Search Federation Options in SharePoint On-Premises


Without going into any technicalities, there are two ways to do search federation in SharePoint 2013. 

  • SharePoint 2013 Search Service Application Federation
  • SharePoint 2013 Search Results Federation

Both of these approaches are good but depends on your requirements.  Now let’s take a a look at this following scanario.

You have two large SharePoint 2013 Farms (ContentFarmA, ContentFarmB) in the same domain and search is configured in both farms.  Most of you would agree that SharePoint 2013 search requires lot of resources to make it work optimal. Now you have two farms to manage search.  The best and recommended approach in this scanario is to move the search out of these two farms to a third service farm. 

Service Application Federation using Service Farm

  1. Configure a Service Farm (FarmC)
  2. Create Search Service Application, Extend the Search Topology then Content Sources and point to Both (ContentFarmA, ContentFarmB)  Farms and configure crawls as required.
  3. From here you have two options
  1. Build an Enterprise Search Web Application and Search Center from Service Farm.  Train users of both content farms to use central search center for search. 
  2. Setup the OAuth Trust betwen Service Farm and Content Farms and Publish the Search Service application.  This will allow you to use local search site collections in each farm

I have just published a Post on this topic here.

Service Application Federation accross Farm (Not Possible)

I know a thought may have come to your mind to setup a two way trusts betwen Content Farms and publish Search service application in each other but it is not possible.  You can only use a Single Search Service Application as default.

Search Result Federation using Result Sources

I am sure you have heard Hybrid Search few times since Office 365 is evolved.  We are not talking about Office365 at this moment but I am thinkg about writing a guide on it already.  We can setup server to server trust between both content farms to provide results to each other using Result source.  This is much simpler way to bring search results from one to the other.  Although the process is easy but can be confusing.  Below are the steps to configure and test the process. 

Prerequisties

  • You must have two farms FarmA and FarmB. 
  • Both must have atleast one web application and some sample content.  Web application must be configured with SSL (Non SSL did not worked for me).
  • Search Service application must be configured and crawling content.
  • You have Farm configured for App Management (App Management and Subscription Settings Service Application)
  • End users from Farm A would like to get federated search results from Farm B. 

Steps to Configure and Test the Trust

You have two farms FarmA and FarmB. 

Farm A is Sender of Search Request.

This means that “Farm B” is Receiver (of Search Requests and Results from Farm A).  This is the core steps.  Make sure you put the text in front of you to remember who is sender and who is receiver.

You will login in to Farm B and Execute the following cmdlets on Management Shell or PowerShell ISE (Please load the snapin).

# Create a trusted security token issuer

$i = New-SPTrustedSecurityTokenIssuer -Name "SendingFarm" -IsTrustBroker:$false -MetadataEndpoint "https://FarmA_WebApplication/_layouts/15/metadata/json/1"

New-SPTrustedRootAuthority -Name "SendingFarm" -MetadataEndPoint https://FarmA_WebApplication>/_layouts/15/metadata/json/1/rootcertificate

Now Run for Each Web application on Farm B to provide access.

$realm = $i.NameId.Split("@")

$s1 = Get-SPSite -Identity https://FarmB_WebApplication>

$sc1 = Get-SPServiceContext -Site $s1

# Set up an authentication realm for' # a web application that hosts content in ReceivingFarm 

Set-SPAuthenticationRealm -ServiceContext $sc1 -Realm $realm[1]

# Get a reference to the application principal' # for that web application in Farm B

$p = Get-SPAppPrincipal -Site https://<ReceivingFarm_web_application> -NameIdentifier $i.NameId

# Grant rights to the application principal' # that SendingFarm will use' # when it sends queries to ReceivingFarm

Set-SPAppPrincipalPermission -Site https://FarmB_WebApplication> -AppPrincipal $p -Scope SiteCollection -Right FullControl

#IISRESET on both farms.  Browse the sites on receiver and otherwsie you may get time out.

Note: According to TechNet we should repeat this for all web application.  We got the results from all web apps with trust for one because it only applies if we have multiple search service application or external search using BCS or you have multiple proxy group.  It is not the case for me in my lab.

Creating and Testing Results Source

Now we should go to FarmA Central Administration => Application Management => Manage Service Application => Click on Search Service Applicatio ==>  Result Source

  1. Click Create a result source
  2. Enter Name as Farm B Results
  3. Choose Type as Remote SharePoint type
  4. Provide Provide URL of FarmB web application in site url. 
  5. Click OK.
  6. Click on the Dropdown next to Farm B Results Result Source and choose Test
  7. If you get Timeout error then go to Farm B and Browse the site.  If you get 401 Unauthorized then you have not setup the trust correct.  You must remove all existing certificates from Central Administration => Security ==> Manage Trust.
  8. To remove SPTrustedSecurityTokenIssuer and SPTrustedRootAuthority you must use
  9. Get-SPTrustedSecurityTokenIssuer and Get-SPTrustedRootAuthority then Remove them using Remove-SPTrustedRootAuthority and Remove-SPTrustedSecurityTokenIssuer  cmdlets.  Please do not remove local.

Note: We must Test the result source to make sure we are not getting 401 unauthorized.

Adding Web parts to Search Results page

Now Go to Search Center on Farm A and search for SharePoint.  Once you get the results use Site Actions button ==> Edit apge ==> now add another search results web part on results.aspx page, Edit the Web part properties and set the web part result source to “Farm B Results”.  Change the Tile and Chrome Type to Title and Border to make sure you see the difference.  Click OK Save and Publish the web part pages.  Perform the search.  You can also get the results in one web part.  This TechNet article provides the steps.

Now to setup the two way trust for search result federation Farm B will become sender and Farm A will be receiver and same steps will be executed (Urls will be updated in scripts).  Here is an example of how this looks like

clip_image002

Looks Nice :).  Next step would be to test Remote Result sources with Office365.

Advertisements

Establishing Trust Relationships between SharePoint Farms


Step 1: Export certificates from the consumer farm

1. Log on to the Consumer Farm CA Server.

2. On the Start screen, type SharePoint 2013 Management Shell, and then press Enter.

3. At the command prompt, type the following command, and then press Enter:

$rootcert = (Get-SPCertificateAuthority).RootCertificate 

4. At the command prompt, type the following command, and then press Enter:

$rootcert.Export("Cert") | Set-Content "C:\Certs\ConsumerFarmRoot.cer" -Encoding byte 

5. At the command prompt, type the following command, and then press Enter:

$stscert = (Get-SPSecurityTokenServiceConfig).LocalLoginProvider.SigningCertificate

6. At the command prompt, type the following command, and then press Enter:

$stscert.Export("Cert") | Set-Content "C:\Certs\ConsumerFarmSTS.cer" -Encoding byte 

7. Close the SharePoint 2013 Management Shell.

8. On the Windows taskbar, click File Explorer.

9. In the File Explorer window, expand Computer, and then double-click C

10. Select the ConsumerFarmRoot and ConsumerFarmSTS files, right-click the files, and then click Copy.

11. In the File Explorer address bar, type \\ProviderFarmServer\c$\Certs, and then press Enter.

12. Right-click the content area, and then click Paste.

13. Close the File Explorer window.

Step 2: Export the root certificate from the provider farm

1. Switch to the Provider Farm virtual machine.

2. On the Start screen, type SharePoint 2013 Management Shell, and then press Enter.

3. At the command prompt, type the following command, and then press Enter:

$rootcert = (Get-SPCertificateAuthority).RootCertificate 

4. At the command prompt, type the following command, and then press Enter:

$rootcert.Export("Cert") | Set-Content "C:\Certs\ProviderFarmRoot.cer" -Encoding byte 

5. Close the SharePoint 2013 Management Shell.

6. On the Windows taskbar, click File Explorer.

7. In the File Explorer window, double-click C Drive.

8. Right-click the ProviderFarmRoot.cer, and then click Copy.

9. In the File Explorer address bar, type \\ConsumerFarmServer\c$\Certs, and then press Enter.

10. Right-click the content area, and then click Paste.

11. Close the File Explorer window.

Step 3: Create a trusted root authority on the consumer farm

1. Switch to the Consumer Farm virtual machine.

2. On the Start screen, type SharePoint 2013 Management Shell, and then press Enter.

3. At the command prompt, type the following command, and then press Enter:

$trustcert = Get-PfxCertificate "C:\Certs\ProviderFarmRoot.cer" 

4. At the command prompt, type the following command, and then press Enter:

New-SPTrustedRootAuthority "Contoso Provider Farm" -Certificate $trustcert 

5. Close the SharePoint 2013 Management Shell.

Step 4: Create a trusted root authority on the provider farm

1. Switch to the Provider Farm virtual machine.

2. On the Start screen, type SharePoint 2013 Management Shell, and then press Enter.

3. At the command prompt, type the following command, and then press Enter:

$trustcert = Get-PfxCertificate "C:\Certs\ConsumerFarmRoot.cer" 

4. At the command prompt, type the following command, and then press Enter:

New-SPTrustedRootAuthority "Contoso Consumer Farm" -Certificate $trustcert

Step 5: Create a trusted token issuer on the provider farm

1. At the command prompt, type the following command, and then press Enter:

$stscert = Get-PfxCertificate "C:\Certs\ConsumerFarmSTS.cer" 

2. At the command prompt, type the following command, and then press Enter:

New-SPTrustedServiceTokenIssuer "Contoso Consumer Farm" -Certificate $stscert 

3. Close the SharePoint 2013 Management Shell.

4. On the Start screen, type SharePoint 2013 Central Administration, and then press Enter.

5. On the Quick Launch navigation menu, click Security.

6. Under General Security, click Manage trust.

7. Verify that Contoso Consumer Farm is listed with a type of Trusted Service Provider.

Note: The term Trusted Service Provider on this page can be confusing, as in this case you want to use Contoso Consumer Farm to consume services. In this context, the term Trusted Service Provider means that the local farm includes a trust relationship that enables it to provide services to Contoso Consumer Farm

Publishing and Consuming Service Applications

Step 1: Get the ID of the consumer farm

1. Switch to the Consumer Farm virtual machine.

2. On the Start screen, type SharePoint 2013 Management Shell, and then press Enter.

3. At the command prompt, type the following command, and then press Enter:

Get-SPFarm | Select Id | Set-Content "C:\Certs\ConsumerFarmID.txt" 

Close the SharePoint 2013 Management Shell.

5. On the Windows taskbar, click File Explorer.

6. In the File Explorer window, double-click C:\

7. Right-click the ConsumerFarmID.txt file, and then click Copy.

8. In the File Explorer address bar, type \\ProviderServer\c$\Certs, and then press Enter.

9. Right-click the content area, and then click Paste.

10. Close the File Explorer window

Step 2: Grant the consumer farm permissions on the provider farm Application Discovery and Load Balancing Service Application

1. Switch to the Provider VM virtual machine.

2. On the Windows taskbar, click File Explorer.

3. In the File Explorer window, browse to C:\Certs, and then open the ConsumerFarmID.txt file.

4. In the ConsumerFarmID.txt file, select the GUID, and then on the Edit menu, click Copy.

Note: Only copy the GUID. Do not include the curly brackets {} or the preceding text.

5. Close the Notepad window and the File Explorer window.

6. On the Start screen, type SharePoint 2013 Management Shell, and then press Enter.

7. At the command prompt, type the following command, and then press Enter:

$security = Get-SPTopologyServiceApplication | Get-SPServiceApplicationSecurity

8. At the command prompt, type the following command, and then press Enter:

$claimprovider = (Get-SPClaimProvider System).ClaimProvider

9. At the command prompt, type the following command, and then press Enter (replace <ConsumerFarmID> with the value you copied from the text file):

$principal = New-SPClaimsPrincipal -ClaimType "http://schemas.microsoft.com/sharepoint/2009/08/claims/farmid" -ClaimProvider $claimprovider -ClaimValue <ConsumerFarmID> 

10. At the command prompt, type the following command, and then press Enter:

Grant-SPObjectSecurity -Identity $security -Principal $principal -Rights “Full

Control”

11. At the command prompt, type the following command, and then press Enter:

Get-SPTopologyServiceApplication | Set-SPServiceApplicationSecurity –ObjectSecurity $security 

12. Close the SharePoint 2013 Management Shell

Step 3: Grant the consumer farm permissions on the provider farm Search Service application

1. On Provider Farm – On the Start screen, type SharePoint 2013 Central Administration, and then press Enter.

2. Under Application Management, click Manage service applications.

3. On the list of service applications, click the uppermost Contoso Search Service.

4. On the ribbon, click Permissions.

5. In the Connection Permissions for Contoso Search Service dialog box, in the text box, paste the GUID that you copied from the text file, and then click Add.

6. In the list of permissions, select the Full Access to Term Store check box, and then click OK.

Step 4: Publish the Search Service application on the provider farm

1. In the list of service applications, ensure that Contoso Search Service is still selected.

2. On the ribbon, click Publish.

3. In the Publish Service Application dialog box, in the Connection Type list, ensure http is selected.

4. Select the Publish this Service Application to other farms check box.

5. Under Published URL, select the URL, right-click the selected text, and then click Copy.

6. In the Description text box, type Search Service application from the Contoso provider farm, and then click OK.

7. Close Internet Explorer.

8. On the Windows taskbar, click File Explorer.

9. In the File Explorer address bar, type \\ConsumerFarm\c$\Certs, and then press Enter.

10. Right-click in the content area, point to New, and then click Text Document.

11. Type ContosoServiceServiceApplicationURL, and then press Enter.

12. Open the ContosoServiceServiceApplication.txt file, paste the URL you copied, and then save and close the file

Step 5: Connect to the Search Service application from the consumer farm

1. Switch to the ConsumerFarm virtual machine.

2. On the Windows taskbar, click File Explorer.

3. Browse to C:\Certs, and then double-click ContosoServiceServiceApplicationURL.txt.

4. Select and copy the URL, and then close the file.

5. Close the File Explorer window.

6. On the Start screen, type SharePoint 2013 Central Administration, and then press Enter.

7. Under Application Management, click Manage service applications.

8. On the ribbon, on the Connect menu, click Search Service Connection.

9. In the Connect to a Remote Service Application dialog box, in the text box, paste the URL you copied from the text file, and then click OK.

10. When the dialog box displays the Contoso Search Service application, click Contoso Search Service, and then click OK.

11. When you are prompted to name the connection, accept the default name and click OK.

12. When the dialog box displays a success message, click OK.

13. On the list of service applications, click Connection to: Contoso Search Service.

14. On the ribbon, click Manage.

15. On the Term Store Management Tool page, verify that the Organization term set is displayed from the remote farm.

.clip_image002

Click on Search Service Application Link and Click OK. Now rename the connection if you like.

clip_image004

Click OK and OK.

clip_image006

The connection will appear right below.

clip_image009

Setting Published Service Application as Default

You can only use a single web application as default search proxy so Next Step would be to make this service application proxy as Default. To do that you have to do the following steps.

Open SharePoint 2013 Central Administration

Click on Application Management à Manage Service Application Association

Click on Default or the service application proxy group of your choice.

clip_image011

Click on Set as Default next to Service Application you just published.

Now open a Search Center and perform a search. You must see the search result.

Creating extended Search topology in Sharepoint 2013


If you are creating a Search service application in SharePoint 2013 and want to create an extedned search topology, the only choice you have is “PowerShell”.  Althought the process looks bit complex but if you spent few minutes in the PowerShell script below it will become very simply.

Below script uses a 9 server Search Toplogy where we have the following configuration

  • 3 Query Processing Components
  • 3 Index Partitions and their Replicas
  • 2 Admin Components
  • 2 Crawl Components
  • 2 Analytic Components
  • 2 Content Processing Components

You can see that we have redundant search components.  with this topology we were planning to cover 30 content sources with over 40 Million items in in the index.

First on the root define the Servers and mention their role.  I used an Excel sheet first then moved them in Powershell

Note: If you are extending an existing which already has index.  it is good to RESET Index.  Additonally you can pause the Search Service Applicaton and then resume it after new topology.

$hostA = Get-SPEnterpriseSearchServiceInstance -Identity “SERVER135" #Query Processing -  Index 0
$hostB = Get-SPEnterpriseSearchServiceInstance -Identity "SERVER136" #Query Processing -  Index 1
$hostC = Get-SPEnterpriseSearchServiceInstance -Identity "SERVER137" #Query Processing -  Index 2

$hostD = Get-SPEnterpriseSearchServiceInstance -Identity “SERVER138" #Analytics Processing - Index 2 (Replica)
$hostE = Get-SPEnterpriseSearchServiceInstance -Identity “SERVER139” #Analytics Processing - Index 1 (Replica)
$hostF = Get-SPEnterpriseSearchServiceInstance -Identity “SERVER140" #Index 0 (Replica)

$hostG = Get-SPEnterpriseSearchServiceInstance -Identity “SERVER141" #Content Processing - Crawl Component
$hostH = Get-SPEnterpriseSearchServiceInstance -Identity “SERVER142" #Content Processing - Crawl Component

Next Step is to Start the Enterprise Search Service Instance on these servers

Start-SPEnterpriseSearchServiceInstance -Identity $hostA
Start-SPEnterpriseSearchServiceInstance -Identity $hostB
Start-SPEnterpriseSearchServiceInstance -Identity $hostC
Start-SPEnterpriseSearchServiceInstance -Identity $hostD
Start-SPEnterpriseSearchServiceInstance -Identity $hostE
Start-SPEnterpriseSearchServiceInstance -Identity $hostF
Start-SPEnterpriseSearchServiceInstance -Identity $hostG
Start-SPEnterpriseSearchServiceInstance -Identity $hostH

Now wait for all services to be provisioned.  You can check the status using the script below

//Wait until status of “OnLine” by issuing the following commands:

Get-SPEnterpriseSearchServiceInstance -Identity $hostA | Select Server, Status
Get-SPEnterpriseSearchServiceInstance -Identity $hostB | Select Server, Status
Get-SPEnterpriseSearchServiceInstance -Identity $hostC | Select Server, Status
Get-SPEnterpriseSearchServiceInstance -Identity $hostD | Select Server, Status
Get-SPEnterpriseSearchServiceInstance -Identity $hostE | Select Server, Status
Get-SPEnterpriseSearchServiceInstance -Identity $hostF | Select Server, Status
Get-SPEnterpriseSearchServiceInstance -Identity $hostG | Select Server, Status
Get-SPEnterpriseSearchServiceInstance -Identity $hostH | Select Server, Status

Now Next Step is to Get the Search Service Application and Create a new Search Topology.  To reuse existing topology you can clone it.

$ssa = Get-SPEnterpriseSearchServiceApplication
$newTopology  = New-SPEnterpriseSearchTopology -SearchApplication $ssa

Next Step would be to Create the Components.  Below i am creating by the hosts for easier managment.

Host A

New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostA
New-SPEnterpriseSearchIndexComponent -SearchTopology $newTopology -SearchServiceInstance $hostA -IndexPartition 0

Host B

New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostB
New-SPEnterpriseSearchIndexComponent -SearchTopology $newTopology -SearchServiceInstance $hostB -IndexPartition 1

Host C

New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostC
New-SPEnterpriseSearchIndexComponent -SearchTopology $newTopology -SearchServiceInstance $hostC -IndexPartition 2

Host D

New-SPEnterpriseSearchAnalyticsProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostD
New-SPEnterpriseSearchIndexComponent -SearchTopology $newTopology -SearchServiceInstance $hostD -IndexPartition 2

Host E

New-SPEnterpriseSearchAnalyticsProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostE
New-SPEnterpriseSearchIndexComponent -SearchTopology $newTopology -SearchServiceInstance $hostE -IndexPartition 1

Host F

New-SPEnterpriseSearchIndexComponent -SearchTopology $newTopology -SearchServiceInstance $hostF -IndexPartition 0
New-SPEnterpriseSearchAdminComponent -SearchTopology $newTopology -SearchServiceInstance $hostF

Host G

New-SPEnterpriseSearchCrawlComponent -SearchTopology $newTopology -SearchServiceInstance $hostG
New-SPEnterpriseSearchContentProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostG

Host H

New-SPEnterpriseSearchCrawlComponent -SearchTopology $newTopology -SearchServiceInstance $hostH
New-SPEnterpriseSearchContentProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostH

Now Set the topology as Active Topolgy

Set-SPEnterpriseSearchTopology -Identity $newTopology
iisreset 
Get-SPEnterpriseSearchTopology -SearchApplication $ssa

Remove the Old Toplogy

#Removing Old Topology Get-SPEnterpriseSearchTopology -SearchApplication $ssa #Note the ID of the old toplogy $OldTopology = Get-SPEnterpriseSearchTopology -SearchApplication $ssa -Identity 64e1b2ba-bfc6-44d4-9ebe-9f9b5952bdd0 #To Verify $OldTopology Remove-SPEnterpriseSearchTopology -Identity $OldTopology.Id -SearchApplication $ssa

#You are all Set now Open Central Administration –> Manage Service Applications –> Search #Service Application –> Review the components and all should show
#Green check mark. Refresh the browse if they take time to get green