Search Federation Options in SharePoint On-Premises – Part II


In the first post Search Federation Options in SharePoint On-Premises I explained and demonstrated the steps required to setup Federated Search Results from 2 different farms.  To show the results I used two seperate web parts.  One confired to show results from Local SharePoint Result Source and other for the Remote resuls.  In this post I will explain the steps required to merge the results and show them as Result Block. 

To achieve this we hav to rely on two things. 

A Query Rule

Allow us to setup rule to make a search query against a different result source or promote the results based on based on some condition and action.  We can create Query Rules at Central Administration, Site Collection and Site Level.   We can also control when to make this rule active. 

A Result Block

A block of search results that will be displayed based ont he query rule.  we can define how much results will be shown, provide “More” links etc.

For more information please read How To Customize SharePoint 2013 Search Results Using Query Rules and Result Sources

Let’s setup the Query Rule and Result block.

image

The image above is what my results look like before I started.  Two web parts on the search results page.  First I removed the web part from page.  I saw something interesting.  The refinement web part broke. 

image

When I edit the webpart I found Target not found error.  I may have set the default web part to use the remote result source and 2nd web part to local search results.  Once I set the Refinement Targett to Local Search Results, it did not broke.

Creating the Query Rule

Open Central Adminsitration

Manage Service Application

Search Service Application

Under Queries and Results Click on Query Rules

image

From  “For what context do you want to configure rule”  select the Local SharePoint Results. 

Note: I am selecting it because I want to add this result block in default search results.

Then Click New Rule

image 
image

Type in the Rule name.  You can limit the query based on results Sources, Query Categories and User Seagments.  I am not going into that details.

image
Select Query Contdition if any based on your requirements.  In my case I remove the condition entirely.  image
image
In the Actions section. You can either
Add a Promoted Results
It looks like Search keywords and best bets in SharePoint 2010 Search.  You just enter the title and URL and Click Save.

image
image
Add result Block
First you enter the Block Title to make it more clear
If you want to configure Query variable and change and test queries you can launc the Query Builder. 
From Search this Source Select the Remote result Source created in last blog post.
Select the number of Items to show in search results.
image
Expand the Settings Tab
Choose “More” link goes to the following URL and enter the URL of your Search center /pages/results.apsx with ?k={subjectTerms}

image
image
In case needed you can control when this query rule will be available and providethe contact.  image
The result block will be visible now.
Clic Save.
image
The rule will be visible on the Manage Query Rule page. image
Now Let’s test the search results. 
The first part of search results are Federated. 
image
Click on Show More took me to the remote search center. image
I perfomed the exact above step on my other farm to setup two way fedearted results.  Now when I clicked on more to go to other farm the results from first showed up as Federated. image

The same procedure is used to get federated results from Office365.  The link to Technet is here https://technet.microsoft.com/en-us/library/dn197173.aspx

Adding Users to Web App as Full Control


One of the most common issues I face while working with different environments is full control to all web applications in a SharePoint Farms.  The best way to tackle this issue for a team would be to create a AD Group and add the group to web application level using the same script but it is an issue where this method is not used so I faces issues browsing different web applications in different farms.  You can use the script below to fix this issues.

function Add-FullControl($UserName, $WebApp) 
{
    Write-Host "Adding $UserName as Full Control." -ForegroundColor Green
    $User = New-SPClaimsPrincipal -IdentityType WindowsSamAccountName -Identity $UserName
    $Policy = $WebApp.Policies.Add($User.ToEncodedString(), $UserName)   
    $Policy.PolicyRoleBindings.Add($WebApp.PolicyRoles.GetSpecialRole([Microsoft.SharePoint.Administration.SPPolicyRoleType]::FullControl))   
    $WebApp.Update()
    Write-Host "Done for $UserName." -ForegroundColor Green
} 

foreach($WebApp in Get-SPWebApplication)
{
    $UserAccount = "contoso\sphelpdesk"
    Add-FullControl -UserName $UserAccount -WebApp $WebApp
}
Write-Host "All Done..." -ForegroundColor Green

Not a huge thing to do from UI but if you have big number of farms with many web apps this script will be handy.  The same script can be used for Cache Super User and Super Reader Accounts but minor changes to GetSpecialRole([Microsoft.SharePoint.Administration.SPPolicyRoleType]:: :)

SharePoint Server MVP for Another Year


I have been honourd to be SharePoint Server MVP for another year.  This will be my 6th year.  Special Thanks to Microsoft and to friends and family. 

Everything is slowly transitioning to Office365 lately but I am still involved in large On-Premises projects at HP.  I will keep sharing as much information as I have.  Best of luck to each and every one of you.  Once again huge thanks to Microsoft for recognizing the community.

Clearning up Large Lists and Recycle Bin with PowerShell


If you want to dump large lists quickly you can use batch delete script mentioend below.  Deleing items one by one in loop will take a lot of time.  Even if you delete the items they will get stuck in site collection recyle bin.  You can modify the script as needed.

Add-PSSnapin Microsoft.SharePoint.powerShell -ErrorAction SilentlyContinue 
$sitecollectionUrl = https://portal.contoso.com
$siteCollection = New-Object Microsoft.SharePoint.SPSite($sitecollectionUrl)
$web = $siteCollection.OpenWeb('Web') 

$list = $web.Lists["Logging List"] 

write-host("Logging List has : $list.ItemCount Items.") -ForegroundColor Yellow
write-host("Recyle Bin has : $siteCollection.RecycleBin.Count Items.") -ForegroundColor Yellow 

$itemCount = 0;
$listId = $list.ID;
[System.Text.StringBuilder]$batchXml = New-Object "System.Text.StringBuilder";
$batchXml.Append("<?xml version=`"1.0`" encoding=`"UTF-8`"?><Batch>");
$command = [System.String]::Format( "<Method><SetList>{0}</SetList><SetVar Name=`"ID`">{1}</SetVar><SetVar Name=`"Cmd`">Delete</SetVar></Method>", $listId, "{0}" ); 

$listItems = $list.Items
foreach ($item in $listItems)
{
    if($item -ne $null){$batchXml.Append([System.String]::Format($command, $item.ID.ToString())) | Out-Null;$itemCount++;}
} 

$batchXml.Append("</Batch>");
$itemCount; 

$web.ProcessBatchData($batchXml.ToString()) | Out-Null;Write-Host("Starting to Clear Site Collection Recyle Bin: $siteCollection.RecycleBin.Count") -ForegroundColor Yellow 
$siteCollection.RecycleBin.DeleteAll();
Write-Host("Cleared Log List and Site Collection Recyle Bin.")
$web.Dispose()
$siteCollection.Dispose()
 

Changing Site Collection Quota for Personal Sites


During migration of personal Sites from SharePoint 2010 to SharePoint 2013 I saw an issue where most of Personal Sites quota is not assigned properly.  The reason could be the my site databases are not upgraded successfully.  I tried some basic PowerShell script to change the personal site quota but got the following error.

image

I even tried with Server OM with PowerShell but result was same.  So for now I will say just fix the database upgrade issue.

Get List of Personal Sites and Quota

Start-SPAssignment -Global
$PersonalSites = @();
$MySites = get-spwebapplication https://mysites.contoso.com/
foreach($Site in $MySites.Sites)
{
    if($Site.Url.Contains("personal")) #Ignore MySite Host
    {
        $Site = New-Object PSObject
        Add-Member -input $Site noteproperty 'Url' $Site.Url
        Add-Member -input $Site noteproperty 'Quota' $Site.Quota
        $PersonalSites += $Site
    }
}
Stop-SPAssignment -Global
$PersonalSites | Out-File C:\MySiteQuota.txt 

Change the Quota for Personal Sites

Start-SPAssignment -Global
$TemplateName = "Personal Site"
$contentService = [Microsoft.SharePoint.Administration.SPWebService]::ContentService
$quotaTemplate = $contentService.QuotaTemplates[$TemplateName] 

$MySites = Get-SPWebApplication https://mysites.contoso.com/ | Get-SPSite -Limit ALL | 
  Where-Object {$_.ServerRelativeUrl.Contains("/personal/")} |
  ForEach-Object { $_.Quota = $quotaTemplate }
Stop-SPAssignment -Global

Controlling Maximum Site Count for Content Databases


Here is a quick and directy Powershell script to get and change the Maximum Site Count for the Content databases in SharePoint.  We had a large farm with hundred of databases so I used the following script to set the Maximum Site Count equal to current Sites in the database and Warning to less than 1.  You can change it based on the logic of your choice.

Get the List of Databases in a text file

$Databases = @();
foreach($aDB in Get-SPContentDatabase)
{
    $DB = New-Object PSObject
    Add-Member -input $DB noteproperty 'Name' $aDB.Name
    Add-Member -input $DB noteproperty 'Maximum' $aDB.MaximumSiteCount
    Add-Member -input $DB noteproperty 'Warning' $aDB.WarningSiteCount
    Add-Member -input $DB noteproperty 'Current' $aDB.Sites.Count
    $Databases += $DB
    Write-Host $aDB.Name + " " + $aDB.MaximumSiteCount + " " + $aDB.WarningSiteCount 
}
$Databases  | Out-File C:\DatabasesBefore.txt
#$Databases  | Out-GridView

Change the values

foreach($aDB in Get-SPContentDatabase)
{
    if(!$aDB.Name.Contains("MySite"))
    {
        Write-Host $aDB.Name
        $MaxSiteCount = $aDB.MaximumSiteCount
        $WarningCount = $aDB.WarningSiteCount
        $CurrentCount = $aDB.Sites.Count 

        Write-Host "$aDB.Name $MaxSiteCount $WarningCount $CurrentCount" -ForegroundColor Yellow
            $aDB.MaximumSiteCount = $CurrentCount
            $aDB.WarningSiteCount = $CurrentCount - 1
            $aDB.Update()
            Write-Host "$aDB.Name $aDB.MaximumSiteCount $WarningCount $CurrentCount" -ForegroundColor Green 

    }
}

Note: The script is excluding mysites databases and will fail for content DB which has no site collection so please add the logic based on your environment

You can re-run the first script if you want to verify the change but Central Administration –> Manage Content Database will show you the updates straight away.

Search Federation Options in SharePoint On-Premises


Without going into any technicalities, there are two ways to do search federation in SharePoint 2013. 

  • SharePoint 2013 Search Service Application Federation
  • SharePoint 2013 Search Results Federation

Both of these approaches are good but depends on your requirements.  Now let’s take a a look at this following scanario.

You have two large SharePoint 2013 Farms (ContentFarmA, ContentFarmB) in the same domain and search is configured in both farms.  Most of you would agree that SharePoint 2013 search requires lot of resources to make it work optimal. Now you have two farms to manage search.  The best and recommended approach in this scanario is to move the search out of these two farms to a third service farm. 

Service Application Federation using Service Farm

  1. Configure a Service Farm (FarmC)
  2. Create Search Service Application, Extend the Search Topology then Content Sources and point to Both (ContentFarmA, ContentFarmB)  Farms and configure crawls as required.
  3. From here you have two options
  1. Build an Enterprise Search Web Application and Search Center from Service Farm.  Train users of both content farms to use central search center for search. 
  2. Setup the OAuth Trust betwen Service Farm and Content Farms and Publish the Search Service application.  This will allow you to use local search site collections in each farm

I have just published a Post on this topic here.

Service Application Federation accross Farm (Not Possible)

I know a thought may have come to your mind to setup a two way trusts betwen Content Farms and publish Search service application in each other but it is not possible.  You can only use a Single Search Service Application as default.

Search Result Federation using Result Sources

I am sure you have heard Hybrid Search few times since Office 365 is evolved.  We are not talking about Office365 at this moment but I am thinkg about writing a guide on it already.  We can setup server to server trust between both content farms to provide results to each other using Result source.  This is much simpler way to bring search results from one to the other.  Although the process is easy but can be confusing.  Below are the steps to configure and test the process. 

Prerequisties

  • You must have two farms FarmA and FarmB. 
  • Both must have atleast one web application and some sample content.  Web application must be configured with SSL (Non SSL did not worked for me).
  • Search Service application must be configured and crawling content.
  • You have Farm configured for App Management (App Management and Subscription Settings Service Application)
  • End users from Farm A would like to get federated search results from Farm B. 

Steps to Configure and Test the Trust

You have two farms FarmA and FarmB. 

Farm A is Sender of Search Request.

This means that “Farm B” is Receiver (of Search Requests and Results from Farm A).  This is the core steps.  Make sure you put the text in front of you to remember who is sender and who is receiver.

You will login in to Farm B and Execute the following cmdlets on Management Shell or PowerShell ISE (Please load the snapin).

# Create a trusted security token issuer

$i = New-SPTrustedSecurityTokenIssuer -Name "SendingFarm" -IsTrustBroker:$false -MetadataEndpoint "https://FarmA_WebApplication/_layouts/15/metadata/json/1"

New-SPTrustedRootAuthority -Name "SendingFarm" -MetadataEndPoint https://FarmA_WebApplication>/_layouts/15/metadata/json/1/rootcertificate

Now Run for Each Web application on Farm B to provide access.

$realm = $i.NameId.Split("@")

$s1 = Get-SPSite -Identity https://FarmB_WebApplication>

$sc1 = Get-SPServiceContext -Site $s1

# Set up an authentication realm for' # a web application that hosts content in ReceivingFarm 

Set-SPAuthenticationRealm -ServiceContext $sc1 -Realm $realm[1]

# Get a reference to the application principal' # for that web application in Farm B

$p = Get-SPAppPrincipal -Site https://<ReceivingFarm_web_application> -NameIdentifier $i.NameId

# Grant rights to the application principal' # that SendingFarm will use' # when it sends queries to ReceivingFarm

Set-SPAppPrincipalPermission -Site https://FarmB_WebApplication> -AppPrincipal $p -Scope SiteCollection -Right FullControl

#IISRESET on both farms.  Browse the sites on receiver and otherwsie you may get time out.

Note: According to TechNet we should repeat this for all web application.  We got the results from all web apps with trust for one because it only applies if we have multiple search service application or external search using BCS or you have multiple proxy group.  It is not the case for me in my lab.

Creating and Testing Results Source

Now we should go to FarmA Central Administration => Application Management => Manage Service Application => Click on Search Service Applicatio ==>  Result Source

  1. Click Create a result source
  2. Enter Name as Farm B Results
  3. Choose Type as Remote SharePoint type
  4. Provide Provide URL of FarmB web application in site url. 
  5. Click OK.
  6. Click on the Dropdown next to Farm B Results Result Source and choose Test
  7. If you get Timeout error then go to Farm B and Browse the site.  If you get 401 Unauthorized then you have not setup the trust correct.  You must remove all existing certificates from Central Administration => Security ==> Manage Trust.
  8. To remove SPTrustedSecurityTokenIssuer and SPTrustedRootAuthority you must use
  9. Get-SPTrustedSecurityTokenIssuer and Get-SPTrustedRootAuthority then Remove them using Remove-SPTrustedRootAuthority and Remove-SPTrustedSecurityTokenIssuer  cmdlets.  Please do not remove local.

Note: We must Test the result source to make sure we are not getting 401 unauthorized.

Adding Web parts to Search Results page

Now Go to Search Center on Farm A and search for SharePoint.  Once you get the results use Site Actions button ==> Edit apge ==> now add another search results web part on results.aspx page, Edit the Web part properties and set the web part result source to “Farm B Results”.  Change the Tile and Chrome Type to Title and Border to make sure you see the difference.  Click OK Save and Publish the web part pages.  Perform the search.  You can also get the results in one web part.  This TechNet article provides the steps.

Now to setup the two way trust for search result federation Farm B will become sender and Farm A will be receiver and same steps will be executed (Urls will be updated in scripts).  Here is an example of how this looks like

clip_image002

Looks Nice :).  Next step would be to test Remote Result sources with Office365.