Introduction to SharePoint 2013 Search

Editor's Notes

• #2: SharePoint Server 2013 will, without a doubt, be a huge success for Microsoft. The SharePoint platform has come a long way and SharePoint 2013 is another leap forward. Search is one of the areas where Microsoft has, once again, made huge investments and the new functionality is astounding. They have taken a great deal of the functionality of their FAST product and tightly integrated it into the SharePoint platform, molding what will finally become the leader in the Enterprise Search space. There are still some bumps along the road but, overall, SharePoint fans are going to be very pleased. First, it is important to know that what we all know as the FAST Search Server 2010 for SharePoint will most definitely be dropped as a SKU. The FAST product will no longer be developed (it hasn’t been touched for a couple years anyways), and FAST Farms will lose support. This is kind of bad news for anyone who has invested in FAST already but very good news for anyone who hasn’t! Some of the most important functionality in FAST and its scalability has been folded into SharePoint Search and the cumbersome structure of the FAST product is gone. This includes the 2 big value propositions of FAST; scalability, and property extraction. However, don’t expect SharePoint 2013 search to work like FAST. Some parts of FAST will disappear, never to return, like the FAST Web Crawler and JDBC connectors. Let’s look in detail at some of the new cool features of SharePoint Server 2013.
• #4: Native OOB PDF indexing.SharePoint 2013 has made significant improvements to ‘nail the basics’ with native support for PDF indexing. Although the PDF ifilter from Adobe is free and 90% effective, native support is a no-brainer. Faster Office document and PDF indexingNot only is PDF indexing out of the box, that and office documents have their own conversion mechanism in SharePoint 2013. This is, from the metrics we have seen, much faster and more reliable. You can still use, and develop your own, ifilters if you want but indexing Office and PDF content out of the box is going to be a huge improvement.  Federated Locations and Search Scopes are combined into Results SourcesFederated Locations allows you to consume results from OpenSearch compliant search engines and a number of different sources could be added. Search Scopes gave a logical separation to the search index and allowed for pre-defined queries to be submitted. Federated Locations has been replaced with Result Sources and Search Scopes have been folded into that menu. Result sources differ from Content Sources by not actually being crawled by this farm. What that means is that you can have results consumed from another SharePoint farm and searchable from this SharePoint farm. In a result source, you can also restrict queries to a subset of content by using a query transform which is similar to a query restriction in SP2010. The primary improvement to result sources is that site collection administrators, site owners, and site designers can now configure result sources.  Mailbox search! Although SharePoint 2013 still only supports indexing of Public Folders, the same back end search engine will be used in Exchange and the Result Sources capability will allow you to add an Exchange index as a result source. This means users will have native search of their own mail within SharePoint.  NOTE: These results will appear in a separate results web part (perhaps best in a separate mail tab?)   ScalabilityThere’s no word on the tested boundaries of SharePoint 2013 search but we are expecting indexing capabilities in the billions of documents and unlimited scalability with impressive crawl times. Look for more blog posts from SurfRay at www.surfray.co as we explore deeper into the beta release for further updates.
• #5: Crawl Health ReportsSharePoint 2010 had some great analytics in their Search Administration Reports. These reports covered crawl rates, crawl processing, and query latency but were not very well used and kind of buried. Microsoft has expanded the reporting available and given a more interesting look at your crawl activity as well as promoting this information to the top of the Search Service Application’s (SSA) menu. In SP2013 expect to see a lot deeper level of information about what your crawler is doing and it should be easier to navigate as well.
• #6: Managed properties can be made Searchable, Queryable, Retrievable, Refinable, or SortableManaged properties in SharePoint 2013 get a slight overhaul. They are now set under Search Schema in the SSA. They can be set as searchable, queryable, retrievable, refinable, sortable or any combination. The differentiation is made to change functionality and limit storage requirements. Searchable: Includes the content of the property in the index, making it searchable as a query term alone. Documents with a searchable property on them will be returned as keyword matches. Queryable: Making a property queryable allows you to query it using property queries such as Property:Value. Retrievable: Retrievable properties can be returned to the result set for display or programmatic usage.Refinable: Allows properties to be returned in refiners. Sortable: Allows properties to be used for sorting.
• #7: Managed properties can be made Searchable, Queryable, Retrievable, Refinable, or SortableManaged properties in SharePoint 2013 get a slight overhaul. They are now set under Search Schema in the SSA. They can be set as searchable, queryable, retrievable, refinable, sortable or any combination. The differentiation is made to change functionality and limit storage requirements. Searchable: Includes the content of the property in the index, making it searchable as a query term alone. Documents with a searchable property on them will be returned as keyword matches. Queryable: Making a property queryable allows you to query it using property queries such as Property:Value. Retrievable: Retrievable properties can be returned to the result set for display or programmatic usage.Refinable: Allows properties to be returned in refiners. Sortable: Allows properties to be used for sorting.
• #8: Complete matching of property valuesProperty queries in SharePoint 2010 were limited by a set of reserved characters and searching for terms with these characters often returned errors. Now a property can be set to have complete matching which will allow ID’s such as 12-3-45.ABC to be queried instead of recognizing the reserved terms as spaces and returning documents with 12 3 45 and ABC on them.  Entity/Property Extraction – company and custom entities. Another option in search schema is extracting properties from the documents, even if they are not in property fields or metadata. Out of the box, company extractions can be performed but custom entity extraction can also be set up. This will prove to be a massively useful feature and a major value add from the FAST expertise in MS.  
• #9: Continuous crawlSharePoint Server 2013 can be configured so that incremental crawls are performed continuously for selected content sources. This option helps to keep index content as fresh as possible. Administrators still need to configure full crawls and evaluate the resource consumption of continuous crawls.
• #10: Search dictionaries- Links to MMSA new link appears in the Site Collection Administration pointing to Search dictionaries where search terms can be matched with others. This actually just links to the Managed Metadata Service and allows for terms to be matched in the term store. New Query SyntaxA new query syntax has been implemented to allow for more logical and complicated queries. The full details of this are not yet published but doing searches from the advanced search page will actually preserve the query in the query box and the user can glean the new syntax from it.It doesn’t look like you’ll get FAST like query language but the new syntax should make custom queries easier to manage.  Figure 12 – A query for documents containing bees in documents and authored by guys with Gordon in their name. ALL(bees) (DetectedLanguage="en") (IsDocument="True") Author:gordonQuery SuggestionsTwo types– what you see when you are entering a query, and what you see when you get resultsQuery box suggestions: A list of items you have clicked on before from your personal query log A list of items that others are typing for their queriesResults page suggestions: They are a list of links that you have clicked through at least twice before and match your search criteria
• #11: Result TypesResult types is a new mechanism that will allow you to apply a particular template to a given result, meaning documents or searchable items of a different type (like people) can be displayed with a picture or different metadata. These have several elements that enable results to be displayed differently including:Rules – Rules determine when a result type should be applied.Property List – Each result type contains a list of managed properties. A property must be added to this list before it can be used in the display template. Rendering Template – These define the visual layout of the result type.Result Type RulesYou can create a variety of different rules on which to match queries with result types:Equality(= or !=), comparison (< or >), or logical (AND or ORor NOT)Each result type can have multiple rules based on queries, result types, and managed propertiesResult Type ManagementResult Types can apply to an entire search service application or to a specific site or webOut of box Result Types cannot be edited but they can be copied and the new rule set to override.There is an inheritance hierarchy to result types in a site: the current web, plus the site collection root, plus the Search Service Application. Result types are not inherited unless you are working in the site collection root web.Display TemplatesDisplay templates define the visual HTML layout of a Result TypeTemplates can be edit it with any HTML designer you want (ex. DreamWeaver, SharePoint Designer, Visual Studio, Notepad)You can add placeholders to your HTML file where managed properties are displayed
• #12: Result Blocks – A group of related results defined by the administratorQuery RulesQuery rules allow you specify actionscorrelated to a search term. When a query meets the conditions in a query rule, the search system performs the action you establish such as displaying a Best Bet.A query rule can specify the following types of actions:Add Promoted Results (new name for Best Bets) that appear above ranked results. For example, for the query “tax documents” administrators could specify a particular Promoted Result, such as a link to a site where employees can retrieve their 1099. Add one or more Result Blocks. For example, for a query that contains “logos" a query rule might display a result block with people in the graphic design department and another result block with a selection of the most popular company logos.Trigger Action Terms to improve search result ranking. For example, if a user searches for “support for widget C”, a query rule could recognize the words “support“ and “widget C” as action terms and boost results from the support portal that also have a tag for “widget C”.
• #13: User InterfaceOne of the most profound changes is the new user interface. Microsoft has really taken a departure from previous versions which were intentionally kept simple to provide a much richer experience. There is a floating information pane that can potentially provide previews of documents if the Office Web Apps Server (separate farm) is used. Here are a few of the new UI features:Ajax UIThe user interface is now heavily dependent on javascript and inline calls to items are made in the search results. This helps to update results and show additional information without page refresh. Video and Reports tab OOB in Enterprise Search CenterIf you have the Publishing feature enabled and the Enterprise Search center deployed you will get new tabs for Video search and Reports alongside Everything (All Sites) and People. I haven’t tried these out yet but am looking forward to trying video search. In-document-library search boxLibraries now have their own search box that is submitting a contextual scoped search for that library or list allowing users much better in library search capabilities. This box also appears on the libraries page, instead of in the universal search box location, making it very handy and easy to use.
• #14: Visual RefinersRefiners get an overhaul with a few new options for displaying in graphs and a search within refiners box. Refinement styles are set with display templates. This can help with long lists of refiners and narrowing a date becomes extremely easy with the date slider.   Deep refiners with countsThe refiners in SharePoint 2013 are now deep refiners, able to count the absolute number of matches in an index. These managed properties need to be made refinable, but all items will be considered instead of the previous shallow dataset. Refiner selection UI in Edit Web Part PaneA major improvement and probably my personal favourite is the extensive capabilities for adding refiners that has been added. You can now easily throw a new refiner into the refinement panel, change its name, and change its attributes without coding any XSLTs.
• #15: Floating info/preview paneWe will certainly have a lot to say about the new floating info pane because it includes many features that Ontolica has been providing but in a slightly different user experience. When mousing over a result, a floating pane appears to the right of the result item with some result actions (not as many as a library item) and potentially a preview of the document. Document previews will require a separate install and likely a separate license.  
• #16: Result set enhancements:Finally, there is a number of great result set enhancement capabilities added including:Social definitions  - Definitions for search terms appear based on content in documents. Personal favorites – Items you have clicked frequently.Response time – How fast the engine returned the results.Language selection – Allow users to filter on languages or have default languages set. Sorting dropdown – Allows for sorting of the result set based on properties.  
• #17: What is Missing?Better advanced searchAdvanced search seems to be about the same as in 2010. I would have expected some of the new search UI love to be given to the advanced search page but perhaps that’s left to customize.Configurable web crawlerWe are losing the web crawler from FAST but we don’t seem to have any more configurability in the way that SharePoint crawls any content with out of the box connectors. Web sites are complicated and messy. Having some control of how it crawls beyond simple rules could be useful.Search Result actions beyond view library and sendIt’s great to see search result actions on the hover pane but these are limited to view library and send document link. Items like view properties or edit properties on the document should be included. Preview only works on Office documentsThe new Office Web Apps server preview is massively cool but only works on Office documents. First page, inline preview only works on PowerPoint. Forget about previewing your PDF documents.  Saved Searches/Alerts/RSS FeedsApparently Microsoft didn’t think people were using alerts and RSS feeds because they are gone. Also, doing anything else with a search result is not (yet) supported. We will have to see if this stuff makes it into the final release. In result preview without additional infrastructureHaving a preview pane is a big bonus but it comes at a massive cost. The Office Web Apps Server is very hard to install and set up and will require a lot of additional infrastructure and probably licensing. Also, the hover pane is a little slow and cumbersome. It detracts from the user experience and is not terribly intuitive. Ideally there should be inline first page previews for all documents, not just PowerPoint.
• #18: Ontolica Search & PreviewOntolica Search & Preview vastly extends the basic capabilities of SharePoint search and transforms them into a true enterprise search platform, with radically improved efficiency, faster information access, and end-user satisfaction.End –User Search PersonalizationAdd personalization to SharePoint search, as user search preferences are automatically detected and remembered across SharePoint. Precise and relevant search results are then displayed according to user preferences. At any time users can easily tweak the automatic search preferences to optimize the search experience to their own personal design preference and needs.Dynamic Result ViewsDisplay best practice search result views that allow users to easily toggle between multiple styles to see and identify relevant information. The multi-view framework in Ontolica Enterprise Search can be easily extended with minimum effort to accommodate specialized search user interfaces.No-coding cross-farm configurationKeep up with an evolving environment as changes to the search experience is a simple matter of adjusting settings in the administration interface. No need for lengthy development and test cycles. Through Ontolica Enterprise Search, changes across a SharePoint farm are easily done with full support for hierarchical inheritance of configuration customizations.
• #19: Preview 500+ File Types & External Content SourcesWith Ontolica, content previews are quickly available to view, scale, and print without native client applications. For example, Ontolica Preview enables users to view CAD drawings and InDesign layouts without the need to install AutoCAD or InDesign. Ontolica supports over 500 different documents and file formats making it the most comprehensive file support platform available. Ontolica Preview also delivers transparent support for content residing on file shares to provide a consistent user interface experience regardless of where the item is stored.
• #21: Overall, the advancements to SharePoint Server 2013 search are pretty huge and this is the most impressive search product released by Microsoft yet. There are still a few gaps and I am uncertain about the adoption of the new UI but we are definitely getting closer to a true Enterprise Search experience with SharePoint. Disclaimer: This post is based on the first SharePoint Preview release so functionality may change before the final release. For a deeper introduction to the new SP2013 preview functionality please see David Gordon’s blog. For information on how to install the new Office Web Apps server make sure to check out Robert Piddocke’s latest post. Both of these as well as more blogs, quick tips, and video instruction on all SharePoint Search releases can be found at www.surfray.com.