Search add a custom field to a product document

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is the default configuration when using Elasticsearch version 7.10?

Search rebuild version 1

No specific version

Search rebuild version 3

Search rebuild version 2 (correct)

Which role is required to opt into search rebuild version 2 in the Admin Console?

ISC_Implementer (correct)

System Administrator

Product Manager

Search Developer

What should you do to optimize the performance of a custom extension during the search rebuild process?

Use a JOIN operation to gather related data

Cache all results in memory

Select only the needed columns from the database (correct)

Use a full database query retrieving all fields

What is the recommended method to gather data for modifying base results in the search rebuild process?

Utilize a PrepareToRetrieveIndexableProducts extension Signup and view all the answers

What must be done after modifying the IndexableProducts property in the pipeline sequence?

Create a new pipeline after the base code Signup and view all the answers

What does C#'s 'iterator methods' feature allow you to do in the context of modifying IndexableProducts?

Yield modified results while enumerating Signup and view all the answers

What happens if you try to override base code in version 2 of the search rebuild process?

It can cause performance issues Signup and view all the answers

What happens to support for search rebuild version 1?

It is no longer supported Signup and view all the answers

What should you avoid doing inside your iteration to ensure good performance?

Running database queries or network activities Signup and view all the answers

What is the purpose of extending CreateElasticsearchProductResult?

To modify a record for Elasticsearch or suppress it Signup and view all the answers

Which namespace is used for extending the ElasticsearchProduct class?

Extensions.Search.Elasticsearch.DocumentTypes.Product Signup and view all the answers

What is required to create a custom property in the extended ElasticsearchProduct?

Inheriting from ElasticsearchProduct and adding a field Signup and view all the answers

What should be gathered in the PrepareToRetrieveIndexableProducts extension?

Essential data needed for customization Signup and view all the answers

What is the role of the Order property in the PrepareToRetrieveIndexableProducts class?

To specify the processing priority within a pipeline Signup and view all the answers

Which of the following is true about the constructor of the ElasticsearchProductCustom class?

It replicates all properties from the base class. Signup and view all the answers

What type of data handling is discouraged within the method discussed?

Invoking external services or databases Signup and view all the answers

What is being prepared in the method Execute of PrepareToRetrieveIndexableProductsResult?

Indexable products prepared for retrieval Signup and view all the answers

How does the GroupBy function work in the Execute method?

It groups order lines by ProductId. Signup and view all the answers

What is the purpose of the MyCustomField in the CreateElasticsearchProductResult?

To represent the total count of indexable products. Signup and view all the answers

Where does the ExtendElasticsearchProduct class fit in the pipeline?

It is situated after the FormSortOrder pipe. Signup and view all the answers

What happens if the TryGetValue method does not find a matching ProductId?

MyCustomField remains unchanged. Signup and view all the answers

What type of data structure is used to store the indexed products preparation result?

A Dictionary Signup and view all the answers

What impact does the new custom field have in the search output?

It influences the relevance scoring of search results. Signup and view all the answers

What is the primary role of the Execute method within the ExtendElasticsearchProduct class?

To enhance product data with additional fields. Signup and view all the answers

What does the Order property in the FormCustomSortOrder class indicate?

The execution priority of the pipeline Signup and view all the answers

Under what condition will the result.SortOrderFields be set in the Execute method?

When SortBy is not Relevance and SearchCriteria is not blank Signup and view all the answers

Which pipeline is responsible for building filters for product search in various categories?

RunProductSearch Signup and view all the answers

In the SortOrderField instantiation, what does the second 'true' parameter signify?

The field is sortable Signup and view all the answers

What type of search does the RunProductFacetSearch pipeline perform?

Faceted searches over various attributes Signup and view all the answers

What is the purpose of the FormRestrictionGroupFilter in relation to the product search?

To apply restrictions on product visibility Signup and view all the answers

Which of the following is NOT mentioned as a feature of the RunProductSearch pipeline?

Multi-value field indexing Signup and view all the answers

What does MyCustomField represent in the context of the SortOrderField?

An optional field for added sorting capabilities Signup and view all the answers

Study Notes

Search Rebuild V2

Search rebuild version 2 increases index build speed and performance
It's recommended to upgrade to version 2
Some customization may require extra effort to ensure they properly function

Opting in to version 2

Users with the ISC_Implementer role can opt in to version 2
Go to Administration > System > Settings
Click the Search tab
Select Version 2 on the Build Version dropdown
Click Save

Modifying the Search Build

Use a PrepareToRetrieveIndexableProducts extension to gather data for modifying base results
- Attach data to the RetrieveIndexableProductsPreparation property of the result
For best performance, use a custom entity framework database query with a general form of .ToDictionary(record => record.ProductId)
Store the dictionary in RetrieveIndexableProductsPreparation and use .TryGetValue to look up product extension data
Select only needed columns from the database
Create a new pipeline after the base code sequence in the pipeline that modifies the IndexableProducts property
You cannot override base code in version 2, doing so in version 1 causes performance and compatibility issues
The data in PrepareToRetrieveIndexableProducts is available in the RetrieveIndexableProductsPreparation property
Use iterator methods to add, remove, or modify the content of IndexableProducts by enumerating it in your extension and yielding the modified results
Do not perform database queries or other network activity inside the iteration, gather the data in PrepareToRetrieveIndexableProducts and use it in the pipeline
If you are not adding new records, you may be able to use an extension of CreateElasticsearchProductResult alone
Use CreateElasticsearchProductResult extensions to modify a record destined for Elasticsearch or suppress it by returning null
Extend the record by inheriting ElasticsearchProduct and adding custom fields
Do not do any database queries or other network activity inside this method

Extension example code

Adding a custom field to Elasticsearch product

Extend ElasticsearchProduct with a custom property
C# Elasticsearch v5*

namespace Extensions.Search.Elasticsearch.DocumentTypes.Product
{
  using Insite.Search.Elasticsearch.DocumentTypes.Product;
  using Nest;

 [ElasticsearchType(Name = "product")]
 public class ElasticsearchProductCustom : ElasticsearchProduct
 {
     public ElasticsearchProductCustom(ElasticsearchProduct source)
         : base(source) 
     {
     }

     [Keyword(Name = "myCustomField ", Index = true)]
     public int MyCustomField { get; set; }
 }
}

C# Elasticsearch v7*

namespace Extensions.Search.Elasticsearch.DocumentTypes.Product
{
  using Insite.Search.Elasticsearch.DocumentTypes.Product;
  using Nest;

 [ElasticsearchType(Name = "product")]
 public class ElasticsearchProductCustom : ElasticsearchProduct
 {
     public ElasticsearchProductCustom(ElasticsearchProduct source)
         : base(source) 
     {
     }

     [Keyword(Name = "myCustomField ", Index = true)]
     public int MyCustomField { get; set; }
 }
}

Gathers Data for customization

Gather data in an extension of PrepareToRetrieveIndexableProducts
C# Elasticsearch v5*

namespace Extensions.Search.Elasticsearch.DocumentTypes.Product.Index.Pipelines.Pipes.PrepareToRetrieveIndexableProducts
{
  using Insite.Core.Interfaces.Data;
  using Insite.Core.Plugins.Pipelines;
  using Insite.Data.Entities;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Index.Pipelines.Parameters;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Index.Pipelines.Results;
  using System.Linq;

 public sealed class PrepareToRetrieveIndexableProducts : IPipe
 {
     public int Order => 0; 
     public PrepareToRetrieveIndexableProductsResult Execute(IUnitOfWork unitOfWork, PrepareToRetrieveIndexableProductsParameter parameter, PrepareToRetrieveIndexableProductsResult result)
     {
         result.RetrieveIndexableProductsPreparation = unitOfWork.GetRepository().GetTableAsNoTracking().Where(order => order.Status == "Submitted").SelectMany(order => order.OrderLines).GroupBy(orderLine => orderLine.ProductId).ToDictionary(group => group.Key, group => group.Count());
         return result;
     }
 }
}

C# Elasticsearch v7*

namespace Extensions.Search.Elasticsearch.DocumentTypes.Product.Index.Pipelines.Pipes.PrepareToRetrieveIndexableProducts
{
  using Insite.Core.Interfaces.Data;
  using Insite.Core.Plugins.Pipelines;
  using Insite.Data.Entities;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Index.Pipelines.Parameters;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Index.Pipelines.Results;
  using System.Linq;

 public sealed class PrepareToRetrieveIndexableProducts : IPipe
 {
     public int Order => 0; 
     public PrepareToRetrieveIndexableProductsResult Execute(IUnitOfWork unitOfWork, PrepareToRetrieveIndexableProductsParameter parameter, PrepareToRetrieveIndexableProductsResult result)
     {
         result.RetrieveIndexableProductsPreparation = unitOfWork.GetRepository().GetTableAsNoTracking().Where(order => order.Status == "Submitted").SelectMany(order => order.OrderLines).GroupBy(orderLine => orderLine.ProductId).ToDictionary(group => group.Key, group => group.Count());
         return result;
     }
 }
}

Extract data and apply it to the extended record

Extract data from the preparation result in CreateElasticsearchProductResult
C# Elasticsearch v5*

namespace Extensions.Search.Elasticsearch.DocumentTypes.Product.Index.Pipelines.Pipes.CreateElasticsearchProduct
{
  using System;
  using System.Collections.Generic;
  using Insite.Core.Interfaces.Data;
  using Insite.Core.Plugins.Pipelines;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Index.Pipelines.Parameters;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Index.Pipelines.Results;

 public sealed class ExtendElasticsearchProduct : IPipe
 {
     public int Order => 150;
     public CreateElasticsearchProductResult Execute(IUnitOfWork unitOfWork, CreateElasticsearchProductParameter parameter, CreateElasticsearchProductResult result)
     {
         var elasticsearchProductCustom = new ElasticsearchProductCustom(result.ElasticsearchProduct);
         if (((Dictionary)parameter.RetrieveIndexableProductsPreparation).TryGetValue(elasticsearchProductCustom.ProductId, out var count))
         {
             elasticsearchProductCustom.MyCustomField = count;
         }
         result.ElasticsearchProduct = elasticsearchProductCustom;
         return result;
     }
 }
}

C# Elasticsearch v7*

namespace Extensions.Search.Elasticsearch.DocumentTypes.Product.Index.Pipelines.Pipes.CreateElasticsearchProduct
{
  using System;
  using System.Collections.Generic;
  using Insite.Core.Interfaces.Data;
  using Insite.Core.Plugins.Pipelines;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Index.Pipelines.Parameters;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Index.Pipelines.Results;

 public sealed class ExtendElasticsearchProduct : IPipe
 {
     public int Order => 150;
     public CreateElasticsearchProductResult Execute(IUnitOfWork unitOfWork, CreateElasticsearchProductParameter parameter, CreateElasticsearchProductResult result)
     {
         var elasticsearchProductCustom = new ElasticsearchProductCustom(result.ElasticsearchProduct);
         if (((Dictionary)parameter.RetrieveIndexableProductsPreparation).TryGetValue(elasticsearchProductCustom.ProductId, out var count))
         {
             elasticsearchProductCustom.MyCustomField = count;
         }
         result.ElasticsearchProduct = elasticsearchProductCustom;
         return result;
     }
 }
}

Use the new field

The code adds a new field to Elasticsearch index product documents
This data can be used in query pipelines
An example is using the new field to influence the sort order of results by adding a pipe in the RunProductSearch pipeline
This pipe runs after the FormSortOrder pipe which modifies the SortOrderFields field on RunProductSearchResult when the user is on a category page using the default Relevance sort order
C# Elasticsearch v5*

namespace Extensions.Plugins.Search.Elasticsearch.DocumentTypes.Product.Query.Pipelines.Pipes.RunProductSearch
{
  using System;
  using Insite.Core.Interfaces.Data;
  using Insite.Core.Plugins.Pipelines;
  using Insite.Core.Plugins.Search;
  using Insite.Search.Elasticsearch.DocumentTypes.Product;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Query.Pipelines.Parameters;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Query.Pipelines.Results;

 public class FormCustomSortOrder : IPipe
 {
     public int Order => 310;
     public RunProductSearchResult Execute(
         IUnitOfWork unitOfWork,
         RunProductSearchParameter parameter,
         RunProductSearchResult result)
     {
         if (result.SortOrderFields == null)
         {
             return result;
         }
         if (parameter.ProductSearchParameter.SortBy != SortOrderType.Relevance ||
             parameter.ProductSearchParameter.SearchCriteria.IsNotBlank() ||
             parameter.ProductSearchParameter.SearchCriteria.IsNotBlank())
         {
             return result;
         }
         result.SortOrderFields = new[]
         {
             new SortOrderField(nameof(ElasticsearchProduct.SortOrder).ToCamelCase(), true, true),
             new SortOrderField(nameof(ElasticsearchProductCustom.MyCustomField).ToCamelCase(), true, true),
             new SortOrderField(nameof(ElasticsearchProduct.ShortDescriptionSort).ToCamelCase())
         };
         return result;
     }
 }
}

C# Elasticsearch v7*

namespace Extensions.Plugins.Search.Elasticsearch.DocumentTypes.Product.Query.Pipelines.Pipes.RunProductSearch
{
  using System;
  using Insite.Core.Interfaces.Data;
  using Insite.Core.Plugins.Pipelines;
  using Insite.Core.Plugins.Search;
  using Insite.Search.Elasticsearch.DocumentTypes.Product;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Query.Pipelines.Parameters;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Query.Pipelines.Results;

 public class FormCustomSortOrder : IPipe
 {
     public int Order => 310;
     public RunProductSearchResult Execute(
         IUnitOfWork unitOfWork,
         RunProductSearchParameter parameter,
         RunProductSearchResult result)
     {
         if (result.SortOrderFields == null)
         {
             return result;
         }
         if (parameter.ProductSearchParameter.SortBy != SortOrderType.Relevance ||
             parameter.ProductSearchParameter.SearchCriteria.IsNotBlank() ||
             parameter.ProductSearchParameter.SearchCriteria.IsNotBlank())
         {
             return result;
         }
         result.SortOrderFields = new[]
         {
             new SortOrderField(nameof(ElasticsearchProduct.SortOrder).ToCamelCase(), true, true),
             new SortOrderField(nameof(ElasticsearchProductCustom.MyCustomField).ToCamelCase(), true, true),
             new SortOrderField(nameof(ElasticsearchProduct.ShortDescriptionSort).ToCamelCase())
         };
         return result;
     }
 }
}

Shared Search*

namespace Extensions.Plugins.Search.Elasticsearch.DocumentTypes.Product.Query.Pipelines.Pipes.RunProductSearch
{
  using System;
  using Insite.Core.Interfaces.Data;
  using Insite.Core.Plugins.Pipelines;
  using Insite.Core.Plugins.Search;
  using Insite.Search.Elasticsearch.DocumentTypes.Product;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Query.Pipelines.Parameters;
  using Insite.Search.Elasticsearch.DocumentTypes.Product.Query.Pipelines.Results;

 public class FormCustomSortOrder : IPipe
 {
     public int Order => 310;
     public RunProductSearchResult Execute(
         IUnitOfWork unitOfWork,
         RunProductSearchParameter parameter,
         RunProductSearchResult result)
     {
         if (result.SortOrderFields == null)
         {
             return result;
         }
         if (parameter.ProductSearchParameter.SortBy != SortOrderType.Relevance ||
             parameter.ProductSearchParameter.SearchCriteria.IsNotBlank() ||
             parameter.ProductSearchParameter.SearchCriteria.IsNotBlank())
         {
             return result;
         }
         result.SortOrderFields = new[]
         {
             new SortOrderField(nameof(ElasticsearchProduct.SortOrder).ToCamelCase(), true, true),
             new SortOrderField(nameof(ElasticsearchProductCustom.MyCustomField).ToCamelCase(), true, true),
             new SortOrderField(nameof(ElasticsearchProduct.ShortDescriptionSort).ToCamelCase())
         };
         return result;
     }
 }
}

Available pipelines

RunProductSearch - The main product query builder
FormProductFilter builds filter for product search over category, language, and attributes
RunProductFacetSearch performs faceted searches over brand, product line, and category
RunBrandSearch searches for brands within the product index
FormRestrictionGroupFilter builds the restriction group filter for the FormProductFilter pipeline

Note

The Optimizely Dogfood sample repository on Github includes an example of adding a multi-value field to the index and using it to filter products based on warehouse stock

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Description

This quiz covers the enhancements introduced in Search Rebuild Version 2, including index build speed and performance improvements. It guides users on how to opt into the new version and modify search build parameters effectively. Test your knowledge on the recommended practices and settings for optimizing search functionality.