'504 - Gateway Timeout' when Indexing the items in Episerver Find

Question

When Indexing the items, it fails sometimes and it gives,

The remote server returned an error: (504) Gateway Timeout. [The remote server returned an error: (504) Gateway Timeout.]

The Indexing logic is here as below,

var client = EPiServer.Find.Framework.SearchClient.Instance;
List<ItemModel> items = getItems(); // Get more than 1000 items
List<ItemModel> tempItems = new List<ItemModel>();

//Index 50 items at a time
foreach(var item in items)
{
    tempItems.Add(item);
    if (tempItems.Count == 50)
    {
        client.Index(tempItems);
        tempItems.Clear();
    }
}

What causes this to happen ?

Note: The above mentioned ItemModel is a custom model which is not implemented interfaces (such as IContent). And the items is a list of ItemModel objects.

Additional info:
EPiServer.Find.Framework version 13.0.1
EPiServer.CMS.Core version 11.9.2

Answer 1

I always figured the SearchClient to be a bit sketchy when manipulating data in Find, as far as I figured (but I have to check this) the SearchClient obey under the request limitation of Episerver Find and when doing bigger operations in loops it tends to time out.

Instead, use the ContentIndexer, i.e.

// Use this or injected parameter
var loader = ServiceLocator.Current.GetInstance<IContentLoader>();

// Remove all children or not
var cascade = true;

ContentReference entryPoint = ...where you want to start

// Get all indexable languages from Find
Languages languages = SearchClient.Instance.Settings.Languages;

// Remove all current instances of all languages below the selected content node
//languages.ForEach(x => ContentIndexer.Instance.RemoveFromIndex(entryPoint, cascade.Checked, x.FieldSuffix));

foreach (var lang in languages)
{
    if (cascade)
    {
        var descendents = loader.GetDescendents(entryPoint);

        foreach (ContentReference descendent in descendents)
        {
            ContentIndexer.Instance.RemoveFromIndex(descendent, false, lang.FieldSuffix);
        }
    }

    // Try delete the entrypoint
    var entryTest = loader.Get<IContent>(entryPoint, new CultureInfo(lang.FieldSuffix));
    if (entryTest != null)
    {
        var delRes = ContentIndexer.Instance.Delete(entryTest);
    }
}

This is the most bulletproof way to delete stuff from the index as far as I figured.