what value to be used for max_num_segments for Eslastic search 7.9 while doing forcemerge?

Question

I am upgrading ES 1.4.2 to ES 7.9. For that I had to do remote indexing by re-indexing API to get all the data from old cluster to new cluster. But after re-indexning the search query performance of ES 7.9 got decreased a lot. So I am planning to do forcemerge in order to increase the query performance. Which value should I use for max_num_segments if I decide to to forcemerge? there are no guidelines provided. The old cluster have 2 primary shards , one with 18 segments 16 and second with 18 segments. The new cluster after remote indexing has 2 primary shards one with 27 segments and second with 30 segments. Pleased guide me on the value to be used for max_num_segments. Thanks

Answer 1

Although you have not provided any information about your cluster, index size, docs, replicas, heap size, your search query, etc so it's difficult to say what is the root cause of your bad search performance. but in general, having fewer segments improve the search performance and merge process happens automatically and in very rare-case it requires the manual intervention but still you can use the force merge API and it shouldn't cross the 5GB threshold.

Please refer to my this SO answer for more info on the merge process and API, try with max_num_segments=1 and ES will try to reduce it to the minimum number of segments(5 GB threshold might prevent some segments not eligible for merging).

Also, please use segment API to see the size of your segments and add it with other required information.