We have installed and configured the sphinx search in our site for some time now. It was running very well. But recently we noticed that while updating the main and delta indexes through ssh in the sphinx server the servers load average increases drastically. It goes up to 11 while the indexer script is in process. The code that we are running is this:
1)ssh -p 90 root@host "/usr/bin/indexer --rotate IdxDelta_domainname"
2)ssh -p 90 root@host "/usr/bin/indexer --rotate IdxDeltaOutlineSearchIndex_domainname"
3)ssh -p 90 root@host "/usr/bin/indexer --rotate IdxDeltaStatus_grmtech"
4) ssh -p 90 root@host "/usr/bin/indexer --rotate --merge IdxMainSearchIndex_domainname IdxDelta_domainname --merge-klists --sighup-each "
5)ssh -p 90 root@host "/usr/bin/indexer --rotate --merge IdxMainOutlineSearchIndex_grmtech IdxDeltaOutlineSearchIndex_domainname --merge-klists --sighup-each "
6)ssh -p 90 root@host "/usr/bin/indexer --rotate --merge IdxMainStatus_grmtech IdxDeltaStatus_grmtech --merge-klists --sighup-each "
This is running from the domain of the original site through a crawler script
The Sphinx index table has 22 fields and data of 689,325 rows.
The server is strong one (with 16 core processor and 6GB of RAM)
While the indexer process runs all 16 cores shows 100% CPU usage (Through top command) and load average shoots up (from the 4th step)
Any way out? please help