Our application back-end is made of Symfony 2.5.x. We are using MySQL 5.x and Elastic search 1.7.x for storing our data.
In our application, we store data of different companies in their individual database i.e. multi-tenant approach. Similarly we store companies data in their respective aliases in elastic search.
We had to enhance default fos:elastica:populate command with our custom command so that it checks every company database and sync it with corresponding elastic search alias data.
All this is working fine. No problem so far. Till now we have almost 2000K records in elastic search.
Now, due to one of the requirement, we need to update mapping and store another property in elastic search i.e. store value of another column in elastic search.
Last time we add a new column for all existing aliases and perform bulk updating, 340K records were update on an average in per hour. So this time it might take 6 hours to complete the process.
Is there any way to speed up the process to say within 2 hours.
Our MySQL database is in AWS RDS, application code in Heroku and elastic search is in Found Elastic search provider.
Any help/idea will be much appreciated.
Thanks in advance !