handling really long migrations in Heroku

767 views Asked by At

How can I speed up really long migrations (millions of rows, lots of moving stuff around)?

Is there a way to run the migration in concurrency, or to script the migration so it will run in the background and then I can promote it?

How can I run long migrations without timing-out in the heroku run rake console?

1

There are 1 answers

2
Remigius Stalder On

The bottleneck for the migration is the communication between the client and the server. The best approach would probably be to do it online (i.e. as part of the application or as a separate application sharing the same database) and pull the required resources in vie HTTP.

Alternatively - but only with downtime - you can create a dump of the database, manipulate it locally on your workstation and upload it to heroku to restore the database from it. However, depending on the data size, this may take too long and therefore not be feasible.

When migrating data online, you can also migrate related parts on access, e.g. migrate all the data of a single customer account the next time the customer accesses it. It the delay is to long, a little message could inform the user of the migration progress.