-Hi, all,
We have created a website on google cloud that needs time consuming calculation. We want to use parallel computing to speed up the calculation. At the same time, we want other users can query the results in the database. However, if we adopted the parallel computing, when one user is calculating online, other users must wait in queue until he/she finished the job even he/she just want to query some result from the database.
If we allowed multi users can access the website at the same time (no one need to wait in queue), we cannot use the parallel computing at the same time.
It only takes a few seconds to query the result from the database, but it will take a few minutes to finish the calculation online.
Specifically, I’m currently using a library called Celery to handle the parallel computations for multiple API calls, and joblib to handle the parallel computations for the CHO calculations. But they don’t really work well together.
Would you please tell me what framework/software can be used to solve my problem?
Thanks
Erick