I need to give our users the ability to export a bunch of their PDF's (invoices).
The problem is, the PDF's are generated everytime a user opens them.
So I don't have them already stored in a filesystem.
So, my question is, what would be an efficient way of exporting a user's PDF's, when they want to?
For example, a user wants to export 100 invoices from the previous quarter.
I would then need to call a script that generates the PDF's and put them in a .zip file.
But, what if multiple users request a heavy export at the same time?
I was thinking of running a cronjob every 10 minutes or so, but that doesn't solve the possible heavyness of the script.
Do I need to run multiple cronjobs, so they can each handle a portion of the exports?
Or can I somehow create a really efficient script, and maybe separate all PDF's in several batches?
I'm using wkhtmltopdf to generate the PDF's.
So fortunately, it's pretty fast to render the PDF's.
Although it still depends on the content, of course.
On average, a single PDF takes about 3-5 seconds to render.
Any help or guidelines would be greatly appreciated!
Thanks
Follow-up question: If I save every invoice into a filesystem, as soon as it's created or updated, I need to find a way to do this in the background, to avoid unnecessary waiting.
Queues would be my savior, right?
Generate the invoices the moment they are saved in the database and store the filesystem-URL with the database entry. If the user requests a package of invoices of a certain time period, collect the already generated PDFs into a zip-file and provide a download link. Requirement is enough space for file storage on the server.
You can limit the invoice storing to one year. If an user requests older invoices they must be generated otf.
Follow-Up Answer: Since you are in PHP-environment, i would suggest implementing some kind of invoiceChanged handler. If you call it (with parameter invoiceNr), the deprecated PDF-file gets deleted and a new one is generated according to the modified data. If it uses the same file-name, you do not even have to update the file-URL in your database.