system s3cmd sync php timeout

Question

I am trying to sync folders between my remote server and amazon s3 using the system command through a php web page. The web page has option to browse the folder which need to sync between the server and s3

The s3 command is the below one

system('s3cmd sync '.$fldr_name.' '.BUCKET_NAME.' --config="'.AWS_CONF_FILE.'"',$return);

For small folders it is working . But I am syncing large file web pages connection is resetting .

I tried this on my php script

ignore_user_abort(true);

But no luck. Is there any way to make the webpage executable for a long time?

Thanks,

Answer 1

The approach you're using is not ideal. The connection to the server might be resetting for several reasons when it's kept open for a long time, and some of these are out of your control.

I suggest you take another approach to this:

Queue all these sync tasks on the server and only execute a couple of them in parallel using a service that runs in the background (e.g. cron job or other).

When one of these syncs finishes, you record its success/failure in a database.

Then, in the UI, you regularly pull a second service (via AJAX calls executed at regular intervals) that checks the database and returns the pending/success/failure flag that you're storing. If the request is still pending, you simply recheck later.