Broken pipe error while uploading large multipart file via s3cmd: "[Errno 32]" Broken pipe

223 views Asked by At

I'm uploading a large (~200GB) compressed file with s3cmd and keep getting a periodic error.

First we try to upload a part, it fails at 0%. Then I get the error: "Cannot retrieve any response status before encountering an EPIPE or ECONNRESET exception", followed by another upload failure notification, wait 3 seconds, then a seemingly successfully upload of that same chunk. This happened from the very first chunk.

Here's what this routine looks like:

s3cmd -vpH --no-check-md5 --multipart-chunk-size-mb=1000 --progress --stats put /Storage/shared/backups/dump/vzdump-qemu-900-2023_06_16-01_00_00.vma.zst s3://bucket/vzdump-qemu-900-2023_06_16-01_00_00.vma.zst
upload: '/Storage/shared/backups/dump/vzdump-qemu-900-2023_06_16-01_00_00.vma.zst' -> 's3://bucket/vzdump-qemu-900-2023_06_16-01_00_00.vma.zst'  [part 30 of 179, 1000MB] [1 of 1]
     786432 of 1048576000     0% in    7s   100.46 KB/s  failed
ERROR: Cannot retrieve any response status before encountering an EPIPE or ECONNRESET exception
WARNING: Upload failed: /vzdump-qemu-900-2023_06_16-01_00_00.vma.zst?partNumber=30&uploadId=NknuFBqK97L28zSVht2Q_….1K6EZL2F ([Errno 32] Broken pipe)
WARNING: Waiting 3 sec...
upload: '/Storage/shared/backups/dump/vzdump-qemu-900-2023_06_16-01_00_00.vma.zst' -> 's3://bucket/vzdump-qemu-900-2023_06_16-01_00_00.vma.zst'  [part 30 of 179, 1000MB] [1 of 1]
 1048576000 of 1048576000   100% in   30s    32.58 MB/s  done

Should I be worried?

0

There are 0 answers