An often requested feature for rclone's Google drive backend is being able to upload files using multiple chunks at once to speed things up. S3/B2/Box/etc support this so it seems an obvious thing to request for Google Drive.

The closest thing I could find is the Google Drive v3 API resumable upload feature.

You can use this with with multiple chunks. I use this API in rclone already, but one chunk at a time.

Reading the docs on that carefully, I think that it can't upload multiple chunks at once - it requires the chunks to be uploaded sequentially, but I'd be delighted to be proved wrong!

To upload the file in multiple chunks:

  1. Create a PUT request to the resumable session URI.
  2. Add the chunk's data to the request body. Create chunks in multiples of 256 KB (256 x 1024 bytes) in size, except for the final chunk that completes the upload. Keep the chunk size as large as possible so that the upload is efficient.
  3. Add the following HTTP headers:

    • Content-Length. Set to the number of bytes in the current chunk.
    • Content-Range: Set to show which bytes in the file you are uploading. For example, Content-Range: bytes 0-524287/2000000 shows that you are uploading the first 524,288 bytes (256 x 1024 x 2) in a 2,000,000 byte file.
  4. Send the request, and process the response.

If the upload request is interrupted, or if you receive a 5xx response, follow the procedure in Resume an interrupted upload.

  1. Repeat steps 1 through 4 for each remaining chunk in the file. Use the Range header in the response to determine where to start the next chunk. Do not assume that the server received all bytes sent in the previous request.

When the entire file upload is complete, you receive a 200 OK or 201 Created response, along with any metadata associated with the resource.

If that API can't do it, is there a different one that can?

0 Answers