Uploading an array of blobs with javascript and handling them on Node.js

Question

I am trying to workaround Cloudflare's 100mb upload limit, by sending chunked uploads. However I cannot seem to figure out how to actually start sending the chunked data. Say I have 100 blobs I'd like to send to the server, do I stream them? How does the server tell the continuous requests relate to each other?

Here's my code so far:

getChunks(file) {
  const divideBy = 1 * 1024 * 1024
  const availableDivisions = Math.ceil(file.size / divideBy)
  let currentSlice = 0

  const chunks = Array(availableDivisions)
    .fill()
    .map((iteration, index) => {
      const nextDivision = divideBy * (index + 1)
      const chunk = file.slice(currentSlice, nextDivision, file.type)

      currentSlice = nextDivision

      return chunk
    })

  return chunks
}

sendChunk(blob) {
  return new Promise((resolve, reject) => {
    const xhr = new XMLHttpRequest()

    xhr.open('POST', 'http://localhost:4080/test', true)
    xhr.setRequestHeader('Content-type', blob.type)

    xhr.onreadystatechange = () => {
      if (xhr.readyState == XMLHttpRequest.DONE && xhr.status == 200) {
        resolve()
      }
    }

    xhr.send(blob)
  })
}

uploadChunked(file) {
  const chunks = this.getChunks(file)

  let iteration = 0

  const upload = chunk => {
    let nextIteration = iteration + 1
    let nextChunk = chunks[nextIteration]

    this.sendChunk(chunk).then(() => {
      if (nextChunk) {
        iteration = nextIteration
        upload(nextChunk)
      }
    })
  }

  upload(chunks[0])
}

So this works fine, the upload requests are done correctly. My problem is figuring out how the server should tell that all these consecutive requests refer to one file. I've looked online and I am just extremely confused at this part.

Answer 1

I've solved my issue by using the Tus protocol. Now my server can accept chunked (and resumable) uploads, and Cloudflare doesn't complain.

Answer 2

You cannot. 100 MB (or X MB) is not a per request limit.

It is a limit per file. In other words if you chunk them up then each chunk would end up becoming a file on the server.

You could upload them in several chunks as you are doing now and also provide an additional script to help your users to later stitch them up on the client side.