Getting cors errors when trying to perform a single chunk resumable upload to google cloud storage using gcs json api, axios, vue3, quasar and node14

Question

I'm trying to perform a single chunk resumable upload via gcs, node, vue per https://cloud.google.com/storage/docs/performing-resumable-uploads . I'm able to produce the signedUrl but get errors when I try to put or post to the signedUrl via the client.

If navigate to the signedUrl, the browser displays the following error:

<Error>
<Code>MalformedSecurityHeader</Code>
<Message>Your request has a malformed header.</Message>
<ParameterName>content-type</ParameterName>
<Details>Header was included in signedheaders, but not in the request.</Details>
</Error>

In the console, I get the following error (domain.appspot and domain.iam in all of these is set to my bucket name. i just replaced/hid it here just in case):

Access to XMLHttpRequest at 'https://storage.googleapis.com/domain.appspot.com/media/20161017_213931.mp4?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=gcs-server%40domain.iam.gserviceaccount.com%2F20211213%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20211213T202005Z&X-Goog-Expires=900&X-Goog-SignedHeaders=content-type%3Bhost%3Bx-goog-resumable&X-Goog-Signature=920390253e558265309a73ceda3ac981c56d40580c8103d41dd191478bb7186b3aea742891f0fc50ad8c766ff1e262c1f012f021f7687699873f98cf244799539ec86f3f600eb9b2e849f869de677ae8bc75a0343eb474f50e12dd4bebc9594c0d4b309bf94b55a1a9c1e3971004c62ed11ebdb328813d8c860d70714feade4b940b7f14c015d45eaa87c816c83d3ba2a1b41783dcda9a9f9ffe09de6ccd47a0c1d292ee0e4c0e1fa0a61d1109207f8a9b9c67d41ae8797bcacb8102a5b3e09a4c108d07d29697bddbe638b32117c078ec180f50c021b5094a163034f3c3d799295f6dd78279a4fb4f0a03d3037333fdf3a03bacc2bd8edb02c2f63974707561' from origin 'http://localhost:8081' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

PUT https://storage.googleapis.com/domain.appspot.com/media/20161017_213931.mp4?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=gcs-server%40domain.iam.gserviceaccount.com%2F20211213%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20211213T214015Z&X-Goog-Expires=900&X-Goog-SignedHeaders=content-type%3Bhost%3Bx-goog-resumable&X-Goog-Signature=93eda476fe4444b7aa7b51054b56ad5124249fc77c2a268e3a6e4967f2cf037da4981325922a2b646d6254e8b9b64d4f317ad3a63c3c40820a55e51f4520e5b29ca3b92b85644909159201f01004d3a3c087364ab0bacff9db651026fd87401f1aeb567ad3ee663a25f2cd94a65e6bb6e1882bba14cfc2a238d5c725000d0ae6637f9c665e42688372ef1118418bb548254cf9868a8a7d773861295b0299f8caae59525232d9059920b302210b30740dca8ccc2c581264674f627a49f87f10d38ef2e806c39e4ccab712abc6ab4a5ca4905cb1d4e8272f6d030985ebaa72b1f9ca69259670d56af50d099a2b700b0a81c7cf61b2b2468e3f381bf3750a4d0d12 net::ERR_FAILED 400

// node/backend

import { Storage } from '@google-cloud/storage'

const storage = new Storage({
  keyFilename: process.env.GOOGLE_APPLICATION_CREDENTIALS
})
const bucketName = 'xxxx.appspot.com'

async function createResumableUrl (file) {
  const blob = storage.bucket(bucketName).file(`media/${file.originalname}`)

  const options = {
    version: 'v4',
    action: 'resumable',
    contentType: 'application/octet-stream',
    expires: Date.now() + 15 * 60 * 1000 // 15 minutes
  }

  const [signedUrl] = await blob.getSignedUrl(options)

  return signedUrl
}

// vue/quasar frontend form

<q-form
enctype="multipart/form-data"
>
  <q-file
    v-if="!initFile"
    label="Select a file for Upload"
    dense
    outlined
    rounded
    no-error-icon
    hide-bottom-space
    bg-color="grey-1"
    class="q-pt-sm q-mb-md"
    @input="mediaAction"
  >
    <template v-if="media.location" v-slot:append>
      <q-icon name="mdi-autorenew" @click.stop="" class="cursor-pointer" />
    </template>
    <template v-else v-slot:append>
      <q-icon name="mdi-plus" @click.stop="" class="cursor-pointer" />
    </template>
    <template v-if="uploaderHint" v-slot:hint>
      {{ uploaderHint }}
    </template>
  </q-file>
  <q-btn
    unelevated
    dense
    size="0.6rem"
    color="primary"
    class="q-py-xs q-px-sm q-mr-md"
    @click="onOKClick"
  >
    <span>
      <q-icon name="mdi-upload-outline" class="q-mr-xs" />
      <span v-if="initFile">Update</span>
      <span v-else>Upload</span>
    </span>
  </q-btn>
</q-form>

// vue/quasar method

methods: {
  async onOKClick () {
          const formData = new FormData()
          formData.append('file', this.selectedMedia)
          const signedUrl = await this.$store.dispatch('media/createResumableUrl', formData)
          // signedUrl is present
          // console.log(signedUrl.data)
          const upload = await this.$api.put(signedUrl.data, formData, {
            withCredentials: false,
            headers: {
             'Content-Type': 'application/octet-stream',
             'Access-Control-Allow-Origin': 'http://localhost:8081'
            }
          })

  }
}

// gcs bucket config

[
  {
    "origin": [
      "*"
    ],
    "responseHeader": [
      "Content-Type",
      "x-goog-resumable",
      "Access-Control-Allow-Origin"
    ],
    "method": ["PUT", "GET", "HEAD", "DELETE", "POST", "OPTIONS"],
    "maxAgeSeconds": 15
  }
]

things i've tried:

1. I've tried to make the client request with a post instead of put
2. I've tried adding 'Content-Range': 'bytes *' to all request
3. i've tried adding origin to the node request for the signedUrl and I've tried matching up the client request and the cors bucket config to all be the same. I've tried this with the port, no port and '*':

// node

const options = {
  version: 'v4',
  origin: 'http://localhost',
  action: 'resumable',
  contentType: 'application/octet-stream',
  expires: Date.now() + 15 * 60 * 1000 // 15 minutes
}

// client

const upload = await this.$api.put(signedUrl.data, formData, {
  withCredentials: false,
  headers: {
    'Content-Type': 'application/octet-stream',
    'Access-Control-Allow-Origin': 'http://localhost'
  }
})

// cors bucket config

[
  {
    "origin": [
      "http://localhost"
    ],
    "responseHeader": [
      "Content-Type",
      "x-goog-resumable",
      "Access-Control-Allow-Origin"
    ],
    "method": ["PUT", "GET", "HEAD", "DELETE", "POST", "OPTIONS"],
    "maxAgeSeconds": 1
  }
]

// Preflight Info

Response Headers:

access-control-allow-headers: content-type,x-goog-resumable,Access-Control-Allow-Origin access-control-allow-methods: PUT,GET,HEAD,DELETE,POST,OPTIONS
access-control-allow-origin: * access-control-max-age: 15 alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43" cache-control: private, max-age=0 content-length: 0 content-type: text/html; charset=UTF-8 date: Tue, 14 Dec 2021 11:03:32 GMT expires: Tue, 14 Dec 2021 11:03:32 GMT server: UploadServer x-guploader-uploadid: ADPycdsNLbraMoNCtWBN2etY5999RcEAzpzumosHd6kQb-O00g53MwwY1JciRK7UauOU4mbLo84Tmasvl-57QCo41NoP8s-8Pg

Request Headers

:authority: storage.googleapis.com
:method: OPTIONS
:path: /domain.appspot.com/media/20161017_213931.mp4?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=gcs-server%40origin-sports.iam.gserviceaccount.com%2F20211214%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20211214T110332Z&X-Goog-Expires=900&X-Goog-SignedHeaders=content-type%3Bhost%3Bx-goog-resumable&X-Goog-Signature=40d4d9bcfd6b62786144968bf3e7fe3a2820d7a42ab6a510832ddbea3aac65bf31ca59f85694125ac98da6c13d1ba740302f88164d5c53ecdd1e40eb4bdb431f34d103c2f5f2f7d0018a9ef0e4ff15978d834b4b3a2b17699a0dc7f8fabc49f99129d7d9b8de4341c0f6883c03a5ce16303811b278ca72f080167d0f4a1e7cc98076e473b7a65043976ddcf87532f52e9d2efefd48fae38bd3742e3e21ef86702b00cfe71b8b08fa506b886183146c94d61b747150ad2b5ae6ea668a5750dce27c5f212e9b60002e5bd09af0fee43a3566606f9063113a1f14d51d22af65eaf6503270f696e9bf50c9015c2af65ef8ec994f1949a4081d5872b2be09cb070e68
:scheme: https accept: / accept-encoding: gzip, deflate, br accept-language: en-US,en;q=0.9 access-control-request-headers: access-control-allow-origin,content-type,x-goog-resumable access-control-request-method: PUT cache-control: no-cache origin: http://localhost:8081 pragma: no-cache referer: http://localhost:8081/ sec-fetch-dest: empty sec-fetch-mode: cors sec-fetch-site: cross-site user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36

I don't post here too often so let me know if you need anything else or if I need to restructure this question.

// related things i've found that haven't worked

1. XMLHttpRequest CORS to Google Cloud Storage only working in preflight request
2. MalformedSecurityHeader error in signed url - Header was included in signedheaders, but not in the request
3. https://github.com/googleapis/nodejs-storage/issues/347

Answer 1

I can see that there are multiple problems to your question, but I wanted to focus on a part where you wanted to do a resumable upload using Signed URLs.

From what I understand, you're doing a resumable upload by sending a PUT request to a Signed URL. There's also no mention of using a Session URL to your question so there's already something missing. To clarify:

When working with resumable uploads, you only create and use a signed URL for the POST request that initiates the upload. This initial request returns a session URI that you use in subsequent PUT requests to upload the data.

Resumable uploads need a signed URL for the POST requests that would initiate the upload. This initial request will return a Session URI that you will use with PUT object request to upload data.

1. First, create a signed URL that accepts POST method, content-type and x-goog-resumable headers.

2. This would return a Session URI, which is the value of Location response header and will be used for single chunk uploads.

I suggest that you restructure your flow. This Github link should be useful to understand the flow. While the Gist is written in Ruby, you should still be able to see that the resumable upload requires a session URL. I would suggest to follow the same flow as well.