How can I upload a file that is a binary string to Amazon S3 using a signed URL and JavaScript?

Question

I am not having any trouble getting the signed URL, and the file is actually getting uploaded to S3, but when I download the file I can't open it. I have tried PDF and image files.

I get the file like this, where 'e' is the file upload event from the browser file input:

   let fileData = e.target.files[0];
   let reader = new FileReader();
   reader.readAsBinaryString(fileData); // generates a binary representation of the image
   reader.onload = function(e) {
      let bits = e.target.result;
      let data = {
         originalFilename: fileData.name,
         filename: fileData.name,
         mimeType: fileData.type,
         fileSizeBytes: fileData.size,
         lastModified: fileData.lastModified,
         bin: bits
      };
   }

I store the 'data' json in the IndexeddB and later when the browser is online I get a signed URL and try to upload it like this:

// signedUrl is the signed URL
// data is the saved file data from the IndexeddB (above)
let contentType = data.mimeType
let binaryString = data.bin;  // bin is a binary string
let formData = new FormData();
formData.append("file", btoa(data.bin));

// upload the file directly to AWS
return $.ajax({
    url: signedUrl,
    method: "POST",
    contentType: contentType ,
    data: formData,
    processData: false
})
.then(function (response) {
    console.log(response);

})
.catch(function (e) {
    console.log('Error in file upload to AWS:');
    console.log(e);
    throw('Upload failed');
})

There are lots of examples which show how to post the file to the signed URL if you have the File object (or you are using Postman), but my web app allows users to "upload" files offline and they are stored in the IndexeddB as binary strings. This all works fine, and I can easily POST the files to my server, recreate the file, and then send them to S3, but I want to avoid the extra trips.

I have tried creating a Blob and quite a few other things and I am stuck. Any help would be greatly appreciated.

Really, all I need to know is "Exactly what format is the file that gets posted to the signed URL in the post data and how can I convert my file data to that format?"

Answer 1

You can simply pass binary encoding option when doing upload at aws s3 like below -

 const params = {
   Bucket: bucketName,
   Key: fileName,
  Body: Buffer.from(binaryString, "binary"), // when base64 data pass "base64"
};

s3.upload(params, (error, data) => {})

Answer 2

OK, I finally figured out exactly what to do. There is very little documentation on the Amazon site, and a lot of misinformation on the web. You just have to recreate the file Blob (and DO NOT use fileData):

// signedUrl is the signed URL
// data is the saved file data from the IndexeddB (above)
let contentType = data.mimeType
let binaryString = data.bin;  // bin is a binary string

// rebuild the file object as a Blob to send to Amazon
let bytes = new Uint8Array(binaryString.length);

for (let i=0; i < binaryString.length; i++) {
    bytes[i] = binaryString.charCodeAt(i);
}

let file = new Blob([bytes], {type: contentType});

// upload the file directly to AWS
return $.ajax({
    url: signedUrl,
    method: "POST",
    contentType: false,
    data: file,
    processData: false
})
.then(function (response) {
    console.log(response);

})
.catch(function (e) {
    console.log('Error in file upload to AWS:');
    console.log(e);
    throw('Upload failed');
})