Uploading images to S3 using NodeJS and Multer. How to upload whole file onFileUploadComplete

4.9k views Asked by At

I'm using NodeJS and Multer to upload files to S3.

On the surface, everything appears to be working, the files get uploaded, and I can see them in the bucket when I log into the AWS console. However, most of the time when I follow the link to the file, the file is broken, often the file size is much smaller than the original file.

When the file reaches the server, the file size is correct if I log it, but on S3 it is much smaller. For example I just uploaded a file which is 151kb. The post request logs the file size correctly, but on S3 the file says it's 81kb.

Client side:

uploadFile = (file) ->
    formData = new FormData()
    formData.append 'file', file


    xhr = new XMLHttpRequest()
    xhr.open "POST", "/upload-image", true
    # xhr.setRequestHeader("Content-Type","multipart/form-data");

    console.log 'uploadFile'

    xhr.onerror = ->
        alert 'Error uploading file'
    xhr.onreadystatechange = ->
        if xhr.readyState is 4
            console.log xhr.responseText

    xhr.send formData

Server:

app.use(multer({ // https://github.com/expressjs/multer
      inMemory: true,
      limits : { fileSize:3000000 },
      rename: function (fieldname, filename) {
        var time = new Date().getTime();
        return filename.replace(/\W+/g, '-').toLowerCase() + '_' + time;
      },
      onFileUploadData: function (file, data, req, res) {
        var params = {
          Bucket: creds.awsBucket,
          Key: file.name,
          Body: data,
          ACL: 'public-read'
        };

        var s3 = new aws.S3();
        s3.putObject(params, function (perr, pres) {
          if (perr) {
            console.log("Error uploading data: ", perr);
          } else {
            console.log("Successfully uploaded data", pres);
          }
        });
      }
    }));

    app.post('/upload-image', function(req, res){
        if (req.files.file === undefined){
            res.end("error, no file chosen");
        } else if (req.files.file.truncated) {
            res.end("file too large");
        } else {
          console.log(req.files.file.size); //logs the correct file size
          var path = creds.awsPath + req.files.file.name;
          res.type('text/plain');
          res.write(path);
          res.end();
        };

    });

EDIT:

Setting file.buffer to the body perma onFileUploadComplete seems to work, but I have a feeling that this isn't the proper way of doing things, and may come back to bite me later. Is this approach okay, or are there issues I should be aware of doing this?

0

There are 0 answers