ContentMD5 Amazon S3 Upload Bad Digest

3.6k views Asked by At

I am able to upload files to S3, however when passing the ContentMD5 param, I always get the error:

{ [BadDigest: The Content-MD5 you specified did not match what we received.]
  message: 'The Content-MD5 you specified did not match what we received.',
  code: 'BadDigest',
  time: Mon Jun 15 2015 17:47:19 GMT-0400 (EDT),
  statusCode: 400,
  retryable: false,
  retryDelay: 30 }

Edit: Added link Now the docs from amazon say:

   The output of the MD5 algorithm is a 128 bit digest.  When viewed in
   network byte order (big-endian order), this yields a sequence of 16
   octets of binary data.  These 16 octets are then encoded according to
   the base64 algorithm in order to obtain the value that is placed in
   the Content-MD5 field.  Thus, if the application of the MD5 algorithm
   over the raw data of a MIME entity results in a digest having the
   (unlikely) value of "Check Integrity!", then that MIME entity's
   header could contain the field

So it would appear that with a file that returns this md5,

computer:NU isaac$ md5 ~/Desktop/mediumFile.dat 
MD5 (/Users/isaac/Desktop/mediumFile.dat) = ce377789add2698f68d4cb7c021e7f55

I would have to convert the hex representation (2 char bytes) into the base64 representation. In nodejs, I try the following,

var hexBuffer = new Buffer('ce377789add2698f68d4cb7c021e7f55', 'hex');
var base64MD5String = hexBuffer.toString('base64'); // returns 'zjd3ia3SaY9o1Mt8Ah5/VQ=='

however, I get the bad digest error when passing the base64MD5String as the ContentMD5 parameter in AWS.S3.upload. What's wrong with the way I calculate/ encode the MD5?

Edit: I am using the Amazon example titled 'uploading an arbitrarily sized stream' however, I am already working with files that are tar'd and gzip'd so im not piping into zlib.

var fs = require('fs');

var body = fs.createReadStream('bigfile');
var s3obj = new AWS.S3({params: {Bucket: 'myBucket', Key: 'myKey'}});
s3obj.upload({Body: body}).
  on('httpUploadProgress', function(evt) { console.log(evt); }).
  send(function(err, data) { console.log(err, data) });
1

There are 1 answers

0
Thavaprakash Swaminathan On

Below what i did, it works fine.

var md5 = require('md5');
var hash = md5('test');

ContentMD5: new Buffer(hash,'hex').toString('base64');