I am able to upload files to S3, however when passing the ContentMD5 param, I always get the error:
{ [BadDigest: The Content-MD5 you specified did not match what we received.]
message: 'The Content-MD5 you specified did not match what we received.',
code: 'BadDigest',
time: Mon Jun 15 2015 17:47:19 GMT-0400 (EDT),
statusCode: 400,
retryable: false,
retryDelay: 30 }
Edit: Added link Now the docs from amazon say:
The output of the MD5 algorithm is a 128 bit digest. When viewed in
network byte order (big-endian order), this yields a sequence of 16
octets of binary data. These 16 octets are then encoded according to
the base64 algorithm in order to obtain the value that is placed in
the Content-MD5 field. Thus, if the application of the MD5 algorithm
over the raw data of a MIME entity results in a digest having the
(unlikely) value of "Check Integrity!", then that MIME entity's
header could contain the field
So it would appear that with a file that returns this md5,
computer:NU isaac$ md5 ~/Desktop/mediumFile.dat
MD5 (/Users/isaac/Desktop/mediumFile.dat) = ce377789add2698f68d4cb7c021e7f55
I would have to convert the hex representation (2 char bytes) into the base64 representation. In nodejs, I try the following,
var hexBuffer = new Buffer('ce377789add2698f68d4cb7c021e7f55', 'hex');
var base64MD5String = hexBuffer.toString('base64'); // returns 'zjd3ia3SaY9o1Mt8Ah5/VQ=='
however, I get the bad digest error when passing the base64MD5String as the ContentMD5 parameter in AWS.S3.upload. What's wrong with the way I calculate/ encode the MD5?
Edit: I am using the Amazon example titled 'uploading an arbitrarily sized stream' however, I am already working with files that are tar'd and gzip'd so im not piping into zlib.
var fs = require('fs');
var body = fs.createReadStream('bigfile');
var s3obj = new AWS.S3({params: {Bucket: 'myBucket', Key: 'myKey'}});
s3obj.upload({Body: body}).
on('httpUploadProgress', function(evt) { console.log(evt); }).
send(function(err, data) { console.log(err, data) });
Below what i did, it works fine.