How to configure AWS S3 SDK for Node.JS to be used with localhost?

Question

I'm trying to use fakes3 as an endpoint for some simple S3 code I've written. I can't get beyond the connection stage though.

The current error is: NetworkingError: getaddrinfo ENOTFOUND.

I've got configuration setup:

"aws": {                                
    "accessKeyId": "123",               
    "secretAccessKey": "abc",           
    "region": "",                       
    "endpoint": "http://localhost:8081",
    "sslEnabled": false                 
}               

var AWS = require('aws-sdk');
// loaded the config into an object called `config`:

AWS.config.update(config.aws);
s3 = new AWS.S3();
// also tried creating an `EndPoint`:
s3.endpoint = new AWS.Endpoint(config.aws.endpoint);  

When I try simple code like:

s3.putObject({ Bucket: 'logging', Key: "logging123", Body: "started" }, 
   function(err, data) {
   if (err) {
       console.log(err);
   }
});

The error mentioned above occurs. When I leave out directly setting the endPoint, the request is made to the East AWS region (and ignores the endpoint value I've passed in via configuration).

And, I'm running fakes3 using the command line:

fakes3 -r c:\temp\_fakes3 -p 8081
Loading FakeS3 with c:/temp/_fakes3 on port 8081 with hostname s3.amazonaws.com
[2013-11-30 14:20:22] INFO  WEBrick 1.3.1
[2013-11-30 14:20:22] INFO  ruby 2.0.0 (2013-06-27) [x64-mingw32]
[2013-11-30 14:20:22] INFO  WEBrick::HTTPServer#start: pid=11800 port=8081

Answer 1

In the AWS SDK docs there is the following option:

s3ForcePathStyle (boolean)

Returns whether to force path style URLs for S3 objects

I've tested it works with this config:

var config = {
  accessKeyId: "123",
  secretAccessKey: "abc",
  endpoint: "localhost:3000",
  sslEnabled: false,
  s3ForcePathStyle: true
};
AWS.config.update(config);
s3Client = new AWS.S3();

Answer 2

The request is being sent to:

logging.127.0.0.1:8081

The '"logging"` is the bucket name (which makes sense as the SDK for S3 always sends the bucket as part of the URL). As that name doesn't properly resolve to the computer, it was failing.

So, instead try using a simple dynamic DNS from 37signals and change the configuration to be:

"endpoint": "127.0.0.1.xip.io:8081/"

In total:

"aws": {                                
    "accessKeyId": "123",               
    "secretAccessKey": "abc",           
    "region": "",                       
    "endpoint": "127.0.0.1.xip.io:8081/",
    "sslEnabled": false                 
}               

There may be other options for this, but http://xip.io grabs the IP address from the subdomain and always returns that. So, in the case above, the actual url sent is: "logging.127.0.0.1.xip.io", but the DNS server at xip.io returns 127.0.0.1.

I could have edited and maintained local DNS files, but this was a simple solution that was clean enough for my purposes.

In order to track this issue down, I looked at the source code, and in particular the file node.js in the handleRequest function, which is where the actual http request was being sent.

Answer 3

If you're using the aws sdk version 3. The config structure has changed and s3ForcePathStyle is now renamed to forcePathStyle

const { S3 } = require("@aws-sdk/client-s3");

const config = {
          "credentials": {
            "accessKeyId": "minioadmin",
            "secretAccessKey": "minioadmin"
          },
          "endpoint": "http://127.0.0.1:9099",
          "sslEnabled": false,
          "forcePathStyle": true
    };
const s3Client = new S3(config);

// example of usage
const response = await s3Client.listObjectsV2({
      Bucket: 'bucketname',
      Prefix: 'prefix'
    });