What's the correct way of loading AWS credentials from role in a Fargate task using AWS SDK Go?

Question

I have the following snippet:

awsCredentials := credentials.NewChainCredentials(
    []credentials.Provider{
        &ec2rolecreds.EC2RoleProvider{
            Client: ec2metadata.New(newSession, aws.NewConfig()),
        },
        &credentials.SharedCredentialsProvider{},
        &credentials.EnvProvider{},
    })

which works fine whenever the code is running on an EC2 instance or when the access/secret key are passed through variables (used for local testing).

However, this code is failing when running on ECS+Fargate because NoCredentialProviders: no valid providers in chain. Checked the environment variables of the running container and it has the expected AWS_CONTAINER_CREDENTIALS_RELATIVE_URI, so the credentials.EnvProvider should read it.

So, my question is, what's the correct way of reading these credentials? Because the problem I'm facing is not about lack of permissions (which would indicate an error in the policy / role), but that code is not able to get the credentials.

UPDATE

I have narrowed this to the use of ec2rolescreds.

Using this simple example:

package main

import (
    "fmt"
    "log"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/credentials"
    "github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds"
    "github.com/aws/aws-sdk-go/aws/ec2metadata"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
)

func main() {
    newSession, err := session.NewSession()

    if err != nil {
        log.Fatal(err)
    }

    awsCredentials := credentials.NewChainCredentials(
        []credentials.Provider{
            &ec2rolecreds.EC2RoleProvider{
                Client: ec2metadata.New(newSession, aws.NewConfig()),
            },
            &credentials.SharedCredentialsProvider{},
            &credentials.EnvProvider{},
        })

    sess, err := session.NewSession(&aws.Config{
        Region:      aws.String("us-east-1"),
        Credentials: awsCredentials},
    )

    if err != nil {
        log.Fatal(err)
    }
    // Create S3 service client
    svc := s3.New(sess)

    result, err := svc.ListBuckets(nil)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("Buckets:")

    for _, b := range result.Buckets {
        fmt.Printf("* %s created on %s\n",
            aws.StringValue(b.Name), aws.TimeValue(b.CreationDate))
    }
}

If I remove ec2rolescreds, everything works fine both local and in ECS+Fargate.

However, if I run this code as is, I get the same error of NoCredentialProviders: no valid providers in chain

Answer 1

So, a session can be configured using a Config object.

Reading through the specs of this object, it says for Credentials:

// The credentials object to use when signing requests. Defaults to a
// chain of credential providers to search for credentials in environment
// variables, shared credential file, and EC2 Instance Roles.
Credentials *credentials.Credentials

The defaults are already what my snippet was doing, so I removed all the awsCredentials block and now it's working fine everywhere. Locally, EC2, Fargate...

UPDATE

To expand the answer, the reason why removing the awsCredentials made this work is because, if you check the SDK's code, https://github.com/aws/aws-sdk-go/blob/master/aws/defaults/defaults.go#L107, the default credentials check both EnvProvider and RemoteCredProvider.

By overriding the default chain credentials, it was not able to look for credentials in RemoteCredProvider, which is the provider that handles the environment variable AWS_CONTAINER_CREDENTIALS_FULL_URI.

Answer 2

The solution is to initialize clients using sessions instead of credentials, i.e:

conf := aws.NewConfig().WithRegion("us-east-1")
sess := session.Must(session.NewSession(conf))

svc := s3.New(sess)
// others:
//   svc := sqs.New(sess)
//   svc := dynamodb.New(sess)
//   ... 

Because, as @Ay0 points, the default credential chain already includes both EnvProvider and RemoteCredProvider.

In case you still need the credentials, you can use:

creds := stscreds.NewCredentials(sess, "myRoleARN")

as the documentation points out. Notice that the policy of the role must have the sts:AssumeRole action enabled. For more information, here are the stscreds.NewCredentials(...) docs