So on my local machine something like this work fine:
library(arrow)
library(aws.s3)
Sys.setenv(
"AWS_ACCESS_KEY_ID" = Sys.getenv("awsaccesskey"),
"AWS_SECRET_ACCESS_KEY" = Sys.getenv("awssecret"),
"AWS_DEFAULT_REGION" = "eu-west-2"
)
feather_data <- s3read_using(read_feather, bucket = "amazingbucket", object = "somefile.feather")
If I wrap this into a docker image, and I want to avoid hardcoding AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY, which come here from Windows environment variables, how does ECR get this information?
When you run this in AWS there is a notion of
IAM role
you can attach to your execution environment. If you are running your container, say, on ECS, you will attach to your task an IAM role. If you are running your container in EKS you will use this method.The long story short is that AWS will inject those value dynamically (and it will rotate the temp creds) and the AWS SDK will be able to source that information automatically.