dragonfly-s3 with S3 IAM user causing a forbidden 403 response from Amazon

Question

I am using Rails 4.2.0 with dragonfly 1.0.10 and dragonfly-s3_data_store 1.1.0 with an Amazon IAM user set with a key and secret_key for auth and full permissions set for a specific bucket. When I try to save the model with the uploaded file, I get the following output.

logging output

Started PATCH "/admin/sections/1/weeks/1" for ::1 at 2015-06-09 13:20:15 -0400
Started PATCH "/admin/sections/1/weeks/1" for ::1 at 2015-06-09 13:20:15 -0400
  ActiveRecord::SchemaMigration Load (0.3ms)  SELECT "schema_migrations".* FROM "schema_migrations"
  ActiveRecord::SchemaMigration Load (0.3ms)  SELECT "schema_migrations".* FROM "schema_migrations"
Processing by Admin::WeeksController#update as HTML
Processing by Admin::WeeksController#update as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"uBK02MNoYAywWWf1K8zQQ/s/FnjY3LpLrX7O/4mYH/aJgX40CdFhqJEvz+B+A2U7X5NqXNLPM3XGp0mTtgHqAA==", "commit"=>"Save", "week"=>{"title"=>"foo", "video"=>#<ActionDispatch::Http::UploadedFile:0x007f9d888e68b8 @tempfile=#<Tempfile:/var/folders/3j/0zbm_nfs02b214_p2gszlb_r0000gn/T/RackMultipart20150609-20229-1s8vt4p.js>, @original_filename="moment.js", @content_type="text/javascript", @headers="Content-Disposition: form-data; name=\"week[video]\"; filename=\"moment.js\"\r\nContent-Type: text/javascript\r\n">, "description"=>"test", "reminder"=>"", "fla_instructions"=>""}, "section_id"=>"1", "id"=>"1"}
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"uBK02MNoYAywWWf1K8zQQ/s/FnjY3LpLrX7O/4mYH/aJgX40CdFhqJEvz+B+A2U7X5NqXNLPM3XGp0mTtgHqAA==", "commit"=>"Save", "week"=>{"title"=>"title", "video"=>#<ActionDispatch::Http::UploadedFile:0x007f9d888e68b8 @tempfile=#<Tempfile:/var/folders/3j/0zbm_nfs02b214_p2gszlb_r0000gn/T/RackMultipart20150609-20229-1s8vt4p.js>, @original_filename="moment.js", @content_type="text/javascript", @headers="Content-Disposition: form-data; name=\"week[video]\"; filename=\"moment.js\"\r\nContent-Type: text/javascript\r\n">, "description"=>"blah, description", "reminder"=>"", "fla_instructions"=>""}, "section_id"=>"1", "id"=>"1"}
  User Load (0.7ms)  SELECT  "users".* FROM "users" WHERE "users"."id" = $1  ORDER BY "users"."id" ASC LIMIT 1  [["id", 2822]]
  User Load (0.7ms)  SELECT  "users".* FROM "users" WHERE "users"."id" = $1  ORDER BY "users"."id" ASC LIMIT 1  [["id", 2822]]
  Section Load (0.2ms)  SELECT  "sections".* FROM "sections" WHERE "sections"."id" = $1  ORDER BY "sections"."order" ASC LIMIT 1  [["id", 1]]
  Section Load (0.2ms)  SELECT  "sections".* FROM "sections" WHERE "sections"."id" = $1  ORDER BY "sections"."order" ASC LIMIT 1  [["id", 1]]
  Week Load (0.3ms)  SELECT  "weeks".* FROM "weeks" WHERE "weeks"."section_id" = $1 AND "weeks"."id" = $2  ORDER BY "weeks"."order" ASC LIMIT 1  [["section_id", 1], ["id", 1]]
  Week Load (0.3ms)  SELECT  "weeks".* FROM "weeks" WHERE "weeks"."section_id" = $1 AND "weeks"."id" = $2  ORDER BY "weeks"."order" ASC LIMIT 1  [["section_id", 1], ["id", 1]]

   (0.2ms)  BEGIN
   (0.2ms)  BEGIN
[fog][WARNING] Unable to fetch credentials: connect timeout reached
[fog][WARNING] Unable to fetch credentials: connect timeout reached
[fog][WARNING] Unable to fetch credentials: connect timeout reached
   (0.2ms)  ROLLBACK
   (0.2ms)  ROLLBACK
Excon::Errors::Forbidden: Expected(200) <=> Actual(403 Forbidden)
excon.error.response
  :body          => "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>440C1D6D8B3F6067</RequestId><HostId>EqSSx+F4XWu14oL04q6fXaiC7K+24yQcmSVW0twY5HQhnsH2FG4Y9EKJafzjx0hJ</HostId></Error>"
  :headers       => {
    "Content-Type"     => "application/xml"
    "Date"             => "Tue, 09 Jun 2015 17:23:25 GMT"
    "Server"           => "AmazonS3"
    "x-amz-id-2"       => "EqSSx+F4XWu14oL04q6fXaiC7K+24yQcmSVW0twY5HQhnsH2FG4Y9EKJafzjx0hJ"
    "x-amz-request-id" => "440C1D6D8B3F6067"
  }
  :local_address => "172.16.2.86"
  :local_port    => 53540
  :reason_phrase => "Forbidden"
  :remote_ip     => "54.231.244.0"
  :status        => 403
  :status_line   => "HTTP/1.1 403 Forbidden\r\n"
from /Users/dakotabrink/.rvm/gems/ruby-2.2.1/gems/excon-0.45.3/lib/excon/middlewares/expects.rb:6:in `response_call'

config/initializers/dragonfly.rb

require 'dragonfly/s3_data_store'

# Configure
Dragonfly.app.configure do
  plugin :imagemagick

  secret "secret goes here"

  url_format "/media/:job/:name"

  datastore :s3,
            bucket_name: ENV['S3_BUCKET'],
            access_key_id: ENV['S3_KEY'],
            secret_access_key: ENV['S3_SECRET'],
            use_iam_profile: true
end

# Logger
Dragonfly.logger = Rails.logger

# Mount as middleware
Rails.application.middleware.use Dragonfly::Middleware

# Add model functionality
if defined?(ActiveRecord::Base)
  ActiveRecord::Base.extend Dragonfly::Model
  ActiveRecord::Base.extend Dragonfly::Model::Validations
end

IAM Policy Settings

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1433856191000",
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-name/*"
            ]
        }
    ]
}

The failure response seems really ambiguous about where it's failing. The IAM user's "last used" field is being updated updated on each "refused" request so it seems that Amazon is recognizing an attempted login. Any help would be appreciated.

Answer 1

If you set use_iam_profile to true it's going to ignore the access_key_id and secret_access_key, and attempt to load the credentials automatically. When you're running an ec2 box with an IAM profile loaded this would work, but since you're adding keys I'm assuming this isn't the case.

From the dragonfly s3 docs: :use_iam_profile # boolean - if true, no need for access_key_id or secret_access_key

More information on IAM profiles via EC2 and how they interact with fog can be found at http://www.spacevatican.org/2012/6/25/iam-roles/