AWS S3 "ACCESS DENIED" trying on Rails Sitemap_Generator

Question

So I'm following the instructions on the sitemap_generator Gem for uploading the sitemap to S3 (since it wasn't being read from the "public" folder) and I keep getting the following error:

In '/var/www/mysite/releases/20200902316/public/':
rake aborted!
Aws::S3::Errors::AccessDenied: Access Denied

This is kind of funny given that I'm using the exact same credentials for Active_Storage on S3 and that's working just fine.

Here's my sitemap.rb

require 'aws-sdk-s3'

# Set the host name for URL creation
SitemapGenerator::Sitemap.default_host = "https://mywebsite.com"

SitemapGenerator::Sitemap.adapter = SitemapGenerator::AwsSdkAdapter.new('s3_bucket',
  aws_access_key_id: 'my_access_key',
  aws_secret_access_key: 'my_secret',
  aws_region: 'us-east-1'
)

This is all straight from the ReadMe on the site_generator page... So I can't quite understand why it wouldn't be working if the same credentials work fine for Active Storage...

https://github.com/kjvarga/sitemap_generator

Answer 1

Ok... This is going to be a little bit INVOLVED because there's more to getting this to work than just the answer to my question. But let's start there.

The reason why I wanted to do this in the first place is because Google Search Console couldn't access the sitemap in my rails "public" folder. So I decided to have Sitemap Generator send it to my S3 bucket instead.

That's where I encountered the "Access Denied" error. (Be sure to use the "environment" variable... otherwise you won't be hitting the right database.)

RAILS_ENV=production bundle exec rake sitemap:refresh

The reason you get this error is because, by default, Amazon has all S3 buckets set to: "Block Public Access." So you need to disable the "Block Public Access" feature as shown below under the S3 bucket permissions. (This may look different by the time you read this because they've changed that configuration many times before.)

Once you've disabled that you should be able to upload your sitemap just fine. BUT... you still won't be able to connect this sitemap to Google Search Console because you'll need to add your bucket to the search console as well.

http://yourbucketname.s3.amazonaws.com/

You can verify your bucket by uploading the provided goggle html file to your bucket's root directory then clicking "VERIFY."

Then you can finally upload your Sitemap Generator sitemap:

http://yourbucketname.s3.amazonaws.com/sitemaps/sitemap.xml.gz

It will still say it can't fetch it... but after a couple of days it should start updating.