Redshift.MissingS3File - AWS Firehose

Question

Tried creating Firehose and ran into this error.

"deliveryStreamVersionId": 2, "message": "One or more S3 files required by Redshift have been removed from the S3 bucket. Check S3 bucket policies to remove any automatic deletion of S3 files.",

Nothing was removed from the bucket configured.

https://forums.aws.amazon.com/thread.jspa?threadID=231224 -- Other users having the same problem. But no help.

I waited for one day to see if the problem gets solved. Still the same.

Any insight on why this happening?

Answer 1

Did found out the core issue.

Firehose depends upon S3 to transfer data to redshift. Redshift.Missings3file happens due to following reasons.

1. Missing or wrong file in jsonpath
2. Roles not having permissions on S3
3. When firehoses initializes it writes and deletes a file for testing to verify permissions. If permissions is denied it throws the error.

Not straightforward error description but indicates it is missing a file under S3.