Amazon S3: move files out of a bucket and into Glacier

Question

I have an S3 bucket (call it main_bucket) to which various TIFF files are uploaded. On upload to main_bucket, a Lambda function is triggered which converts the TIFF to a JPEG and also stores the JPEG in the bucket. I would like to then store the TIFF in Glacier and remove it from main_bucket, and I am not sure how to do this.

I can, of course, create a second bucket (call it archive), set the lifecycle on archive so that it stores all contents in the Glacier storage class, and set lifecycle rules on main_bucket so that all TIFF files in main_bucket are copied to archive. This will ensure that a copy of an uploaded TIFF ends up in archive, and since all things in archive are in the Glacier storage class, this means that all TIFFs end up in Glacier, which is what I want. But: this approach also keeps a copy of each TIFF in main_bucket, which I don't want. I can delete the TIFFs from main_bucket, but I don't know when to do that; I don't believe there's an event corresponding to the copy to archive being completed. Is there some way to do what I'm looking for?

Answer 1

Moving objects to the Glacier or Glacier Deep Archive storage class is an excellent way to archive objects that are rarely (potentially never) accessed.

It appears that your scenario is:

• TIFF is uploaded to Bucket-A
• This triggers an AWS Lambda function that converts the TIFF to a JPG and stores it in the same bucket
• You would then like the TIFF moved to a Bucket-B and moved into a Glacier storage class

Option 1: Swap bucket order

• Have the TIFF uploaded to Bucket-B
• This then triggers the Lambda function that stores the JPG in Bucket-A (a different bucket!)
• Add a Lifecycle rule to Bucket-B that archives the TIFFs after a certain period

The beauty of this option is that no objects need to be moved. Objects are always created in the bucket where they are desired.

Option 2: Move the object as part of the Lambda function

• After the Lambda function creates the JPG in Bucket-A, it should also:
• Copy the TIFF to Bucket-B
• Delete the TIFF from Bucket-A
• Add a Lifecycle rule to Bucket-B that archives the TIFFs after a certain period