We're currently storing user-uploaded files on our ColdFusion server, and the disk usage is getting out of hand. Given the easy integration we would like to move these files off onto S3 and just allows the users to access the files directly from there, but how to handle permissions is a little unclear to me.
Once I upload a file I can use CF to set the ACL for the file or bucket. However, to create that ACL struct I'll need a list of the users who can be allowed access. Older documentation lists that users can be specified by email address, but a new piece of documentation I found states
You cannot use an email address to specify a grantee for any AWS Region that was created after 12/8/2014
which indicates email lookups are becoming legacy. That same doc provides instructions for looking up my own canonical user ID, or the canonical user ID for another account by reading existing ACLs, but I have no other ACLs to reference.
Assuming I have only a user's first/last name and email, provided to my system during their registration, how can I find their canonical user ID to assign them read access to my S3 bucket? Or in this situation is it possible to generate a one-time-use access key that I can hand off to users before forwarding them to the s3 file?
You have two basic options here, both of which require you to get to know, love, and hate IAM.
Your application can handle all requests to S3 and your application must know that person X is authorized to access file Q. Your application handles all the authorization, and makes requests to S3 to add/retrieve/delete files as needed. You'd want to make the bucket not world-readable, to protect the individual files in the bucket. Any request to any file in the bucket would have to be made through CF, rather than directly to the bucket itself.
You create IAM accounts for all individuals who need to store files on S3. You then pass that individual's IAM credentials in a request to S3 and S3 validates that the provided credentials do indeed have access to the files on S3. You use those same IAM credentials when adding the file to say "hey, only this person with these IAM credentials can access this file" -- effectively supplying the IAM account information in the ACL.
I'd suggest letting your application handle the access control, rather than trying to create IAM accounts for every one of your users. The main reason to have individual IAM accounts for all users is if multiple applications will use the same S3 buckets.
You mention one-time access keys, but that service (Simple Token Service, or STS) is really for virtual machine instances in EC2 to get a temporary token to access other AWS services, as needed (particularly during startup). It's not designed for individual applications outside AWS to provide one-time access to AWS services.
In both cases, you will also have to sign your requests. There are a number of S3-related CFCs out there, but none of them can handle the current Signature v4 signing process. You will need to use the AWS SDK for Java to do this. It's not hard, but it means you have to tap into the .jar files for the SDK for signing requests. I have examples of using the AWS SDK for Java in CF in my sample AWS PlayBox CF application.