AWS: Mount S3 Bucket to an EC2 instance. (Later FTP Tunneling)

Question

what do I want to do?

Step1: Mount a S3 Bucket to an EC2 Instance.

Step2: Install a FTP Server on the EC2 Instance and tunnel ftp-requests to files in the bucket.

What did I do so far?

• create bucket
• create security group with open input ports (FTP:20,21 - SSH:22 - some more)
• connect to ec2

And the following code:

wget https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/s3fs/s3fs-1.74.tar.gz
tar -xvzf s3fs-1.74.tar.gz
yum update all
yum install gcc libstdc++-devel gcc-c++ fuse fuse-devel curl-devel libxml2-devel openssl-devel mailcap
cd s3fs-1.74
./configure  --prefix=/usr
make
make install
vi /etc/passwd-s3fs # set access:secret keys
chmod 640 /etc/passwd-s3fs
mkdir /s3bucket
cd /s3bucket

And cd anwers: Transport endpoint is not connected

Dunno what's wrong. Maybe I am using the wrong user? But currently I only have one user (for test reasons) except for root.

Next step would be the ftp tunnel, but for now I'd like getting this to work.

Answer 1

I followed these instructions now. https://github.com/s3fs-fuse/s3fs-fuse

I guess they are calling the API in background too, but it works as I wished.

Answer 2

Point 1

Whilst the other answerer is correct in saying that S3 is not built for this, it's not true to say a bucket cannot be mounted (I'd seriously consider finding a better way to solve your problem however).

That being said, you can use s3fuse to mount S3 buckets within EC2. There's plenty of good reasons not to do this, detailed here.

Point 2

From there it's just a case of setting up a standard FTP server, since the bucket now appears to your system as if it is any other file system (mostly).

vsftpd could be good choice for this. I'd have a go at both and then post separate questions with any specific problems you run into, but this should give you a rough outline to work from. (Well, in reality I'd have a go at neither and use S3 via app code consuming the API, but still).

Answer 3

One possible solution to mount S3 to an EC2 instance is to use the new file gateway.

Check out this: https://aws.amazon.com/about-aws/whats-new/2017/02/aws-storage-gateway-supports-running-file-gateway-in-ec2-and-adds-file-share-security-options/

http://docs.aws.amazon.com/storagegateway/latest/userguide/WhatIsStorageGateway.html