In our project we have photos and other media that are stored on S3. The server code is able to authenticate (JWT) and authorize requests to the files.
We need to deliver scaled-down thumbnails for the media, currently this goes through the webserver, where the photos are processed on demand and then cached. This does not scale.
I found 2 limitations that make the solutions to be found when googling unsuitable for us:
- It is crucial for us that this system is replicable on our local Docker setup as well as in our CI pipeline, so we can test end-to-end extensively. So a service that will just read from S3 won't work, as it needs to access our local machines for example.
- Authentication and authorization of requests is critical. A one-time presigned URL will not do the job.
I struggle to find designs for these requirements.