Use Axum Multipart for S3 upload

Question

I have a multipart form endpoint in my Axum app. File upload form field is supposed to be read like this:

// let mut field: axum::extract::multipart::Field
while let Some(chunk) = field.chunk().await? {
    // Do something with chunk: Bytes
}
// Done

On other side I have s3::Bucket (rust-s3 crate) and its method put_object_stream expects tokio::io::AsyncRead. How to make them work together without saving the whole file in the memory or on the disk?

Answer 1

If you have a multipart Field and want to use it as a tokio::io::AsyncRead type, you can use tokio_util::io::StreamReader since Field implements Stream<Item = Result<Bytes, MultipartError>>. Though because AsyncRead's interface uses std::io::Error you'll have to convert MultipartError to it first.

[dependencies] # additional
futures = "0.3.28"
tokio-util = { version = "0.7.9", features = ["io"] }

use axum::extract::multipart::Field;
use futures::TryStreamExt;
use tokio_util::io::StreamReader;

let field: Field = ...;

let reader = StreamReader::new(field.map_err(|multipart_error| {
    std::io::Error::new(std::io::ErrorKind::Other, multipart_error)
}));