Streaming a ByteArrayOutputStream to an akka http response

Question

I'm creating a ByteArrayOutputStream using ZIO Streams i.e.:

lazy val byteArrayOutputStream = new ByteArrayOutputStream()
val sink = ZSink.fromOutputStream(byteArrayOutputStream).contramapChunks[String](_.flatMap(_.getBytes)
val data = ZStream.unwrap(callToFunction).run(sink)

This works fine - now I need to stream this data back to the client using akka http. I can do this:

val arr = byteArrayOutputStream.toByteArray
complete(HttpEntity(ContentTypes.`application/octet-stream`, arr)

which works but of course the toByteArray brings the outputstream into memory i.e. I don't stream the data. I'm missing something obvious - is there an easy way to do this?

Answer 1

You can convert output stream to Akka Stream Source:

val byteArrayOutputStream = new ByteArrayOutputStream()
val source = StreamConverters.asOutputStream().mapMaterializedValue(_ => byteArrayOutputStream)

and then simply create a chunked HTTP entity:

HttpResponse(entity = HttpEntity.Chunked.fromData(ContentTypes.`application/octet-stream`, source))

More about chunked transfer: https://datatracker.ietf.org/doc/html/rfc7230#section-4.1

For ZIO, you could probably use something like this:

val zSource = ZStream.fromOutputStreamWriter(os => byteArrayOutputStream.writeTo(os))

However, you need to find a way to convert ZStream to Akka Stream Source.