get RequestBody in httpclient

Question

I want to use Haskell send request to Amazon hosted elasticsearch server either using bloodhound or REST api directly.

Since the server has IAM based access policy, I need sign the signature. In order to sign the signature for every single request, the hash of payload needs to computed. I am not sure how to get RequestBody (chunked or not) into ByteString.

Answer 1

For your particular usecase: it looks to me like the wreq library already supports AWS IAM signing (tutorial), so depending on your particular requirements, it might be easiest to just go with that/look into how it works.

It looks like the easiest way to work with RequestBodys is probably actually to just write a function that calculates the signature for each of the 6 possible kinds of RequestBodys, or at least the ones that you need, without trying to reuse http-client's machinery to convert any of the 6 into one ByteString. This is a particularly useful option because it looks like you may need to do special things for chunked requests. There are only a few options for what a RequestBody can be, and only two (the stream based ones) seem significantly difficult to work with; those ones are also often deserving of special treatment (especially since, depending on how the creator of the request implemented them, it's unclear to me whether it's guaranteed to be possible to read from them & have them still work later). This source may be useful for this approach.

Depending on your experience with Haskell, the stream constructors may be a bit intimidating. However it is actually not bad: expanding out the type synonyms gives that GivesPopper () = (IO ByteString -> IO ()) -> IO (). The IO ByteString -> IO () function is something that you can supply that takes a producer of stream chunks (each evaluation will yield one more chunk) and does something useful with it---for example, write that chunk into a list in an IORef so that you can inspect it later. If you call the GivesPopper on this function, you will get an IO action which runs it with a useful producer as an argument. For example:

foo :: NeedsPopper ()
foo pop = do
  chunk <- pop
  if (chunk == "") then return ()
                   else BS.putStr chunk >> foo pop

Will, when passed to a GivesPopper (), print the streamed response body to stdout.

If you expect that the request can be built multiple times without issue (any stream GivesPoppers must be callable multiple times, etc.), and you wish to reuse http-client's internal response rendering, you may be able to get away with something terribly hacky like this:

getRequest :: Request -> IO BS.ByteString
getRequest req = do
  (conn, out, inp) <- dummyConnection []
  let req' = req { requestHeaders = (CI.mk "Expect", "100-continue")
                                  : requestHeaders req
                 }
  (Just later) <- requestBuilder req' conn
  _ <- out
  later
  BS.concat <$> out

It seems that the only place that http-client renders a Response is in requestBuilder, and when building the request, this will always send the headers, which I assume is not what you want. The _ <- out line clears the header+body from the dummy connection, and, since Expect: 100-continue is given, later should then write the body again to dummy connection. Notice that this only works when the response can be built multiple times without issue. If your request actually expected to use the continue functionality for something different, this may not work terribly well. Notice also that this will write out the encoded version of a chunked request (e.g. "6\r\na body\r\n0\r\n\r\n"), which may or may not be what you want.