Log all requests and responses for http-conduit

Question

I have written this ManagerSettings to log all requests and responses for my http-conduit application. (By the way, I am importing ClassyPrelude).

tracingManagerSettings :: ManagerSettings
tracingManagerSettings =
  tlsManagerSettings { managerModifyRequest = \req -> do
                         putStr "TRACE: "
                         print req
                         putStrLn ""
                         pure req
                     , managerModifyResponse = \r -> do
                         responseChunks <- brConsume $ responseBody r
                         let fullResponse = mconcat responseChunks
                         putStr "TRACE: RESPONSE: "
                         putStrLn $ decodeUtf8 fullResponse
                         pure $ r { responseBody = pure fullResponse }
                     }

However, it's not working - when I use it, the application is hanging and trying to consume all the RAM in the machine after printing the first request and first response, which suggests some kind of infinite loop.

Also, the request is printed twice.

I made a previous attempt that was similar, but didn't modify r. That failed because after I had already read the response completely, there was no more response data to read.

If I replace this with tlsManagerSettings, http-conduit works again.

My application is using libstackexchange, which I have modified to allow the ManagerSettings to be customised. I am using http-conduit version 2.2.4.

How can I diagnose the issue? How can I fix it?

Answer 1

managerModifyResponse doesn't work with a Response ByteString, it works with a Response BodyReader, where type BodyReader = IO ByteString along with the contract that if it produces a non-empty ByteString there is more input that can be read.

The problem you're running into is that pure fullResponse never returns an empty ByteString unless it always does. You need to provide a somewhat more complex IO action to capture the intended behavior. Maybe something along these lines (untested):

returnOnce :: Monoid a => a -> IO (IO a)
returnOnce x = do
    ref <- newIORef x
    pure $ readIORef ref <* writeIORef ref mempty

As for how to debug this? Not sure about generic methods. I was just suspicious that you probably needed a solution along these lines, and the docs for BodyReader confirmed it.