"List like" conduit newtype

Question

We've got a bunch of code currently using lists which probably should be using conduit.

But this occurred to me:

newtype ListyConduit r m i o = ListyConduit (ConduitT i o m r)

instance Functor (ListyConduit r m i) where
  ...

instance Foldable (ListyConduit r m i) where
  ...

instance Traversable (ListyConduit r m i) where
  ...

instance Profunctor (ListConduit r m) where
  ...

This approach would simply the conversion as in most cases I'll just have to generalise [a] to f a and then I can just put conduits in with the existing logic.

And it seems to me that each of these instances should be writable (at least I think) in a way that preserves the constant space usage of conduits.

Is this a reasonable assumption, and if so, has someone already done what I'm proposing above? And if it hasn't been done, is it because my suggestion is quite silly in some way or has no-one just got around to it?

Answer 1

There are quite a few problems here:

• Traversable instances basically "can't" be constant space. The fundamental operation is "do all the effects and then give me all the values." While the effects are being done, the values collected so far must be buffered; Traversable is antithetical to streaming.
  • Conversely, the point of conduit is, to some extent, interleaving values and effects. If/where this is necessary for you, you really can't expect to reuse code that uses lists, since it is not built to handle effects in the middle of a stream.
• Conduits don't seem (to me) to be Traversable in their o slot anyway, for the above reason.
• fmap on lists already streams, with O(1) extra space usage.
• Strict left folds on lists already stream input and only use as much extra space as needed for the accumulator.
• Right folds producing lists from lists have similar streaming properties to fmap.
• Because a conduit produces its o values under a monad, they can't be Foldable since you'd really need fold :: (Monad m, Monoid m) => Conduit () o m r -> m o. Again, the problem is that a Conduit has effects everywhere but a [] doesn't (unless m = Identity).

Basically: list code that can generalize to Conduit is pure and should already be streaming, if that is possible; there's no point in using Conduit here. When list code doesn't stream, the main culprit is effects (i.e. a traverse), and making it stream by using Conduit will not be "free". Such code has already traded away the capacity to stream effects for simplicity.

Answer 2

The closest thing that reminds me of is ListT in pipes. Unlike what you're proposing, it doesn't expose the input type to make a Profunctor, and it doesn't have Foldable/Traversable instances.