Continuations and for comprehensions -- what's the incompatibility?

Question

I am new to Scala and trying to wrap my head around continuations I'm trying to reproduce the yield return C# statement.

Following this post, I have written the following code :

package com.company.scalatest

import scala.util.continuations._;

object GenTest {

  val gen = new Generator[Int] {
    def produce = {
      yieldValue(1)
      yieldValue(2)
      yieldValue(3)
      yieldValue(42)
    }
  }
  // Does not compile :(

  //  val gen2 = new Generator[Int] {
  //    def produce = {
  //      var ints = List(1, 2, 3, 42);
  //
  //      ints.foreach((theInt) => yieldValue(theInt));
  //    }
  //  }

  // But this works?
  val gen3 = new Generator[Int] {
    def produce = {
      var ints = List(1, 2, 3, 42);
      var i = 0;
      while (i < ints.length) {
        yieldValue(ints(i));
        i = i + 1;
      }
    }
  }

  def main(args: Array[String]): Unit = {
    gen.foreach(println);
    //    gen2.foreach(println);
    gen3.foreach(println);
  }
}

abstract class Generator[E] {

  var loopFn: (E => Unit) = null

  def produce(): Unit @cps[Unit]

  def foreach(f: => (E => Unit)): Unit = {
    loopFn = f
    reset[Unit, Unit](produce)
  }

  def yieldValue(value: E) =
    shift { genK: (Unit => Unit) =>
      loopFn(value)
      genK(())
      ()
    }
}

As you can see, gen2 is commented out as it does not compile. Since I can easily iterate over the content of a list using a while loop (see gen3), I expected the foreach loop to work just as well.

The compilation error is the following :

no type parameters for method foreach: (f: Int => B)Unit exist so that 
it can be applied to arguments (Int => Unit @scala.util.continuations.cpsParam[Unit,Unit])  
 --- because --- 
argument expression's type is not compatible with formal parameter type;  
found   : Int => Unit @scala.util.continuations.cpsParam[Unit,Unit]  
required: Int => ?B 

Why do I get this error and is there a way to work around this with something cleaner than a while loop?

Thank you

Answer 1

First let's look at what it will take to get gen2 to compile.

object CpsConversions {

  import scala.collection.IterableLike
  import scala.util.continuations._

  implicit def cpsIterable[A, Repr](xs: IterableLike[A, Repr]) = new {
    def cps = new {
      def foreach[B](f: A => Any@cpsParam[Unit, Unit]): Unit@cpsParam[Unit, Unit] = {
        val it = xs.iterator
        while(it.hasNext) f(it.next)
      }
    }
  }
}

object GenTest {

  import CpsConversions.cpsIterable
  val gen2 = new Generator[Int] {
    def produce = {
      var ints = List(1, 2, 3, 42)
      ints.cps.foreach((theInt) => yieldValue(theInt))
    }
  }

Now let's take a look at what's going on. The original gen2 fails to compile on the following line:

ints.foreach((theInt) => yieldValue(theInt))

Since the type of yieldValue includes an @cpsParam annotation, the continuations plugin transforms the function passed to the foreach method to one of type:

Int => Unit @cpsParam[Unit,Unit]

Way up in the hierarchy of List[Int], you'll see foreach defined as:

foreach [U] (f: (Int) ⇒ U): Unit

This is a problem, as the types do not match and Scala doesn't know how to get from Int => U to Int => Unit @cpsParam[Unit,Unit]. To fix it, I added the CPS version of foreach in an implicit conversion, which you can access by calling cps on any IterableLike.

It would be very nice if this implicit conversion could be done without the explicit cps call, but I have not found a way to make the Scala compiler recognize the applicability of such an implicit conversion to pimp the new foreach onto your list. This might have to do with the order in which the compiler uses the continuations plugin, but I know far too little about this process to be sure.

So that's all well and good for foreach. Your question mentions for comprehensions, which will require any of filter, map, or flatMap to be defined (depending on what goes on in your for comprehension). I have implemented these in the link in my above comment, which extends the CpsConversions object above to allow for general for comprehensions.