Type inference inconsistency between toList and toBuffer

Question

As per the example below, calling xs.toList.map(_.toBuffer) succeeds, but xs.toBuffer.map(_.toBuffer) fails. But when the steps in the latter are performed using an intermediate result, it succeeds. What causes this inconsistency?

scala> "ab-cd".split("-").toBuffer
res0: scala.collection.mutable.Buffer[String] = ArrayBuffer(ab, cd)

scala> res0.map(_.toBuffer)
res1: scala.collection.mutable.Buffer[scala.collection.mutable.Buffer[Char]] = ArrayBuffer(ArrayBuffer(a, b), ArrayBuffer(c, d))

scala> "ab-cd".split("-").toBuffer.map(_.toBuffer)
<console>:8: error: missing parameter type for expanded function ((x$1) => x$1.toBuffer)
              "ab-cd".split("-").toBuffer.map(_.toBuffer)
                                              ^

scala> "ab-cd".split("-").toList.map(_.toBuffer)
res3: List[scala.collection.mutable.Buffer[Char]] = List(ArrayBuffer(a, b), ArrayBuffer(c, d))

Answer 1

Look at the definitions of toBuffer and toList:

def toBuffer[A1 >: A]: Buffer[A1]
def toList: List[A]

As you can see, toBuffer is generic, while toList is not. The reason for this difference is - I believe - that Buffer is invariant, while List is covariant.

Let's say that we have the following classes:

class Foo
class Bar extends Foo

Because List is covariant, you can call toList on an instance of Iterable[Bar] and treat the result as a List[Foo]. If List where invariant, this would not be the case. Buffer being invariant, if toBuffer was defined as def toBuffer: Buffer[A] you would similarly not be able to treat the result of toBuffer (on an instance of Iterable[Bar]) as an instance of Buffer[Foo] (as Buffer[Bar] is not a sub-type of Buffer[Foo], unlike for lists). But by declaring toBuffer as def toBuffer[A1 >: A] (notice the added type parameter A1), we get back the possibility to have toBuffer return an instance of Buffer[Foo] : all we need is to explcitly set A1 to Foo, or let the compiler infer it (if toBuffer is called at a site where a Buffer[Foo] is expected).

I think this explains the reason why toList and toBuffer are defined differently. Now the problem with this is that toBuffer is generic, and this can badly affect inference.

When you do this:

"ab-cd".split("-").toBuffer

You never explicitly say that A1 is String, but because "ab-cd".split("-") has unambiguously the type Array[String], the compiler knows that A is String. It also knows that A1 >: A (in toBuffer), and without any further constraint, it will infer A1 to be exactly A, in other words String. So in the end the whole expression returns a Buffer[String].

But here's the thing: in scala, type inference happens in an expression as a whole. When you have something like a.b.c, you might expect that scala will infer an exact type for a, then from that infer an exact type for a.b, and finally for a.b.c. Not so. Type inference is deferred to the whole expression a.b.c (see scala specification "6.26.4 Local Type Inference ", "case 1: selections")

So, going back to your problem, in the expression "ab-cd".split("-").toBuffer.map(_.toBuffer), the sub-expression "ab-cd".split("-").toBuffer is not typed Buffer[String], but instead it stays typed as something like Buffer[A1] forSome A1 >: String. In other words, A1 is not fixed, we just carry the constraint A1 >: String to the next step of inference. This next step is map(_.toBuffer), where map is defined as map[C](f: (B) ⇒ C): Buffer[B]. Here B is actually the same as A1, but at this point A1 is still not fully known, we only know that A1 >: String. Here lies our problem. The compiler needs to know the exact type of the anonymous function (_.toBuffer) (simply because instantiating a Function1[A,R] requires to know the exact types of A and R, just like for any generic type). So you need to tell him explcitly somehow, as it was not able to infer it exactly.

This means you need to do either:

"ab-cd".split("-").toBuffer[String].map(_.toBuffer)

Or:

"ab-cd".split("-").toBuffer.map((_:String).toBuffer)