Why StreamEx force me to add "? extends" to variable type when collecting to list?

Question

I noticed that in the case below standard Java stream better "calculate" variable type than StreamEx. This is odd, because people love StreamEx and use it everywhere, but then code become polluted with "?". I want to use List<Class<?> but StreamEx force me to use List<? extends Class<?>>. Could someone explain why StreamEx works this way? Can I somehow use the desired variable type with StreamEx ?

private static List<? extends Class<?>> 
fooList_StreamEx_Compiles(List<Integer> input) {
    return StreamEx.of(input)
            .map(x -> foo())
            .toList();
}

private static List<Class<?>> 
fooList_StreamEx_Error(List<Integer> input) {
    return StreamEx.of(input)
            .map(x -> foo())
// Error: incompatible types: java.util.List<java.lang.Class<capture#1 of ?>> 
// cannot be converted to java.util.List<java.lang.Class<?>>
            .toList();
}

private static List<Class<?>> fooList(List<Integer> input) {
    return input
            .stream()
            .map(x -> foo())
            .collect(Collectors.toList());
}

private static Class<?> foo() {
    return String.class;
}

I'm using StreamEx 0.7.0 and Java 11

Answer 1

This is not a StreamEx issue. When you use collect(Collectors.toList()) on the StreamEx, it works equally well. The problem is connected with the toList() convenience method, which the standard Stream doesn’t even offer. It’s a general problem that StreamEx is not to blame for.

The toList method on StreamEx has the following signature:

public List<T> toList()

In a perfect world, it would be legal to create a list parameterized with a super-type, i.e.

public <R super T> List<R> toList()

but this syntax had been omitted when Java’s generics were created. The toArray methods of both, Collection and Stream suffer from a similar limitation; they can not declare the resulting array’s element type to be a super type of the collection’s element type. But since an actual array’s store operation is checked, those methods simply allow any element type. For a List result, subject to type erasure, this is not possible.

When using collect(Collectors.toList()) on the other hand, it’s possible to create a list with a super type due to collect’s signature:

<R,​A> R collect​(Collector<? super T,​A,​R> collector)

which allows to pass in a Collector parameterized with a super type (? super T) that in most scenarios will be inferred from the target type.

---

This limitation of the toList declaration interacts with another limitation, the unreasonable handling of wildcard types. I’m not sure whether the cause of this problem lies in the compiler or specification, but at the map(x -> foo()) step, the wildcard type got captured and such a captured type will be considered different to any other captured wildcard type, even when it stems from the same source.

When I compile your code with javac, it says:

error: incompatible types: List<Class<CAP#1>> cannot be converted to List<Class<?>>
                    .toList();
                           ^
  where CAP#1 is a fresh type-variable:
    CAP#1 extends Object from capture of ?
1 error

The CAP#1 is a captured type. All captured types get numbered, to distinguish them. As said, each of them is considered a distinct type, different from every other.

Class<?> is a supertype of Class<CAP#1>, so it works with collect which allows to collect to a list parameterized with that supertype, as said above, but not with toList.

You can fix this by using the return type List<? extends Class<?>>, to denote that the list’s actual element type is a subtype of Class<?>, but the better option is to force the compiler not to use a captured type:

private static List<Class<?>> fooList_StreamEx_Solved(List<Integer> input) {
    return StreamEx.of(input)
            .<Class<?>>map(x -> foo())
            .toList();
}

Don’t worry if you didn’t fully understand the necessity to insert the explicit type here, I’m not claiming that the behavior of Java compilers was comprehensible at all when wildcard types are involved…