I am trying to understand if there is a way to terminate reduction operation without examining the whole stream and I cannot figure out a way.
The use-case is roughly as follows: let there be a long list of Integer
s which needs to be folded into an Accumulator
. Each element examination is potentially expensive, so within the Accumulator
, I perform a check on the incoming Accumulator
to see if we even need to perform expensive operation - if we don't, then I simply return the accumulator.
This is obviously a fine solution for small(er) lists but huge lists incur unnecessary stream element visiting costs I'd like to avoid.
Here's a code sketch - assume serial reductions only.
class Accumulator {
private final Set<A> setA = new HashSet<>;
private final Set<B> setB = new HashSet<>;
}
class ResultSupplier implements Supplier<Result> {
private final List<Integer> ids;
@Override
public Result get() {
Accumulator acc = ids.stream().reduce(new Accumulator(), f(), (x, y) -> null);
return (acc.setA.size > 1) ? Result.invalid() : Result.valid(acc.setB);
}
private static BiFunction<Accumulator, Integer, Accumulator> f() {
return (acc, element) -> {
if (acc.setA.size() <= 1) {
// perform expensive ops and accumulate results
}
return acc;
};
}
}
In addition to having to traverse the whole Stream
, there is another fact I dislike - I have to check the same condition twice (namely, setA
size check).
I have considered map()
and collect()
operations but they just seemed like more of the same and didn't find they materially change the fact that I just can't finish the fold operation without examining the entire stream.
Additionally, my thinking is that imaginary takeWhile(p : (A) => boolean)
Stream API correspondent would also buy us nothing, as the terminating condition depends on the accumulator, not stream elements per se.
Bear in mind I am a relative newcomer to FP so - is there a way to make this work as I expect it? Have I set up the whole problem improperly or is this limitation by design?
Instead of starting with
ids.stream()
you canids.spliterator()
tryAdvance
return false if the flag is changedStreamSupport.stream(Spliterator<T>, boolean)
Add some static helper methods to keep it functional.
the resulting API could look about this
It does work if the condition is dependent on the accumulator state and not on the stream members. That's essentially the approach i've outlined above.
It probably would be forbidden in a
takeWhile
provided by the JDK but a custom implementation using spliterators is free to take a stateful approach.