Develop Reference

given an infinite sequence break it into intervals, and return a new infinite sequence with the average of each interval

i have to calculate the average of a Infinite Sequence using Stream API
Input:
Stream<Double> s = a,b,c,d ...
int interval = 3
Expected Result:
Stream<Double> result = avg(a,b,c), avg(d,e,f), ....
the result can be also an Iterator, or any other type
as long as it mantains the structure of an infinite list
of course what i written is pseudo code and doesnt run

There is a #Beta API termed mapWithIndex within Guava that could help here with certain assumption:
static Stream<Double> stepAverage(Stream<Double> stream, int step) {
return Streams.mapWithIndex(stream, (from, index) -> Map.entry(index, from))
.collect(Collectors.groupingBy(e -> (e.getKey() / step), TreeMap::new,
Collectors.averagingDouble(Map.Entry::getValue)))
.values().stream();
}
The assumption that it brings in is detailed in the documentation clearly(emphasized by me):
The resulting stream is efficiently splittable if and only if stream
was efficiently splittable and its underlying spliterator reported
Spliterator.SUBSIZED. This is generally the case if the underlying
stream comes from a data structure supporting efficient indexed random
access, typically an array or list.

This should work fine using vanilla Java
I'm using Stream#mapMulti and a Set external to the Stream to aggregate the doubles
As you see, I also used DoubleSummaryStatistics to count the average.
I could have use the traditional looping and summing then dividing but I found this way more explicit
Update:
I changed the Collection used from Set to List as a Set could cause unexpected behaviour
int step = 3;
List<Double> list = new ArrayList<>();
Stream<Double> averagesStream =
infiniteStream.mapMulti(((Double aDouble, Consumer<Double> doubleConsumer) -> {
list.add(aDouble);
if (list.size() == step) {
DoubleSummaryStatistics doubleSummaryStatistics = new DoubleSummaryStatistics();
list.forEach(doubleSummaryStatistics::accept);
list.clear();
doubleConsumer.accept(doubleSummaryStatistics.getAverage());
}
}));

Example of stream reduction with distinct combiner and accumulator

The question is about java.util.stream.Stream.reduce(U identity,BiFunction<U, ? super T, U> accumulator, BinaryOperator<U> combiner) method.
One of the requirements is that the combiner function must be compatible with the accumulator function; for all u and t, the following must hold:
combiner.apply(u, accumulator.apply(identity, t)) == accumulator.apply(u, t) (*)
If the combiner and accumulator are the same, the above equality is automatically true.
A BinaryOperator is actually extending BiFunction, therefore I can use it when BiFunction is required. If U and T are identical, the following is always legal:
operator<T> op = (x,y) -> something;
stream.reduce(id, op, op);
Of course, one cannot always use the combiner as acumulator since, in the general case, they serve for different purposes and are different Java types.
My question
Is there an example of stream reduction with distinct combiner and accumulator?
Also, I'm not interested in trivial examples, but natural examples that I can encounter in practice while doing reduction on parallel streams.
For trivial examples, there are many tutorials, like this one
Why am I asking this question
Basically, the reason this reduction method exists is for parallel streams. It seems to me the condition (*) is so strong that, in practice, it renders this reduction useless since rarely the reduction operations fulfill it.

If the combiner and accumulator are the same? You are confusing things here.
accumulator transforms from X to Y for example (using the identity), while combiner merges two Y into one. Also notice that one is a BiFunction and the other one is a BinaryOperator (which is actually a BiFunction<T, T, T>).
Is there an example of stream reduction with distinct combiner and accumulator?
These look pretty different to me:
Stream.of("1", "2")
.reduce(0, (x, y) -> x + y.length(), Integer::sum);
I think you might be confused with things like:
Stream.of("1", "2")
.reduce("", String::concat, String::concat);
How is it possible to do?
BiFunction<String, String, String> bi = String::concat;
Well there is a hint here.
EDIT
Addressing the part where "different" means different operations, accumulator might sum, while accumulator might multiply. This is exactly what the rule :
combiner.apply(u, accumulator.apply(identity, t)) == accumulator.apply(u, t)
is about, to protected itself from two separate associative functions, but different operations. Let's take an example of two lists (equal, but with different order). This, btw, would be a lot funner with a Set::of from java-9 that adds an internal randomization, so theoretically for the same exact input, you would get different result on the same VM from run to run. But to keep it simple:
List.of("a", "bb", "ccc", "dddd");
List.of("dddd", "a", "bb", "ccc");
And we want to perform:
....stream()
.parallel()
.reduce(0,
(x, y) -> x + y.length(),
(x, y) -> x * y);
Under the current implementation, this will yield the same result for both lists; but that is an implementation artifact.
There is nothing stopping an internal implementation in saying: "I will split the list to the smallest chunk possible, but not smaller than two elements in each of them". In such a case, this could have been translated to these splits:
["a", "bb"] ["ccc", "dddd"]
["dddd", "a" ] ["bb" , "ccc" ]
Now, "accumulate" those splits:
0 + "a".length = 1 ; 1 + "bb".length = 3 // thus chunk result is 3
0 + "ccc".length = 3 ; 3 + "dddd".length = 7 // thus chunk result is 7
Now we "combine" these chunks: 3 * 7 = 21.
I am pretty sure you already see that the second list in such a scenario would result in 25; as such different operations in the accumulator and combiner can result in wrong results.

So, here's are a few examples. Some of these may count as "trivial", particularly where there's already a function to do it for you.
An example where T and U are the same
These are quite difficult to come up with, and are a bit contrived, as they generally involve assuming that the elements of stream and the object being accumulated have different meanings, even though they have the same type.
Counting
If we have a stream of integers, we could count them using reduce:
stream.reduce(0, (count, item) -> count+1, (a, b) -> a+b);
Obviously, we could just use stream.count() here, but I'm willing to bet count uses the 3 argument version of reduce internally.
An example where T and U are different
This gives us quite a lot of freedom, and obviously, the accumulator and combiner are never going to be the same here, as they have different types.
One of the most common ways we may want to aggregate is gathering into a collection. We could use reduce for that, but since in Java collection types are typically mutable, using collect will generally be more efficient. This rule applies generally: if the result type mutable, use collect rather than reduce.
Determining the range of a stream of numbers
class Range {
static Range NONE = new Range(Double.NaN, Double.NaN);
final double min, max;
static Range of(double min, double max) {
if(Double.isNaN(min) || Double.isNaN(max) || min>max) {
throw new IllegalArgumentException();
}
return new Range(min, max);
}
private Range(double min, double max) {
this.min = min;
this.max = max;
}
boolean contains(double value) {
return this!=Range.NONE && min<=value && max>=value;
}
boolean spans(Range other) {
return this==other
|| other==Range.NONE
|| (contains(other.min) && contains(other.max));
}
}
Range range = streamOfDoubles.reduce(
Range.NONE,
(range, value) -> {
if(range==Range.NONE)
return Range.of(value, value);
else if(range.contains(value))
return range;
else
return Range.of(Math.min(value, range.min), Math.max(value, range.max));
},
(a, b) -> {
if(b.spans(a))
return b;
else if(a.spans(b))
return a;
else
return Range.of(Math.min(a.min, b.min), Math.max(a.max, b.max));
}
);

Optimising multiple streams to single loop

I am trying to find the best way to optimise the converters below to follow the flow I call 'convertAndGroupForUpdate' first which triggers the conversions and relevant mappings.
Any help to optimise this code would be massively appreciated.
public List<GroupedOrderActionUpdateEntity> convertAndGroupForUpdate(List<SimpleRatifiableAction> actions) {
List<GroupedOrderActionUpdateEntity> groupedActions = new ArrayList<>();
Map<String, List<SimpleRatifiableAction>> groupSimple = actions.stream()
.collect(Collectors.groupingBy(x -> x.getOrderNumber() + x.getActionType()));
groupSimple.entrySet().stream()
.map(x -> convertToUpdateGroup(x.getValue()))
.forEachOrdered(groupedActions::add);
return groupedActions;
}
public GroupedOrderActionUpdateEntity convertToUpdateGroup(List<SimpleRatifiableAction> actions) {
List<OrderActionUpdateEntity> actionList = actions.stream().map(x -> convertToUpdateEntity(x)).collect(Collectors.toList());
return new GroupedOrderActionUpdateEntity(
actions.get(0).getOrderNumber(),
OrderActionType.valueOf(actions.get(0).getActionType()),
actions.get(0).getSource(),
12345,
actions.stream().map(SimpleRatifiableAction::getNote)
.collect(Collectors.joining(", ", "Group Order Note: ", ".")),
actionList);
}
public OrderActionUpdateEntity convertToUpdateEntity(SimpleRatifiableAction action) {
return new OrderActionUpdateEntity(action.getId(), OrderActionState.valueOf(action.getState()));
}

You can’t elide a grouping operation, but you don’t need to store the intermediate result in a local variable.
Further, you should not add to a list manually, when you can collect to a List. Just do it like you did in the other method.
Also, creating a grouping key via string concatenation is tempting, but very dangerous, depending on the contents of the properties, the resulting strings may clash. And string concatenation is rather expensive. Just create a list of the property values, as long as you don’t modify it, it provides the right equality semantics and hash code implementation.
If you want to process the values of a map only, don’t call entrySet(), to map each entry via getValue(). Just use values() in the first place.
public List<GroupedOrderActionUpdateEntity> convertAndGroupForUpdate(
List<SimpleRatifiableAction> actions) {
return actions.stream()
.collect(Collectors.groupingBy( // use List.of(…, …) in Java 9 or newer
x -> Arrays.asList(x.getOrderNumber(), x.getActionType())))
.values().stream()
.map(x -> convertToUpdateGroup(x))
.collect(Collectors.toList());
}
Since convertToUpdateGroup is processing the list of actions of each group multiple times, there is not much that can be simplified and I wouldn’t inline it either. If there was only one operation, e.g. joining them to a string, you could do that right in the groupingBy operation, but there is no simply way to collect to multiple results.

Passing a collection using a reduce (3 parameters) function - streams java 8

I am trying to calculate the multiplication of a value using the previous two values using java 8's stream. I want to call a function that will return an array/list/collection. I am creating a List and adding 1,2 to it.
Let's say the list name is result.
public static void main (String[] args) {
List<Integer> result = new ArrayList<Integer>();
result.add(1);
result.add(2);
int n = 5; //n can be anything, choosing 5 for this example
res(n, result);
//print result which should be [1, 2, 2, 4, 8]
}
public static List<Integer> res(int n, List<Integer> result ) {
result.stream()
.limit(n)
.reduce(identity, (base,index) -> base);
//return result;
}
Now the issue is trying to try to pass result into the stream to keep updating the list with the new values using the stream. According to the java tutorials, it is possible, albeit inefficient.
"If your reduce operation involves adding elements to a collection, then every time your accumulator function processes an element, it creates a new collection that includes the element, which is inefficient."
Do I need to use the optional third parameter, BinaryOperator combiner, to combine the list + result??
<U> U reduce(U identity,
BiFunction<U,? super T,U> accumulator,
BinaryOperator<U> combiner)
In short; I want to pass a list with two values and have the function find the multiplication of the first two values (1,2), add it to the list, and find the multiplication of the last two values (2,2), and add it to the list, and until the stream hits the limit.

It looks like you're trying to implement a recurrence relation. The reduce method applies some function to a bunch of pre-existing values in the stream. You can't use reduce and take an intermediate result from the reducer function and "feed it back" into the stream, which is what you need to do in order to implement a recurrence relation.
The way to implement a recurrence relation using streams is to use one of the streams factory methods Stream.generate or Stream.iterate. The iterate factory seems to suggest the most obvious approach. The state that needs to be kept for each application of the recurrence function requires two ints in your example, so unfortunately we have to create an object to hold these for us:
static class IntPair {
final int a, b;
IntPair(int a_, int b_) {
a = a_; b = b_;
}
}
Using this state object you can create a stream that implements the recurrence that you want:
Stream.iterate(new IntPair(1, 2), p -> new IntPair(p.b, p.a * p.b))
Once you have such a stream, it's a simple matter to collect the values into a list:
List<Integer> output =
Stream.iterate(new IntPair(1, 2), p -> new IntPair(p.b, p.a * p.b))
.limit(5)
.map(pair -> pair.a)
.collect(Collectors.toList());
System.out.println(output);
[1, 2, 2, 4, 8]
As an aside, you can use the same technique to generate the Fibonacci sequence. All you do is provide a different starting value and iteration function:
Stream.iterate(new IntPair(0, 1), p -> new IntPair(p.b, p.a + p.b))
You could also implement a similar recurrence relation using Stream.generate. This will also require a helper class. The helper class implements Supplier of the result value but it also needs to maintain state. It thus needs to be mutable, which is kind of gross in my book. The iteration function also needs to be baked into the generator object. This makes it less flexible than the IntPair object, which can be used for creating arbitrary recurrences.

Just for completeness, here is a solution which does not need an additional class.
List<Integer> output = Stream.iterate(
(ToIntFunction<IntBinaryOperator>)f -> f.applyAsInt(1, 2),
prev -> f -> prev.applyAsInt((a, b) -> f.applyAsInt(b, a*b) )
)
.limit(9).map(pair -> pair.applyAsInt((a, b)->a))
.collect(Collectors.toList());
This is a functional approach which doesn’t need an intermediate value storage. However, since Java is not a functional programming language and doesn’t have optimizations for such a recursive function definition, this is not recommended for larger streams.
Since for this example a larger stream would overflow numerically anyway and the calculation is cheap, this approach works. But for other use cases you will surely prefer a storage object when solving such a problem with plain Java (as in Stuart Marks’ answer)

Can you split a stream into two streams?

I have a data set represented by a Java 8 stream:
Stream<T> stream = ...;
I can see how to filter it to get a random subset - for example
Random r = new Random();
PrimitiveIterator.OfInt coin = r.ints(0, 2).iterator();
Stream<T> heads = stream.filter((x) -> (coin.nextInt() == 0));
I can also see how I could reduce this stream to get, for example, two lists representing two random halves of the data set, and then turn those back into streams.
But, is there a direct way to generate two streams from the initial one? Something like
(heads, tails) = stream.[some kind of split based on filter]
Thanks for any insight.

A collector can be used for this.
For two categories, use Collectors.partitioningBy() factory.
This will create a Map<Boolean, List>, and put items in one or the other list based on a Predicate.
Note: Since the stream needs to be consumed whole, this can't work on infinite streams. And because the stream is consumed anyway, this method simply puts them in Lists instead of making a new stream-with-memory. You can always stream those lists if you require streams as output.
Also, no need for the iterator, not even in the heads-only example you provided.
Binary splitting looks like this:
Random r = new Random();
Map<Boolean, List<String>> groups = stream
.collect(Collectors.partitioningBy(x -> r.nextBoolean()));
System.out.println(groups.get(false).size());
System.out.println(groups.get(true).size());
For more categories, use a Collectors.groupingBy() factory.
Map<Object, List<String>> groups = stream
.collect(Collectors.groupingBy(x -> r.nextInt(3)));
System.out.println(groups.get(0).size());
System.out.println(groups.get(1).size());
System.out.println(groups.get(2).size());
In case the streams are not Stream, but one of the primitive streams like IntStream, then this .collect(Collectors) method is not available. You'll have to do it the manual way without a collector factory. It's implementation looks like this:
[Example 2.0 since 2020-04-16]
IntStream intStream = IntStream.iterate(0, i -> i + 1).limit(100000).parallel();
IntPredicate predicate = ignored -> r.nextBoolean();
Map<Boolean, List<Integer>> groups = intStream.collect(
() -> Map.of(false, new ArrayList<>(100000),
true , new ArrayList<>(100000)),
(map, value) -> map.get(predicate.test(value)).add(value),
(map1, map2) -> {
map1.get(false).addAll(map2.get(false));
map1.get(true ).addAll(map2.get(true ));
});
In this example I initialize the ArrayLists with the full size of the initial collection (if this is known at all). This prevents resize events even in the worst-case scenario, but can potentially gobble up 2NT space (N = initial number of elements, T = number of threads). To trade-off space for speed, you can leave it out or use your best educated guess, like the expected highest number of elements in one partition (typically just over N/2 for a balanced split).
I hope I don't offend anyone by using a Java 9 method. For the Java 8 version, look at the edit history.

I stumbled across this question to my self and I feel that a forked stream has some use cases that could prove valid. I wrote the code below as a consumer so that it does not do anything but you could apply it to functions and anything else you might come across.
class PredicateSplitterConsumer<T> implements Consumer<T>
{
private Predicate<T> predicate;
private Consumer<T> positiveConsumer;
private Consumer<T> negativeConsumer;
public PredicateSplitterConsumer(Predicate<T> predicate, Consumer<T> positive, Consumer<T> negative)
{
this.predicate = predicate;
this.positiveConsumer = positive;
this.negativeConsumer = negative;
}
#Override
public void accept(T t)
{
if (predicate.test(t))
{
positiveConsumer.accept(t);
}
else
{
negativeConsumer.accept(t);
}
}
}
Now your code implementation could be something like this:
personsArray.forEach(
new PredicateSplitterConsumer<>(
person -> person.getDateOfBirth().isPresent(),
person -> System.out.println(person.getName()),
person -> System.out.println(person.getName() + " does not have Date of birth")));

Unfortunately, what you ask for is directly frowned upon in the JavaDoc of Stream:
A stream should be operated on (invoking an intermediate or terminal
stream operation) only once. This rules out, for example, "forked"
streams, where the same source feeds two or more pipelines, or
multiple traversals of the same stream.
You can work around this using peek or other methods should you truly desire that type of behaviour. In this case, what you should do is instead of trying to back two streams from the same original Stream source with a forking filter, you would duplicate your stream and filter each of the duplicates appropriately.
However, you may wish to reconsider if a Stream is the appropriate structure for your use case.

You can get two Streams out of one
since Java 12 with teeing
counting heads and tails in 100 coin flips
Random r = new Random();
PrimitiveIterator.OfInt coin = r.ints(0, 2).iterator();
List<Long> list = Stream.iterate(0, i -> coin.nextInt())
.limit(100).collect(teeing(
filtering(i -> i == 1, counting()),
filtering(i -> i == 0, counting()),
(heads, tails) -> {
return(List.of(heads, tails));
}));
System.err.println("heads:" + list.get(0) + " tails:" + list.get(1));
gets eg.: heads:51 tails:49

Not exactly. You can't get two Streams out of one; this doesn't make sense -- how would you iterate over one without needing to generate the other at the same time? A stream can only be operated over once.
However, if you want to dump them into a list or something, you could do
stream.forEach((x) -> ((x == 0) ? heads : tails).add(x));

This is against the general mechanism of Stream. Say you can split Stream S0 to Sa and Sb like you wanted. Performing any terminal operation, say count(), on Sa will necessarily "consume" all elements in S0. Therefore Sb lost its data source.
Previously, Stream had a tee() method, I think, which duplicate a stream to two. It's removed now.
Stream has a peek() method though, you might be able to use it to achieve your requirements.

not exactly, but you may be able to accomplish what you need by invoking Collectors.groupingBy(). you create a new Collection, and can then instantiate streams on that new collection.

This was the least bad answer I could come up with.
import org.apache.commons.lang3.tuple.ImmutablePair;
import org.apache.commons.lang3.tuple.Pair;
public class Test {
public static <T, L, R> Pair<L, R> splitStream(Stream<T> inputStream, Predicate<T> predicate,
Function<Stream<T>, L> trueStreamProcessor, Function<Stream<T>, R> falseStreamProcessor) {
Map<Boolean, List<T>> partitioned = inputStream.collect(Collectors.partitioningBy(predicate));
L trueResult = trueStreamProcessor.apply(partitioned.get(Boolean.TRUE).stream());
R falseResult = falseStreamProcessor.apply(partitioned.get(Boolean.FALSE).stream());
return new ImmutablePair<L, R>(trueResult, falseResult);
}
public static void main(String[] args) {
Stream<Integer> stream = Stream.iterate(0, n -> n + 1).limit(10);
Pair<List<Integer>, String> results = splitStream(stream,
n -> n > 5,
s -> s.filter(n -> n % 2 == 0).collect(Collectors.toList()),
s -> s.map(n -> n.toString()).collect(Collectors.joining("|")));
System.out.println(results);
}
}
This takes a stream of integers and splits them at 5. For those greater than 5 it filters only even numbers and puts them in a list. For the rest it joins them with |.
outputs:
([6, 8],0|1|2|3|4|5)
Its not ideal as it collects everything into intermediary collections breaking the stream (and has too many arguments!)

I stumbled across this question while looking for a way to filter certain elements out of a stream and log them as errors. So I did not really need to split the stream so much as attach a premature terminating action to a predicate with unobtrusive syntax. This is what I came up with:
public class MyProcess {
/* Return a Predicate that performs a bail-out action on non-matching items. */
private static <T> Predicate<T> withAltAction(Predicate<T> pred, Consumer<T> altAction) {
return x -> {
if (pred.test(x)) {
return true;
}
altAction.accept(x);
return false;
};
/* Example usage in non-trivial pipeline */
public void processItems(Stream<Item> stream) {
stream.filter(Objects::nonNull)
.peek(this::logItem)
.map(Item::getSubItems)
.filter(withAltAction(SubItem::isValid,
i -> logError(i, "Invalid")))
.peek(this::logSubItem)
.filter(withAltAction(i -> i.size() > 10,
i -> logError(i, "Too large")))
.map(SubItem::toDisplayItem)
.forEach(this::display);
}
}

Shorter version that uses Lombok
import java.util.function.Consumer;
import java.util.function.Predicate;
import lombok.RequiredArgsConstructor;
/**
* Forks a Stream using a Predicate into postive and negative outcomes.
*/
#RequiredArgsConstructor
#FieldDefaults(makeFinal = true, level = AccessLevel.PROTECTED)
public class StreamForkerUtil<T> implements Consumer<T> {
Predicate<T> predicate;
Consumer<T> positiveConsumer;
Consumer<T> negativeConsumer;
#Override
public void accept(T t) {
(predicate.test(t) ? positiveConsumer : negativeConsumer).accept(t);
}
}

How about:
Supplier<Stream<Integer>> randomIntsStreamSupplier =
() -> (new Random()).ints(0, 2).boxed();
Stream<Integer> tails =
randomIntsStreamSupplier.get().filter(x->x.equals(0));
Stream<Integer> heads =
randomIntsStreamSupplier.get().filter(x->x.equals(1));