Develop Reference

Java stream, filter, and then do something with resulting collection

Goal
final List<T> listOfThings = ...;
listOfThings.stream()
.filter(...) // returns a Stream<T>
.then(filteredListOfThings -> {
// How do I get here so I can work on the newly filtered collection
// in a fluent way w/out collecting the result to a variable?
// For example, if I need to process the elements but don't
// care about them in their current form outside this chain.
});
Problem
In English, given a list of something, I'd like to stream the list, filter it, and then operate on the entire filtered result. I can accomplish this with optional but it's not clean IMO:
final List<T> listOfThings = ...;
Optional
.of(listOfThings.stream()
.filter(...) // returns a Stream<T>
.collect(Collectors.toList()))
.map(filteredListOfThings -> {
// I'm here, now, but would like to not have to wrap it in an Optional<T>
});
It'd be cool if there was a then or similar method on a Stream<T> which returns Stream<T> to allow for further chaining, which allows me to work with the entire set of results within the lambda without declaring an outside variable.

Don't make it more complicated than it needs to be.
Assign the result of the collect to a variable, then operate on that variable:
List<T> filteredListOfThings = ... .collect(toList());
// Now use filteredListOfThings.
filteredListOfThings will always have a value, even if it's the empty list, so there's no point in using Optional.
And there's not much syntactic difference between filteredListOfThings being a lambda parameter and it being an explicit variable; but you have more flexibility in what you can do whilst processing it (returning from the methods, throwing checked exceptions etc).

I'd like to stream the list, filter it, and then operate on the entire filtered result.
Note that the stream can be infinite as well ;)
So getting the infinite list of results is not a good idea.
Basically streams are lazy and applying an intermediate operations to stream without having a terminal operation does nothing:
For example the following code prints nothing:
Stream<String> stream = Stream.of("hello","how","are", "you").filter(this::startsWithH)
private boolean startsWithH(String elem) {
System.out.println("Filtering element " + elem);
return elem.startsWith("h");
}
Now, when you do apply a terminal operation, it will still work element-by-element usually:
Example of execution:
Stream<String> stream = Stream.of("hello","how","are", "you")
.filter(this::startsWithH)
.map(String::toUpperCase)
stream.collect(toList());
This example yields the following execution chain:
filter("hello")
map("hello")
filter("how")
map("how")
filter("are") <--- filtered out, no call to map will be done
filter("you") <--- filtered out, no call to map will be done
But if so, you can't really operate on the "whole" stream in the example provided in this question (ok, there are stateful operations that must work on the whole stream, like sort, but its an entirely different story).
In other words if you want to get the data as a collection you should, well, collect the data. It won't be a stream anymore.
For this, you should use .collect(). And if you do have an infinite stream, don't forget to call limit beforehands ;)

Terminal operation to evaluate intermediate operation

Let says i have a list of strings and i want to use those strings as input to a fluent builder.
List<String> scripts;
//initialize list
ScriptRunnerBuilder scriptRunnerBuilder = new ScriptRunnerBuilder();
BiFunction<String,ScriptRunnerBuilder,ScriptRunnerBuilder> addScript =
(script,builder) -> builer.addScript(script);
scriptRunnerBuilder = scripts.stream.map(script ->
addScript.apply(script,scriptRunnerBuilder)).......
scriptRunnerBuilder.build();
which terminal operation can i use so that the addScript function gets called for all elements in the list?
The issue is that the ScriptRunnerBuilder is immutable whereby ScriptRunnerBuilder.addScript() returns a new ScriptRunnerBuilder object rather than modifying existing – so i can't just us a foreach.
My intentions are to carry the result of the addScript() call and use that as input for the next element in the stream

In simplest way this should:
// create your builder
ScriptRunnerBuilder builder = new ScriptRunnerBuilder();
// add all scripts
scripts.forEach(script-> builder.addScript(script))
build results
scriptRunnerBuilder.build();
Because builder aggregates all data, and you have created it outside forEach lambda, you can access it directly. This will lead to less code and same result.
Or as #Holger suggested:
scripts.forEach(builder::addScript);

Use forEach instead of map and don't assign the result of the stream anymore
scripts.forEach(script -> addScript.apply(script,scriptRunnerBuilder));

i could use reduce operation but that is unnecessary as we are not combining results
Combining is exactly what you are doing.
You combine all scripts from List<String> to ScriptRunnerBuilder aren't you?
I agree that the #Beri's solution without stream probably is the simplest. But also there is a way with reduce(identity, accumulator, combiner) method where you don't need to create ScriptRunnerBuilder before:
ScriptRunnerBuilder builder = scripts.stream()
.reduce(new ScriptRunnerBuilder(), ScriptRunnerBuilder::addScript, (b1, b2) -> b1);
See more: Why is a combiner needed for reduce method that converts type in java 8
Update To not to rely on the fact that combiner not being invoked for sequential stream and to make it works with parallel one you have to implement the real combiner.
If you could add an overrided method addScript(ScriptRunnerBuilder otherBuilder) then the reduce will look like:
.reduce(new ScriptRunnerBuilder(), ScriptRunnerBuilder::addScript,
ScriptRunnerBuilder::addScript)

Considering list elements that are added after filtered stream creation

Given the following code:
List<String> strList = new ArrayList<>(Arrays.asList("Java","Python","Php"));
Stream<String> jFilter = strList.stream().filter(str -> str.startsWith("J"));
strList.add("JavaScript"); // element added after filter creation
strList.add("JQuery"); // element added after filter creation
System.out.println(Arrays.toString(jFilter.toArray()));
which outputs:
[Java, JavaScript, JQuery]
Why do JavaScript and JQuery appear in the filtered result even though they were added after creating the filtered stream?

Short Answer
You're assuming after this point:
Stream<String> jFilter = strStream.filter(str -> str.startsWith("J"));
That a new stream of the elements starting with "J" are returned i.e. only Java. However this is not the case;
streams are lazy i.e. they don't perform any logic unless told otherwise by a terminal operation.
The actual execution of the stream pipeline starts on the toArray() call and since the list was modified before the terminal toArray() operation commenced the result will be [Java, JavaScript, JQuery].
Longer Answer
here's part of the documentation which mentions this:
For well-behaved stream sources, the source can be modified before
the terminal operation commences and those modifications will be
reflected in the covered elements. For example, consider the following
code:
List<String> l = new ArrayList(Arrays.asList("one", "two"));
Stream<String> sl = l.stream();
l.add("three");
String s = sl.collect(joining(" "));
First a list is created consisting of two strings: "one"; and "two". Then a stream is created
from that list. Next the list is modified by adding a third string:
"three". Finally the elements of the stream are collected and joined
together. Since the list was modified before the terminal collect
operation commenced the result will be a string of "one two three".
All the streams returned from JDK collections, and most other JDK
classes, are well-behaved in this manner;

Until the statement
System.out.println(Arrays.toString(jFilter.toArray()));
runs, the stream doesn't do anything. A terminal operation (toArray in the example) is required for the stream to be traversed and your intermediate operations (filter in this case) to be executed.
In this case, what you can do is, for example, capture the size of the list before adding other elements:
int maxSize = strList.size();
Stream<String> jFilter = strStream.limit(maxSize)
.filter(str -> str.startsWith("J"));
where limit(maxSize) will not allow more than the initial elements to go through the pipeline.

Its because the stream never got evaluated. you never called a "Terminal operation" on that stream for it to get executed as they're lazy.
Look at a modification of your code and the output. The filtering actually takes place when you call the Terminal Operator.
public static void main(String []args){
List<String> strList = new ArrayList<>();
strList.add("Java");
strList.add("Python");
strList.add("Php");
Stream<String> strStream = strList.stream();
Stream<String> jFilter = strStream.filter(str -> {
System.out.println("Filtering" + str);
return str.startsWith("J");
});
System.out.println("After Stream creation");
strList.add("JavaScript"); // element added after filter creation
strList.add("JQuery"); // element added after filter creation
System.out.println(Arrays.toString(jFilter.toArray()));
}
Output:
After Stream creation
FilteringJava
FilteringPython
FilteringPhp
FilteringJavaScript
FilteringJQuery
[Java, JavaScript, JQuery]

As explained in the official documentation at ,https://docs.oracle.com/javase/8/docs/api/java/util/stream/package-summary.html, streams have no storage, and so are more like iterators than collections, and are evaluated lazily.
So, nothing really happens with respect to the stream until you invoke the terminal operation toArray()

#Hadi J's comment but it should be answer according to the rules.
Because streams are lazy and when you call terminal operation it executed.

The toArray method is the terminal operation and it works on that full content of your list. To get predictable result do not save the stream to a temporary variable as it will lead to misleading results. A better code is:
String[] arr = strList.stream().filter(str -> str.startsWith("J")).toArray();

Flattening streams from a singleton Optional

This question is related to the Java 8 map and flatMap that is present both in Streams and Optionals. It is worth to note that C# has a similar construct named SelectMany.
I have learned about the two methods, in particular that in Streams you can use flatMap to get a Collection<T> from a Collection<Collection<T>> which is what I want.
In my example I have a nested class structure (from a DTD I have no control over) in which I want to compute a sum of values. I will not redact the class names for laziness.
class DatiTelematico {
private Adempimento adempimento;
}
class Adempimento {
private List<DatiNegozio> datiNegozio;
}
class DatiNegozio {
private List<Negozio> negozio;
}
class Negozio {
private List<Tassazione> tassazione;
}
class Tassazione {
private BigDecimal importo;
}
Given an Optional instance of a DatiTelematico class I would like to sum (importo) from telematico join adempimento join datiNegozio join negozio join tassazione.
The best I could do was to use nested lambdas and plain map method
optionalTelematico.map(DatiTelematico::getAdempimento)
.map(Adempimento::getDatiNegozio)
.map(l -> l.stream().map(DatiNegozio::getNegozio)
.map(n -> n.stream()
.map(Negozio::getTassazione)
.map(t -> t.stream()
.map(Tassazione::getImporto)
.reduce(BigDecimal.ZERO,
BigDecimal::add))
.reduce(BigDecimal.ZERO, BigDecimal::add))
.reduce(BigDecimal.ZERO, BigDecimal::add))
.orElse(BigDecimal.ZERO));
I have tried to start writing something like
optionalTelematico.map(DatiTelematico::getAdempimento)
.map(Adempimento::getDatiNegozio)
.map(l->l.stream().flatMap(n->n.getNegozio().stream().flatMap(s->s.getTassazione().stream().flatMap(Tassazione::getImporto)))....TBD
But then I get a compiler error
Method binding must be directly contained in a class (OTJLD A.3.1).
How do I smartly switch from an Optional<T> (singleton) to a Collection<U> that is to be summed over?
I am asking this to increase my knowledge of Java lambdas.

Both Optional and Stream actually only represent a single piece of data - in case of optional, that piece of data might be either absent or present, in case of stream, there may be other pieces of data coming before or after, but in current moment we have this piece only.
Now,
map method is essentially a kind of type transformation for both Optional and Stream: mapping takes a function I -> R, applying which one can make transformation Optional<I> -> Optional<R> (or Stream<I> -> Stream<R>).
flatMap method is a kind of transformation that can:
Transform optional value into another optional (possibly empty). That means a function type I -> Optional<R>
Transform each item in stream into another stream (having 0..n number of elements in it). That means function type I -> Stream<R>. Note that for streams this operation can change the number of elements that are contained in stream (but it won't change the fact that there always effectively one stream element being processed at a time).
In your particular case, by making transformation to optional, you may obtain up to Optional<List<DatiNegozio>> directly:
Optional<List<DatiNegozio>> optDatiNegozio = optionalDatiTelematico
.map(DatiTelematico::getAdempimento) // Optional<Adempimento>
.map(Adempimento::getDatiNegozio);
Every List<DatiNegozio> you can easily convert to Optional<BigDecimal> summing and accessing elements via Stream:
static Optional<BigDecimal> sumImporto(List<DatiNegozio> datiNegozio) {
return datiNegozio.stream() // Stream<DatiNegozio>
.map(DatiNegozio::getNegozio) // Stream<List<Negozio>>
// unroll stream of collections into a stream of collection elements
.flatMap(List::stream) // Stream<Negozio>
.map(Negozio::getTassazione) // Stream<List<Tassazione>>
// again, unroll stream of collections into a stream of collection elements
.flatMap(List::stream)
.map(Tassazione::getImporto) // Stream<BigDecimal>
// last thing we need to do is just reduce
.reduce(BigDecimal::add);
}
As you can see, second snippet allows you to convert List<DatiNegozio> into an Optional<BigDecimal>. After this, you have two options (stylistic choice):
There is a variant of reduce that yields BigDecimal instead of Optional<BigDecimal>:
.reduce(BigDecimal.ZERO, BigDecimal::add); // it yields concrete type instead of optional because even in case when there is no elements in stream, we can at least return value from which we started - ZERO
You can use second code snippet to produce Function that is usable in flatMap-ing an optional:
optionalDatiTelematico
.map(DatiTelematico::getAdempimento)
.map(Adempimento::getDatiNegozio)
.flatMap(Example::sumImporto) // reference to method from 2nd code snippet
.orElse(BigDecimal.ZERO); // what to do if we had Optional.empty at any point

You can use Collection.stream() method to convert Collection to Stream and use it in flatMap. So combination of .map(d -> d.getList()).flatMap(Collection::stream) returns stream for all internal lists of Stream<D>.
In your case it can looks like:
Optional.of(datiTelematico)
.map(DatiTelematico::getAdempimento)
.map(Adempimento::getDatiNegozio)
.map(Collection::stream)
.orElseGet(Stream::empty)
.map(DatiNegozio::getNegozio)
.flatMap(Collection::stream)
.map(Negozio::getTassazione)
.flatMap(Collection::stream)
.map(Tassazione::getImporto)
.reduce(BigDecimal.ZERO, BigDecimal::add);

Let say
Optional<DatiNegozio> abc = optionalTelematico.map(DatiTelematico::getAdempimento)
.map(Adempimento::getDatiNegozio)
Now when you say abc.map(xyz). xyz must be function which takes instance of DatiNegozio as one and only argument. In your case xyz is a lambda which takes one parameter l whose type should be DatiNegozio. You are now doing l.stream() which throws compile error beacsue stream() does not exist in l(instance ofDatiNegozio).

Combine stream of Collections into one Collection - Java 8

So I have a Stream<Collection<Long>> that I obtain by doing a series of transformations on another stream.
What I need to do is collect the Stream<Collection<Long>> into one Collection<Long>.
I could collect them all into a list like this:
<Stream<Collection<Long>> streamOfCollections = /* get the stream */;
List<Collection<Long>> listOfCollections = streamOfCollections.collect(Collectors.toList());
And then I could iterate through that list of collections to combine them into one.
However, I imagine there must be a simple way to combine the stream of collections into one Collection<Long> using a .map() or .collect(). I just can't think of how to do it. Any ideas?

This functionality can be achieved with a call to the flatMap method on the stream, which takes a Function that maps the Stream item to another Stream on which you can collect.
Here, the flatMap method converts the Stream<Collection<Long>> to a Stream<Long>, and collect collects them into a Collection<Long>.
Collection<Long> longs = streamOfCollections
.flatMap( coll -> coll.stream())
.collect(Collectors.toList());

You could do this by using collect and providing a supplier (the ArrayList::new part):
Collection<Long> longs = streamOfCollections.collect(
ArrayList::new,
ArrayList::addAll,
ArrayList::addAll
);

You don't need to specify classes when not needed.
A better solution is:
Collection<Long> longs = streamOfCollections.collect(
ArrayList::new,
Collection::addAll,
Collection::addAll
);
Say, you don't need an ArrayList but need a HashSet, then you also need to edit only one line.