This question is related to the Java 8 map and flatMap that is present both in Streams and Optionals. It is worth to note that C# has a similar construct named SelectMany.
I have learned about the two methods, in particular that in Streams you can use flatMap to get a Collection<T> from a Collection<Collection<T>> which is what I want.
In my example I have a nested class structure (from a DTD I have no control over) in which I want to compute a sum of values. I will not redact the class names for laziness.
class DatiTelematico {
private Adempimento adempimento;
}
class Adempimento {
private List<DatiNegozio> datiNegozio;
}
class DatiNegozio {
private List<Negozio> negozio;
}
class Negozio {
private List<Tassazione> tassazione;
}
class Tassazione {
private BigDecimal importo;
}
Given an Optional instance of a DatiTelematico class I would like to sum (importo) from telematico join adempimento join datiNegozio join negozio join tassazione.
The best I could do was to use nested lambdas and plain map method
optionalTelematico.map(DatiTelematico::getAdempimento)
.map(Adempimento::getDatiNegozio)
.map(l -> l.stream().map(DatiNegozio::getNegozio)
.map(n -> n.stream()
.map(Negozio::getTassazione)
.map(t -> t.stream()
.map(Tassazione::getImporto)
.reduce(BigDecimal.ZERO,
BigDecimal::add))
.reduce(BigDecimal.ZERO, BigDecimal::add))
.reduce(BigDecimal.ZERO, BigDecimal::add))
.orElse(BigDecimal.ZERO));
I have tried to start writing something like
optionalTelematico.map(DatiTelematico::getAdempimento)
.map(Adempimento::getDatiNegozio)
.map(l->l.stream().flatMap(n->n.getNegozio().stream().flatMap(s->s.getTassazione().stream().flatMap(Tassazione::getImporto)))....TBD
But then I get a compiler error
Method binding must be directly contained in a class (OTJLD A.3.1).
How do I smartly switch from an Optional<T> (singleton) to a Collection<U> that is to be summed over?
I am asking this to increase my knowledge of Java lambdas.
Both Optional and Stream actually only represent a single piece of data - in case of optional, that piece of data might be either absent or present, in case of stream, there may be other pieces of data coming before or after, but in current moment we have this piece only.
Now,
map method is essentially a kind of type transformation for both Optional and Stream: mapping takes a function I -> R, applying which one can make transformation Optional<I> -> Optional<R> (or Stream<I> -> Stream<R>).
flatMap method is a kind of transformation that can:
Transform optional value into another optional (possibly empty). That means a function type I -> Optional<R>
Transform each item in stream into another stream (having 0..n number of elements in it). That means function type I -> Stream<R>. Note that for streams this operation can change the number of elements that are contained in stream (but it won't change the fact that there always effectively one stream element being processed at a time).
In your particular case, by making transformation to optional, you may obtain up to Optional<List<DatiNegozio>> directly:
Optional<List<DatiNegozio>> optDatiNegozio = optionalDatiTelematico
.map(DatiTelematico::getAdempimento) // Optional<Adempimento>
.map(Adempimento::getDatiNegozio);
Every List<DatiNegozio> you can easily convert to Optional<BigDecimal> summing and accessing elements via Stream:
static Optional<BigDecimal> sumImporto(List<DatiNegozio> datiNegozio) {
return datiNegozio.stream() // Stream<DatiNegozio>
.map(DatiNegozio::getNegozio) // Stream<List<Negozio>>
// unroll stream of collections into a stream of collection elements
.flatMap(List::stream) // Stream<Negozio>
.map(Negozio::getTassazione) // Stream<List<Tassazione>>
// again, unroll stream of collections into a stream of collection elements
.flatMap(List::stream)
.map(Tassazione::getImporto) // Stream<BigDecimal>
// last thing we need to do is just reduce
.reduce(BigDecimal::add);
}
As you can see, second snippet allows you to convert List<DatiNegozio> into an Optional<BigDecimal>. After this, you have two options (stylistic choice):
There is a variant of reduce that yields BigDecimal instead of Optional<BigDecimal>:
.reduce(BigDecimal.ZERO, BigDecimal::add); // it yields concrete type instead of optional because even in case when there is no elements in stream, we can at least return value from which we started - ZERO
You can use second code snippet to produce Function that is usable in flatMap-ing an optional:
optionalDatiTelematico
.map(DatiTelematico::getAdempimento)
.map(Adempimento::getDatiNegozio)
.flatMap(Example::sumImporto) // reference to method from 2nd code snippet
.orElse(BigDecimal.ZERO); // what to do if we had Optional.empty at any point
You can use Collection.stream() method to convert Collection to Stream and use it in flatMap. So combination of .map(d -> d.getList()).flatMap(Collection::stream) returns stream for all internal lists of Stream<D>.
In your case it can looks like:
Optional.of(datiTelematico)
.map(DatiTelematico::getAdempimento)
.map(Adempimento::getDatiNegozio)
.map(Collection::stream)
.orElseGet(Stream::empty)
.map(DatiNegozio::getNegozio)
.flatMap(Collection::stream)
.map(Negozio::getTassazione)
.flatMap(Collection::stream)
.map(Tassazione::getImporto)
.reduce(BigDecimal.ZERO, BigDecimal::add);
Let say
Optional<DatiNegozio> abc = optionalTelematico.map(DatiTelematico::getAdempimento)
.map(Adempimento::getDatiNegozio)
Now when you say abc.map(xyz). xyz must be function which takes instance of DatiNegozio as one and only argument. In your case xyz is a lambda which takes one parameter l whose type should be DatiNegozio. You are now doing l.stream() which throws compile error beacsue stream() does not exist in l(instance ofDatiNegozio).
Related
Goal
final List<T> listOfThings = ...;
listOfThings.stream()
.filter(...) // returns a Stream<T>
.then(filteredListOfThings -> {
// How do I get here so I can work on the newly filtered collection
// in a fluent way w/out collecting the result to a variable?
// For example, if I need to process the elements but don't
// care about them in their current form outside this chain.
});
Problem
In English, given a list of something, I'd like to stream the list, filter it, and then operate on the entire filtered result. I can accomplish this with optional but it's not clean IMO:
final List<T> listOfThings = ...;
Optional
.of(listOfThings.stream()
.filter(...) // returns a Stream<T>
.collect(Collectors.toList()))
.map(filteredListOfThings -> {
// I'm here, now, but would like to not have to wrap it in an Optional<T>
});
It'd be cool if there was a then or similar method on a Stream<T> which returns Stream<T> to allow for further chaining, which allows me to work with the entire set of results within the lambda without declaring an outside variable.
Don't make it more complicated than it needs to be.
Assign the result of the collect to a variable, then operate on that variable:
List<T> filteredListOfThings = ... .collect(toList());
// Now use filteredListOfThings.
filteredListOfThings will always have a value, even if it's the empty list, so there's no point in using Optional.
And there's not much syntactic difference between filteredListOfThings being a lambda parameter and it being an explicit variable; but you have more flexibility in what you can do whilst processing it (returning from the methods, throwing checked exceptions etc).
I'd like to stream the list, filter it, and then operate on the entire filtered result.
Note that the stream can be infinite as well ;)
So getting the infinite list of results is not a good idea.
Basically streams are lazy and applying an intermediate operations to stream without having a terminal operation does nothing:
For example the following code prints nothing:
Stream<String> stream = Stream.of("hello","how","are", "you").filter(this::startsWithH)
private boolean startsWithH(String elem) {
System.out.println("Filtering element " + elem);
return elem.startsWith("h");
}
Now, when you do apply a terminal operation, it will still work element-by-element usually:
Example of execution:
Stream<String> stream = Stream.of("hello","how","are", "you")
.filter(this::startsWithH)
.map(String::toUpperCase)
stream.collect(toList());
This example yields the following execution chain:
filter("hello")
map("hello")
filter("how")
map("how")
filter("are") <--- filtered out, no call to map will be done
filter("you") <--- filtered out, no call to map will be done
But if so, you can't really operate on the "whole" stream in the example provided in this question (ok, there are stateful operations that must work on the whole stream, like sort, but its an entirely different story).
In other words if you want to get the data as a collection you should, well, collect the data. It won't be a stream anymore.
For this, you should use .collect(). And if you do have an infinite stream, don't forget to call limit beforehands ;)
I'm interested in sorting an list of object based on date attribute in that object. I can either use list sort method.
list.sort( (a, b) -> a.getDate().compareTo(b.getDate()) );
Or I can use stream sorted method
List<E> l = list.stream()
.sorted( (a, b) -> a.getDate().compareTo(b.getDate()))
.collect(Collectors.toList());
Out of both above option which should we use and why?
I know the former one will update my original list and later one will not update the original but instead give me a fresh new list object.
So, I don't care my original list is getting updated or not. So which one is good option and why?
If you only need to sort your List, and don't need any other stream operations (such as filtering, mapping, etc...), there's no point in adding the overhead of creating a Stream and then creating a new List. It would be more efficient to just sort the original List.
If you wish to known which is best, your best option is to benchmark it: you may reuse my answer JMH test.
It should be noted that:
List::sort use Arrays::sort. It create an array before sorting. It does not exists for other Collection.
Stream::sorted is done as state full intermediate operation. This means the Stream need to remember its state.
Without benchmarking, I'd say that:
You should use collection.sort(). It is easier to read: collection.stream().sorted().collect(toList()) is way to long to read and unless you format your code well, you might have an headache (I exaggerate) before understanding that this line is simply sorting.
sort() on a Stream should be called:
if you filter many elements making the Stream effectively smaller in size than the collection (sorting N items then filtering N items is not the same than filtering N items then sorting K items with K <= N).
if you have a map transformation after the sort and you loose a way to sort using the original key.
If you use your stream with other intermediate operation, then sort might be required / useful:
collection.stream() // Stream<U> #0
.filter(...) // Stream<U> #1
.sorted() // Stream<U> #2
.map(...) // Stream<V> #3
.collect(toList()) // List<V> sorted by U.
;
In that example, the filter apply before the sort: the stream #1 is smaller than #0, so the cost of sorting with stream might be less than Collections.sort().
If all that you do is simply filtering, you may also use a TreeSet or a collectingAndThen operation:
collection.stream() // Stream<U> #0
.filter(...) // Stream<U> #1
.collect(toCollection(TreeSet::new))
;
Or:
collection.stream() // Stream<U>
.filter(...) // Stream<U>
.collect(collectingAndThen(toList(), list -> {
list.sort();
return list;
})); // List<V>
Streams have some overheads because it creates many new objects like a concrete Stream, a Collector, and a new List. So if you just want to sort a list and doesn't care about whether the original gets changed or not, use List.sort.
There is also Collections.sort, which is an older API. The difference between it and List.sort can be found here.
Stream.sorted is useful when you are doing other stream operations alongside sorting.
Your code can also be rewritten with Comparator:
list.sort(Comparator.comparing(YourClass::getDate)));
First one would be better in term of performance. In the first one, the sort method just compares the elements of the list and orders them. The second one will create a stream from your list, sort it and create a new list from that stream.
In your case, since you can update the first list, the first approach is the better, both in term of performance and memory consumption. The second one is convenient if you need to and with a stream, or if you have a stream and want to end up with a sorted list.
You use the first method
list.sort((a, b) -> a.getDate().compareTo(b.getDate()));
it's much faster than the second one and it didn't create a new intermediate object. You could use the second method when you want to do some additional stream operations (e.g. filtering, map).
Let says i have a list of strings and i want to use those strings as input to a fluent builder.
List<String> scripts;
//initialize list
ScriptRunnerBuilder scriptRunnerBuilder = new ScriptRunnerBuilder();
BiFunction<String,ScriptRunnerBuilder,ScriptRunnerBuilder> addScript =
(script,builder) -> builer.addScript(script);
scriptRunnerBuilder = scripts.stream.map(script ->
addScript.apply(script,scriptRunnerBuilder)).......
scriptRunnerBuilder.build();
which terminal operation can i use so that the addScript function gets called for all elements in the list?
The issue is that the ScriptRunnerBuilder is immutable whereby ScriptRunnerBuilder.addScript() returns a new ScriptRunnerBuilder object rather than modifying existing – so i can't just us a foreach.
My intentions are to carry the result of the addScript() call and use that as input for the next element in the stream
In simplest way this should:
// create your builder
ScriptRunnerBuilder builder = new ScriptRunnerBuilder();
// add all scripts
scripts.forEach(script-> builder.addScript(script))
build results
scriptRunnerBuilder.build();
Because builder aggregates all data, and you have created it outside forEach lambda, you can access it directly. This will lead to less code and same result.
Or as #Holger suggested:
scripts.forEach(builder::addScript);
Use forEach instead of map and don't assign the result of the stream anymore
scripts.forEach(script -> addScript.apply(script,scriptRunnerBuilder));
i could use reduce operation but that is unnecessary as we are not combining results
Combining is exactly what you are doing.
You combine all scripts from List<String> to ScriptRunnerBuilder aren't you?
I agree that the #Beri's solution without stream probably is the simplest. But also there is a way with reduce(identity, accumulator, combiner) method where you don't need to create ScriptRunnerBuilder before:
ScriptRunnerBuilder builder = scripts.stream()
.reduce(new ScriptRunnerBuilder(), ScriptRunnerBuilder::addScript, (b1, b2) -> b1);
See more: Why is a combiner needed for reduce method that converts type in java 8
Update To not to rely on the fact that combiner not being invoked for sequential stream and to make it works with parallel one you have to implement the real combiner.
If you could add an overrided method addScript(ScriptRunnerBuilder otherBuilder) then the reduce will look like:
.reduce(new ScriptRunnerBuilder(), ScriptRunnerBuilder::addScript,
ScriptRunnerBuilder::addScript)
I have a Set<String> of "hostname:port" pairs and from that I'd like to create a Set<InetSocketAddress>. I tried it like:
Set<InetSocketAddress> ISAAddresses = StrAddresses
.stream().map(addr -> new InetSocketAddress(
addr.split(":")[0],
Integer.parseInt(addr.split(":")[1])));
But this produces the following error in IntelliJ:
Incompatible types. Required Set<InetSocketAddress> but 'map' was
inferred to Stream<R>: no instance(s) of type variable(s) R exist so
that Stream<R> conforms to Set<InetSocketAddress>
Something must be wrong with how I'm using the map and the lambda.
The Stream#map function does not return a Map. It transforms (maps) the current elements of your stream to other elements. So it generates from a Stream<X> a Stream<Y> using the given transformation function which takes X and outputs Y.
StrAddresses.stream() // String
.map(addr -> new InetSocketAddress(
addr.split(":")[0],
Integer.parseInt(addr.split(":")[1]))); // InetSocketAddress
You start with a Stream<String> and end up with a Stream<InetSocketAddress>.
To quote from its documentation:
Returns a stream consisting of the results of applying the given function to the elements of this stream.
If you want to transform that stream into a Set you need to use the Stream#collect method like so:
StrAddresses.stream()
.map(addr -> new InetSocketAddress(
addr.split(":")[0],
Integer.parseInt(addr.split(":")[1])))
.collect(Collectors.toSet());
The utility method Collectors.toSet() returns a collector for a well optimized Set. If you for example explicitly want a HashSet you can use this instead:
.collect(Collectors.toCollection(HashSet::new));
From its documentation:
Performs a mutable reduction operation on the elements of this stream. A mutable reduction is one in which the reduced value is a mutable result container, such as an ArrayList [...]
As a small note, you currently split the same element twice each time:
addr.split(":")[0], // First
Integer.parseInt(addr.split(":")[1]))) // Second
You could save that additional split procedure by memorizing the value before. In this case this can be done elegantly by using a second Stream#map call. First we transform from Stream<String> to Stream<String[]> and then to Stream<InetSocketAddress>:
StrAddresses.stream() // String
.map(addr -> addr.split(":")) // String[]
.map(addrData -> new InetSocketAddress(
addrData[0], Integer.parseInt(addrData[1]))) // InetSocketAddress
.collect(Collectors.toSet());
Note that Stream#map is a lazy operation. This means that Java will not transform the whole Stream from A to B once you call the method. It will wait until a non-lazy (finalizing) operation like Stream#collect comes, then traverse the Stream and apply each lazy operation element-wise. So you can add as many Stream#map calls as you like without producing extra loops over the whole Stream.
You need to collect the Stream of InetSocketAddress addresses returned after mapping to Set. This can be done as -
Set<InetSocketAddress> ISAAddresses = StrAddresses.stream()
.map(addr -> new InetSocketAddress(addr.split(":")[0], Integer.parseInt(addr.split(":")[1])))
.collect(Collectors.toSet());
So I have a Stream<Collection<Long>> that I obtain by doing a series of transformations on another stream.
What I need to do is collect the Stream<Collection<Long>> into one Collection<Long>.
I could collect them all into a list like this:
<Stream<Collection<Long>> streamOfCollections = /* get the stream */;
List<Collection<Long>> listOfCollections = streamOfCollections.collect(Collectors.toList());
And then I could iterate through that list of collections to combine them into one.
However, I imagine there must be a simple way to combine the stream of collections into one Collection<Long> using a .map() or .collect(). I just can't think of how to do it. Any ideas?
This functionality can be achieved with a call to the flatMap method on the stream, which takes a Function that maps the Stream item to another Stream on which you can collect.
Here, the flatMap method converts the Stream<Collection<Long>> to a Stream<Long>, and collect collects them into a Collection<Long>.
Collection<Long> longs = streamOfCollections
.flatMap( coll -> coll.stream())
.collect(Collectors.toList());
You could do this by using collect and providing a supplier (the ArrayList::new part):
Collection<Long> longs = streamOfCollections.collect(
ArrayList::new,
ArrayList::addAll,
ArrayList::addAll
);
You don't need to specify classes when not needed.
A better solution is:
Collection<Long> longs = streamOfCollections.collect(
ArrayList::new,
Collection::addAll,
Collection::addAll
);
Say, you don't need an ArrayList but need a HashSet, then you also need to edit only one line.