This question already has answers here:
Copy a stream to avoid "stream has already been operated upon or closed"
(10 answers)
Closed 3 years ago.
I'm learning the new Java 8 features, and while experimenting with streams (java.util.stream.Stream) and collectors, I realized that a stream can't be used twice.
Is there any way to reuse it?
If you want to have the effect of reusing a stream, you might wrap the stream expression in a Supplier and call myStreamSupplier.get() whenever you want a fresh one. For example:
Supplier<Stream<String>> sup = () -> someList.stream();
List<String> nonEmptyStrings = sup.get().filter(s -> !s.isEmpty()).collect(Collectors.toList());
Set<String> uniqueStrings = sup.get().collect(Collectors.toSet());
From the documentation:
A stream should be operated on (invoking an intermediate or terminal stream operation) only once.
A stream implementation may throw IllegalStateException if it detects that the stream is being reused.
So the answer is no, streams are not meant to be reused.
As others have said, "no you can't".
But it's useful to remember the handy summaryStatistics() for many basic operations:
So instead of:
List<Person> personList = getPersons();
personList.stream().mapToInt(p -> p.getAge()).average().getAsDouble();
personList.stream().mapToInt(p -> p.getAge()).min().getAsInt();
personList.stream().mapToInt(p -> p.getAge()).max().getAsInt();
You can:
// Can also be DoubleSummaryStatistics from mapToDouble()
IntSummaryStatistics stats = personList.stream()
.mapToInt(p-> p.getAge())
.summaryStatistics();
stats.getAverage();
stats.getMin();
stats.getMax();
The whole idea of the Stream is that it's once-off. This allows you to create non-reenterable sources (for example, reading the lines from the network connection) without intermediate storage. If you, however, want to reuse the Stream content, you may dump it into the intermediate collection to get the "hard copy":
Stream<MyType> stream = // get the stream from somewhere
List<MyType> list = stream.collect(Collectors.toList()); // materialize the stream contents
list.stream().doSomething // create a new stream from the list
list.stream().doSomethingElse // create one more stream from the list
If you don't want to materialize the stream, in some cases there are ways to do several things with the same stream at once. For example, you may refer to this or this question for details.
As others have noted the stream object itself cannot be reused.
But one way to get the effect of reusing a stream is to extract the stream creation code to a function.
You can do this by creating a method or a function object which contains the stream creation code. You can then use it multiple times.
Example:
public static void main(String[] args) {
List<Integer> list = Arrays.asList(1, 2, 3, 4, 5);
// The normal way to use a stream:
List<String> result1 = list.stream()
.filter(i -> i % 2 == 1)
.map(i -> i * i)
.limit(10)
.map(i -> "i :" + i)
.collect(toList());
// The stream operation can be extracted to a local function to
// be reused on multiple sources:
Function<List<Integer>, List<String>> listOperation = l -> l.stream()
.filter(i -> i % 2 == 1)
.map(i -> i * i)
.limit(10)
.map(i -> "i :" + i)
.collect(toList());
List<String> result2 = listOperation.apply(list);
List<String> result3 = listOperation.apply(Arrays.asList(1, 2, 3));
// Or the stream operation can be extracted to a static method,
// if it doesn't refer to any local variables:
List<String> result4 = streamMethod(list);
// The stream operation can also have Stream as argument and return value,
// so that it can be used as a component of a longer stream pipeline:
Function<Stream<Integer>, Stream<String>> streamOperation = s -> s
.filter(i -> i % 2 == 1)
.map(i -> i * i)
.limit(10)
.map(i -> "i :" + i);
List<String> result5 = streamOperation.apply(list.stream().map(i -> i * 2))
.filter(s -> s.length() < 7)
.sorted()
.collect(toCollection(LinkedList::new));
}
public static List<String> streamMethod(List<Integer> l) {
return l.stream()
.filter(i -> i % 2 == 1)
.map(i -> i * i)
.limit(10)
.map(i -> "i :" + i)
.collect(toList());
}
If, on the other hand, you already have a stream object which you want to iterate over multiple times, then you must save the content of the stream in some collection object.
You can then get multiple streams with the same content from than collection.
Example:
public void test(Stream<Integer> stream) {
// Create a copy of the stream elements
List<Integer> streamCopy = stream.collect(toList());
// Use the copy to get multiple streams
List<Integer> result1 = streamCopy.stream() ...
List<Integer> result2 = streamCopy.stream() ...
}
Come to think of it, this will of "reusing" a stream is just the will of carry out the desired result with a nice inline operation. So, basically, what we're talking about here, is what can we do to keep on processing after we wrote a terminal operation?
1) if your terminal operation returns a collection, the problem is solved right away, since every collection can be turned back into a stream (JDK 8).
List<Integer> l=Arrays.asList(5,10,14);
l.stream()
.filter(nth-> nth>5)
.collect(Collectors.toList())
.stream()
.filter(nth-> nth%2==0).forEach(nth-> System.out.println(nth));
2) if your terminal operations returns an optional, with JDK 9 enhancements to Optional class, you can turn the Optional result into a stream, and obtain the desired nice inline operation:
List<Integer> l=Arrays.asList(5,10,14);
l.stream()
.filter(nth-> nth>5)
.findAny()
.stream()
.filter(nth-> nth%2==0).forEach(nth-> System.out.println(nth));
3) if your terminal operation returns something else, i really doubt that you should consider a stream to process such result:
List<Integer> l=Arrays.asList(5,10,14);
boolean allEven=l.stream()
.filter(nth-> nth>5)
.allMatch(nth-> nth%2==0);
if(allEven){
...
}
The Functional Java library provides its own streams that do what you are asking for, i.e. they're memoized and lazy. You can use its conversion methods to convert between Java SDK objects and FJ objects, e.g. Java8.JavaStream_Stream(stream) will return a reusable FJ stream given a JDK 8 stream.
Related
I have two streams which I must materialize into two lists to get the permutations from both streams:
public Stream<Permutation> getAll() {
Stream.Builder<Permutation> all = Stream.builder();
// unfortunately, I must collect it into a list
var list1 = IntStream.iterate(0, d -> d - 1).limit(1000000).boxed().collect(Collectors.toList());
var list2 = IntStream.iterate(0, d -> d + 1).limit(1000000).boxed().collect(Collectors.toList());
// I must use a classic loop (and cannot operate on those streams) to avoid java.lang.IllegalStateException
for(var l1: list1) {
for(var l2: list2) {
// the permutation class consists of two int properties
all.add(new Permutation(l1, l2));
}
}
return all.build();
}
Is there a way to avoid to materialize list1 and list2 and operate only on those streams to return the permutations? I have tried it but I get
java.lang.IllegalStateException: stream has already been operated upon or closed
Therefore I have materialized the two lists and used a classic loop. However, I would like to improve the performance by doing the following steps:
avoid the materialization of list1 and list2
and maybe also use parallelStream for list1 and list2 to get the permutations faster
Is this possible? If so, how?
EDIT:
Thanks to #Andreas for the solution which works so far. However, I wonder how I can create a permutation from two getAll()-streams without the need to materialize it in between:
// The `Permutations` class holds two `Permuation`-instances.
Stream<Permutations> allPermutations(){
Stream<Permutation> stream1 = getAll();
Stream<Permutation> stream2 = getAll();
// returns java.lang.IllegalStateException: stream has already been operated upon or closed
return stream1.flatMap(s1->stream2.map(s2->new Permutations(s1,s2));
}
You can do it like this:
public Stream<Permutation> getAll() {
return IntStream.iterate(0, d -> d - 1).limit(1000000).boxed()
.flatMap(l1 -> IntStream.iterate(0, d -> d + 1).limit(1000000)
.mapToObj(l2 -> new Permutation(l1, l2)));
}
The caller can decide whether or not to use parallel processing:
// Sequential
Stream<Permutation> stream = getAll();
// Parallel
Stream<Permutation> stream = getAll().parallel();
No need to call sequential(), since iterate() returns a new sequential IntStream.
I'm trying to take a list of elements, make some manipulation on part of these elements and put the output of these manipulations in a new list.
I want to do it with only one iteration on the list.
The only way I found to do it is:
List<Integer> newList = numList.stream().reduce(new ArrayList<Integer>(),
(acc, value) -> {
if (value % 2 == 0) {
acc.add(value * 10);
}
return acc;
},
(l1, l2) -> {
l1.addAll(l2);
return l1;
}
);
As you can see, it's very cumbersome.
I can of course use filter and then map, but in this case I iterate the list twice.
In other languages (Javascript for example) this kind of reduce operation is very straightforward, for example:
arr.reduce((acc, value) => {
if (value % 2 == 0) {
acc.push(value * 10);
}
return acc;
}, new Array())
Amazing!
I was thinking to myself whether Java has some nicer version for this reduce or the Java code I wrote is the shortest way to do such an operation.
I can of course use filter and they map, but in this case I iterate the list twice.
That's not true. The elements of the Stream are iterated once, no matter how many intermediate operations you have.
Besides, the correct way to perform mutable reduction is to use collect:
List<Integer> newList =
numList.stream()
.filter(v -> v % 2 == 0)
.map(v -> v * 10)
.collect(Collectors.toList());
What you're doing is the opposite of functional programming, because you're using functions only to get a side effect.
You can simply not use reduce to make it even more explicit:
List<Integer> newList = numList.stream()
.filter(value -> value % 2 == 0)
.map(value -> value * 10)
.collect(Collectors.toList());
Edit: Streams iterate over the collection once by definition. According to the javadoc:
A stream should be operated on (invoking an intermediate or terminal stream operation) only once.
I need a trick to solve this problem i'm using Java 1.8 And I retrieved a an object from a method that returns a DoubleStream object. So, the problem is I could not reuse the stream after it has been consumed.
Here is the first version of the code :
DoubleStream stream = object.getvalueStream(a,b);
if(condtion)
stream.map(v -> v * 2);
stream.forEach(value -> {
// do something
}
The problem is that once the condition is true, the stream is consumed And I can not reuse it. So I tried to use a supplier to return a doubleStream from supplier and iterate overt it.
But still the same problem as I try to recover the stream from my stream object which is already used.
Here is my updated code :
DoubleStream stream = object.getvalueStream(a,b);
if(condtion)
stream.map(v -> v * 2);
Supplier>DoubleStream> streamSupplier = () -> DoubleStream.of(stream.toArray());
streamSupplier.get().forEach(value -> {
//Do something
But I still have the same problem since I create my supplier from my stream already used if the condition is true.
Thanks for your help.
once the condition is true, the stream is consumed And I can not reuse it
Intermediate operations (e.g. map) return a new stream, so you need to reassign the stream after the intermediate operation (map).
I.e.
DoubleStream stream = object.getvalueStream(a,b);
if (condition) {
stream = stream.map(v -> v * 2);
}
stream.forEach(value -> {
// do something
}
Note terminal operations (e.g. foreach) do not return a stream. So if you want many terminal operations, you should collect the stream so it can be reused.
Note also, there is also an intermediate version of foreach called peek if you wish to chain foreach (peek) calls.
Streams in Java are not up to be reused. You should collect the result and stream twice like
List<Double> doubles = object.getvalueStream(a,b).boxed().collect(toList());
if(condition) {
doubles = doubles.stream().map(v -> v * 2).boxed().collect(toList);
}
// and further processing here
doubles.stream().forEach(v ->
...
);
You can use map() if it is important for you to keep the stream without collecting it. The only drawback in this approach is that you have to check the condition each time
DoubleStream stream = object.getvalueStream(a,b).map(v-> condition ? v*2 : v).forEach(...);
Or just assign the right Stream to the variable
DoubleStream stream = condition ? object.getvalueStream(a,b).map(v->v*2) : object.getvalueStream(a,b).map(v->v*2).forEach(...);
I'd like to perform such action using streams:
List<String> listX = new ArrayList<>();
for(int i=0;i<listY.size();i++){
listX.add(String.format(ABC,i));
}
so I have to iterate over a list using the length of another list and additionally use counter i. After all i'd like to return the new list
using IntStream.range & IntStream.mapToObj methods. and don't operates variables out of lambda expression in stream api, that will resulting in Thread Interference or Memory Consistency Errors in parallel stream.
List<String> listX = IntStream.range(0, listY.size())
.mapToObj(i -> String.format(ABC,i))
.collect(Collectors.toList());
You can use IntStream to iterate using index of listY as shown below:
IntStream.range(0, listY.size()).forEach(i -> listX.add(String.format(ABC,i)));
You could also use Collectors.mapping, something like this:
IntStream.range(0, listY.size())
.collect(Collectors.mapping(i -> String.format(ABC,i),
Collectors.toList()));
I am trying to join two streams together. I ran into the problem that my stream closes and I don't understand why. Couldn't anyone please explain to me why the following happens.
The code below doesn't work. I get an exception on the flatMap function that the stream is already closed.
private Stream<KeyValuePair<T, U>> joinStreams(Stream<T> first, Stream<U> second) {
return
first
.map(x -> second
.map(y -> new KeyValuePair<T, U>(x, y))
)
.flatMap(x -> x);
}
When I first collect a list from the second stream and then grab a stream from that list it does work. See the example below.
private Stream<KeyValuePair<T, U>> joinStreams(Stream<T> first, Stream<U> second) {
List<U> secondList = second.collect(Collectors.toList());
return
first
.map(x -> secondList.stream()
.map(y -> new KeyValuePair<T, U>(x, y))
)
.flatMap(x -> x);
}
I can't figure out why this happens. Could anyone please explain this?
Edit:
Example of the code calling this function.
List<Integer> numbers1 = Arrays.asList(1, 2);
List<Integer> numbers2 = Arrays.asList(3, 4);
List<KeyValuePair<Integer, Integer>> combined = joinStreams(numbers1.stream(), numbers2.stream())
.collect(Collectors.toList());
// Expected result
// 1 3
// 1 4
// 2 3
// 2 4
The problem is that your code attempts to process the second Stream twice (once for each element of the first Stream). A Stream can only be processed once, just like an Iterator can only iterate over the elements of the underlying class once.
If your first Stream had only one element, the code would work, since the second Stream would only be processed once.
In the code that does work, you produce a new Stream (from secondList) for each element of the first Stream, so each Stream is processed once, regardless of how many elements are in the first Stream.