I want to work out if I can do something like this. Suppose I have a stream of the numbers 1 - 20. I want to utilise a feature such as a drop 3 (limit or skip I guess in Java terms?) and produce a stream of streams that is the numbers:
1 - 20, 4 - 20, 7- 20, etc
then possibly flat map these alll into one stream. I've tried various combinations of using Stream.iterate primarily to generate streams from streams, but I keep geting an IllegalStateException saying the stream has already operated upon or closed.
For example one may expect this code:
Stream.iterate(Stream.of(1,2,3,4,5), x -> x.skip(1).collect(Collectors.toList()).stream()).limit(5).flatMap(x -> x).forEach(x -> System.out.println(x));
To produce: 1,2,3,4,5,2,3,4,5,3,4,5,4,5,5
But it doesn't, it throws an exception. Am I missing something obvious here. I know of a takeWhile operation but don't think it's out until Java 9.
EDIT: I have managed to get an ugly solution to what I was trying to achieve using the following:
List<Integer> list = IntStream.rangeClosed(1, 20).boxed().collect(Collectors.toList());
Stream.iterate(list, x -> x.stream().skip(3).collect(Collectors.toList())).limit(7).map(x -> x.stream().limit(3).collect(Collectors.toList())).flatMap(List::stream).forEach(System.out::println);
So I have a list 1..20, then end up with a stream of lists 1..20, 4..20, 7..20, etc. If I then take the first 3 elements of each of these I end up with 1..3, 4..6, 7..9, etc. If I flatMap this, I get back to 1..20.
My question is, A. is there a way for me to not have to hardcode limit(7) above, so it automatically stops when I've processed all of the input element set, and B. I don't like having to collect into lists - can I not do this purely with streams prior to a collection at the end or even just printing after a flat map?
Perhaps you're looking for something like
IntStream.rangeClosed(1, 20)
.flatMap(i -> IntStream.rangeClosed(i, 20))
...which you can then do whatever you like with; e.g. .forEach(System.out::println).
If you want to do this for every third number, you're better off doing something like
IntStream.rangeClosed(0, 20/3)
.map(i -> 3 * i + 1)
.flatMap(i -> IntStream.rangeClosed(i, 20))
Related
I was practicing stream. Then got this strange behavior. I can't figure out the reason. That's why asking for help.
I wrote this:
IntStream.iterate(10, i -> i - 2)
.limit(5)
.sorted()
.takeWhile(i -> i > 2)
.forEach(System.out::println);
This is giving me nothing in the console, which is expected. When I am seeing the stream chain in IntelliJ IDEA, getting this:
So, after the execution of sorted() it is returning only one element (that element is 2 in the image) to the pipeline.
Now I commented the takeWhile() call. Like this:
IntStream.iterate(10, i -> i - 2)
.limit(5)
.sorted()
// .takeWhile(i -> i > 2)
.forEach(System.out::println);
This is also printing in the console as expected(from 10 to 2 with a difference of 2).
But problem is, this time sorted() returned 5 elements to pipeline. When I am seeing the stream chain in IntelliJ IDEA, getting this:
My question is, why the difference is seen in calling sorted()? If there is a takeWhile() after sorted() then it is returning one element. If I am commenting takeWhile(), sorted() is returning stream of 5 elements.
Is JVM doing anything here for achieving better performance?
Thanks in advance.....
Stream.sorted() is a stateful operation, so it is actually maintaining the [2, 4, 6, 8, 10] state. However, it will only send 1 item at a time.
So, Initially it will send 1 entry: '2' to takeWhile operator which can then reject the entry and stop the stream chain. That is why you are seeing sorted returning only 2 to takeWhile operator.
But, when sorted is not present, takeWhile will get 10 as the entry. And as the condition is true, stream will continue until takeWhile receives 2.
Edit:
When takeWhile is not present, you are getting all 5 items as output because you have 5 items in stream chain and there is no takeWhile to stop the stream chain.
In Intellij's Stream Trace, On the right side of sorted you are seeing how many elements are being passed to next operator after performing sorted() operation.
In the first case, only 1 element is being passed to takeWhile() as takeWhile() is terminating the stream chain after receiving '2'. But in the second case all the elements are being passed to forEach.
Akka Stream code:
Source.range(1, 100).map(i -> {
return Stream.of(i*4).toList();
}).runWith(Sink.foreach(a ->System.out.println(a)), materializer);
I want to create a method say printList() and move the runWith code(Sink.foreach(a ->System.out.println(a))), so it should be like this runWith(printList(), materializer)
The printList() method would have to return a Sink, or a runnable graph / sink shape. I don’t understand what you’re trying to do here exactly, so I’ll break down some of the things that confuse me, and maybe that will help you work out better code:
Source.range(1, 100) is a good start and will output integers one at a time from 1 to 100.
The next step, map(i -> Stream.of(i * 4).toList()) is where my confusion starts. This will return a List<Integer> with exactly one element: i x 4 (so 4, 8, 12, 16, Etc in succession). The same effect, without a List, can be achieved simply with map(i -> i * 4)... this will return each element of the upstream Source multiplied by four.
Now, the runWith(Sink.foreach(a -> System.out.println(a)), mat)... in the code as you have it, it will receive a List with one element in it, 100 times.
If you want to take a Source of 100 Integer elements and put them into a List, there’s a Sink operator that will do exactly that: Sink.seq(). However you’d have to be sure that the Source is actually finite otherwise you’ll never get to the end of the collection that this Sink operator creates.
Guessing at your purpose, I would expect you want something like:
Source.range(1, 100)
.map(i -> i * 4)
.runWith(
Sink.forEach(a -> System.out.println(a)),
materializer);
This will print
4
8
12
16
20
...
400
I'm practicing streams in java 8 and im trying to make a Stream<Integer> containing the multiples of 2. There are several tasks in one main class so I won't link the whole block but what i got so far is this:
Integer twoToTheZeroth = 1;
UnaryOperator<Integer> doubler = (Integer x) -> 2 * x;
Stream<Integer> result = ?;
My question here probably isn't related strongly to the streams, more like the syntax, that how should I use the doubler to get the result?
Thanks in advance!
You can use Stream.iterate.
Stream<Integer> result = Stream.iterate(twoToTheZeroth, doubler);
or using the lambda directly
Stream.iterate(1, x -> 2*x);
The first argument is the "seed" (ie first element of the stream), the operator gets applied consecutively with every element access.
EDIT:
As Vinay points out, this will result in the stream being filled with 0s eventually (this is due to int overflow). To prevent that, maybe use BigInteger:
Stream.iterate(BigInteger.ONE,
x -> x.multiply(BigInteger.valueOf(2)))
.forEach(System.out::println);
Arrays.asList(1,2,3,4,5).stream().map(x -> x * x).forEach(x -> System.out.println(x));
so you can use the doubler in the map caller
I have a question on the intermediate stages sequential state - are the operations from a stage applied to all the input stream (items) or are all the stages / operations applied to each stream item?
I'm aware the question might not be easy to understand, so I'll give an example. On the following stream processing:
List<String> strings = Arrays.asList("Are Java streams intermediate stages sequential?".split(" "));
strings.stream()
.filter(word -> word.length() > 4)
.peek(word -> System.out.println("f: " + word))
.map(word -> word.length())
.peek(length -> System.out.println("m: " + length))
.forEach(length -> System.out.println("-> " + length + "\n"));
My expectation for this code is that it will output:
f: streams
f: intermediate
f: stages
f: sequential?
m: 7
m: 12
m: 6
m: 11
-> 7
-> 12
-> 6
-> 11
Instead, the output is:
f: streams
m: 7
-> 7
f: intermediate
m: 12
-> 12
f: stages
m: 6
-> 6
f: sequential?
m: 11
-> 11
Are the items just displayed for all the stages, due to the console output? Or are they also processed for all the stages, one at a time?
I can further detail the question, if it's not clear enough.
This behaviour enables optimisation of the code. If each intermediate operation were to process all elements of a stream before proceeding to the next intermediate operation then there would be no chance of optimisation.
So to answer your question, each element moves along the stream pipeline vertically one at a time (except for some stateful operations discussed later), therefore enabling optimisation where possible.
Explanation
Given the example you've provided, each element will move along the stream pipeline vertically one by one as there is no stateful operation included.
Another example, say you were looking for the first String whose length is greater than 4, processing all the elements prior to providing the result is unnecessary and time-consuming.
Consider this simple illustration:
List<String> stringsList = Arrays.asList("1","12","123","1234","12345","123456","1234567");
int result = stringsList.stream()
.filter(s -> s.length() > 4)
.mapToInt(Integer::valueOf)
.findFirst().orElse(0);
The filter intermediate operation above will not find all the elements whose length is greater than 4 and return a new stream of them but rather what happens is as soon as we find the first element whose length is greater than 4, that element goes through to the .mapToInt which then findFirst says "I've found the first element" and execution stops there. Therefore the result will be 12345.
Behaviour of stateful and stateless intermediate operations
Note that when a stateful intermediate operation as such of sorted is included in a stream pipeline then that specific operation will traverse the entire stream. If you think about it, this makes complete sense as in order to sort elements you'll need to see all the elements to determine which elements come first in the sort order.
The distinct intermediate operation is also a stateful operation, however, as #Holger has mentioned unlike sorted, it does not require traversing the entire stream as each distinct element can get passed down the pipeline immediately and may fulfil a short-circuiting condition.
stateless intermediate operations such as filter , map etc do not have to traverse the entire stream and can freely process one element at a time vertically as mentioned above.
Lastly, but not least it's also important to note that, when the terminal operation is a short-circuiting operation the terminal-short-circuiting methods can finish before traversing all the elements of the underlying stream.
reading: Java 8 stream tutorial
Your answer is loop fusion. What we see is that the four
intermediate operations filter() – peek() – map() – peek() – println using forEach() which is a kinda terminal operation have been logically
joined together to constitute a single pass. They are executed in
order for each of the individual element. This joining
together of operations in a single pass is an optimization technique
known as loop fusion.
More for reading: Source
An intermediate operation is always lazily executed. That is to say
they are not run until the point a terminal operation is reached.
A few of the most popular intermediate operations used in a stream
filter – the filter operation returns a stream of elements that
satisfy the predicate passed in as a parameter to the operation. The
elements themselves before and after the filter will have the same
type, however the number of elements will likely change
map – the map operation returns a stream of elements after they have
been processed by the function passed in as a parameter. The
elements before and after the mapping may have a different type, but
there will be the same total number of elements.
distinct – the distinct operation is a special case of the filter
operation. Distinct returns a stream of elements such that each
element is unique in the stream, based on the equals method of the
elements
.java-8-streams-cheat-sheet
Apart from optimisation, the order of processing you'd describe wouldn't work for streams of indeterminate length, like this:
DoubleStream.generate(Math::random).filter(d -> d > 0.9).findFirst();
Admittedly this example doesn't make much sense in practice, but the point is that rather than backed by a fixed-size collection,DoubleStream.generate() creates a potentially infinite stream. The only way to process this is element by element.
Why this code in java 8:
IntStream.range(0, 10)
.peek(System.out::print)
.limit(3)
.count();
outputs:
012
I'd expect it to output 0123456789, because peek preceeds limit.
It seems to me even more peculiar because of the fact that this:
IntStream.range(0, 10)
.peek(System.out::print)
.map(x -> x * 2)
.count();
outputs 0123456789 as expected (not 02481012141618).
P.S.: .count() here is used just to consume stream, it can be replaced with anything else
The most important thing to know about streams are that they do not contain elements themselves (like collections) but are working like a pipe whose values are lazily evaluated. That means that the statements that build up a stream - including mapping, filtering, or whatever - are not evaluated until the terminal operation runs.
In your first example, the stream tries to count from 0 to 9, one at each time doing the following:
print out the value
check whether 3 values are passed (if yes, terminate)
So you really get the output 012.
In your second example, the stream again counts from 0 to 9, one at each time doing the following:
print out the value
maping x to x*2, thus forwarding the double of the value to the next step
As you can see the output comes before the mapping and thus you get the result 0123456789. Try to switch the peek and the map calls. Then you will get your expected output.
From the docs:
limit() is a short-circuiting stateful intermediate operation.
map() is an intermediate operation
Again from the docs what that essentially means is that limit() will return a stream with x values from the stream it received.
An intermediate operation is short-circuiting if, when presented with infinite input, it may produce a finite stream as a result.
Streams are defined to do lazy processing. So in order to complete your count() operation it doesn’t need to look at the other items. Otherwise, it would be broken, as limit(…) is defined to be a proper way of processing infinite streams in a finite time (by not processing more than limit items).
In principle, it would be possible to complete your request without ever looking at the int values at all, as the operation chain limit(3).count() doesn’t need any processing of the previous operations (other than verifying whether the stream has at least 3 items).
Streams use lazy evaluation, the intermediate operations, i.e. peek() are not executed till the terminal operation runs.
For instances, the following code will just print 1 .In fact, as soon as the first element of the stream,1, will reach the terminal operation, findAny(), the stream execution will be ended.
Arrays.asList(1,2,3)
.stream()
.peek(System.out::print)
.filter((n)->n<3)
.findAny();
Viceversa, in the following example, will be printed 123. In fact the terminal operation, noneMatch(), needs to evaluate all the elements of the stream in order to make sure there is no match with its Predicate: n>4
Arrays.asList(1, 2, 3)
.stream()
.peek(System.out::print)
.noneMatch(n -> n > 4);
For future readers struggling to understand how the count method doesn't execute the peek method before it, I thought I add this additional note:
As per Java 9, the Java documentation for the count method states that:
An implementation may choose to not execute the stream pipeline
(either sequentially or in parallel) if it is capable of computing the
count directly from the stream source.
This means terminating the stream with count is no longer enough to ensure the execution of all previous steps, such as peek.