According to this question, peek() is lazy it means we should somehow activate it. In fact, to activate it to print something out to the console I tried this :
Stream<String> ss = Stream.of("Hi","Hello","Halo","Hacker News");
ss.parallel().peek(System.out::println);
System.out.println("lol"); // I wrote this line to print sth out to terminal to wake peek method up
But that doesn't work and the output is :
lol
Thus, how can I make the peek function actually work?
If there is no way to that so whats the point of using peek?
You have to use terminal operation on a stream for it to execute (peek is not terminal, it is an intermediate operation, that returns a new Stream), e.g. count():
Stream<String> ss = Stream.of("Hi","Hello","Halo","Hacker News");
ss.parallel().peek(System.out::println).count();
Or replace peek with forEach (which is terminal):
ss.parallel().forEach(System.out::println);
peek() method uses Consumer as parameter which means that potentially you can mutate the state of the incoming element. At the same time Java documentation says that peek should be mostly used for debugging purposes. It is an intermediate operator and requires a terminal operator like forEach.
stream().peek(Consumer).forEach(Consumer);
Related
I found this bug on Sonarqube:
private String getMacAdressByPorts(Set<Integer> ports) {
ports.stream().sorted(); // sonar list show "Refactor the code so this stream pipeline is used"
return ports.toString();
} //getMacAdressByPorts
I have been searching for a long time on the Internet, but it was no use. Please help or try to give some ideas how to achieve this.
The sorted() method has no effect on the Set you pass in; actually, it's a non-terminal operation, so it isn't even executed. If you want to sort your ports, you need something like
return ports.stream().sorted().collect(Collectors.joining(","));
EDIT:
as #Slaw correctly points out, to get the same format you had before (ie [item1, item2, item3], you also need to add the square brackets to the joining collector, ie Collectors.joining(", ", "[", "]"). I left those out for simplicity.
From the Sonar Source documentation about this warning (emphasis of mine):
Stream operations are divided into intermediate and terminal operations, and are combined to form stream pipelines. After the terminal operation is performed, the stream pipeline is considered consumed, and cannot be used again. Such a reuse will yield unexpected results.
Official JavaDoc for stream() gives more details on sorted() (emphasis of mine):
Returns a stream consisting of the elements of this stream, sorted according to natural order. If the elements of this stream are not Comparable, a java.lang.ClassCastException may be thrown when the terminal operation is executed. [...] This is a stateful intermediate operation.
This implies that only using sorted() will yield no result. From the Oracle Stream package documentation (still emphasis of mine):
Stream operations are divided into intermediate and terminal operations, and are combined to form stream pipelines. A stream pipeline consists of a source (such as a Collection, an array, a generator function, or an I/O channel); followed by zero or more intermediate operations such as Stream.filter or Stream.map; and a terminal operation such as Stream.forEach or Stream.reduce.
Intermediate operations return a new stream. They are always lazy; executing an intermediate operation such as filter() does not actually perform any filtering, but instead creates a new stream that, when traversed, contains the elements of the initial stream that match the given predicate. Traversal of the pipeline source does not begin until the terminal operation of the pipeline is executed.
sorted() returns another stream(), and not a sorted list. To solve your Sonar issue (and maybe your code issue, in that manner), you have to call a terminal operation in order to run all the intermediate operations. You can find a list (non-exhausive, I think) of terminal operations on CodeJava for instance.
In your case, the solution might look like:
private String getMacAdressByPorts(Set<Integer> ports) {
/* Ports > Stream > Sort (intermediate operation) >
/ Collector (final operation) > List > String
/ Note that you need to import the static method toList()
/ from the Collector API, otherwise it won't compile
*/
return ports.stream().sorted().collect(toList()).toString();
}
I finally solved the problem used the code below.
private String getMacAdressByPorts(Set<Integer> ports) {
return ports.stream().sorted().collect(Collectors.toList()).toString();
In our project we are migrating to java 8 and we are testing the new features of it.
On my project I'm using Guava predicates and functions to filter and transform some collections using Collections2.transform and Collections2.filter.
On this migration I need to change for example guava code to java 8 changes. So, the changes I'm doing are the kind of:
List<Integer> naturals = Lists.newArrayList(1,2,3,4,5,6,7,8,9,10,11,12,13);
Function <Integer, Integer> duplicate = new Function<Integer, Integer>(){
#Override
public Integer apply(Integer n)
{
return n * 2;
}
};
Collection result = Collections2.transform(naturals, duplicate);
To...
List<Integer> result2 = naturals.stream()
.map(n -> n * 2)
.collect(Collectors.toList());
Using guava I was very confortable debugging the code since I could debug each transformation process but my concern is how to debug for example .map(n -> n*2).
Using the debugger I can see some code like:
#Hidden
#DontInline
/** Interpretively invoke this form on the given arguments. */
Object interpretWithArguments(Object... argumentValues) throws Throwable {
if (TRACE_INTERPRETER)
return interpretWithArgumentsTracing(argumentValues);
checkInvocationCounter();
assert(arityCheck(argumentValues));
Object[] values = Arrays.copyOf(argumentValues, names.length);
for (int i = argumentValues.length; i < values.length; i++) {
values[i] = interpretName(names[i], values);
}
return (result < 0) ? null : values[result];
}
But it isn't as straighforward as Guava to debug the code, actually I couldn't find the n * 2 transformation.
Is there a way to see this transformation or a way to easy debug this code?
EDIT: I've added answer from different comments and posted answers
Thanks to Holger comment that answered my question, the approach of having lambda block allowed me to see the transformation process and debug what happened inside lambda body:
.map(
n -> {
Integer nr = n * 2;
return nr;
}
)
Thanks to Stuart Marks the approach of having method references also allowed me to debug the transformation process:
static int timesTwo(int n) {
Integer result = n * 2;
return result;
}
...
List<Integer> result2 = naturals.stream()
.map(Java8Test::timesTwo)
.collect(Collectors.toList());
...
Thanks to Marlon Bernardes answer I noticed that my Eclipse doesn't show what it should and the usage of peek() helped to display results.
I usually have no problem debugging lambda expressions while using Eclipse or IntelliJ IDEA. Just set a breakpoint and be sure not to inspect the whole lambda expression (inspect only the lambda body).
Another approach is to use peek to inspect the elements of the stream:
List<Integer> naturals = Arrays.asList(1,2,3,4,5,6,7,8,9,10,11,12,13);
naturals.stream()
.map(n -> n * 2)
.peek(System.out::println)
.collect(Collectors.toList());
UPDATE:
I think you're getting confused because map is an intermediate operation - in other words: it is a lazy operation which will be executed only after a terminal operation was executed. So when you call stream.map(n -> n * 2) the lambda body isn't being executed at the moment. You need to set a breakpoint and inspect it after a terminal operation was called (collect, in this case).
Check Stream Operations for further explanations.
UPDATE 2:
Quoting Holger's comment:
What makes it tricky here is that the call to map and the lambda
expression are in one line so a line breakpoint will stop on two
completely unrelated actions.
Inserting a line break right after map(
would allow you to set a break point for the lambda expression only.
And it’s not unusual that debuggers don’t show intermediate values of
a return statement. Changing the lambda to n -> { int result=n * 2; return result; }
would allow you to inspect result. Again, insert line
breaks appropriately when stepping line by line…
IntelliJ has such a nice plugin for this case as a Java Stream Debugger plugin. You should check it out: https://plugins.jetbrains.com/plugin/9696-java-stream-debugger?platform=hootsuite
It extends the IDEA Debugger tool window by adding the Trace Current Stream Chain button, which becomes active when debugger stops inside of a chain of Stream API calls.
It has nice interface for working with separate streams operations and gives you opportunity to follow some values that u should debug.
You can launch it manually from the Debug window by clicking here:
Debugging lambdas also works well with NetBeans. I'm using NetBeans 8 and JDK 8u5.
If you set a breakpoint on a line where there's a lambda, you actually will hit once when the pipeline is set up, and then once for each stream element. Using your example, the first time you hit the breakpoint will be the map() call that's setting up the stream pipeline:
You can see the call stack and the local variables and parameter values for main as you'd expect. If you continue stepping, the "same" breakpoint is hit again, except this time it's within the call to the lambda:
Note that this time the call stack is deep within the streams machinery, and the local variables are the locals of the lambda itself, not the enclosing main method. (I've changed the values in the naturals list to make this clear.)
As Marlon Bernardes pointed out (+1), you can use peek to inspect values as they go by in the pipeline. Be careful though if you're using this from a parallel stream. The values can be printed in an unpredictable order across different threads. If you're storing values in a debugging data structure from peek, that data structure will of course have to be thread-safe.
Finally, if you're doing a lot of debugging of lambdas (especially multi-line statement lambdas), it might be preferable to extract the lambda into a named method and then refer to it using a method reference. For example,
static int timesTwo(int n) {
return n * 2;
}
public static void main(String[] args) {
List<Integer> naturals = Arrays.asList(3247,92837,123);
List<Integer> result =
naturals.stream()
.map(DebugLambda::timesTwo)
.collect(toList());
}
This might make it easier to see what's going on while you're debugging. In addition, extracting methods this way makes it easier to unit test. If your lambda is so complicated that you need to be single-stepping through it, you probably want to have a bunch of unit tests for it anyway.
Just to provide more updated details (Oct 2019), IntelliJ has added a pretty nice integration to debug this type of code that is extremely useful.
When we stop at a line that contains a lambda if we press F7 (step into) then IntelliJ will highlight what will be the snippet to debug. We can switch what chunk to debug with Tab and once we decided it then we click F7 again.
Here some screenshots to illustrate:
1- Press F7 (step into) key, will display the highlights (or selection mode)
2- Use Tab multiple times to select the snippet to debug
3- Press F7 (step into) key to step into
Intellij IDEA 15 seems to make it even easier, it allows to stop in a part of the line where lambda is, see the first feature: http://blog.jetbrains.com/idea/2015/06/intellij-idea-15-eap-is-open/
Debugging using IDE's are always-helpful, but the ideal way of debugging through each elements in a stream is to use peek() before a terminal method operation since Java Steams are lazily evaluated, so unless a terminal method is invoked, the respective stream will not be evaluated.
List<Integer> numFromZeroToTen = Arrays.asList(1,2,3,4,5,6,7,8,9,10);
numFromZeroToTen.stream()
.map(n -> n * 2)
.peek(n -> System.out.println(n))
.collect(Collectors.toList());
According to Javadocs for SE 8 Stream.map() does the following
Returns a stream consisting of the results of applying the given function to the elements of this stream.
However, a book I'm reading (Learning Network Programming with Java, Richard M. Reese) on networking implements roughly the following code snippet in an echo server.
Supplier<String> inputLine = () -> {
try {
return br.readLine();
} catch(IOException e) {
e.printStackTrace();
return null;
}
};
Stream.generate(inputLine).map((msg) -> {
System.out.println("Recieved: " + (msg == null ? "end of stream" : msg));
out.println("echo: " + msg);
return msg;
}).allMatch((msg) -> msg != null);
This is supposed to be a functional way to accomplish getting user input to print to the socket input stream. It works as intended, but I don't quite understand how. Is it because map knows the stream is infinite so it lazily executes as new stream tokens become available? It seems like adding something to a collection currently being iterated over by map is a little black magick. Could someone please help me understand what is going on behind the scenes?
Here is how I restated this in order to avoid the confusing map usage. I believe the author was trying to avoid an infinite loop since you can't break out of a forEach.
Stream.generate(inputLine).allMatch((msg) -> {
boolean alive = msg != null;
System.out.println("Recieved: " + (alive ? msg : "end of stream"));
out.println("echo: " + msg);
return alive;
});
Streams are lazy. Think of them as workers in a chain that pass buckets to each other. The laziness is in the fact that they will only ask the worker behind them for the next bucket if the worker in front of them asks them for it.
So it's best to think about this as allMatch - being a final action, thus eager - asking the map stream for the next item, and the map stream asking the generate stream for the next item, and the generate stream going to its supplier, and providing that item as soon as it arrives.
It stops when allMatch stops asking for items. And it does so when it knows the answer. Are all items in this stream not null? As soon as the allMatch receives an item that is null, it knows the answer is false, and will finish and not ask for any more items. Because the stream is infinite, it will not stop otherwise.
So you have two factors causing this to work the way it work - one is allMatch asking eagerly for the next item (as long as the previous ones weren't null), and the generate stream that - in order to supply that next item - may need to wait for the supplier that waits for the user to send more input.
But it should be said that map shouldn't have been used here. There should not be side effects in map - it should be used for mapping an item of one type to an item of another type. I think this example was used only as a study aid. The much simpler and straightforward way would be to use BufferedReader's method lines() which gives you a finite Stream of the lines coming from the buffered reader.
Yes - Streams are setup lazily until and unless you perform a terminal operation (final action) on the Stream. Or simpler:
For as long as the operations on your stream return another Stream, you do not have a terminal operation, and you keep on chaining until you have something returning anything other than a Stream, including void.
This makes sense, as to be able to return anything other than a Stream, the operations earlier in your stream will need to be evaluated to actually be able to provide the data.
In this case, and as per documentation, allMatch returns a boolean, and thus final execution of your stream is required to calculate that boolean. This is the point also where you provide a Predicate limiting your resulting Stream.
Also note that in the documentation it states:
This is a short-circuiting terminal operation.
Follow that link for more information on those terminal operations, but a terminal operation basically means that it will actually execute the operation. Additionally, the limiting of your infinite stream is the 'short-circuiting' aspect of that method.
Here are two the most relevant sentences of the java-stream documentation. The snippet you provided is a perfect example of these working together:
Stream::generate(Supplier<T> s) says that it returns:
Returns an infinite sequential unordered stream where each element is generated by the provided Supplier.
3rd dot of Stream package documentation:
Laziness-seeking. Many stream operations, such as filtering, mapping, or duplicate removal, can be implemented lazily, exposing opportunities for optimization. For example, "find the first String with three consecutive vowels" need not examine all the input strings. Stream operations are divided into intermediate (Stream-producing) operations and terminal (value- or side-effect-producing) operations. Intermediate operations are always lazy.
In a shortcut, this generated stream await the further elements until the terminal operation is reached. As long as the execution inside the supplied Supplier<T>, the stream pipeline continues.
As an example, if you provide the following Supplier, the execution has no chance to stop and will continue infinitely:
Supplier<String> inputLine = () -> {
return "Hello world";
};
Are the lambda expressions evaluated at the place where we write them or in any other class of Java?
For example :
Stream<Student> absent = students.values().stream().filter(s -> !s.present());
Will the above lambda expression passed to the filter method be executed immediately in a given class where the code is written OR in another class and will it take some more time (in terms of nano seconds) than if the code was written in conventional coding style prior to Java 8?
When you compile your sources, the compiler will insert an invokedynamic byte code instruction for the lambda expression that you use. The actual implementation (which in your case is a Predicate) will be created at runtime via ASM. It will not even be present on hard disk when you run it - meaning the class is generated in memory, there will be no .class file for Predicate. That's a big difference between an anonymous class for example - that will generate a class file when you compile it.
You can see the generated file for the Predicate if you run your example with :
-Djdk.internal.lambda.dumpProxyClasses=/Your/Path/Here
Otherwise Eran's answer is correct, Streams are driven by the terminal operation, if such is not present nothing gets executed. You should absolutely read the excellent Holger's answer about even more interesting differences.
The body of the lambda expression passed to the filter method in your example won't be executed at all, since filter is an intermediate operation, which only gets executed for Streams that end in a terminal operation, such as collect, forEach, etc...
If you add a terminal operation, such as collecting the elements of the Stream to a List:
List<Student> absent = students.values().stream().filter(s -> !s.present()).collect(Collectors.toList());
the body of the lambda expression will be executed for each element of your Stream, in order for the terminal operation to be able to produce its output.
Note that this behavior would not change if you passed an anonymous class instance or some other implementation of the Predicate interface to your filter method instead of the lambda expression.
The expressions are lazy evaluated, which means they'll only actually be evaluated when you actually try to 'terminate' the stream - i.e. use an operation that takes a stream but returns something else, like collect, min, max, reduce, etc. Operations which take a stream as input and return a stream as output are usually lazy.
Lambda expressions are essentially objects with a single method, so they're evaluated whenever that method is called.
In your particular case they're never evaluated. A Stream does not evaluate the expressions until you call a terminating operation (collect, findAny, etcetera)
I have a need in a block of code to consume 'n' items from a stream then finish, in essence:
public static <T> void eat(Stream<T> stream, int n)
// consume n items of the stream (and throw them away)
}
In my situation, I can't change the signature to return Stream<T> and simply return stream.skip(n); I have to actually throw away some elements from the stream (not simple logic) - to be ready for a down stream consumer which doesn't need to know how, or even that, this has happened.
The simplest way to do this is to use limit(n), but I have to call a stream terminating method to activate the stream, so in essence I have:
public static <T> void skip(Stream<T> stream, int n) {
stream.limit(n).forEach(t -> {});
}
Note: This code is a gross over simplification of the actual code and is for illustrative purposes only. Actually, limit won't work because there is logic around what/how to consume elements. Think of it like consuming "header" elements from a stream, then having a consumer consume the "body" elements.
This question is about the "do nothing" lambda t -> {}.
Is there a "do nothing" consumer somewhere in the JDK, like the "do nothing" function Function.identity()?
No, JDK does not provide dummy consumer as well as other predefined functions like dummy runnable, always-true predicate or supplier which always returns zero. Just write t -> {}, it's anyways shorter than calling any possible ready method which will do the same.
Introducing the dummy (empty) consumer was considered in the scope of the ticket:
[JDK-8182978] Add default empty consumer - Java Bug System.
Archived: [JDK-8182978] Add default empty consumer - Java Bug System.
According to the ticket, it was decided not to introduce it.
Therefore, there is no dummy (empty) consumer in the JDK.
Yes. Well, more or less yes...
Since a Function is also a Consumer, you can use Function.identity() as a "do nothing" Consumer.
However, the compiler needs a little help to make the leap:
someStream.forEach(identity()::apply);