I would like to clarify a behavior of Java parallel streams. If I were to use a parallel stream as shown below, can I take it as a guarantee that the line "Should happen at the end" shown in the code below will only print after all parallel tasks have executed?
public static void main(String[] args) {
Stream.of(1, 2, 3, 5, 7, 8, 9, 10, 11, 23, 399, 939).parallel().forEach(
integer -> System.out
.println(Thread.currentThread().getName() + ", for number " + integer));
System.out.println("Should happen at the end");
}
The repeated trials always print "Should happen at the end" predictably at the end, as shown below. This maybe happening because main thread is also used to process a few requests. Is it possible that in some scenario that main thread is not used and in that case "Should happen at the end" will print before all the ForkJoinPool.commonPool-worker have finished executing their tasks?
main, for number 7
main, for number 6
ForkJoinPool.commonPool-worker-9, for number 3
ForkJoinPool.commonPool-worker-9, for number 4
ForkJoinPool.commonPool-worker-4, for number 1
ForkJoinPool.commonPool-worker-4, for number 10
ForkJoinPool.commonPool-worker-2, for number 2
ForkJoinPool.commonPool-worker-11, for number 5
ForkJoinPool.commonPool-worker-9, for number 8
main, for number 9
Should happen at the end
Stream terminal operations are not asynchronous. It means that Java will give back control to the calling thread only after forEach is terminated. Therefore, what you print after forEach is necessarily printed after in console.
Note, however, that if you were to use java.util.Logger API instead of System.out, output ordering would not be as predictable, as logging API itself cannot (for performance reason mainly) guarantee ordering of outputs.
Further reading on stream operations: Oracle official documentation.
Related
I was investigating Mockito's behavior in one of our tests and I didn't understand its behavior. The following code snippets shows the same behavior :
#Test
public void test(){
var mymock = Mockito.mock(Object.class);
var l = List.of(1,2,3);
l.parallelStream().forEach(val -> {
int finalI = val;
doAnswer(invocationOnMock -> {
log.info("{}",finalI);
return null;
}).when(mymock).toString();
mymock.toString();
});
}
The output I was expecting is prints of 1,2,3 in some order (not nessaseraly sorted) :
The output I received :
2
2
2
Why I got this output ?
This is a classic race condition: your parallel stream executes the forEach lambda three times in parallel. In this case, all three threads managed to execute the doAnswer() call, before any of them got to the mymock.toString() line. It also just so happens that the execution where val=2 happened to run last. Happily, Mockito is reasonably thread-safe, so you don't just get exceptions thrown.
One way your scenario might occur is if the threads happened to run like this:
Main thread: calls l.parallelStream().forEach(), which spawns 3 threads with val equal to 1, 2 and 3
val=1 thread: executes finalI=val, giving it a value of 1 in this thread
val=2 thread: executes finalI=val, giving it a value of 2 in this thread
val=3 thread: executes finalI=val, giving it a value of 3 in this thread
val=3 thread: calls doAnswer(...) which makes future calls to mymock.toString() print out 3
val=1 thread: calls doAnswer(...) which makes future calls to mymock.toString() print out 1
val=2 thread: calls doAnswer(...) which makes future calls to mymock.toString() print out 2
val=1 thread: calls mymock.toString(), which prints out 2
val=2 thread: calls mymock.toString(), which prints out 2
val=3 thread: calls mymock.toString(), which prints out 2
Note that this isn't necessarily happening in order, or even in this order:
some parts of this may literally be happening at the same time, in different cores of your processor.
while any steps within a single thread would always run in order, they might run before, after, or interleaved with any steps from any other thread.
Because the threads are running in parallel, and you have not put anything in place to manage this, there is no guarantee whatsoever about the order in which these calls occur: your output could just as easily have been 1 1 1, 1 2 3, 3 2 1 or 3 2 2 or 1 1 3 etc.
Here is my code:
int count=20;
Stream.iterate(4,i->i<count,i->i+1).parallel().forEach(i -> {
try {
Thread.sleep(new Random().nextInt(2000));
} catch (InterruptedException ignored) {
}
System.out.println(i);
});
I expect this will go parallel, but!
When i run it, parallel() seems "dead", numbers from 4 to 19 comes with a "perfect" sequence which was not i want.
So i modified the "iterate" part like this:
.iterate(4,i->i+1).limit(count-4)
There comes fixed, the parallel() works again.
So, Why??? The iterate-limit-parallel combo seems too silly.
...Or in fact, there may be a reason?
please help me.
THX!
ps: I print the thread ID, in the first case, that always printing "main". in the second case, at least 5 threads shows up.
pps: I've tried 2000 as a big number during this weird stuck a few minutes ago. now even with several larger num(even using Integer.MAX_VALUE). Still stucks.
It's very hard to construct a meaningful spliterator for a stream which is generated by iterate() with 3 arguments. For example, if you have a simple stream of integers from range from 1 to 1000 you can easily split this stream into two streams, one from 1 to 500 and another from 501 to 1000 and process them in parallel. But consider this example:
Stream.iterate(4,i->i<Integer.MAX_VALUE,i->i+ThreadLocalRandom.current().nextInteger())
How would you make this stream parallel? Which number should the second thread start from? There is no way to effectively split this stream into parts without calculating all its elements first.
When you add limit() to the stream you actually buffer the results of the previous stream, so limit() can provide a non-trivial spliterator for it's internal buffer. However, neither of the streams are guaranteed to do something special in parallel streams, parallel() is just a "recommendation".
Can someone tell me why this is happening and if it's expected behaviour or a bug
List<Integer> a = Arrays.asList(1,1,3,3);
a.parallelStream().filter(Objects::nonNull)
.filter(value -> value > 2)
.reduce(1,Integer::sum)
Answer: 10
But if we use stream instead of parallelStream I'm getting the right & expected answer 7
The first argument to reduce is called "identity" and not "initialValue".
1 is no identity according to addition. 1 is identity for multiplication.
Though you need to provide 0 if you want to sum the elements.
Java uses "identity" instead of "initialValue" because this little trick allows to parallelize reduce easily.
In parallel execution, each thread will run the reduce on a part of the stream, and when the threads are done, they will be combined using the very same reduce function.
Though it will look something like this:
mainThread:
start thread1;
start thread2;
wait till both are finished;
thread1:
return sum(1, 3); // your reduce function applied to a part of the stream
thread2:
return sum(1, 3);
// when thread1 and thread2 are finished:
mainThread:
return sum(sum(1, resultOfThread1), sum(1, resultOfThread2));
= sum(sum(1, 4), sum(1, 4))
= sum(5, 5)
= 10
I hope you can see, what happens and why the result is not what you expected.
I have defined the following instance variable:
private final AtomicInteger tradeCounter = new AtomicInteger(0);
I have a method called onTrade defined as below being called by 6 threads:
public void onTrade(Trade trade) {
System.out.println(tradeCounter.incrementAndGet());
}
Why is the output:
2
5
4
3
1
6
Instead of
1
2
3
4
5
6
?
I want to avoid using synchronization.
You can think of
tradeCounter.incrementAndGet();
And
System.out.println();
as two separate statements.
So here
System.out.println(tradeCounter.incrementAndGet());
there are basically two statements, and those statements together are not atomical.
Imagine such example with 2 threads :
Thread 1 invokes tradeCounter.incrementAndGet()
Thread 2 invokes tradeCounter.incrementAndGet()
Thread 2 prints value 2
Thread 1 prints value 1
It all depends in what order threads will invoke inctructions in your method.
I have a method called onTrade defined as below being called by 6
threads:
public void onTrade(Trade trade) {
System.out.println(tradeCounter.incrementAndGet());
}
Why is the output:
2 5 4 3 1 6
Instead of 1 2 3 4 5 6 ?
Why shouldn't that be the output? Or why not 3 1 4 6 5 2? Or any of the other permutations of 1 2 3 4 5 6?
Using an AtomicInteger and its incrementAndGet() method ensures that each each thread gets a different value, and that the six values obtained are sequential, without synchronization. But that has nothing to do with the order in which the resulting values are printed afterward.
If you want the results to be printed in the same order that they are obtained, then synchronization is the easiest way to go. In that case, using an AtomicInteger does not gain you anything over using a plain int (for this particular purpose):
int tradeCounter = 0;
synchronized public void onTrade(Trade trade) {
System.out.println(++tradeCounter);
}
Alternatively, don't worry about the order in which the output is printed. Consider: is the output order actually meaningful for any reason?
incrementAndGet() increments in the expected order : 1, 2, 3 etc...
But system.out doesn't invoke the println() in an atomic way along incrementAndGet(). So the random ordering is expected.
I want to avoid using synchronization.
You could not in this case.
System.out.println(...) and tradeCounter.incrementAndGet() are two separate operations and most likely when thread-i gets new value, some other threads can get value and print it before thread-i prints it. There is no way to avoid synchronization (direct or indirect) here.
Having a tough time with the below code, I have answered the first call correctly as the condition is immediately correct. However, the second call of 4 is causing me great confusion, I came to the answer 2, 1 however it is incorrect- I am clearly getting things mixed up. Can someone explain to me exactly why my answer is wrong and the correct breakdown of the process of Recursive tracing in this example. I do understand elements of recursion however I am having issues following the process of tracing.
public void mystery1(int n) {
if (n <= 1) {
System.out.print(n);
} else {
mystery1(n / 2);
System.out.print(", " + n);
}
}
mystery1(1);
mystery1(4);
mystery1(16);
Recursion is beautiful yet very powerful and when it comes to observe it, people get tricked.
Let me explain you how i learnt it when i was preparing for my GATE(Graduate Aptitude Test in Engineering).
Think of each Function call as 2 parts:
Printing
Calling with halving itself
Now all the operations will be stacked upon the older one until unless we are out of recursion and only Printing is left in the function cycle.
We will print out in stack format i.e FIFO
Example, if we have these elements in stack:
Top:
4
3
7
2
Bottom
It will print 4 3 7 2 .
Now taking your Question:
Lets break your Conditional statements in two halves, 1st will be if condition with Print 2nd will be else with its print.
Note: Only one part i.e 1st will print 1 and that too without commas (,).
I have attached the Image below kindly refer it, think as print statement as on stack, and the lowest print statement will be executed first as we will end out of function calls.
Now combining mystery1(1),mystery1(4),mystery1(16) the final output will be: 1
1
, 2
, 4
1
, 2
, 4
, 8
, 16
PS: If you are going to give interviews of Amazon , Google or any Reputed Product based company they will ask question with flavor of recursion and that too without using any compiler, they wanna test your concepts via programming.
When you call mystery1(4), it goes to the else part and calls mystery1(2) since n>1. The print statement will be executed only after mystery1(2) has finished executing.
The same thing happens for mystery1(2) and it calls mystery1(1). It waits for mystery(1) to finish execution.
When mystery1(1) is called, it prints "1".
Then, mystery1(2) will continue and print ",2".
Finally, mystery1(4) will continue and print ",4".
So your output will be 1,2,4
In your method:
public static void mystery1(int n) {
if (n <= 1) {
System.out.print(n);
} else {
mystery1(n / 2);
System.out.print(", " + n);
}
}
try to replace the position of System.out.print(", " + n); before mystery1(n / 2); call.
In the first case, when System.out after mystery1 call, you have result as 1, 2, 4, because, first you have to get result, then print it.
In the second case, when System.out before mystery1 call, you have result as , 4, 21, because, first you print the result, then calculate the next result in function.