I've a map of key-value and iterating over keys, and calling service and based on the response, I am adding all the response to some uberList
How can I execute the different operations concurrently? Will changing stream() to parallelStream() do the trick? Does it synchronize when it adds to uberList?
The idea is to minimize the response time.
List<MyClass> uberList = new LinkedList<>();
Map<String, List<MyOtherClass>> map = new HashMap();
//Populate map
map.entrySet().stream().filter(s -> s.getValue().size() > 0 && s.getValue().values().size() > 0).forEach(
y -> {
// Do stuff
if(noError) {
uberList.add(MyClass3);
}
}
}
//Do stuff on uberList
How can I execute the different operations concurrently?
One thread can do one task at a time. If you want to do multiple operations concurrently, you have to offwork to other threads.
You can either creating new Thread or using ExecutorService to manage thread pool, queue the task and execute task for you.
Will changing stream() to parallelStream() do the trick?
Yes it does. Internally, parallelStream() use the ForkJoinPool.commonPool() to run tasks for you. But keep in mind that the parallelStream() has no guarantee about if the returned stream is paralleled (but for now, the current implementation return a paralleled one)
Does it synchronize when it adds to uberList?
It's up to you to do the synchronization part in forEach pipeline. Normally you do not want to call collection.add() inside forEach to create collection. Instead you should use .map().collect(toX()) methods. It frees you from synchronizatin part:
It does not required to know about your local variable (in this case uberlist. And it will not modify it on execution, help reduce a lot of strange bugs caused of concurrency
You can freely change the type of collection in .collect() part. It give you more control over the result type.
It does not require thread-safe or synchronization on given collection when using with parallel stream. Because "multiple intermediate results may be instantiated, populated, and merged so as to maintain isolation of mutable data structures" (Read more about this here)
So what you want is to execute multiple similar service call at the same time and collect your result into a list.
You can do it simply by parallel stream:
uberList = map.entrySet().stream()
.parallel() // Use .stream().parallel() to force parallism. The .parallelStream() does not guarantee that the returned stream is parallel stream
.filter(yourCondition)
.map(e -> yourService.methodCall(e))
.collect(Collectors.toList());
Pretty cool, isn't it?
But as I stated, the default parallel stream use ForkJoinPool.commonPool() for thread queueing and executing.
The bad part is if your yourService.methodCall(e) do heavy IO stuff (like HTTP call, even db call...) or long running task then it may exhaust the pool, other incoming tasks will queued forever to wait for execution.
So typically all other tasks depend on this common pool (not only your own yourService.methodCall(e), but all other parallel stream) will be slow down due to queueing time.
To solve this problem, you can force execute parallelism on your own fork-join pool:
ForkJoinPool forkJoinPool = new ForkJoinPool(4); // Typically set it to Runtime.availableProcessors()
uberlist = forkJoinPool.submit(() -> {
return map.entrySet().stream()
.parallel() // Use .stream().parallel() to force parallism. The .parallelStream() does not guarantee that the returned stream is parallel stream
.filter(yourCondition)
.map(e -> yourService.methodCall(e))
.collect(Collectors.toList());
}).get();
You probably don't want to use parallelStream for concurrency, only for parallelism. (That is: use it for tasks where you want to use multiple physical processes efficiently on a task that's conceptually sequential, not for tasks where you want multiple things going on at the same time conceptually.)
In your case you would probably be better off using an ExecutorService, or more specifically com.google.common.util.concurrent.ListenableExecutorService from Google Guava (warning: I haven't tried to compile the below code, there may be syntax errors):
int MAX_NUMBER_OF_SIMULTANEOUS_REQUESTS = 100;
ListeningExecutorService myExecutor =
MoreExecutors.listeningDecorator(
Executors.newFixedThreadPool(MAX_NUMBER_OF_SIMULTANEOUS_REQUESTS));
List<ListenableFuture<Optional<MyClass>>> futures = new ArrayList<>();
for (Map.Entry<String, List<MyOtherClass>> entry : map.entrySet()) {
if (entry.getValue().size() > 0 && entry.getValue().values().size() > 0) {
futures.add(myExecutor.submit(() -> {
// Do stuff
if(noError) {
return Optional.of(MyClass3);
} else {
return Optional.empty();
}
}));
}
}
List<MyClass> uberList = Futures.successfulAsList(futures)
.get(1, TimeUnit.MINUTES /* adjust as necessary */)
.stream()
.filter(Optional::isPresent)
.map(Optional::get)
.collect(Collectors.toList());
The advantage of this code is that it allows you to explicitly specify that the tasks should all start at the "same time" (at least conceptually) and allows you to control your concurrency explicitly (how many simultaneous requests are allowed? What do we do if some of the tasks fail? How long are we willing to wait? etc). Parallel streams aren't really for that.
Parallel Stream will help in execution concurrently. But it is not recommended to do forEach loop and add element in outside list. If you do that, you have to make sure of synchnising external list. Better way of doing it is to use map and collect result into list. In this case, parallelStream takes care of synchronisation.
List<MyClass> uberList = map.entrySet().parallelStream().filter(s ->
s.getValue().size() > 0 && s.getValue().values().size() >
0).map(
y -> {
// Do stuff
return MyClass3;
}
}
.filter(t -> check no ertor condition)
.collect (Collectors.toList())
Related
Java version : 11
I have List of objects, I want to perform certain operations on them, where one operations depends on the result of another.
To achieve this workflow in Async fashion, I am using CompletableFuture.
Currently I am doing this by partitioning the List into sub lists and giving each of the sub list to CompletableFuture, so each thread in my thread-pool can start working on that sublist.
The code for above approach I used and working is :
List<SomeObject> someObjectList // original list
List<List<SomeObject>> partitionList = ListUtils.partition(someObjectList, partionSize);
partitionList.forEach(subList -> {
CompletableFuture.supplyAsync(() -> firstOperation(subList), executorService)
.thenAcceptAsync(firstOpresult -> secondOperationWithFirstOpResult(firstOpresult),executorService);
});
public static List<String> firstOperation(List<SomeObject> subList){
//perform operation
return List<String>;
}
public static void secondOperationWithFirstOpResult(List<String> firstOpProducedList) {
//perform operation
//print results.
}
Here the problem is partitioning the original list,
because if my original list has 100 thousand records and partion size
is say 100(which means I want 100 items in each sublist), I will have
1000 sublist objects each holding 100 elements, which may not be good considering this many objects in memory, moreover if the partion size is user controlled/config controller, a smaller partion size would result in huge no of sublist objects.
instead of partitioning the original list,
I want to take the original list.(say 100 elements)
Have a startIndex and endIndex on the original list(say 0 to 9, 10 to 19...)
And give each of those batches to a thread, in threadpool with CompletableFuture
So this thread can perform the two operations.
I know SO is not place for exact solution, but if you could nudge me in the right direction, a pseudo code or even if this is possible with CompletableFuture in the first place, would be great help :)
Since ListUtils.partition is not a standard method, it’s impossible to say, how it works. But if it works “the smart way”, it will use subList on the original list, instead of copying data.
If you want to be on the safe side, you can do the trivial partitioning yourself:
IntStream.range(0, (someObjectList.size() + partionSize - 1) / partionSize)
.forEach(p -> {
int start = p * partionSize,
end = Math.min(someObjectList.size(), start + partionSize);
List<SomeObject> subList = someObjectList.subList(start, end);
CompletableFuture.supplyAsync(() -> firstOperation(subList), executorService)
.thenAcceptAsync(r -> secondOperationWithFirstOpResult(r), executorService);
});
Since these sublists do not store the elements, they consume less memory than the CompletableFuture instances, for example. So this is nothing to worry about.
But if you can live with using the default thread pool¹ instead of executorService, you could use
IntStream.range(0, (someObjectList.size() + partionSize - 1) / partionSize)
.parallel()
.forEach(p -> {
int start = p * partionSize,
end = Math.min(someObjectList.size(), start + partionSize);
List<SomeObject> subList = someObjectList.subList(start, end);
secondOperationWithFirstOpResult(firstOperation(subList));
});
where each sublist only exists while being processed.
This will already use Fork/Join tasks under the hood. There is no need to implement those Fork/Join operations yourself.
¹ the default pool is unspecified, but will be ForkJoinPool.commonPool() in practice.
Consider the following code:
AtomicInteger counter1 = new AtomicInteger();
AtomicInteger counter2 = new AtomicInteger();
Flux<Object> source = Flux.generate(emitter -> {
emitter.next("item");
});
Executor executor1 = Executors.newFixedThreadPool(32);
Executor executor2 = Executors.newFixedThreadPool(32);
Flux<String> flux1 = Flux.merge(source).concatMap(item -> Mono.fromCallable(() -> {
Thread.sleep(1);
return "1_" + counter1.incrementAndGet();
}).subscribeOn(Schedulers.fromExecutor(executor1)));
Flux<String> flux2 = Flux.merge(source).concatMap(item -> Mono.fromCallable(() -> {
Thread.sleep(100);
return "2_" + counter2.incrementAndGet();
}).subscribeOn(Schedulers.fromExecutor(executor2)));
Flux.merge(flux1, flux2).subscribe(System.out::println);
You can see that one publisher is 100x faster than the other one. Still, when running the code it seems that all data is printed, but there's a huge gap between the two publishers which increases overtime.
What's interesting to note is that when changing the numbers so executer2 will have 1024 threads, and executer1 will have only 1 thread, then still we see a gap that is getting larger and larger overtime.
My expectation was that after tweaking the thread-pools accordingly the publishers will get balanced.
I'd like to achieve a balance between publishers (relative to the thread-pool sizes and processing time)
What would happen if I waited long enough? In other words, is a back-pressure can occur? (Which by default I guess it's a runtime exception, right?)
I don't want to drop items nor want to have a runtime exception. Instead, as I mentioned, I'd like the system to get balanced with respect to the resources it has and the processing times - Does the code above promise that?
Thanks!
Your Flux objects in this example are not ParallelFlux objects, so they'll only ever use one thread.
It doesn't matter if you create a scheduler that's capable of handling thousands of threads, and pass that to one of the Flux objects - they'll just sit there going unused, which is exactly what's happening in this example. There's no backpressure, and it won't result in an exception - it's just going as fast as it can using one thread.
If you want to make sure that the Flux takes full advantage of the 1024 threads available to it, then you need to call .parallel(1024):
ParallelFlux<String> flux1 = Flux.merge(source).parallel(1).concatMap(item -> Mono.fromCallable(() -> {
Thread.sleep(1);
return "1_" + counter1.incrementAndGet();
}).subscribeOn(Schedulers.fromExecutor(executor1)));
ParallelFlux<String> flux2 = Flux.merge(source).parallel(1024).concatMap(item -> Mono.fromCallable(() -> {
Thread.sleep(100);
return "2_" + counter2.incrementAndGet();
}).subscribeOn(Schedulers.fromExecutor(executor1)));
If you do that to your code, then you start to see results much closer to what you seem to be expecting, with 2_ sailing past 1_ despite the fact it's sleeping for 100 times as long:
...
2_17075
2_17076
1_863
1_864
2_17077
1_865
2_17078
2_17079
...
However, a word of warning:
I'd like to achieve a balance between publishers (relative to the thread-pool sizes and processing time)
You can't pick numbers here to balance the outputs, at least not reliably or in any meaningful way - the thread scheduling will be completely arbitrary. If you want to do that, then you could use this variant of the subscribe method, allowing you to explicitly call request() on the subscription consumer. This then allows you to provide backpressure by only requesting as many elements as you're prepared to deal with.
I have a legacy code which have dozen database calls to populate a report, it takes noticeable amount of time which I try to reduce using CompletableFuture.
I have some doubts that I do things correctly and not overuse this technology.
My code now looks like this:
Start asynchronous population of document sections with many database calls inside each methods
CompletableFuture section1Future = CompletableFuture.supplyAsync(() -> populateSection1(arguments));
CompletableFuture section2Future = CompletableFuture.supplyAsync(() -> populateSection2(arguments));
...
CompletableFuture section1oFuture = CompletableFuture.supplyAsync(() -> populateSection10(arguments));
Then I'm arranging futures in specific order in arrayList and joining all of them to make sure that my code will run further only when all futures are finished.
List<CompletableFuture> futures = Arrays.asList(
section1Future,
section2Future, ...
section10Future);
List<Object> futureResults = futures.stream()
.map(CompletableFuture::join)
.collect(Collectors.toList());
Then I'm populating PDF document itself with its pieces
Optional.ofNullable((PdfPTable) futureResults.get(0)).ifPresent(el -> populatePdfElement(document, el));
Optional.ofNullable((PdfPTable) futureResults.get(1)).ifPresent(el -> populatePdfElement(document, el));
...
Optional.ofNullable((PdfPTable) futureResults.get(10)).ifPresent(el -> populatePdfElement(document, el));
return document
My concerns are:
1) Is it okay to create and instantiate many Completable Futures in such way? Order them in required sequence in arrayList, join them to make sure that they are all finished, and then get result by casting them into specific object?
2) Is it okay to run without specifying an executor service but to rely on common ForkJoinPool? However this code runs in web container, so probably in order to use JTA I need to use container provided thread pool executor via JNDI?
3) If this code is surrounded in try-catch I should be able to catch CompletionException in main thread, right? Or In order to do that I should declare each features like following:
CompletableFuture.supplyAsync(() -> populateSection1(arguments))
.exceptionally (ex -> {
throw new RuntimeException(ex.getCause());
});
4) Is it possible to overuse CompletableFutures so they become a performance bottleneck itself? Like many futures waits one executor to start running? How to avoid that? Use container provided executor service?
If yes, could someone please point me to some best practice on how to correctly configure executor service taking to account processors and memory amount?
5) A memory impact. I read in parallel thread that there can be a problem with OOME as many object are created and garbage collected. Is there a best practice on how to calculate correct amount of memory required for application?
The approach is not wrong in general, but there are things to improve.
Most notably, you should not use raw types, like CompletableFuture.
When populateSection… returns a PdfPTable, you should use use CompletableFuture<PdfPTable> consistently throughout the code.
I.e.
CompletableFuture<PdfPTable> section1Future = CompletableFuture.supplyAsync(() -> populateSection1(arguments));
CompletableFuture<PdfPTable> section2Future = CompletableFuture.supplyAsync(() -> populateSection2(arguments));
...
CompletableFuture<PdfPTable> section10Future = CompletableFuture.supplyAsync(() -> populateSection10(arguments));
even if these methods do not declare the return type you are assuming to be always returned at runtime, you should insert the type cast at this early stage:
CompletableFuture<PdfPTable> section1Future = CompletableFuture.supplyAsync(() -> (PdfPTable)populateSection1(arguments));
CompletableFuture<PdfPTable> section2Future = CompletableFuture.supplyAsync(() -> (PdfPTable)populateSection2(arguments));
...
CompletableFuture<PdfPTable> section10Future = CompletableFuture.supplyAsync(() -> (PdfPTable)populateSection10(arguments));
Then, you can use
Stream.of(section1Future, section2Future, ..., section10Future)
.map(CompletableFuture::join)
.filter(Objects::nonNull)
.forEachOrdered(el -> populatePdfElement(document, el));
By not using raw types, you already get the desired result type and you can do the 3rd step’s operations, i.e. filtering and performing the final action, right in this stream operation.
If you still need the list, you may use
List<PdfPTable> results = Stream.of(section1Future, section2Future, ..., section10Future)
.map(CompletableFuture::join)
.filter(Objects::nonNull)
.collect(Collectors.toList());
results.forEach(el -> populatePdfElement(document, el));
That said, the parallelism depends on the thread pool used for the operation (specified to supplyAsync). When you don’t specify an executor, you get the default Fork/Join pool used by parallel streams, so in this specific case, you get the same result much simpler as
List<PdfPTable> results = Stream.<Supplier<PdfPTable>>.of(
() -> populateSection1(arguments),
() -> populateSection2(arguments));
...
() -> populateSection10(arguments)))
.parallel()
.map(Supplier::get)
.filter(Objects::nonNull)
.forEachOrdered(el -> populatePdfElement(document, el));
or
List<PdfPTable> results = Stream.<Supplier<PdfPTable>>.of(
() -> populateSection1(arguments),
() -> populateSection2(arguments));
...
() -> populateSection10(arguments)))
.parallel()
.map(Supplier::get)
.filter(Objects::nonNull)
.collect(Collectors.toList());
results.forEach(el -> populatePdfElement(document, el));
While both variants ensure that populatePdfElement will be called in the right order and one at a time, only the latter will perform all calls from the initiating thread.
Regarding exception handling, you’ll get any exception thrown by a supplier wrapped in a CompletionException when you call CompletableFuture::join. Chaining something like .exceptionally (ex -> { throw new RuntimeException(ex.getCause()); }); makes no sense, the new RuntimeException will also be wrapped in a CompletionException when you call CompletableFuture::join.
In the Stream variant, you’ll get the exception without a wrapper. Since Supplier does not allow checked exceptions, only subtypes of RuntimeException or Error are possible.
The other questions are too broad for the Q&A.
The code snippet below updates a not-thread-safe map (itemsById is not thread safe) from a parallel stream's forEach block.
// Update stuff in `itemsById` by iterating over all stuff in newItemsById:
newItemsById.entrySet()
.parallelStream()
.unordered()
.filter(...)
.forEach(entry -> {
itemsById.put(entry.getKey(), entry.getValue()); <-- look
});
To me, this looks like not-thread-safe, because the parallel stream will call the forEach block in many threads at the same time, and thus call itemsById.put(..) in many threads at the same time, and itemsById isn't thread safe. (However, with a ConcurrentMap the code would be safe I think)
I wrote to a colleague: "Please note that the map might allocate new memory when you insert new data. That's likely not thread safe, since the collection is not thread safe. -- Whether or not writing to different keys from many threads, is thread safe, is implementation dependent, I would think. It's nothing I would choose to rely on."
He however says that the above code is thread safe. -- Is it?
((Please note: I don't think this question is too localized. Actually now with Java 8 I think fairly many people will do something like: parallelStream()...foreach(...) and then it might be good know about thread safety issues, for many people))
You're right: this code is not thread-safe and depending on the Map implementation and race condition may produce any random effect: correct result, silent loss of data, some exception or endless loop. You may easily check it like this:
int equal = 0;
for(int i=0; i<100; i++) {
// create test input map like {0 -> 0, 1 -> 1, 2 -> 2, ...}
Map<Integer, Integer> input = IntStream.range(0, 200).boxed()
.collect(Collectors.toMap(x -> x, x -> x));
Map<Integer, Integer> result = new HashMap<>();
// write it into another HashMap in parallel way without key collisions
input.entrySet().parallelStream().unordered()
.forEach(entry -> result.put(entry.getKey(), entry.getValue()));
if(result.equals(input)) equal++;
}
System.out.println(equal);
On my machine this code usually prints something between 20 and 40 instead of 100. If I change HashMap to TreeMap, it usually fails with NullPointerException or becomes stuck in the infinite loop inside TreeMap implementation.
I'm no expert on streams but I assume there is no fancy synchronization employed here and thus I wouldn't consider adding elements to itemsById in parallel as threadsafe.
One of the things that could happen would be an endless loop since if both elements would happen to end up in the same bucket the underlying list might be messed up and elements could refer to each other in a cycle (A.next = B, B.next = A). A ConcurrentHashMap would prevent that by synchronizing write access on the bucket, i.e. unless the elements end up in the same bucket it would not block but if they do the add is sequential.
This code is not thread-safe.
Oracle docs state:
Operations like forEach and peek are designed for side effects; a
lambda expression that returns void, such as one that invokes
System.out.println, can do nothing but have side effects. Even so, you
should use the forEach and peek operations with care; if you use one
of these operations with a parallel stream, then the Java runtime may
invoke the lambda expression that you specified as its parameter
concurrently from multiple threads.
I've got a significant set of data, and want to call slow, but clean method and than call fast method with side effects on result of the first one. I'm not interested in intermediate results, so i would like not to collect them.
Obvious solution is to create parallel stream, make slow call , make stream sequential again, and make fast call. The problem is, ALL code executing in single thread, there is no actual parallelism.
Example code:
#Test
public void testParallelStream() throws ExecutionException, InterruptedException
{
ForkJoinPool forkJoinPool = new ForkJoinPool(Runtime.getRuntime().availableProcessors() * 2);
Set<String> threads = forkJoinPool.submit(()-> new Random().ints(100).boxed()
.parallel()
.map(this::slowOperation)
.sequential()
.map(Function.identity())//some fast operation, but must be in single thread
.collect(Collectors.toSet())
).get();
System.out.println(threads);
Assert.assertEquals(Runtime.getRuntime().availableProcessors() * 2, threads.size());
}
private String slowOperation(int value)
{
try
{
Thread.sleep(100);
}
catch (InterruptedException e)
{
e.printStackTrace();
}
return Thread.currentThread().getName();
}
If I remove sequential, code executing as expected, but, obviously, non-parallel operation would be call in multiple threads.
Could you recommend some references about such behavior, or maybe some way to avoid temporary collections?
Switching the stream from parallel() to sequential() worked in the initial Stream API design, but caused many problems and finally the implementation was changed, so it just turns the parallel flag on and off for the whole pipeline. The current documentation is indeed vague, but it was improved in Java-9:
The stream pipeline is executed sequentially or in parallel depending on the mode of the stream on which the terminal operation is invoked. The sequential or parallel mode of a stream can be determined with the BaseStream.isParallel() method, and the stream's mode can be modified with the BaseStream.sequential() and BaseStream.parallel() operations. The most recent sequential or parallel mode setting applies to the execution of the entire stream pipeline.
As for your problem, you can collect everything into intermediate List and start new sequential pipeline:
new Random().ints(100).boxed()
.parallel()
.map(this::slowOperation)
.collect(Collectors.toList())
// Start new stream here
.stream()
.map(Function.identity())//some fast operation, but must be in single thread
.collect(Collectors.toSet());
In the current implementation a Stream is either all parallel or all sequential. While the Javadoc isn't explicit about this and it could change in the future it does say this is possible.
S parallel()
Returns an equivalent stream that is parallel. May return itself, either because the stream was already parallel, or because the underlying stream state was modified to be parallel.
If you need the function to be single threaded, I suggest you use a Lock or synchronized block/method.