Develop Reference

CompletableFuture: Await percentage complete

I am writing identical data in parallel to n nodes of a distributed system.
When n% of these nodes have been written to successfully, the remaining writes to the other nodes are unimportant as n% guarantees replication between the other nodes.
Java's CompletableFuture seems to have very close to what I want eg:
CompletableFuture.anyOf()
(Returns when the first future is complete) - avoids waiting unnecessarily, but returns too soon as I require n% completions
CompletableFuture.allOf()
(Returns when all futures complete) - avoids returning too soon but waits unnecessarily for 100% completion
I am looking for a way to return when a specific percentage of futures have completed.
For example if I supply 10 futures, return when 6 or 60% of these complete successfully.
For example, Bluebird JS has this feature with
Promise.some(promises, countThatNeedToComplete)
I was wondering if I could do something similar with TheadExecutor or vanilla CompletableFuture in java

I believe you can achieve what you want using only what's already provided by CompletableFuture, but you'll have to implement additional control to know how many future tasks were already completed and, when you reach the number/percentage that you need, cancel the remaining tasks.
Below is a class to illustrate the idea:
public class CompletableSome<T>
{
private List<CompletableFuture<Void>> tasks;
private int tasksCompleted = 0;
public CompletableSome(List<CompletableFuture<T>> tasks, int percentOfTasksThatMustComplete)
{
int minTasksThatMustComplete = tasks.size() * percentOfTasksThatMustComplete / 100;
System.out.println(
String.format("Need to complete at least %s%% of the %s tasks provided, which means %s tasks.",
percentOfTasksThatMustComplete, tasks.size(), minTasksThatMustComplete));
this.tasks = new ArrayList<>(tasks.size());
for (CompletableFuture<?> task : tasks)
{
this.tasks.add(task.thenAccept(a -> {
// thenAccept will be called right after the future task is completed. At this point we'll
// check if we reached the minimum number of nodes needed. If we did, then complete the
// remaining tasks since they are no longer needed.
tasksCompleted++;
if (tasksCompleted >= minTasksThatMustComplete)
{
tasks.forEach(t -> t.complete(null));
}
}));
}
}
public void execute()
{
CompletableFuture.allOf(tasks.toArray(new CompletableFuture<?>[0])).join();
}
}
You would use this class as in the example below:
public static void main(String[] args)
{
int numberOfNodes = 4;
// Create one future task for each node.
List<CompletableFuture<String>> nodes = new ArrayList<>();
for (int i = 1; i <= numberOfNodes; i++)
{
String nodeId = "result" + i;
nodes.add(CompletableFuture.supplyAsync(() -> {
try
{
// Sleep for some time to avoid all tasks to complete before the count is checked.
Thread.sleep(100 + new Random().nextInt(500));
}
catch (InterruptedException e)
{
e.printStackTrace();
}
// The action here is just to print the nodeId, you would make the actual call here.
System.out.println(nodeId + " completed.");
return nodeId;
}));
}
// Here we're saying that just 75% of the nodes must be called successfully.
CompletableSome<String> tasks = new CompletableSome<>(nodes, 75);
tasks.execute();
}
Please note that with this solution you could end up executing more tasks than the minimum required -- for instance, when two or more nodes respond almost simultaneously, you may reach the minimum required count when the first node responds, but there will be no time to cancel the other tasks. If that's an issue, then you'd have to implement even more controls.

Best way to use Future in java

I have a set of jobs which I am submitting using executor framework and Future. Let's say that I have 100 futures. As of now, I am using Future.get and using the output for subsequent processing. However for further tuning, I want to change the flow as below:
iterate through the set of future tasks and start consuming the result as soon as a future task is complete. I am reading the API doc to understand what might be a good way to accomplish this but reaching out to see if there is a better way to accomplish what I am looking for.
Here is the sample code:
public class ImplCallable implements Callable<String> {
int timeOut;
ImplCallable(int timeOut) {
this.timeOut=timeOut;
}
public String call() throws Exception {
Thread.sleep(timeOut);
return Thread.currentThread().getName();
}
}
and the main class:
public class MainProg {
public static void main(String...args) throws Exception {
long startTimeInMillis = System.currentTimeMillis();
ImplCallable callable1 = new ImplCallable(1000);
ImplCallable callable2 = new ImplCallable(2000);
ExecutorService service = Executors.newFixedThreadPool(4);
Future<String> task1 = service.submit(callable1);
Future<String> task2 = service.submit(callable2);
List<Future<String>> futureList = new ArrayList();
futureList.add(task1);
futureList.add(task2);
String retVal;
for(Future<String> task:futureList) {
retVal = task.get();
//do something with the retVal
}
long endTimeInMillis = System.currentTimeMillis();
System.out.println("time taken by code - " + (endTimeInMillis-startTimeInMillis) + "-ms");
}
}
Basically I don't want to use Future.get() and wait for its completion. I want to know if either of the task is complete and use the result as soon as its done.

There are many ways do this so, without a concrete example, you won't get a concrete answer. Likely want to look at CompletableFuture which has many methods for defining follow-on work, combining work, splitting work etc.
Future<String> f = CompletableFuture.supplyAsync(() -> "INITIAL WORK")
.thenApply(String::toLowerCase) // Do some more work
.thenAccept(queue::add); // put results onto a queue something is reading from
f.join();
// Batch complete

I hope you are using Java 8 or later version.
Whenever you mention "as soon as a future task is complete", you want to use CompletableFuture and its .thenApply() method, as #drekbour suggests.
Then you have multiple threads running different tasks in non-determenistic sequence. But at the end you want to get all the results in the single (Main) thread. To achieve it, you can use CompletableFuture.allOf method, .join() it - and then iterate over all the (already completed) future results without waiting.

Java8 - idiomatic way to process a Stream<Callable<...>> in parallel delivering to a non-thread-safe consumer?

Suppose I have a Stream<Callable<SomeClass>> stream;. The stream is accessing over a million objects which will not fit in memory.
What is the idiomatic way to convert this to a Stream<SomeClass> in a manner that ensures the Callable::call are executed in parallel before being delivered to a consumer that is non-threaded-safe (perhaps by calling .sequential().forEach() or some other bottlenecking mechanism)?
i.e. Process the stream in parallel but deliver the output sequentially (random order ok, as long as it's single-threaded).
I know I could do what I want by setting up an ExecutionService and a Queue between the original stream and the consumer. But that seems like a lot of code, is there a magic one-liner?

You could still employ an ExecutorService for parallelization. Like this:
ExecutorService service = Executors.newFixedThreadPool(4);
stream.map(c -> service.submit(c)).map(future -> {
try {
return future.get(); //retrieve callable result
} catch (InterruptedException | ExecutionException ex) {
//Exception handling
throw new RuntimeException(ex);
}
});
You can process the resulting Stream<SomeClass> further sequentially.
If you use forEach/forEachOrdered directly on the Stream<Future<SomeClass>> you can process a resulting SomeClass-object directly once the current future is done (different from when you use invokeAll() which blocks until every task is done).
If you want to process the results of the callables in the exact order they are available you will have to use CompletionService which can't be used along with a single chain of stream operations due to the necessary call of Future<SomeClass> f = completionService.take() after submitting the callables.
EDIT:
Using an ExecutorService within streams doesn't work the way I showed above, because every Callable is submitted and requested via future.get() one after the other.
I found a possible even side-effect heavier solution dividing the Callables in fixed parallelized chunks.
I use a class TaskMapper as mapping-function for submitting the Callables and mapping them to chunks:
class TaskMapper implements Function<Callable<Integer>, List<Future<Integer>>>{
private final ExecutorService service;
private final int chunkSize;
private List<Future<Integer>> chunk = new ArrayList<>();
TaskMapper(ExecutorService service, int chunkSize){
this.service = service;
this.chunkSize = chunkSize;
}
#Override
public List<Future<Integer>> apply(Callable<Integer> c) {
chunk.add(service.submit(c));
if(chunk.size() == chunkSize){
List<Future<Integer>> fList = chunk;
chunk = new ArrayList<>();
return fList;
}else{
return null;
}
}
List<Future<Integer>> getChunk(){
return chunk;
}
}
This how the chain of stream-operations looks like:
ExecutorService service = Executors.newFixedThreadPool(4);
TaskMapper taskMapper = new TaskMapper(service, 4);
stream.map(taskMapper)
.filter(fl -> fl != null) //filter for the chunks
.flatMap(fl -> fl.stream()) //flat-map the chunks to futures
.map(future -> {
try {
return future.get();
} catch (InterruptedException | ExecutionException ex) {
throw new RuntimeException(ex);
}
});
//process the remaining futures
for(Future<Integer> f : taskMapper.getChunk()){
try {
Integer i = f.get();
//process i
} catch (InterruptedException | ExecutionException ex) {
//exception handling
}
}
This works as follows: The TaskMapper takes 4 callables each time submits them to the service and maps them to a chunk of futures (without Spliterator). This is solved by mapping to null for the 1st, 2nd and 3rd callable each time. null could be replaced by a dummy object for example. The mapping function that maps the futures to the results waits for the result of each future of the chunk. I use Integer in my example instead of SomeClass. When all results of the futures in the current chunk are mapped, a new chunk will be created and parallelized. Finally, if the number of elements in the stream is not dividable by the chunkSize(4 in my example), the remaining futures will have to be retrieved from the TaskMapper and processed outside of the stream.
This construct works for the tests I carried out, but I am aware that it is possible fragile due to the side-effects, statefullness and the undefined evaluation behavior of the stream.
EDIT2:
I made a version of the construct from the previous EDIT using a custom Spliterator:
public class ExecutorServiceSpliterator<T> extends AbstractSpliterator<Future<T>>{
private final Spliterator<? extends Callable<T>> srcSpliterator;
private final ExecutorService service;
private final int chunkSize;
private final Queue<Future<T>> futures = new LinkedList<>();
private ExecutorServiceSpliterator(Spliterator<? extends Callable<T>> srcSpliterator) {
this(srcSpliterator, Executors.newFixedThreadPool(8), 30); //default
}
private ExecutorServiceSpliterator(Spliterator<? extends Callable<T>> srcSpliterator, ExecutorService service, int chunkSize) {
super(Long.MAX_VALUE, srcSpliterator.characteristics() & ~SIZED & ~CONCURRENT);
this.srcSpliterator = srcSpliterator;
this.service = service;
this.chunkSize = chunkSize;
}
public static <T> Stream<T> pipeParallelized(Stream<? extends Callable<T>> srcStream){
return getStream(new ExecutorServiceSpliterator<>(srcStream.spliterator()));
}
public static <T> Stream<T> pipeParallelized(Stream<? extends Callable<T>> srcStream, ExecutorService service, int chunkSize){
return getStream(new ExecutorServiceSpliterator<>(srcStream.spliterator(), service, chunkSize));
}
private static <T> Stream<T> getStream(ExecutorServiceSpliterator<T> serviceSpliterator){
return StreamSupport.stream(serviceSpliterator, false)
.map(future -> {
try {
return future.get();
} catch (InterruptedException | ExecutionException ex) {
throw new RuntimeException(ex);
}
}
);
}
#Override
public boolean tryAdvance(Consumer<? super Future<T>> action) {
boolean didAdvance = true;
while((didAdvance = srcSpliterator.tryAdvance(c -> futures.add(service.submit(c))))
&& futures.size() < chunkSize);
if(!didAdvance){
service.shutdown();
}
if(!futures.isEmpty()){
Future<T> future = futures.remove();
action.accept(future);
return true;
}
return false;
}
}
This class provides functions (pipeParallelized()) which take a stream of Callable-elements execute them chunk-wise in parallel and then ouput a sequential stream containing the results. Spliterators are allowed to be stateful. Therefore this version should hopefully not violate any stream operation constraints. This is how the Splitterator can be used (close to a "magic oneliner"):
ExecutorServiceSpliterator.pipeParallelized(stream);
This line takes the stream of Callables stream parallelizes the execution of them and returns a sequential stream containing the results (piping happens lazily -> should work with millions of callables) which can be processed further with regular stream operations.
The implementation of ExecutorServiceSpliteratoris very basic. It should mainly demonstrate how it could be done in principle. The resupplying of the service and the retrieving of the results could be optimized. For example if the resulting stream is allowed to be unordered, a CompletionService could be used.

You are asking for an idiomatic solution. Streams with sideeffects in its behavioral parameters are discouraged (explicitly stated in the javadoc of Stream).
So the idiomatic solution is basically ExecutorService + Futures and some loops/forEach(). If you have a Stream as parameter, just transform it to a List with the standard Collector.
Something like that:
ExecutorService service = Executors.newFixedThreadPool(5);
service.invokeAll(callables).forEach( doSomething );
// or just
return service.invokeAll(callables);

First Example:
ExecutorService executor = Executors.newWorkStealingPool();
List<Callable<String>> callables = Arrays.asList(
() -> "job1",
() -> "job2",
() -> "job3");
executor.invokeAll(callables).stream().map(future -> {
return future.get();
}).forEach(System.out::println);
Second Example:
Stream.of("1", "2", "3", "4", "", "5")
.filter(s->s.length() > 0)
.parallel()
.forEachOrdered(System.out::println);

public static void main(String[] args) {
testInfititeCallableStream();
}
private static void testInfititeCallableStream() {
ExecutorService service = Executors.newFixedThreadPool(100);
Consumer<Future<String>> consumeResult = (Future<String> future)->{
try {
System.out.println(future.get());
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
};
getCallableStream().parallel().map(callable -> service.submit(callable)).forEach(consumeResult);
}
private static Stream<Callable<String>> getCallableStream() {
Random randomWait = new Random();
return Stream.<Callable<String>>generate(() ->
new Callable<String>() {
public String call() throws Exception {
//wait for testing
long time = System.currentTimeMillis();
TimeUnit.MILLISECONDS.sleep(randomWait.nextInt(5000));
return time + ":" +UUID.randomUUID().toString();
};
}).limit(Integer.MAX_VALUE);
}

None of the other answers worked for me.
I finally settled on something like this (pseudo-code):
ExecutorService executor = Executors.newWorkStealingPool();
CompletionService completor = new CompletionService(executor);
int count = stream.map(completor::submit).count();
while(count-- > 0) {
SomeClass obj = completor.take();
consume(obj);
}
The consume(obj) loop is executed sequentially in a single thread while the individual callable tasks asynchronously work their way through the CompletionService's multiple threads. Memory consumption is limited as the CompletionService will have only as many items in progress at a time as there are threads available. The Callables waiting for execution are eagerly materialized from the stream, but the impact of that is negligible compared to the memory each consumes once it starts executing (your use-case may vary).

How to calculate run-time for a multi-threaded program?

I am trying to test the performance (in terms of execution time) for my webcrawler but I am having trouble timing it due to multi-threading taking place.
My main class:
class WebCrawlerTest {
//methods and variables etc
WebCrawlerTest(List<String> websites){
//
}
if(!started){
startTime = System.currentTimeMillis();
executor = Executors.newFixedThreadPool(32); //this is the value I'm tweaking
started=true;
}
for(String site : websites){
executor.submit(webProcessor = new AllWebsiteProcessorTest(site, deepSearch));
}
executor.shutdown();
//tried grabbing end time here with no luck
AllWebsiteProcessorTest class:
class AllWebsiteProcessorTest implements Runnable{
//methods and var etc
AllWebsiteProcessorTest(String site, boolean deepSearch) {
}
public void run() {
scanSingleWebsite(websites);
for(String email:emails){
System.out.print(email + ", ");
}
private void scanSingleWebsite(String website){
try {
String url = website;
Document document = Jsoup.connect(url).get();
grabEmails(document.toString());
}catch (Exception e) {}
With another class (with a main method), I create an instance of WebCrawlerTest and then pass in an array of websites. The crawler works fine but I can't seem to figure out how to time it.
I can get the start time (System.getCurrentTime...();), but the problem is the end time. I've tried adding the end time like this:
//another class
public static void main(.....){
long start = getCurrent....();
WebCrawlerTest w = new WebCrawlerTest(listOfSites, true);
long end = getCurrent....();
}
Which doesn't work. I also tried adding the end after executor.shutdown(), which again doesn't work (instantly triggered). How do I grab the time for the final completed thread?

After shutting down your executors pool
executor.shutdown();
//tried grabbing end time here with no luck
You can simply
executor.awaitTermination(TimeUnit, value)
This call will block untill all tasks are completed. Take the time, subtract T0 from it and voila, we have execution time.
shutdown() method just assures that no new tasks will be accepted into excution queue. Tasks already in the queue will be performed (shutdownNow() drops pending tasks). To wait for all currently running tasks to complete, you have to awaitTermination().

Running time of a job sent to a ExecutorService

Good day,
I am writing a program where a method is called for each line read from a text file. As each call of this method is independent of any other line read I can call them on parallel. To maximize cpu usage I use a ExecutorService where I submit each run() call. As the text file has 15 million lines, I need to stagger the ExecutorService run to not create too many jobs at once (OutOfMemory exception). I also want to keep track of the time each submitted run has been running as I have seen that some are not finishing. The problem is that when I try to use the Future.get method with timeout, the timeout refers to the time since it got into the queue of the ExecutorService, not since it started running, if it even started. I would like to get the time since it started running, not since it got into the queue.
The code looks like this:
ExecutorService executorService= Executors.newFixedThreadPool(ncpu);
line = reader.readLine();
long start = System.currentTimeMillis();
HashMap<MyFut,String> runs = new HashMap<MyFut, String>();
HashMap<Future, MyFut> tasks = new HashMap<Future, MyFut>();
while ( (line = reader.readLine()) != null ) {
String s = line.split("\t")[1];
final String m = line.split("\t")[0];
MyFut f = new MyFut(s, m);
tasks.put(executorService.submit(f), f);
runs.put(f, line);
while (tasks.size()>ncpu*100){
try {
Thread.sleep(100);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Iterator<Future> i = tasks.keySet().iterator();
while(i.hasNext()){
Future task = i.next();
if (task.isDone()){
i.remove();
} else {
MyFut fut = tasks.get(task);
if (fut.elapsed()>10000){
System.out.println(line);
task.cancel(true);
i.remove();
}
}
}
}
}
private static class MyFut implements Runnable{
private long start;
String copy;
String id2;
public MyFut(String m, String id){
super();
copy=m;
id2 = id;
}
public long elapsed(){
return System.currentTimeMillis()-start;
}
#Override
public void run() {
start = System.currentTimeMillis();
do something...
}
}
As you can see I try to keep track of how many jobs I have sent and if a threshold is passed I wait a bit until some have finished. I also try to check if any of the jobs is taking too long to cancel it, keeping in mind which failed, and continue execution. This is not working as I hoped. 10 seconds execution for one task is much more than needed (I get 1000 lines done in 70 to 130s depending on machine and number of cpu).
What am I doing wrong? Shouldn't the run method in my Runnable class be called only when some Thread in the ExecutorService is free and starts working on it? I get a lot of results that take more than 10 seconds. Is there a better way to achieve what I am trying?
Thanks.

If you are using Future, I would recommend change Runnable to Callable and return total time in execution of thread as result. Below is sample code:
import java.util.concurrent.Callable;
public class MyFut implements Callable<Long> {
String copy;
String id2;
public MyFut(String m, String id) {
super();
copy = m;
id2 = id;
}
#Override
public Long call() throws Exception {
long start = System.currentTimeMillis();
//do something...
long end = System.currentTimeMillis();
return (end - start);
}
}

You are making your work harder as it should be. Java’s framework provides everything you want, you only have to use it.
Limiting the number of pending work items works by using a bounded queue, but the ExecutorService returned by Executors.newFixedThreadPool() uses an unbound queue. The policy to wait once the bounded queue is full can be implemented via a RejectedExecutionHandler. The entire thing looks like this:
static class WaitingRejectionHandler implements RejectedExecutionHandler {
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
try {
executor.getQueue().put(r);// block until capacity available
} catch(InterruptedException ex) {
throw new RejectedExecutionException(ex);
}
}
}
public static void main(String[] args)
{
final int nCPU=Runtime.getRuntime().availableProcessors();
final int maxPendingJobs=100;
ExecutorService executorService=new ThreadPoolExecutor(nCPU, nCPU, 1, TimeUnit.MINUTES,
new ArrayBlockingQueue<Runnable>(maxPendingJobs), new WaitingRejectionHandler());
// start flooding the `executorService` with jobs here
That’s all.
Measuring the elapsed time within a job is quite easy as it has nothing to do with multi-threading:
long startTime=System.nanoTime();
// do your work here
long elpasedTimeSoFar = System.nanoTime()-startTime;
But maybe you don’t need it anymore once you are using the bounded queue.
By the way the Future.get method with timeout does not refer to the time since it got into the queue of the ExecutorService, it refers to the time of invoking the get method itself. In other words, it tells how long the get method is allowed to wait, nothing more.