Develop Reference

Which threading mechanism to use for tasks that enqueue other tasks?

I'm using a task that creates other tasks. Those tasks in turn may or may not create subsequent tasks. I don't know beforehand how many tasks will be created in total. At some point, no more tasks will be created, and all the task will finish.
When the last task is done, I must do some extra stuff.
Which threading mechanism should be used? I've read about CountDownLatch, Cyclic Barrier and Phaser but none seem to fit.
I've also tried using ExecutorService, but I've encountered some issues such as the inability to execute something at the end, and you can see my attempt below:
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;
import static java.util.concurrent.TimeUnit.MILLISECONDS;
public class Issue {
public static void main(String[] args) throws InterruptedException {
var count = new AtomicInteger(1);
var executor = Executors.newFixedThreadPool(3);
class Task implements Runnable {
final int id = count.getAndIncrement();
#Override
public void run() {
try {
MILLISECONDS.sleep((long)(Math.random() * 1000L + 1000L));
} catch (InterruptedException e) {
// Do nothing
}
if (id < 5) {
executor.submit(new Task());
executor.submit(new Task());
}
System.out.println(id);
}
}
executor.execute(new Task());
executor.shutdown();
// executor.awaitTermination(20, TimeUnit.SECONDS);
System.out.println("Hello");
}
}
This outputs an exception because tasks are added after shutdown() is called, but the expected output would be akin to:
1
2
3
4
5
6
7
8
9
Hello
Which threading mechanism can help me do that?

It seems pretty tricky. If there is even a single task that's either in the queue or currently executing, then since you can't say whether or not it will spawn another task, you have no way to know how long it may run for. It may be the start of a chain of tasks that takes another 2 hours.
I think all the information you'd need to achieve this is encapsulated by the executor implementations. You need to know what's running and what's in the queue.
I think you're unfortunately looking at having to write your own executor. It needn't be complicated and it doesn't have to conform to the JDK's interfaces if you don't want it to. Just something that maintains a thread pool and a queue of tasks. Add the ability to attach listeners to the executor. When the queue is empty and there are no actively executing tasks then you can notify the listeners.
Here's a quick code sketch.
class MyExecutor
{
private final AtomicLong taskId = new AtomicLong();
private final Map<Long, Runnable> idToQueuedTask = new ConcurrentHashMap<>();
private final AtomicLong runningTasks = new AtomicLong();
private final ExecutorService delegate = Executors.newFixedThreadPool(3);
public void submit(Runnable task) {
long id = taskId.incrementAndGet();
final Runnable wrapped = () -> {
taskStarted(id);
try {
task.run();
}
finally {
taskEnded();
}
};
idToQueuedTask.put(id, wrapped);
delegate.submit(wrapped);
}
private void taskStarted(long id) {
idToQueuedTask.remove(id);
runningTasks.incrementAndGet();
}
private void taskEnded() {
final long numRunning = runningTasks.decrementAndGet();
if (numRunning == 0 && idToQueuedTask.isEmpty()) {
System.out.println("Done, time to notify listeners");
}
}
public static void main(String[] args) {
MyExecutor executor = new MyExecutor();
executor.submit(() -> {
System.out.println("Parent task");
try {
Thread.sleep(1000);
}
catch (Exception e) {}
executor.submit(() -> {
System.out.println("Child task");
});
});
}
}

If you change your ExecutorService to this:
ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(3);
You could then use the count functions to wait:
while(executor.getTaskCount() > executor.getCompletedTaskCount())
{
TimeUnit.SECONDS.sleep(10L);
}
executor.shutdown();
System.out.println("Hello");

Check for thread shutdown and execute a task once it shutdown

I am creating a fixed-size thread pool with 10 threads as follows:
ExecutorService executorService = Executors.newFixedThreadPool(10);
for (int i = 0; i < poolSize; ++i) {
executorService.execute(factory.get());
}
Now, after spawning 10 threads, a runnable is being executed by each of them. Runnable here is provided by factory.get()
Now, I wanted to do a thing that as soon as a thread gets shutdown, i.e. it has completed a task, it picks another runnable again and start running it. So, basically check for all 10 threads state and execute a runnable again if the thread is shutdown.
I know I can do something like, in which I can call the executorService shutdown() method as follows to do that:
while (!executorService.isShutdown()) {
try {
executorService.execute(factory.get());
} catch (RejectedExecutionException e) {
if (!executorService.isShutdown())
log.warn("task submission rejected");
}
}
But in this approach, issue is that I am keep on calling the execute function which would increase LinkedBlockingQueue and won't serve my purpose.
If I can somehow check the Threads state, that would be good because then, it would avoid the overhead of going back to get more work from the executor service.
Please suggest how can I do that in code.

You can use a bounded executor. The following is code by Brian Goetz from jcip
/**
* BoundedExecutor
* <p/>
* Using a Semaphore to throttle task submission
*
* #author Brian Goetz and Tim Peierls
*/
public class BoundedExecutor {
private final Executor exec;
private final Semaphore semaphore;
public BoundedExecutor(Executor exec, int bound) {
this.exec = exec;
this.semaphore = new Semaphore(bound);
}
public void submitTask(final Runnable command)
throws InterruptedException {
semaphore.acquire();
try {
exec.execute(new Runnable() {
public void run() {
try {
command.run();
} finally {
semaphore.release();
}
}
});
} catch (RejectedExecutionException e) {
semaphore.release();
}
}
}
Then just create and use it:
ExecutorService executorService = Executors.newFixedThreadPool(10);
BoundedExecutor boundedExecutor = new BoundedExecutor(executorService,10);
while (!executorService.isShutdown()) {
try {
boundedExecutor.submitTask(factory.get());
} catch (InterruptedException e) {
}
}
This way you will always have 10 threads running, new tasks will only be submitted after the old ones will complete and you can stop the execution by shutting down the executor.

Why use a thread pool if you're going to run 10 threads continually? Just start 10 threads yourself.
Why have a "controller" thread on top of the 10 worker threads? Just let each thread ask for the next task to execute.
The following assumes that factory.get() is thread-safe.
for (int i = 0; i < 10; i++) {
new Thread(() -> {
for (;;) {
Runnable r = factory.get(); // <-- must be thread-safe
if (r == null)
break; // stop thread if no more tasks to execute
try {
r.run();
} catch (Exception e) {
// log exception here
}
}
}).start();
}
If needed, save the 10 thread objects in an array.
If you want to be able to shutdown the threads without having factory.get() returning null, add a volatile boolean, and check it in the loop.

ArrayBlockingQueue NoSuchElementException

Just for learning, I have written the following code for custom thread pool referring and editing the code shown here.
As shown in the code I am using ArrayBlockingQueue for task queue.
Code:
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.TimeUnit;
public class ThreadPoolService {
private final BlockingQueue<Runnable> taskQueue;
private final int corePoolSize;
private ThreadPoolService(int corePoolSize) {
this.corePoolSize = corePoolSize;
this.taskQueue = new ArrayBlockingQueue<>(corePoolSize);
ThreadPool[] threadPool = new ThreadPool[corePoolSize];
for (int i = 0; i < corePoolSize; i++) {
threadPool[i] = new ThreadPool();
threadPool[i].start();
}
}
public static ThreadPoolService newFixedThreadPool(int size) {
return new ThreadPoolService(size);
}
public void execute(Runnable task) {
try {
taskQueue.offer(task, 10, TimeUnit.SECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private class ThreadPool extends Thread {
Runnable task;
#Override
public void run() {
while (true) {
try {
while (!taskQueue.isEmpty()) {
task = taskQueue.remove();
task.run();
}
} catch (RuntimeException ex) {
ex.printStackTrace();
}
}
}
}
public static void main(String[] args) {
ThreadPoolService pool = ThreadPoolService.newFixedThreadPool(10);
Runnable task1 = () -> {
System.out.println(" Wait for sometime: -> " + Thread.currentThread().getName());
try {
TimeUnit.SECONDS.sleep(2);
} catch (InterruptedException e) {
e.printStackTrace();
}
};
Runnable task2 = () -> System.out.println(" Do Task 2 -> " + Thread.currentThread().getName());
Runnable task3 = () -> System.out.println(" Do Task 3 -> " + Thread.currentThread().getName());
Runnable task4 = () -> System.out.println(" Do Task 4 -> " + Thread.currentThread().getName());
List<Runnable> taskList = new ArrayList<>();
taskList.add(task1);
taskList.add(task2);
taskList.add(task3);
taskList.add(task4);
for (Runnable task : taskList) {
pool.execute(task);
}
}
}
This code runs fine sometimes and sometimes gives an error.
Success output:
Do Task 2 -> Thread-2
Wait for sometime: -> Thread-8
Do Task 3 -> Thread-6
Do Task 4 -> Thread-7
Failure output:
Do Task 4 -> Thread-3
Do Task 3 -> Thread-6
Wait for sometime: -> Thread-4
Do Task 2 -> Thread-7
java.util.NoSuchElementException
at java.util.AbstractQueue.remove(AbstractQueue.java:117)
at com.interview.java.ThreadPoolService$ThreadPool.run(ThreadPoolService.java:43)
java.util.NoSuchElementException
at java.util.AbstractQueue.remove(AbstractQueue.java:117)
at com.interview.java.ThreadPoolService$ThreadPool.run(ThreadPoolService.java:43)
java.util.NoSuchElementException
at java.util.AbstractQueue.remove(AbstractQueue.java:117)
at com.interview.java.ThreadPoolService$ThreadPool.run(ThreadPoolService.java:43)
I see the reason for the error is the attempt to remove the element when the queue is empty. But it should not because i am doing queue empty check at line no 42 (while (!taskQueue.isEmpty())). What is wrong with code and also why it runs without error sometimes ?

Between the 'while' check and the actual removal, the queue could be modified by another Thread, possibly resulting in the error you mention. That's called a 'race condition'.
So in order to fix that you'll need a way to block the access to the queue by other threads, either by 'locking', using a 'synchronized' block with a shared lock object. Or simply by 'polling' instead of removing.

BlockingQueue is thread-safe only on an individual operation level.I'm seeing check-then-actoperation in the code which is a compound operation which is not thread-safe. To make this code thread-safe perform the check-then-act inside a synchronized block and lock on the queue itself.
synchronized(taskQueue) {
while (!taskQueue.isEmpty()) {
task = taskQueue.remove();
task.run();
}};
optimization: If the task is a time consuming one, you can execute it outside the synchronized block. So that other threads don't have to wait till the current task completes.

What is wrong with code?
You access taskQueue field from several threads without synchronization. You must do queue empty check and remove operation atomically, which can be done with synchronized keyword:
private class ThreadPool extends Thread {
#Override
public void run() {
Runnable task;
while (true) {
synchronized(queue) {
// give access to taskQueue to one thread at a time
if (!taskQueue.isEmpty()) {
task = taskQueue.remove();
}
}
try {
task.run();
} catch (RuntimeException ex) {
ex.printStackTrace();
}
}
}
}
why it runs without error sometimes?
Because of nature of JVM thread scheduler: sometimes it plans thread execution in such a way that they access taskQueue synchronously by themselves. But when you deal with multithreading, you can't rely on thread execution order, and must synchronize access to shared objects by yourself.

Simple queue and multi-threading

I have to process a lot of files. I wrote simple Java program that does the job, but it is too slow.
I need more than 1 working thread.
Im totally new with Java and Java multithreading.
Here is my code (simplified):
public static void main(String[] args)
{
// some queue here?
for (int i = 1; i < 8000000; i++)
{
processId(i);
}
}
public static void processId(int id)
{
try
{
// do work
System.out.println("Im working on: " + Integer.toString(id));
}
catch (Exception e)
{
// do something with errors
System.out.println("Error while working on: " + Integer.toString(id));
}
}
How can I add simple queue with 8 threads?

You should look into Executors.
You can create a thread pool of 8 threads using:
ExecutorService executor = Executors.newFixedThreadPool(8);
Then submit your tasks inside your loop the following way:
final int finalId = i; // final is necessary to be enclosed in lambda
executor.submit(() -> processId(finalId));
Or prior to java 8:
final int temp = i; // final is necessary to be enclosed in anonymous class
executor.submit(new Runnable() {
public void run() {
processId(finalId);
}
});
Don't forget to shutdown the thread pool when not needed anymore, as mentioned in the documentation. Here is an example from the doc:
private void shutdownAndAwaitTermination(ExecutorService pool) {
pool.shutdown(); // Disable new tasks from being submitted
try {
// Wait a while for existing tasks to terminate
if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
pool.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if (!pool.awaitTermination(60, TimeUnit.SECONDS))
System.err.println("Pool did not terminate");
}
} catch (InterruptedException ie) {
// (Re-)Cancel if current thread also interrupted
pool.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}
}

You should look into ExecutorService. This will make multithreading easy. An example:
Main code:
ExecutorService pool = Executors.newFixedThreadPool(8);
for (int i = 1; i < 8000000; i++) {
pool.submit(new intProcessingTask(i));
}
pool.shutdown();
pool.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
// all tasks have now finished (unless an exception is thrown above)
intProcessingTask code:
private static class DownloadTask implements Runnable {
private int id;
public DownloadTask(int id) {
this.id = id;
}
#Override
public void run() {
System.out.println("Im working on: " + Integer.toString(id));
}
}
This is slightly longer than the the other answer, but does pretty much the same thing, and works on Java 7 and earlier.

There are many ways in Java for processing mulithreading. Base on your question that you need a queue, I think the most simple version is use Java ExecutorService. You can see this code:
public static void main(String[] args) {
// creating a thread pool with maximum thread will be 8
ExecutorService executorService = Executors.newFixedThreadPool(8);
for (int i = 0; i < 8000000; i++) {
final int threadId = i;
executorService.execute(new Runnable() {
public void run() {
processId(threadId);
}
});
}
}
ExecutorService has some methods:
execute(Runnable)
submit(Runnable)
submit(Callable)
invokeAny(...)
invokeAll(...)
I recommend you view this link: ExecutorService tutorial for clear explanation.
Hope this help :)

Wait until any of Future<T> is done

I have few asynchronous tasks running and I need to wait until at least one of them is finished (in the future probably I'll need to wait util M out of N tasks are finished).
Currently they are presented as Future, so I need something like
/**
* Blocks current thread until one of specified futures is done and returns it.
*/
public static <T> Future<T> waitForAny(Collection<Future<T>> futures)
throws AllFuturesFailedException
Is there anything like this? Or anything similar, not necessary for Future. Currently I loop through collection of futures, check if one is finished, then sleep for some time and check again. This looks like not the best solution, because if I sleep for long period then unwanted delay is added, if I sleep for short period then it can affect performance.
I could try using
new CountDownLatch(1)
and decrease countdown when task is complete and do
countdown.await()
, but I found it possible only if I control Future creation. It is possible, but requires system redesign, because currently logic of tasks creation (sending Callable to ExecutorService) is separated from decision to wait for which Future. I could also override
<T> RunnableFuture<T> AbstractExecutorService.newTaskFor(Callable<T> callable)
and create custom implementation of RunnableFuture with ability to attach listener to be notified when task is finished, then attach such listener to needed tasks and use CountDownLatch, but that means I have to override newTaskFor for every ExecutorService I use - and potentially there will be implementation which do not extend AbstractExecutorService. I could also try wrapping given ExecutorService for same purpose, but then I have to decorate all methods producing Futures.
All these solutions may work but seem very unnatural. It looks like I'm missing something simple, like
WaitHandle.WaitAny(WaitHandle[] waitHandles)
in c#. Are there any well known solutions for such kind of problem?
UPDATE:
Originally I did not have access to Future creation at all, so there were no elegant solution. After redesigning system I got access to Future creation and was able to add countDownLatch.countdown() to execution process, then I can countDownLatch.await() and everything works fine.
Thanks for other answers, I did not know about ExecutorCompletionService and it indeed can be helpful in similar tasks, but in this particular case it could not be used because some Futures are created without any executor - actual task is sent to another server via network, completes remotely and completion notification is received.

simple, check out ExecutorCompletionService.

ExecutorService.invokeAny

Why not just create a results queue and wait on the queue? Or more simply, use a CompletionService since that's what it is: an ExecutorService + result queue.

This is actually pretty easy with wait() and notifyAll().
First, define a lock object. (You can use any class for this, but I like to be explicit):
package com.javadude.sample;
public class Lock {}
Next, define your worker thread. He must notify that lock object when he's finished with his processing. Note that the notify must be in a synchronized block locking on the lock object.
package com.javadude.sample;
public class Worker extends Thread {
private Lock lock_;
private long timeToSleep_;
private String name_;
public Worker(Lock lock, String name, long timeToSleep) {
lock_ = lock;
timeToSleep_ = timeToSleep;
name_ = name;
}
#Override
public void run() {
// do real work -- using a sleep here to simulate work
try {
sleep(timeToSleep_);
} catch (InterruptedException e) {
interrupt();
}
System.out.println(name_ + " is done... notifying");
// notify whoever is waiting, in this case, the client
synchronized (lock_) {
lock_.notify();
}
}
}
Finally, you can write your client:
package com.javadude.sample;
public class Client {
public static void main(String[] args) {
Lock lock = new Lock();
Worker worker1 = new Worker(lock, "worker1", 15000);
Worker worker2 = new Worker(lock, "worker2", 10000);
Worker worker3 = new Worker(lock, "worker3", 5000);
Worker worker4 = new Worker(lock, "worker4", 20000);
boolean started = false;
int numNotifies = 0;
while (true) {
synchronized (lock) {
try {
if (!started) {
// need to do the start here so we grab the lock, just
// in case one of the threads is fast -- if we had done the
// starts outside the synchronized block, a fast thread could
// get to its notification *before* the client is waiting for it
worker1.start();
worker2.start();
worker3.start();
worker4.start();
started = true;
}
lock.wait();
} catch (InterruptedException e) {
break;
}
numNotifies++;
if (numNotifies == 4) {
break;
}
System.out.println("Notified!");
}
}
System.out.println("Everyone has notified me... I'm done");
}
}

As far as I know, Java has no analogous structure to the WaitHandle.WaitAny method.
It seems to me that this could be achieved through a "WaitableFuture" decorator:
public WaitableFuture<T>
extends Future<T>
{
private CountDownLatch countDownLatch;
WaitableFuture(CountDownLatch countDownLatch)
{
super();
this.countDownLatch = countDownLatch;
}
void doTask()
{
super.doTask();
this.countDownLatch.countDown();
}
}
Though this would only work if it can be inserted before the execution code, since otherwise the execution code would not have the new doTask() method. But I really see no way of doing this without polling if you cannot somehow gain control of the Future object before execution.
Or if the future always runs in its own thread, and you can somehow get that thread. Then you could spawn a new thread to join each other thread, then handle the waiting mechanism after the join returns... This would be really ugly and would induce a lot of overhead though. And if some Future objects don't finish, you could have a lot of blocked threads depending on dead threads. If you're not careful, this could leak memory and system resources.
/**
* Extremely ugly way of implementing WaitHandle.WaitAny for Thread.Join().
*/
public static joinAny(Collection<Thread> threads, int numberToWaitFor)
{
CountDownLatch countDownLatch = new CountDownLatch(numberToWaitFor);
foreach(Thread thread in threads)
{
(new Thread(new JoinThreadHelper(thread, countDownLatch))).start();
}
countDownLatch.await();
}
class JoinThreadHelper
implements Runnable
{
Thread thread;
CountDownLatch countDownLatch;
JoinThreadHelper(Thread thread, CountDownLatch countDownLatch)
{
this.thread = thread;
this.countDownLatch = countDownLatch;
}
void run()
{
this.thread.join();
this.countDownLatch.countDown();
}
}

If you can use CompletableFutures instead then there is CompletableFuture.anyOf that does what you want, just call join on the result:
CompletableFuture.anyOf(futures).join()
You can use CompletableFutures with executors by calling the CompletableFuture.supplyAsync or runAsync methods.

Since you don't care which one finishes, why not just have a single WaitHandle for all threads and wait on that? Whichever one finishes first can set the handle.

See this option:
public class WaitForAnyRedux {
private static final int POOL_SIZE = 10;
public static <T> T waitForAny(Collection<T> collection) throws InterruptedException, ExecutionException {
List<Callable<T>> callables = new ArrayList<Callable<T>>();
for (final T t : collection) {
Callable<T> callable = Executors.callable(new Thread() {
#Override
public void run() {
synchronized (t) {
try {
t.wait();
} catch (InterruptedException e) {
}
}
}
}, t);
callables.add(callable);
}
BlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(POOL_SIZE);
ExecutorService executorService = new ThreadPoolExecutor(POOL_SIZE, POOL_SIZE, 0, TimeUnit.SECONDS, queue);
return executorService.invokeAny(callables);
}
static public void main(String[] args) throws InterruptedException, ExecutionException {
final List<Integer> integers = new ArrayList<Integer>();
for (int i = 0; i < POOL_SIZE; i++) {
integers.add(i);
}
(new Thread() {
public void run() {
Integer notified = null;
try {
notified = waitForAny(integers);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
System.out.println("notified=" + notified);
}
}).start();
synchronized (integers) {
integers.wait(3000);
}
Integer randomInt = integers.get((new Random()).nextInt(POOL_SIZE));
System.out.println("Waking up " + randomInt);
synchronized (randomInt) {
randomInt.notify();
}
}
}