I have 4-5 worker threads processing large message queue. And I also have another piece of code which runs using 2-3 workers. I wanted to block all other workers when large message queue is being processed.
I am using JDK 6, and Jms
EDITED:
Queue process workers never terminated. They blocked on queue when no message. These workers are managed by executor thread pool.If I use read-write lock, one of these workers will get blocked too.
Also, if used cyclic barrier then I have to terminate threads in order to relase the blocking second process. Since workers are managed by thread pool, it is not assured that all workers will be busy processing messages.
Let me know,
final ExecutorService executor = getExecutorManager().getExecutor();
for (int i = 0; i < threadPoolSize; i++) {
executor.submit(new MessageWorker(qConn));
}
Following is second module, where i want all workers to be blocked while queue processors worker threads are working.
final ExecutorService executor = getExecutorManager().getExecutor();
for (int i = 0; i < threadPoolSize; i++) {
executor.submit(new DbUpdateWorker());
}
You need to use CyclicBarrier.
A synchronization aid that allows a set of threads to all wait for each other to reach a common barrier point. CyclicBarriers are useful in programs involving a fixed sized party of threads that must occasionally wait for each other. The barrier is called cyclic because it can be re-used after the waiting threads are released.
Use as:
CyclicBarrier barrier = new CyclicBarrier(numWorkers, runnable);
Where runnable is a Runnable that you want to call when your worker threads are finished. Each thread calls barrier.await() when it's complete.
Related
I am having a scenario of around inserting millions of data into the back end and currently using executor framework to load this. I will explain my problem in simpler terms.
In the below case, I am having 10 runnable and three threads to execute the same. Consider my runnable is doing an insert operation and it is taking time to complete the task. When I checked ,It is understood that ,if all the threads are busy, the other tasks will go to the queue and once the threads completed the tasks ,it will fetch the tasks from the pool and complete it.
So in this case, object of SampleRunnable 4 to 10 will be created and this will be in the pool.
Problem: Since I need to load millions of tasks,I cannot load all the records in queue which can lead to memory issues. So my question is instead of taking all tasks in the queue ,is it possible to make the main thread waiting until any one of the executor worker threads becomes available.
Following approaches I tried as a work around instead of queuing this much tasks:
Approach 1: Used Array Blocking Queue for executor and gave the size as 5 (for e.g.)
So in this case, when the 9th task comes ,this will throw RejectedExecutionException and in the catch clause,put a sleep for 1 minute and recursively trying the same.This will get picked up on any of the retry when the thread is available.
Approach 2: Used shut down and await termination. i.e. if the task count is 5, i am putting shut down and await termination. In the await Termination 'if' block (executor.awaitTermination(60000,TimeUnit.SECONDS)),I am instantiating the thread pool again.
public class SampleMain {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(3);
for (int i=0;i<10;i++){
executorService.execute(new SampleRunnable(i));
}
executor.shutdown();
}
Sounds like the problem is, you want to throttle the main thread, so that it does not get ahead of the workers. If that's the case, then consider explicitly constructing a ThreadPoolExecutor instance instead of calling Executors.newFixedThreadPool().
That class has several different constructors, and most of them allow you to supply your own blocking queue. If you create an ArrayBlockingQueue with a limited size, then every time the queue becomes full, the main thread will be automatically blocked until a worker makes room by taking another task.
final int work_queue_size = 30;
BlockingQueue work_queue = new ArrayBlockingQueue(work_queue_size);
ExecutorService executor = new ThreadPoolExecutor(..., work_queue);
for (int i=0;i<10;i++){
executorService.execute(new SampleRunnable(i));
}
...
I am fetching messages from Kafka and processing them using Executor service similar to below. I am not calling executorService.shutdown(). I have occasionally been seeing heapSize exception , but not sure if this could be one of the causes. How do the unused Runnable instances get removed after completion? Should I do anything specific to explicitly make it eligible for GC?
public class Consumer implements CommandLineRunner{
ExecutorService executorService;
executorService = Executors.newFixedThreadPool(50)
executorService.submit(runnable);
}
}
From documentation for Executors.newFixedThreadPool:
Creates a thread pool that reuses a fixed number of threads operating
off a shared unbounded queue. At any point, at most nThreads threads
will be active processing tasks. If additional tasks are submitted
when all threads are active, they will wait in the queue until a
thread is available. If any thread terminates due to a failure during
execution prior to shutdown, a new one will take its place if needed
to execute subsequent tasks. The threads in the pool will exist until
it is explicitly shutdown.
For given example there will be at most 50 threads active in the thread pool and they are released when you call shutdown method.
If you do not keep references to your Runnables they will be GCed when Executor is done with them. If you get out of memory exceptions this can be due to queued Runnables in cases when executor can not keep up with the work submitted to him.
EDIT: Also out of memory exceptions can happen if your tasks take a lot of memory (obviously).
I put a bunch of runnable objects into an ExecutorService:
// simplified content of main method
ExecutorService threadPool = Executors.newCachedThreadPool();
for(int i = 0; i < workerCount; i++) {
threadPool.execute(new Worker());
}
I would expect my program/process to stop immediately after all workers are done. But according to my log, it takes another 20-30 seconds until that happens. The workers do not allocate any resources, in fact, they do nothing at the moment.
Don't get me wrong, this is not a crucial problem for me, I'm just trying to understand what is happening and I'm wondering if this is normal behavior.
Executors.newCachedThreadPool() uses Executors.defaultThreadFactory() for its ThreadFactory. defaultThreadFactory's javadocs say that "each new thread is created as a non-daemon thread" (emphasis added). So, the threads created for the newCachedThreadPool are non-daemon. That means that they'll prevent the JVM from exiting naturally (by "naturally" I mean that you can still call System.exit(1) or kill the program to cause the JVM to halt).
The reason the app finishes at all is that each thread created within the newCachedThreadPool times out and closes itself after some time of inactivity. When the last one of them closes itself, if your application doesn't have any non-daemon threads left, it'll quit.
You can (and should) close the ExecutorService down manually via shutdown or shutdownNow.
See also the JavaDoc for Thread, which talks about daemon-ness.
I would expect my program/process to stop immediately after all workers are done. But according to my log, it takes another 20-30 seconds until that happens. The workers do not allocate any resources, in fact, they do nothing at the moment.
The problem is that you are not shutting down your ExecutorService. After you submit all of the jobs to the service, you should shutdown the service or the JVM will not terminate unless all of the threads in it are daemon threads. If you do not shutdown the thread-pool then any threads associated with the ExecutorService, again if not daemon, will stop the JVM from finishing. If you've submitted any tasks to a cached thread pool then you will have to wait for the threads to timeout and get reaped before the JVM will finish.
ExecutorService threadPool = Executors.newCachedThreadPool();
for(int i = 0; i < workerCount; i++) {
threadPool.execute(new Worker());
}
// you _must_ do this after submitting all of your workers
threadPool.shutdown();
Starting the threads as daemon is most likely not what you want to do because your application may stop before the tasks have completed and all of the tasks will be terminated immediately at that time. I just did a quick audit and of the 178 times we use ExecutorService classes in our production code, only 2 of them were started as daemon threads. The rest are properly shutdown.
If you need to force an ExecutorService to stop when the application is exiting then using shutdownNow() with proper handling of the thread interrupt flags is in order.
Basically on an ExecutorService you call shutdown() and then awaitTermination():
ExecutorService taskExecutor = Executors.newFixedThreadPool(4);
while(...) {
taskExecutor.execute(new MyTask());
}
taskExecutor.shutdown();
try {
taskExecutor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
...
}
From the javadoc for Executors.newCachedThreadPool():
Threads that have not been used for sixty seconds are terminated and removed from the cache.
It is usually a good idea to call shutdown() on an ExecutorService if you know that no new tasks will be submitted to it. Then all tasks in the queue will complete, but the service will then shut down immediately.
(Alternately, if you don't care if all the tasks complete - for example, if they are handling background calculations that are irrelevant once your main UI is gone - then you can create a ThreadFactory that sets all the threads in that pool to be daemon.)
For multi threading of ExecutorService
Solution is
threadPool.shutdown();
It is due to combination keepAliveTime=60L, timeunit=TimeUnit.SECONDS and corePoolSize=0*: when thread completes task, it does not terminate immediately, it may** wait during keepAliveTime for a new task.
public static ExecutorService newCachedThreadPool() {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
}
*if core poolSize != 0 see method allowCoreThreadTimeOut() of ThreadPoolExecutor
**waiting depends on combination of current quantity of running threads in pool, corePoolSize and maximumPoolSize
When I create ExecutorService with below code in JAVA,can someone explain how the ExecutorService works ?
ExecutorService executor = Executors.newFixedThreadPool(400);
for (int i = 0; i < 500; i++) {
Runnable worker = new MyRunnable(10000000L + i);
executor.execute(worker);
}
I believe that there will be a single Queue of work and my for loop will add 500 Runnable tasks to this queue. Now the ExecutorService has been created with a Thread Pool of 400 threads.
So of those 500 tasks in the queue, the 400 threads in the ExecutorService will execute this 400 tasks at a time, and the remaining as slots are freed up?
Am I correct in my understanding ?
JavaDoc newFixedThreadPool
Creates a thread pool that reuses a fixed number of threads operating
off a shared unbounded queue. At any point, at most nThreads threads
will be active processing tasks. If additional tasks are submitted
when all threads are active, they will wait in the queue until a
thread is available. If any thread terminates due to a failure during
execution prior to shutdown, a new one will take its place if needed
to execute subsequent tasks. The threads in the pool will exist until
it is explicitly shutdown.
If tasks are more than number of processing threads, the tasks which haven't been picked up by threads will wait. Once the thread completes one task, it will pick-up one more waiting task.
But these thread pools (other than ForkJoinPool) are not efficient in stealing worker thread tasks.
Assume that one thread is backlog of 10 tasks to be executed and it's running first task. At that same time, some other thread in pool is idle. In this scenario, once the task is allocated a Thread, only that thread will execute the task even though other threads are idle.
ForkJoinPool differs from other kinds of ExecutorService mainly by virtue of employing work-stealing: all threads in the pool attempt to find and execute tasks submitted to the pool and/or created by other active tasks (eventually blocking waiting for work if none exist)
One more new API has been added in Java 8.
public static ExecutorService newWorkStealingPool()
Creates a work-stealing thread pool using all available processors as its target parallelism level.
Relate SE question : ThreadPoolExecutor vs ForkJoinPool: stealing subtasks
I'm preparing an application where a single producer generates several million tasks, which will then be processed by a configurable number of consumers. Communication from producer to consumer is (probably) going to be queue-based.
From the thread that runs the producer/generates the tasks, what method can I use to wait for completion of all tasks? I'd rather not resume to any periodic polling to see if my tasks queue is empty. In any case, the task queue being empty isn't actually a guarantee that the last tasks have completed. Those tasks can be relatively long-running, so it's quite possible that the queue is empty while the consumer threads are still happily processing.
Rgds, Maarten
You might want to have a look at the java.util.concurrent package.
ExecutorService
Executors
Future
The executor framework already provides means to execute tasks via threadpool. The Future abstraction allows to wait for the completition of tasks.
Putting both together allows you coordinate the executions easily, decoupling tasks, activities (threads) and results.
Example:
ExecutorService executorService = Executors.newFixedThreadPool(16);
List<Callable<Void>> tasks = null;
//TODO: fill tasks;
//dispatch
List<Future<Void>> results = executorService.invokeAll(tasks);
//Wait until all tasks have completed
for(Future<Void> result: results){
result.get();
}
Edit: Alternative Version using CountDownLatch
ExecutorService executorService = Executors.newFixedThreadPool(16);
final CountDownLatch latch;
List<Callable<Void>> tasks = null;
//TODO: fill tasks;
latch = new CountDownLatch(tasks.size());
//dispatch
executorService.invokeAll(tasks);
//Wait until all tasks have completed
latch.await();
And inside your tasks:
Callable<Void> task = new Callable<Void>()
{
#Override
public Void call() throws Exception
{
// TODO: do your stuff
latch.countDown(); //<---- important part
return null;
}
};
You want to know where every tasks completes. I would have another queue of completed task reports. (One object/message per task) When this count reaches the number of tasks you created, they have all completed. This task report can also have any errors and timing information for the task.
You could have each consumer check to see if the queue is empty when they dequeue, and, if it is, pulse a condvar (or a Monitor, since I believe that's what Java has) on which the main thread is waiting.
Having the threads check a global boolean variable (marked as volatile) is a way to let the threads know that they should stop.
You can use join() method for each thread ..so that till all the threads are done your main thread will not end! And by this way you can actually find out whether all the threads are done or not!