I am using ExecutorService for ease of concurrent multithreaded program. Take following code:
while(xxx) {
ExecutorService exService = Executors.newFixedThreadPool(NUMBER_THREADS);
...
Future<..> ... = exService.submit(..);
...
}
In my case the problem is that submit() is not blocking if all NUMBER_THREADS are occupied. The consequence is that the Task queue is getting flooded by many tasks. The consequence of this is, that shutting down the execution service with ExecutorService.shutdown() takes ages (ExecutorService.isTerminated() will be false for long time). Reason is that the task queue is still quite full.
For now my workaround is to work with semaphores to disallow to have to many entries inside the task queue of ExecutorService:
...
Semaphore semaphore=new Semaphore(NUMBER_THREADS);
while(xxx) {
ExecutorService exService = Executors.newFixedThreadPool(NUMBER_THREADS);
...
semaphore.aquire();
// internally the task calls a finish callback, which invokes semaphore.release()
// -> now another task is added to queue
Future<..> ... = exService.submit(..);
...
}
I am sure there is a better more encapsulated solution?
The trick is to use a fixed queue size and:
new ThreadPoolExecutor.CallerRunsPolicy()
I also recommend using Guava's ListeningExecutorService.
Here is an example consumer/producer queues.
private ListeningExecutorService producerExecutorService = MoreExecutors.listeningDecorator(newFixedThreadPoolWithQueueSize(5, 20));
private ListeningExecutorService consumerExecutorService = MoreExecutors.listeningDecorator(newFixedThreadPoolWithQueueSize(5, 20));
private static ExecutorService newFixedThreadPoolWithQueueSize(int nThreads, int queueSize) {
return new ThreadPoolExecutor(nThreads, nThreads,
5000L, TimeUnit.MILLISECONDS,
new ArrayBlockingQueue<Runnable>(queueSize, true), new ThreadPoolExecutor.CallerRunsPolicy());
}
Anything better and you might want to consider a MQ like RabbitMQ or ActiveMQ as they have QoS technology.
A true blocking ThreadPoolExecutor has been on the wishlist of many, there's even a JDC bug opened on it.
I'm facing the same problem, and came across this:
http://today.java.net/pub/a/today/2008/10/23/creating-a-notifying-blocking-thread-pool-executor.html
It's an implementation of a BlockingThreadPoolExecutor, implemented using a RejectionPolicy that uses offer to add the task to the queue, waiting for the queue to have room. It looks good.
You can call ThreadPoolExecutor.getQueue().size() to find out the size of the waiting queue. You can take an action if the queue is too long. I suggest running the task in the current thread if the queue is too long to slow down the producer (if that is appropriate).
You're better off creating the ThreadPoolExecutor yourself (which is what Executors.newXXX() does anyway).
In the constructor, you can pass in a BlockingQueue for the Executor to use as its task queue. If you pass in a size constrained BlockingQueue (like LinkedBlockingQueue), it should achieve the effect you want.
ExecutorService exService = new ThreadPoolExecutor(NUMBER_THREADS, NUMBER_THREADS, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>(workQueueSize));
you can add another bloquing queue that's has limited size to controle the size of internal queue in executorService, some thinks like semaphore but very easy.
before executor you put() and whene the task achive take(). take() must be inside the task code
I know this is too old but might be useful for other developers. So submitting one of the solution.
As you asked for better encapsulated solution. It is done by extending ThreadPoolExecutor and overriding submit method.
BoundedThreadpoolExecutor implemented using Semaphore. Java executor service throws RejectedExecutionException when the task queue becomes full. Using unbounded queue may result in out of memory error. This can be avoided by controlling the number of tasks being submitted using executor service. This can be done by using semaphore or by implementing RejectedExecutionHandler.
Related
I am making critical use of:
CompletableFuture
.delayedExecutor(1, TimeUnit.MILLISECONDS).execute(() -> {});
From what I have read online, it's common for this to use a new thread for every call. I am wondering if there is a way to re-use a thread instead of creating new threads?
Update:
I wasn't clear - I want to use CompletableFuture, but I want CompletableFuture to reuse a certain thread, instead of managing its own threads.
I see this question:
CompletableFuture reuse thread from pool
but it recommends using an environment variable - I am wondering if there is a way to do this programmatically.
From what I have read online, it's common for this to use a new thread for every call.
(1) It's the case only if the machine doesn't support parallelism or you made it to not support it by setting the system property java.util.concurrent.ForkJoinPool.common.parallelism to 0 or 1.
8 processors
(2) If the machine does support parallelism, ForkJoinPool.commonPool() is used and the parallelism level is set, I guess, to the number of available processors (which can be determined by Runtime#availableProcessors).
In a scenario with 8 processors, 7-8 threads will probably be created to serve the common ForkJoinPool.
I want to use CompletableFuture, but I want CompletableFuture to reuse a certain thread, instead of managing its own threads.
A DelayedExecutor just submits tasks to the underlying Executor, which is either a ThreadPerTaskExecutor (1) or a ForkJoinPool (2).
Fortunately, you can manually specify an Executor which will be employed by the DelayedExecutor to delegate tasks to.
Executor delayedExecutor =
CompletableFuture.delayedExecutor(1, TimeUnit.MILLISECONDS, executor);
It gets us back to your previous question, where I pointed out that an Executor can be defined with a ThreadFactory.
Executor executor = Executors.newCachedThreadPool(YourThreadClass::new);
Executor new Thread is created for every set of tasks
An Executor is normally used instead of explicitly creating threads. For example, rather than invoking new Thread(new(RunnableTask())).start() for each of a set of tasks, you might use: for each of a set of tasks
Executor executor = anExecutor;
executor.execute(new RunnableTask1());
executor.execute(new RunnableTask2());
So if you want to reuse the threads, create a thread pool by using ExecutorService or ThreadPoolExecutor, so one of the threads from the pool will execute the runnable tasks.
If all the threads are busy, tasks will be queued up to a certain limit and after that will get rejected through a RejectedExecutionException.
Example
public class NewMain {
private static final ExecutorService ex = Executors.newFixedThreadPool(3);
public static void main(String[] args) {
Runnable r = () -> System.out.println(Thread.currentThread().getName());
ex.execute(r);
CompletableFuture<Void> c = CompletableFuture.runAsync(r, ex);
}
}
Jdk-8 Use CompletableFuture.runAsync and pass runnable, Executor
public static CompletableFuture runAsync(Supplier supplier,
Executor executor)
Returns a new CompletableFuture that is asynchronously completed by a task running in the given executor after it runs the given action.
When my application launches, a executor service (using Executors.newFixedThreadPool(maxThreadNum) in java.util.concurrent) object is created. When requests come, the executor service will creates threads to handle them.
Because it takes time to create threads at run time, I want to make threads available when launching application, so that when requests come, it would take less time to process.
What I did is following:
executorService = Executors.newFixedThreadPool(200);
for (int i=0; i<200; i++) {
executorService.execute(new Runnable() {
#Override
public void run() {
System.out.println("Start thread in pool " );
}
});
}
It will creates 200 threads in the executorService pool when application launches.
Just wonder is this a correct way of creating threads when application starts?
Or is there a better way of doing it?
You are missing shutdown().It is very important to shutdown the Executor service once the operation is completed. So have try,catch and Finally block
try{
executorService.execute(...);
}catach(Exception e){
...
}finally{
executorService.shutdown(); //Mandatory
}
If you can use a ThreadPoolExecutor directly rather than an ExecutorService from Executors1, then there's perhaps a more standard/supported way to start all the core threads immediately.
int nThreads = 200;
ThreadPoolExecutor executor = new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>());
executor.prestartAllCoreThreads();
The above uses prestartAllCoreThreads().
Note that, currently, the implementation of Executors.newFixedThreadPool(int) creates a ThreadPoolExecutor in the exact same manner as above. This means you could technically cast the ExecutorService returned by the factory method to a ThreadPoolExecutor. There's nothing in the documentation that guarantees it will be a ThreadPoolExecutor, however.
1. ThreadPoolExecutor implements ExecutorService but provides more functionality. Also, many of the factory methods in Executors either returns a ThreadPoolExecutor directly or a wrapper that delegates to one. Some, like newWorkStealingPool, use the ForkJoinPool. Again, the return types of these factory methods are implementation details so don't rely too much on it.
The number of threads which could run parallel depends on your processor core. Unless you have 200 cores it would be pretty useless to make a thread pool of 200.
A great way to find out how many processors cores you have is:
int cores = Runtime.getRuntime().availableProcessors();
Moreover the overhead which develops during creating a new thread and executing it is unavoidable, so unless the task is heavily computed it would not be worth to create a new single thread for this task.
But after all your code is total fine so far.
Your code is totally fine if it works for your scenario. Since we don't know your use case, only you can answer your question with enough tests and benchmark.
However, do take note that the ThreadPool will reclaim idle threads after some time. That may bite you if you don't pay attention to it.
Just wonder is this a correct way of creating threads when application
starts?
Yes. That's a correct way of creating threads.
Or is there a better way of doing it?
Maybe. Under some workloads you might want to use a Thread pool with a variable number of threads (unlike the one created by newFixedThreadPool) - one that removes from the pool threads that have been idle for some time.
I want to to create an something like ExecutorService in which there will be single thread initially, and based on the work load given to the ExecutorService, the thread count has to be increased gradually upto certain count, say 50 for example. I could not find any way to do that.
Is there anyway make this happen in NETTY NIO?
Appreciate your help. Thanks.
You can use a ThreadPoolExecutor. You don't really add threads yourself, rather, the pool will instantiate new threads as needed according to the load, so you start with core threads and new threads will be added automatically as needed up to maxThreads. Threads above the core count will stick around once they idle for the timeout period.
import java.util.concurrent.*;
....
ThreadPoolExecutor tpe = new ThreadPoolExecutor(core, maxThreads,
timeout, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>(),
threadFactory);
In netty, ChannelHandlerContext.executor() returns the Netty's EventExecutor. You can submit your Runnables to Netty's executor.
ctx.executor().submit(aRunnable);
If you want to perform an event, you might want to avoid creating your own executor service, and rely on netty provided executor.
I'm faced with a very slow data import console app and I'm trying to speed it up by multithreading it.
Is there an example pattern for starting and managing X number of worker threads in a console app? Ideally I was hoping that there would be something like:
ThreadManager tm = new ThreadManager(maxthreads=10);
while (moreWork = true) {
tm.addThread(new Thread(new MyClass));
}
The ThreadManager would add threads until the max was reached and then wait patiently until a slot became available for a new thread.
Anything like that out there? I can't be the first one that's faced this problem.
#jeshurun's answer is correct but for posterity, I thought I'd add some more information. If you utilize the great ExecutorService code then your code would turn into:
ExecutorService threadPool = Executors.newFixedThreadPool(10);
while (moreWork) {
threadPool.submit(new MyClass);
}
// stop pool after you've submitted the last job, submitted jobs will still run
threadPool.shutdown();
// you can wait for the last job to finish if you'd like
threadPool.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
Your MyClass should either implement Runnable or Callable. If it is a Runnable then the thread pool will execute the MyClass.run() method when it has an available thread. Same with Callable except that the Future<?> returned by the submit() method can be used to get the value returned by your MyClass.call() method. You can also get any exceptions that were thrown by call(). Here's a decent tutorial.
If you are using Java 5 or above, why don't you use the ExecutorService interface from the java.util.concurrent framework, and its implementation ThreadPoolExecutor for managing a fixed number of threads? Normally you would use one of the static methods in the Executors class to obtain an instance of a ThreadPool with a fixed size, and submit as many threads as you want to it for execution.
The documentation for ThreadPoolExecutor and all its friends in the java.util.concurrent package is available here.
I can't use shutdown() and awaitTermination() because it is possible new tasks will be added to the ThreadPoolExecutor while it is waiting.
So I'm looking for a way to wait until the ThreadPoolExecutor has emptied it's queue and finished all of it's tasks without stopping new tasks from being added before that point.
If it makes any difference, this is for Android.
Thanks
Update: Many weeks later after revisiting this, I discovered that a modified CountDownLatch worked better for me in this case. I'll keep the answer marked because it applies more to what I asked.
If you are interested in knowing when a certain task completes, or a certain batch of tasks, you may use ExecutorService.submit(Runnable). Invoking this method returns a Future object which may be placed into a Collection which your main thread will then iterate over calling Future.get() for each one. This will cause your main thread to halt execution until the ExecutorService has processed all of the Runnable tasks.
Collection<Future<?>> futures = new LinkedList<Future<?>>();
futures.add(executorService.submit(myRunnable));
for (Future<?> future:futures) {
future.get();
}
My Scenario is a web crawler to fetch some information from a web site then processing them. A ThreadPoolExecutor is used to speed up the process because many pages can be loaded in the time. So new tasks will be created in the existing task because the crawler will follow hyperlinks in each page. The problem is the same: the main thread do not know when all the tasks are completed and it can start to process the result. I use a simple way to determine this. It is not very elegant but works in my case:
while (executor.getTaskCount()!=executor.getCompletedTaskCount()){
System.err.println("count="+executor.getTaskCount()+","+executor.getCompletedTaskCount());
Thread.sleep(5000);
}
executor.shutdown();
executor.awaitTermination(60, TimeUnit.SECONDS);
Maybe you are looking for a CompletionService to manage batches of task, see also this answer.
(This is an attempt to reproduce Thilo's earlier, deleted answer with my own adjustments.)
I think you may need to clarify your question since there is an implicit infinite condition... at some point you have to decide to shut down your executor, and at that point it won't accept any more tasks. Your question seems to imply that you want to wait until you know that no further tasks will be submitted, which you can only know in your own application code.
The following answer will allow you to smoothly transition to a new TPE (for whatever reason), completing all the currently-submitted tasks, and not rejecting new tasks to the new TPE. It might answer your question. #Thilo's might also.
Assuming you have defined somewhere a visible TPE in use as such:
AtomicReference<ThreadPoolExecutor> publiclyAvailableTPE = ...;
You can then write the TPE swap routine as such. It could also be written using a synchronized method, but I think this is simpler:
void rotateTPE()
{
ThreadPoolExecutor newTPE = createNewTPE();
// atomic swap with publicly-visible TPE
ThreadPoolExecutor oldTPE = publiclyAvailableTPE.getAndSet(newTPE);
oldTPE.shutdown();
// and if you want this method to block awaiting completion of old tasks in
// the previously visible TPE
oldTPE.awaitTermination();
}
Alternatively, if you really no kidding want to kill the thread pool, then your submitter side will need to cope with rejected tasks at some point, and you could use null for the new TPE:
void killTPE()
{
ThreadPoolExecutor oldTPE = publiclyAvailableTPE.getAndSet(null);
oldTPE.shutdown();
// and if you want this method to block awaiting completion of old tasks in
// the previously visible TPE
oldTPE.awaitTermination();
}
Which could cause upstream problems, the caller would need to know what to do with a null.
You could also swap out with a dummy TPE that simply rejected every new execution, but that's equivalent to what happens if you call shutdown() on the TPE.
If you don't want to use shutdown, follow below approaches:
Iterate through all Future tasks from submit on ExecutorService and check the status with blocking call get() on Future object as suggested by Tim Bender
Use one of
Using invokeAll on ExecutorService
Using CountDownLatch
Using ForkJoinPool or newWorkStealingPool of Executors(since java 8)
invokeAll() on executor service also achieves the same purpose of CountDownLatch
Related SE question:
How to wait for a number of threads to complete?
You could call the waitTillDone() on Runner class:
Runner runner = Runner.runner(10);
runner.runIn(2, SECONDS, runnable);
runner.run(runnable); // each of this runnables could submit more tasks
runner.waitTillDone(); // blocks until all tasks are finished (or failed)
// and now reuse it
runner.runIn(500, MILLISECONDS, callable);
runner.waitTillDone();
runner.shutdown();
To use it add this gradle/maven dependency to your project: 'com.github.matejtymes:javafixes:1.0'
For more details look here: https://github.com/MatejTymes/JavaFixes or here: http://matejtymes.blogspot.com/2016/04/executor-that-notifies-you-when-task.html
Try using queue size and active tasks count as shown below
while (executor.getThreadPoolExecutor().getActiveCount() != 0 || !executor.getThreadPoolExecutor().getQueue().isEmpty()){
try {
Thread.sleep(500);
} catch (InterruptedException e) {
}
}