Please note that I usually ask a question after googling for more than 20 times about the issue. But I can't still understand it. So I need your help.
Basically, I don't understand the exact usage of newFixedThreadPool
Does newFixedThreadPool(10) mean having ten different threads? Or does it mean it can have 10 of the same threads? or the both?
I executed with submit() methods more than 20 times and it's working.
Does submit() print a value? Or are you putting threads in the ExecutorService?
Briefly, tasks are small units of code that could be executed in parallel (code sections). The threads (in a thread pool) are what execute them. You can think of the threads like workers and the tasks like jobs. Jobs can be done in parallel, and workers can work in parallel. Workers work on jobs.
So, to answer your questions:
newFixedThreadPool(int nThreads) creates a thread pool of nThread threads that operate on the same input queue. nThreads is the maximum number of threads that can be running at any given time. Each thread can run a different task. With your example, you can be running up to 10 tasks at the same time. (The documentation can be found here with credit to #hovercraft-full-of-eels)
submit() pushes the given task into an event queue that is shared by the threads in the thread pool. Once a thread is available, it will take a task from the front of the queue and execute it. It shouldn't print anything, unless the Runnable you pass it has a print statement in it. However, the print statement may not be printed right when you submit the task! It will print once a thread is executing that particular task. (The documentation can be found here)
Just refer java docs or JAVA API's description rather than googling it.
For your questions I have below comments .
Question 1 ->
ExecutorService executorService = Executors.newFixedThreadPool(10);
First an ExecutorService is created using the Executors newFixedThreadPool() factory method. This creates a thread pool with 10 threads executing tasks.
Executors.newFixedThreadPool API creates, a thread pool that reuses a fixed number of threads and these threads work on a s***hared unbounded queue***.
At any point, at most nThreads threads will be active processing tasks.
If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available.
If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks. The threads in the pool will exist until it is explicitly SHUTDOWN.
After submitting even 20 tasks ,it worked with this thread pool.
Internally it calls below line of codes .
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue());
}
Question 2- > Submits a Runnable task for execution in Queue and it can also return an Object of type Future Object representing task. we can use Future's get method to check whether submitted task has successfully completed or not because it will return null upon successful completion.
Related
I am having a scenario of around inserting millions of data into the back end and currently using executor framework to load this. I will explain my problem in simpler terms.
In the below case, I am having 10 runnable and three threads to execute the same. Consider my runnable is doing an insert operation and it is taking time to complete the task. When I checked ,It is understood that ,if all the threads are busy, the other tasks will go to the queue and once the threads completed the tasks ,it will fetch the tasks from the pool and complete it.
So in this case, object of SampleRunnable 4 to 10 will be created and this will be in the pool.
Problem: Since I need to load millions of tasks,I cannot load all the records in queue which can lead to memory issues. So my question is instead of taking all tasks in the queue ,is it possible to make the main thread waiting until any one of the executor worker threads becomes available.
Following approaches I tried as a work around instead of queuing this much tasks:
Approach 1: Used Array Blocking Queue for executor and gave the size as 5 (for e.g.)
So in this case, when the 9th task comes ,this will throw RejectedExecutionException and in the catch clause,put a sleep for 1 minute and recursively trying the same.This will get picked up on any of the retry when the thread is available.
Approach 2: Used shut down and await termination. i.e. if the task count is 5, i am putting shut down and await termination. In the await Termination 'if' block (executor.awaitTermination(60000,TimeUnit.SECONDS)),I am instantiating the thread pool again.
public class SampleMain {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(3);
for (int i=0;i<10;i++){
executorService.execute(new SampleRunnable(i));
}
executor.shutdown();
}
Sounds like the problem is, you want to throttle the main thread, so that it does not get ahead of the workers. If that's the case, then consider explicitly constructing a ThreadPoolExecutor instance instead of calling Executors.newFixedThreadPool().
That class has several different constructors, and most of them allow you to supply your own blocking queue. If you create an ArrayBlockingQueue with a limited size, then every time the queue becomes full, the main thread will be automatically blocked until a worker makes room by taking another task.
final int work_queue_size = 30;
BlockingQueue work_queue = new ArrayBlockingQueue(work_queue_size);
ExecutorService executor = new ThreadPoolExecutor(..., work_queue);
for (int i=0;i<10;i++){
executorService.execute(new SampleRunnable(i));
}
...
I have a java program deployed in Weblogic server which is executed using Quartz scheduler. The program executes in every 10 seconds. In the java code I have created two threads using ExcutorService and I have called service.shutdown() at the end. But every time the quartz scheduler runs the program it creates a new pool of threads by incrementing the pool id like "pool-109-thread-1" and "pool-109-thread-2" then pool-110-thread-1" and "pool-110-thread-2". So this pool count is increasing. Is it ok or do I need to change something in my code ?
Sample Code below:`
public void post(){
ExecutorService service = Executors.newFixedThreadPool(2);
for (String filePath : strArray) {
service.submit(new PostImages(postURL,filePath));
}
service.shutDown();
}
`
Every time you call the Executors.newFixedThreadPool(2) a new thread pool of 2 threads is created. It will be a problem if running too much times because the number of process of your OS will crash.
You must to convert your local variable to a static variable and to have only 1 instance of this thread pool, keeping only 2 threads to execute the jobs.
I had the same problem, this is what you want to use
If you are interested in knowing when a certain task completes, or a certain batch of tasks, you may use ExecutorService.submit(Runnable). Invoking this method returns a Future object which may be placed into a Collection which your main thread will then iterate over calling Future.get() for each one. This will cause your main thread to halt execution until the ExecutorService has processed all of the Runnable tasks.
Collection<Future<?>> futures = new LinkedList<Future<?>>();
futures.add(executorService.submit(myRunnable));
for (Future<?> future:futures) {
future.get();
}
How to wait for all tasks in an ThreadPoolExecutor to finish without shutting down the Executor?
I think it's ok. It's probably incrementing the pool ID because old ones are still waiting for the garbage collector.
How to i implement such a function ?
I have a dynamic queue which gets filled at unknown times with runnables, which have to be executed. The ExecutorService should only start a limited amount of threads, when the maximum thread size is reached, it should stop executing more threads, until one thread finishes, then the next task should be executed.
So far i came across this:
ExecutorService executor = new ThreadPoolExecutor(20, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
databaseConnectionQueue);
The ExecutorService is created before the queue is filled, and should stay alive until the queue is deleted, not when its empty, because this can happen. Can anybody help me ?
ThreadPoolExecutor will will not shutdown when it is empty. From the JavaDoc:
A pool that is no longer referenced in a program AND has no remaining
threads will be shutdown automatically.
I believe you should use FixedThreadPool (Executors.newFixedThreadPool)
As per javadoc [Executors.newFixedThreadPool]
Creates a thread pool that reuses a fixed number of threads operating
off a shared unbounded queue. At any point, at most nThreads threads
will be active processing tasks. If additional tasks are submitted
when all threads are active, they will wait in the queue until a
thread is available. If any thread terminates due to a failure during
execution prior to shutdown, a new one will take its place if needed
to execute subsequent tasks. The threads in the pool will exist until
it is explicitly shutdown.
Hope it helps. Thanks.
I've inherited some code that uses Executors.newFixedThreadPool(4); to run the 4 long-lived threads that do all the work of the application.
Is this recommended? I've read the Java Concurrency in Practice book and there does not seem to be much guidance around how to manage long-lived application threads.
What is the recommended way to start and manage several threads that each live for the entire live of the application?
You mentioned that code is using Executors, it should be returning an ExecutorService
ExecutorService executor = Executors.newFixedThreadPool(NTHREDS);
ExecutorService is an Executor that provides methods to manage termination and methods that can produce a Future for tracking progress of one or more asynchronous tasks.
As long as returned ExecutorService is performing graceful shutdown there should not be an issue.
You can check that your code is doing shutodwn by finding following in your code:
// This will make the executor accept no new threads
// and finish all existing threads in the queue
executor.shutdown();
// Wait until all threads are finish
executor.awaitTermination();
Cheers !!
I assume that your long-lived thread do some periodic job in a loop. What you can do is the following:
Make sure that each runnable in the pool checks the pool's state before looping.
while( ! pool.isShutdown() ) { ... }
Your runnable must thus have a reference to their parent pool.
Install a JVM shutdown hook with Runtime.addShutdownHook(). The hook calls pool.shutdown() then pool.awaitTermination(). The pool will transition to the SHUTDOWN state and eventually the threads will stop, after which it will transition to the TERMINATED state.
--
That said, I'm a bit suspicious of your 4 threads. Shouldn't there be only 1 long-live threads, which fetches tasks, and submits them to an executor service? Do you really have 4 different long-lived processes? (This consideration is orthogonal to the main question).
I would like to implement a piece of software that will make sure 10 threads running the same class all the time
All threads will terminate let's say after a certain number of seconds at random, but then, I want to make sure once they're terminated, they're replaced instantly by newly created threads.
Would you know if there's any library available?
Many thanks,
Vlad
Rather you use Thread Pool. Java Executor framework provides it.
ExecutorService executor = Executors.newFixedThreadPool(10);
ExecutorService executorPool = Executors.newFixedThreadPool(noOfThreads);
public static ExecutorService newFixedThreadPool(int nThreads)
Creates a thread pool that reuses a fixed set of threads operating off a
shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.
Parameters:
nThreads - the number of threads in the pool
Returns:
the newly created thread pool