I have a java program deployed in Weblogic server which is executed using Quartz scheduler. The program executes in every 10 seconds. In the java code I have created two threads using ExcutorService and I have called service.shutdown() at the end. But every time the quartz scheduler runs the program it creates a new pool of threads by incrementing the pool id like "pool-109-thread-1" and "pool-109-thread-2" then pool-110-thread-1" and "pool-110-thread-2". So this pool count is increasing. Is it ok or do I need to change something in my code ?
Sample Code below:`
public void post(){
ExecutorService service = Executors.newFixedThreadPool(2);
for (String filePath : strArray) {
service.submit(new PostImages(postURL,filePath));
}
service.shutDown();
}
`
Every time you call the Executors.newFixedThreadPool(2) a new thread pool of 2 threads is created. It will be a problem if running too much times because the number of process of your OS will crash.
You must to convert your local variable to a static variable and to have only 1 instance of this thread pool, keeping only 2 threads to execute the jobs.
I had the same problem, this is what you want to use
If you are interested in knowing when a certain task completes, or a certain batch of tasks, you may use ExecutorService.submit(Runnable). Invoking this method returns a Future object which may be placed into a Collection which your main thread will then iterate over calling Future.get() for each one. This will cause your main thread to halt execution until the ExecutorService has processed all of the Runnable tasks.
Collection<Future<?>> futures = new LinkedList<Future<?>>();
futures.add(executorService.submit(myRunnable));
for (Future<?> future:futures) {
future.get();
}
How to wait for all tasks in an ThreadPoolExecutor to finish without shutting down the Executor?
I think it's ok. It's probably incrementing the pool ID because old ones are still waiting for the garbage collector.
Related
I am having a scenario of around inserting millions of data into the back end and currently using executor framework to load this. I will explain my problem in simpler terms.
In the below case, I am having 10 runnable and three threads to execute the same. Consider my runnable is doing an insert operation and it is taking time to complete the task. When I checked ,It is understood that ,if all the threads are busy, the other tasks will go to the queue and once the threads completed the tasks ,it will fetch the tasks from the pool and complete it.
So in this case, object of SampleRunnable 4 to 10 will be created and this will be in the pool.
Problem: Since I need to load millions of tasks,I cannot load all the records in queue which can lead to memory issues. So my question is instead of taking all tasks in the queue ,is it possible to make the main thread waiting until any one of the executor worker threads becomes available.
Following approaches I tried as a work around instead of queuing this much tasks:
Approach 1: Used Array Blocking Queue for executor and gave the size as 5 (for e.g.)
So in this case, when the 9th task comes ,this will throw RejectedExecutionException and in the catch clause,put a sleep for 1 minute and recursively trying the same.This will get picked up on any of the retry when the thread is available.
Approach 2: Used shut down and await termination. i.e. if the task count is 5, i am putting shut down and await termination. In the await Termination 'if' block (executor.awaitTermination(60000,TimeUnit.SECONDS)),I am instantiating the thread pool again.
public class SampleMain {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(3);
for (int i=0;i<10;i++){
executorService.execute(new SampleRunnable(i));
}
executor.shutdown();
}
Sounds like the problem is, you want to throttle the main thread, so that it does not get ahead of the workers. If that's the case, then consider explicitly constructing a ThreadPoolExecutor instance instead of calling Executors.newFixedThreadPool().
That class has several different constructors, and most of them allow you to supply your own blocking queue. If you create an ArrayBlockingQueue with a limited size, then every time the queue becomes full, the main thread will be automatically blocked until a worker makes room by taking another task.
final int work_queue_size = 30;
BlockingQueue work_queue = new ArrayBlockingQueue(work_queue_size);
ExecutorService executor = new ThreadPoolExecutor(..., work_queue);
for (int i=0;i<10;i++){
executorService.execute(new SampleRunnable(i));
}
...
Please note that I usually ask a question after googling for more than 20 times about the issue. But I can't still understand it. So I need your help.
Basically, I don't understand the exact usage of newFixedThreadPool
Does newFixedThreadPool(10) mean having ten different threads? Or does it mean it can have 10 of the same threads? or the both?
I executed with submit() methods more than 20 times and it's working.
Does submit() print a value? Or are you putting threads in the ExecutorService?
Briefly, tasks are small units of code that could be executed in parallel (code sections). The threads (in a thread pool) are what execute them. You can think of the threads like workers and the tasks like jobs. Jobs can be done in parallel, and workers can work in parallel. Workers work on jobs.
So, to answer your questions:
newFixedThreadPool(int nThreads) creates a thread pool of nThread threads that operate on the same input queue. nThreads is the maximum number of threads that can be running at any given time. Each thread can run a different task. With your example, you can be running up to 10 tasks at the same time. (The documentation can be found here with credit to #hovercraft-full-of-eels)
submit() pushes the given task into an event queue that is shared by the threads in the thread pool. Once a thread is available, it will take a task from the front of the queue and execute it. It shouldn't print anything, unless the Runnable you pass it has a print statement in it. However, the print statement may not be printed right when you submit the task! It will print once a thread is executing that particular task. (The documentation can be found here)
Just refer java docs or JAVA API's description rather than googling it.
For your questions I have below comments .
Question 1 ->
ExecutorService executorService = Executors.newFixedThreadPool(10);
First an ExecutorService is created using the Executors newFixedThreadPool() factory method. This creates a thread pool with 10 threads executing tasks.
Executors.newFixedThreadPool API creates, a thread pool that reuses a fixed number of threads and these threads work on a s***hared unbounded queue***.
At any point, at most nThreads threads will be active processing tasks.
If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available.
If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks. The threads in the pool will exist until it is explicitly SHUTDOWN.
After submitting even 20 tasks ,it worked with this thread pool.
Internally it calls below line of codes .
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue());
}
Question 2- > Submits a Runnable task for execution in Queue and it can also return an Object of type Future Object representing task. we can use Future's get method to check whether submitted task has successfully completed or not because it will return null upon successful completion.
I would like to implement a piece of software that will make sure 10 threads running the same class all the time
All threads will terminate let's say after a certain number of seconds at random, but then, I want to make sure once they're terminated, they're replaced instantly by newly created threads.
Would you know if there's any library available?
Many thanks,
Vlad
Rather you use Thread Pool. Java Executor framework provides it.
ExecutorService executor = Executors.newFixedThreadPool(10);
ExecutorService executorPool = Executors.newFixedThreadPool(noOfThreads);
public static ExecutorService newFixedThreadPool(int nThreads)
Creates a thread pool that reuses a fixed set of threads operating off a
shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.
Parameters:
nThreads - the number of threads in the pool
Returns:
the newly created thread pool
I got a loop {Loop-1}, where I start Threads. The class which contains the {Loop-1} implements Daemon and Runnable.
In the {Loop-1} the thread, which is started, calls a method coordinate() of a class Coordinate.java where I use the ExecutorService.
When the object of Coordinate.java is created (this happens once BEFORE {Loop-1}), I instantiate a ExecutorService
pool = Executors.newFixedThreadPool(2);
In coordinate() I create two Objects of a class which implements Callable and I start them then and store the result in a List of Future results.
callableResults = pool.invokeAll(threads);
After, I try to get the results in a loop with result = future.get();
Then, I return to {Loop-1} and the whole process starts again (call coordinate(), invokeAll(), future.get()
Now Ive got the following question:
1. Do I need to shutdown the pool of ExecutorService after I got the results in coordinate()?
2. Do I need to recreate the pool everytime my {Loop-1} calls coordinate()?
Thanks for answers! :-)
No you do not. The threads in the fixed thread pool can be used until you call shutdown on it. So, you can simply resubmit new tasks to be executed and fetch their results, exactly as you did in the first go-round.
You need to shutdown the executorService once you're done processing all your tasks.
The submission of tasks can be in multiple cycles.
Once you call executorService.shutDown(), you can wait until all tasks are completed after calling shutDown() using executorService.awaitTermination(10, TimeUnit.SECONDS).
Alternatively, you can do: while (!executorService.isTerminated()) { }
I'm new to java concurrency so this may be a question already answered many time over or too obvious that I maybe missing something.
I am running as task like so:
Executors.newSingleThreadExecutor().execute(task)
My question is when its comes to end of executing the run method of task why does it not exit or why is the thread still alive? My understanding was once a threads run() completes the thread is no more and ceases to exist, right?
newSingleThreadExecutor returns an ExecutorService which uses a single thread - it can still execute multiple tasks. It doesn't exit because you may want to supply more tasks.
You can use:
ExecutorService service = Executors.newSingleThreadExecutor();
service.execute(task);
service.shutdown();
to shut it down after the task has executed.
the thread remains alive because its lifecycle is not tied to that of the tasks assigned to the executor; take a look at:
javadoc for Executors.newSingleThreadExecutor
you'll find that internally, the returned ExecutorService uses a single thread to sequentially run as many tasks as you assign to it, potentially instantiating a new thread if one of your tasks kills the original one.