When my application launches, a executor service (using Executors.newFixedThreadPool(maxThreadNum) in java.util.concurrent) object is created. When requests come, the executor service will creates threads to handle them.
Because it takes time to create threads at run time, I want to make threads available when launching application, so that when requests come, it would take less time to process.
What I did is following:
executorService = Executors.newFixedThreadPool(200);
for (int i=0; i<200; i++) {
executorService.execute(new Runnable() {
#Override
public void run() {
System.out.println("Start thread in pool " );
}
});
}
It will creates 200 threads in the executorService pool when application launches.
Just wonder is this a correct way of creating threads when application starts?
Or is there a better way of doing it?
You are missing shutdown().It is very important to shutdown the Executor service once the operation is completed. So have try,catch and Finally block
try{
executorService.execute(...);
}catach(Exception e){
...
}finally{
executorService.shutdown(); //Mandatory
}
If you can use a ThreadPoolExecutor directly rather than an ExecutorService from Executors1, then there's perhaps a more standard/supported way to start all the core threads immediately.
int nThreads = 200;
ThreadPoolExecutor executor = new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>());
executor.prestartAllCoreThreads();
The above uses prestartAllCoreThreads().
Note that, currently, the implementation of Executors.newFixedThreadPool(int) creates a ThreadPoolExecutor in the exact same manner as above. This means you could technically cast the ExecutorService returned by the factory method to a ThreadPoolExecutor. There's nothing in the documentation that guarantees it will be a ThreadPoolExecutor, however.
1. ThreadPoolExecutor implements ExecutorService but provides more functionality. Also, many of the factory methods in Executors either returns a ThreadPoolExecutor directly or a wrapper that delegates to one. Some, like newWorkStealingPool, use the ForkJoinPool. Again, the return types of these factory methods are implementation details so don't rely too much on it.
The number of threads which could run parallel depends on your processor core. Unless you have 200 cores it would be pretty useless to make a thread pool of 200.
A great way to find out how many processors cores you have is:
int cores = Runtime.getRuntime().availableProcessors();
Moreover the overhead which develops during creating a new thread and executing it is unavoidable, so unless the task is heavily computed it would not be worth to create a new single thread for this task.
But after all your code is total fine so far.
Your code is totally fine if it works for your scenario. Since we don't know your use case, only you can answer your question with enough tests and benchmark.
However, do take note that the ThreadPool will reclaim idle threads after some time. That may bite you if you don't pay attention to it.
Just wonder is this a correct way of creating threads when application
starts?
Yes. That's a correct way of creating threads.
Or is there a better way of doing it?
Maybe. Under some workloads you might want to use a Thread pool with a variable number of threads (unlike the one created by newFixedThreadPool) - one that removes from the pool threads that have been idle for some time.
Related
I'm new to this concurrent programming in java and came up with following scenarios where I'm getting confusion which to use when.
Scenario 1: In the following code I was trying to run threads by calling .start() on GPSService class which is a Runnable implementation.
int clientNumber = 0;
ServerSocket listener = new ServerSocket(port);
while (true) {
new GPSService(listener.accept(), clientNumber++, serverUrl).start();
}
Scenario 2: In the following code I was trying to run threads by using ExecutorService class as shown
int clientNumber = 0;
ServerSocket listener = new ServerSocket(port);
while(true) {
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.execute(new GPSService(listener.accept(), client++, serverUrl));
executor.shutdown();
while (!executor.awaitTermination(1, TimeUnit.SECONDS)) {
// Threads are still running
System.out.println("Thread is still running");
}
// All threads are completed
System.out.println("\nThread completed it's execution and terminated successfully\n");
}
My Questions are
Which is the best practice to invoke a thread in concurrent programming?
What will be result(troubles) I'll end up with when I use first or second?
Note: I've been facing an issue with the first scenario where the program is getting hanged after every few days. So, is that issue related/expected when I use first method.?
Any good/helpful answer will be appreciated :) Thank you
There are no big differences in the two scenario you posted, except from managing thread termination in Scenario2; you always create a new thread for each incoming request. If you want to use ThreadPool my advice is not to create one for every request but to create one for each server and reuse threads. Something like:
public class YourClass {
//in init method or constructor
ExecutorService executor = Executors....;// choose from newCachedThreadPool() or newFixedThreadPool(int nThreads) or some custom option
int clientNumber = 0;
ServerSocket listener = new ServerSocket(port);
while(true) {
executor.execute(new GPSService(listener.accept(), client++, serverUrl));
}
This will allow you to use a thread pool and to control how many threads to use for your server. If you want to use a Executor this is the preferred way to go.
With a server pool you need to decide how many threads there are in the pool; you have different choices but you can start or with a fixed number or threads or with a pool that tries to use a non busy thread and if all threads are busy it creates a new one (newCachedThreadPool()). The number of threads to allocate depends form many factors: the number of concurrents requests and it durations. The more your server side code takes time the more you need for additional thread. If your server side code is very faster there are very high chances that the pool can recycle threads already allocated (since the requests do not come all in the same exact instant).
Say for example that you have 10 request during a second and each request lasts 0.2 seconds; if the request arrive at 0, 0.1, 0.2, 0.3, 0.4, 0.5, .. part of the second (for example 23/06/2015 7:16:00:00, 23/06/2015 7:16:00:01, 23/06/2015 7:16:00:02) you need only three threads since the request coming at 0.3 can be performed by the thread that server the first request (the one at 0), and so on (the request at time 0.4 can reuse thread used for the request that came at 0.1). Ten requests managed by three threads.
I recommend you (if you did not it already) to read Java Concurrency in practice (Task Execution is chapter 6); which is an excellent book on how to build concurrent application in Java.
From oracle documentation from Executors
public static ExecutorService newCachedThreadPool()
Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available. These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks.
Calls to execute will reuse previously constructed threads if available. If no existing thread is available, a new thread will be created and added to the pool. Threads that have not been used for sixty seconds are terminated and removed from the cache.
Thus, a pool that remains idle for long enough will not consume any resources. Note that pools with similar properties but different details (for example, timeout parameters) may be created using ThreadPoolExecutor constructors.
public static ExecutorService newFixedThreadPool(int nThreads)
Creates a thread pool that reuses a fixed number of threads operating off a shared unbounded queue. At any point, at most nThreads threads will be active processing tasks. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available.
If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks. The threads in the pool will exist until it is explicitly shutdown.
#Giovanni is saying that you don' have to provide number of threads to newCachedThreadPool unlike newFixedThreadPool(), where you have to pass maximum cap on number of threads in ThreadPool.
But between these two, newFixedThreadPool() is preferred. newCachedThread Pool may cause leak and you may reach maximum number of available threads due to unbounded nature. Some people consider it as an evil.
Have a look at related SE question:
Why is an ExecutorService created via newCachedThreadPool evil?
In my Android project I had a lot of places where I need to run some code asynchronously (a web request, call to db etc.). This is not long running tasks (maximum a few seconds).
Until now I was doing this kind of stuff with creating a new thread, passing it a new runnable with the task. But recently I have read an article about threads and concurrency in Java and understood that creating a new Thread for every single task is not a good decision.
So now I have created a ThreadPoolExecutor in my Application class which holds 5 threads.
Here is the code:
public class App extends Application {
private ThreadPoolExecutor mPool;
#Override
public void onCreate() {
super.onCreate();
mPool = (ThreadPoolExecutor)Executors.newFixedThreadPool(5);
}
}
And also I have a method to submit Runnable tasks to the executor:
public void submitRunnableTask(Runnable task){
if(!mPool.isShutdown() && mPool.getActiveCount() != mPool.getMaximumPoolSize()){
mPool.submit(task);
} else {
new Thread(task).start();
}
}
So when I want to run an asynchronous task in my code I get the instance of App and call the submitRunnableTask method passing the runnable to it. As you can see, I also check, if the thread pool has free threads to execute my task, if not, I create a new Thread (I don't think that this will happen, but in any case... I don't want my task to wait in a queue and slow down the app).
In the onTerminate callback method of Application I shutdown the pool.
So my question is the following: Is this kind of pattern better then creating new Threads in code? What pros and cons my new approach has? Can it cause problems that I am not aware off yet? Can you advice me something better than this to manage my asynchronous tasks?
P.S. I have some experience in Android and Java, but I am far from being a concurrency guru ) So may be there are aspects that I don't understand well in this kind of questions. Any advice will be appreciated.
This answer assumes your tasks are short
Is this kind of pattern better then creating new Threads in code?
It's better, but it's still far from ideal. You are still creating threads for short tasks. Instead you just need to create a different type of thread pool - for example by Executors.newScheduledThreadPool(int corePoolSize).
What's the difference in behaviour?
A FixedThreadPool will always have a set of threads to use and if all threads are busy, a new task will be put into a queue.
A (default) ScheduledThreadPool, as created by the Executors class, has a minimum thread pool that it keeps, even when idle. If all threads are busy when a new task comes in, it creates a new thread for it, and disposes of the thread 60 seconds after it is done, unless it's needed again.
The second one can allow you to not create new threads by yourself. This behaviour can be achieved without the "Scheduled" part, but you will then have to construct the executor yourself. The constructor is
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue)
The various options allow you to fine-tune the behaviour.
If some tasks are long...
And I mean long. As in most of your application lifetime (Realtime 2-way connection? Server port? Multicast listener?). In that case, putting your Runnable in an executor is detrimental - standard executors are not designed to cope with it, and their performance will deteriorate.
Think about your fixed thread pool - if you have 5 long-running tasks, then any new task will spawn a new thread, completely destroying any possible gains of the pool. If you use a more flexible executor - some threads will be shared, but not always.
The rule of thumb is
If it's a short task - use an executor.
If it's a long task - make sure your executor can handle it (i.e. it either doesn't have a max pool size, or enough max threads to deal with 1 more thread being gone for a while)
If it's a parallel process that needs to always run alongside your main thread - use another Thread.
To answer your question — Yes, using Executor is better than creating new threads because:
Executor provides a selection of different thread pools. It allows re-use of already existing threads which increases performance as thread creation is an expensive operation.
In case a thread dies, Executor can replace it with a new thread without affecting the application.
Changes to multi-threading policies are much easier, as only the Executor implementation needs to be changed.
Based on the comment of Ordous I have modified my code to work with only one pool.
public class App extends Application {
private ThreadPoolExecutor mPool;
#Override
public void onCreate() {
super.onCreate();
mPool = new ThreadPoolExecutor(5, Integer.MAX_VALUE, 1, TimeUnit.MINUTES, new SynchronousQueue<Runnable>());
}
}
public void submitRunnableTask(Runnable task){
if(!mPool.isShutdown() && mPool.getActiveCount() != mPool.getMaximumPoolSize()){
mPool.submit(task);
} else {
new Thread(task).start(); // Actually this should never happen, just in case...
}
}
So, I hope this can be useful to someone else, and if more experienced people have some comments on my approach, I will very appreciate their comments.
What is the advantage of using ExecutorService over running threads passing a Runnable into the Thread constructor?
ExecutorService abstracts away many of the complexities associated with the lower-level abstractions like raw Thread. It provides mechanisms for safely starting, closing down, submitting, executing, and blocking on the successful or abrupt termination of tasks (expressed as Runnable or Callable).
From JCiP, Section 6.2, straight from the horse's mouth:
Executor may be a simple interface, but it forms the basis for a flexible and powerful framework for asynchronous task execution that supports a wide variety of task execution policies. It provides a standard means of decoupling task submission from task execution, describing tasks as Runnable. The Executor implementations also provide lifecycle support and hooks for adding statistics gathering, application management, and monitoring.
...
Using an Executor is usually the easiest path to implementing a producer-consumer design in your application.
Rather than spending your time implementing (often incorrectly, and with great effort) the underlying infrastructure for parallelism, the j.u.concurrent framework allows you to instead focus on structuring tasks, dependencies, potential parallelism. For a large swath of concurrent applications, it is straightforward to identify and exploit task boundaries and make use of j.u.c, allowing you to focus on the much smaller subset of true concurrency challenges which may require more specialized solutions.
Also, despite the boilerplate look and feel, the Oracle API page summarizing the concurrency utilities includes some really solid arguments for using them, not least:
Developers are likely to already
understand the standard library
classes, so there is no need to learn
the API and behavior of ad-hoc
concurrent components. Additionally,
concurrent applications are far
simpler to debug when they are built
on reliable, well-tested components.
Java concurrency in practice is a good book on concurrency. If you haven't already, get yourself a copy. The comprehensive approach to concurrency presented there goes well beyond this question, and will save you a lot of heartache in the long run.
An advantage I see is in managing/scheduling several threads. With ExecutorService, you don't have to write your own thread manager which can be plagued with bugs. This is especially useful if your program needs to run several threads at once. For example you want to execute two threads at a time, you can easily do it like this:
ExecutorService exec = Executors.newFixedThreadPool(2);
exec.execute(new Runnable() {
public void run() {
System.out.println("Hello world");
}
});
exec.shutdown();
The example may be trivial, but try to think that the "hello world" line consists of a heavy operation and you want that operation to run in several threads at a time in order to improve your program's performance. This is just one example, there are still many cases that you want to schedule or run several threads and use ExecutorService as your thread manager.
For running a single thread, I don't see any clear advantage of using ExecutorService.
The following limitations from traditional Thread overcome by Executor framework(built-in Thread Pool framework).
Poor Resource Management i.e. It keep on creating new resource for every request. No limit to creating resource. Using Executor framework we can reuse the existing resources and put limit on creating resources.
Not Robust : If we keep on creating new thread we will get StackOverflowException exception consequently our JVM will crash.
Overhead Creation of time : For each request we need to create new resource. To creating new resource is time consuming. i.e. Thread Creating > task. Using Executor framework we can get built in Thread Pool.
Benefits of Thread Pool
Use of Thread Pool reduces response time by avoiding thread creation during request or task processing.
Use of Thread Pool allows you to change your execution policy as you need. you can go from single thread to multiple thread by just replacing ExecutorService implementation.
Thread Pool in Java application increases stability of system by creating a configured number of threads decided based on system load and available resource.
Thread Pool frees application developer from thread management stuff and allows to focus on business logic.
Source
Below are some benefits:
Executor service manage thread in asynchronous way
Use Future callable to get the return result after thread completion.
Manage allocation of work to free thread and resale completed work from thread for assigning new work automatically
fork - join framework for parallel processing
Better communication between threads
invokeAll and invokeAny give more control to run any or all thread at once
shutdown provide capability for completion of all thread assigned work
Scheduled Executor Services provide methods for producing repeating invocations of runnables and callables
Hope it will help you
Is it really that expensive to create a new thread?
As a benchmark, I just created 60,000 threads with Runnables with empty run() methods. After creating each thread, I called its start(..) method immediately. This took about 30 seconds of intense CPU activity. Similar experiments have been done in response to this question. The summary of those is that if the threads do not finish immediately, and a large number of active threads accumulate (a few thousand), then there will be problems: (1) each thread has a stack, so you will run out of memory, (2) there might be a limit on the number of threads per process imposed by the OS, but not necessarily, it seems.
So, as far as I can see, if we're talking about launching say 10 threads per second, and they all finish faster than new ones start, and we can guarantee that this rate won't be exceeded too much, then the ExecutorService doesn't offer any concrete advantage in visible performance or stability. (Though it may still make it more convenient or readable to express certain concurrency ideas in code.) On the other hand, if you might be scheduling hundreds or thousands of tasks per second, which take time to run, you could run into big problems straight away. This might happen unexpectedly, e.g. if you create threads in response to requests to a server, and there is a spike in the intensity of requests that your server receives. But e.g. one thread in response to every user input event (key press, mouse motion) seems to be perfectly fine, as long as the tasks are brief.
ExecutorService also gives access to FutureTask which will return to the calling class the results of a background task once completed. In the case of implementing Callable
public class TaskOne implements Callable<String> {
#Override
public String call() throws Exception {
String message = "Task One here. . .";
return message;
}
}
public class TaskTwo implements Callable<String> {
#Override
public String call() throws Exception {
String message = "Task Two here . . . ";
return message;
}
}
// from the calling class
ExecutorService service = Executors.newFixedThreadPool(2);
// set of Callable types
Set<Callable<String>>callables = new HashSet<Callable<String>>();
// add tasks to Set
callables.add(new TaskOne());
callables.add(new TaskTwo());
// list of Future<String> types stores the result of invokeAll()
List<Future<String>>futures = service.invokeAll(callables);
// iterate through the list and print results from get();
for(Future<String>future : futures) {
System.out.println(future.get());
}
Prior to java 1.5 version, Thread/Runnable was designed for two separate services
Unit of work
Execution of that unit of work
ExecutorService decouples those two services by designating Runnable/Callable as unit of work and Executor as a mechanism to execute ( with lifecycling) the unit of work
Executor Framework
//Task
Runnable someTask = new Runnable() {
#Override
public void run() {
System.out.println("Hello World!");
}
};
//Thread
Thread thread = new Thread(someTask);
thread.start();
//Executor
Executor executor = new Executor() {
#Override
public void execute(Runnable command) {
Thread thread = new Thread(someTask);
thread.start();
}
};
Executor is just an interface which accept Runnable. execute() method can just call command.run() or working with other classes which use Runnable(e.g. Thread)
interface Executor
execute(Runnable command)
ExecutorService interface which extends Executor and adds methods for managing - shutdown() and submit() which returns Future[About] - get(), cancel()
interface ExecutorService extends Executor
Future<?> submit(Runnable task)
shutdown()
...
ScheduledExecutorService extends ExecutorService for planning executing tasks
interface ScheduledExecutorService extends ExecutorService
schedule()
Executors class which is a Factory to provide ExecutorService realisations for running async tasks[About]
class Executors
newFixedThreadPool() returns ThreadPoolExecutor
newCachedThreadPool() returns ThreadPoolExecutor
newSingleThreadExecutor() returns FinalizableDelegatedExecutorService
newWorkStealingPool() returns ForkJoinPool
newSingleThreadScheduledExecutor() returns DelegatedScheduledExecutorService
newScheduledThreadPool() returns ScheduledThreadPoolExecutor
...
Conclusion
Working with Thread is an expensive operation for CPU and memory.
ThreadPoolExecutor consist of Task Queue(BlockingQueue) and Thread Pool(Set of Worker) which have better performance and API to handle async tasks
Creating a large number of threads with no restriction to the maximum threshold can cause application to run out of heap memory. Because of that creating a ThreadPool is much better solution. Using ThreadPool we can limit the number of threads can be pooled and reused.
Executors framework facilitate process of creating Thread pools in java. Executors class provide simple implementation of ExecutorService using ThreadPoolExecutor.
Source:
What is Executors Framework
Does it make sense to use a ThreadPool with a poolsize of just 1 to basically just recycle that one thread over and over again for different uses in the application? Rather then doing new Thread(Runnable()) etc and then letting the garbage collector handle the removal of the thread, I thought it would be more efficient to just use that one thread for different jobs that dont need to run together.
This is what I am currently doing to define 1 poolsize threadpool.
private static int poolSize = 1;
private static int maxPoolSize = 1;
private static long keepAliveTime= 10;
private static final ArrayBlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(100);
private static ThreadPoolExecutor threadPool = new ThreadPoolExecutor(poolSize, maxPoolSize, keepAliveTime, TimeUnit.SECONDS, queue);
There is nothing wrong with a single threaded thread pool if it fits with how you application should function. For example, in an application I work on we have a number of services where we need to ensure that data is strictly processed in order of arrival. To do this we simply execute tasks on a single threaded executor.
Also using Executors means that it is easy to adjust the thread pool parameters in the future if you need to.
With new Thread(Runnable) you can execute N threads concurrently. It may be an advantage, but it also may bring synchronization issues.
With reusing one Thread you lose the ability to execute tasks in parallel, but you are spared the sync/concurrency issues.
Defining a one-thread pool this way is perfectly compatible with modern coding standards. it has the only drawback of not letting you parallelize any fragment of yhe code. However, I guess that's what you wanted.
One of the advantage of using the ThreadPoolExecutor being that once the thread is created, it will get reused as against creation of new Thread everytime when using new Thread.
Have you tried it without a Thread ? Threads are no efficient unless really needed and you need to do a lot of I/O specific stuff in parallel. If what you are looking for is a simple internal message queue, then it is fine.
Well title says it, what is the difference between Executors.newSingleThreadExecutor().execute(command) and new Thread(command).start();
Behaviourally, pretty much nothing.
However, once you have an Executor instance, you can submit multiple tasks to it, and have them executed one after another. You can't do that simply with a raw Thread.
One noticeable difference, is when you run new Thread(someRunnable).start(); when the runnable is finished the thread will die quietly.
The Executor though will persist until you shut it down. So running Executors.newSingleThreadExecutor().execute(command) When you think your application or the JVM may be finished the Executor may still be running in a background thread.
With Executor.execute, if an Error or RuntimeException is thrown in the Executor it will be swallowed silently, while the new Thread() will print it to System.err.
There can be many differences, but I will show you one difference which I found very important to understand:
public void sendEventSingleThreadExecutor(Event e){
Executor.singleThreadExecutor().execute(()->{//send the event here})
}
Now, even if you call sendEventSingleThreadExecutor method 10 times, it will use only a single thread to send them. It will not create a new thread every time. Which means that the events will be sent sequentially or synchronously!
You can read more from here:
Now see the below example with new thread
public void sendEventThread(Event e){
Thread(//send the event here).start();
}
If you call it 10 times, it will create 10 new threads. Which means, the executions will be asynchronous! And it could be dangerous, it can create a lot of threads depending on how many times you call sendEventThread functions.
Please note that, the code are only for demonstration purpose, it might have syntax error!
If you find any wrong description here, I will be happy to be corrected.
Some more information from here
newSingleThreadExecutor. A single-threaded executor creates a single
worker thread to process tasks, replacing it if it dies unexpectedly.
Tasks are guaranteed to be processed sequentially according to the
order imposed by the task queue (FIFO, LIFO, priority order).[4]
[4] Single-threaded executors also provide sufficient internal
synchronization to guarantee that any memory writes made by tasks are
visible to subsequent tasks; this means that objects can be safely
confined to the “task thread” even though that thread may be replaced
with another from time to time.
I prefer to use ExecutorService or ThreadPoolExecutor even for single digit threads. They offer more flexibility.
Have a look at ExecutorService & ThreadPoolExecutor sections in related SE questions :
java Fork/Join pool, ExecutorService and CountDownLatch
Java's Fork/Join vs ExecutorService - when to use which?
Assume that you have started with your own thread instead of ExecutorService. In future, if there is a need for supporting multiple threads, ExecutorService or ThreadPoolExecutor will offer better control and flexibility for you. You can fine-tune required number of parameters in these below APIs.
ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime,
TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory,
RejectedExecutionHandler handler)
Executors.newSingleThreadExecutor().execute(command) will reuse previously constructed thread, it will not created new thread as in case of new Thread(). If the thread that have not been used for sixty seconds are terminated, It's a kind of pool which contains a single thread which make its equivalent newFixedThreadPool(1).