Develop Reference

CompletableFuture with ExecutorService in java

I am trying to implement a thread pool using ExecutorService and CompletableFuture in java. Here I have passed the fixed-size pool to completable future tasks. if I don't pass the executor service here in completable future tasks it will use Fork/Join common pool internally as I read. Now my question is should I pass executor service here externally or not and let it use Fork/Join common pool internally? which is better in which case?
ExecutorService es = Executors.newFixedThreadPool(4);
List<CompletableFuture<?>> futures = new ArrayList<>();
for(item i: item[]){
CompletableFuture<Void> future = CompletableFuture.runAsync(() -> MyClass.service(i), es);
futures.add(future);
}
CompletableFuture.allOf(futures.toArray(new CompletableFuture[]{})).join();

It's not possible to give a fully accurate answer to your question without knowing the specifics. Which kinds of tasks do you intend to run using CompletableFuture? How many CPUs does your machine have?
In general, I would suggest avoiding using the common pool, since it may bring you some unexpected bugs up to blocking all parts of your application that rely on using the common pool.
Talking more specifically, you need to understand which type of tasks will you be running: blocking or non-blocking? Blocking tasks are those ones that use any external dependencies like database invocation and communication with other services/applications, synchronization, thread sleeps, or any other blocking utilities.
The best approach will be the creation of a separate thread pool for blocking tasks that you can fully manage. This will allow you to isolate other parts of your application and have stable and predictable execution and performance behavior for all your non-blockings tasks.
However, there is one a bit advanced workaround that may allow you to use the common pool even for blocking tasks. It's ForkJoinPool.ManagedBlocker. You may check the official documentation if you want to do some digging and experiments.

Force re-use of thread by CompletableFuture

I am making critical use of:
CompletableFuture
.delayedExecutor(1, TimeUnit.MILLISECONDS).execute(() -> {});
From what I have read online, it's common for this to use a new thread for every call. I am wondering if there is a way to re-use a thread instead of creating new threads?
Update:
I wasn't clear - I want to use CompletableFuture, but I want CompletableFuture to reuse a certain thread, instead of managing its own threads.
I see this question:
CompletableFuture reuse thread from pool
but it recommends using an environment variable - I am wondering if there is a way to do this programmatically.

From what I have read online, it's common for this to use a new thread for every call.
(1) It's the case only if the machine doesn't support parallelism or you made it to not support it by setting the system property java.util.concurrent.ForkJoinPool.common.parallelism to 0 or 1.
8 processors
(2) If the machine does support parallelism, ForkJoinPool.commonPool() is used and the parallelism level is set, I guess, to the number of available processors (which can be determined by Runtime#availableProcessors).
In a scenario with 8 processors, 7-8 threads will probably be created to serve the common ForkJoinPool.
I want to use CompletableFuture, but I want CompletableFuture to reuse a certain thread, instead of managing its own threads.
A DelayedExecutor just submits tasks to the underlying Executor, which is either a ThreadPerTaskExecutor (1) or a ForkJoinPool (2).
Fortunately, you can manually specify an Executor which will be employed by the DelayedExecutor to delegate tasks to.
Executor delayedExecutor =
CompletableFuture.delayedExecutor(1, TimeUnit.MILLISECONDS, executor);
It gets us back to your previous question, where I pointed out that an Executor can be defined with a ThreadFactory.
Executor executor = Executors.newCachedThreadPool(YourThreadClass::new);

Executor new Thread is created for every set of tasks
An Executor is normally used instead of explicitly creating threads. For example, rather than invoking new Thread(new(RunnableTask())).start() for each of a set of tasks, you might use: for each of a set of tasks
Executor executor = anExecutor;
executor.execute(new RunnableTask1());
executor.execute(new RunnableTask2());
So if you want to reuse the threads, create a thread pool by using ExecutorService or ThreadPoolExecutor, so one of the threads from the pool will execute the runnable tasks.
If all the threads are busy, tasks will be queued up to a certain limit and after that will get rejected through a RejectedExecutionException.
Example
public class NewMain {
private static final ExecutorService ex = Executors.newFixedThreadPool(3);
public static void main(String[] args) {
Runnable r = () -> System.out.println(Thread.currentThread().getName());
ex.execute(r);
CompletableFuture<Void> c = CompletableFuture.runAsync(r, ex);
}
}
Jdk-8 Use CompletableFuture.runAsync and pass runnable, Executor
public static CompletableFuture runAsync(Supplier supplier,
Executor executor)
Returns a new CompletableFuture that is asynchronously completed by a task running in the given executor after it runs the given action.

Java's Fork/Join vs ExecutorService - when to use which?

I just finished reading this post: What's the advantage of a Java-5 ThreadPoolExecutor over a Java-7 ForkJoinPool? and felt that the answer is not straight enough.
Can you explain in simple language and examples, what are the trade-offs between Java 7's Fork-Join framework and the older solutions?
I also read the Google's #1 hit on the topic Java Tip: When to use ForkJoinPool vs ExecutorService from javaworld.com but the article doesn't answer the title question when, it talks about api differences mostly ...

Fork-join allows you to easily execute divide and conquer jobs, which have to be implemented manually if you want to execute it in ExecutorService. In practice ExecutorService is usually used to process many independent requests (aka transaction) concurrently, and fork-join when you want to accelerate one coherent job.

Fork-join is particularly good for recursive problems, where a task involves running subtasks and then processing their results. (This is typically called "divide and conquer" ... but that doesn't reveal the essential characteristics.)
If you try to solve a recursive problem like this using conventional threading (e.g. via an ExecutorService) you end up with threads tied up waiting for other threads to deliver results to them.
On the other hand, if the problem doesn't have those characteristics, there is no real benefit from using fork-join.
References:
Java Tutorials: Fork/Join.
Java Tip: When to use ForkJoinPool vs ExecutorService:

Java 8 provides one more API in Executors
static ExecutorService newWorkStealingPool()
Creates a work-stealing thread pool using all available processors as its target parallelism level.
With addition of this API,Executors provides different types of ExecutorService options.
Depending on your requirement, you can choose one of them or you can look out for ThreadPoolExecutor which provides better control on Bounded Task Queue Size, RejectedExecutionHandler mechanisms.
static ExecutorService newFixedThreadPool(int nThreads)
Creates a thread pool that reuses a fixed number of threads operating off a shared unbounded queue.
static ScheduledExecutorService newScheduledThreadPool(int corePoolSize)
Creates a thread pool that can schedule commands to run after a given delay, or to execute periodically.
static ExecutorService newCachedThreadPool(ThreadFactory threadFactory)
Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available, and uses the provided ThreadFactory to create new threads when needed.
static ExecutorService newWorkStealingPool(int parallelism)
Creates a thread pool that maintains enough threads to support the given parallelism level, and may use multiple queues to reduce contention.
Each of these APIs are targeted to fulfil respective business needs of your application. Which one to use will depend on your use case requirement.
e.g.
If you want to process all submitted tasks in order of arrival, just use newFixedThreadPool(1)
If you want to optimize performance of big computation of recursive tasks, use ForkJoinPool or newWorkStealingPool
If you want to execute some tasks periodically or at certain time in future, use newScheduledThreadPool
Have a look at one more nice article by PeterLawrey on ExecutorService use cases.
Related SE question:
java Fork/Join pool, ExecutorService and CountDownLatch

Brian Goetz describes the situation best: https://www.ibm.com/developerworks/library/j-jtp11137/index.html
Using conventional thread pools to implement fork-join is also challenging because fork-join tasks spend much of their lives waiting for other tasks. This behavior is a recipe for thread starvation deadlock, unless the parameters are carefully chosen to bound the number of tasks created or the pool itself is unbounded. Conventional thread pools are designed for tasks that are independent of each other and are also designed with potentially blocking, coarse-grained tasks in mind — fork-join solutions produce neither.
I recommend reading the whole post, as it has a good example of why you'd want to use a fork-join pool. It was written before ForkJoinPool became official, so the coInvoke() method he refers to became invokeAll().

Fork-Join framework is an extension to Executor framework to particularly address 'waiting' issues in recursive multi-threaded programs. In fact, the new Fork-Join framework classes all extend from the existing classes of the Executor framework.
There are 2 characteristics central to Fork-Join framework
Work Stealing (An idle thread steals work from a thread having tasks
queued up more than it can process currently)
Ability to recursively decompose the tasks and collect the results.
(Apparently, this requirement must have popped up along with the
conception of the notion of parallel processing... but lacked a solid
implementation framework in Java till Java 7)
If the parallel processing needs are strictly recursive, there is no choice but to go for Fork-Join, otherwise either of executor or Fork-Join framework should do, though Fork-Join can be said to better utilize the resources because of the idle threads 'stealing' some tasks from busier threads.

Fork Join is an implementation of ExecuterService. The main difference is that this implementation creates DEQUE worker pool. Where task is inserted from oneside but withdrawn from any side. It means if you have created new ForkJoinPool() it will look for the available CPU and create that many worker thread. It then distribute the load evenly across each thread. But if one thread is working slowly and others are fast, they will pick the task from the slow thread. from the backside. The below steps will illustrate the stealing better.
Stage 1 (initially):
W1 -> 5,4,3,2,1
W2 -> 10,9,8,7,6
Stage 2:
W1 -> 5,4
W2 -> 10,9,8,7,
Stage 3:
W1 -> 10,5,4
W2 -> 9,8,7,
Whereas Executor service creates asked number of thread, and apply a blocking queue to store all the remaining waiting task. If you have used cachedExecuterService, it will create single thread for each job and there will be no waiting queue.

What are the advantages of using an ExecutorService?

What is the advantage of using ExecutorService over running threads passing a Runnable into the Thread constructor?

ExecutorService abstracts away many of the complexities associated with the lower-level abstractions like raw Thread. It provides mechanisms for safely starting, closing down, submitting, executing, and blocking on the successful or abrupt termination of tasks (expressed as Runnable or Callable).
From JCiP, Section 6.2, straight from the horse's mouth:
Executor may be a simple interface, but it forms the basis for a flexible and powerful framework for asynchronous task execution that supports a wide variety of task execution policies. It provides a standard means of decoupling task submission from task execution, describing tasks as Runnable. The Executor implementations also provide lifecycle support and hooks for adding statistics gathering, application management, and monitoring.
...
Using an Executor is usually the easiest path to implementing a producer-consumer design in your application.
Rather than spending your time implementing (often incorrectly, and with great effort) the underlying infrastructure for parallelism, the j.u.concurrent framework allows you to instead focus on structuring tasks, dependencies, potential parallelism. For a large swath of concurrent applications, it is straightforward to identify and exploit task boundaries and make use of j.u.c, allowing you to focus on the much smaller subset of true concurrency challenges which may require more specialized solutions.
Also, despite the boilerplate look and feel, the Oracle API page summarizing the concurrency utilities includes some really solid arguments for using them, not least:
Developers are likely to already
understand the standard library
classes, so there is no need to learn
the API and behavior of ad-hoc
concurrent components. Additionally,
concurrent applications are far
simpler to debug when they are built
on reliable, well-tested components.
Java concurrency in practice is a good book on concurrency. If you haven't already, get yourself a copy. The comprehensive approach to concurrency presented there goes well beyond this question, and will save you a lot of heartache in the long run.

An advantage I see is in managing/scheduling several threads. With ExecutorService, you don't have to write your own thread manager which can be plagued with bugs. This is especially useful if your program needs to run several threads at once. For example you want to execute two threads at a time, you can easily do it like this:
ExecutorService exec = Executors.newFixedThreadPool(2);
exec.execute(new Runnable() {
public void run() {
System.out.println("Hello world");
}
});
exec.shutdown();
The example may be trivial, but try to think that the "hello world" line consists of a heavy operation and you want that operation to run in several threads at a time in order to improve your program's performance. This is just one example, there are still many cases that you want to schedule or run several threads and use ExecutorService as your thread manager.
For running a single thread, I don't see any clear advantage of using ExecutorService.

The following limitations from traditional Thread overcome by Executor framework(built-in Thread Pool framework).
Poor Resource Management i.e. It keep on creating new resource for every request. No limit to creating resource. Using Executor framework we can reuse the existing resources and put limit on creating resources.
Not Robust : If we keep on creating new thread we will get StackOverflowException exception consequently our JVM will crash.
Overhead Creation of time : For each request we need to create new resource. To creating new resource is time consuming. i.e. Thread Creating > task. Using Executor framework we can get built in Thread Pool.
Benefits of Thread Pool
Use of Thread Pool reduces response time by avoiding thread creation during request or task processing.
Use of Thread Pool allows you to change your execution policy as you need. you can go from single thread to multiple thread by just replacing ExecutorService implementation.
Thread Pool in Java application increases stability of system by creating a configured number of threads decided based on system load and available resource.
Thread Pool frees application developer from thread management stuff and allows to focus on business logic.
Source

Below are some benefits:
Executor service manage thread in asynchronous way
Use Future callable to get the return result after thread completion.
Manage allocation of work to free thread and resale completed work from thread for assigning new work automatically
fork - join framework for parallel processing
Better communication between threads
invokeAll and invokeAny give more control to run any or all thread at once
shutdown provide capability for completion of all thread assigned work
Scheduled Executor Services provide methods for producing repeating invocations of runnables and callables
Hope it will help you

Is it really that expensive to create a new thread?
As a benchmark, I just created 60,000 threads with Runnables with empty run() methods. After creating each thread, I called its start(..) method immediately. This took about 30 seconds of intense CPU activity. Similar experiments have been done in response to this question. The summary of those is that if the threads do not finish immediately, and a large number of active threads accumulate (a few thousand), then there will be problems: (1) each thread has a stack, so you will run out of memory, (2) there might be a limit on the number of threads per process imposed by the OS, but not necessarily, it seems.
So, as far as I can see, if we're talking about launching say 10 threads per second, and they all finish faster than new ones start, and we can guarantee that this rate won't be exceeded too much, then the ExecutorService doesn't offer any concrete advantage in visible performance or stability. (Though it may still make it more convenient or readable to express certain concurrency ideas in code.) On the other hand, if you might be scheduling hundreds or thousands of tasks per second, which take time to run, you could run into big problems straight away. This might happen unexpectedly, e.g. if you create threads in response to requests to a server, and there is a spike in the intensity of requests that your server receives. But e.g. one thread in response to every user input event (key press, mouse motion) seems to be perfectly fine, as long as the tasks are brief.

ExecutorService also gives access to FutureTask which will return to the calling class the results of a background task once completed. In the case of implementing Callable
public class TaskOne implements Callable<String> {
#Override
public String call() throws Exception {
String message = "Task One here. . .";
return message;
}
}
public class TaskTwo implements Callable<String> {
#Override
public String call() throws Exception {
String message = "Task Two here . . . ";
return message;
}
}
// from the calling class
ExecutorService service = Executors.newFixedThreadPool(2);
// set of Callable types
Set<Callable<String>>callables = new HashSet<Callable<String>>();
// add tasks to Set
callables.add(new TaskOne());
callables.add(new TaskTwo());
// list of Future<String> types stores the result of invokeAll()
List<Future<String>>futures = service.invokeAll(callables);
// iterate through the list and print results from get();
for(Future<String>future : futures) {
System.out.println(future.get());
}

Prior to java 1.5 version, Thread/Runnable was designed for two separate services
Unit of work
Execution of that unit of work
ExecutorService decouples those two services by designating Runnable/Callable as unit of work and Executor as a mechanism to execute ( with lifecycling) the unit of work

Executor Framework
//Task
Runnable someTask = new Runnable() {
#Override
public void run() {
System.out.println("Hello World!");
}
};
//Thread
Thread thread = new Thread(someTask);
thread.start();
//Executor
Executor executor = new Executor() {
#Override
public void execute(Runnable command) {
Thread thread = new Thread(someTask);
thread.start();
}
};
Executor is just an interface which accept Runnable. execute() method can just call command.run() or working with other classes which use Runnable(e.g. Thread)
interface Executor
execute(Runnable command)
ExecutorService interface which extends Executor and adds methods for managing - shutdown() and submit() which returns Future[About] - get(), cancel()
interface ExecutorService extends Executor
Future<?> submit(Runnable task)
shutdown()
...
ScheduledExecutorService extends ExecutorService for planning executing tasks
interface ScheduledExecutorService extends ExecutorService
schedule()
Executors class which is a Factory to provide ExecutorService realisations for running async tasks[About]
class Executors
newFixedThreadPool() returns ThreadPoolExecutor
newCachedThreadPool() returns ThreadPoolExecutor
newSingleThreadExecutor() returns FinalizableDelegatedExecutorService
newWorkStealingPool() returns ForkJoinPool
newSingleThreadScheduledExecutor() returns DelegatedScheduledExecutorService
newScheduledThreadPool() returns ScheduledThreadPoolExecutor
...
Conclusion
Working with Thread is an expensive operation for CPU and memory.
ThreadPoolExecutor consist of Task Queue(BlockingQueue) and Thread Pool(Set of Worker) which have better performance and API to handle async tasks

Creating a large number of threads with no restriction to the maximum threshold can cause application to run out of heap memory. Because of that creating a ThreadPool is much better solution. Using ThreadPool we can limit the number of threads can be pooled and reused.
Executors framework facilitate process of creating Thread pools in java. Executors class provide simple implementation of ExecutorService using ThreadPoolExecutor.
Source:
What is Executors Framework

ExecutorService, standard way to avoid to task queue getting too full

I am using ExecutorService for ease of concurrent multithreaded program. Take following code:
while(xxx) {
ExecutorService exService = Executors.newFixedThreadPool(NUMBER_THREADS);
...
Future<..> ... = exService.submit(..);
...
}
In my case the problem is that submit() is not blocking if all NUMBER_THREADS are occupied. The consequence is that the Task queue is getting flooded by many tasks. The consequence of this is, that shutting down the execution service with ExecutorService.shutdown() takes ages (ExecutorService.isTerminated() will be false for long time). Reason is that the task queue is still quite full.
For now my workaround is to work with semaphores to disallow to have to many entries inside the task queue of ExecutorService:
...
Semaphore semaphore=new Semaphore(NUMBER_THREADS);
while(xxx) {
ExecutorService exService = Executors.newFixedThreadPool(NUMBER_THREADS);
...
semaphore.aquire();
// internally the task calls a finish callback, which invokes semaphore.release()
// -> now another task is added to queue
Future<..> ... = exService.submit(..);
...
}
I am sure there is a better more encapsulated solution?

The trick is to use a fixed queue size and:
new ThreadPoolExecutor.CallerRunsPolicy()
I also recommend using Guava's ListeningExecutorService.
Here is an example consumer/producer queues.
private ListeningExecutorService producerExecutorService = MoreExecutors.listeningDecorator(newFixedThreadPoolWithQueueSize(5, 20));
private ListeningExecutorService consumerExecutorService = MoreExecutors.listeningDecorator(newFixedThreadPoolWithQueueSize(5, 20));
private static ExecutorService newFixedThreadPoolWithQueueSize(int nThreads, int queueSize) {
return new ThreadPoolExecutor(nThreads, nThreads,
5000L, TimeUnit.MILLISECONDS,
new ArrayBlockingQueue<Runnable>(queueSize, true), new ThreadPoolExecutor.CallerRunsPolicy());
}
Anything better and you might want to consider a MQ like RabbitMQ or ActiveMQ as they have QoS technology.

A true blocking ThreadPoolExecutor has been on the wishlist of many, there's even a JDC bug opened on it.
I'm facing the same problem, and came across this:
http://today.java.net/pub/a/today/2008/10/23/creating-a-notifying-blocking-thread-pool-executor.html
It's an implementation of a BlockingThreadPoolExecutor, implemented using a RejectionPolicy that uses offer to add the task to the queue, waiting for the queue to have room. It looks good.

You can call ThreadPoolExecutor.getQueue().size() to find out the size of the waiting queue. You can take an action if the queue is too long. I suggest running the task in the current thread if the queue is too long to slow down the producer (if that is appropriate).

You're better off creating the ThreadPoolExecutor yourself (which is what Executors.newXXX() does anyway).
In the constructor, you can pass in a BlockingQueue for the Executor to use as its task queue. If you pass in a size constrained BlockingQueue (like LinkedBlockingQueue), it should achieve the effect you want.
ExecutorService exService = new ThreadPoolExecutor(NUMBER_THREADS, NUMBER_THREADS, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>(workQueueSize));

you can add another bloquing queue that's has limited size to controle the size of internal queue in executorService, some thinks like semaphore but very easy.
before executor you put() and whene the task achive take(). take() must be inside the task code

I know this is too old but might be useful for other developers. So submitting one of the solution.
As you asked for better encapsulated solution. It is done by extending ThreadPoolExecutor and overriding submit method.
BoundedThreadpoolExecutor implemented using Semaphore. Java executor service throws RejectedExecutionException when the task queue becomes full. Using unbounded queue may result in out of memory error. This can be avoided by controlling the number of tasks being submitted using executor service. This can be done by using semaphore or by implementing RejectedExecutionHandler.