I have been reading up on how the settings of Spring's ThreadPoolTaskExecutor work together and how the thread pool and queue work. This stackoverflow answer as well as this and this article from Baeldung have been useful to me.
As far as I understand thus far, corePoolSize number of threads are kept alive at all time (assuming allowCoreThreadTimeOut is not set to true). If all of these threads are currently in use, any additional requests will be put on the queue. Once queueCapacity is reached, the thread pool size will be increased until maxPoolSize is reached.
Intuitively, I would have thought it would instead work as follows:
corePoolSize number of threads are kept alive at all time (again assuming allowCoreThreadTimeOut is not set to true). If all of these threads are currently in use and new requests come in, the pool size will be increased until maxPoolSize is reached. If there are then still more requests coming in, they will be put on the queue until queueCapacity is reached.
I wonder what would be the reasoning behind it working the way it is?
The first reference you should check is the documentation.
Right from the documentation for ThreadPoolExecutor (ThreadPoolTaskExecutor is "just" a wrapper):
A ThreadPoolExecutor will automatically adjust the pool size (see getPoolSize()) according to the bounds set by corePoolSize (see getCorePoolSize()) and maximumPoolSize (see getMaximumPoolSize()). When a new task is submitted in method execute(Runnable), if fewer than corePoolSize threads are running, a new thread is created to handle the request, even if other worker threads are idle. Else if fewer than maximumPoolSize threads are running, a new thread will be created to handle the request only if the queue is full. [...]
If the pool currently has more than corePoolSize threads, excess threads will be terminated if they have been idle for more than the keepAliveTime (see getKeepAliveTime(TimeUnit)). This provides a means of reducing resource consumption when the pool is not being actively used. If the pool becomes more active later, new threads will be constructed. [...]
(You haven't mentioned the parameter for the BlockingQueue but I suggest you to read about it as well. It's very interesting.)
Why do the parameters not work like you've suggested they should?
If the pool size would be increased up to maximumPoolSize before tasks are queued (like you've proposed), you'd have one problem: You'd have removed the thread pool's ability to determine when a new worker is worth it.
The corePoolSize is the amount of workers that stay in the pool. The benefit is that you don't have to create, terminate, create, terminate, create ... new workers for a given workload. If you can determine how much work there will always be, it's a smart idea to set the corePoolSize accordingly.
The maximumPoolSize determines the maximum amount of workers in the pool. You want to have control over that as you could have multiple thread pools, hardware restrictions or just a specific program where you don't need as many workers.
Now why does the work queue get filled up first? Because the queue capacity is an indicator for when the amount of work is so high, that it's worth it to create new workers. As long the queue is not full, the core workers are supposed to be enough to handle the given work. If the capacity is reached, then new workers are created to handle further work.
With this mechanism the thread pool dynamically creates workers when there is a need for them and only keeps so many workers as there is usually need for. This is the point of a thread pool.
Related
newCachedThreadPool() versus newFixedThreadPool()
When should I use one or the other? Which strategy is better in terms of resource utilization?
I think the docs explain the difference and usage of these two functions pretty well:
newFixedThreadPool
Creates a thread pool that reuses a
fixed number of threads operating off
a shared unbounded queue. At any
point, at most nThreads threads will
be active processing tasks. If
additional tasks are submitted when
all threads are active, they will wait
in the queue until a thread is
available. If any thread terminates
due to a failure during execution
prior to shutdown, a new one will take
its place if needed to execute
subsequent tasks. The threads in the
pool will exist until it is explicitly
shutdown.
newCachedThreadPool
Creates a thread pool that creates new
threads as needed, but will reuse
previously constructed threads when
they are available. These pools will
typically improve the performance of
programs that execute many short-lived
asynchronous tasks. Calls to execute
will reuse previously constructed
threads if available. If no existing
thread is available, a new thread will
be created and added to the pool.
Threads that have not been used for
sixty seconds are terminated and
removed from the cache. Thus, a pool
that remains idle for long enough will
not consume any resources. Note that
pools with similar properties but
different details (for example,
timeout parameters) may be created
using ThreadPoolExecutor constructors.
In terms of resources, the newFixedThreadPool will keep all the threads running until they are explicitly terminated. In the newCachedThreadPool Threads that have not been used for sixty seconds are terminated and removed from the cache.
Given this, the resource consumption will depend very much in the situation. For instance, If you have a huge number of long running tasks I would suggest the FixedThreadPool. As for the CachedThreadPool, the docs say that "These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks".
Just to complete the other answers, I would like to quote Effective Java, 2nd Edition, by Joshua Bloch, chapter 10, Item 68 :
"Choosing the executor service for a particular application can be tricky. If you’re writing a small program, or a lightly loaded server, using Executors.new- CachedThreadPool is generally a good choice, as it demands no configuration and generally “does the right thing.” But a cached thread pool is not a good choice for a heavily loaded production server!
In a cached thread pool, submitted tasks are not queued but immediately handed off to a thread for execution. If no threads are available, a new one is created. If a server is so heavily loaded that all of its CPUs are fully utilized, and more tasks arrive, more threads will be created, which will only make matters worse.
Therefore, in a heavily loaded production server, you are much better off using Executors.newFixedThreadPool, which gives you a pool with a fixed number of threads, or using the ThreadPoolExecutor class directly, for maximum control."
If you see the code in the grepcode, you will see, they are calling ThreadPoolExecutor. internally and setting their properties. You can create your one to have a better control of your requirement.
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
}
public static ExecutorService newCachedThreadPool() {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
}
The ThreadPoolExecutor class is the base implementation for the executors that are returned from many of the Executors factory methods. So let's approach Fixed and Cached thread pools from ThreadPoolExecutor's perspective.
ThreadPoolExecutor
The main constructor of this class looks like this:
public ThreadPoolExecutor(
int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
ThreadFactory threadFactory,
RejectedExecutionHandler handler
)
Core Pool Size
The corePoolSize determines the minimum size of the target thread pool. The implementation would maintain a pool of that size even if there are no tasks to execute.
Maximum Pool Size
The maximumPoolSize is the maximum number of threads that can be active at once.
After the thread pool grows and becomes bigger than the corePoolSize threshold, the executor can terminate idle threads and reach to the corePoolSize again.
If allowCoreThreadTimeOut is true, then the executor can even terminate core pool threads if they were idle more than keepAliveTime threshold.
So the bottom line is if threads remain idle more than keepAliveTime threshold, they may get terminated since there is no demand for them.
Queuing
What happens when a new task comes in and all core threads are occupied? The new tasks will be queued inside that BlockingQueue<Runnable> instance. When a thread becomes free, one of those queued tasks can be processed.
There are different implementations of the BlockingQueue interface in Java, so we can implement different queuing approaches like:
Bounded Queue: New tasks would be queued inside a bounded task queue.
Unbounded Queue: New tasks would be queued inside an unbounded task queue. So this queue can grow as much as the heap size allows.
Synchronous Handoff: We can also use the SynchronousQueue to queue the new tasks. In that case, when queuing a new task, another thread must already be waiting for that task.
Work Submission
Here's how the ThreadPoolExecutor executes a new task:
If fewer than corePoolSize threads are running, tries to start a
new thread with the given task as its first job.
Otherwise, it tries to enqueue the new task using the
BlockingQueue#offer method. The offer method won't block if the queue is full and immediately returns false.
If it fails to queue the new task (i.e. offer returns false), then it tries to add a new thread to the thread pool with this task as its first job.
If it fails to add the new thread, then the executor is either shut down or saturated. Either way, the new task would be rejected using the provided RejectedExecutionHandler.
The main difference between the fixed and cached thread pools boils down to these three factors:
Core Pool Size
Maximum Pool Size
Queuing
+-----------+-----------+-------------------+---------------------------------+
| Pool Type | Core Size | Maximum Size | Queuing Strategy |
+-----------+-----------+-------------------+---------------------------------+
| Fixed | n (fixed) | n (fixed) | Unbounded `LinkedBlockingQueue` |
+-----------+-----------+-------------------+---------------------------------+
| Cached | 0 | Integer.MAX_VALUE | `SynchronousQueue` |
+-----------+-----------+-------------------+---------------------------------+
Fixed Thread Pool
Here's how the Excutors.newFixedThreadPool(n) works:
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>());
}
As you can see:
The thread pool size is fixed.
If there is high demand, it won't grow.
If threads are idle for quite some time, it won't shrink.
Suppose all those threads are occupied with some long-running tasks and the arrival rate is still pretty high. Since the executor is using an unbounded queue, it may consume a huge part of the heap. Being unfortunate enough, we may experience an OutOfMemoryError.
When should I use one or the other? Which strategy is better in terms of resource utilization?
A fixed-size thread pool seems to be a good candidate when we're going to limit the number of concurrent tasks for resource management purposes.
For example, if we're going to use an executor to handle web server requests, a fixed executor can handle the request bursts more reasonably.
For even better resource management, it's highly recommended to create a custom ThreadPoolExecutor with a bounded BlockingQueue<T> implementation coupled with reasonable RejectedExecutionHandler.
Cached Thread Pool
Here's how the Executors.newCachedThreadPool() works:
public static ExecutorService newCachedThreadPool() {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
}
As you can see:
The thread pool can grow from zero threads to Integer.MAX_VALUE. Practically, the thread pool is unbounded.
If any thread is idle for more than 1 minute, it may get terminated. So the pool can shrink if threads remain too much idle.
If all allocated threads are occupied while a new task comes in, then it creates a new thread, as offering a new task to a SynchronousQueue always fails when there is no one on the other end to accept it!
When should I use one or the other? Which strategy is better in terms of resource utilization?
Use it when you have a lot of predictable short-running tasks.
If you are not worried about an unbounded queue of Callable/Runnable tasks, you can use one of them. As suggested by bruno, I too prefer newFixedThreadPool to newCachedThreadPool over these two.
But ThreadPoolExecutor provides more flexible features compared to either newFixedThreadPool or newCachedThreadPool
ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime,
TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory,
RejectedExecutionHandler handler)
Advantages:
You have full control of BlockingQueue size. It's not un-bounded, unlike the earlier two options. I won't get an out of memory error due to a huge pile-up of pending Callable/Runnable tasks when there is unexpected turbulence in the system.
You can implement custom Rejection handling policy OR use one of the policies:
In the default ThreadPoolExecutor.AbortPolicy, the handler throws a runtime RejectedExecutionException upon rejection.
In ThreadPoolExecutor.CallerRunsPolicy, the thread that invokes execute itself runs the task. This provides a simple feedback control mechanism that will slow down the rate that new tasks are submitted.
In ThreadPoolExecutor.DiscardPolicy, a task that cannot be executed is simply dropped.
In ThreadPoolExecutor.DiscardOldestPolicy, if the executor is not shut down, the task at the head of the work queue is dropped, and then execution is retried (which can fail again, causing this to be repeated.)
You can implement a custom Thread factory for the below use cases:
To set a more descriptive thread name
To set thread daemon status
To set thread priority
That’s right, Executors.newCachedThreadPool() isn't a great choice for server code that's servicing multiple clients and concurrent requests.
Why? There are basically two (related) problems with it:
It's unbounded, which means that you're opening the door for anyone to cripple your JVM by simply injecting more work into the service (DoS attack). Threads consume a non-negligible amount of memory and also increase memory consumption based on their work-in-progress, so it's quite easy to topple a server this way (unless you have other circuit-breakers in place).
The unbounded problem is exacerbated by the fact that the Executor is fronted by a SynchronousQueue which means there's a direct handoff between the task-giver and the thread pool. Each new task will create a new thread if all existing threads are busy. This is generally a bad strategy for server code. When the CPU gets saturated, existing tasks take longer to finish. Yet more tasks are being submitted and more threads created, so tasks take longer and longer to complete. When the CPU is saturated, more threads is definitely not what the server needs.
Here are my recommendations:
Use a fixed-size thread pool Executors.newFixedThreadPool or a ThreadPoolExecutor. with a set maximum number of threads;
You must use newCachedThreadPool only when you have short-lived asynchronous tasks as stated in Javadoc, if you submit tasks which takes longer time to process, you will end up creating too many threads. You may hit 100% CPU if you submit long running tasks at faster rate to newCachedThreadPool (http://rashcoder.com/be-careful-while-using-executors-newcachedthreadpool/).
I do some quick tests and have the following findings:
1) if using SynchronousQueue:
After the threads reach the maximum size, any new work will be rejected with the exception like below.
Exception in thread "main" java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask#3fee733d rejected from java.util.concurrent.ThreadPoolExecutor#5acf9800[Running, pool size = 3, active threads = 3, queued tasks = 0, completed tasks = 0]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
2) if using LinkedBlockingQueue:
The threads never increase from minimum size to maximum size, meaning the thread pool is fixed size as the minimum size.
I have a long running process that listens to events and do some intense processing.
Currently I use Executors.newFixedThreadPool(x) to throttle the number of jobs that runs concurrently, but depending of the time of the day, and other various factors, I would like to be able to dynamically increase or decrease the number of concurrent threads.
If I decrease the number of concurrent threads, I want the current running jobs to finish nicely.
Is there a Java library that let me control and dynamically increase or decrease the number of concurrent threads running in a Thread Pool ? (The class must implement ExecutorService).
Do I have to implement it myself ?
Have a look at below API in ThreadPoolExecutor
public void setCorePoolSize(int corePoolSize)
Sets the core number of threads. This overrides any value set in the constructor.
If the new value is smaller than the current value, excess existing threads will be terminated when they next become idle.
If larger, new threads will, if needed, be started to execute any queued tasks.
Initialization:
ExecutorService service = Executors.newFixedThreadPool(5);
On need basis, resize Thread pool by using below API
((ThreadPoolExecutor)service).setCorePoolSize(newLimit);//newLimit is new size of the pool
Important note:
If the queue is full, and new value of number of threads is greater than or equal to maxPoolSize defined earlier, Task will be rejected.
So set values of maxPoolSize and corePoolSize properly.
I'm doing some load tests agains my Spring application and now I'm a little bit confused about the configuration of the ThreadPoolTaskExecutor.
The documentation of the internally used ThreadPoolExecutor describes the corePoolSize as "the number of threads to keep in the pool, even if they are idle, [...]" and maximumPoolSize as "the maximum number of threads to allow in the pool".
That obviously means that the maximumPoolSize limits the number of thread in the pool. But instead the limit seems the be set by the corePoolSize. Actually I configured just the corePoolSize with 100 an let the maximumPoolSize unconfigured (that means the default value is used: Integer.MAX_VALUE = 2147483647).
When I run the load test I can see (by reviewing the logs), that the executed worker thread are numbered from worker-1 to worker-100. So in this case the thread pool size is limited by corePoolSize. Even if I set maximumPoolSize to 200 or 300, the result is exactly the same.
Why the value of maximumPoolSize has no affect in my case?
#Bean
public TaskExecutor taskExecutor() {
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setCorePoolSize(100);
taskExecutor.setThreadNamePrefix("worker-");
return taskExecutor;
}
SOLUTION
I've found the solution in the documentation: "If there are more than corePoolSize but less than maximumPoolSize threads running, a new thread will be created only if the queue is full". The default queue size is Integer.MAX_VALUE. If I limit the queue, everything works fine.
I have done some testing on ThreadPoolTaskExecutor and there is three things that you have to understand:
corePoolSize
queueCapacity
maxPoolSize
When you start the process there is no threads in the pool.
Each time a task comes one new executor thread will be created to handle this new load as long the corePoolSize is not reached.
When the corePoolSize is reached the next task will be shift to the queue and wait for a free executor thread.
If the load is too high and queueCapacity is full, the new executor threads will be created unless the maxPoolSize is reached. These additional threads will expire as soon as the queue is empty.
If the corePoolSize is exhausted, queueCapacity is full and maxPoolSize is also reached then the new submitteds tasks will be rejected and called will get an exception.
You have not mentioned the queueCapacity of your configuration so it might be set to highest integer number and thus maxPoolSize is never getting triggered. Try with small corePoolSize and queueCapacity and you will observe the desired result.
If you have 100 threads in a pool and you are executing CPU bound code on 4 physical CPU cores, most of your core threads are idle in the pool waiting to be re-used. That is probably why you don't see more than worker-100.
You didn't show us code you are executing in workers, therefore I assume it is not I/O bound. If it would be I/O bound code and 100 of your core threads would be occupied by waiting for blocking I/O operations to finish, ThreadPoolExecutor would need to create additional workers.
Try it with corePoolSize lower than number of cores on your machine to confirm. Another option is to put Thread.sleep(1000) into your worker code and observe how your workers count will be raising.
EDIT:
You suggested to use SimpleAsyncTaskExecutor in comment. Notice this section of Spring Framework docs:
SimpleAsyncTaskExecutor This implementation does not reuse any
threads, rather it starts up a new thread for each invocation.
However, it does support a concurrency limit which will block any
invocations that are over the limit until a slot has been freed up. If
you are looking for true pooling, see the discussions of
SimpleThreadPoolTaskExecutor and ThreadPoolTaskExecutor below.
So with SimpleAsyncTaskExecutor you don't have pooling at all and a lot of resources (CPU cycles included) are wasted on creation and deletion of Thread objects, which may be quite expensive operation.
So SimpleAsyncTaskExecutor executor type does more harm than good to your load testing. If you want to have more workers, use more machines. It's naive to use only one machine if you want to have accurate load testing.
Will the thread creating method wait for a thread to get free?
can I reduce the number of threads generated using thread pooling?
If you use a cached thread pool, the pool will create more threads. However this will only be the maximum needed at any one time and might be far less than the number of tasks you submit.
If you use a fixed size thread pool, it will create a fixed number of threads regardless of whether you give it any tasks, or if you give it more tasks than it can do. It will queue any tasks which are waiting.
Will the thread creating method wait for a thread to get free?
While you could create a queue which did this, this is not the default behaviour. A more common solution is to make the caller execute the task if this is required.
can I reduce the number of threads generated using thread pooling?
Thread pooling is likely to produce far less threads than tasks esp if you limit the number of threads.
Will the thread creating method wait for a thread to get free?
That contradicts with your title. You'd normally submit a task and the pool would pass that task to a worker thread when one is available. So you'd not create a thread but submit a task. Whether you wait for the task to be executed or just trigger asynchronous execution (which in most cases would be the default) depends on your system and requirements.
Can I reduce the number of threads generated using thread pooling?
Thread pooling is often used to reduce the number of threads created, i.e. instead of having a thread per task you have a defined (maximum) number of worker threads and thus if #tasks > max threads in pool you'll reduce the number of threads needed.
From the ThreadPoolExecutor documentation:
A ThreadPoolExecutor will automatically adjust the pool size (see
getPoolSize()) according to the bounds set by corePoolSize (see
getCorePoolSize()) and maximumPoolSize (see getMaximumPoolSize()).
When a new task is submitted in method execute(java.lang.Runnable),
and fewer than corePoolSize threads are running, a new thread is
created to handle the request, even if other worker threads are idle.
If there are more than corePoolSize but less than maximumPoolSize
threads running, a new thread will be created only if the queue is
full. By setting corePoolSize and maximumPoolSize the same, you create
a fixed-size thread pool. By setting maximumPoolSize to an essentially
unbounded value such as Integer.MAX_VALUE, you allow the pool to
accommodate an arbitrary number of concurrent tasks. Most typically,
core and maximum pool sizes are set only upon construction, but they
may also be changed dynamically using setCorePoolSize(int) and
setMaximumPoolSize(int).
Basically, you can set two sizes: the 'core' size and the 'max' size. When tasks are submitted, if there are fewer than 'core' threads, a new thread will be created to execute that task. If there are greater than 'core' threads, one of the current threads will be used to execute tasks, unless all current threads are busy. If all current threads are busy, more threads will be created up to 'max' size. Once 'max' number of threads are reached, no more will be created, and new tasks will be queued until a thread is available to run them.
In the general case, there is no 'right' way that thread pools work. Any given implementation could be used: a fixed size thread pool that always has X threads, or a thread pool that always grows up to a maximum limit, etc.
ThreadPoolExecutor's submitted method throws RejectedExecutionException if the task cannot be scheduled for execution.
I'm trying to use a ThreadPoolExecutor to schedule tasks, but running into some problems with its policies. Here's its stated behavior:
If fewer than corePoolSize threads are running, the Executor always prefers adding a new thread rather than queuing.
If corePoolSize or more threads are running, the Executor always prefers queuing a request rather than adding a new thread.
If a request cannot be queued, a new thread is created unless this would exceed maximumPoolSize, in which case, the task will be rejected.
The behavior I want is this:
same as above
If more than corePoolSize but less than maximumPoolSize threads are running, prefers adding a new thread over queuing, and using an idle thread over adding a new thread.
same as above
Basically I don't want any tasks to be rejected; I want them to be queued in an unbounded queue. But I do want to have up to maximumPoolSize threads. If I use an unbounded queue, it never generates threads after it hits coreSize. If I use a bounded queue, it rejects tasks. Is there any way around this?
What I'm thinking about now is running the ThreadPoolExecutor on a SynchronousQueue, but not feeding tasks directly to it - instead feeding them to a separate unbounded LinkedBlockingQueue. Then another thread feeds from the LinkedBlockingQueue into the Executor, and if one gets rejected, it simply tries again until it is not rejected. This seems like a pain and a bit of a hack, though - is there a cleaner way to do this?
It probably isn't necessary to micro-manage the thread pool as being requested.
A cached thread pool will re-use idle threads while also allowing potentially unlimited concurrent threads. This of course could lead to runaway performance degrading from context switching overhead during bursty periods.
Executors.newCachedThreadPool();
A better option is to place a limit on the total number of threads while discarding the notion of ensuring idle threads are used first. The configuration changes would be:
corePoolSize = maximumPoolSize = N;
allowCoreThreadTimeOut(true);
setKeepAliveTime(aReasonableTimeDuration, TimeUnit.SECONDS);
Reasoning over this scenario, if the executor has less than corePoolSize threads, than it must not be very busy. If the system is not very busy, then there is little harm in spinning up a new thread. Doing this will cause your ThreadPoolExecutor to always create a new worker if it is under the maximum number of workers allowed. Only when the maximum number of workers are "running" will workers waiting idly for tasks be given tasks. If a worker waits aReasonableTimeDuration without a task, then it is allowed to terminate. Using reasonable limits for the pool size (after all, there are only so many CPUs) and a reasonably large timeout (to keep threads from needlessly terminating), the desired benefits will likely be seen.
The final option is hackish. Basically, the ThreadPoolExecutor internally uses BlockingQueue.offer to determine if the queue has capacity. A custom implementation of BlockingQueue could always reject the offer attempt. When the ThreadPoolExecutor fails to offer a task to the queue, it will try to make a new worker. If a new worker can not be created, a RejectedExecutionHandler would be called. At that point, a custom RejectedExecutionHandler could force a put into the custom BlockingQueue.
/** Hackish BlockingQueue Implementation tightly coupled to ThreadPoolexecutor implementation details. */
class ThreadPoolHackyBlockingQueue<T> implements BlockingQueue<T>, RejectedExecutionHandler {
BlockingQueue<T> delegate;
public boolean offer(T item) {
return false;
}
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
delegate.put(r);
}
//.... delegate methods
}
Your use case is common, completely legit and unfortunately more difficult than one would expect. For background info you can read this discussion and find a pointer to a solution (also mentioned in the thread) here. Shay's solution works fine.
Generally I'd be a bit wary of unbounded queues; it's usually better to have explicit incoming flow control that degrades gracefully and regulates the ratio of current/remaining work to not overwhelm either producer or consumer.
Just set corePoolsize = maximumPoolSize and use an unbounded queue?
In your list of points, 1 excludes 2, since corePoolSize will always be less or equal than maximumPoolSize.
Edit
There is still something incompatible between what you want and what TPE will offer you.
If you have an unbounded queue, maximumPoolSize is ignored so, as you observed, no more than corePoolSize threads will ever be created and used.
So, again, if you take corePoolsize = maximumPoolSize with an unbounded queue, you have what you want, no?
Would you be looking for something more like a cached thread pool?
http://download.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/Executors.html#newCachedThreadPool()