The ThreadPoolExecutor class in the Java SE 6 docs has the following method:
public int getActiveCount()
Returns the approximate number of threads
that are actively executing tasks.
What is the meaning of approximate and actively executing here?
Is there any guarantee that, if before, during and after the call to getActiveCount()
N threads have been allotted from the pool for task execution, and
None of these N threads are available for further task assignment,
the integer returned by getActiveCount() will be exactly N?
If getActiveCount() does not provide this guarantee, is there any other way to obtain this information in a more precise manner?
Prior SO Questions:
I have looked at Thread Pool Executor Monitoring Requirement and How to tell if there is an available thread in a thread pool in java, but they do not answer my queries.
The reason that it is approximate is because the number could change during the calculation; you're multi-threading. Before the calculation completes a different number of threads could now be active (A thread that was inactive when checked is now active).
When you say "particular time instance" ... that doesn't really mean anything. The calculation isn't instantaneous. The number you get back is the best possible answer given the fluid/dynamic nature of the pool.
If by chance the calculation starts and completes while none of the threads in the pool change state, then yes that number is "exact" but only until a thread in the pool changes state, which means it might only be "exact" for 1ms (or less).
I think you're possibly confusing things by introducing a notion of "rejoining the pool" that doesn't really exist in the implementation of ThreadPoolExecutor.
Each worker thread sits continually waiting for a task (it effectively sits on top of a blocking queue). Each a task comes in to its queue, that worker is "locked", then any pre-task housekeeping is run, then the actual task is run, then post-task housekeeping, then the worker is "unlocked".
activeCount() gives you the number of threads in the "locked" state: notice that this means that they could actually be conducting 'housekeeping' at the precise moment of calling activeCount(), but that to be counted as 'active', there must be a task actually involved, either about to be, currently, or having just been executed.
Whether that equates with your notion of "rejoining the pool" I'm not sure-- as I say, you seem to be inventing a notion that strictly speaking doesn't exist from the point of view of ThreadPoolExecutor.
Related
I'm trying to run a number of jobs concurrently using Java's ForkJoinPool. The main task (which is already running in the pool) spawns all the jobs and then does a series of joins. I was sure that a task calling join would free the thread it is running in, but it seems like it is actually blocked on it, and therefore it is "wasting" the thread, i.e., since the number of threads equals the number of CPU cores, one core will be inactive.
I know that if I run invokeAll instead, then the first of the sub-jobs gets to run in the same thread, and indeed this works. However, this seems sub-optimal, because if the first task is actually a very fast one, i have the same problem. One of the threads is blocked waiting on join. There are more jobs than threads, so I would rather another one of the jobs gets started.
I can try and bypass all this manually but its not so nice, and it seems like I am redoing what ForkJoinPool is supposed to do.
So the question is: Am I understanding ForkJoinPool wrong? or if what I'm saying is correct, then is there simple way to utilize the threads more efficiently?
ForkJoinPool is designed to prevent you having to think about thread utilization in this way. The 'work stealing' algorithm ensures that each thread is always busy so long as there are tasks in the queue.
Check out these notes for a high-level discussion:
https://www.dre.vanderbilt.edu/~schmidt/cs891f/2018-PDFs/L4-ForkJoinPool-pt3.pdf
To see the ugly details go down the rabbit hole of the ForkJoinPool#awaitJoin source.
Roughly, if I'm reading the (very complex) code correctly: When a thread joins a sub-task, it attempts to complete that task itself, otherwise if the sub-task's worker queue is non-empty (i.e. it is also depending on other tasks), the joining thread repeatedly attempts to complete one of those tasks, via ForkJoinPool#tryHelpStealer, whose Javadoc entry provides some insight:
Tries to locate and execute tasks for a stealer of the given
task, or in turn one of its stealers, Traces currentSteal ->
currentJoin links looking for a thread working on a descendant
of the given task and with a non-empty queue to steal back and
execute tasks from. The first call to this method upon a
waiting join will often entail scanning/search, (which is OK
because the joiner has nothing better to do), but this method
leaves hints in workers to speed up subsequent calls. The
implementation is very branchy to cope with potential
inconsistencies or loops encountering chains that are stale,
unknown, or so long that they are likely cyclic.
Notice that ForkJoinTask does not extend Thread, so 'blocking' of the join operation means something different here than usual. It doesn't mean that the underlying thread is in a blocked state, rather it means that the computation of the current task is held up further up the call stack while join goes off and attempts to resolve the tree of sub-tasks impeding progress.
Could you explain me, why in ScheduledThreadPoolExecutor javadoc is this:
Additionally, it is almost never a good idea to set corePoolSize to
zero or use allowCoreThreadTimeOut because this may leave the pool without
threads to handle tasks once they become eligible to run.
I've tried to analyze how new threads are created in this thread pool when a new task has to be executed and I think problem described in javadoc shouldn't happen.
The thread pool trys to make the number of work threads equals corePoolSize, to improve efficiency by caching threads. Allowing core thread time out is contrary to this purpose. If you allow core thread time out, new task will be executed, but leads to repeatly create and destroy work threads.
If you set allowCoreThreadTimeOut = true, then after the work thread find no task in task queue and time out, they will be destroyed even the number of working threads is less than corePoolSize. So, if you submit new task at this time, the thread pool has to create new thread.
If you set allowCoreThreadTimeOut = false, then after the work thread find no task in task queue and number of work threads less than corePoolSize, they will be not be destroyed and keep waiting for new task.
My guess is that the answer is stale Javadoc, for the most part. As you note, ensurePrestart ensures that as long as corePoolSize > 0, the number of core pool threads is nonzero after the call. This has been the case since https://github.com/openjdk/jdk/commit/2d19ea519b17529a083a62eb219da532693bbef3, but notably that commit did not update the Javadoc on ScheduledThreadPoolExecutor.
However, the details aren't quite so simple either. Rather than only worrying about new task submissions and task reschedule on completion, you also need to worry about core pool threads idling out because all scheduled tasks are too far in the future to trigger before the pool timeout.
Not sure if I'm reading the JRE code correctly, but it looks like in such a case, the pool will:
Start a worker thread (due to ensurePrestart)
Regardless of thread type (core or not), the thread is eligible for timeout because allowCoreThreadTimeout is true
Worker thread polls DelayedWorkQueue for next task, with timeout
poll returns null (times out), because the next scheduled task is beyond pool timeout
The core thread will terminate because it thinks there is nothing to do
ThreadPoolExecutor.processWorkerExit will run on worker termination. It will check the queue, notice that it's nonempty, and thus require a minimum of at least one thread
If the thread being terminated is the last thread, it will notice that the minimum is not met and immediately start a new (non-core) worker
Repeat from step 1
So, the pool will work as you intend, but probably won't be in an ideal state either (you really want a single core thread polling without timeout here, not threads constantly spinning up, polling with timeout, timing out, then starting a new thread to replace itself). In that sense, step 6 is really what prevents the case mentioned in Javadoc (by the time tasks are eligible, all pool threads have timed out), but it does so imperfectly because of the unnecessary thread creation/destruction loop.
This weirdness is really because DelayedWorkQueue semantically breaks the BlockingQueue contract. All else equal, you'd assume that size() > 0 implies that a subsequent poll(...) will successfully retrieve an element and not time out, but DelayedWorkQueue allows those elements to be held back for some time (even though they're already visible via size() and isEmpty()).
NOTE: Seems like this code changed in Java 8, with the addition of this condition to ThreadPoolExecutor.getTask(). This will keep that last worker thread alive and avoid the thread create/destroy loop, but it will busy-poll the work queue for work instead.
When I have this code in an application:
Executors.newFixedThreadPool(4);
but I never use this thread pool. Will the idle threads consume CPU time? If so - why?
No, these threads are created lazily, or 'on-demand'. As stated in the documentation (emphasis mine):
On-demand construction
By default, even core threads are initially created and started only when new tasks arrive
Java provides methods to override this default and allow for eager creation, namely prestartCoreThread and prestartAllCoreThreads.
Once threads are actually created, idle ones (generally) won't take CPU time as there is no reason for them to be scheduled on a core when they have no work to do.
They will still hold on to some memory however, for their stack and whatnot.
The javadoc states:
Creates a thread pool that reuses a fixed number of threads operating off a shared unbounded queue. At any point, at most nThreads threads will be active processing tasks.
This might lead to assume: we don't know exactly. But as the other answer clearly finds - we can know, and the implementation is actually fully lazy. Thus:
ExecutorService service = Executors.newFixedThreadPool(4);
does not even cause a lot of setup. The implementation would be free to wait for any
service.submit(...
to happen.
On the other hand, that thread pool could (theoretically) be created immediately, and also 4 OS threads could be created. If the later is the case, all those threads would be idle'ing, so they should not consume any CPU resources.
But of course, as Elliott is correctly pointing out, that initial step of creating the pool and (potentially) creating 1 to 4 threads requires CPU activity.
And of course: threads are a OS resource. So even when they "only exist" and do nothing; they have that "cost". Where again, it might depend on the OS if that could ever turn into a problem (like reaching some "maximum number of existing threads" limit).
Finally: as this got me curious, I had a look into the current Java8 source code (from newFixedThreadPoo() and ThreadPoolExcecutor() down into DefaultThreadFactory). If I am not mistaken: those constructors only prepare for thread creation.
So: the "current" implementation is "fully" lazy; and if you really only call newFixedThreadPool() without ever using the resulting ExecutorService ... nothing happens (in terms of new threads being created).
I have read many similar questions . However I was not quite satisfied with answers.
I would like to build an algorithm that would adjust the number of threads depending on the average speed.
Let's say as I introduce a new thread, the average speed of task execution increases , it means that the new thread is good. Then the algorithm should try to add another thread ... until the optimal number of threads is achieved .......
Also the algorithm should be keeping track of the average speed. If at some point the average speed goes down significantly, let's say by 10 % (for any reason e.g. i open a different application or whatever) , then the algorithm should terminate one thread and see if the speed goes up ...
Maybe such an API exists. Please, give me any directions or any code example how I could implement such an algorithm
Thank You !
I do not know self-tune system that you are describing but it sounds like not so complicated task once you are using ready thread pool. Take thread pool from concurrency package, implement class TimeConsumptionCallable implements Callable that wraps any other callable and just measures the execution time.
Now you just have to change (increase or decrease) number of working threads when average execution time increases or decreases.
Just do not forget that you need enough statistics before you decide to change number of working threads. Otherwise various random effects that do not depend on your application can cause your thread pool to grow and go down all the time that can itself kill overall performance.
newCachedThreadPool() V/s newFixedThreadPool suggests that perhaps you should be looking at ExecutorService.newCachedThreadPool().
Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available. These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks. Calls to execute will reuse previously constructed threads if available. If no existing thread is available, a new thread will be created and added to the pool. Threads that have not been used for sixty seconds are terminated and removed from the cache. Thus, a pool that remains idle for long enough will not consume any resources. Note that pools with similar properties but different details (for example, timeout parameters) may be created using ThreadPoolExecutor constructors.
If your threads do not block at any time, then the maximum execution speed is reached when you have as many threads as cores, as simply more than 100% CPU usage is not possible.
In other situations it is very difficult to measure how much a new thread will increase/decrease the execution speed, as you just watch a moment in time and make assumptions based on something that could be entirely different the next second.
One idea would be to use an Executor class in combination with a Queue that you specified. So you can measure the size of the queue and make assumptions based on that. If the queue is empty, threads are idle and you can remove one. If the queue fills up, threads cannot handle the load, you need to add more. If the queue is stable, you are about right.
You can come up with your own algorithm by using existing API of java :
public void setCorePoolSize(int corePoolSize) in ThreadPoolExecutor
Sets the core number of threads. This overrides any value set in the constructor.
If the new value is smaller than the current value, excess existing threads will be terminated when they next become idle.
If larger, new threads will, if needed, be started to execute any queued tasks.
Initialization:
ExecutorService service = Executors.newFixedThreadPool(5); // initializaiton
On your need basis, resize the pool by using below API
((ThreadPoolExecutor)service).setCorePoolSize(newLimit);//newLimit is new size of the pool
And one important point: If the queue is full, and new value of number of threads is greater than or equal to maxPoolSize defined earlier, Task will be rejected.
Be careful when setting maxPoolSize so that setCorePoolSize works properly.
I have a bit of an issue with an application running multiple Java threads.
The application runs a number of working threads that peek continuously at an input queue and if there are messages in the queue they pull them out and process them.
Among those working threads there is another verification thread scheduled to perform at a fixed period a check to see if the host (on which the application runs) is still in "good shape" to run the application. This thread updates an AtomicBoolean value which in turn is verified by the working thread before they start peeking to see if the host is OK.
My problem is that in cases with high CPU load the thread responsible with the verification will take longer because it has to compete with all the other threads. If the AtomicBoolean does not get updated after a certain period it is automatically set to false, causing me a nasty bottleneck.
My initial approach was to increase the priority of the verification thread, but digging into it deeper I found that this is not a guaranteed behavior and an algorithm shouldn't rely on thread priority to function correctly.
Anyone got any alternative ideas? Thanks!
Instead of peeking into a regular queue data structure, use the java.util.concurrent package's LinkedBlockingQueue.
What you can do is, run an pool of threads (you could use executer service's fixed thread pool, i.e., a number of workers of your choice) and do LinkedBlockingQueue.take().
If a message arrives at the queue, it is fed to one of the waiting threads (yeah, take does block the thread until there is something to be fed with).
Java API Reference for Linked Blocking Queue's take method
HTH.
One old school approach to throttling rate of work, that does not use a health check thread at all (and so by-passes these problems) is to block or reject requests to add to the queue if the queue is longer than say 100. This applies dynamic back pressure on to the clients generating the load, slowing them down when the worker threads are over loaded.
This approach was added to the Java 1.5 library, see java.util.concurrent.ArrayBlockingQueue. Its put(o) method blocks if the queue is full.
Are u using Executor framework (from Java's concurrency package)? If not give it a shot. You could try using ScheduledExecutorService for the verification thread.
More threads does not mean better performance. Usually if you have dual core, 2 threads gives best performance, 3 or more starts getting worse. Quad core should handle 4 threads best, etc. So be careful how much threads you use.
You can put the other threads to sleep after they perform their work, and allow other threads to do their part. I believe Thread.yield() will pause the current thread to give time to other threads.
If you want your thread to run continuously, I would suggest creating two main threads, thread A and B. Use A for the verification thread, and from B, create the other threads. Therefore thread A gets more execution time.
Seems you need to utilize Condition variables. Peeking will take cpu cycles.
http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/Condition.html