I'm going to use ConcurrentLinkedQueue and Java as an example to a more general question. Let me first explain the question with regards to ConcurrentLinkedQueue. Consider:
ConcurrentLinkedQueue<Integer> queue = new ConcurrentLinkedQueue<>();
while (true) {
Integer item = queue.poll();
if (item != null) {
// do some stuff
}
}
ConcurrentLinkedQueue::poll does not block. So if I was to run this code (and only this code) in its own thread. It would constantly do a redundant operation. Compare this to using something like LinkedBlockingQueue::take that blocks until something is available. How much of a difference does it make?
I realize that the question is very vague and specific to the language and data-structure's implementation. But the questions generalizes to something like this:
How resource consuming is it to run a forever loop that does some small, repetitive operation (like queue.poll())?
Because the operation is small but repetitive, each iteration finishes faster but the loop also runs at a higher frequency, which makes me think that it's worse.
As you have an infinite number of operations to execute, scheduled by the while (true) loop, you would bring one CPU core to constantly 100% utilisation with the polling implementation. Which is not a good idea.
On the other hand take blocks the thread until an item is available. The thread can be put in the background, while other threads can be executed on the CPU. A blocked thread does not consume resources. It is wake up, when an item is available and only then scheduled to be executed on the CPU.
The context switch of the scheduling of a background thread might give you a slightly slower reaction time (this depends on the operation system implementation of thread scheduling and handling interrupts), but over all it should be way better than putting constantly 100% utilisation on the CPU.
Related
I have a highly concurrent code that should be able to increment \ decrement a few counters concurrently and read the values. I don't require exact values at every read, so it might as well be eventually consistent. My main objective is that the write operations are non blocking and require no wait time, basically a few threads want to increment the same counter, call some increment function and don't wait for it to be processed.
However, I'm having hard figuring out a way to make such a counter.
I was thinking about using ConcurrentLinkedQueue<Boolean>. Add an element to the queue, and have another thread to pop elements and count increments \ decrements. However, it's not a BlockingQueue so I'd have to make a thread that constantly tries to poll the queue, feels like a huge waste to have a thread fully dedicated to this task. Just asking for size() is not an option because ConcurrentLinkedQueue the operation isn't constant time and every call has to traverse the entire queue, that would be insane waste of time.
I also looked at AtomicInteger but there is only lazy set operation, no lazy increments, incrementAndGet if I understand correctly would result in a locking based increment-read behavior which is definitely not what I need.
Is using a ConcurrentLinkedQueue<Boolean> and a dedicated polling thread my only option for an eventually consistent counter? Especially considering that I do not know how many threads will be trying to write and read this counter at any moment of time, these are spawned dynamically.
Sounds like java.util.concurrent.LongAdder might be what you're looking for:
This class is usually preferable to AtomicLong when multiple threads update a common sum that is used for purposes such as collecting statistics, not for fine-grained synchronization control. Under low update contention, the two classes have similar characteristics. But under high contention, expected throughput of this class is significantly higher, at the expense of higher space consumption.
My main objective is that the write operations are non-blocking and
require no wait time
It makes me think that like you need to introduce kind of async write.
I'd have to make a thread that constantly tries to poll the queue
You may try to create kind of increment-task using CompletableFuture which uses ForkJoinPool (or created dedicated single thread pool) with one thread which will eventually apply all write operations to some AtomicLong so the worker threads will not be blocked:
public class AsyncCounter {
private final AtomicLong counter = new AtomicLong();
// no-blocking here
public CompletableFuture<Long> inc() {
return CompletableFuture.supplyAsync(counter::incrementAndGet);
}
public long get() {
return counter.get();
}
}
But in any case, it should be tested using JMH because it usually happens that our consideration regarding performance is wrong.
I have some misunderstanding with advantages of wait/notify. As i understand processor core will do nothing helpful in both cases so what's the reason tro write complex wait/notify block codes instead of just waiting in cycle?
I'm clear that wait/notify will not steal processor time in case when two threads are executed on only one core.
"Waiting in a cycle" is most commonly referred to as a "busy loop" or "busy wait":
while ( ! condition()) {
// do nothing
}
workThatDependsOnConditionBeingTrue();
This is very disrespectful of other threads or processes that may need CPU time (it takes 100% time from that core if it can). So there is another variant:
while ( ! condition()) {
sleepForShortInterval();
// do nothing
}
workThatDependsOnConditionBeingTrue();
The small sleep in this variant will drop CPU usage dramatically, even if it is ~100ms long, which should not be noticeable unless your application is real-time.
Note that there will generally be a delay between when the condition actually becomes true and when sleepForShortInterval() ends. If, to be more polite to others, you sleep longer -- the delay will increase. This is generally unacceptable in real-time scenarios.
The nice way to do this, assuming that whatever condition() is checking is being changed from another thread, is to have the other thread wake you up when it finishes whatever you are waiting for. Cleaner code, no wasted CPU, and no delays.
Of course, it's quicker to implement a busy wait, and it may justified for quick'n'dirty situations.
Beware that, in a multithreaded scenario where condition() can be changed to false as well as true, you will need to protect your code between the while and the workThatDependsOnConditionBeingTrue() to avoid other threads changing its value in this precise point of time (this is called a race codition, and is very hard to debug after the fact).
I think you answered your question almost by saying
I'm clear that wait/notify will not steal processor time in case.
Only thing I would add is, this true irrespective of one core or multi-core. wait/notify wont keep the cpu in a busy-wait situation compared to while loop or periodic check.
what's the reason not to run core but wait? There's no helpful work in any case and you're unable to use core when it's in waiting state.
I think you are looking at it from a single application perspective where there is only one application with one thread is running. Think of it from a real world application (like web/app servers or standalone) where there are many threads running and competing for cpu cycles - you can see the advantage of wait/notify. You would definitely not want even a single thread to just do a busy-wait and burn the cpu cycles.
Even if it a single application/thread running on the system there are always OS process running and its related processes that keep competing for the CPU cycles. You don't want them to starve them because the application is doing a while busy-wait.
Quoting from Gordon's comment
waiting in cycle as you suggest you are constantly checking whether the thing you are waiting for has finished, which is wasteful and if you use sleeps you are just guessing with timing, whereas with wait/notify you sit idle until the process that you are waiting on tells you it is finished.
In general, your application is not the only one running on the CPU. Using non-spinning waiting is, first of all, an act of courtesy towards the other processes/threads which are competing for the CPU in order to do some useful job. The CPU scheduler cannot know a-priori if your thread is going to do something useful or just spin on a false flag. So, it can't tune itself based on that, unless you tell it you don't want to be run, because there's nothing for you to do.
Indeed, busy-waiting is faster than getting the thread to sleep, and that's why usually the wait() method is implemented in a hybrid way. It first spins for a while, and then it actually goes to sleep.
Besides, it's not just waiting in a loop. You still need to synchronize access to the resources you're spinning on. Otherwise, you'll fall victim of race conditions.
If you feel the need of a simpler interface, you might also consider using CyclicBarrier, CountDownLatch or a SynchronousQueue.
There's a list of things that my program needs to periodically check - no events can be assigned to trigger when their state changes. These things are stored in array list as Robot class instances:
public class RobotManager extends Thread {
protected final List<Robot> robots = new ArrayList<>();
}
Every robot has canRun task which returns true if there's someting the robot can do. This includes updating availability of GUI buttons and so on.
My current plan was to sleep for some while, like 800ms, then loop through list and canRun (and eventually start()) every Robot in the list. But this doesn't seem very nice - if there's sufficient number of tasks, the program will lag the system every 800ms. It would be much nicer if the program could:
Tell the OS to sleep for something around 800ms with less precision and try to run where there are spare resources
Do these unprecise sleeps while looping the list to reduce the peak in required resources.
In other words: Can I, in Java, make sleep less precise in favour of running when system has spare resources?
I think you are looking for the Thread.yield()method.
Javadoc:
A hint to the scheduler that the current thread is willing to yield
its current use of a processor. The scheduler is free to ignore this
hint.
Yield is a heuristic attempt to improve relative progression
between threads that would otherwise over-utilise a CPU. Its use
should be combined with detailed profiling and benchmarking to
ensure that it actually has the desired effect.
It is rarely appropriate to use this method. It may be useful
for debugging or testing purposes, where it may help to reproduce
bugs due to race conditions. It may also be useful when designing
concurrency control constructs such as the ones in the
java.util.concurrent.locks package.
With a combination of sleep(...) and yield() you can find a tradeoff between "robots list is not processed often enough" and "it's eating up to much cpu". The amount of time you sleep and the number of yield calls (within the robots and/or between robot calls) depends on the stuff your robots actually do.
What you should do is to set the process/thread priority to Idle or very low priority. A thread/process that has an Idle priority will only be scheduled if no other tasks with higher priority is ready to run. Note that this opens the possibility of starvation if the current machine is actively very busy, the idle thread won't run at all. A low priority thread would still let you get some time slice, only that it'll yield to higher priority threads first. The specific behaviour of thread priority varies depending on the JVM implementation and the OS, but generally a low priority thread will likely be preempted if a higher priority thread becomes ready to run, and it will be less likely to be scheduled if a higher priority thread is ready to run.
Another comment is that I'd recommend to avoid polling for available task, but rather use a BlockingQueue instead of ArrayList and sleeping. If your thread is waiting for a BlockingQueue, it won't be scheduled until there's something in the queue, so you don't have the unpredictable wake-up checks. It's also nicer to the machine as a blocked thread would allow the CPU to enter low power mode (unlike a constantly waking thread, which keeps the CPU at its toes), this can be important if your program is running on a machine with battery.
I have read many similar questions . However I was not quite satisfied with answers.
I would like to build an algorithm that would adjust the number of threads depending on the average speed.
Let's say as I introduce a new thread, the average speed of task execution increases , it means that the new thread is good. Then the algorithm should try to add another thread ... until the optimal number of threads is achieved .......
Also the algorithm should be keeping track of the average speed. If at some point the average speed goes down significantly, let's say by 10 % (for any reason e.g. i open a different application or whatever) , then the algorithm should terminate one thread and see if the speed goes up ...
Maybe such an API exists. Please, give me any directions or any code example how I could implement such an algorithm
Thank You !
I do not know self-tune system that you are describing but it sounds like not so complicated task once you are using ready thread pool. Take thread pool from concurrency package, implement class TimeConsumptionCallable implements Callable that wraps any other callable and just measures the execution time.
Now you just have to change (increase or decrease) number of working threads when average execution time increases or decreases.
Just do not forget that you need enough statistics before you decide to change number of working threads. Otherwise various random effects that do not depend on your application can cause your thread pool to grow and go down all the time that can itself kill overall performance.
newCachedThreadPool() V/s newFixedThreadPool suggests that perhaps you should be looking at ExecutorService.newCachedThreadPool().
Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available. These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks. Calls to execute will reuse previously constructed threads if available. If no existing thread is available, a new thread will be created and added to the pool. Threads that have not been used for sixty seconds are terminated and removed from the cache. Thus, a pool that remains idle for long enough will not consume any resources. Note that pools with similar properties but different details (for example, timeout parameters) may be created using ThreadPoolExecutor constructors.
If your threads do not block at any time, then the maximum execution speed is reached when you have as many threads as cores, as simply more than 100% CPU usage is not possible.
In other situations it is very difficult to measure how much a new thread will increase/decrease the execution speed, as you just watch a moment in time and make assumptions based on something that could be entirely different the next second.
One idea would be to use an Executor class in combination with a Queue that you specified. So you can measure the size of the queue and make assumptions based on that. If the queue is empty, threads are idle and you can remove one. If the queue fills up, threads cannot handle the load, you need to add more. If the queue is stable, you are about right.
You can come up with your own algorithm by using existing API of java :
public void setCorePoolSize(int corePoolSize) in ThreadPoolExecutor
Sets the core number of threads. This overrides any value set in the constructor.
If the new value is smaller than the current value, excess existing threads will be terminated when they next become idle.
If larger, new threads will, if needed, be started to execute any queued tasks.
Initialization:
ExecutorService service = Executors.newFixedThreadPool(5); // initializaiton
On your need basis, resize the pool by using below API
((ThreadPoolExecutor)service).setCorePoolSize(newLimit);//newLimit is new size of the pool
And one important point: If the queue is full, and new value of number of threads is greater than or equal to maxPoolSize defined earlier, Task will be rejected.
Be careful when setting maxPoolSize so that setCorePoolSize works properly.
What's the proper way for a Java command line application to do background work without hogging resources? Should it use sleep() in the loop or is there a more elegant/efficient way?
Some heuristics:
Don't attempt to make scheduling decisions in your application. The operating system's scheduler is way better than yours will be. Let it do its job.
Don't poll if you don't have to. For instance, instead of sleeping n seconds, then waking up to check a non-blocked socket, block on the socket. This second strategy plays better with the operating system's scheduler.
Don't use an enormous heap if you don't have to, and try not to allocate enormous chunks of memory at one time. A thrashing application tends to have a negative effect on system performance.
Use buffered I/O. Always. If you think you need non-buffered I/O, be absolutely sure you're right. (You're probably wrong.)
Don't spawn a lot of threads. Threads are surprisingly expensive; beyond a certain point, more threads will reduce your application's performance profile. If you have lots of work to do concurrently, learn and use java.util.concurrent.
Of course, this is just a starter list...
I'd only use sleep() if there's no work to be done. For example, if you're doing something like polling a task queue periodically and there's nothing there, sleep for a while then check again, etc.
If you're just trying to make sure you don't hog the CPU but you're still doing real work, you could call Thread.yield() periodically. That will relinquish control of the CPU and let other threads run, but it won't put you to sleep. If other processes don't need the CPU you'll get control back and continue to do your work.
You can also set your thread to a low priority:
myThread.setPriority(Thread.MIN_PRIORITY);
As Ishmael said, don't do this in your main thread. Create a "worker thread" instead. That way your UI (GUI or CLI) will still be responsive.
There are several ways. I would use ExecutorService... for example:
ExecutorService service = Executors.newCachedThreadPool();
Callable<Result> task = new Callable<Result>() {
public Result call() throws Exception {
// code which will be run on background thread
}
};
Future<Result> future = service.submit(task);
// Next line wait until background task is complete
// without killing CPU. Of course, you can do something
// different here and check your 'future' later.
//
// Note also that future.get() may throw various exceptions too,
// you'll need to handle them properly
Result resultFromBackgroundThread = future.get();
This is Java 5 code, ExecutorService, Callable, Future and similar are in java.util.concurrent package.
One place to start is to make sure that only those resources are being used and no other objects (so that they become garbage collected).
Placing sleep() in a single threading application is only going to halt the current thread. If you're trying to accomplish data being processed in the background while information still needs to be presented to the user then it is best to put the background process in a seperate thread.