I have some misunderstanding with advantages of wait/notify. As i understand processor core will do nothing helpful in both cases so what's the reason tro write complex wait/notify block codes instead of just waiting in cycle?
I'm clear that wait/notify will not steal processor time in case when two threads are executed on only one core.
"Waiting in a cycle" is most commonly referred to as a "busy loop" or "busy wait":
while ( ! condition()) {
// do nothing
}
workThatDependsOnConditionBeingTrue();
This is very disrespectful of other threads or processes that may need CPU time (it takes 100% time from that core if it can). So there is another variant:
while ( ! condition()) {
sleepForShortInterval();
// do nothing
}
workThatDependsOnConditionBeingTrue();
The small sleep in this variant will drop CPU usage dramatically, even if it is ~100ms long, which should not be noticeable unless your application is real-time.
Note that there will generally be a delay between when the condition actually becomes true and when sleepForShortInterval() ends. If, to be more polite to others, you sleep longer -- the delay will increase. This is generally unacceptable in real-time scenarios.
The nice way to do this, assuming that whatever condition() is checking is being changed from another thread, is to have the other thread wake you up when it finishes whatever you are waiting for. Cleaner code, no wasted CPU, and no delays.
Of course, it's quicker to implement a busy wait, and it may justified for quick'n'dirty situations.
Beware that, in a multithreaded scenario where condition() can be changed to false as well as true, you will need to protect your code between the while and the workThatDependsOnConditionBeingTrue() to avoid other threads changing its value in this precise point of time (this is called a race codition, and is very hard to debug after the fact).
I think you answered your question almost by saying
I'm clear that wait/notify will not steal processor time in case.
Only thing I would add is, this true irrespective of one core or multi-core. wait/notify wont keep the cpu in a busy-wait situation compared to while loop or periodic check.
what's the reason not to run core but wait? There's no helpful work in any case and you're unable to use core when it's in waiting state.
I think you are looking at it from a single application perspective where there is only one application with one thread is running. Think of it from a real world application (like web/app servers or standalone) where there are many threads running and competing for cpu cycles - you can see the advantage of wait/notify. You would definitely not want even a single thread to just do a busy-wait and burn the cpu cycles.
Even if it a single application/thread running on the system there are always OS process running and its related processes that keep competing for the CPU cycles. You don't want them to starve them because the application is doing a while busy-wait.
Quoting from Gordon's comment
waiting in cycle as you suggest you are constantly checking whether the thing you are waiting for has finished, which is wasteful and if you use sleeps you are just guessing with timing, whereas with wait/notify you sit idle until the process that you are waiting on tells you it is finished.
In general, your application is not the only one running on the CPU. Using non-spinning waiting is, first of all, an act of courtesy towards the other processes/threads which are competing for the CPU in order to do some useful job. The CPU scheduler cannot know a-priori if your thread is going to do something useful or just spin on a false flag. So, it can't tune itself based on that, unless you tell it you don't want to be run, because there's nothing for you to do.
Indeed, busy-waiting is faster than getting the thread to sleep, and that's why usually the wait() method is implemented in a hybrid way. It first spins for a while, and then it actually goes to sleep.
Besides, it's not just waiting in a loop. You still need to synchronize access to the resources you're spinning on. Otherwise, you'll fall victim of race conditions.
If you feel the need of a simpler interface, you might also consider using CyclicBarrier, CountDownLatch or a SynchronousQueue.
Related
I'm writing a backtesting raw data collector for my crypto trading bot and I've run into a weird optimization issue.
I constantly have 30 runnables in an Executors.newCachedThreadPool() running get requests from an API. Since the API has a request limit of 1200 per minute I have this bit of code in my runnable:
while (minuteRequests.get() >= 1170) {
Thread.onSpinWait();
}
Yes, minuteRequests is an AtomicInteger, so I'm not running into any issues there.
Everything works, the issue is that even though I'm using the recommended busy-waiting onSpinWait method, I shoot from 24% CPU usage or so to 100% when the waiting is initiated. For reference I'm running this on a 3900X (24 thread).
Any recommendations on how to better handle this situation?
My recommendation would be to not do busy waiting at all.
The javadocs for Thread.onSpinWait say this:
Indicates that the caller is momentarily unable to progress, until the occurrence of one or more actions on the part of other activities. By invoking this method within each iteration of a spin-wait loop construct, the calling thread indicates to the runtime that it is busy-waiting. The runtime may take action to improve the performance of invoking spin-wait loop constructions.
Note the highlighted section uses the word may rather than will. That means that it also may not do anything. Also "improve the performance" does not mean that your code will be objectively efficient.
The javadoc also implies that the improvements may be hardware dependent.
In short, this is the right way to use onSpinwait ... but you are expecting too much of it. It won't make your busy-wait code efficient.
So what would I recommend you actually do?
I would recommend that you replace the AtomicInteger with a Semaphore (javadoc). This particular loop would be replaced by the following:
semaphore.acquire();
This blocks1 until 1 "permit" is available and acquires it. Refer to the class javadocs for an explanation of how semaphores work.
Note: since you haven't show us the complete implementation of your rate limiting, it is not clear how your current approach actually works. Therefore, I can't tell you exactly how to replace AtomicInteger with Semaphore throughout.
1 - The blocked thread is "parked" until some other thread releases a permit. While it is parked, the thread does not run and is not associated with a CPU core. The core is either left idle (typically in a low power state) or it is assigned to some other thread. This is typically handled by the operating system's thread scheduler. When another thread releases a permit, the Semaphore.release method will tell the OS to unpark one of the threads that is blocked in acquire.
I have a consumer thread blocking on removing from a queue.
There are going to be periods during which I know nothing will be added to the queue.
My question is: is it worth adding the complexity of managing when to start/stop the thread, or should I just leave it waiting until queue starts getting elements again?
If the concurrent queue implementation that you're using is worth it's salt then the thread will not be busy-waiting for very long. Some implementations may do this briefly for performance reasons but after that then it will block and will not be consuming CPU cycles. Therefore the difference between a stopped thread and a blocked thread becomes more or less meaningless.
Use a concurrent queue. See Which concurrent Queue implementation should I use in Java?
When dealing with Multithreading its a best practice to just act when you have a performance problem. Otherwise I would just leave it like it is to avoid trouble.
I dont think there is a big impact on the performance since the thread is blocked (inactive waiting). It could make sense if the thread is holding expensive resources which can be released for that time. I would keep this as simple as possible, especially in a concurrent enviroment complexity can lead to strange errors.
There's a list of things that my program needs to periodically check - no events can be assigned to trigger when their state changes. These things are stored in array list as Robot class instances:
public class RobotManager extends Thread {
protected final List<Robot> robots = new ArrayList<>();
}
Every robot has canRun task which returns true if there's someting the robot can do. This includes updating availability of GUI buttons and so on.
My current plan was to sleep for some while, like 800ms, then loop through list and canRun (and eventually start()) every Robot in the list. But this doesn't seem very nice - if there's sufficient number of tasks, the program will lag the system every 800ms. It would be much nicer if the program could:
Tell the OS to sleep for something around 800ms with less precision and try to run where there are spare resources
Do these unprecise sleeps while looping the list to reduce the peak in required resources.
In other words: Can I, in Java, make sleep less precise in favour of running when system has spare resources?
I think you are looking for the Thread.yield()method.
Javadoc:
A hint to the scheduler that the current thread is willing to yield
its current use of a processor. The scheduler is free to ignore this
hint.
Yield is a heuristic attempt to improve relative progression
between threads that would otherwise over-utilise a CPU. Its use
should be combined with detailed profiling and benchmarking to
ensure that it actually has the desired effect.
It is rarely appropriate to use this method. It may be useful
for debugging or testing purposes, where it may help to reproduce
bugs due to race conditions. It may also be useful when designing
concurrency control constructs such as the ones in the
java.util.concurrent.locks package.
With a combination of sleep(...) and yield() you can find a tradeoff between "robots list is not processed often enough" and "it's eating up to much cpu". The amount of time you sleep and the number of yield calls (within the robots and/or between robot calls) depends on the stuff your robots actually do.
What you should do is to set the process/thread priority to Idle or very low priority. A thread/process that has an Idle priority will only be scheduled if no other tasks with higher priority is ready to run. Note that this opens the possibility of starvation if the current machine is actively very busy, the idle thread won't run at all. A low priority thread would still let you get some time slice, only that it'll yield to higher priority threads first. The specific behaviour of thread priority varies depending on the JVM implementation and the OS, but generally a low priority thread will likely be preempted if a higher priority thread becomes ready to run, and it will be less likely to be scheduled if a higher priority thread is ready to run.
Another comment is that I'd recommend to avoid polling for available task, but rather use a BlockingQueue instead of ArrayList and sleeping. If your thread is waiting for a BlockingQueue, it won't be scheduled until there's something in the queue, so you don't have the unpredictable wake-up checks. It's also nicer to the machine as a blocked thread would allow the CPU to enter low power mode (unlike a constantly waking thread, which keeps the CPU at its toes), this can be important if your program is running on a machine with battery.
I have a server program which polls a database for new requests , I want this polling to be done at 1 minute intervals so , I've set up a Thread.sleep() in the program while loop.
The problem is that whenever this program is supposed to "sleep" the CPU consumption goes up drastically (viz. about 25 - 30%).
Paradoxically, when the program is not dormant and is busy processing requests , the CPU consumption drops to 0.4%.
I read online and found out that there are performance hits associated with thread.sleep, but I could not find any viable alternative (Thread.wait requires notification on an object, something which I feel is useless in my scenario)
The main loop (when there are no new requests) doesn't do anything, here is a skeleton of all that is being done when the CPU consumption is 25%
-> poll
-> No new records ?
-> Sleep
->repeat
Check what the CPU consumption is for individual CPU cores. If you are using a 4 core machine, maybe one thread is going rogue and is eating up once core (25%). This usually happens when the thread is in a tight loop.
You could use Thread.wait with a timeout (which indeed the Timer class does), but my bet is that it won't make any difference. Both Thread.sleep and Thread.wait changes the threads' state to not runnable. Although it depends on your JVM implementation etc., the thread shouldn't consume that much CPU in such situation. So my bet is that there is some bug at work.
Another thing you can do is taking a thread dump and see what the thread is doing when this happens. Use kill -3 on a Linux box, or use ctrl+break on the java console window if you are using Windows. Then, examine the thread dump that is dumped to the standard output. Then you can be sure if the thread was actually sleeping or was doing something else.
As many people pointed out, Thread.sleep should and actually does help with dropping the CPU usage drastically.
I omitted certain facts from my original question as I thought they were not relevant.
The main thread was the producer, there was another thread running asynchronously which was the consumer. It turns out that the "sleep" on this thread was inside some weird condition that wasn't getting triggered properly. So the loop on that thread was never sleeping.
Once the sleep thing was eliminated I went ahead and analyzed it closely to realize the problem.
Inline Java IDE hint states, "Invoking Thread.sleep in loop can cause performance problems." I can find no elucidation elsewhere in the docs re. this statement.
Why? How? What other method might there be to delay execution of a thread?
It is not that Thread.sleep in a loop itself is a performance problem, but it is usually a hint that you are doing something wrong.
while(! goodToGoOnNow()) {
Thread.sleep(1000);
}
Use Thread.sleep only if you want to suspend your thread for a certain amount of time. Do not use it if you want to wait for a certain condition.
For this situation, you should use wait/notify instead or some of the constructs in the concurrency utils packages.
Polling with Thread.sleep should be used only when waiting for conditions external to the current JVM (for example waiting until another process has written a file).
It depends on whether the wait is dependent on another thread completing work, in which case you should use guarded blocks, or high level concurrency classes introduced in Java 1.6. I recently had to fix some CircularByteBuffer code that used Thread sleeps instead of guarded blocks. With the previous method, there was no way to ensure proper concurrency. If you just want the thread to sleep as a game might, in the core game loop to pause execution for a certain amount of time so that over threads have good period in which to execute, Thread.sleep(..) is perfectly fine.
It depends on why you're putting it to sleep and how often you run it.
I can think of several alternatives that could apply in different situations:
Let the thread die and start a new one later (creating threads can be expensive too)
Use Thread.join() to wait for another thread to die
Use Thread.yield() to allow another thread to run
Let the thread run but set it to a lower priority
Use wait() and notify()
http://www.jsresources.org/faq_performance.html
1.6. What precision can I expect from Thread.sleep()?
The fundamental problem with short sleeps is that a call to sleep finishes the current scheduling time slice. Only after all other threads/process finished, the call can return.
For the Sun JDK, Thread.sleep(1) is reported to be quite precise on Windows. For Linux, it depends on the timer interrupt of the kernel. If the kernel is compiled with HZ=1000 (the default on alpha), the precision is reported to be good. For HZ=100 (the default on x86) it typically sleeps for 20 ms.
Using Thread.sleep(millis, nanos) doesn't improve the results. In the Sun JDK, the nanosecond value is just rounded to the nearest millisecond. (Matthias)
why? that is because of context switching (part of the OS CPU scheduling)
How? calling Thread.sleep(t) makes the current thread to be moved from the running queue to the waiting queue. After the time 't' reached the the current thread get moved from the waiting queue to the ready queue and then it takes some time to be picked by the CPU and be running.
Solution: call Thread.sleep(t*10); instead of calling Thread.Sleep(t) inside loop of 10 iterations ...
I have face this problem before when waiting for asynchronous process to return a result.
Thread.sleep is a problem on multi thread scenario. It tends to oversleep. This is because internally it rearrange its priority and yields to other long running processes (thread).
A new approach is using ScheduledExecutorService interface or the ScheduledThreadPoolExecutor introduce in java 5.
Reference: http://download.oracle.com/javase/1,5.0/docs/api/java/util/concurrent/ScheduledExecutorService.html
It might NOT be a problem, it depends.
In my case, I use Thread.sleep() to wait for a couple of seconds before another reconnect attempt to an external process. I have a while loop for this reconnect logic till it reaches the max # of attemps. So in my case, Thread.sleep() is purely for timing purpose and not coordinating among multithreads, it's perfectly fine.
You can configure you IDE in how this warning should be handled.
I suggest looking into the CountDownLatch class. There are quite a few trivial examples out there online. Back when I just started multithreaded programming they were just the ticket for replacing a "sleeping while loop".