Develop Reference

CPU usage is 100% during Thread.onSpinWait()

I'm writing a backtesting raw data collector for my crypto trading bot and I've run into a weird optimization issue.
I constantly have 30 runnables in an Executors.newCachedThreadPool() running get requests from an API. Since the API has a request limit of 1200 per minute I have this bit of code in my runnable:
while (minuteRequests.get() >= 1170) {
Thread.onSpinWait();
}
Yes, minuteRequests is an AtomicInteger, so I'm not running into any issues there.
Everything works, the issue is that even though I'm using the recommended busy-waiting onSpinWait method, I shoot from 24% CPU usage or so to 100% when the waiting is initiated. For reference I'm running this on a 3900X (24 thread).
Any recommendations on how to better handle this situation?

My recommendation would be to not do busy waiting at all.
The javadocs for Thread.onSpinWait say this:
Indicates that the caller is momentarily unable to progress, until the occurrence of one or more actions on the part of other activities. By invoking this method within each iteration of a spin-wait loop construct, the calling thread indicates to the runtime that it is busy-waiting. The runtime may take action to improve the performance of invoking spin-wait loop constructions.
Note the highlighted section uses the word may rather than will. That means that it also may not do anything. Also "improve the performance" does not mean that your code will be objectively efficient.
The javadoc also implies that the improvements may be hardware dependent.
In short, this is the right way to use onSpinwait ... but you are expecting too much of it. It won't make your busy-wait code efficient.
So what would I recommend you actually do?
I would recommend that you replace the AtomicInteger with a Semaphore (javadoc). This particular loop would be replaced by the following:
semaphore.acquire();
This blocks1 until 1 "permit" is available and acquires it. Refer to the class javadocs for an explanation of how semaphores work.
Note: since you haven't show us the complete implementation of your rate limiting, it is not clear how your current approach actually works. Therefore, I can't tell you exactly how to replace AtomicInteger with Semaphore throughout.
1 - The blocked thread is "parked" until some other thread releases a permit. While it is parked, the thread does not run and is not associated with a CPU core. The core is either left idle (typically in a low power state) or it is assigned to some other thread. This is typically handled by the operating system's thread scheduler. When another thread releases a permit, the Semaphore.release method will tell the OS to unpark one of the threads that is blocked in acquire.

Letting a thread wait vs stopping and starting

I have a consumer thread blocking on removing from a queue.
There are going to be periods during which I know nothing will be added to the queue.
My question is: is it worth adding the complexity of managing when to start/stop the thread, or should I just leave it waiting until queue starts getting elements again?

If the concurrent queue implementation that you're using is worth it's salt then the thread will not be busy-waiting for very long. Some implementations may do this briefly for performance reasons but after that then it will block and will not be consuming CPU cycles. Therefore the difference between a stopped thread and a blocked thread becomes more or less meaningless.
Use a concurrent queue. See Which concurrent Queue implementation should I use in Java?

When dealing with Multithreading its a best practice to just act when you have a performance problem. Otherwise I would just leave it like it is to avoid trouble.

I dont think there is a big impact on the performance since the thread is blocked (inactive waiting). It could make sense if the thread is holding expensive resources which can be released for that time. I would keep this as simple as possible, especially in a concurrent enviroment complexity can lead to strange errors.

Why prefer wait/notify to while cycle?

I have some misunderstanding with advantages of wait/notify. As i understand processor core will do nothing helpful in both cases so what's the reason tro write complex wait/notify block codes instead of just waiting in cycle?
I'm clear that wait/notify will not steal processor time in case when two threads are executed on only one core.

"Waiting in a cycle" is most commonly referred to as a "busy loop" or "busy wait":
while ( ! condition()) {
// do nothing
}
workThatDependsOnConditionBeingTrue();
This is very disrespectful of other threads or processes that may need CPU time (it takes 100% time from that core if it can). So there is another variant:
while ( ! condition()) {
sleepForShortInterval();
// do nothing
}
workThatDependsOnConditionBeingTrue();
The small sleep in this variant will drop CPU usage dramatically, even if it is ~100ms long, which should not be noticeable unless your application is real-time.
Note that there will generally be a delay between when the condition actually becomes true and when sleepForShortInterval() ends. If, to be more polite to others, you sleep longer -- the delay will increase. This is generally unacceptable in real-time scenarios.
The nice way to do this, assuming that whatever condition() is checking is being changed from another thread, is to have the other thread wake you up when it finishes whatever you are waiting for. Cleaner code, no wasted CPU, and no delays.
Of course, it's quicker to implement a busy wait, and it may justified for quick'n'dirty situations.
Beware that, in a multithreaded scenario where condition() can be changed to false as well as true, you will need to protect your code between the while and the workThatDependsOnConditionBeingTrue() to avoid other threads changing its value in this precise point of time (this is called a race codition, and is very hard to debug after the fact).

I think you answered your question almost by saying
I'm clear that wait/notify will not steal processor time in case.
Only thing I would add is, this true irrespective of one core or multi-core. wait/notify wont keep the cpu in a busy-wait situation compared to while loop or periodic check.
what's the reason not to run core but wait? There's no helpful work in any case and you're unable to use core when it's in waiting state.
I think you are looking at it from a single application perspective where there is only one application with one thread is running. Think of it from a real world application (like web/app servers or standalone) where there are many threads running and competing for cpu cycles - you can see the advantage of wait/notify. You would definitely not want even a single thread to just do a busy-wait and burn the cpu cycles.
Even if it a single application/thread running on the system there are always OS process running and its related processes that keep competing for the CPU cycles. You don't want them to starve them because the application is doing a while busy-wait.
Quoting from Gordon's comment
waiting in cycle as you suggest you are constantly checking whether the thing you are waiting for has finished, which is wasteful and if you use sleeps you are just guessing with timing, whereas with wait/notify you sit idle until the process that you are waiting on tells you it is finished.

In general, your application is not the only one running on the CPU. Using non-spinning waiting is, first of all, an act of courtesy towards the other processes/threads which are competing for the CPU in order to do some useful job. The CPU scheduler cannot know a-priori if your thread is going to do something useful or just spin on a false flag. So, it can't tune itself based on that, unless you tell it you don't want to be run, because there's nothing for you to do.
Indeed, busy-waiting is faster than getting the thread to sleep, and that's why usually the wait() method is implemented in a hybrid way. It first spins for a while, and then it actually goes to sleep.
Besides, it's not just waiting in a loop. You still need to synchronize access to the resources you're spinning on. Otherwise, you'll fall victim of race conditions.
If you feel the need of a simpler interface, you might also consider using CyclicBarrier, CountDownLatch or a SynchronousQueue.

Deadlock in a single threaded java program [duplicate]

This question already has answers here:
Is it possible for a thread to Deadlock itself?
(20 answers)
Closed 9 years ago.
Read that deadlock can happen in a single threaded java program. I am wondering how since there won't be any competition after all. As far as I can remember, books illustrate examples with more than one thread. Can you please give an example if it can happen with a single thread.

It's a matter of how exactly you define "deadlock".
For example, this scenario is somewhat realistic: a single-threaded application that uses a size-limited queue that blocks when its limit is reached. As long as the limit is not reached, this will work fine with a single thread. But when the limit is reached, the thread will wait forever for a (non-existing) other thread to take something from the queue so that it can continue.

Before multicore processors became cheap, all desktop computers had single-core processors. Single-core processors runs only on thread. So how multithreading worked then? The simplest implementation for Java would be:
thread1's code:
doSomething();
yield(); // may switch to another thread
doSomethingElse();
thread2's code:
doSomething2();
yield(); // may switch to another thread
doSomethingElse2();
This is called cooperative multithreading - all is done with just 1 thread, and so multithreading was done in Windows 3.1.
Today's multithreading called preemptive multithreading is just a slight modification of cooperative multithreading where this yield() is called automatically from time to time.
All that may reduce to the following interlacings:
doSomething();
doSomething2();
doSomethingElse2();
doSomethingElse();
or:
doSomething();
doSomething2();
doSomethingElse();
doSomethingElse2();
And so on... We converted multithreaded code to single-threaded code. So yes, if a deadlock is possible in multithreaded programs in single-threaded as well. For example:
thread1:
queue.put(x);
yield();
thread2:
x = queue.waitAndGet()
yield();
It's OK with this interlace:
queue.put(x);
x = queue.waitAndGet()
But here we get deadlock:
x = queue.waitAndGet()
queue.put(x);
So yes, deadlocks are possible in single-threaded programs.

Well I dare say yes
If you try to acquire the same lock within the same thread consecutively, it depends on the type of lock or locking implementation whether it checks if the lock is acquired by the same thread. If the implementation does not check this, you have a deadlock.
For synchronized this is checked, but I could not find the guarantee for Semaphore.
If you use some other type of lock, you have to check the spec as how it is guaranteed to behave!
Also as has already been pointed out, you may block (which is different from deadlock) by reading/ writing to a restricted buffer. For instance you write things into a slotted buffer and only read from it on certain conditions. When you can no longer insert, you wait until a slot becomes free, which won't happen since you yourself do the reading.
So I daresay the answer should be yes, albeit not that easy and usually easier to detect.
hth
Mario

Even if your java stuff is single-threaded there are still signal handlers, which are executed in a different thread/context than the main thread.
So, a deadlock can indeed happen even on single-threaded solutions, if/when java is running on linux.
QED.
-pbr

No, Sounds pretty impossible to me.
But you could theoretically lock a system resource while another app locks another that you're going to request and that app is going to request the one you've already locked. Bang Deadlock.
But the OS should be able to sort this thing out by detecting that and give both resources to one app at the time. Chances for this to happen is slim to none, but any good OS should be able to handle this one-in-a billion chance.
If you make the design carefully and only locks one resource at a time, this can not happen.

No.
Deadlock is a result of multiple threads (or processes) attempting to acquire locks in such a way that neither can continue.
Consider a quote from the Wikipedia article: (http://en.wikipedia.org/wiki/Deadlock)
"When two trains approach each other at a crossing, both shall come to a full stop and neither shall start up again until the other has gone."

It is actually quite easy:
BlockingQueue bq = new ArrayBlockingQueue(1);
bq.take();
will deadlock.

What is the benefit of ThreadGroup in java over creating separate threads?

Many methods like stop(), resume(), suspend() etc are deprecated.
So is it useful to create threads using ThreadGroup?

Using ThreadGroup can be a useful diagnostic technique in big application servers with thousands of threads. If your threads are logically grouped together, then when you get a stack trace you can see which group the offending thread was part of (e.g. "Tomcat threads", "MDB threads", "thread pool X", etc), which can be a big help in tracking down and fixing the problem.

Don't use ThreadGroup for new code. Use the Executor stuff in java.util.concurrent instead.

Somewhat complimentary to the answer provided (6 years ago or so). But, while the Concurrency API provides a lot of constructs, the ThreadGroup might still be useful to use. It provides the following functionality:
Logical organisation of your threads (for diagnostic purposes).
You can interrupt() all the threads in the group. (Interrupting is perfectly fine, unlike suspend(), resume() and stop()).
You can set the maximum priority of the threads in the group. (not sure how widely useful is that, but there you have it).
Sets the ThreadGroup as a daemon. (So all new threads added to it will be daemon threads).
It allows you to override its uncaughtExceptionHandler so that if one of the threads in the group throws an Exception, you have a callback to handle it.
It provides you some extra tools such as getting the list of threads, how many active ones you have etc. Useful when having a group of worker threads, or some thread pool of some kind.

The short answer is - no, not really. There's little if any benefit to using one.
To expand on that slightly, if you want to group worker threads together you're much better off using an ExecutorService. If you want to quickly count how many threads in a conceptual group are alive, you still need to check each Thread individually (as ThreadGroup.activeCount() is an estimation, meaning it's not useful if the correctness of your code depends on its output).
I'd go so far as to say that the only thing you'd get from one these days, aside from the semantic compartmentalisation, is that Threads constructed as part of a group will pick up the daemon flag and a sensible name based on their group. And using this as a shortcut for filling in a few primitives in a constructor call (which typically you'd only have to write once anyway, sicne you're probably starting the threads in a loop and/or method call).
So - I really don't see any compelling reason to use one at all. I specifically tried to, a few months back, and failed.
EDIT - I suppose one potential use would be if you're running with a SecurityManager, and want to assert that only threads in the same group can interrupt each other. Even that's pretty borderline, as the default implementation always returns true for a Thread in any non-system thread group. And if you're implementing your own SecurityManager, you've got the possibility to have it make its decision on any other criteria (including the typical technique of storing Threads in collections as they get created).

Great answer for #skaffman. I want to add one more advantage:
Thread groups helps manipulating all the threads which are defined in this at once.