One of the first things we all learnt about concurrency in Java is that we use locks (synchronized keyword, Lock/ReadWriteLock interface) to protect against concurrent access. For example:
synchronized void eat(){
//some code
}
In theory, this method eat() can be executed by a single thread. Even though there are many threads waiting to execute it, only one will take the lock. But then comes parallelism, which made me think twice about what I just said.
I have 4 core CPU. That means I can do 4 tasks parallelly. Yes, lock can be taken by a single thread. But could it happen that 4 threads call method eat() and take a lock at the LITERALLY same time, although there is a lock that needs to be acquired to actually do anything?
Can something like that even happen in Java? I guess it can't but I had to ask this. And how is it even dealing with a case I just said?
...4 threads...take a lock at the LITERALLY same time...
Can't happen. Any "synchronized" operation (e.g., "take a lock") must operate on the system's main memory, and in any conventional computer system, there is only one memory bus. It is physically impossible for more than one CPU to access the main memory at the same time.
If two CPUs decide to access the memory at literally the same time, the hardware guarantees that one of them will "win" the race and go first, while the other one is forced to wait its turn.
Short answer - JVM keeps a list of threads that are trying to get the lock.
More details
https://wiki.openjdk.java.net/display/HotSpot/Synchronization
Also, worth mentioning that list (aka "inflated lock") is only created when really required (== when contention on a lock happens).
Related
I'm writing a backtesting raw data collector for my crypto trading bot and I've run into a weird optimization issue.
I constantly have 30 runnables in an Executors.newCachedThreadPool() running get requests from an API. Since the API has a request limit of 1200 per minute I have this bit of code in my runnable:
while (minuteRequests.get() >= 1170) {
Thread.onSpinWait();
}
Yes, minuteRequests is an AtomicInteger, so I'm not running into any issues there.
Everything works, the issue is that even though I'm using the recommended busy-waiting onSpinWait method, I shoot from 24% CPU usage or so to 100% when the waiting is initiated. For reference I'm running this on a 3900X (24 thread).
Any recommendations on how to better handle this situation?
My recommendation would be to not do busy waiting at all.
The javadocs for Thread.onSpinWait say this:
Indicates that the caller is momentarily unable to progress, until the occurrence of one or more actions on the part of other activities. By invoking this method within each iteration of a spin-wait loop construct, the calling thread indicates to the runtime that it is busy-waiting. The runtime may take action to improve the performance of invoking spin-wait loop constructions.
Note the highlighted section uses the word may rather than will. That means that it also may not do anything. Also "improve the performance" does not mean that your code will be objectively efficient.
The javadoc also implies that the improvements may be hardware dependent.
In short, this is the right way to use onSpinwait ... but you are expecting too much of it. It won't make your busy-wait code efficient.
So what would I recommend you actually do?
I would recommend that you replace the AtomicInteger with a Semaphore (javadoc). This particular loop would be replaced by the following:
semaphore.acquire();
This blocks1 until 1 "permit" is available and acquires it. Refer to the class javadocs for an explanation of how semaphores work.
Note: since you haven't show us the complete implementation of your rate limiting, it is not clear how your current approach actually works. Therefore, I can't tell you exactly how to replace AtomicInteger with Semaphore throughout.
1 - The blocked thread is "parked" until some other thread releases a permit. While it is parked, the thread does not run and is not associated with a CPU core. The core is either left idle (typically in a low power state) or it is assigned to some other thread. This is typically handled by the operating system's thread scheduler. When another thread releases a permit, the Semaphore.release method will tell the OS to unpark one of the threads that is blocked in acquire.
I'm reading a book called "Java Concurrency In Practice" and in the first chapter the following code is demonstrated as thread unsafe
public class UnsafeSequence {
private int value;
/** Returns a unique value. */
public int getNext() {
return value++;
}
}
So if two threads run this code we can get unwanted results because they will interchange in different steps such as reading, modifying and writing the value. Is this determined only by OS, or do threads switch between each other on different "bytecode commands" for example? Is there any way to know all possible places where threads might switch from one to another, not just for this code but in general?
As several comments note, no. Two things you can do:
Write your classes in a thread-safe manner, so that thread scheduling isn't an issue.
Use concurrency support to prevent issues.
Keep reading the book.
Is there any way to know all possible places where threads might switch from one to another, not just for this code but in general?
This question is a bit vagiue. Let me split it up in two parts:
Two threads can wander over the same piece of code and happily interleave, except:
inside atomic operations (including complex operations inside of thread-safe classes)
inside guarded blocks (e.g. using a synchronized block, lock, semaphore, or some other memory fence)
Threads can switch all the time, which is 100% up to the OS. In theory a thread might even never get a chance to be 'scheduled in' again if the OS decides so. Threads may die spuriously (e.g. killed in ProcessExplorer). You never know when a thread will be stopped in it's tracks (suspended), but you do know that if it happens inside an atomic operation, no other thread will enter that code until the suspended thread resumes and completes the operation.
It happens whenever the system scheduler feels like. It has nothing to do with the JVM if the JVM only passes that scheduling to the native processor.
Do mutex locks ensure bounded waiting condition ? Is it possible if two threads are trying to get hold of a lock, but only one process (just by luck) gets it again and again. Since Peterson's Algorithm ensures bounded waiting, is it better to use that instead of mutex locks ?
It is possible to have unbounded wait with mutices, if for instance locking attempts keep coming in on a mutex, at least in C++ std::mutex there's no guaranteed first comes first gets.
However this shouldn't really be a concern - Unless you have some lock with many many threads locking all the time (and even in that case it's very unlikely to cause some starvation situation).
The best thing to do is always use standard library locking mechanism and not write your own mutices.
Mutex with "bounded waiting condition" is called Fair sometimes. As gbehar correctly mention above, C++ standard doesn't define fairness for std::mutex. If you really need fair mutex, you can look at Intel TBB, where fairness is guaranteed for some kinds of them. I would like to remember that fairness comes not without overhead.
See https://www.threadingbuildingblocks.org/docs/help/tbb_userguide/Mutex_Flavors.html for details.
Update: Current link https://github.com/oneapi-src/oneTBB/blob/master/doc/main/tbb_userguide/Mutex_Flavors.rst
I read this in an upvoted comment on StackOverflow:
But if you want to be safe, you can add simple synchronized(this) {}
at the end of you #PostConstruct [method]
[note that variables were NOT volatile]
I was thinking that happens-before is forced only if both write and read is executed in synchronized block or at least read is volatile.
Is the quoted sentence correct? Does an empty synchronized(this) {} block flush all variables changed in current method to "general visible" memory?
Please consider some scenerios
what if second thread never calls lock on this? (suppose that second thread reads in other methods). Remember that question is about: flush changes to other threads, not give other threads a way (synchronized) to poll changes made by original thread. Also no-synchronization in other methods is very likely in Spring #PostConstruct context - as original comment says.
is memory visibility of changes forced only in second and subsequent calls by another thread? (remember that this synchronized block is a last call in our method) - this would mark this way of synchronization as very bad practice (stale values in first call)
Much of what's written about this on SO, including many of the answers/comments in this thread, are, sadly, wrong.
The key rule in the Java Memory Model that applies here is: an unlock operation on a given monitor happens-before a subsequent lock operation on that same monitor. If only one thread ever acquires the lock, it has no meaning. If the VM can prove that the lock object is thread-confined, it can elide any fences it might otherwise emit.
The quote you highlight assumes that releasing a lock acts as a full fence. And sometimes that might be true, but you can't count on it. So your skeptical questions are well-founded.
See Java Concurrency in Practice, Ch 16 for more on the Java Memory Model.
All writes that occur prior to a monitor exit are visible to all threads after a monitor enter.
A synchronized(this){} can be turned into bytecode like
monitorenter
monitorexit
So if you have a bunch of writes prior to the synchronized(this){} they would have occurred before the monitorexit.
This brings us to the next point of my first sentence.
visible to all threads after a monitor enter
So now, in order for a thread to ensure the writes ocurred it must execute the same synchronization ie synchornized(this){}. This will issue at the very least a monitorenter and establish your happens before ordering.
So to answer your question
Does an empty synchronized(this) {} block flush all variables changed
in current method to "general visible" memory?
Yes, as long as you maintain the same synchronization when you want to read those non-volatile variables.
To address your other questions
what if second thread never calls lock on this? (suppose that second
thread reads in other methods). Remember that question is about: flush
changes to other threads, not give other threads a way (synchronized)
to poll changes made by original thread. Also no-synchronization in
other methods is very likely in Spring #PostConstruct context
Well in this case using synchronized(this) without any other context is relatively useless. There is no happens-before relationship and it's in theory just as useful as not including it.
is memory visibility of changes forced only in second and subsequent
calls by another thread? (remember that this synchronized block is a
last call in our method) - this would mark this way of synchronization
as very bad practice (stale values in first call)
Memory visibility is forced by the first thread calling synchronized(this), in that it will write directly to memory. Now, this doesn't necessarily mean each threads needs to read directly from memory. They can still read from their own processor caches. Having a thread call synchronized(this) ensures it pulls the value of the field(s) from memory and retrieve most up to date value.
I was trying to dive into synchronization.
here, its mentioned "Every object has an intrinsic lock associated with it.".
I understand Object class. Its ( quite known to us ) does not have any property like a lock ( thus i guess its called intrinsic lock ). What exactly is this lock (ie is it a Lock.java class ? it is some sort of hidden field ? ) and how is it associated with an object (ie is there some mystery implicit reference to the lock from an object, something happening natively ) ?
When several threads try to acquire the same lock, one or more threads will be suspended and they will be resumed later. Well where are these threads stored ? What data structure keeps records of waiting threads ?
What logic is used under the covers to pick a thread waiting to enter synchronized method, when there are many waiting ?
Any reference to what happens under the covers (step by step) from "synchronized keyword" to "intrinsic lock aquisition" ?
Is there an upper limit on number of threads that are allowed to wait on synchronized ?
1) Yes, the lock is essentially a hidden field on the object. You can't access it except through synchronization.
2) Waiting threads essentially just sleep until the lock is available. They aren't "stored" anyplace special, nor is there a visible queue of waiting threads. Details of implementation are hidden.
3) I don't think there's any promised order. If you explicitly need round-robin scheduling or prioritization or something of that sort, it's your responsibility to implement it (or use a class which implements it for you) on top of the synchronization lock mechanism.
4) This is likely to be handled as an OS semaphore, if you understand those. If you don't, defining them strikes me as too much detail to be addressed properly here... and you don't really need to understand this unless you're reimplementing it.
5) There is no explicit limit, as far as I know. (I haven't checked the official Java Specification, but I understand how this kind of thing gets implemented at the OS level.) Of course at some point you're going to run out of system resources, but I think you'll generally run out of other resources (like memory to run those threads in) first.
One additional note: It's also worth looking at the Atomic... classes. When these can be used, they will be somewhat more efficient in modern processors than traditional Java synchronization can be.