One of the first things we all learnt about concurrency in Java is that we use locks (synchronized keyword, Lock/ReadWriteLock interface) to protect against concurrent access. For example:
synchronized void eat(){
//some code
}
In theory, this method eat() can be executed by a single thread. Even though there are many threads waiting to execute it, only one will take the lock. But then comes parallelism, which made me think twice about what I just said.
I have 4 core CPU. That means I can do 4 tasks parallelly. Yes, lock can be taken by a single thread. But could it happen that 4 threads call method eat() and take a lock at the LITERALLY same time, although there is a lock that needs to be acquired to actually do anything?
Can something like that even happen in Java? I guess it can't but I had to ask this. And how is it even dealing with a case I just said?
...4 threads...take a lock at the LITERALLY same time...
Can't happen. Any "synchronized" operation (e.g., "take a lock") must operate on the system's main memory, and in any conventional computer system, there is only one memory bus. It is physically impossible for more than one CPU to access the main memory at the same time.
If two CPUs decide to access the memory at literally the same time, the hardware guarantees that one of them will "win" the race and go first, while the other one is forced to wait its turn.
Short answer - JVM keeps a list of threads that are trying to get the lock.
More details
https://wiki.openjdk.java.net/display/HotSpot/Synchronization
Also, worth mentioning that list (aka "inflated lock") is only created when really required (== when contention on a lock happens).
I'm writing a backtesting raw data collector for my crypto trading bot and I've run into a weird optimization issue.
I constantly have 30 runnables in an Executors.newCachedThreadPool() running get requests from an API. Since the API has a request limit of 1200 per minute I have this bit of code in my runnable:
while (minuteRequests.get() >= 1170) {
Thread.onSpinWait();
}
Yes, minuteRequests is an AtomicInteger, so I'm not running into any issues there.
Everything works, the issue is that even though I'm using the recommended busy-waiting onSpinWait method, I shoot from 24% CPU usage or so to 100% when the waiting is initiated. For reference I'm running this on a 3900X (24 thread).
Any recommendations on how to better handle this situation?
My recommendation would be to not do busy waiting at all.
The javadocs for Thread.onSpinWait say this:
Indicates that the caller is momentarily unable to progress, until the occurrence of one or more actions on the part of other activities. By invoking this method within each iteration of a spin-wait loop construct, the calling thread indicates to the runtime that it is busy-waiting. The runtime may take action to improve the performance of invoking spin-wait loop constructions.
Note the highlighted section uses the word may rather than will. That means that it also may not do anything. Also "improve the performance" does not mean that your code will be objectively efficient.
The javadoc also implies that the improvements may be hardware dependent.
In short, this is the right way to use onSpinwait ... but you are expecting too much of it. It won't make your busy-wait code efficient.
So what would I recommend you actually do?
I would recommend that you replace the AtomicInteger with a Semaphore (javadoc). This particular loop would be replaced by the following:
semaphore.acquire();
This blocks1 until 1 "permit" is available and acquires it. Refer to the class javadocs for an explanation of how semaphores work.
Note: since you haven't show us the complete implementation of your rate limiting, it is not clear how your current approach actually works. Therefore, I can't tell you exactly how to replace AtomicInteger with Semaphore throughout.
1 - The blocked thread is "parked" until some other thread releases a permit. While it is parked, the thread does not run and is not associated with a CPU core. The core is either left idle (typically in a low power state) or it is assigned to some other thread. This is typically handled by the operating system's thread scheduler. When another thread releases a permit, the Semaphore.release method will tell the OS to unpark one of the threads that is blocked in acquire.
I'm reading a book called "Java Concurrency In Practice" and in the first chapter the following code is demonstrated as thread unsafe
public class UnsafeSequence {
private int value;
/** Returns a unique value. */
public int getNext() {
return value++;
}
}
So if two threads run this code we can get unwanted results because they will interchange in different steps such as reading, modifying and writing the value. Is this determined only by OS, or do threads switch between each other on different "bytecode commands" for example? Is there any way to know all possible places where threads might switch from one to another, not just for this code but in general?
As several comments note, no. Two things you can do:
Write your classes in a thread-safe manner, so that thread scheduling isn't an issue.
Use concurrency support to prevent issues.
Keep reading the book.
Is there any way to know all possible places where threads might switch from one to another, not just for this code but in general?
This question is a bit vagiue. Let me split it up in two parts:
Two threads can wander over the same piece of code and happily interleave, except:
inside atomic operations (including complex operations inside of thread-safe classes)
inside guarded blocks (e.g. using a synchronized block, lock, semaphore, or some other memory fence)
Threads can switch all the time, which is 100% up to the OS. In theory a thread might even never get a chance to be 'scheduled in' again if the OS decides so. Threads may die spuriously (e.g. killed in ProcessExplorer). You never know when a thread will be stopped in it's tracks (suspended), but you do know that if it happens inside an atomic operation, no other thread will enter that code until the suspended thread resumes and completes the operation.
It happens whenever the system scheduler feels like. It has nothing to do with the JVM if the JVM only passes that scheduling to the native processor.
I am reading JDK source code(1.8.0_05). Found that ReentrantLock is used to ensure thread-safe workers. Author just told 2 reasons.
Reason 1:
This serializes interruptIdleWorkers, which avoids unnecessary
interrupt storms, especially during shutdown.Otherwise exiting threads
would concurrently interrupt those that have not yet interrupted.
Reason 2:
It also simplifies some of the associated statistics bookkeeping of largestPoolSize etc.
Can anybody elaborate more details about the reason, especially Reason 1? I just don't understand the idea on design level.
P.S. How about using Collections.synchronizedSet?
Can anybody elaborate more details about the reason, especially Reason
1? I just don't understand the idea on design level.
Imagine if it were not a ReentrantLock but instead a concurrent set. Also, let's imagine if 10 threads invoked shutdown. Shutdown will run interruptIdleWorkers, so each of those 10 threads will run interruptIdleWorkers.
If it were a concurrent set, then all 10 threads shutting down will also interrupt every thread. Since the collection is concurrent, then it each of those 10 threads doesn't need to wait for the others to succeed. The result here is, as the docs said, a flood of interrupts when all you really want is 1.
You can use Collections.synchronizedSet, but you would have to synchronize on the entire collection. This could be fine, but if you can achieve Reason #2 with ReentrantLock than it is a better fit than the synchronizedSet.
The stop(), suspend(), and resume() in java.lang.Thread are deprecated because they are unsafe. The Oracle recommended work around is to use Thread.interrupt(), but that approach doesn't work in all cases. For example, if you are call a library method that doesn't explicitly or implicitly check the interrupted flag, you have no choice but to wait for the call to finish.
So, I'm wondering if it is possible to characterize situations where it is (provably) safe to call stop() on a Thread. For example, would it be safe to stop() a thread that did nothing but call find(...) or match(...) on a java.util.regex.Matcher?
(If there are any Oracle engineers reading this ... a definitive answer would be really appreciated.)
EDIT: Answers that simply restate the mantra that you should not call stop() because it is deprecated, unsafe, whatever are missing the point of this question. I know that that it is genuinely unsafe in the majority of cases, and that if there is a viable alternative you should always use that instead.
This question is about the subset cases where it is safe. Specifically, what is that subset?
Here's my attempt at answering my own question.
I think that the following conditions should be sufficient for a single thread to be safely stopped using Thread.stop():
The thread execution must not create or mutate any state (i.e. Java objects, class variables, external resources) that might be visible to other threads in the event that the thread is stopped.
The thread execution must not use notify to any other thread during its normal execution.
The thread must not start or join other threads, or interact with then using stop, suspend or resume.
(The term thread execution above covers all application-level code and all library code that is executed by the thread.)
The first condition means that a stopped thread will not leave any external data structures or resources in an inconsistent state. This includes data structures that it might be accessing (reading) within a mutex. The second condition means that a stoppable thread cannot leave some other thread waiting. But it also forbids use of any synchronization mechanism other that simple object mutexes.
A stoppable thread must have a way to deliver the results of each computation to the controlling thread. These results are created / mutated by the stoppable thread, so we simply need to ensure that they are not visible following a thread stop. For example, the results could be assigned to private members of the Thread object and "guarded" with a flag that is atomically by the thread to say it is "done".
EDIT: These conditions are pretty restrictive. For example, for a "regex evaluator" thread to be safely stopped, if we must guarantee that the regex engine does not mutate any externally visible state. The problem is that it might do, depending on how you implement the thread!
The Pattern.compile(...) methods might update a static cache of compiled
patterns, and if they did they would (should) use a mutex to do it. (Actually, the OpenJDK 6.0 version doesn't cache Patterns, but Sun might conceivably change this.)
If you try to avoid 1) by compiling the regex in the control thread and supplying a pre-instantiated Matcher, then the regex thread does mutate externally visible state.
In the first case, we would probably be in trouble. For example, suppose that a HashMap was used to implement the cache and that the thread was interrupted while the HashMap was being reorganized.
In the second case, we would be OK provided that the Matcher had not been passed to some other thread, and provided that the controller thread didn't try to use the Matcher after stopping the regex matcher thread.
So where does this leave us?
Well, I think I have identified conditions under which threads are theoretically safe to stop. I also think that it is theoretically possible to statically analyse the code of a thread (and the methods it calls) to see if these conditions will always hold. But, I'm not sure if this is really practical.
Does this make sense? Have I missed something?
EDIT 2
Things get a bit more hairy when you consider that the code that we might be trying to kill could be untrusted:
We can't rely on "promises"; e.g. annotations on the untrusted code that it is either killable, or not killable.
We actually need to be able to stop the untrusted code from doing things that would make it unkillable ... according to the identified criteria.
I suspect that this would entail modifying JVM behaviour (e.g. implementing runtime restrictions what threads are allowed to lock or modify), or a full implementation of the Isolates JSR. That's beyond the scope of what I was considering as "fair game".
So lets rule the untrusted code case out for now. Or at least, acknowledge that malicious code can do things to render itself not safely killable, and put that problem to one side.
The lack of safety comes from the idea idea of critical sections
Take mutex
do some work, temporarily while we work our state is inconsistent
// all consistent now
Release mutex
If you blow away the thread and it happend to be in a critical section then the object is left in an inconsistent state, that means not safely usable from that point.
For it to be safe to kill the thread you need to understand the entire processing of whatever is being done in that thread, to know that there are no such critical sections in the code. If you are using library code, then you may not be able to see the source and know that it's safe. Even if it's safe today it may not be tomorrow.
(Very contrived) Example of possible unsafety. We have a linked list, it's not cyclic. All the algorithms are really zippy because we know it's not cyclic. During our critical section we temporarily introduce a cycle. We then get blown away before we emerge from the critical section. Now all the algorithms using the list loop forever. No library author would do that surely! How do you know? You cannot assume that code you use is well written.
In the example you point to, it's surely possible to write the requreid functionality in an interruptable way. More work, but possible to be safe.
I'll take a flyer: there is no documented subset of Objects and methods that can be used in cancellable threads, because no library author wants to make the guarantees.
Maybe there's something I don't know, but as java.sun.com said, it is unsafe because anything this thread is handling is in serious risk to be damaged. Other objects, connections, opened files... for obvious reasons, like "don't shut down your Word without saving first".
For this find(...) exemple, I don't really think it would be a catastrophe to simply kick it away with a sutiless .stop()...
A concrete example would probably help here. If anyone can suggest a good alternative to the following use of stop I'd be very interested. Re-writing java.util.regex to support interruption doesn't count.
import java.util.regex.*;
import java.util.*;
public class RegexInterruptTest {
private static class BadRegexException extends RuntimeException { }
final Thread mainThread = Thread.currentThread();
TimerTask interruptTask = new TimerTask() {
public void run() {
System.out.println("Stopping thread.");
// Doesn't work:
// mainThread.interrupt();
// Does work but is deprecated and nasty
mainThread.stop(new BadRegexException());
}
};
Timer interruptTimer = new Timer(true);
interruptTimer.schedule(interruptTask, 2000L);
String s = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaab";
String exp = "(a+a+){1,100}";
Pattern p = Pattern.compile(exp);
Matcher m = p.matcher(s);
try {
System.out.println("Match: " + m.matches());
interruptTimer.cancel();
} catch(BadRegexException bre) {
System.out.println("Oooops");
} finally {
System.out.println("All over");
}
}
}
There are ways to use Thread.stop() relatively stable w/o leaking memory or file descriptors (FDs are exceptionally leak prone on *NIX) but you shall rely on it only if you are forced to manage 3rd party code. Never do use it to achieve the result if you can have control over the code itself.
If I use Thread.stop along w/ interrupt() and some more hacks stuff like adding custom logging handlers to re-throw the trapped ThreadDeath, adding unhandleExceltionHandler, running into your own ThreadGroup (sync over 'em), etc...
But that deserves an entire new topic.
But in this case it's the Java Designers telling you; and
they're more authorative on their language then either of us :)
Just a note: quite a few of them are pretty clueless
If my understanding is right, the problem has to do with synchronization locks not being released as the generated ThreadInterruptedException() propagates up the stack.
Taking that for granted, it's inherently unsafe because you can never know whether or not any "inner method call" you happened to be in at the very moment stop() was invoked and effectuated, was effectively holding some synchronization lock, and then what the java engineers say is, seemingly, unequivocally right.
What I personally don't understand is why it should be impossible to release any synchronization lock as this particular type of Exception propagates up the stack, thereby passing all the '}' method/synchronization block delimiters, which do cause any locks to be release for any other type of exception.
I have a server written in java, and if the administrator of that service wants a "cold shutdown", then it is simply NECESSARY to be able to stop all running activity no matter what. Consistency of any object's state is not a concern because all I'm trying to do is to EXIT. As fast as I can.
There is no safe way to kill a thread.
Neither there is a subset of situations where it is safe. Even if it is working 100% while testing on Windows, it may corrupt JVM process memory under Solaris or leak thread resources under Linux.
One should always remember that underneath the Java Thread there is a real, native, unsafe thread.
That native thread works with native, low-level, data and control structures. Killing it may leave those native data structures in an invalid state, without a way to recover.
There is no way for Java machine to take all possible consequences into account, as the thread may allocate/use resources not only within JVM process, but within the OS kernel as well.
In other words, if native thread library doesn't provide a safe way to kill() a thread, Java cannot provide any guarantees better than that. And all known to me native implementations state that killing thread is a dangerous business.
All forms of concurrency control can be provided by the Java synchronization primitives by constructing more complex concurrency controls that suit your problem.
The reasons for deprecation are clearly given in the link you provide. If you're willing to accept the reasons why, then feel free to use those features.
However, if you choose to use those features, you also accept that support for those features could stop at any time.
Edit: I'll reiterate the reason for deprecation as well as how to avoid them.
Since the only danger is that objects that can be referenced by the stoped thread could be corrupted, simply clone the String before you pass it to the Thread. If no shared objects exist, the threat of corrupted objects in the program outside the stoped Thread is no longer there.