I am using Android Studio (intelliJ-idea) for Android development. I get a warning for this piece of code:
if (status == STATUS_SOLVING) {
if (!solverThread.isAlive())
if (status != STATUS_SOLVED) // <<<<<< WARNING THIS LINE
status = STATUS_DATA_COLLECTING;
}
The line indicated above gives me a warning, saying that this condition is always true. I can see why this is true if the whole program runs on a single thread.
But since my program uses a parallel thread to change the value of status, does it not mean that this condition might change between line #1 and #3 in the snippet above?
Is this a valid warning? Am I missing something?
Do things change if I change the nested ifs into one single if with && operator?
It means that if you're counting on a value being changed by another thread that's a dangerous programming tactic and that the compiler won't know that its being accessed by different friends and will optimize assuming it isn't.
Any variable that may be touched by multiple threads like that needs to be declared volatile to inform the compiler that its touched by multiple threads, and special care needs to be taken in treating it. If you didn't know that I'll bet you have a dozen other multithreading bugs waiting to happen.
If a second thread is involved and changes status then certainly the value can change between lines 1 and 3. Especially since you don't seem to have any kind of thread synchronization in place, deliberate or not.
Granted, the code seems a bit unsafe, because you would not normally have two threads accessing the same variable without some kind of concurrency control. But this is speculation since I don't see the rest of your code.
Anyway, this is a warning, not an error. Sometimes warnings are wrong, that's why they can be suppressed.
You could see this as a sign that you're up to some unsafe stuff though. Perhaps you could post some more code in case you want to discuss the actual synchronization aspects?
Related
Summary
From my studies, I don't remember that a concept such "uninterruptible block" exists, and I did not find it either with a quick Google search.
Expected answer
yes, it does exist, and the proper term for that is ... (in this case, it would be nice, if someone could explain me, why it does not exist in Java)
no, it does not exist, because ...
Definition
By "uninterruptible block", I mean a section of code, in a multi-threading context, which, once starts execution, cannot be interrupted by other threads. I.e., the CPU (or the JVM), won't run any other thread at all, until the "atomic block" is left.
Note, that this is not the same as a section marked by lock/mutex/... etc., because such section can not be interrupted only by other threads, which acquire the same lock or mutex. But other threads can still interrupt it.
EDIT, in response to comments It would be fine also, if it affected only the threads of the current process.
RE. multiple cores: I would say, yes, also the other cores should stop, and we accept the performance hit (or, if it is exclusive only for the current process, then the other cores could still run threads of other processes).
Background
First of all, it is clear, that, at least in Java, this concept does not exist:
Atomic as in uninterruptible: once the block starts, it can't be interrupted, even by task switching.
...
[this] cannot be guaranteed in Java - it doesn't provide access to the
"critical sections" primitives required for uninterruptibility.
However, it would have come in handy in the following case: a system sends a request and receives response A. After receiving the response, it has max. 3 seconds to send request B. Now, if multiple threads are running, doing this, then it can happen, that after receiving response A, the thread is interrupted, and one or more threads run, before the original thread has the chance to send out request B, and thus misses the 3 seconds deadline. The more threads are running, the bigger the risk that this happens. By marking the "receive A to send B" section "uninterruptible", this could be avoided.
Note, that locking this section would not solve the issue. (It would not prevent the JVM, from e.g. processing 10 new threads at the "send request A" phase, right after our thread received response A.)
EDIT: Re. global mutex. That would also not solve the issue. Basically, I want the threads to make Request A's (and some other stuff) simultaneously, but I want them to stop, when another thread received Response A, and is going to make Request B.
Now, I know, that this would not be a 100% solution either, because those threads that don't get scheduled right after receiving response A still could miss the deadline. But, at least, those who do, would for sure send out the second request in time.
Some further speculation
The classic concurrency problem a++ could be simply solved by uninterruptible { a++; }, without the need for locks (which can cause dead-lock, and, in any case, would probably be more expensive in terms of performance, than simply executing the three instructions required by a++, with a simple flag, that they must not be interrupted).
EDIT RE. CAS: of course, that's another solution too. However, it involves retrying, until the write succeeds, and it is also slightly more complex to use (at least in Java, we have to use AtomicXXX, instead of the primitive types for that).
I know, of course, that this could be easily abused, by marking large blocks of code as uninterruptible, but that is true for many concurrency primitives as well. (What's more, I also know, that my original use case would be also kind of an "abuse", since I'd be doing I/O in an uninterruptible block, still it would have been worth at least a try, if such concept did exist in Java.)
The prevailing religion of Java today heavily forbids the use of the stop() instance method on a Thread[1] [2] [3]. It's marked as Deprecated in the official documentation, with a message that begins with:
This method is inherently unsafe. Stopping a thread with Thread.stop causes it to unlock all of the monitors that it has locked (as a natural consequence of the unchecked ThreadDeath exception propagating up the stack). If any of the objects previously protected by these monitors were in an inconsistent state, the damaged objects become visible to other threads, potentially resulting in arbitrary behavior. [...]
There is even a whole article supplied that goes into further detail. (Especially that ThreadDeath is silent when it gets to the top, and even if you guard against it, you can't guard against Thread.stop(Throwable t) anyway.)
In general, I agree with what that article is worried about, and with the answers to the related question on StackOverflow — in almost all the use cases in the world there is a much better way than stop(). But not all.
Here is a use case that I'm struggling to think of a better way for. I invite your suggestions.
Use case
I'm working with an interactive console running on top of Java, that allows the user run arbitrary Java statements and Python functions that call Java, in an interactive interpreted way. (See Jython console for screenshots.)
Now, the user can write arbitrary functions. Some of them might call Java methods that result in infinite loops that print a lot of output, almost saturating the system, so that it becomes painfully slow to respond.
When the user messes up like this, I need to give them a chance to kill the thread (using a button), and salvage the remainder of the work that they have produced in their session and stored in local variables.
If calling interrupt() on the thread fails, (e.g. if it never goes into waiting state while stuck in its infinite loop), then we are faced with two options that I can see: (1) kill the application outright or (2) use stop() on the bad thread, and let the user salvage whatever is left that hadn't been corrupted.
Yes I know that some objects may have been damaged and won't work properly. But we're talking about salvaging anything that may have been left in a valid state (which is probably pretty much everything, except one or two things).
Can anybody see any problems with this reasoning? And if this turns out to be a valid use-case, then does it mean the method shouldn't be deprecated :) ?
Calling Thread.stop() is a bad idea. End of story.
It may work in practice but you sacrifice most of the JVM's concurrency guarantees in the process. Your whole program is essentially running undefined behavior. It isn't simply the thread or the data in the thread that may become corrupted, but any part of the JVM that happens to be in a vulnerable state when the thread is killed.
It sounds like you're looking for someone to confirm that your use case somehow avoids the risks to the JVM. It doesn't, so you're not likely to get such a confirmation. If you don't see issues more power to you, but don't be surprised when it fails in inexplicable or dangerous ways.
As Peter Lawrey suggests you should be running this untrusted code in an isolated JVM, where the OS's process management can support killing resource-hogging processes. If you can't do that because you're passing around strange resources like open file descriptors that is your problem.
I am experimenting with a game mechanic in which players can run scripts on in-game computers. Script execution will be resource limited at a gameplay level to some amount of instructions per tick.
The following proof-of-concept demonstrates a basic level of sandboxing and throttling of arbitrary user code. It successfully runs ~250 instructions of poorly crafted 'user input' and then discards the coroutine. Unfortunately, the Java process never terminates. A little investigation in shows that the LuaThread created by LuaJ for the coroutine is hanging around forever.
SandboxTest.java:
public static void main(String[] args) {
Globals globals = JsePlatform.debugGlobals();
LuaValue chunk = globals.loadfile("res/test.lua");
chunk.call();
}
res/test.lua:
function sandbox(fn)
-- read script and set the environment
f = loadfile(fn, "t")
debug.setupvalue(f, 1, {print = print})
-- create a coroutine and have it yield every 50 instructions
local co = coroutine.create(f)
debug.sethook(co, coroutine.yield, "", 50)
-- demonstrate stepped execution, 5 'ticks'
for i = 1, 5 do
print("tick")
coroutine.resume(co)
end
end
sandbox("res/badfile.lua")
res/badfile.lua:
while 1 do
print("", "badfile")
end
The docs suggest that a coroutine that is considered unresumable will be garbage collected and an OrphanedThread exception will be thrown, signalling the LuaThread to end - but this is never happening. My question is in two parts:
Am I doing something fundamentally wrong to cause this behaviour?
If not, how should I handle this situation? From the source it appears that if I can get a reference to the LuaThread in Java I may be able to forcibly abandon it by issuing an interrupt(). Is this a good idea?
Reference: Lua / Java / LuaJ - Handling or Interrupting Infinite Loops and Threads
EDIT: I have posted a bug report over at the LuaJ SourceForge. It discusses the underlying issue (threads not being garbage collected as in the Lua spec) and suggests some ways to work around it.
It seems to be a limitation of LuaJ. I submitted a ticket earlier this year on Sourceforge as I see you've also done. The LuaThread class doesn't store references to the Java threads it creates, so you can't interrupt() those threads without modifying the LuaJ core to expose them:
new Thread(this, "Coroutine-"+(++coroutine_count)).start();
It may be dangerous to interrupt those threads without adding appropriate cleanup code to LuaJ.
Documentation that you provided for OrphanedThread also tells us that scope is the defining condition:
"Error sublcass that indicates a lua thread that is no longer referenced has been detected. The java thread in which this is thrown should correspond to a LuaThread being used as a coroutine that could not possibly be resumed again because there are no more references to the LuaThread with which it is associated. Rather than locking up resources forever, this error is thrown, and should fall through all the way to the thread's Thread.run() method."
Your code example doesn't cause all LuaThread references to disappear, so you shouldn't expect an exception to be thrown. CoroutineLib documentation indicates: Coroutines that are yielded but never resumed to complete their execution may not be collected by the garbage collector, so an OutOfMemoryError should actually be expected from the code you listed on SourceForge, if I'm not mistaken. LuaThread:52 also specifies: Applications should not catch OrphanedThread, because it can break the thread safety of luaj., which is yet another obstacle.
There also seem to be differences between empty and non-empty while loops in Lua/J. IIRC, empty loops (while true do end) don't obey all coroutine hook/tick rules. *Because no actions occur in an empty loop, there's no opportunity for certain hooks to occur (I need to test this again so please correct me otherwise!).
A forked version of LuaJ with the functionality we're looking for is used in the ComputerCraft mod for Minecraft, though it's designed only for the mod and isn't open source.
Why is the following multithreading related example code broken?
public void method1(){
synchronized(intVariable){
}
synchronized(stringVariable){
}
}
public void method2(){
synchronized(stringVariable){
}
synchronized(intVariable){
}
}
Above two methods are from same class where stringVariable and intVariable are instance variables.
I thought it will not cause any problem, at least with Thread deadlocks. Is there any other reason why this code is broken?
Either you didn't understand the problem, or you are right that this wouldn't cause a deadlock.
Perhaps he was looking for something more obscure like,
you can't lock an int field.
locking a String object is a very bad idea because you don't know how it is shared.
But I doubt it. In any case, he should have clarified the question and your answer because perhaps he might have learnt something, if only how to make the question clearer next time.
If you, as an interviewer, have a set of screening questions, you should make sure they are covered before you even bring in a candidate. A questionnaire to give to HR or an agent can be useful. A phone interview is often a good first set. As a candidate, I sometimes ask for a phone interview, just to see if it is worth my time going to a face to face. (e.g. if I have serious doubts its worth it)
Not only are you trying to convince them you are a good fit for them, but they are trying to convince you they are a good fit for you. It appears they failed both technically to explain the problem to you, and how they handled it HR wise, so I would count yourself lucky you didn't waste any more time with them.
BTW: Most big companies are diverse and working for one team can be very different to another team. It would be unfair to characterise a company based on one experience.
The problem is, assuming that both variables have a reference type (otherwise you couldn’t synchronize on them), that synchronizing on a variable whose contents could change is broken.
The first read of the variable is done without synchronization and whatever reference the thread will see (which could be a completely outdated value) is used to synchronize on, which does not prevent other threads from synchronizing on a different value of that variable as it will be a completely different object.
Since String and Integer are immutable each change of the variable’s value implies changing the reference contained in the variable, allowing another thread to enter the synchronized block while the thread performing the change is still inside that block.
And due to legal reordering of operations it might even appear as if the second thread performs actions inside the synchronized block before the first thread performs the write. Just recall that the read of the reference to use for synchronization is not synchronized. So it’s like having no synchronization at all.
I'm building a program based on the open source software JPiv, that is used to do digital image correlation and strain analysis. The algorithm in JPiv is, unfortunately, very slow, so I've been trying to use multithreading to decrease the time taken to analyse an image set. The algorithm uses JAI for something, not quite sure what as I've never used it before. When I run it, on the second 'pass', I get a OutOfMemory exception on at least one of the threads, and occasionally get other errors, seemingly at random, but usually IllegalArgument exceptions - sometimes from JAI, sometimes from standard Java libraries. The exceptions get thrown by different libraries at different points in the program, which makes it hard to debug - especially as they don't always give traceback messages in the console, for some reason.
I think the error is in the use of the JAI.create() method, and the way that the different threads access the corr[c] variable to use the above method. Is it possible to use JAI.create() in the way I'm trying to do so?
The code is quite lengthy, so I've put it on Pastebin here: http://pastebin.com/EX92YjXA
Below is a bit of pseudo-code to get a general sense of what I'm attempting.
public doPivEvaluation{
corr = new BufferedImage
start threads
send corr array to threads
loop until threads have finished
do stuff with corr}
public class threads{
on start{
do the analysis using tmpCorr
pb = new ParameterBlock()
pb.removeSources();
pb.removeParameters();
pb.addSource(PivEvaluation.corr[c]);
pb.addSource(tmpCorr);
PivEvaluation.corr[c] = JAI.create("add",pb,null).getAsBufferedImage();
end threads}
Uh, yeah, so maybe not the best pseudo-code ever, but yeah. The position in corr, c, is different for each thread (and refers to a set of pixels, so could be up to 5000 depending on the size of the image, split between the different threads), and corr has been declared volatile, so in theory there should be no overriding of data. Likewise, in theory, if JAI.create() only acts on that position, there should be no problem either. The problem arises because I don't understand how .create() actually works... I know the whole thing works in a single thread, because I've not changed anything of the actual algorithm, only moved it into multiple threads.
Also, apologies for any bad coding practices that may be in the code, I'm relatively new to Java still, so I'm more just muddling along. If I try something and it fixes a problem, I'm likely to go with it even it means multiple variable declarations or whatever other inefficiencies I've made. This is the first problem I've encountered I haven't been able to fix by guesswork and Google.