I'm building a program based on the open source software JPiv, that is used to do digital image correlation and strain analysis. The algorithm in JPiv is, unfortunately, very slow, so I've been trying to use multithreading to decrease the time taken to analyse an image set. The algorithm uses JAI for something, not quite sure what as I've never used it before. When I run it, on the second 'pass', I get a OutOfMemory exception on at least one of the threads, and occasionally get other errors, seemingly at random, but usually IllegalArgument exceptions - sometimes from JAI, sometimes from standard Java libraries. The exceptions get thrown by different libraries at different points in the program, which makes it hard to debug - especially as they don't always give traceback messages in the console, for some reason.
I think the error is in the use of the JAI.create() method, and the way that the different threads access the corr[c] variable to use the above method. Is it possible to use JAI.create() in the way I'm trying to do so?
The code is quite lengthy, so I've put it on Pastebin here: http://pastebin.com/EX92YjXA
Below is a bit of pseudo-code to get a general sense of what I'm attempting.
public doPivEvaluation{
corr = new BufferedImage
start threads
send corr array to threads
loop until threads have finished
do stuff with corr}
public class threads{
on start{
do the analysis using tmpCorr
pb = new ParameterBlock()
pb.removeSources();
pb.removeParameters();
pb.addSource(PivEvaluation.corr[c]);
pb.addSource(tmpCorr);
PivEvaluation.corr[c] = JAI.create("add",pb,null).getAsBufferedImage();
end threads}
Uh, yeah, so maybe not the best pseudo-code ever, but yeah. The position in corr, c, is different for each thread (and refers to a set of pixels, so could be up to 5000 depending on the size of the image, split between the different threads), and corr has been declared volatile, so in theory there should be no overriding of data. Likewise, in theory, if JAI.create() only acts on that position, there should be no problem either. The problem arises because I don't understand how .create() actually works... I know the whole thing works in a single thread, because I've not changed anything of the actual algorithm, only moved it into multiple threads.
Also, apologies for any bad coding practices that may be in the code, I'm relatively new to Java still, so I'm more just muddling along. If I try something and it fixes a problem, I'm likely to go with it even it means multiple variable declarations or whatever other inefficiencies I've made. This is the first problem I've encountered I haven't been able to fix by guesswork and Google.
Related
In our Netty application. We are moving all blocking calls in our code to run in a special backgroundThreadGroup.
I'd like to be able to log in production the threadName and the lineNumber of the java code that is about to execute a blocking operation. (i.e. sync File and Network IO)
That way I can grep for the logs looking at places were we might have missed to move our blocking code to the backgroundThreadGroup.
Is there a way to instrument the JVM so that it can tell me that?
Depends on what you mean by a "blocking operation".
In a broad sense, any operation that causes a voluntary context switch is blocking. Trying to do something special about them is absolutely impractical.
For example, in Java, any method containing synchronized is potentially blocking. This includes
ConcurrentHashMap.put
SecureRandom.nextInt
System.getProperty
and many more. I don't think you really want to avoid calling all these methods that look normal at a first glance.
Even simple methods without any synchronization primitives can be blocking. E.g., ByteBuffer.get may result in a page fault and a blocking read on the OS level. Furthermore, as mentioned in comments, there are JVM level blocking operations that are not under your control.
In short, it's impractical if not impossible to find all places in the code where a blocking operation happens.
If, however, you are interested in finding particular method calls that you believe are bad (like Thread.sleep and Socket.read), you can definitely do so. There is a BlockHound project specifically for this purpose. It already has a predefined list of "bad" methods, but can be customized with your own list.
There is a library called BlockHound, that will throw an exception unless you have configured BlockHound to ignore that specific blocking call
This is how you configure BlockHound for Netty: https://github.com/violetagg/netty/blob/625f9d5781ed85bfaca6fa4e826d0d46d70fdbd8/common/src/main/java/io/netty/util/internal/Hidden.java
(You can improve the above code by replacing the last line with builder.nonBlockingThreadPredicate(
p -> p.or(thread -> thread instanceof FastThreadLocalThread)); )
see https://github.com/reactor/BlockHound
see https://blog.frankel.ch/blockhound-how-it-works/
I personally used it to find all blocking call within our Netty based service.
Good Luck
The prevailing religion of Java today heavily forbids the use of the stop() instance method on a Thread[1] [2] [3]. It's marked as Deprecated in the official documentation, with a message that begins with:
This method is inherently unsafe. Stopping a thread with Thread.stop causes it to unlock all of the monitors that it has locked (as a natural consequence of the unchecked ThreadDeath exception propagating up the stack). If any of the objects previously protected by these monitors were in an inconsistent state, the damaged objects become visible to other threads, potentially resulting in arbitrary behavior. [...]
There is even a whole article supplied that goes into further detail. (Especially that ThreadDeath is silent when it gets to the top, and even if you guard against it, you can't guard against Thread.stop(Throwable t) anyway.)
In general, I agree with what that article is worried about, and with the answers to the related question on StackOverflow — in almost all the use cases in the world there is a much better way than stop(). But not all.
Here is a use case that I'm struggling to think of a better way for. I invite your suggestions.
Use case
I'm working with an interactive console running on top of Java, that allows the user run arbitrary Java statements and Python functions that call Java, in an interactive interpreted way. (See Jython console for screenshots.)
Now, the user can write arbitrary functions. Some of them might call Java methods that result in infinite loops that print a lot of output, almost saturating the system, so that it becomes painfully slow to respond.
When the user messes up like this, I need to give them a chance to kill the thread (using a button), and salvage the remainder of the work that they have produced in their session and stored in local variables.
If calling interrupt() on the thread fails, (e.g. if it never goes into waiting state while stuck in its infinite loop), then we are faced with two options that I can see: (1) kill the application outright or (2) use stop() on the bad thread, and let the user salvage whatever is left that hadn't been corrupted.
Yes I know that some objects may have been damaged and won't work properly. But we're talking about salvaging anything that may have been left in a valid state (which is probably pretty much everything, except one or two things).
Can anybody see any problems with this reasoning? And if this turns out to be a valid use-case, then does it mean the method shouldn't be deprecated :) ?
Calling Thread.stop() is a bad idea. End of story.
It may work in practice but you sacrifice most of the JVM's concurrency guarantees in the process. Your whole program is essentially running undefined behavior. It isn't simply the thread or the data in the thread that may become corrupted, but any part of the JVM that happens to be in a vulnerable state when the thread is killed.
It sounds like you're looking for someone to confirm that your use case somehow avoids the risks to the JVM. It doesn't, so you're not likely to get such a confirmation. If you don't see issues more power to you, but don't be surprised when it fails in inexplicable or dangerous ways.
As Peter Lawrey suggests you should be running this untrusted code in an isolated JVM, where the OS's process management can support killing resource-hogging processes. If you can't do that because you're passing around strange resources like open file descriptors that is your problem.
I'm working on a Java project and I've come upon an interesting design issue. It's not exactly a problem, but it is a bit ugly with the obvious solution.
I have a class implementing Callable, although with the current implementation it could just as easily be a Runnable as I'm not interested in the outcome, at least not as far as the calling thread is concerned. The calling thread will drop a number of these into a thread pool. Each of these Callables will have a batch of data that was obtained from an external service. The Callables will perform a number of actions, many of which involve making further calls to external services. As a result, there are a good number of places where various Exceptions could be thrown.
The issue I find is that depending on where the Exception occurs, I may need to take different actions. If it happens at point A, then delete the data on the external service. If it happens at point B, move the data to a different location on the server. If it happens at point C, just log it and do nothing further, etc. Any number of Exception types could be thrown at multiple points, although I don't think I'll need to do much filtering on the type itself, but more that one occurred.
The Callable itself isn't terribly huge, so there's not a great deal of code to mess with. However, I am hesitant to kludge it up with a ton of try/catch blocks to handle every possible point/Exception that may need different handling. I realize that this may really be the only viable solution. I don't really have control over most of the Exceptions that will be thrown (maybe a handful) without catching an existing one and rethrowing my own, which seems a bit redundant. I'm wondering if there's a good pattern or method to handle this sort of thing.
I've considered an exception handling class, but I'd still need to catch each Exception somehow and pass it to the handler as the point at which the Exception was thrown is important. I could break the Callable down into more atomic classes, each with their own small block and handling, but that would be trading one kludge for another. Catching everything in the call() method outright or by grabbing the Exception from the Future in the calling thread really isn't an option as this will lose the data on where it occurred unless I want to parse the stack trace, which isn't exactly viable.
Can anyone shed some light? Maybe I'm just quibbling over the try/catch blocks and should just go ahead with it, but I feel like there must be a better way...
Hmmm, it does occur to me that annotations on methods might help here. I could break down all methods until there's only one possible exception-throwing piece of code in each. Annotate each of these with a custom annotation that dictates what is done when that method throws an exception. I'm not sure if it is possible (an exception would somehow need to be caught right there as it may happen within a loop going over each piece of data and only one piece may be problematic, or at least somehow mark that piece for processing further up the chain), but perhaps this could mitigate the need for lots of try/catch blocks and instead handle the behavior with a single annotation and a handler class to deal with the exceptions. I don't believe it's possible to dictate behavior this way with an annotation, but I'd be happy to be wrong on that.
Why is the following multithreading related example code broken?
public void method1(){
synchronized(intVariable){
}
synchronized(stringVariable){
}
}
public void method2(){
synchronized(stringVariable){
}
synchronized(intVariable){
}
}
Above two methods are from same class where stringVariable and intVariable are instance variables.
I thought it will not cause any problem, at least with Thread deadlocks. Is there any other reason why this code is broken?
Either you didn't understand the problem, or you are right that this wouldn't cause a deadlock.
Perhaps he was looking for something more obscure like,
you can't lock an int field.
locking a String object is a very bad idea because you don't know how it is shared.
But I doubt it. In any case, he should have clarified the question and your answer because perhaps he might have learnt something, if only how to make the question clearer next time.
If you, as an interviewer, have a set of screening questions, you should make sure they are covered before you even bring in a candidate. A questionnaire to give to HR or an agent can be useful. A phone interview is often a good first set. As a candidate, I sometimes ask for a phone interview, just to see if it is worth my time going to a face to face. (e.g. if I have serious doubts its worth it)
Not only are you trying to convince them you are a good fit for them, but they are trying to convince you they are a good fit for you. It appears they failed both technically to explain the problem to you, and how they handled it HR wise, so I would count yourself lucky you didn't waste any more time with them.
BTW: Most big companies are diverse and working for one team can be very different to another team. It would be unfair to characterise a company based on one experience.
The problem is, assuming that both variables have a reference type (otherwise you couldn’t synchronize on them), that synchronizing on a variable whose contents could change is broken.
The first read of the variable is done without synchronization and whatever reference the thread will see (which could be a completely outdated value) is used to synchronize on, which does not prevent other threads from synchronizing on a different value of that variable as it will be a completely different object.
Since String and Integer are immutable each change of the variable’s value implies changing the reference contained in the variable, allowing another thread to enter the synchronized block while the thread performing the change is still inside that block.
And due to legal reordering of operations it might even appear as if the second thread performs actions inside the synchronized block before the first thread performs the write. Just recall that the read of the reference to use for synchronization is not synchronized. So it’s like having no synchronization at all.
We have JSF2.1 app deployed in weblogic10.3.4 ,in one of our backing bean ,when we try to assign the reference ArrayList to a List instance ,weblogic ends up in Struck thread ,during peak traffic to our application.
java.util.ArrayList.indexOf(ArrayList.java:210)
java.util.ArrayList.contains(ArrayList.java:199)
Any one has faced this issue before.
It is not entirely clear what you mean, so I'm going to assume that you mean a "stuck thread", and that the thread is stuck in the sense that it is continually executing at that point.
I can think of three plausible causes.
The object that is being searched for has a buggy equals(Object) method that in some circumstances goes into an infinite loop.
There are two (or more) threads accessing and/or updating the list roughly simultaneously, and you are not synchronizing properly. If you don't synchronize properly, there is a risk that the threads will see inconsistent views of the data structure, and that that will cause it behave in a way that seems impossible.
You've somehow set up a pathological situation that is causing one thread to be both reading and updating the list in the (incorrect) belief that it has two distinct lists.
My bet is that it is the second problem, since "heisenbugs" like that are more likely to occur when your server is under heavy load.
Finally, it is possible that the thread is not in an infinite loop, but is just taking a long time to do something. And it is possible that the loop involves other code, but each time you look at it is at that point.