I have a big program in Java that uses multithreading. In some cases, the program starts using 100% of three cores of my eight core system. In normal use, the program use all cores at 1-2%. How can I find the class that's overloading cores?
Use a profiler such as the jvisualvm that is bundled with jdk-1.6.0_10
The best solution is to use a profiler - that's what they're built for, and there's a great one bundled with Java 6.
Another (far from being as ideal a solution) is to run your program in the Eclipse IDE (if that's what you use) in debug mode. You can then look at the running threads. IF a lot of them are suspended, the one that is not might be your culprit. Force it to break (from the toolbar) and you can see where it is. There are many chances that you'll find a clear loop or busy waiting.
Use a profiler to figure out which thread(s) are using all of your CPU cycles, and the method(s) they are executing.
If you are using Eclipse, you can use the TPTP profiling tool.
If you are going the commercial profiler route then I would recommend using Dynatrace.
Try taking threads dump (see jps, jstack commands) and then see which methods are executed.
If you are using Java over UNIX or some versions of Linux look into DTrace with Java.
Related
What is the easiest way in Java for achieving multi-core? And by this, I mean, to specifically point out on what core to execute some parts of the project, so good-old "normal" java threads are not an option.
So far, I was suggested JConqurr (which is an Eclipse toolkit for multi-core programming in java), JaMP (which extends Java for OpenMP), and MPJ express, of which I don't know much. Which of the above do you consider the best, or do you have other suggestions? It would be preferable to somehow mesure the performance boost/gain, but not exclusive.
Any help would be much appreciated.
Thanks,
twentynine.
Even though it is easy to write multi-threaded code in Java, there is nothing in the Java standard runtime which generically allow you to tell the JVM or the operating system how to schedule your program.
Hence you will need to have code specifically for your JVM and/or your operating system, and that code may not be doable in Java (unless you dive into JNI or JNA). External programs can pin processes to a CPU in many Unix versions (and probably Windows too), but I don't think you can do this for individual threads.
Scala is quite popular for this. It runs on the JVM and has bindings for Java to hook them together.
I'm currently using VisualVM, but the problem I'm having is that I can't save the graphs it generates. I need to report some data about its memory usage and running time, though running time is easy to get with System.nanoTime(). I've also tried the NetBeans profiler but it isn't what I want, since I'm not looking for specific parts that would be slowing it down or anything, so that would be overkill. The biggest problem with it is that it eats up too much processing time. Also doesn't let me capture/transfer the data easily, like VisualVM, at least as far as I can tell.
Ideally the best way to go about it would be some method call because then I'd be able to get the information a lot more easily, but anything like VisualVM that actually lets me save the graph is fine. Performance with VisualVM is pretty good too, compared to the NetBeans profiler, though I suppose that's because I wasn't using its profiler.
I'm currently using Ubuntu, but Windows 7 is fine. I'd rather have a program that specializes in doing this though, since the information gotten by programs who don't is likely to include the JVM and other things that would be better left out.
Well, apparently, you can save snapshots of the current session and maximize the window in VisualVM, so you could make the charts bigger, take a snapshot and cut them... But that's kind of a hack. Better suggestions welcome.
Runtime.getRuntime().freeMemory();
Runtime.getRuntime().totalMemory();
Look at the Runtime class. It has freeMemory, maxMemory, and totalMemory. That's probably close enough for your purposes.
You may prefer graceful method to measure memory, rather than hack image.
JConsole is known to Monitor Applications by JMX,it provides program API. I guess it is what you need.
See: Using JConsole to Monitor Applications
Try JProfiler. Although its not free you can try evaluation version first.
The HPjmeter console is free. Run your Java process with -Xloggc:<file> and open the <file> with it. Not only can you save your sessions, but you can compare runs. Other options to consider including in your command line are:
-XX:+PrintGCTimeStamps
-XX:+PrintGCDetails
I have a fairly complex Java application that is utilizing a lot of CPU, and would like to know if there is any recommendation on a profile tool that I could use to determine the cause of the CPU utilization. My goal is to pinpoint the location in the code where it is using most of the CPU time.
I've used jProfiler and YourKit on Linux. But you might find the information you're looking for by running the jconsole that comes with recent JDKs. Good information on how to use it: http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html
Netbeans has good built-in profiler. NB runs quite well on Ubuntu 9.10
If your issue cannot be observed in developement, you can attach the profiler to a remote JVM (your production environment). You have to configure the host JVM to accept the JMS/JConsole connections from the profilers. The JMS/JConsole realtime monitoring visualization charts look pretty awesome.
I have used Visual VM in the eclipse IDE. If you are not on an IDE, jhat and jconsole will help. Google jhat for more info
If you are comfortable with Eclipse, I'd recommend TPTP.
If you use Sun Java 6 then learn to use jvisualvm in the JDK on the machine running the troublesome program.
It will answer most of your questions, can attach effortlessly to an already running program and is gratis.
This method is low-tech but works best. Here's a short explanation why.
Since you are not running under an IDE, you can use pstack or lsstack to get stack samples. If the app is using lots more time than it should, then most likely the problem is one or a few rogue function calls, which will be clearly visible on most of the stack samples.
A Java application I support that runs on JRE 1.4.2_12 is hanging near midnight every night. I'd like to try and record as much profiling information as I can to discover if there is an issue in the JVM or external to the app.
I'd like to use HPROF to collect as much information as possible.
Is there a way to have HPROF dump its cpu sample and memory allocation report every minute instead of at the termination of the JVM?
Is there a different, more appropriate profiler that can collect information like this?
Rather than relying on dump files, I would try hooking up a profiler to the VM and leave it attached until the hang up occurs. Then use the profiler to introspect the state of the threads.
The use of Java 1.4 is a minor issue here, since 1.4's debug interface is not great, but some profilers still support it. I can particularly recommend YourKit, which is commercial, but offers an evaluation licence. It's the best profiler I've used, but some margin.
First things first: did you analyze the thread dump when your application hangs? A lot of the time that has enough information to troubleshoot a hanging java app...
Ctrl-Break in the process window on Windows, or kill -QUIT [pid] on Linux.
I would first try to determine if its actually your app or something else.
Are there any other apps on the box, if so do they run any batch around midnight. It could be a situation of your app suffering from a lack of resources due to other things running on the box or chewing up bandwidth.
Was this always the case or did it start recently. If this is new look at what changed on the box as a whole not just your own app.
I am writing a simple checkers game in Java. When I mouse over the board my processor ramps up to 50% (100% on a core).
I would like to find out what part of my code(assuming its my fault) is executing during this.
I have tried debugging, but step-through debugging doesn't work very well in this case.
Is there any tool that can tell me where my problem lies? I am currently using Eclipse.
This is called "profiling". Your IDE probably comes with one: see Open Source Profilers in Java.
Use a profiler (e.g yourkit )
Profiling? I don't know what IDE you are using, but Eclipse has a decent proflier and there is also a list of some open-source profilers at java-source.
In a nutshell, profilers will tell you which part of your program is being called how many often.
I don't profile my programs much, so I don't have too much experience, but I have played around with the NetBeans IDE profiler when I was testing it out. (I usually use Eclipse as well. I will also look into the profiling features in Eclipse.)
The NetBeans profiler will tell you which thread was executing for how long, and which methods have been called how long, and will give you bar graphs to show how much time each method has taken. This should give you a hint as to which method is causing problems. You can take a look at the Java profiler that the NetBeans IDE provides, if you are curious.
Profiling is a technique which is usually used to measure which parts of a program is taking up a lot of execution time, which in turn can be used to evaluate whether or not performing optimizations would be beneficial to increase the performance of a program.
Good luck!
1) It is your fault :)
2) If you're using eclipse or netbeans, try using the profiling features -- it should pretty quickly tell you where your code is spending a lot of time.
3) failing that, add console output where you think the inner loop is -- you should be able to find it quickly.
Yes, there are such tools: you have to profile the code. You can either try TPTP in eclipse or perhaps try JProfiler. That will let you see what is being called and how often.
Use a profiler. There are many. Here is a list: http://java-source.net/open-source/profilers.
For example you can use JIP, a java coded profiler.
Clover will give a nice report showing hit counts for each line and branch. For example, this line was executed 7 times.
Plugins for Eclipse, Maven, Ant and IDEA are available. It is free for open source, or you can get a 30 day evaluation license.
If you're using Sun Java 6, then the most recent JDK releases come with JVisualVM in the bin directory. This is a capable monitoring and profiling tool that will require very little effort to use - you don't even need to start your program with special parameters - JVisualVM simply lists all the currently running java processes and you choose the one you want to play with.
This tool will tell you which methods are using all the processor time.
There are plenty of more powerful tools out there, but have a play with a free one first. Then, when you read about what other features are available out there, you'll have an inking about how they might help you.
This is a typically 'High CPU' problem.
There are two kind of high CPU problems
a) Where on thread is using 100% CPU of one core (This is your scenario)
b) CPU usage is 'abnormally high' when we execute certain actions. In such cases CPU may not be 100% but will be abnormally high. Typically this happens when we have CPU intensive operations in the code like XML parsing, serialization de-serialization etc.
Case (a) is easy to analyze. When you experience 100% CPU 5-6 thread dumps in 30 sec interval. Look for a thread which is active (in "runnable" state) and which is inside the same method (you can infer that by monitoring the thread stack). Most probably that you will see a 'busy wait' (see code below for an example)
while(true){
if(status) break;
// Thread.sleep(60000); // such a statement would have avoided busy wait
}
Case (b) also can be analyzed using thread dumps taken in equal interval. If you are lucky you will be able to find out the problem code, If you are not able to identify the problem code by using thread dump. You need to resort to profilers. In my experience YourKit profiler is very good.
I always try with thread dumps first. Profilers will only be last resort. In 80% of the cases we will be able to identify using thread dumps.
Or use JUnit test cases and a code coverage tool for some common components of yours. If there are components that call other components, you'll quickly see those executed many more times.
I use Clover with JUnit test cases, but for open-source, I hear EMMA is pretty good.
In single-threaded code, I find adding some statements like this:
System.out.println("A: "+ System.currentTimeMillis());
is simpler and as effective as using a profiler. You can soon narrow down the part of the code causing the problem.