I'm currently using VisualVM, but the problem I'm having is that I can't save the graphs it generates. I need to report some data about its memory usage and running time, though running time is easy to get with System.nanoTime(). I've also tried the NetBeans profiler but it isn't what I want, since I'm not looking for specific parts that would be slowing it down or anything, so that would be overkill. The biggest problem with it is that it eats up too much processing time. Also doesn't let me capture/transfer the data easily, like VisualVM, at least as far as I can tell.
Ideally the best way to go about it would be some method call because then I'd be able to get the information a lot more easily, but anything like VisualVM that actually lets me save the graph is fine. Performance with VisualVM is pretty good too, compared to the NetBeans profiler, though I suppose that's because I wasn't using its profiler.
I'm currently using Ubuntu, but Windows 7 is fine. I'd rather have a program that specializes in doing this though, since the information gotten by programs who don't is likely to include the JVM and other things that would be better left out.
Well, apparently, you can save snapshots of the current session and maximize the window in VisualVM, so you could make the charts bigger, take a snapshot and cut them... But that's kind of a hack. Better suggestions welcome.
Runtime.getRuntime().freeMemory();
Runtime.getRuntime().totalMemory();
Look at the Runtime class. It has freeMemory, maxMemory, and totalMemory. That's probably close enough for your purposes.
You may prefer graceful method to measure memory, rather than hack image.
JConsole is known to Monitor Applications by JMX,it provides program API. I guess it is what you need.
See: Using JConsole to Monitor Applications
Try JProfiler. Although its not free you can try evaluation version first.
The HPjmeter console is free. Run your Java process with -Xloggc:<file> and open the <file> with it. Not only can you save your sessions, but you can compare runs. Other options to consider including in your command line are:
-XX:+PrintGCTimeStamps
-XX:+PrintGCDetails
Related
This might be a long shot of a question, but I have ran into a very complicated issue and I am unsure on how to solve it.
Long story short, we have a Java application running, it's currently using JDBC to pull in data from a MysQL Database on startup.
We have had a meltdown and that database is no longer active and has been lost forever and so has the data to go along with it which internally is very valuable.
However the data is still stored in the heap of the running JVM that pulled it in.
My only hope now is to somehow extract the data from the running JVM, in an ideal world i would be able to attach to it and have the flexibility to run code which could access the internal running classes..
So my questions today are:
Is my approach reasonable and possible?
If so how can I attach to the JVM and 'Inject' code
Thank you for reading
It seems that what you want to use is the jmap command. jmap can be used to dump the heap of a running JVM into a file, which you can then analyze "off-line", using tools such as jhat or JVisualVM.
It allows you to do so without killing the JVM and/or injecting code into it, and since the heap dump file is "inert", you can analyze it at your leisure without fear of harming the running VM by probing it further. Admittedly, I haven't used it extensively, so I'm not sure exactly what its capabilities are, but theoretically, you could perhaps also use JVisualVM's OQL language to run automated sequences on data in the heap and dump it to files in a format you want.
See, for instance, this question for usage examples.
In a situation like this the Eclipse Memory Analyzer Tool can be a good solution. It works on heap dumps too and shows you which objects take up memory.
In addition to this it can show the content of objects / memory locations.
I sometimes found MAT goes beyond what VisualVM does, but perhaps a view like this helps you find your data already:
(This is a screenshot of a made-up example where I create some custom objects in
order to show them with their value in the heapdump)
Perhaps you can even attach Eclipse to the running application. There is a certain trick where you can run custom code in a breakpoint. This one could somehow dump your data to disk.
I need to optimise a Java application. It makes some 3rd party calls. I need some good tool to accurately measure the time taken by individual API calls.
To give an idea of complexity-
the application takes a data source file containing 1 million rows, and it takes around one hour to complete the processing. As a part of processing , it makes some 3rd party calls (including some network calls). I need to identify which calls are taking more time then others, and based on that, find out a way to optimise the application.
Any suggestions would be appreciated.
I can recommend JVisualVM. It's a great monitoring / profiling tool that is bundled with the Oracle/Sun JDK. Just fire it up, connect to your application and start the CPU-profiling. You should get great histograms over where the time is spent.
Getting Started with VisualVM has a great screen-cast showing you how to work with it.
Screen shot:
Another more rudimentary alternative is to go with the -Xprof command line option:
-Xprof
Profiles the running program, and sends profiling data to
standard output. This option is provided as a utility that is
useful in program development and is not intended to be be
used in production systems.
I've been using YourKit a few times and what quite happy with it. I've however never profiled a long-running operation.
Is the processing the same for each row? In which case the size of the input file doesn't really matter. You could profile a subset to figure out which calls are expensive.
Just wanted to mention the inspectIT tool. It recently became completely open source (https://github.com/inspectIT/inspectIT). It provides complete and detailed call graph with contextual information, there are many out-of the box sensor for database calls, http monitoring, exceptions, etc.
Seams perfect for your use-case..
Try OPNET's Panorama software product
It sounds like a normal profiler might not be the right tool in this case, since they're geared towards measuring the CPU time taken by the program being profiled rather than external APIs that it calls, and they tend to incur a high overhead of their own and collect a large amount of data that would probably overwhelm your system if left running for a long time.
If you really need to collect performance data over such a long time, and mainly for external calls, then Perf4J is probably a better tool.
In our office we use YourKit profiler on a day to day basis. It's really light weight and serves most of the performance related use cases we have had.
But I have also used Visual VM. It's free and fast. You may first want to give Visual VM a try before going towards YourKit (YourKit is not freeware).
visualvm (part of the SDK) and Java 7 can produce detailed profiling.
I use profiler in NetBeans (it is really brilliant and already built in, no need to install plugin) or JVisualVM when not using NetBeans.
I have started to use visualvm for profiling my application which I launch in Eclipse. Then I launch visualvm which initially gives believable results.
After some time two processes appear in the monitor which consume huge amounts of time.
I have not deliberately invoked these. After a time they disappear. Are they an artefact of the profiling process and do I need to worry?
Very few of my routines appear in the profile, mainly the libraries they call. Is there a way of showing which routines call the most heavily used ones?
It is better to start with CPU sampling, if you don't know which part of the code is slow. Once you know better (based on the sampling results) what is going on, you can profile just part of your application, which is slow. You need to set profiling roots and instrumentation filter and don't forget to take the snapshot of collected results. See Profiling With VisualVM, Part 1 and Profiling With VisualVM, Part 2 to get more information about profiling and how to set profiling roots and instrumentation filter.
VisualVM uses Java to perform it's work. This means you will see some artefacts which relate to the RMI calls it makes. You can ignore them.
I use YourKit which doesn't do this, but it's not free ;)
VisualVM will track all methods being called by the java program it's monitoring, so either your program or one of its libraries is calling those methods. VisualVM is also connecting to it so there might be some small artifacts.
As for searching, probably the easiest way is to filter by your own packages. There is a space at the bottom where you can enter those so you can see which of your own methods is really taking time. Also you should pay attention to which thread you are in, usually you will want to look at the whatever is your "main" thread. The other threads are interesting but won't always give you the best idea of how your program behaves.
Is it possible tell the JVM to hint the OS that a certain object should preferably not get pushed out to swap space?
The short answer is no.
Java doesn't allow you any control over what is swapped in and what is 'pinned' into RAM.
Worrying about this sort of thing is usually a sign that there is something else wrong in your project. The OS will on the whole do a much better job of working out what should be swapped and what shouldn't. Your job is to write your software such that it doesn't try to second guess what underlying VM/OS is going to do, just concentrate on delivering your features and a good design.
This problem has also been very noticeable in Eclipse and the KeepResident dirty hack plugin (http://suif.stanford.edu/pub/keepresident/) avoids it.
It might be a good place to start? I have not seen it in widespread use so perhaps this has been integrated in the standard Eclipse distribution?
Hey! You are programming in a managed language. Why are you thinking about these? If you can't get these stuff out of your mind, you can always choose to program in C.
The short answer is (as given above): Dont' do it :-).
It would however be possible in principle. Most OS do allow to "lock" certain memory areas from being swapped (e.g. mlock(2) under Linux, VirtualLock under Windows).
The VM could expose this functionality to Java applications via a suitable API. However, no VM I know of does that, so to do it, you would first have to modify your VM...
If you access it regularly, that whatever page it happens to be in at the time (the JVM moves stuff around during garbage collection) will not be paged out unless something else is requesting memory even more aggressively. But there is no way of telling the JVM to not move it to another page, and the OS only knows about pages.
Not an answer, but lacking points to comment, I reserve this option :)
There are reasons to not store information in swap. Be it passwords or other confidential information that should not spend eternity on disk. Also, coming back after a weekend to my pc, I'd like some things to be in memory immediately available.
(Non Java) Natively there is probably some way to do this for each/most operating systems. With windows this is definitely possible. But not straight out of java (think JNI).
Depending on how desperate this option is, you could always look at using video memory, or some other hardware device that does not swap out. This allows you to still use a standardish java api, like jogl to store information. But somehow I doubt that is in context with the implementation/results you are looking for.
Basically you want to keep the whole JVM in main memory the whole time.
I am writing a simple checkers game in Java. When I mouse over the board my processor ramps up to 50% (100% on a core).
I would like to find out what part of my code(assuming its my fault) is executing during this.
I have tried debugging, but step-through debugging doesn't work very well in this case.
Is there any tool that can tell me where my problem lies? I am currently using Eclipse.
This is called "profiling". Your IDE probably comes with one: see Open Source Profilers in Java.
Use a profiler (e.g yourkit )
Profiling? I don't know what IDE you are using, but Eclipse has a decent proflier and there is also a list of some open-source profilers at java-source.
In a nutshell, profilers will tell you which part of your program is being called how many often.
I don't profile my programs much, so I don't have too much experience, but I have played around with the NetBeans IDE profiler when I was testing it out. (I usually use Eclipse as well. I will also look into the profiling features in Eclipse.)
The NetBeans profiler will tell you which thread was executing for how long, and which methods have been called how long, and will give you bar graphs to show how much time each method has taken. This should give you a hint as to which method is causing problems. You can take a look at the Java profiler that the NetBeans IDE provides, if you are curious.
Profiling is a technique which is usually used to measure which parts of a program is taking up a lot of execution time, which in turn can be used to evaluate whether or not performing optimizations would be beneficial to increase the performance of a program.
Good luck!
1) It is your fault :)
2) If you're using eclipse or netbeans, try using the profiling features -- it should pretty quickly tell you where your code is spending a lot of time.
3) failing that, add console output where you think the inner loop is -- you should be able to find it quickly.
Yes, there are such tools: you have to profile the code. You can either try TPTP in eclipse or perhaps try JProfiler. That will let you see what is being called and how often.
Use a profiler. There are many. Here is a list: http://java-source.net/open-source/profilers.
For example you can use JIP, a java coded profiler.
Clover will give a nice report showing hit counts for each line and branch. For example, this line was executed 7 times.
Plugins for Eclipse, Maven, Ant and IDEA are available. It is free for open source, or you can get a 30 day evaluation license.
If you're using Sun Java 6, then the most recent JDK releases come with JVisualVM in the bin directory. This is a capable monitoring and profiling tool that will require very little effort to use - you don't even need to start your program with special parameters - JVisualVM simply lists all the currently running java processes and you choose the one you want to play with.
This tool will tell you which methods are using all the processor time.
There are plenty of more powerful tools out there, but have a play with a free one first. Then, when you read about what other features are available out there, you'll have an inking about how they might help you.
This is a typically 'High CPU' problem.
There are two kind of high CPU problems
a) Where on thread is using 100% CPU of one core (This is your scenario)
b) CPU usage is 'abnormally high' when we execute certain actions. In such cases CPU may not be 100% but will be abnormally high. Typically this happens when we have CPU intensive operations in the code like XML parsing, serialization de-serialization etc.
Case (a) is easy to analyze. When you experience 100% CPU 5-6 thread dumps in 30 sec interval. Look for a thread which is active (in "runnable" state) and which is inside the same method (you can infer that by monitoring the thread stack). Most probably that you will see a 'busy wait' (see code below for an example)
while(true){
if(status) break;
// Thread.sleep(60000); // such a statement would have avoided busy wait
}
Case (b) also can be analyzed using thread dumps taken in equal interval. If you are lucky you will be able to find out the problem code, If you are not able to identify the problem code by using thread dump. You need to resort to profilers. In my experience YourKit profiler is very good.
I always try with thread dumps first. Profilers will only be last resort. In 80% of the cases we will be able to identify using thread dumps.
Or use JUnit test cases and a code coverage tool for some common components of yours. If there are components that call other components, you'll quickly see those executed many more times.
I use Clover with JUnit test cases, but for open-source, I hear EMMA is pretty good.
In single-threaded code, I find adding some statements like this:
System.out.println("A: "+ System.currentTimeMillis());
is simpler and as effective as using a profiler. You can soon narrow down the part of the code causing the problem.