Please explain about the steps involved in profiling a JAVA application? This is irrespective of what ever profiling tools that is used. What are the best practices and steps involved in profiling the java applications?
Experts, any links or documents are really appreciated.
Thanks.
Thanks. The thing I want to know is there are so many profilers available but when we profile a Java applications for OutOfMemory or Memoryleaks , etc . What are the steps we need to go through in profiling the application. Let say I am using VisualVM which does have a profiler , I am getting an OutOfMemory Error in my application my application so huge that I don't know where exactly is the problem even the logger is of no use (just for assumption). In such case how we can figure out where exactly is the problem by using the profiler tool like VisualVM? And what are the steps we need to look into ? Whether we can directly use the CPU and Memory profiling or still we need to go and get the thread dump and analyse it , then create a Heap dump analyze it and then go for a CPU and Memory profiling? I am little confused here. Hence please point me to the right direction as well as the steps involved in profiling a JAVA application to find the memory leaks. Hope I am clear with my question.
Depending on why you need to profile your application you have to decide what filters you will need, as mentioned inthe comments the question is very general, you should provide some more percise information to get some help hier.
Try the following link (in Eclipse):
An introduction to profiling Java applications
And check this List or Open source java profilers
i have used JProfiler and yourkit but i am not satisfied with output for actual performance tuning. currently we have been switched to java melody. This not only help performance optimization in dev but also in production system. Java melody is very easy to integrate and configure and in production you can enable or disable by just updating web.xml
This series of articles should give you a good idea on how you go about serious performance investigation of a relatively complex Java application.
http://www.jinspired.com/solutions/case-studies/scala-compiler
Related
I need to optimise a Java application. It makes some 3rd party calls. I need some good tool to accurately measure the time taken by individual API calls.
To give an idea of complexity-
the application takes a data source file containing 1 million rows, and it takes around one hour to complete the processing. As a part of processing , it makes some 3rd party calls (including some network calls). I need to identify which calls are taking more time then others, and based on that, find out a way to optimise the application.
Any suggestions would be appreciated.
I can recommend JVisualVM. It's a great monitoring / profiling tool that is bundled with the Oracle/Sun JDK. Just fire it up, connect to your application and start the CPU-profiling. You should get great histograms over where the time is spent.
Getting Started with VisualVM has a great screen-cast showing you how to work with it.
Screen shot:
Another more rudimentary alternative is to go with the -Xprof command line option:
-Xprof
Profiles the running program, and sends profiling data to
standard output. This option is provided as a utility that is
useful in program development and is not intended to be be
used in production systems.
I've been using YourKit a few times and what quite happy with it. I've however never profiled a long-running operation.
Is the processing the same for each row? In which case the size of the input file doesn't really matter. You could profile a subset to figure out which calls are expensive.
Just wanted to mention the inspectIT tool. It recently became completely open source (https://github.com/inspectIT/inspectIT). It provides complete and detailed call graph with contextual information, there are many out-of the box sensor for database calls, http monitoring, exceptions, etc.
Seams perfect for your use-case..
Try OPNET's Panorama software product
It sounds like a normal profiler might not be the right tool in this case, since they're geared towards measuring the CPU time taken by the program being profiled rather than external APIs that it calls, and they tend to incur a high overhead of their own and collect a large amount of data that would probably overwhelm your system if left running for a long time.
If you really need to collect performance data over such a long time, and mainly for external calls, then Perf4J is probably a better tool.
In our office we use YourKit profiler on a day to day basis. It's really light weight and serves most of the performance related use cases we have had.
But I have also used Visual VM. It's free and fast. You may first want to give Visual VM a try before going towards YourKit (YourKit is not freeware).
visualvm (part of the SDK) and Java 7 can produce detailed profiling.
I use profiler in NetBeans (it is really brilliant and already built in, no need to install plugin) or JVisualVM when not using NetBeans.
I need to do a performance analysis of Java EE web application and optimize the code.
Please suggest ways of doing it?
To start with, I am checking the server logs.
Based on your vague question an answer can just be vague:
Depending on what you want to improve, the first rule is to measure what you want to improve. Furthermore alsways measure again after you tried to improve!
Memory
Regarding memory optimizations you should acquire heap dumps of the running application and analyze those. A very helpful tool for an anylysis is the eclipse memory analyzer tools.
Profiling
If you want to improve the performance and minimize runtime of code, you should start with profiling. JVisualVM is then a good tool. To get some load on your application JMeter can help you in the context of a web based application.
Rules of Performance tuning
First measure to identify the bottlenecks, then pick the "biggest" leaks for optimization. After optimizing measure again to verify your result. If you are not happy afterwards, start again with measuring.
Know the real slow parts of your application
Before even starting with measuring you should exactly identify the situations where your application is really slow, otherwise you might not notice a difference or even "de-optimize".
Use some good java profiler and figure out problem points like high memory usage, high CPU usage etc.
Look at YourKit and/or jprofiler. You can use their trial version for your case
Multiple tools are available to do performance analysis.
You can use Jmeter to do some load testing and see what performance you are getting. If you find performance bad for certain features then dig into that to find the bottlenecks.
You can use JProfiler to analyse JVM of the web application.
Try using a application monitoring tool like newrelic , it will tell you which server side components have the slowest response times, and then it will let you drill down to which calls within that application consume the most resources, that should be a good start ...
Anyone know of any good plugins for eclipse (Helios 3.6.1) so that I can investigate particular metrics relating to the run-time performance of a java application? It would be nice if I could get my hands on something that provides me with extensive runtime metrics such that I can investigate why a program might take longer to run than another program.
I tried TPTP documentation is out of date and I don't know how to use it.
You could try out VisualVM which has a runtime CPU and memory profiler. Or you can go with the common stack traces method.
There is a Java Struts application running on Tomcat, that have some memory errors. Sometimes it becomes slowly and hoard all of the memory of Tomcat, until it crashes.
I know how to find and repair "normal code errors", using tests, debugging, etc, but I don't know how to deal with memory errors (How can I reproduce? How can I test? What are the places of code where is more common create a memory error? ).
In one question: Where can I start? Thanks
EDIT:
A snapshot sended by the IT Department (I haven't direct access to the production application)
Use one of the many "profilers". They hook into the JVM and can tell you things like how many new objects are being created per second, and what type they are etc.
Here's just one of many: http://www.ej-technologies.com/products/jprofiler/overview.html
I've used this one and it's OK.
http://kohlerm.blogspot.com/
It is quite good intro how to find memory leaks using eclipse memory analyzer.
If you prefer video tutorials, try youtube, although it is android specific it is very informative.
If your application becomes slowly you could create a heap dump and compare it to another heap dump create when the system is in a healthy condition. Look for differences in larger data structures.
You should run it under profiler (jprofile or yourkit, for example) for some time and see for memory/resource usage. Also try to make thread dumps.
There are couple of options profiler is one of them, another is to dump java heap to a file and analyze it with a special tool (i.e. IBM jvm provides a very cool tool called Memory Analizer that presents very detailed report of allocated memory in the time of jvm crash - http://www.ibm.com/developerworks/java/jdk/tools/memoryanalyzer/).
3rd option is to start your server with jmx server enabled and connect to it via JConsole with this approach you would be able to monitor memory ussage/allocation in the runtime. JConsole is provided with standard sun jdk under bin directory (here u may find how to connect to tomcat via jconsole - Connecting remote tomcat JMX instance using jConsole)
Occasionally I have to do some profiling work on Java code, and I would like to know why I should have my boss investigate in a commercial profiler as opposed to just use the one in Netbeans or JConsole?
What would the killer features be that would warrant the investment?
In my experience with JProfiler, it's just an all-round slicker experience than the NetBeans profiler. It's easier to get started, easier to interpret the information and, although I haven't measured it, it seems that JProfiler has less of a negative impact on the performance of the application being profiled.
Also, JProfiler integrates nicely with IntelliJ IDEA. I have to use NetBeans to use the NetBeans profiler, which is an inconvenience because I have to manually configure a free-form project to match the layout of my project.
The NetBeans profiler is usable. Unlike IntelliJ, I wouldn't buy a JProfiler licence for my personal projects because, unlike an IDE, it's not a tool you use all day every day. However, for paid work there's no reason not to buy a better tool. It's not expensive compared to the cost of a developer's time.
I have experience using both NetBeans profiler and JProbe. For performance profiling I have found Netbeans quite useful but where JProbe is superior is for memory profiling.
JProbe has superior tools for comparing heap snapshots and finding the root cause of a memory leak. For example, in JProbe you can view heap shapshots visually as a graph, select nodes to investigate and then delete references to see if the instance could then be garbage collected.
If you are using Netbeans already then starting up the profiler is easy (unless you are using a Maven based project... sigh).
I have used paid profilers as well as the Netbeans one. Netbeans does the job well enough (it was a bit rough when it first came out... but much better now).
The code I profile isn't HUGE so I cannot say if the time spent in profiling is a major factor.
The answer is highly subjective and totally depends on your needs. Things to look at:
1) ease of use in your environment (in the case of NetBeans it is likely that the built in profiler is easiest.
2) time spent starting the prfiler to it actually getting you usable results
3) is it a sampling or tracing profiler? (An overview is here: http://docs.hp.com/en/5992-0757/ch05s01.html
4) can you view the results live or do you have to wait for the profiling to finish?
Here is a link to a slashdot discussion on Java profilers: http://ask.slashdot.org/article.pl?sid=06/06/30/0053237
I've not used Netbeans profiler, but tried JProfiler, Yourkit and JProbe. I found Yourkit slightly better (mainly bought by the usability aspect). Some of the useful features in it are: (you can check if it is available in Netbeans)
J2EE Profiling (Eg. It shows how much time an SQL query took).
Snapshot comparison and annotation
Deadlock detector
Exception telemetry
You can check for more details at their site.
I would say that, ready to use and more performance statistics. I was assigned a profiling job last year when I was interning at a multinational. I used the InfraRED profiler which uses Java aspect oriented API (works with both Aspectwerkz and AspectJ). But I had to extend the profiler to get what my manager wanted. Also, the performance statistics given by the profiler was limited.
But before selecting the profiler I researched a few other opensource profilers. Some of them were trivial and didnt suit what we wanted.
I would also add that, some of them just doesnt work. For example, if we want to collect performance statistics of a web application, all the profilers doesnt support those statistics required for us.
With a completely independent profiler, it's much easier to integrate it with other applications in your toolchain. For example, say you want to run the profiler as part of your build process (say, once a night). Something like JProfiler easily integrates with ANT, whereas profilers built into IDEs may or may not. If you have a separate build machine, installing a local copy of a profiler makes sense, but installing a whole IDE just to get access to one component does not.
If you are using Tomcat you might consider lambdaprobe
http://www.lambdaprobe.org/
(It is for free)
From my experience, YourKit profiler is most usable one. Small usability things really make the difference, but also it is most comprehensive one, containing:
most comprehensive and usable memory snapshots (working also with 1GB+ heaps), with detail object view and primitive data, for every single object. (for example in hashmap you can see if objects are evenly distributed or most are stored in same bucket!) This detail of memory snapshots and its ease of use is my main reason for yourkit.
very little overhead (far less then many other profilers I used)
comparing snapshots
J2EE profiling
deadlock detector, lock status (I think it still misses java.concurrent.locks, bud for synchronized it is great)
Among other things, it's also constantly improving, so who knows what is future holding :)
Compare the features and see if you really need the features provided by commercial software over the free one. If yes then its worth investing.