Is there a way to remote start a CPU profiler (and specify package exclusions) like you can with the VisualVM tool (through I assume JMX).
We want to be able to automatically start a server run a stress test with a specific CPU profiler configuration enabled, and then export the results. Also is there any library that would allow you to parse the binary CPU profiler output (.nps file).
I think my best bet would be to dig through the VisualVM source code and see how they're doing it and copy that.
It appears that VisualVM uses some Netbeans library to do the parsing of the CPU profiler:
* org.netbeans.lib.profiler.results.cpu.StackTraceSnapshotBuilder
If a simple profiler is enough, you can integrate it into your application. I wrote one. It's also available with Apache license. I and my co-workers use it quite a lot. It's not perfect, but very easy to integrate.
Related
in java programs, running the programs with java profiler will generate profiling output
Ex- java -prof classfile_name
How to get same profiling output of java programs running on Tomcat?
I want to get an profiling output like this
count callee caller time , like this in profiling output
1 java.util.LinkedHashMap.init()V java.util.HashMap.(IF)V 0
Can you tell me how to get an output like this for programs running on Tomcat?
Tomcat is also just a Java application, so you can add the same VM parameters by defining the JAVA_OPTS environment variable in startup.bat/startup.sh of your Tomcat distribution.
However, I would really recommend to use a real profiler, there are several free and commercial alternatives. The major free one is visualvm and as a commercial solution I recommend JProfiler (disclaimer: my company develops that tool).
In those profilers, look for the "Hot spot" views which will show you the kind of information that you are looking for.
I have developed a nice multi-Threaded genetic algorithm in Java that runs on a 16-core system running CentOS with 128GB of RAM.
I want to use a code profiler to see if I can figure out which portion of the code is getting bogged down when I increase the number of mutations in my simulation beyond a certain point. Memory doesn't seem to be the issue.
So I installed VNC and Eclipse 3.6SR2 on the server and installed the TPTP plugin.
PROBLEM: The biggest issue is that it doesn't look like eclipse is using more than one core when I am doing the TPTP "execution time analysis" (I checked using 'top'). Normally when the program is run from the command line it uses as many cores as there are threads in the program.
Is there a way to fix this in the eclipse configuration?
Disclaimer: My company develops JProfiler
A profiler that uses JVMTI should not change the multi-core thread distribution with respect to a regular execution. TPTP may not be the best option for you.
There are several powerful Java profilers in the market. The most well-known free option is VisualVM and a commmercial alternative with much more powerful analytic capabilities in the area of multi-threading and monitor contention analysis is JProfiler - there's a fully functional free trial.
Get a real profiler like YourKit and add the agent to your application at startup.
Then you can open an SSH tunnel with the port where the agent is running and you can remotely profile your application. It has a quite good documentation and a healthy community in their forums. And in my opinion YourKit is great for multithreaded applications, I use it a lot.
No need for VNC and installing Eclipse on a production server.
I'm looking for a tool which can profile the java application running as a windows service ( remotely/locally either way) .
I've come across VisualVM as one option. Are there any other products available other than VisualVm.I'm more interested in reputated product. Can Jprofiler do this for me?
VisualVM , Does it give class wise list of profiling ?
See this article, if you want to use Java VisualVM. It describes how java application running as a Windows service can be monitored and/or profiled using VisualVM.
JVisualVM is you best shot. It's free, comes with the JVM and gives you a pretty decent range of functionality. Not sure what you mean by "class wise list of profiling." But it will show you where the majority of your execution time is spent.
You can execute jvsiualvm by going to $JAVA_HOME/bin and typing jvisualvm. Then select the the vm you wish to profile.
You can use BTrace to instrument your application and to measure the parts of the application that you are interested in. BTrace logs its output to files, which you can later or in real-time transfer to a monitoring application, such as EurekaJ (which I've written myself).
Both tools are Open Sourced and free to use. BTrace uses the "GNU Public License v.2 w/Classpath Exception" license, while EurekaJ uses the GPLv3 license.
InfoQ Writeup on the two tool including VisualVM and a few commandline tools: http://www.infoq.com/articles/java-profiling-with-open-source
Link How can I monitor my Java application running as Windows Service with Visual VM? says that on recent Windows versions only JMX mode can be used, but not local.
I have a fairly complex Java application that is utilizing a lot of CPU, and would like to know if there is any recommendation on a profile tool that I could use to determine the cause of the CPU utilization. My goal is to pinpoint the location in the code where it is using most of the CPU time.
I've used jProfiler and YourKit on Linux. But you might find the information you're looking for by running the jconsole that comes with recent JDKs. Good information on how to use it: http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html
Netbeans has good built-in profiler. NB runs quite well on Ubuntu 9.10
If your issue cannot be observed in developement, you can attach the profiler to a remote JVM (your production environment). You have to configure the host JVM to accept the JMS/JConsole connections from the profilers. The JMS/JConsole realtime monitoring visualization charts look pretty awesome.
I have used Visual VM in the eclipse IDE. If you are not on an IDE, jhat and jconsole will help. Google jhat for more info
If you are comfortable with Eclipse, I'd recommend TPTP.
If you use Sun Java 6 then learn to use jvisualvm in the JDK on the machine running the troublesome program.
It will answer most of your questions, can attach effortlessly to an already running program and is gratis.
This method is low-tech but works best. Here's a short explanation why.
Since you are not running under an IDE, you can use pstack or lsstack to get stack samples. If the app is using lots more time than it should, then most likely the problem is one or a few rogue function calls, which will be clearly visible on most of the stack samples.
I am attempting to solve performance issues with a large and complex tomcat java web application. The biggest issue at the moment is that, from time to time, the memory usage spikes and the application becomes unresponsive. I've fixed everything I can fix with log profilers and Bayesian analysis of the log files. I'm considering running a profiler on the production tomcat server.
A Note to the Reader with Gentle Sensitivities:
I understand that some may find the very notion of profiling a production app offensive. Please be assured that I have exhausted most of the other options. The reason I am considering this is that I do not have the resources to completely duplicate our production setup on my test server, and I have been unable to cause the failures of interest on my test server.
Questions:
I am looking for answers which work either for a java web application running on tomcat, or answer this question in a language agnostic way.
What are the performance costs of profiling?
Any other reasons why it is a bad idea to remotely connect and profile a web application in production (strange failure modes, security issues, etc)?
How much does profiling effect the memory foot print?
Specifically are there java profiling tools that have very low performance costs?
Any java profiling tools designed for profiling web applications?
Does anyone have benchmarks on the performance costs of profiling with visualVM?
What size applications and datasets can visualVM scale to?
OProfile and its ancestor DPCI were developed for profiling production systems. The overhead for these is very low, and they profile your full system, including the kernel, so you can find performance problems in the VM and in the kernel and libraries.
To answer your questions:
Overhead: These are sampled profilers, that is, they generate timer or performance counter interrupts at some regular interval, and they take a look at what code is currently executing. They use that to build a histogram of where you spend your time, and the overhead is very low (1-8% is what they claim) for reasonable sampling intervals.
Take a look at this graph of sampling frequency vs. overhead for OProfile. You can tune the sampling frequency for lower overhead if the defaults are not to your liking.
Usage in production: The only caveat to using OProfile is that you'll need to install it on your production machine. I believe there's kernel support in Red Hat since RHEL3, and I'm pretty sure other distributions support it.
Memory: I'm not sure what the exact memory footprint of OProfile is, but I believe it keeps relatively small buffers around and dumps them to log files occasionally.
Java: OProfile includes profiling agents that support Java and that are aware of code running in JITs. So you'll be able to see Java calls, not just the C calls in the interpreter and JIT.
Web Apps: OProfile is a system-level profiler, so it's not aware of things like sessions, transactions, etc. that a web app would have.
That said, it is a full-system profiler, so if your performance problem is caused by bad interactions between the OS and the JIT, or if it's in some third-party library, you'll be able to see that, because OProfile profiles the kernel and libraries. This is an advantage for production systems, as you can catch problems that are due to misconfigurations or particulars of the production environment that might not exist in your test environment.
VisualVM: Not sure about this one, as I have no experience with VisualVM
Here's a tutorial on using OProfile to find performance bottlenecks.
I've used YourKit to profile apps in a high-load production environment, and while there was certainly an impact, it was easily an acceptable one. Yourkit makes a big deal of being able to do this in a non-invasive manner, such as selectively turning off certain profiling features that are more expensive (it's a sliding scale, really).
My favourite aspect of it is that you can run the VM with the YourKit agent running, and it has zero performance impact. it's only when you connect the GUI and start profiling that it has an effect.
There is nothing wrong in profiling production apps. If you work on distributed applications, there are times when a outofmemory exception occurs in a very unique probability scenario which is very difficult to reproduce in a dev/stage/uat environment.
You can try using custom profilers but if you are in a hurry and plugging in/ setting upa profiler on a production box will take time, you can also use the jvm to take a memory dump(jvms memory dump also gives you thread dump)
You can activate the automatic generation on the JVM command line, by using the following option :
-XX:+HeapDumpOnOutOfMemoryError
he Eclipse Memory Analyzer project has a very powerful feature called “group by value”, which makes it possible to build an object query and regroup the instances by a field value. This is useful in the case where you have a lot of instances that are containing a smaller set of possible values, and you can to see which values are being used the most. This has really helped me understand some complex memory dumps so I recommend you try it out.
You may also consider using one of the modern HotSpot JVM - Java Flight Recorder and Java Mission Control. It is a set of tools that allow you to collect low-level runtime information with the CPU overhead about 5% (I cannot prove the last statement anyhow, this is the statement of Oracle engineer who presented the feature and live demo).
You can use this tool as long as your application is running 1_7u40 JVM or higher. To enable the runtime info collection, you need to start JVM with particular flags:
By default, JFR is disabled in the JVM. To enable JFR, you must launch your Java application with the -XX:+FlightRecorder option. Because JFR is a commercial feature, available only in the commercial packages based on Java Platform, Standard Edition (Oracle Java SE Advanced and Oracle Java SE Suite), you also have to enable commercial features using the -XX:+UnlockCommercialFeatures options.
(Quoted http://docs.oracle.com/javase/8/docs/technotes/guides/jfr/about.html#sthref7)
I added this answer as this is viable option for profiling in production IMO.
Also there is an Eclipse plugin that supports JFR and JMC and capable of displaying information user-friendly.
The tools have improved vastly over the years. These days, most people who have needs like these use a tool that hooks into Java's instrumentation API instead of the profiling API. Surely there are more examples, but NewRelic and AppDynamics come to mind. Instrumentation-based solutions usually run as an agent in the JVM and constantly collect data. They report the data at a higher level (business transaction, web transaction, database transaction) than the old profiling approach and allow you to dig deeper (down to the method or line) if necessary. You can even setup monitoring and alerts, so you can track/alert on metrics like page load times and performance against SLAs. With these great tools, you really should have no reason to run a profiler in production any longer. The cost of running them is negligible.