I'm attempting to profile junit tests in eclipse on OSX using TPTP. I installed the TPTP package from eclipse with no problem, I went to profile some code and it tells me:
Agent Controller is unavailable under port 10002.
Make sure that the service is started and the port number is correct
under preferences.
I searched around on the internets and discovered many complains about Agent Controller not being available for os X. Almost all of the java developers I know run eclipse under os x. Why would eclipse not support os x? Why even make the TPTP packages available for os x if os x can't run agent controller? Purely for remote debugging purposes?
Os X is built on BSD, Linux has an Agent Controller port. Has this been ported to os x?
Any ideas or success stories getting TPTP to work on os x (remote debugging on linux, agent controller replacements)?
What other options are there (Netbeans, VisualVM, JProfiler) and how good are they?
What does stackoverflow suggest I do?
Note: I am interested in profiling on class and method, of the number of times each method is called, and the amount of time the program spends in each method. Due to the complexity of the code using System.nanotime() statements are not an option.
According to the Eclipse TPTP FAQ
What platforms and environments does TPTP support?
The user interface components of the
TPTP Infrastructure can run on any
Eclipse-supported platform. The
various test execution, monitoring and
tracing components typically run on a
broad range of infrastructure and
interface with a broad range of
systems. One notable deficiency at
present is that the various trace
facilities are currently
Java-specific, although the project
has a goal to extend additional
language support in due course. For
the other agents there is a reasonable
coverage implemented or planned for
popular operating systems, application
servers, web servers etc, with a
particular focus on providing support
for other open source products.
Related Questions:
How can I see what parts of my code are running the most
Using java to get os level system information
if you have a java application that is consuming cpu when it isnt doing anything
Related Links:
Open Source Profilers for Java
As far as other Java profilers go, I think this is the best Stack Overflow question:
Please Recommend a Java Profiler
I personally would recommend YourKit Java Profiler, I've found it much more user-friendly than TPTP, and (while I haven't tested it) it should work on OS X.
Have you seen if the Netbeans profiler works under OS X? It is the original of which jvisualvm is a subset. It might just work...
Related
As a hobby I have been writing a shell in java. It's very basic, it include the creation, deletion, reading of files through a command line or through a desktop-like environment if a plugin is installed (which I also wrote).
Now my question is a very complicated one, but I hope someone can point me to the right direction.
Is there a layer or an operating system out there that has java
support, so I can run my java application on top of it, to serve as a
shell? Something minimal?
I don't fully understand the library inheritance in java to its full extent yet, and I'm not sure if AWT and swing applications are something that is built for the major OSs out there.
Can someone point me in the right direction? I just want to learn here, and I don't fully understand the full details of the layers underneath my java program.
There are (at least) 2 examples where Java is supported at the operating system level
Sun had a bare-metal operating system called JavaOS which is no longer available. I don't recall the exact details but I think that the target architecture was some kind of embedded processor.
There is an open source bare-metal Java operating system called JNode that runs on x86. It already has a command shell (the "bjorne" shell) and a primitive GUI.
The bad news with JNode is that it doesn't have a working "isolates" implementation, so it is impossible to stop a buggy applications from bringing the operating system to its knees. This has become a roadblock to progress ... especially since a viable isolates implementation probably would require a redesign of the system architecture to support a separate address space for each isolate.
Java doesn't interact directly with the OS, instead it uses the Java VM that serves as a mediator between the Java app and the hosting OS. Having this VM allows Java apps to be portable since it abstracts away the OS specifics. Java VM implementations differ for different OSs but that's probably not something you should be worrying about; the VM is available and distributed for most major OSs, just download and install the one for yours from http://www.java.com. After that you're free to go with your app.
Is there a good tool to do end to end profiling of a system that uses J2EE and has clients?
I realize that you can easily profile the server and the client seperately. However, I do not know of any tools that'll give you information about the effect of the network connection between the two and how it affect's their performance. [An end-to-end profiler, would take in consideration of when the application is stuck on communicating and when it is processing]
I am not aware of any profiling agent that provides an out-of-the-box feature to profile applications end-to-end. At least, not at the time of writing this.
However, you could consider using DTrace to collect profiling information (not necessarily all the info provided by a typical profiler) from both the client and the server at the same time. An example on how to use this in a Java web-application (with Firefox as the client, and Tomcat as the server) is available in this article. The trick lies in having DTrace instrumentation built into the JVM (running the server) and the client, and in writing a DTrace script that writes the collected information from the traces into a parseable output. Since, the question isn't clear about whether the client is in Java I'll assume that the client is also in Java; if not, the application/executable must support DTrace instrumentation.
A few caveats are to be mentioned though:
DTrace is not available in all operating systems.
It is currently available in Solaris 10 and Mac OS Leopard.
I'm not aware of Dtrace support in Linux distributions, where SystemTap is the recommended alternative. FreeBSD has an experimental implementation of DTrace. It should be noted that SystemTap is not an implementation of DTrace for Linux, but a wholly different entity in itself.
On MSFT Windows, you're left to fend for yourself, atleast for the moment.
Assumptions about the JVM and the client (since I know that the server runs on a JVM)
If your client is a JVM then you must use a JVM that has DTrace instrumentation or equivalent (I'm referring to SystemTap here) built into it, or your DTrace/SystemTap scripts will not work. This means that unless the authors of the JRE/JDK for your distribution, have added code to support DTrace/SystemTap, you are constrained to use a JVM that has such a support built in. If it is of importance in your context, OpenJDK 1.6.0 appears have instrumentation support for SystemTap, and the Oracle/Sun Java 6 distributions for Solaris have support for DTrace. On Java 1.4 and 5 on Solaris, you'll need to install a JVMTI agent.
If your client is a web-browser or a thick-client written in a different language, then you must ensure that the process it is running under can support DTrace. For instance, DTrace instrumented Firefox is available on Solaris 10, and presumably on Solaris Express 11 (I have verified DTrace support only on OpenSolaris 2009.06). SystemTap probes haven't been added in Firefox for a lot of Linux distributions, so you'll often need to build Firefox from source, with the appropriate flags in the Firefox build script to add the probes; my previous experience with this process leads me to believe that this is doable, but isn't easy. If you are running a different application (neither Firefox, nor a JVM) as the client, then you'll need to verify DTrace/SystemTap support for it.
Writing good DTrace/SystemTap scripts to collect profiling information is not easy. You'll need to learn another language, and you'll need to ensure that the scripts do not add their own overhead (and validating the Heisenberg Uncertainty Principle in the context of profiling).
You can certainly write DTrace/SystemTap scripts that operate over remote clients and servers, but it is not advisable to write the collected instrumentation data to a socket (to avoid the afore-mentioned overhead).
Since version 8.0, JProfiler has the capability to follow RMI, web services and remote EJB calls between two profiled JVMs.
In the JVM that makes the call, call sites have hyperlinks that take you to the execution site in the remote JVM:
On the remote side, execution sites are recorded separately for each call sites, so you can inspect the call in isolation.
Disclaimer: My company develops JProfiler.
I have developed a nice multi-Threaded genetic algorithm in Java that runs on a 16-core system running CentOS with 128GB of RAM.
I want to use a code profiler to see if I can figure out which portion of the code is getting bogged down when I increase the number of mutations in my simulation beyond a certain point. Memory doesn't seem to be the issue.
So I installed VNC and Eclipse 3.6SR2 on the server and installed the TPTP plugin.
PROBLEM: The biggest issue is that it doesn't look like eclipse is using more than one core when I am doing the TPTP "execution time analysis" (I checked using 'top'). Normally when the program is run from the command line it uses as many cores as there are threads in the program.
Is there a way to fix this in the eclipse configuration?
Disclaimer: My company develops JProfiler
A profiler that uses JVMTI should not change the multi-core thread distribution with respect to a regular execution. TPTP may not be the best option for you.
There are several powerful Java profilers in the market. The most well-known free option is VisualVM and a commmercial alternative with much more powerful analytic capabilities in the area of multi-threading and monitor contention analysis is JProfiler - there's a fully functional free trial.
Get a real profiler like YourKit and add the agent to your application at startup.
Then you can open an SSH tunnel with the port where the agent is running and you can remotely profile your application. It has a quite good documentation and a healthy community in their forums. And in my opinion YourKit is great for multithreaded applications, I use it a lot.
No need for VNC and installing Eclipse on a production server.
I have found the DTrace intriguing but have personally failed to see a use-case that allow me to get information that I cannot get otherwise anyway without using DTrace.
Hence, I would like to hear what I have overlooked. What can I do on my vmware OpenIndiana build 148 with DTrace that can make a difference when creating stand-alone applications and Java EE web applications (most of which communicate heavily with a legacy backend using sockets)?
Non-trivial Dtrace scripts are very welcome.
I had some good experiences with DTrace.
I had a client running our Java code on a production server running 24x7. We had some performance issues with the applications. It was impossible to stop the JVMs in order to attach the profiler. Moreover the behavior was not present in our lab under the same load.
We solved the issue using DTrace with the JVM related probes as I could attach it to the running JVMs and the overhead introduced was minimal (1.3% on a Netra T2000 SPARC machine).
The bonus of the method was that all the debugging was done via a dial-up (33kbps) line to the customer's lab. It is almost impossible to run any other profiler/debugger with this constraints (JDWP is quite verbose for this bandwidth). With my DTrace script I filtered only what it was interesting for me.
For some hints see:
http://java.dzone.com/articles/java-profiling-dtrace?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+javalobby/frontpage+(Javalobby+/+Java+Zone)
http://download.oracle.com/javase/6/docs/technotes/guides/vm/dtrace.html
I have adapted those scripts
http://blogs.oracle.com/ahl/date/20050418#dtracing_java
Another big plus are the aggregated values that can be easily used to create custom statistics.
But, if you have appropriate conditions, everything can be done easier with othe tools. DTrace is very powerfull but it targets more bare-metal conditions (maybe DLight project would help).
I have a fairly complex Java application that is utilizing a lot of CPU, and would like to know if there is any recommendation on a profile tool that I could use to determine the cause of the CPU utilization. My goal is to pinpoint the location in the code where it is using most of the CPU time.
I've used jProfiler and YourKit on Linux. But you might find the information you're looking for by running the jconsole that comes with recent JDKs. Good information on how to use it: http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html
Netbeans has good built-in profiler. NB runs quite well on Ubuntu 9.10
If your issue cannot be observed in developement, you can attach the profiler to a remote JVM (your production environment). You have to configure the host JVM to accept the JMS/JConsole connections from the profilers. The JMS/JConsole realtime monitoring visualization charts look pretty awesome.
I have used Visual VM in the eclipse IDE. If you are not on an IDE, jhat and jconsole will help. Google jhat for more info
If you are comfortable with Eclipse, I'd recommend TPTP.
If you use Sun Java 6 then learn to use jvisualvm in the JDK on the machine running the troublesome program.
It will answer most of your questions, can attach effortlessly to an already running program and is gratis.
This method is low-tech but works best. Here's a short explanation why.
Since you are not running under an IDE, you can use pstack or lsstack to get stack samples. If the app is using lots more time than it should, then most likely the problem is one or a few rogue function calls, which will be clearly visible on most of the stack samples.