Is there a good tool to do end to end profiling of a system that uses J2EE and has clients?
I realize that you can easily profile the server and the client seperately. However, I do not know of any tools that'll give you information about the effect of the network connection between the two and how it affect's their performance. [An end-to-end profiler, would take in consideration of when the application is stuck on communicating and when it is processing]
I am not aware of any profiling agent that provides an out-of-the-box feature to profile applications end-to-end. At least, not at the time of writing this.
However, you could consider using DTrace to collect profiling information (not necessarily all the info provided by a typical profiler) from both the client and the server at the same time. An example on how to use this in a Java web-application (with Firefox as the client, and Tomcat as the server) is available in this article. The trick lies in having DTrace instrumentation built into the JVM (running the server) and the client, and in writing a DTrace script that writes the collected information from the traces into a parseable output. Since, the question isn't clear about whether the client is in Java I'll assume that the client is also in Java; if not, the application/executable must support DTrace instrumentation.
A few caveats are to be mentioned though:
DTrace is not available in all operating systems.
It is currently available in Solaris 10 and Mac OS Leopard.
I'm not aware of Dtrace support in Linux distributions, where SystemTap is the recommended alternative. FreeBSD has an experimental implementation of DTrace. It should be noted that SystemTap is not an implementation of DTrace for Linux, but a wholly different entity in itself.
On MSFT Windows, you're left to fend for yourself, atleast for the moment.
Assumptions about the JVM and the client (since I know that the server runs on a JVM)
If your client is a JVM then you must use a JVM that has DTrace instrumentation or equivalent (I'm referring to SystemTap here) built into it, or your DTrace/SystemTap scripts will not work. This means that unless the authors of the JRE/JDK for your distribution, have added code to support DTrace/SystemTap, you are constrained to use a JVM that has such a support built in. If it is of importance in your context, OpenJDK 1.6.0 appears have instrumentation support for SystemTap, and the Oracle/Sun Java 6 distributions for Solaris have support for DTrace. On Java 1.4 and 5 on Solaris, you'll need to install a JVMTI agent.
If your client is a web-browser or a thick-client written in a different language, then you must ensure that the process it is running under can support DTrace. For instance, DTrace instrumented Firefox is available on Solaris 10, and presumably on Solaris Express 11 (I have verified DTrace support only on OpenSolaris 2009.06). SystemTap probes haven't been added in Firefox for a lot of Linux distributions, so you'll often need to build Firefox from source, with the appropriate flags in the Firefox build script to add the probes; my previous experience with this process leads me to believe that this is doable, but isn't easy. If you are running a different application (neither Firefox, nor a JVM) as the client, then you'll need to verify DTrace/SystemTap support for it.
Writing good DTrace/SystemTap scripts to collect profiling information is not easy. You'll need to learn another language, and you'll need to ensure that the scripts do not add their own overhead (and validating the Heisenberg Uncertainty Principle in the context of profiling).
You can certainly write DTrace/SystemTap scripts that operate over remote clients and servers, but it is not advisable to write the collected instrumentation data to a socket (to avoid the afore-mentioned overhead).
Since version 8.0, JProfiler has the capability to follow RMI, web services and remote EJB calls between two profiled JVMs.
In the JVM that makes the call, call sites have hyperlinks that take you to the execution site in the remote JVM:
On the remote side, execution sites are recorded separately for each call sites, so you can inspect the call in isolation.
Disclaimer: My company develops JProfiler.
Related
I have been asked to investigate Oracle Java Mission Control, so that server-side Java applications may be monitored and actions taken (e.g., alerts emitted and logged, flight recordings saved) under certain conditions. Java Mission Control's trigger system, where you specify conditions and actions, meets our needs, but it seems to depend on the GUI application ("Oracle Java Mission Control") being running, implying that triggers are not the monitored JMX server's responsibility. Is this the case? There are a number of servers usually accessed via terminal...
Is there a way of running Java Mission Control as a daemon, from a terminal session, unattended, while retaining and obeying any specified trigger rules (e.g., imported from an XML file)?
If not, are there competing tools with a similar trigger system that can fill the void?
Thanks! :)
Currently no, you can't run JMC without a GUI.
You are not the first person that wants to do this.
One option is to run JMC in another machine, and make it connect to many servers, which of course requires running the remote JMX agent etc.
We have been discussing server side triggers/rules, but AFAIK, it is not planned for any JDK release.
It is possible to dump flight recordings from code, so you could write your own little agent that uses the DiagnosticMBean to do this on another JVM on the same machine or on remotely. I'm pretty sure this how some people solve the same sort of problem. It is also possible to parse and analyze flight recordings in code. If you're interested in this approach, I'm sure there's some sample code around, of course it's more work than if JMC could run as a daemon :/
You should probably have a look at an APM tool instead of monitoring with JMC. The product is extremely weak, introduces a lot of overhead (making it unsuitable for production) and creates a lot of issues. There are also developer focused tools available out there.
APM : AppDynamics (deepest of the bunch), New Relic, Ruxit
Java Developer Tools : Takipi, Fusion Reactor, Javosize
As a hobby I have been writing a shell in java. It's very basic, it include the creation, deletion, reading of files through a command line or through a desktop-like environment if a plugin is installed (which I also wrote).
Now my question is a very complicated one, but I hope someone can point me to the right direction.
Is there a layer or an operating system out there that has java
support, so I can run my java application on top of it, to serve as a
shell? Something minimal?
I don't fully understand the library inheritance in java to its full extent yet, and I'm not sure if AWT and swing applications are something that is built for the major OSs out there.
Can someone point me in the right direction? I just want to learn here, and I don't fully understand the full details of the layers underneath my java program.
There are (at least) 2 examples where Java is supported at the operating system level
Sun had a bare-metal operating system called JavaOS which is no longer available. I don't recall the exact details but I think that the target architecture was some kind of embedded processor.
There is an open source bare-metal Java operating system called JNode that runs on x86. It already has a command shell (the "bjorne" shell) and a primitive GUI.
The bad news with JNode is that it doesn't have a working "isolates" implementation, so it is impossible to stop a buggy applications from bringing the operating system to its knees. This has become a roadblock to progress ... especially since a viable isolates implementation probably would require a redesign of the system architecture to support a separate address space for each isolate.
Java doesn't interact directly with the OS, instead it uses the Java VM that serves as a mediator between the Java app and the hosting OS. Having this VM allows Java apps to be portable since it abstracts away the OS specifics. Java VM implementations differ for different OSs but that's probably not something you should be worrying about; the VM is available and distributed for most major OSs, just download and install the one for yours from http://www.java.com. After that you're free to go with your app.
I have found the DTrace intriguing but have personally failed to see a use-case that allow me to get information that I cannot get otherwise anyway without using DTrace.
Hence, I would like to hear what I have overlooked. What can I do on my vmware OpenIndiana build 148 with DTrace that can make a difference when creating stand-alone applications and Java EE web applications (most of which communicate heavily with a legacy backend using sockets)?
Non-trivial Dtrace scripts are very welcome.
I had some good experiences with DTrace.
I had a client running our Java code on a production server running 24x7. We had some performance issues with the applications. It was impossible to stop the JVMs in order to attach the profiler. Moreover the behavior was not present in our lab under the same load.
We solved the issue using DTrace with the JVM related probes as I could attach it to the running JVMs and the overhead introduced was minimal (1.3% on a Netra T2000 SPARC machine).
The bonus of the method was that all the debugging was done via a dial-up (33kbps) line to the customer's lab. It is almost impossible to run any other profiler/debugger with this constraints (JDWP is quite verbose for this bandwidth). With my DTrace script I filtered only what it was interesting for me.
For some hints see:
http://java.dzone.com/articles/java-profiling-dtrace?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+javalobby/frontpage+(Javalobby+/+Java+Zone)
http://download.oracle.com/javase/6/docs/technotes/guides/vm/dtrace.html
I have adapted those scripts
http://blogs.oracle.com/ahl/date/20050418#dtracing_java
Another big plus are the aggregated values that can be easily used to create custom statistics.
But, if you have appropriate conditions, everything can be done easier with othe tools. DTrace is very powerfull but it targets more bare-metal conditions (maybe DLight project would help).
I've written a Java file, using Jsp,servlets, that I would like to perform run-time tests on. I've never done this before and was just curious on how to go about it.
What I'm interested in knowing, besides the actual timings, is how to find cpu,memory and io utilization when running the application.Your thoughts are appreciated.
Typically you wouldn't measure these from within the application, but by running another tool on the same host.
If you just want to see the impact on the host operating system, you can use a program like top (on *nix boxes), or good old Task Manager on Windows, to see the CPU/memory/IO utilisation of your Java process (typically the servlet container such as Tomcat).
If you want more detailed information on the actual Java process itself, you can connect JConsole or jvisualvm to get VM information (including memory and CPU) for the process itself. (With Java 6 you should be able to do this from the local machine without passing any parameters to the Java process at startup; for Java 5, or remote connections, you'll need to pass command-line arguments to the Java process to allow (remote) JMX connections.)
Finally, if you want really in-depth details of the resource usage, down to the performance of various methods (which it sounds like you're after), you'll need to use a profiler. There are several of these for Java - with YourKit and JProfiler being the biggest commercial ones (in my unqualified opinion). I believe that the NetBeans IDE also has a decent profiler built-in. The process for connecting these to your application would vary depending on the app itself, but these will all typically allow you to "drill down" into the CPU time to see which classes/methods took the most cycles to execute, and likewise to drill down into memory use to see which classes are taking up the most memory.
The standard way to monitor running Java applications these days is sing JMX through the JConcole
If your a using a commercial application server like Weblogic or WebSphere these have custom and powerful management consoles that provide the monitoring information you are looking for. The technology at the heart of these consoles is still JMX so these can also be monitored and managed using the standard JConsole. This article shows how to do this for Weblogic.
I guess you need this info in the client side (browser). So it's not Java based question.
If so, here is my answer:
I prefer using FireBug and ySlow extensions. They give performance information, memory information and much more.
I combine it with using regular task-manager to view more information about the browser.
BR
I'm attempting to profile junit tests in eclipse on OSX using TPTP. I installed the TPTP package from eclipse with no problem, I went to profile some code and it tells me:
Agent Controller is unavailable under port 10002.
Make sure that the service is started and the port number is correct
under preferences.
I searched around on the internets and discovered many complains about Agent Controller not being available for os X. Almost all of the java developers I know run eclipse under os x. Why would eclipse not support os x? Why even make the TPTP packages available for os x if os x can't run agent controller? Purely for remote debugging purposes?
Os X is built on BSD, Linux has an Agent Controller port. Has this been ported to os x?
Any ideas or success stories getting TPTP to work on os x (remote debugging on linux, agent controller replacements)?
What other options are there (Netbeans, VisualVM, JProfiler) and how good are they?
What does stackoverflow suggest I do?
Note: I am interested in profiling on class and method, of the number of times each method is called, and the amount of time the program spends in each method. Due to the complexity of the code using System.nanotime() statements are not an option.
According to the Eclipse TPTP FAQ
What platforms and environments does TPTP support?
The user interface components of the
TPTP Infrastructure can run on any
Eclipse-supported platform. The
various test execution, monitoring and
tracing components typically run on a
broad range of infrastructure and
interface with a broad range of
systems. One notable deficiency at
present is that the various trace
facilities are currently
Java-specific, although the project
has a goal to extend additional
language support in due course. For
the other agents there is a reasonable
coverage implemented or planned for
popular operating systems, application
servers, web servers etc, with a
particular focus on providing support
for other open source products.
Related Questions:
How can I see what parts of my code are running the most
Using java to get os level system information
if you have a java application that is consuming cpu when it isnt doing anything
Related Links:
Open Source Profilers for Java
As far as other Java profilers go, I think this is the best Stack Overflow question:
Please Recommend a Java Profiler
I personally would recommend YourKit Java Profiler, I've found it much more user-friendly than TPTP, and (while I haven't tested it) it should work on OS X.
Have you seen if the Netbeans profiler works under OS X? It is the original of which jvisualvm is a subset. It might just work...