Does attaching a profiler cause some things to run slower then others? - java

Is it possible that attaching a profiler to a JVM (let's say VisualVM) could make some methods run slower, while not effecting others and thus causing a skew in the results that makes it look like a certain piece of code is a hotspot when in fact it's not. I will ask specifically about reflection calls for an example. I'm running some code that shows a lot of time spent in Spring AOP calls (specifically invokeJoinpointUsingReflection) - which the author says runs fine in testing (using an in code microbenchmark) but when they profiled it showed this method to take longer then other non-reflection methods. (sorry if that' a little unclear) So it got my wondering if the profiler could really have this effect and lead a developer down a false trail. Feel free to answer with any examples, the reflection part is just my example.

Profilers regularly give mis-leading information, but in generally they are usually right. Where they tend to skew the result is in very simple methods which might be further optimised if profiling wasn't enabled.
If in doubt I suggest you use another profiler, such as YourKit (evalation version should be fine) It has more light weight recording, but can have the same issues.

Heisenberg famously observed that collecting information from a system always disturbs it, so you can't get an undisturbed observation. (Thus the software term, "Heisenbug"). Yes, collecting profiling information can cause the actual performance to be changed in ways that will misdirect you.
Whether that is true in a significant way for your particular JVM or profiler, and how much disturbance occurs, is a matter of engineering.

Most profilers are sample based, and thus the more data you collect, the more accurate the results are. As far as I know, there is no bias for or against methods written purely in Java.

Certain profilers require a calibration step, e.g. NetBeans and VisualVM. You might verify the vintage and settings for your chosen profiler.

Related

JVM and hardware interactions profiling

Hi.
I searched for a while and stumbled about the lack of specific instrument. There're many good, even free profilers for Java, which allow to see how much time the code parts took overall, but nothing for the next tasks.
Given.
A Java multithreading program, doing some calculations through the misc.unsafe, using a simple fork-join pool, ThreadPoolExecutor, running on one Oracle JVM and Intel Xeon based server-class node. Looks like nothing hard, hot-spots are obvious. I need to figure out is any optimizations possible, which are promising and their potential assessment.
Needed.
For this, among the other information, I need a decent information about what exactly happens inside the hardware. For the research start, I want at least the cache misses and memory boundary overhead, byte-code or hardware instructions' overall timings. This would be very useful for avoiding "box ticking" approach. For clarification what exactly I need, I put here a similar instrument's wiki link which I used for a C or Fortran legacy code https://en.wikipedia.org/wiki/VTune (especially look briefly at the "Hardware event sampling" paragraph) and it fulfilled all my purposes.
Thanks in advance.

Detecting and pinpointing performance regressions

Are there any known techniques (and resources related to them, like research papers or blog entries) which describe how do dynamically programatically detect the part of the code that caused a performance regression, and if possible, on the JVM or some other virtual machine environment (where techniques such as instrumentation can be applied relatively easy)?
In particular, when having a large codebase and a bigger number of committers to a project (like, for example, an OS, language or some framework), it is sometimes hard to find out the change that caused a performance regression. A paper such as this one goes a long way in describing how to detect performance regressions (e.g. in a certain snippet of code), but not how to dynamically find the piece of the code in the project that got changed by some commit and caused the performance regression.
I was thinking that this might be done by instrumenting pieces of the program to detect the exact method which causes the regression, or at least narrowing the range of possible causes of the performance regression.
Does anyone know about anything written about this, or any project using such performance regression detection techniques?
EDIT:
I was referring to something along these lines, but doing further analysis into the codebase itself.
Perhaps not entirely what you are asking, but on a project I've worked on with extreme performance requirements, we wrote performance tests using our unit testing framework, and glued them into our continuous integration environment.
This meant that every check-in, our CI server would run tests that validated we hadn't slowed down the functionality beyond our acceptable boundaries.
It wasn't perfect - but it did allow us to keep an eye on our key performance statistics over time, and it caught check-ins that affected the performance.
Defining "acceptable boundaries" for performance is more an art than a science - in our CI-driven tests, we took a fairly simple approach, based on the hardware specification; we would fail the build if the performance tests exceeded a response time of more than 1 second with 100 concurrent users. This caught a bunch of lowhanging fruit performance issues, and gave us a decent level of confidence on "production" hardware.
We explicitly didn't run these tests before check-in, as that would slow down the development cycle - forcing a developer to run through fairly long-running tests before checking in encourages them not to check in too often. We also weren't confident we'd get meaningful results without deploying to known hardware.
With tools like YourKit you can take a snapshot of the performance breakdown of a test or application. If you run the application again, you can compare performance breakdowns to find differences.
Performance profiling is more of an art than a science. I don't believe you will find a tool which tells you exactly what the problem is, you have to use your judgement.
For example, say you have a method which is taking much longer than it used to do. Is it because the method has changed or because it is being called a different way, or much more often. You have to use some judgement of your own.
JProfiler allows you to see list of instrumented methods which you can sort by average execution time, inherent time, number of invocations etc. I think if this information is saved over releases one can get some insight into regression. Offcourse the profiling data will not be accurate if the tests are not exactly same.
Some people are aware of a technique for finding (as opposed to measuring) the cause of excess time being taken.
It's simple, but it's very effective.
Essentially it is this:
If the code is slow it's because it's spending some fraction F (like 20%, 50%, or 90%) of its time doing something X unnecessary, in the sense that if you knew what it was, you'd blow it away, and save that fraction of time.
During the general time it's being slow, at any random nanosecond the probability that it's doing X is F.
So just drop in on it, a few times, and ask it what it's doing.
And ask it why it's doing it.
Typical apps are spending nearly all their time either waiting for some I/O to complete, or some library function to return.
If there is something in your program taking too much time (and there is), it is almost certainly one or a few function calls, that you will find on the call stack, being done for lousy reasons.
Here's more on that subject.

Profile CPU usage in Java on a Mac

I'm looking for a way to measure the cpu usage for different methods in my java code. I understand that this can be achieved using JNI and C, but I wouldn't know where to start...
The purpose of this is to compare different algorithms, and provide qualitative results.
Probably the most common way is to use sampling. The JVM provides facilities to ask it the current stack trace of all threads (or ones you're interested in), along with how much CPU they've consumed. So you periodically do this. On each call, if a thread is inside the method you're interested in, then assume that it's spent half of the reported CPU time since the last poll inside that method.
If this method sounds appropriate, a little while back I wrote some material on the Java 5 profiling facilities that might help you.
Java 5 also provides an Instrumentation framework, by which you can doctor classes as they're being loaded in to include calls on the entry and exit to your given method, so you can measure CPU usage just inside that method. However, this is a little more complex to program because you need to doctor the actual class binaries as they're being loaded.
I don't think you can really identify CPU usage down to the method level with the current range of profilers. For most methods it's pretty obvious (if the method is compute-bound and single-threaded then it'll use 100% of CPU subject to allocation by the OS).
You may want to identify hot-spots though (methods consuming more CPU than you'd anticipate - or possibly less?) and I'd recommend looking at YourKit for an easy-to-configure profiler.
Failing that, take a look at the JVM Profiling Interface (JVMPI), which may give you some further pointers.
Sun VisualVM is integrated in recent JDK's and its profiling capabilities are explained here. Note that it seems to require a pretty up to date version of OSX if I understand this correctly.
Netbeans has basically the same profiling machinery on board, I don't know if that's of any help.
If you want to use one of already available profiling tools, then you can try Shark
Have a look at the OperatingSystemMXBean perhaps you could look at something before and after your method
long startProcessCpuTime = operatingSystemMXBean.getProcessCpuTime();
long endProcessCpuTime = operatingSystemMXBean.getProcessCpuTime();
Java6 only
I I'm not sure if this is what you want but I've used jrat for profiling in the past with decent results.
Check JaMon. Is very easy to use.
Call your methods in separate threads and measure the delta of time before and after execution of given procedure.
To measure the time of a process call:
ManagementFactory.getThreadMXBean().setThreadCpuTimeEnabled(true);
long threadTime = ManagementFactory.getThreadMXBean().getCurrentThreadCpuTime();

Jython Optimizations

Are their any ways to optimize Jython without resorting to profiling or significantly changing the code?
Specifically are there any flags that can be passed to the compiler, or code hints in tight loops.
No flags, no code hints. You can optimize by tweaking your code much as you would for any other Python implementation (hoisting, etc), but profiling helps by telling you where it's worth your while to expend such effort -- so, sure, you can optimize "without resorting to profiling" (and the code changes to do so may well be deemed to be not significant), but you're unlikely to guess right about where your time and energy are best spent, while profiling helps you determine exactly that.
Jython compiler does not offer lots of optimization choices. However, since the Java virtual machine (java) and perhaps compiler (javac) are getting invoked in the back end or at runtime, you should take a look at them.
Java has different runtime switches to use depending on whether you are going to launch it as a server process, client process, etc. You can also tell how much memory to allocate too.
I know this is an old question, but I'm just putting this for completeness.
You can use:-J-server flag to launch Jython in the Java server mode, which can help speed up the hot loops. (JVM will look to aggressively optimize, but might slow up the start up time)

Taking a snapshot of optimized JVM runtime

I know that the JVM can do some pretty serious optimizations at runtime, especially in -server mode. Of course, it takes a little while for the JVM to settle down and reach peak performance. Is there any way to take a snapshot of those optimizations so they can be applied immediately the next time you run your app?
"Hey JVM! Great job optimizing my code. Could you write that down for me for later?"
Basically not yet with Sun's VM, but they have it in mind.
See various postings/comments under here:
http://blogs.oracle.com/fatcatair/category/Java
(Sorry: I can't find quite the right one about retaining stats over restart for immediate C1 compilation of known-hot-at-startup methods.)
But I don't know where all this stuff is right now.
Note that optimisations appropriate in steady-state may well not be appropriate at start-up and might indeed reduce start-up performance, and indeed two runs may not have the same hotspots...
Perhaps this might help: http://wikis.sun.com/display/HotSpotInternals/PrintAssembly.

Categories

Resources