How to make performance tests on Java? Such as time of execution, a track of using memory, a track of operations per second and may be there are another helpful tests. How to realize this tests without an influence to the real application work (without this test)? I mean that I could be confident that my application would be work with the same performance on average without those tests.
Note: I need to attach a specific tool to my test class. I know that there is many different tools to test a huge amount of VM parameters of an application. I need to write a test class with distinct test parameters which values I can handle as I want. It would be good if API supported graphical GUI for some those parameters.
You can use tools like jconsole, visual vm especially for the memory. These come bundled with the Jdk.
As for the speed of your application, typically log messages can help you with that. If the log messages follow a standard you can pretty much write your own script to give you a pretty formatted result. Also, it is a good idea to write test classes to give you a fair idea of the performance for varying loads.
http://visualvm.java.net/gettingstarted.html
I would recommend jProfile for performance test and jConsole for Monitoring memory usage etc...
I am under the assumption that your servers are hosted on the linux env. Jconsole and Visualvm are indeed helpful to monitor your memory and performance. Just as a work around you can also use sar, top and mpstat commands to get your required values.
You will have to install the sar package on your linux server by command sudo apt-get install sysstat. You can refer "http://www.thegeekstuff.com/2011/03/sar-examples/?utm_source=feedburner" to learn more on the commnads.
You don't need admin access to configure any of these commnds..
Related
I am evaluating a product which works by instrumenting bytecode.
I launch my application via the provided agent which instruments bytecode, profiles and generates reports about my application.
What would be the best way to determine the performance penalty imposed by running the agent?
Would it be enough if I were to capture latency of a few of operation in my app via JMH with and without the agent?
Also, is there a baseline expected drop in performance by using an agent which does bytecode instrumentation?
Thanks
You can download an existing Java performance benchmark like SPECjvm2008 and run it with/without your agent. I wouldn't only write a microbenchmark sourced from the application you are monitoring because that might not highlight the bottlenecks of various operations and instrumentation techniques used by this product(method/memory/system).
The baseline that is advertised for a typical agent is 5%, which is a number I would take with a huge grain of salt.
If your application is some kind of a server, you can launch it with an agent and run a separate JMH benchmark that would measure calls to your app. That way you have your app running with or without this agent and benchmark code wouldn't interfere with it. But you'll have to isolate a lot of factors like call latency, cpu usage by the caller (if you run caller from the same host), calls distribution (you have a lot of different calls on a server, but only a couple in a benchmark), etc.
If instead you would benchmark inside a process agent is attached to, I wouldn't trust this result, unless actions take really long time, like >5ms.
I need to optimise a Java application. It makes some 3rd party calls. I need some good tool to accurately measure the time taken by individual API calls.
To give an idea of complexity-
the application takes a data source file containing 1 million rows, and it takes around one hour to complete the processing. As a part of processing , it makes some 3rd party calls (including some network calls). I need to identify which calls are taking more time then others, and based on that, find out a way to optimise the application.
Any suggestions would be appreciated.
I can recommend JVisualVM. It's a great monitoring / profiling tool that is bundled with the Oracle/Sun JDK. Just fire it up, connect to your application and start the CPU-profiling. You should get great histograms over where the time is spent.
Getting Started with VisualVM has a great screen-cast showing you how to work with it.
Screen shot:
Another more rudimentary alternative is to go with the -Xprof command line option:
-Xprof
Profiles the running program, and sends profiling data to
standard output. This option is provided as a utility that is
useful in program development and is not intended to be be
used in production systems.
I've been using YourKit a few times and what quite happy with it. I've however never profiled a long-running operation.
Is the processing the same for each row? In which case the size of the input file doesn't really matter. You could profile a subset to figure out which calls are expensive.
Just wanted to mention the inspectIT tool. It recently became completely open source (https://github.com/inspectIT/inspectIT). It provides complete and detailed call graph with contextual information, there are many out-of the box sensor for database calls, http monitoring, exceptions, etc.
Seams perfect for your use-case..
Try OPNET's Panorama software product
It sounds like a normal profiler might not be the right tool in this case, since they're geared towards measuring the CPU time taken by the program being profiled rather than external APIs that it calls, and they tend to incur a high overhead of their own and collect a large amount of data that would probably overwhelm your system if left running for a long time.
If you really need to collect performance data over such a long time, and mainly for external calls, then Perf4J is probably a better tool.
In our office we use YourKit profiler on a day to day basis. It's really light weight and serves most of the performance related use cases we have had.
But I have also used Visual VM. It's free and fast. You may first want to give Visual VM a try before going towards YourKit (YourKit is not freeware).
visualvm (part of the SDK) and Java 7 can produce detailed profiling.
I use profiler in NetBeans (it is really brilliant and already built in, no need to install plugin) or JVisualVM when not using NetBeans.
It's difficult to find all bottlenecks, deadlocks, and memory leaks in a Java application using unit tests alone.
I'd like to add some level of stress testing for my application. I want to test the limits of the application and determine how it reacts under high load.
I'd like to gauge the following:
Availablity under high load
Performance under high load
Memory / CPU / Disk Usage under high load
Does it crash under high load or react gracefully
It would also be interesting to measure and contrast such characteristics under normal load.
Are their well known, standard techniques to address stress testing.
I am looking for help / direction in setting up such an environment.
Ideally, I would like to run these tests regularly, so that wecan determine if recent deliveries impact performance.
I am a big fan of JMeter. You can set up calls directly against the server just as users would access it. You can control the number of user (concurrent threads) and accesses. It can follow a workflow, scraping pertinent information page to page. It takes 1 to 2 days to learn it well enough to be productive. (You can do the basics within an hour of downloading!)
As for seeing how all that affects the server, that is a tougher question. I have used professional tools from CA and IBM. (I am drawing a blank on specific tool names - maybe due to PTSD!) I have used out-of-the-box JVM profilers. I have used native linux and windows tools. If you are not too concerned about profiling what parts of your application causes issues, then you can just use the native tools for your OS to monitor CPU/Memory/IO.
One of our standard techniques is running stepped-ramp load tests to measure scalability.
There are mainly two approaches for performance on an application:
Performance test and System Test
How do they differ? Well it's easy, it's based on their scope, Performance tests' scope is limited and are highly unrealistic. Example: Test the IncomingMessage handler on some App X, for this you would setup a test which sends meesages to this handler on a X,Y,Z basis. This approach will help you pin down problems and measure performance of individual and limited zones on your application.
So this should now take you to the question, so am I to benchmark and performance test each one of the components in my app individually? Yes if you believe the component's behavior is critical and changes on newer versions are likely to induce performance penalties. But, if you want to get a feel on your application as a whole, the bunch of components interacting with each other and see how performance comes out, then you need a system test.
A system test will always, try to replicate as close as possible any customer production environment. Here you can observe what a real world feel of your app's performance is like and act accordingly to correct it.
So as conclusion,setup a system test on your app and measure what you were saying you wanted to measure. Then Stress the system as a whole and see how it reacts, you will be surprised on the outcome.
Finally, Performance test individually any critical components you have identified or would like to keep tracked on your app.
As a general guideline, when doing performance you should always:
1.- Get a baseline for the system on an idle state.
2.- Get a baseline for the system under normal expected load.
3.- Get a baseline for the system under stress conditions.
Keep in mind that Normal load results should be extrapolated to stress conditions, and a nice system will always be that one which scales linearly.
Hope this helps.
P.S. Tests, envirnoment setup and even data collection should be as fully automated as possible, this will help you run this on a basis and spend time diagnosing performance problems and not setting up the test.
As mentioned by others; tools like JMeter (Commercial tools like LoadRunner and more) can help you generate concurrent test load.
Many monitoring tools (some provided within JDK like MissionControl, some other open source/ free tools like java Melody and many commercial one's) can help you do generic monitoring of various system (memory, CPU, network bandwidth) and JVM resources (Heap, CPU, GC overheads etc).
But to really identify bottlenecks within your code as well as other dependencies of your applications (like external services invoked, DB queries/updates etc) in a very quick and easy way; I recommend considering a good APM i.e. Application Performance Monitoring Tools like AppDynamics/ DynaTrace and more. They can help you pinpoint bottlenecks for specific request level, highlight slower parts of apps, generate percentile metrics at individual service end point or component / method level etc. They can be immensely useful , if one is dealing with very high concurrent users and stringent response time NFR's. They help uncover many bottlenecks across the layers of your application. Many even configure these tools in production (expected to cause 2-3% overheads; but worth it per me for the benefits they provide) - as production logging is not at debug level by default; so once some errors or slowness is observed; it's often extremely difficult to reproduce in lower environments or debug in absence of debug level logs from specific past duration.
There's no one tool to tackle this as far as I know. So build you own environment
Load Injecting & Scripting: JMeter, SOAP UI, LoadUI
Scheduling Tests & Automation: Jenkins, Rundeck
Analytics on transaction data, resources, application performance logs: AppDynamics, ElasticSearch, Splunk
Profiling: AppDynamics, YouKit, Java Mission Control, VisualVm
How can we increase the performance of an application. My application is written using Java, Hibernate, Servlets, Wsdl i have used for web services. I have executed some of the tests on linux machine, so that i can get proper TPS of the execution.
but still , i am not satisfied by the performance.
So for this, what all steps i should try to increase the performance.
adding to above, i have executed code coverage and used find bugs in the code prominently for each and every test and every service i have written.
Individual suggestions are invited.
Thanks.
Profile your application, and remove all of your bottlenecks.
In addition, or better before, take a day or two and read as much from the Java Performance Tuning newsletters as you understand.
You should monitor your application with a tool like VisualVM, JProfiler etc. to determine the performance bottleneck(s). It is pointless to tune the application without knowing where the actual performance problems are located.
In a professional environment, I suggest dynaTrace that can show you performance bottlenecks along the execution path. The tool can show you exactly where the application spends its time.
Is the performance related to disk I/O or network I/O? In a high throughput system (from DB point of view) Hibernate might not be the best way to go. If you have a lot of writes I would recommend you use a different mechanism to write to database -- perhaps simply switching to simple JDBC might speed it up?
Secondly, is it the case that your webservices are taking too long to get back with results? SOAP is not the fastest protocols really -- have you looked at something like REST maybe coupled with JSON ?
I have found the DTrace intriguing but have personally failed to see a use-case that allow me to get information that I cannot get otherwise anyway without using DTrace.
Hence, I would like to hear what I have overlooked. What can I do on my vmware OpenIndiana build 148 with DTrace that can make a difference when creating stand-alone applications and Java EE web applications (most of which communicate heavily with a legacy backend using sockets)?
Non-trivial Dtrace scripts are very welcome.
I had some good experiences with DTrace.
I had a client running our Java code on a production server running 24x7. We had some performance issues with the applications. It was impossible to stop the JVMs in order to attach the profiler. Moreover the behavior was not present in our lab under the same load.
We solved the issue using DTrace with the JVM related probes as I could attach it to the running JVMs and the overhead introduced was minimal (1.3% on a Netra T2000 SPARC machine).
The bonus of the method was that all the debugging was done via a dial-up (33kbps) line to the customer's lab. It is almost impossible to run any other profiler/debugger with this constraints (JDWP is quite verbose for this bandwidth). With my DTrace script I filtered only what it was interesting for me.
For some hints see:
http://java.dzone.com/articles/java-profiling-dtrace?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+javalobby/frontpage+(Javalobby+/+Java+Zone)
http://download.oracle.com/javase/6/docs/technotes/guides/vm/dtrace.html
I have adapted those scripts
http://blogs.oracle.com/ahl/date/20050418#dtracing_java
Another big plus are the aggregated values that can be easily used to create custom statistics.
But, if you have appropriate conditions, everything can be done easier with othe tools. DTrace is very powerfull but it targets more bare-metal conditions (maybe DLight project would help).