I have a few executable tools. In my Java application I need to launch each of them for a few hundred times and measure their memory consumption based on different inputs. I am using
Runtime.getRuntime().exec(externalToolCommand);
to execute external tools. But I don't know how to measure the max memory usage of the external tools.
To make more clear I will exemplify it;
Let say I have prism.exe, mrmc.exe, and plasma.exe which are three executable external tools. I have want to know when I launch one of the tools e.g. prism.exe, how much memory it consumes. I don't need to measure my Java application memory consumption. I need only to know the external memory consumption.
Thanks.
I don't know the exact code but here is what I could think of. On Windows, you can launch a batch script say p.bat from your Java application and launch a powershell script from p.bat say q.ps1 and now you have got access to a powershell script. You can run a process monitor tool(I don't know maybe perfmon...) to measure one time memory consumption of it, log it in a text file. Terminate the process from the script. Do the whole process in loop in your Java application. Finally you have got the file containing the process memory consumption n number of times.
But Beware! it is really expensive as it involves File I/O, context switch back and forth, pipelining on the top.
With powershell, the possibilities are endless. But I am no powershell expert so pardon, I can't write the exact code for you. This answer requires a lot of research on your side on various steps involved.
Try "jvisualvm", you can find it at /bin/jvisualvm.exe
Related
This might be a long shot of a question, but I have ran into a very complicated issue and I am unsure on how to solve it.
Long story short, we have a Java application running, it's currently using JDBC to pull in data from a MysQL Database on startup.
We have had a meltdown and that database is no longer active and has been lost forever and so has the data to go along with it which internally is very valuable.
However the data is still stored in the heap of the running JVM that pulled it in.
My only hope now is to somehow extract the data from the running JVM, in an ideal world i would be able to attach to it and have the flexibility to run code which could access the internal running classes..
So my questions today are:
Is my approach reasonable and possible?
If so how can I attach to the JVM and 'Inject' code
Thank you for reading
It seems that what you want to use is the jmap command. jmap can be used to dump the heap of a running JVM into a file, which you can then analyze "off-line", using tools such as jhat or JVisualVM.
It allows you to do so without killing the JVM and/or injecting code into it, and since the heap dump file is "inert", you can analyze it at your leisure without fear of harming the running VM by probing it further. Admittedly, I haven't used it extensively, so I'm not sure exactly what its capabilities are, but theoretically, you could perhaps also use JVisualVM's OQL language to run automated sequences on data in the heap and dump it to files in a format you want.
See, for instance, this question for usage examples.
In a situation like this the Eclipse Memory Analyzer Tool can be a good solution. It works on heap dumps too and shows you which objects take up memory.
In addition to this it can show the content of objects / memory locations.
I sometimes found MAT goes beyond what VisualVM does, but perhaps a view like this helps you find your data already:
(This is a screenshot of a made-up example where I create some custom objects in
order to show them with their value in the heapdump)
Perhaps you can even attach Eclipse to the running application. There is a certain trick where you can run custom code in a breakpoint. This one could somehow dump your data to disk.
Everyone knows that a heap dump can be obtained from a running JVM. Is the other way possible? Can we start a JVM using a heap dump?
I have been having this question in mind for a long time now. If this is possible it would solve a lot of time and make thinks easy for a support engineer. It helps big time in cases where if we have to recreate some the rare problems our customer face. [Just imagine that the underlying hardware and Java runtime are the same and also all the supporting files are also present in the respective location in file system].
Added note: The intention of doing this is not when OOM occurs but at any given point after JVM starts.
No you can't.
You would need things like the current position in each open file. That affects what data is returned on a simple sequential read. The restorer would need to open each file and get it to the correct position. That may not be possible for non-seekable streams.
Program specific serialization is a much more feasible path, then setup the program from there.
Also, as Heap Dumps are usually from OutOfMemory situtaions, recreating JVM from same would again throw OutOfMemoryException. If you are taking heap dump in between, then serialize your objects and restore them when you bring up jvm.
(contents copied from comments of this question, Authors almas-shaikh and patricia-shanahan)
To create a heap dump from a running JVM you can also use jhat, or jcmd (with the GC.heap_dump command), both exist in the JDK/bin folder. MAT is one way of analyzing the contents of the dump. Java Mission Control has a tool called JOverflow that analyzes heap dumps, but only to look at memory waste patterns.
I have never heard of any way to restart a JVM from sort of image, a heap dump would not be enough at all, since it only contains the Java objects, and not the compiled code and other things.
I think you are looking for tools like Java Mission Control and Chronon DVR (commercial). These help you incident analysis, event collection and profiling, time travel debugging (as phrased by chronon)
As per their documentation:
Java Mission Control
Java Flight Recorder and Java Mission Control together create a
complete tool chain to continuously collect low level and detailed
runtime information enabling after-the-fact incident analysis. Java
Flight Recorder is a profiling and event collection framework built
into the Oracle JDK. It allows Java administrators and developers to
gather detailed low level information about how the Java Virtual
Machine (JVM) and the Java application are behaving. Java Mission
Control is an advanced set of tools that enables efficient and
detailed analysis of the extensive of data collected by Java Flight
Recorder. The tool chain enables developers and administrators to
collect and analyze data from Java applications running locally or
deployed in production environments.Starting with the release of
Oracle JDK 7 Update 40 (7u40)
Some key features of Chronon Recording Server which will be useful in your case:
The Recording Server is specifically designed for long running, server
side applications that run for weeks or months at a time. The
Recording Server will take care of splitting the recording if it get
too large and flushing out old recordings.
Get rid of the need to look at long sparsely detailed log files to
debug your program. Just play back the entire execution and see
exactly what took place in your program.
The Recording Server makes it share recordings on different machines
among team members or across multiple teams.
I have a Java program running in JVM. I need to find out at any moment what are the variables stored in the memory. I need to list their names and values. Can use any language or scripting to write this program.
I have used JMAP and JHAT to get a heap dump and see the dump values. But it does not give what I want.
Any help is appreciated. Thanks!
YourKit is one of the best profilers in town, but is pretty expensive. And, while they might offer you a free trial period, that doesn't do you any good the next time you need to do similar testing on your next project.
I'd recommend using what the good 'ole JRE ships with: JVisualVM. It has all the features you're looking for (inspecting variables/objects dynamically at runtime), as well as the ability to monitor heap/thread activity, perform heap/thread dumps and analyze them, and even has its own object query language (OQL) for query object values out of a dump based on certain criteria.
The downside is that it introduces a moderate performance overhead, so you would only want to run this in a non-production environment. (For low-overhead production monitoring using something like AppDynamics or New Relic). Will JVisualVM's performance overhead affect how fast you code runs? Yes. But since you'll be running it for each and every test case, all test cases will be affected equally.
So, it sounds like you want this as your basic workflow:
Start your app/JVM
Start JVisualVM
Start profiling ("Snapshot Mode") your app with JVisualVM
Run your next test.
Stop profiling your app
Do a heap and/or thread dump and save it off somewhere on your file system
If more tests, go back to #3.
Else stop your app
In JVisualVM, load each heap/thread dump one by one, possibly using OQL to fetch the objects/variables you are looking for, and comparing each test case's performance to the others
Good luck & HTH.
I have a Java process which has been running for over a week. In the process, I have been processing some data and have been storing some intermediate result into a hashmap in memory.
Now, I need to stop the process due to some bugs in the code. But if i kill the process, then i lose the data in the hashmap n will have to reprocess it again, the next time i run the code.
Is their a way by which i can take a dump of the hashmap present in the memory ?
You can trigger a memory dump which you can read with various tools such as profilers. This is very difficult to read and I have never heard of some one using this to restart a program.
Restarting a program is not something you can do as an afterthought, it needs to designed into your application and thoroughly tested. One way people get around having to worry about this issue is to use a database. This is because databases are, designed and tested professionally, to be restarted without losing data.
You can use JConsole, which is included with the Standard JDK, or visualvm to attach to an already-running process and trigger a heap dump.
However, as Peter said, actually reading this dump is not going to be fun unless you can find a tool to assist you.
I have an application running on Websphere Application Server 6.0 and it crashes nearly every day because of Out-Of-Memory. From verbose GC is certain there are the memory leaks(many of them)
Unfortunately the application is provided by external vendor and getting things fixed is slow & painful process. As part of the process I need to gather the logs and heapdumps each time the OOM occurs.
Now I'm looking for some way how to automate it. Fundamental problem is how to detect OOM condition. One way would be to create shell script which will periodically search for new heapdumps. This approach seems me a kinda dirty. Another approach might be to leverage the JMX somehow. But I have little or no experience in this area and don't have much idea how to do it.
Or is in WAS some kind of trigger/hooks for this? Thank you very much for every advice!
You can pass the following arguments to the JVM on startup and a heap dump will be automatically generated on an OutOfMemoryError. The second argument lets you specify the path for the heap dump file. By using this at least you could check for the existence of a specific file to see if a heap dump has occurred.
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=<value>
I see two options if you want heap dumping automated but #Mark's solution with heap dump on OOM isn't satisfactory.
You can use the MemoryMXBean to detect high memory pressure, and then programmatically create a heap dump if the usage (or usage delta) seems high.
You can periodically get memory usage info and generate heap dumps with a cron'd shell script using jmap (works both locally and remote).
It would be nice if you could have a callback on OOM, but, uhm, that callback probably would just crash with an OOM error. :)
Have you looked at JConsole ? It uses JMX to give you visibility of a variety of JVM metrics, including memory info. It would probably be worth monitoring your application using this to begin with, to get a feel for how/when the memory is consumed. You may find the memory is consumed uniformly over the day, or when using certain features.
Take a look at the detecting low memory section of the above link.
If you need you can then write a JMX client to watch the application automatically and trigger whatever actions required. JConsole will indicate which JMX methods you need to poll.
And alternative to waiting until the application has crashed may be to script a controlled restart like every night if you're optimistic that it can survive for twelve hours..
Maybe even websphere can do that for you !?
You could add a listener (Session scoped or Application scope attribute listener) class that would be called each time a new object is added in session/app scope.
In this - you can attempt to check the total memory used by app (Log it) as as call run gc (note that invoking it will not imply gc will always run)
(The above is for the logging part and gc based on usage growth)
For scheduled gc:
In addition you can keep a timer task class that runs after every few hrs and does a request for gc.
Our experience with ITCAM has been less than stellar from the monitoring perspective. We dumped it in favor of CA Wily Introscope.
Have you had a look on the jvisualvm tool in the latest Java 6 JDK's?
It is great for inspecting running code.
I'd dispute that the you need the heap dumps when the OOM occurs. Periodic gathering of the information over time should give the picture of what's going on.
As has been observed various tools exist for analysing these problems. I have had success with ITCAM for WebSphere, as an IBMer I have ready access to that. We were very quickly able to indentify the exact lines of code in out problem situation.
If there's any way you can get a tool of that nature then that's the way to go.
It should be possible to write a simple program to get the process list from the kernel and scan it to see if your WAS process is still running. On a Unix box you could probably whip up something in Perl in a few minutes (if you know Perl), not sure how difficult it would be under Windows. Run it as a scheduled task every five minutes or so, and if the process doesn't show up you could have it fork off another process that would deal with the heap dump and re-start WAS.