Yesterday I read something abouth application optimization and how a programmer should find the most used parts of the program and by profiling and modifying them getting the most benefit (when looking at the time/work invested vs. memory/speed gains). Now, I've run the Eclipse profiler, got VisualVM but I don't how to use this data properly. My primary concerns are memory usage (i'm generating an XML and either storing it to disk as a zip or flushing it as zip to the user for download) and slowdowns from the database (i'm suspecting my indexes aren't there or aren't good, and in any case, don't know much about them so I can't tell you more :) but I don't even know how to start this. For the first case VisalVM shows that the program uses up to 200MB, but when I inspect the Heap Dump and click the most used object (or how it's called), the information is overwhelming. For the second case I know even less, other than that Toad has some tools.
What I want to know is how to start doing this, and when I'm satisfied with the local performance, how to do it on the production application.
Edit1: So, for a concrete example of memory usage (i'm generating an XML and either storing it to disk as a zip or flushing it as zip to the user for download). This is what I get when I choose "Heap dump", then choose top 20 objects by retained size and open the details:
and this is what I get when I opened Profiler on the same use case:
The question is, what do this screens tell me? :)
As far a database applications go, I would start from reading Cary Millsap's excellent articles:
http://method-r.com/downloads/cat_view/38-papers-and-articles
Search for "Making friends with the Oracle Database" for example...
Related
This might be a long shot of a question, but I have ran into a very complicated issue and I am unsure on how to solve it.
Long story short, we have a Java application running, it's currently using JDBC to pull in data from a MysQL Database on startup.
We have had a meltdown and that database is no longer active and has been lost forever and so has the data to go along with it which internally is very valuable.
However the data is still stored in the heap of the running JVM that pulled it in.
My only hope now is to somehow extract the data from the running JVM, in an ideal world i would be able to attach to it and have the flexibility to run code which could access the internal running classes..
So my questions today are:
Is my approach reasonable and possible?
If so how can I attach to the JVM and 'Inject' code
Thank you for reading
It seems that what you want to use is the jmap command. jmap can be used to dump the heap of a running JVM into a file, which you can then analyze "off-line", using tools such as jhat or JVisualVM.
It allows you to do so without killing the JVM and/or injecting code into it, and since the heap dump file is "inert", you can analyze it at your leisure without fear of harming the running VM by probing it further. Admittedly, I haven't used it extensively, so I'm not sure exactly what its capabilities are, but theoretically, you could perhaps also use JVisualVM's OQL language to run automated sequences on data in the heap and dump it to files in a format you want.
See, for instance, this question for usage examples.
In a situation like this the Eclipse Memory Analyzer Tool can be a good solution. It works on heap dumps too and shows you which objects take up memory.
In addition to this it can show the content of objects / memory locations.
I sometimes found MAT goes beyond what VisualVM does, but perhaps a view like this helps you find your data already:
(This is a screenshot of a made-up example where I create some custom objects in
order to show them with their value in the heapdump)
Perhaps you can even attach Eclipse to the running application. There is a certain trick where you can run custom code in a breakpoint. This one could somehow dump your data to disk.
I'm testing data structure performance with very large data.
As a temporary workaround (see here) I want to write memory to disk.
I want to test with very big datasets - how can I make it so that when the java VM runs out of memory it writes some of it to disk?
Since we're talking about temporary fixes here you could always increase your page file if you need a little extra space (swap file in most linux distros)
Here's a link from Microsoft:
http://windows.microsoft.com/en-us/windows-vista/change-the-size-of-virtual-memory
Linux:
http://www.cyberciti.biz/faq/linux-add-a-swap-file-howto/
Now let me say that this isn't a good long term fix, but I understand that sometimes developers just need to make it work. If this is something that will ever see a production environment you may want to look at a tool like Hadoop. It allows you to distribute your data processing over multiple JVM's--a tool built for a "big data" application like the one you're describing
Maybe you can use stream, or some buffered one. I think that will be the best choice for testing such structure. If you can read from disk using stream and that will be not make any additional objects(only that which are necessary) so you can have all jvm memory for your structure. But maybe you can describe your problem more?
I have a Java process which has been running for over a week. In the process, I have been processing some data and have been storing some intermediate result into a hashmap in memory.
Now, I need to stop the process due to some bugs in the code. But if i kill the process, then i lose the data in the hashmap n will have to reprocess it again, the next time i run the code.
Is their a way by which i can take a dump of the hashmap present in the memory ?
You can trigger a memory dump which you can read with various tools such as profilers. This is very difficult to read and I have never heard of some one using this to restart a program.
Restarting a program is not something you can do as an afterthought, it needs to designed into your application and thoroughly tested. One way people get around having to worry about this issue is to use a database. This is because databases are, designed and tested professionally, to be restarted without losing data.
You can use JConsole, which is included with the Standard JDK, or visualvm to attach to an already-running process and trigger a heap dump.
However, as Peter said, actually reading this dump is not going to be fun unless you can find a tool to assist you.
I am currently trying to determine the cause of high memory usage in a Java application running on an exotic platform where I know of no instrumented JVM.
I have the source to the application, and can make changes to the source for the purposes of testing.
How can I debug memory usage under these conditions?
If more info is needed, I'll be happy to provide. I'm just a little lost trying to use such an old jvm without much tooling to speak of.
If I were in your shoes I would approach it with:
Find the functional areas you know
need attention.
Make backup copy of code
Start inserting print statements
with start and end times
See what takes a lot of time and
narrow it down.
For Java 5 and later this can be done using Java agents. For earlier versions - including 1.1.8 - you must load native agents to do this. If you cannot instrument your code, you must do the work needed yourself.
One approach to get most of the way is to use a Java 1.1 compatible version of log4j which allows you to essentially write out strings prepended with a timestamp. This can then be massaged afterwards into extracting answers to whatever you want to know.
If you need memory profiling - and I'd recommend against this - you could start serializing objects out to disk, then measuring disk size as a rough estimate of memory size.
If you really want to dig into where you're usually not supposed to be, try the sun.misc package, although I don't know how much of that was around in 1.1.x.
I am trying to reproduce java.lang.OutOfMemoryException in Jboss4, which one of our client got, presumably by running the J2EE applications over days/weeks.
I am trying to find a way for the webapp to spitout java.lang.OutOfMemoryException in a matter of minutes (instead of days/weeks).
One thing come into mind is to write a selenium script and has the script bombards the webapps.
One other thing that we can do is to reduce JVM heap size, but we would prefer not to do this, as we want to see the limit of our system.
Any suggestions?
ps: I don't have access to the source code, as we just provide a hosting service (of course I could decompile the class files...)
If you don't have access to the source code of the J2EE app in question, the options that come to mind are:
Reduce the amount of RAM available to the JVM. You've already identified this one and said you don't want to do it.
Create a J2EE app (it could probably just be a JSP) and configure it to run within the same JVM as the target app, and have that app allocate a ridiculous amount of memory. That will reduce the amount of memory available to the target app, hopefully such that it fails in the way you're trying to force.
Try to use some profiling tools to investigate memory leakage. Also good to investigate memory damps that was taken after OOM happens and logs. IMHO: reducing memory is not the rightest way to investigate cose you can get issues not connected with real production one.
Do both, but in a controlled fashion :
Reduce the available memory to the absolute minimum (using -Xms1M -Xmx2M, as an example, but I fear your app won't even load with such limitations)
Do controlled "nuclear irradiation" : do Selenium scripts or each of the known working urls before to attack the presumed guilty one.
Finally, unleash the power that shall not be raised : start VisualVM and any other monitoring software you can think of (DB execution is a usual suspect).
If you are using Sun Java 6, you may want to consider attaching to the application with jvisualvm in the JDK. This will allow you to do in-place profiling without needing to alter anything in your scenario, and may possibly immediately reveal the culprit.
If you don't have the source use decompile it, at least if you think the terms of usage allows this and you live in a free country. You can use:
Java Decompiler or JAD.
In addition to all the others I must say that even if you can reproduce an OutOfMemory error, and find out where it occurred, you probably haven't found out anything worth knowing.
The trouble is that an OOM occurs when an allocation can not take place. The real problem however is not that allocation, but the fact that other allocations, in other parts of the code, have not been de-allocated (de-referenced and garbage collected). The failed allocation here might have nothing to do with the source of the trouble (no pun intended).
This problem is larger in your case as it might take weeks before trouble starts, suggesting either a sparsely used application, or an abnormal code path, or a relatively HUGE amount of memory in relation to what would be necessary if the code was OK.
It might be a good idea to ask around why this amount of memory is configured for JBoss and not something different. If it's recommended by the supplier than maybe they already know about the leak and require this to mitigate the effects of the bug.
For these kind of errors it really pays to have some idea in which code path the problem occurs so you can do targeted tests. And test with a profiler so you can see during run-time which objects (Lists, Maps and such) are growing without shrinking.
That would give you a chance to decompile the correct classes and see what's wrong with them. (Closing or cleaning in a try block and not a finally block perhaps).
In any case, good luck. I think I'd prefer to find a needle in a haystack. When you find the needle you at least know you have found it:)
The root of the problem is most likely a memory leak in the webapp that the client is running. In order to track it down, you need to run the app with a representative workload with memory profiling enabled. Take some snapshots, and then use the profiler to compare the snapshots to see where objects are leaking. While source-code would be ideal, you should be able to at least figure out where the leaking objects are being allocated. Then you need to track down the cause.
However, if your customer won't release binaries so that you can run an identical system to what he is running, you are kind of stuck, and you'll need to get the customer to do the profiling and leak detection himself.
BTW - there is not a lot of point causing the webapp to throw an OutOfMemoryError. It won't tell you why it is happening, and without understanding "why" you cannot do much about it.
EDIT
There is not point "measuring the limits", if the root cause of the memory leak is in the client's code. Assuming that you are providing a servlet hosting service, the best thing to do is to provide the client with instructions on how to debug memory leaks ... and step out of the way. And if they have a support contract that requires you to (in effect) debug their code, they ought to provide you with the source code to do your job.