In our Environment the tomcat server got hanged frequently then we increase the heap and restart the tomcat.
There is any another way to analyze the heap dumps in tomcat??
can we create the heap dumps in tomcat? if possible how?
Thanks
Surya
First, you should analyze exactly what causes your Tomcat to hang. There are many reasons which can cause an application to "hang", e.g. dead locks, long GC pauses, etc.
Looking at the heap dump makes sense if your Tomcat crashes with an OutOfMemoryError.
In that case you can use a tool like MAT to analyze the heap dump.
You can create heap dumps any time with jcmd <pid> GC.heap_dump <file>. You can also set the VM option -XX:+HeapDumpOnOutOfMemoryError. This will dump the heap automatically when you get an OutOfMemoryError.
Related
from time to time jboss serwer throws "An unhandled exception has occurred:
java.lang.OutOfMemoryError: Java heap space".
Could you please explain me how it works?
A few infromation:
Jboss is running all the time in standalone mode on Windows.
From standalone.conf "JAVA_OPTS=-Xms1G -Xmx1G -XX:MaxPermSize=256M"
I deploy ~50MB war file, after tests I remove it.
What is possible cause of this Java heap space exception?
Should I restart server between following deploys?
Is there any command to clean heap space?
If I understand corectly increasing of -Xmx argument will not help. It will only delay the appearance of the exception. Right?
Thanks in advance
What is possible cause of this Java heap space exception?
On the face of it, the explanation is simple. The JVM has run out of heap space, and the GC is unable to reclaim enough space to continue.
But what caused the JVM to get into that state?
There are a number of possible explanations, but they mostly fall into three classes:
Your application has a memory leak.
Repeated deploys are causing memory to leak.
There are no memory leaks, but occasionally your application is getting a request that simply needs too much memory.
Should I restart server between following deploys?
That may help if the deploys are causing the leaks. If not, it won't.
Is there any command to clean heap space?
There is no command that will do this. The JVM will already have run a "full" GC before throwing the OOME.
If I understand corectly increasing of -Xmx argument will not help. It will only delay the appearance of the exception. Right?
That depends. If the root cause is #3 above, then increasing the heap size may solve the problem. But if the root cause is #1 or #2, then tweaking the heap size will (at best) cause the JVM to survive longer between crashes.
My recommendation is to start by treating this as a "normal" (cause #1) memory leak, and use a memory profiler to identify and fix leaks that are likely to build over time.
If / when you can definitively eliminate cause #1, consider the others.
I absolutely agree with the answer from Stephen C, i just want to show you a possible way how you can analyse it.
A minimal tool for monitoring the memory is jstat and comes with JDK.
After starting JBoss you can start monitoring the memory and GC with
jstat -gc <JBOSS_PID> 2s
The output can then be loaded for example in an excel.
Now when you recognize something strange happens with the memory, take a heap dump:
jcmd <JBOSS_PID> GC.heap_dump <filename>
jcmd also comes with JDK.
You can than load the heap dump into MAT and analyse it. It takes some practice and patience to work with MAT. They also have a good Tutorial. You can also compare heap dumps in MAT.
I also suggest you add XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=<path>" to your JAVA_OPTS.
Please increase the MaxPermSize variable and check it will work.
JAVA_OPTS=-Xms1G -Xmx1G -XX:MaxPermSize=512M
I have a java.lang.OutOfMemoryError:GC Overhead limit exceeded.
There is no HeapDumpOnOutOfMemoryError command line option for my application.
I need a heap dump but when I try to capture the dump with jmap or jcmd tools they are not responding:
jmap
D:\program>jmap -dump:live,format=b,file=d:\dump4.hprof 8280
Dumping heap to D:\dump4.hprof ...
jcmd
D:\program>jcmd 8280 GC.heap_dump d:\dump6.hprof
8280:
Processes are not completing but dump files are created. When I open them with VisualVM, they are loading infinitely.
If I capture a heap dump of e.g. VisualVM, Tools complete successfully and dumps are created and opened.
Could you please explain why jmap and jcmd are not completing? And how can I capture a dump of the application with OutOfMemoryError exception? Application is still working but there are only a few live threads.
One possibility is that the heap size you intend to dump is too large in size.
Please specify the size of the heap and RAM.
It is not due to your intended heap size is more than allocated heap size. This error occurs when the JVM spent too much time performing Garbage Collection and was only able to reclaim very little heap space. Your application might have ended up using almost all the RAM and Garbage collector has spent too much time trying to clean it and failed repeatedly.
Your application's performance will be slow comparatively, This is because the CPU is using its entire capacity for Garbage Collection and hence cannot perform any other tasks.
Following questions need to be addressed:
What are the objects in the application that occupy large portions of the heap?
In which parts of the source code are these objects being allocated?
You can also use automated graphical tools such as JConsole which helps to detect performance problems in the code including java.lang.OutOfMemoryErrors.
I am getting "OutOfMemoryError: GC overhead limit exceeded" with my Tomcat server in Linux Machine. Please let me know if there are any tools that helps me to analyze which program in my Java application is consuming lot of memory. Do we have any debugging tools that gives some information to know where tomcat sever failed with this error?
Thanks in advance.
Be sure you set command line paramter: -XX:+HeapDumpOnOutOfMemoryError to make heapdump on OOM. Also can be usefull: -XX:HeapDumpPath=<folder for heap dump>.
When OOM occurred you can analyze heap dump with MAT. It is very useful tool to analyze heapdumps.
Also you can use jmap to make heap dumps manually. Example: jmap -dump:file=<output-filename> <java process id>
My tomcat application crashed due to memory leak.
I want to take the heap dump on the crashed system/jvm.
Is it possible? I am using windows/tomcat 6
How?
The process does not exists anymore. So there is no heap to dump.
Use '-XX:+HeapDumpOnOutOfMemoryError' for the next time.
You can get the heap dump at runtime by:
jmap -dump:live,format=b,file=heap.dump
You can't get a heap dump on a process that is no longer running. Next time you start Tomcat, you're going to have to edit the file in the /bin directory called catalina.sh first so that it contains options to automatically dump the heap if it runs out of memory.
What you need to do is to edit the JAVA_OPTS variable so that contains the JVM options you need. So near the top of the file, after JAVA_OPTS has been created, you need to do something like
JAVA_OPTS="$JAVA_OPTS -XX:+HeapDumpOnOutOfMemoryError"
You can also take heap dumps using JConsole, but in order to do this, you need to know roughly when Tomcat is running out of memory in order for the heap dump to help you diagnose the problem.
If your application is not responding but the JVM is still limping, you may try using JConsole and trigger a Heap Dump. Search for Heap Dump on this link
I am running load against Tomcat 6 running on Java 6. I want to collect a heapdump of the Java heap while the Tomcat server is under load. I normally use jmap -dump to collect my heapdumps.
However, when I try to do this when Tomcat is handling a high load I find that the heapdump collection fails.
Is jmap the best tool for collecting a heap dump from a process under load? What are the possible causes which would cause jmap to fail to collect a heapdump?
If jmap is not the best tool - what is better?
It is entirely acceptable to me for jmap (or some other tool) to stop the world within the Java process while the heap dump is taken.
Is jmap the best tool for collecting a heap dump from a process under load?
I think: No it isn't. From this link:
NOTE - This utility is unsupported and
may or may not be available in future
versions of the JDK.
I've also found jmap can pretty temperamental. If you're having problems:
Try it again. It often manages to get a heap dump after a couple of attempts if it first fails
Use the -F option
Add -XX:+HeapDumpOnOutOfMemoryError as a standard configuration to proactively take heap dumps when an OOM error is thrown
Run Tomcat interactively and add the heap dump on ctrl-break option. This gives you a thread dump too, something you'll probably need anyway
If your heap size is especially large and you have a repeatable condition, temporarily lower your heap size. It makes the resulting file much easier to handle, takes less time and is more likely to succeed
I have found that running Tomcat with a JMX port allows me to take a remote heapdump using visualvm. This succeeded for me when jmap failed.