In my application I have a PermGen out of memory error and I want to see what led to it.
I have connected to my application via VisualVM.
I want to see what exactly consumes so much PermGen memory in my application but I cannot take a special "PermGen Heap" in VisualVM. I can only take a standard HeapDump.
Is it possible to find out what exactly consumes all my PermGen memory using VIsualVM or some other tool?
UPDATE
This question was marked as "duplicate" but in fact I dont see an answer to my question in the topic mentioned. The problem is that I dont see a way to list exactly what consumes memory in my permgen space.
Currently from that topic I know the following hints:
Run jmap -permstat and see how much of PermGen is used by intern Strings. Good idea, it works: now I know that String Pool is not a problem in my case.
Run jmap -permstat and see how the memory is consumed by the classloaders. It works, but there are hundreds of classes in AppClassLoader, this doesn't help me a lot.
Run PrintStringTable.java util to see which Strings occupy memory. I cannot run it in my Windows 7 PC because I get
Attaching to process ID XXXX, please wait... Error attaching to
process: Timed out while attempting to connect to debug server (please
start SwDbgSrv.exe).
Whatever, its not my case.
Trace the classloading in the runtime by specifying JVM properties. It works, but does not help me to understand what takes the memory away, there thousands of classes loaded.
So I cannot see an answer to my question here.
I know that PermGen is consumed not only by classes themselfs, but also by their static members. How can I track which static members of which classes consume the memory?
Please let me know if I have missed this answer.
VisualVM's heap dump should be enough to investigate your permgen problem - I have had to solve far too many PermGen memory leaks and VisualVM is always my starting point.
If you perform the heap dump it will include the info about permgen also - classes, classloader etc - Personally, I then use Eclipse Memory Analyser tool - just load the entire dump in and it can provide leak suspect reports, duplicate class reports etc. Generally, I just go to the duplicate classes list, right-click and select "Find GC Roots" and from there you can normally narrow in on some problems.
this might help (although not read in detail):
http://java.jiderhamn.se/2011/12/11/classloader-leaks-i-how-to-find-classloader-leaks-with-eclipse-memory-analyser-mat/
This seems to be the same question/answer: How to analyze PermGen space?
As an aside, are you doing anything particular with classloading etc? Explicitly reloading or loading with classloaders? Is it just straight Java? Is it in a container etc?
Related
In a java heap dump how do I know exactly where in the code/which thread caused the dump?
For reading the memory dumps:
I would recommend you to try "eclipse memory analyzer" From here
Another option (free) would be opening this with JVisualVM (available at $JAVA_HOME/bin)
jhat is cool too but was already recommended :)
Now, you're asking about the thread that caused memory heap-dump and not about how to proceed with memory dump...
It depends on how did you obtain the memory dump.
There are different ways to obtain the dump.
On your process you can instruct the JVM to produce memory dump once OutOfMemory error is encountered, in this case I believe it will be the JVM itself.
You can trigger the heap dump creation from the MBean given you have a JMX Server running along with your JVM
Example
You can even use system calls (on linux) externally to your application: kill -3 _YOUR_JAVA_PROCESS_ID_ will generation the heapdump.
But I hardly can imagine why would you need such an information. Later in comments you mention 'exact line of code' but these ways are usually external to your JVM... Are you sure you need a line of code that generated heap dump itself, or you're trying to identify the real issue?
Hope this helps
In java you create object some where, use it in many places and the let GC to collect it back. There is no single line causing a leak.
What you should look for in tools like MAP is objects count and heap used by them. Pick each of the target class and see why they are not garbage collected. (some one is holding reference more than needed- say a static field )
You may find instructions from this page more useful - http://scn.sap.com/people/krum.tsvetkov/blog/2007/07/02/finding-memory-leaks-with-sap-memory-analyzer (also linked from MAT homepage)
Try to use Java Heap Analysis Tool(jhat) or jconsole (that is in your JAVA_HOME\bin).
I'm running a GWT+Hibernate app on Glassfish 3.1. After a few hours, I run out of Permgen space. This is without any webapp reloads. I'm running with –XX:MaxPermSize=256m –XmX1024m.
I took the advice from this page, and found that I'm leaking tons of classes- all of my Hibernate models and all of my GWT RequestFactory proxies.
The guide referenced above says to "inspect the chains, locate the accidental reference, and fix the code". Easier said than done.
The classloader always points back to an instance of org.glassfish.web.loader.WebappClassLoader. Digging further, I find lots of references from $Proxy135 and similar-named objects. But I don't know how else to follow through.
new class objects get placed into the PermGen and thus occupy an ever increasing amount of space. Regardless of how large you make the PermGen space, it will inevitably top out after enough deployments. What you need to do is take measures to flush the PermGen so that you can stabilize its size. There are two JVM flags which handle this cleaning:
-XX:+CMSPermGenSweepingEnabled
This setting includes the PermGen in a garbage collection run. By default, the PermGen space is never included in garbage collection (and thus grows without bounds).
-XX:+CMSClassUnloadingEnabled
This setting tells the PermGen garbage collection sweep to take action on class objects. By default, class objects get an exemption, even when the PermGen space is being visited during a garabage collection.
There are some OK tools to help with this, though you'd never know it. The JDK (1.6 u1 and above) ships with jhat and jmap. These tools will help significantly, especially if you use the jhat JavaScript query support.
See:
http://blog.ringerc.id.au/2011/06/java-ee-application-servers-learning.html
http://blogs.oracle.com/fkieviet/entry/classloader_leaks_the_dreaded_java
http://www.mhaller.de/archives/140-Memory-leaks-et-alii.html
http://blogs.oracle.com/sundararajan/entry/jhat_s_javascript_interface
I "solved" this by moving to Tomcat.
(I can't view the link you provided as it's blocked by websense so if I'm restating anything I apologize)
It sounds like you have a class loader leak. These are difficult to track down, add these options to the JVM Options in your instance configuration
-XX:+PrintGCDetails
-XX:+TraceClassUnloading
-XX:+TraceClassLoading
Now when you run your app, you can look at the jvm.log located in your domain/logs folder and see what's loading and unloading. Mostly likely, you'll see the same class(es) loading over and over again.
A good culprit is JAXB, especially if you're creating a new JAXBContext over and over again.
I'm trying to determine whether or not I have a memory leak in my webapp. I'm using VisualVM and JMeter to load test and watch the heap.
I saved a heap dump to file and downloaded Eclipse Memory Analyzer yesterday...after much frustration with VisualVM, I thought Eclipse would pinpoint the leak, if any, better than VisualVM.
I opened the heap file in Eclipse and ran what they call a Leak Suspects Report. I thought it would point to a particular class in my webapp, but it doesn't. So I have no clue how to use the info its provided in order to find out where in any particular class of mine the leak suspect is.
Here's the results of the Leak Suspect Report for one of my heap dump files.
One instance of "org.apache.catalina.session.StandardManager" loaded by "org.apache.catalina.loader.StandardClassLoader # 0x261bdac0" occupies 16,977,376 (48.54%) bytes. The memory is accumulated in one instance of "java.util.concurrent.ConcurrentHashMap$Segment[]" loaded by "".
Keywords
org.apache.catalina.loader.StandardClassLoader # 0x261bdac0
org.apache.catalina.session.StandardManager
java.util.concurrent.ConcurrentHashMap$Segment[]
The rest of the Details in the report are as shown in the attached image. I hope the image can be expanded for a closer look....
I know that Eclipse is supposed to be really good software. This is my last attempt to use something like this to find a memory leak - I just have very, very, limited knowledge in HOW this software can be used for such. Tutorial and help pages describe things as though you should know what to do after a few clicks... I need more help than that.
While I don't have any experience with using Eclipse for finding leaks, I would ask a question first: How sure are you that you have a memory leak? From your question, it doesn't sound like you are sure you have a leak, but you are testing to see if you do have one. The simplest way to test that would be to start your application, note how much memory it is consuming, have JMeter hit it continuously for 24 hours, and see how much memory it is consuming (probably after executing GC). If your application is consuming a significantly large portion of memory, or has died from an OutOfMemoryError, then you have a memory leak.
If you find that you actually do have a memory leak, then I would first suggest running your application through FindBugs to see if it can find the memory leaks through a quick static analysis. If that doesn't work, then this article (although it is rather old) might help you understand the results given to you by Eclipse.
This question already has answers here:
Closed 12 years ago.
I am new to Java and given the task to fix a bug and the issue is as follows. It would be really great if you give suggestions/ideas what is this issue and how can I fix this.:
HTTP Status 500 -
--------------------------------------------------------------------------------
type Exception report
message
description The server encountered an internal error () that prevented it from fulfilling this request.
exception
org.apache.jasper.JasperException
org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:453)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:375)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:314)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264)
javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
org.netbeans.modules.web.monitor.server.MonitorFilter.doFilter(MonitorFilter.java:368)
root cause
javax.servlet.ServletException
org.apache.jasper.runtime.PageContextImpl.doHandlePageException(PageContextImpl.java:858)
org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContextImpl.java:791)
org.apache.jsp.CustMaint.Jsp.ProfProfileDetails_jsp._jspService(ProfProfileDetails_jsp.java:4016)
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:332)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:314)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264)
javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
org.netbeans.modules.web.monitor.server.MonitorFilter.doFilter(MonitorFilter.java:368)
root cause
java.lang.OutOfMemoryError
note The full stack trace of the root cause is available in the Apache Tomcat/5.5.17 logs.
--------------------------------------------------------------------------------
Apache Tomcat/5.5.17
Here's what the Tomcat guys have to say:
An Out Of Memory can be thrown by
several causes:
A servlet trying to load a several
GBytes file into memory will surely
kill the server. These kind of errors
must be considered a simple bug in our
program.
To compensate for the data
your servlet tries to load, you
increase the heap size so that there
is no room to create the stack size
for the threads that need to be
created. The memory required by each
thread will vary by OS but can be as
high as 2M by default and in some OS's
(like Debian Sarge) is not reducible
with the -Xss parameter. 1 Rule of
Thumb, use no more than 1G for heap
space in a 32-bit web application.
Deep recursive algorithms can also
lead to Out Of Memory problems. In
this case, the only fixes are
increasing the thread stack size
(-Xss), or refactoring the algorithms
to reduce the depth, or the local data
size per call.
A webapp that uses lots
of libraries with many dependencies,
or a server maintaining lots of
webapps could exhauste the JVM PermGen
space. This space is where the VM
stores the classes and methods data.
In those cases, the fix is to increase
this size. The Sun VM has the flag
-XX:MaxPermSize that allows to set its size (the default value is 64M)
Hard references to classes can prevent the
garbage collector from reclaiming the
memory allocated for them when a
ClassLoader is discarded. This will
occur on JSP recompilations, and
webapps reloads. If these operations
are common in a webapp having these
kinds of problems, it will be a matter
of time, until the PermGen space gets
full and an Out Of Memory is thrown.
Source: Tomcat Wiki: OutOfMemory
Well... who really caused the out of memory error?
If you ate 8 slices of pizza and you are full, is it the last slice that caused the out of stomach error?
Use Java Heap Analysis Tool (JHAT) with Eclipse MAT http://www.eclipse.org/mat/ to analyse what's going on inside JVM. What is eating how much memory. See the profile and then see the code causing that.
You can also use JConsole, it's dead easy to set it up. And you can see stuffs 'live'. TPTP is also a good option, unfortunately, I find it hard to configure.
This kind of problem is not easy to nail down based on just the stacktrace. It at least boils down to that you've either a memory leak in your application (the code is keeping (unnecessarily) too much objects for an (unnecessarily) long time in memory), or the server simply doesn't have enough memory in order to be able to run your webapp (simply because it is designed that way to require many memory).
To detect and fix memory leaks, use a Java profiler. If you don't have any memory leaks, i.e. the memory usage is stable but the code just really need that much memory, then simply give the server more memory to work with. The profiler is however still useful to spot memory holes in your webapp and optimize code accordingly.
If you're using Eclipse, use TPTP profiler or if you're using Netbeans, use builtin VisualVM profiler. Or when you're using standalone VisualVM, check this blog how to monitor Tomcat with it.
I have an application that is using a lot more than expected short lived objects this is causing significant performance issues which I need to resolve. To give you an idea of the scale of the problem the Eden space jumps from around 200mb to 1800mb in 10 seconds before the garbage collection runs again and brings Eden right back down. I have taken 2 heap dumps one just after GC and one before the next run. I want to analyse these and see what the unreferenced objects are so I can find the source of the issue however, when I load the heap dumps in to either eclipse or netbeans both seems to remove the data associated with the unreferenced objects and just show the 200mb of referenced objects in the application at that time. Does anyone know of a tool/way for me to analyse the heap dump to see what the unreferenced objects are?
Thanks,
James
Last time I debugged such problem I used http://www.yourkit.com/ that helped me very much.
Maybe Unreachable Objects histogram available in Eclipse Memory Analyzer can help you. You may also try enabling -keep_unreachable_objects as stated on the FAQ.
I don't know of a tool to analyze the heap dumps, but jvisualvm that comes with the jdk distribution has a rather nice memory profiler, that I've used to debug similar issues. It should be in your jdk bin-directory.
I've been able to track down a lot of heap related issues with the IBM Heap Analzyer
But I guess you have to be running the IBM SDK to use it, but maybe it worth a shot?