Develop Reference

Memory leaks in application

I seems to have memory leak problems in my javaFX application, i have been using the net beans profiler tool to help me locate the source of the leaks but i have hit a wall as I'm not sure exactly what I'm doing.
now i have looked at tutorials,and they say to monitor the surviving generations of a object but I'm still unsure what to with this information. I have taken a screenshot of the highest surviving generation objects while i was monitoring my application.
Can someone explain to me what exactly i should be doing with these objects or are they actually causing a memory leak?

A common source of unexpected high memory usage is adding objects to a collection and then not removing them when you are done (I see the image you posted shows a HashMap that may be an issue). The typical solution is to use a collection which will not keep an object alive once all other references to it are gone, see WeakHashMap for more info.
To dig into these types of issues further capture a hrpof (you can use VisualVM, JConsole, etc.) and then analyze it in Eclipse MAT or a similar tool.

How can I see what exactly is stored in PermGen?

In my application I have a PermGen out of memory error and I want to see what led to it.
I have connected to my application via VisualVM.
I want to see what exactly consumes so much PermGen memory in my application but I cannot take a special "PermGen Heap" in VisualVM. I can only take a standard HeapDump.
Is it possible to find out what exactly consumes all my PermGen memory using VIsualVM or some other tool?
UPDATE
This question was marked as "duplicate" but in fact I dont see an answer to my question in the topic mentioned. The problem is that I dont see a way to list exactly what consumes memory in my permgen space.
Currently from that topic I know the following hints:
Run jmap -permstat and see how much of PermGen is used by intern Strings. Good idea, it works: now I know that String Pool is not a problem in my case.
Run jmap -permstat and see how the memory is consumed by the classloaders. It works, but there are hundreds of classes in AppClassLoader, this doesn't help me a lot.
Run PrintStringTable.java util to see which Strings occupy memory. I cannot run it in my Windows 7 PC because I get
Attaching to process ID XXXX, please wait... Error attaching to
process: Timed out while attempting to connect to debug server (please
start SwDbgSrv.exe).
Whatever, its not my case.
Trace the classloading in the runtime by specifying JVM properties. It works, but does not help me to understand what takes the memory away, there thousands of classes loaded.
So I cannot see an answer to my question here.
I know that PermGen is consumed not only by classes themselfs, but also by their static members. How can I track which static members of which classes consume the memory?
Please let me know if I have missed this answer.

VisualVM's heap dump should be enough to investigate your permgen problem - I have had to solve far too many PermGen memory leaks and VisualVM is always my starting point.
If you perform the heap dump it will include the info about permgen also - classes, classloader etc - Personally, I then use Eclipse Memory Analyser tool - just load the entire dump in and it can provide leak suspect reports, duplicate class reports etc. Generally, I just go to the duplicate classes list, right-click and select "Find GC Roots" and from there you can normally narrow in on some problems.
this might help (although not read in detail):
http://java.jiderhamn.se/2011/12/11/classloader-leaks-i-how-to-find-classloader-leaks-with-eclipse-memory-analyser-mat/
This seems to be the same question/answer: How to analyze PermGen space?
As an aside, are you doing anything particular with classloading etc? Explicitly reloading or loading with classloaders? Is it just straight Java? Is it in a container etc?

java: General approach to out-of-memory error

I have been working on this project on Java with multiple modules. Since quite some time, I have been occasionally getting "java: Out Of Memory" error! I am pretty new to this 'popular' error and wanted to know the general approach to solve such errors.
Also, are there standard tools accepted by the industry to help figure out the cause of such errors?
The modules in my project include every minute polling from a third party (using web service), multi-threading among other things. However, this is just a pointer and I seek a general approach and not something very specific to my project.
Thanks.

Sometimes you just have an class that uses a lot of memory and you need to increase the heap size or make a more space-efficient algorithm. Other times it is a leak and you need to deference objects.
Run jvisualvm (it's included in the JDK).
Connect to your process and try if you can to recreate the
out-of-memory error while keeping an eye on the heap size.
Perform a heap dump when the memory grows large. Search for the
largest objects by size - often that will give you the culprit
class.
Look at the dependencies to see what is holding a references. If it is a memory leak make sure to dereference unneeded objects.

Also, are there standard tools accepted by the industry to help figure out the cause of such errors?
Yes, there are memory profilers such as VisualVM and YourKit. I use the latter extensively, for both CPU and memory profiling, and find it extremely useful. To get some idea of what it's capable of, take a look at this page: link.

If you can't increase the available memory you have to consume less.
Don't keep references to Objects that you don't need at the time of execution (like data you can reload dynamically) and if necessary redesign your flow (e.g. don't process all objects in parallel and do it sequentially) to require less memory at that time. The garbage collection should do the rest for you.
Especially if you load big data objects into memory consider to use a streaming approach if possible. E.g. you don't need to load a whole file into memory if you want to search through it. You can just step through it.
Besides architectural problems you can also have leaks: keeping unintentional references to objects you don't need anymore. Since they are referenced, the garbage collector can't free the memory and you run out of memory at some point. That is probably the #1 reason for OutOfMemoryExceptions and it usually has to do with static references since classes and therefore the statics are usually not unloaded after the first time you touch a class. The internet has many articles on finding / fixing those, e.g. How to Fix Memory Leaks in Java
one tool I know of is MAT

You likely have a memory leak. Finding it is a challenge. Netbeans has some tools to help you profile the VM . You can profile your project and view men usage while it runs. Apache JMeter is also available as a plug-in or you can run it on its own.
JMeter.apache.org

If you get OOM too often, then start java with correct options, get a heap dump and analyze it with jhat or with memory analyzer from eclipse (http://www.eclipse.org/mat/)
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=<path to dump file>

What free JVM implementation has the best PermGen handling?

I'm running Tomcat6 in Sun's JRE6 and every couple deploys I get OutOfMemoryException: PermGen. I've done the Googling of PermGen solutions and tried many fixes. None work. I read a lot of good things about Oracle's JRockit and how its PermGen allocation can be gigs in size (compare to Sun's 128M) and while it doesn't solve the problem, it would allow me to redeploy 100 times between PermGen exceptions compared to 2 times now.
The problem with JRockit is to use it in production you need to buy WebLogic which costs thousands of dollars. What other (free) options exist that are more forgiving of PermGen expansion? How do the below JVMs do in this area?
IBM JVM
Open JDK
Blackdown
Kaffe
...others?
Update: Some people have asked why I thought PermGen max was 128M. The reason is because any time I try to raise it above 128M my JVM fails to initialize:
[2009-06-18 01:39:44] [info] Error occurred during initialization of VM
[2009-06-18 01:39:44] [info] Could not reserve enough space for object heap
[2009-06-18 01:39:44] [395 javajni.c] [error] CreateJavaVM Failed
It's strange that it fails trying to reserve space for the object heap, though I'm not sure it's "the" heap instead of "a" heap.
I boot the JVM with 1024MB initial and 1536MB max heap.
I will close this question since it has been answered, ie. "switching is useless" and ask instead Why does my Sun JVM fail with larger PermGen settings?

I agree with Michael Borgwardt in that you can increase the PermGen size, I disagree that it's primarily due to memory leaks. PermGen space gets eaten up aggressively by applications which implement heavy use of Reflection. So basically if you have a Spring/Hibernate application running in Tomcat, be prepared to bump that PermGen space up a lot.

What gave you the idea that Sun's JVM is restricted to 128M PermGen? You can set it freely with the -XX:MaxPermSize command line option; the default is 64M.
However, the real cause of your problem is probably a memory leak in your application that prevents the classes from getting garbage collected; these can be very subtle, especially when ClassLoaders are involved, since all it takes is a single reference to any of the classes, anywhere. This article describes the problem in detail, and this one suggests ways to fix it.

Technically, the "PermGen" memory pool is a Sun JVM thing. Other JVMs don't call it that, but they all have the idea of one or more non-heap memory pools.
But if you have a problem with permgen in your Sun JVM, moving to another JVM is very unlikely solve anything, it'll just manifest itself under a different name.
If multiple redeployments are causing your problems, just boost the VM's PermGen up to large values. We tried JRockit a while back because of this very problem, and it suffers from the same redeployment exhaustion. We moved back to SUn JVM.

Changing JVM is not a panacea. You can get new unexpected issues (e.g. see an article about launching an application under 4 different JVM).
You can have a class leak (e.g. via classloaders) that mostly often happen on redeploy. Frankly, I've never saw working hot redeploy on Tomcat (hope to see one day).
You can have incorrect JVM paramaters (e.g. for Sun JDK 6 64 bits -XX:+UseParNewGC switch leads to leak PermGen segment of memory. If you add additional switches: -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled-XX:+CMSPermGenSweepingEnabled the situation will be resolved. Funny, but I never met above mentioned leak with Sun JDK 6 32 bits). Link to an article "Tuning JVM Garbage Collection for Production Deployments".
Your PermGen chunk can be not enough to load classes and related information (actually that most often happens after redeploy under Tomcat, old classes stay in memory and new ones are loading)
From my past experience, debugging that kind of leak is one of the most tricky kind of debugging that I've ever had.
[UPDATED]
Useful article how to eliminate classloader link on an application redeploy.

I use JRockit and I still get PermGen errors if I don't bump up (via -XX:MaxPermSize) the memory. I also can't get anything to work to avoid getting this (other than increasing it).

Perm gen is probably the simplest memory to handle, I doubt there'd be much difference between the various vm implementations.
Make sure all those Tomcat configs that are marked turn off in production are turned off in production.
Yes, some frameworks that do generate a lot of classes an the fly, but they should be cleaning up after themselves, and, in any case, you can fit more than a few classes in 128Mb.
Seriously, if perm gen keeps going up then thats a leak a should be fixed, though it may not be your problem to fix.

The IBM JVM does not (and did not in 2009) have a permgen. You can read more about its Generational Concurrent Garbage Collector which is its default GC for Java 7.
I have sometimes run the Eclipse IDE on IBM JVM specifically because with my favorite plugins it would frequently fill up the HotSpot JVM's permgen. Sure, there was probably a memory leak that someone should have fixed, but meanwhile my IDE was not crashing and I was not busy experimenting with different settings.

How do I discover what is in the permanent generation

Given a heapdump or a running VM, how do I discover what the contents of the permanent generation is ? I know about 'jmap -permstat' but that's not available on Windows.

This article Memory Monitoring with Java SE 5 describes how to programmatically discover information on heap usage, memory pools (including permgen space) and so on. Its very simple:
MemoryUsage usage = ManagementFactory.getMemoryMXBean().getNonHeapMemoryUsage();
long nonHeapFree = usage.getMax() - usage.getUsed();
long nonHeapTotal = usage.getMax();
In my testing on OSX (Sun VM) "non heap memory usage" matches the values returned for the permgen pool closely and presumably will do something useful on VMs that don't have permgen.

The permanent generation contains the class object.
So you should check the heap dump or other form of object list for classes.
If you have problem with the size of permanent generation usually it is caused by two reason:
your program or a library you use creates classes dynamically and the default size of permanent generation is too small - simple increate the size with -XX:MaxPermSize=256m
your program or a library you use creates new classes dynamically every time it is called, so the size of permanent generation increases non-stop - this is a programming error you should fix it (or search a fix/create a bug report)
To see which is your case check the size of the permanent generation over a larger period.
And a good overview about permanent generation:
http://blogs.oracle.com/jonthecollector/entry/presenting_the_permanent_generation

See my blog post about thr permsize of Eclipse
In short the Memory Analyzer can doit, but you need the SAP JVM.

Do you have a specific problem to solve? The use of String.intern() is one of the typical causes for permgen problems. Additionally projects with a lot of classes also have permgen problems.
I do not know how to get into the permgen and see what it is there...

Permanent generation really only contains two kinds of things: Class definitions and interned strings. Latter very rarely gives you problems, but it is often blamed for problems. More often former is the one giving problems, due to code generation and partial hot reloading (dangling references).
Unlike name suggests, permgen does eventually get GC'ed too, just not part of regular GC cycle. Hence unreferenced interned Strings and unused classes do get cleaned up.
But permgen also does not grow dynamically which means that it is sometimes necessary to manually resize its settings for JVM start.

I'm looking into the same thing but due to memory constraints of an embedded platform.
Look at the code for jmap, the permstat tool is only available if the sun.jvm.hotspot.tools.HeapSummary class is available. This class is part of the serviceability agent. According to OpenJDK documentation (http://openjdk.java.net/groups/serviceability/svcjdk.html#bsa):
Serviceability Agent components are built as part of the standard build of the HotSpot repository. These components are:
-libsaproc.so: this is the native code component of SA.
-sa-jdi.jar: This is contains the Java classes of SA. It includes an implementation of JDI which allows JDI clients to do read-only debugging on core files and hung processes.
SA is used by jinfo, jmap, jstack
NOTE: The Serviceability Agent and the technologies that use it are not currently included in JDK releases on the Windows platforms.
Looks to be the case for Oracle JDK as well. I'm looking to modify the jmap tool to get more info.

One technique that helped me was to use the -verbose:class command-line option to java, and you'll get log output telling you as classes are loaded/unloaded. Since classes are loaded to the permgen, this can help in certain circumstances.

you can use JConsole or jvisualvm.exe(with jdk 1.6 7) to find what is where. If you want to know how all of your objects are related to each other and tree of objects, then you might want to try Eclipse Memory Analyzer -- http://www.eclipse.org/mat/.
IN summary, you will get want you want from "http://www.eclipse.org/mat/".
Good luck,