In one of our java application we have got
OutOfMemoryError:GC Overhead limit exceeded.
We have used HashMaps in someplaces for storing some data.From logs we can I identify that its reproducing at the same place.
I wanted to ask if Garbage Collector spends more time in clearing up the hashmaps?
Upon looking at the code( i cant share here ), I have found that that there is a Hashmap created like
Hashmap topo = new HashMap();
but this hashmap is never used.
Is this a kind of memory leak in my application ?
If this Hashmap is created inside a method which is doing some processing and it is not used elsewhere also this method is accessed my multiple threads say 20 .Then in such a case would it impact,creating Hashmap as above, Garbage collector to spend more time in recovering heap and throw OOME.
Please let me know if you need some more details.
n one of our java application we have got OutOfMemoryError:GC Overhead
limit exceeded. We have used HashMaps in someplaces for storing some
data.From logs we can I identify that its reproducing at the same
place.
If the Hashmap is simply ever building and most likely marked as static, which means you keep adding things to this hashmap and never delete. Then one fine day it will lead to OutOfMemoryError.
I wanted to ask if Garbage Collector spends more time in clearing up
the hashmaps?
Garbage collector spends time on the objects which are not referenced, weakly referenced, soft referenced. Wherever it find such objects, depending on the need it will clear them.
Upon looking at the code( i cant share here ), I have found that that there is a Hashmap
created like Hashmap topo = new HashMap(); , but this hashmap is never used. Is this a
kind of memory leak in my application ?
if this Hashmap is created inside a method which is doing some
processing and it is not used elsewhere also this method is accessed
my multiple threads say 20 . Then in such a case would it
impact,creating Hashmap as above, Garbage collector to spend more time
in recovering heap and throw OOME.
If it is hashmap local to a methid, and the method exits after doing some processing, then it should be garbage collected as soon as method exits. As the hashmap is local to the method, so each thread will have a separate copy of this map and once thread finishes the method execution, map is eligible for GC.
You need to look for long-lifetime objects & structures, which might be the actual problem, rather than wildly grasping at some clueless manager's idea of potential problem.
See:
How to find memory leaks using visualvm
How to find a Java Memory Leak
Look out especially for static/ or application-lifetime Maps or Lists, which are added to during the lifetime rather than just at initialization. It will most likely be one, or several, of these that are accumulating.
Note also that inner classes (Listeners, Observers) can capture references to their containing scope & prevent these from being GC'ed indefinitely.
Please let me know if you need some more details.
You need some more details. You need to profile your application to see what objects are consuming the heap space.
Then, if some of the sizeable objects are no longer actually being used by your application, you have a memory leak. Look at the references to these objects to find out why they're still being held in memory when they're no longer useful, and then modify your code to no longer hold these references.
Alternatively, you may find that all of the objects in memory are what you would expect as your working set. Then either you need to increase the heap size, or refactor your application to work with a smaller working set (e.g. streaming events one at a time rather than reading an entire list; storing the last seesion details in the database rather than memory; etc.).
Related
I have a memory leak in Java in which I have 9600 ImapClients in my heap dump and only 7800 MonitoringTasks. This is a problem since every ImapClient should be owned by a MonitoringTask, so those extra 1800 ImapClients are leaked.
One problem is I can't isolate them in the heap dump and see what's keeping them alive. So far I've only been able to pinpoint them by using external evidence to guess at which ImapClients are dangling. I'm learning OQL which I believe can solve this but it's coming slowly, and it'll take a while before I can understand how to perform something recursive like this in a new query language.
Determining a leak exists is difficult, so here is my full situation:
this process was spewing OOMEs a week ago. I thought I fixed it and I'm trying to verify whether my fixed worked without waiting another full week to see if it spews OOMEs again.
This task creates 7000-9000 ImapClients on start then under normal operation connects and disconnects very few of them.
I checked another process running older pre-OOME code, and it showed numbers of 9000/9100 instead of 7800/9600. I do not know why old code will be different from new code but this is evidence of a leak.
The point of this question is so I can determine if there is a leak. There is a business rule that every ImapClient should be a referee of a MonitoringTask. If this query I am asking about comes up empty, there is not a leak. If it comes up with objects, together with this business rule, it is not only evidence of a leak but conclusive proof of one.
Your expectations are incorrect, there is no actual evidence of any leaks occuring
The Garbage Collector's goal is to free space when it is needed and
only then, anything else is a waste of resources. There is absolutely
no benefit in attempting to keep as much free space as possible
available all the time and only down sides.
Just because something is a candidate for garbage collection doesn't
mean it will ever actually be collected, and there is no way to
force garbage collection either.
I don't see any mention of OutOfMemoryError anywhere.
What you are concerned about you can't control, not directly anyway
What you should focus on is what in in your control, which is making sure you don't hold on to references longer than you need to, and that you are not duplicating things unnecessarily. The garbage collection routines in Java are highly optimized, and if you learn how their algorithms work, you can make sure your program behaves in the optimal way for those algorithms to work.
Java Heap Memory isn't like manually managed memory in other languages, those rules don't apply
What are considered memory leaks in other languages aren't the same thing/root cause as in Java with its garbage collection system.
Most likely in Java memory isn't consumed by one single uber-object that is leaking ( dangling reference in other environments ).
Intermediate objects may be held around longer than expected by the garbage collector because of the scope they are in and lots of other things that can vary at run time.
EXAMPLE: the garbage collector may decide that there are candidates, but because it considers that there is plenty of memory still to be had that it might be too expensive time wise to flush them out at that point in time, and it will wait until memory pressure gets higher.
The garbage collector is really good now, but it isn't magic, if you are doing degenerate things, it will cause it to not work optimally. There is lots of documentation on the internet about the garbage collector settings for all the versions of the JVMs.
These un-referenced objects may just have not reached the time that the garbage collector thinks it needs them to for them to be expunged from memory, or there could be references to them held by some other object ( List ) for example that you don't realize still points to that object. This is what is most commonly referred to as a leak in Java, which is a reference leak more specifically.
I don't see any mention of OutOfMemoryError
You probably don't have a problem in your code, the garbage collection system just might not be getting put under enough pressure to kick in and deallocate objects that you think it should be cleaning up. What you think is a problem probably isn't, not unless your program is crashing with OutOfMemoryError. This isn't C, C++, Objective-C, or any other manual memory management language / runtime. You don't get to decide what is in memory or not at the detail level you are expecting you should be able to.
Check your code for finalizers, especially anything relating to IMapclient.
It could be that your MonitoringTasks are being easily collected whereas your IMapclient's are finalized, and therefore stay on the heap (though dead) until the finalizer thread runs.
The obvious answer is to add a WeakHashMap<X, Object> (and Y) to your code -- one tracking all instances of X and another tracking all instances of Y (make them static members of the class and insert every object into the map in the constructor with a null 'value'). Then you can at any time iterate over these maps to find all live instances of X and Y and see which Xs are not referenced by Ys. You might want to trigger a full GC first, to ignore objects that are dead and not yet collected.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Forcing Garbage Collection in Java?
Can I Force Garbage Collection in Java by any means?
System.gc() is just a suggestion.It's useless.
When I know for sure that some resources won't be used any more,why can't I force to clean them?
Just like delete() in C++ and free() in C?
When there are lots of resources that can't be reused,this can really suck the performance.All that we can do is sleep().
Any solutions?Thanks
Nope, System.gc() is as close as you can get. Java isn't C or C++, the JVM manages memory for you, so you don't have that kind of fine grained control. If you set objects you're no longer using to null, or loose all references, they will get cleaned up. And the GC is pretty smart, so it should take good care of you.
That said, if you are on a unix box, and force a thread dump (kill -3), it'll pretty much force garbage collection.
You shouldn't be trying to force GC - if you are running low on memory then you have a memory leak somewhere. Forcing GC at that point won't help, because if you are holding a reference to the object then it still won't be garbage collected.
What you need to do is solve the real problem, and make sure you are not holding references to objects you are not using any more.
Some common culprits:
Holding lots of references in a large object graph that never get cleared up. Either set references to null when you don't need them any more, or better still simplify your object graph so it doesn't need all the extra long-term references.
Caching objects in a hashmap or something similar that grows huge over time. Stop doing this, or use something like Google's CacheBuilder to create a proper soft reference cache.
Using String.intern() excessively on large numbers of different strings over time.
References with larger scope than they need. Are you using an instance variable when it could be a local variable, for example?
There is no way to explicitly instruct the JVM to collect garbage. This is only performed when the system needs the resources.
The only two actions I'm aware of to potentially get the GC running are the following:
As you stated, attempt to "suggest" that GC now would be a good time by called System.gc().
Set any references you are not using to the null reference to make the elements eligible for collection.
On my second point, see the answer here: Garbage collector in java - set an object null. In essence, if you don't make the objects you don't need available for garbage collection (by losing the reference you have to it) then there is no reason for the garbage collector to run, because it's unaware of any available garbage.
In addition, it's important to consider why/how those objects in memory are affecting performance:
Are you getting lots of OutOfMemoryExceptions? This could be resolved by point #2 and by increasing the available heap space for the JVM.
Have you done measurements to see that more objects in the JVM's allocated heap space makes a difference in performance? Determining when you could let references to objects go earlier could help reduce these issues.
I am working on an application which reads in a huge amount of data from a database into a Map<String,Map<String,Map<String,String>>>, processes it, and writes the processed reports to a spreadsheet using an in-house xml writer. The whole run can take about 12 hours.
I'm finding I'm getting
Exception in thread "CursorController-Thread-0" java.lang.OutOfMemoryError: Java heap space
at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:45)
at java.lang.StringBuilder.<init>(StringBuilder.java:68)
When I attempt to write this jumbo file. For this reason I think it would be best to write each Map<String,Map<String,String>> (notice that's a layer deeper)as it's finished processing.
My question is, how can I make sure that the Map<String,Map<String,String>> is not retained in memory after I write it, since the Map>> will still contain it?
Once you're done with the Map<String,Map<String,String>> mapped to by the key "key" you simply do
hugeMap.remove("key");
This will "null" out the entry in the hugeMap and make the Map<String,Map<String,String>> eligible for garbage collection (i.e., never be part of causing a heap space out of memory).
I would choose a different solution for this kind of problem. 12 hours for processing the source data is heavy.
Have you considered any scalable solutions? For e.g. Hadoop?
Use map.remove(key) method on your Map>>. You can call from time to time System.gc(); to force garbage collection.
You can keep the Map> that was written in the outer map if you want to keep your structure. but probably you should clear its contents so that it is empty. also, make sure that when you are processing it and writing it, you don't keep references to its members (mappings) anywhere before clearing the content. please see the following post for picking the approach that suits your needs best Using .clear() or letting the GC take care of it
You can't.
The garbage collector runs whenever it likes and frees whatever it likes.
That said, it is worth trying that after you delete all references to the data you no longer need, call to System.gc().
Anyway, you have written that the out of memory error is while writting the data. Maybe you have a memory leak there.
I have memory leak in the web application (servlet) that I am working on. I am suspicious of 1 cause and wanted to hear your ideas about it.
I use hashmaps,hashsets etc. as DB (about 20MB data loaded). These maps,sets get reloaded once in every 10 min. There are huge amount of simultaneous requests. I read that, GC passes objects, that are not collected for a time period/cycle, to a generation (old and permanent generations) which is less checked or garbage collected. I think that my usage for static maps,sets is causing me leak problem. What do you think ?
As Romain noted, the static map is a suspect. If for some reason you can't regularly clean it up explicitly, you may consider using a WeakHashMap instead, which is
A hashtable-based Map implementation with weak keys. An entry in a WeakHashMap will automatically be removed when its key is no longer in ordinary use. More precisely, the presence of a mapping for a given key will not prevent the key from being discarded by the garbage collector, that is, made finalizable, finalized, and then reclaimed. When a key has been discarded its entry is effectively removed from the map, so this class behaves somewhat differently than other Map implementations.
Unfortunately, as of Java6 there seems to be no WeakHashSet in the standard library, but several implementations can be found on the net.
It is not a leak if you have removed all references to it. If you're clearing out your map entirely, then it's not the source of a leak. You should consider the fact that the JVM chooses not to GC tenured generation very often as irrelevant to you - all that matters is that you don't have a reference to it, so the JVM could GC it if it wants to.
There are different strategies that JVMs can use to manage GC, so I'm speaking in generalities instead of specifics here, but GCing tenured spaces tends to be very expensive and has a high impact on the application, so the JVM chooses not to do it often in general.
If you're looking at the amount of heap space used, you'll see a sawtooth pattern as items are added and eventually collected. Don't be concerned about where the top of the sawtooth is, be concerned about where the bottom is (and how close to the maximum amount of heap space available that is).
One way to test if it really is a leak is to load test your app for a long period of time. If you have a leak, the base amount of memory that your app is using will go up over time (the bottom of the sawtooth). If you don't, it will remain constant. If you do have a leak, you can use a profiler to help you find it.
Static Maps are a known source of leaks. The reason being that people put stuff in and do not remove them. If every ten minutes you simply clear the cache and then reload then you should be fine.
I would bet that you are not clearing it properly. The GC part is working properly, I would not worry that it is the issue.
You may also want to consider using WeakReference if you have some way of falling back to the real data if part of your cache is GC-ed but then subsequently required.
I suggest that you check the heap contents using a heap dump and a heap analyzer (such as JVisualVM). This will help you find leakage suspects. The fact that the old generation is collected less frequently does not mean that more memory is leaking; remember that while it may seem full, only a portion of it represents live objects, and the other portion is clearned by the next major GC. Like others said, the problem may be because of incomplete cleaning of the static collections.
The permanent generation never receives promoted objects. It is an out-of-heap area reserved for other purposes, such as reflective information of loaded classes, and interned strings.
I have this class and I'm testing insertions with different data distributions. I'm doing this in my code:
...
AVLTree tree = new AVLTree();
//insert the data from the first distribution
//get results
...
tree = new AVLTree();
//inser the data from the next distribution
//get results
...
I'm doing this for 3 distributions. Each one should be tested an average of 14 times, and the 2 lowest/highest values removed from to compute the average. This should be done 2000 times, each time for 1000 elements. In other words, it goes 1000, 2000, 3000, ..., 2000000.
The problem is, I can only get as far as 100000. When I tried 200000, I ran out of heap space. I increased the available heap space with -Xmx in the command line to 1024m and it didn't even complete the tests with 200000. I tried 2048m and again, it wouldn't work.
What I'm thinking is that the garbage collector isn't getting rid of the old trees once I do tree = new AVL Tree(). But why? I thought that the elements from the old trees would no longer be accessible and their memory would be cleaned up.
The garbage collector should have no trouble cleaning up your old tree objects, so I can only assume there's some other allocation that you're doing that's not being cleaned up.
Java has a good tool to watch the GC in progress (or not in your case), JVisualVM, which comes with the JDK.
Just run that and it will show you which objects are taking up the heap, and you can both trigger and see the progress of GC's. Then you can target those for pools so they can be re-used by you, saving the GC the work.
Also look into this option, which will probably stop the error you're getting that stops the program, and you program will finish, but it may take a long time because your app will fill up the heap then run very slowly.
-XX:-UseGCOverheadLimit
Which JVM you are using and what JVM parameters you have used to configure GC?
Your explaination shows there is a memory leak in your code. If you have any tool like jprofiler then use it to find out where is the memory leak.
There's no reason those trees shouldn't be collected, although I'd expect that before you ran out of memory you should see long pauses as the system ran a full GC. As it's been noted here that that's not what you're seeing, you could try running with flags like -XX:-PrintGC, -XX:-PrintGCDetails,-XX:-PrintGCTimeStamps to give you some more information on exactly what's going on, along with perhaps some sort of running count of roughly where you are. You could also explicitly tell the garbage collector to use a different garbage-collection algorithm.
However, it still seems unlikely to me. What other code is running? is it possible there's something in the AVLTree class itself that's keeping its instances from being GC'd? What about manually logging the finalize() on that class to insure that (some of them, at least) are collectible (e.g. make a few and manually call System.gc())?
GC params here, a nice ref on garbage collection from sun here that's well worth reading.
The Java garbage collector isn't guaranteed to garbage collect after each object's refcount becomes zero. So if you're writing code that is only creating and deleting a lot of objects, it's possible to expend all of the heap space before the gc has a chance to run. Alternatively, Pax's suggestion that there is a memory leak in your code is also a strong possibility.
If you are only doing benchmarking, then you may want to use the java gc function (in the System class I think) between tests, or even re-run you program for each distribution.
We noticed this in a server product. When making a lot of tiny objects that quickly get thrown away, the garbage collector can't keep up. The problem is more pronounced when the tiny objects have pointers to larger objects (e.g. an object that points to a large char[]). The GC doesn't seem to realize that if it frees up the tiny object, it can then free the larger object. Even when calling System.gc() directly, this was still a huge problem (both in 1.5 and 1.6 VMs)!
What we ended up doing and what I recommend to you is to maintain a pool of objects. When your object is no longer needed, throw it into the pool. When you need a new object, grab one from the pool or allocate a new one if the pool is empty. This will also save a small amount of time over pure allocation because Java doesn't have to clear (bzero) the object.
If you're worried about the pool getting too large (and thus wasting memory), you can either remove an arbitrary number of objects from the pool on a regular basis, or use weak references (for example, using java.util.WeakHashMap). One of the advantages of using a pool is that you can track the allocation frequency and totals, and you can adjust things accordingly.
We're using pools of char[] and byte[], and we maintain separate "bins" of sizes in the pool (for example, we always allocate arrays of size that are powers of two). Our product does a lot of string building, and using pools showed significant performance improvements.
Note: In general, the GC does a fine job. We just noticed that with small objects that point to larger structures, the GC doesn't seem to clean up the objects fast enough especially when the VM is under CPU load. Also, System.gc() is just a hint to help schedule the finalizer thread to do more work. Calling it too frequently causes a significant performance hit.
Given that you're just doing this for testing purposes, it might just be good housekeeping to invoke the garbage collector directly using System.gc() (thus forcing it to make a pass). It won't help you if there is a memory leak, but if there isn't, it might buy you back enough memory to get through your test.