Is Java GC run when I'm out of memory? - java

I'm developing an Android-based 3D game in Java (not using the NDK).
One of the must-haves for any game is a resource cache, in which I can store loaded textures, meshes, sounds, etc.
My plan is to develop a resource cache that hands out weak references to certain resources, so that only the cache itself keeps a strong reference to the resource, meaning that if I remove a resource from my cache, it will be garbage collected without me having to worry about it.
I'm having a hard time finding an answer to the following question though:
Say I have filled my resource cache to the point where my heap space is almost completely full. I now want to load an additional resource, but its memory footprint is so large that it won't fit in my available heap space.
To accommodate for the new object's size, I remove some entries from my cache and I load in the new object. Will this possibly cause an OutOfMemory exception?
As long as the garbage collector hasn't sweeped through my recently removed objects, the memory stays in use, so is the garbage collector smart enough to run a new sweep in order to accommodate the newly allocated object?

The JVM will run a Full GC before throwing an OutOfMemoryError (it doesn't want to throw OOME, so it'll attempt everything possible). It's just that considerate.

GC is not guaranteed to run, although it usually does. I know I've seen OOME without GC and GC could have made a difference but it's rare. Usually if you've dereferenced objects and they are eligible for GC, then when you need memory allocated for new objects the JVM will do its job great and you will be fine. Properly dereferencing objects is usually the problem if you get an OOME and you think enough memory should be available.
See this post:
Is the garbage collector guaranteed to run before Out of Memory Error?
Also you can request GC, but that also does not guarantee that it will be run, and it may not be a good idea. See this post:
Garbage collector in Android

The [Java Machine specification states(#6.3)] states this:
OutOfMemoryError: The Java Virtual Machine implementation has run out of either virtual
or physical memory, and the **automatic storage manager was unable to reclaim enough memory**
to satisfy an object creation request.
So JVM does guarantee that it will try what it can to free up memory through GC before it throws an OutOfMemoryError.

To accommodate for the new object's size, I remove some entries from my cache and I load in the new object. Will this possibly cause an OutOfMemory exception? As long as the garbage collector hasn't sweeped through my recently removed objects, the memory stays in use, so is the garbage collector smart enough to run a new sweep in order to accommodate the newly allocated object?
By remove I understand you set them to null to make them eligible for GC. Although you have set them to null, the GC might decide not to do a memory sweep at all. Now you could explicitly call System.gc but then again the GC will be running in a separate low-priority thread. And collection of Objects will not be immediate. Now, once you try to load the large resource, the GC finds out that the memory is less, it starts its task to free as much memory as possible. But it will still throw OutOfMemory because you don't have memory when trying to load your resource.

I just found out the answer to my own question in accordance with what Kayaman said earlier:
The Java Virtual Machine implementation has run out of either virtual or physical memory, and the automatic storage manager was unable to reclaim enough memory to satisfy an object creation request.
Found at http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html#jvms-6.3

Related

Clearing an arraylist in many ways, but it's not freeing up memory

I made a particle system thingie for fun and giggles, and it's working pretty well, even this problem is not catastrophic at all, it's just making me super curious.
I store my particles in an ArrayList. When I middle mouse click on a particle emitter, it clears the arraylist, thus deltes all particles. And it's working. However, even after the arraylist is cleared, there seems to be not a single byte freed up in my RAM.
I believe I don't have a way to reference individual particles, except through the arraylist, because I create all of them with this line:
particles.add(new Particle()); //particles is an ArrayList<Particle>, ofc.
Thus, I don't understand why the garbage collector isn't collecting them.
I tried:
using the empty() method(that's what I still use), and after/before(logically) that:
setting all about-to-be-deleted Particle objects to null
calling System.gc()
overwriting the arraylist with a brand new, empty arraylist
using trimToSize()
starting the .jar with these parameters I found on the internets:
-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
waiting entire minutes, maybe the garbage collector will suddenly work.
But none of this resulted in any noticable change.
I'm checking my ram usage through good old task manager. Could the problem be simply that it's displaying incorrectly?
There is a tool included in the JDK called "JVisualVM". It can be attached onto a java process and gives you quite detailed information about the garbage collection and the heap. It will also enable you to see if the objects were garbage collected or not.
Concerning the task manager: This is not a reliable source to check whether objects have been collected, as the heap stays at its size (unless it needs to increase) for quite some time.
As already mentioned in a comment above, a call to System.gc() may be completely ignored by your JVM.
Java uses a mechanism called garbage collection to return memory to the pool. Entire books have been written about how it works, but the gist of it is that once memory is used it is not returned to the OS free memory pool until a deep garbage collelction occurs.
In the deep garbage collection, the JVM computes the storage requirement for everything that is a candidate for retention, claims a new heap from the OS, and copies the object being retained from the old heap into the new heap. Once the objects are copied, the old heap is released to the OS.
This approach means (1) you will only see memory freed by your process inside the JVM most of the time, and (2) you need to have twice as much RAM on your system as the size of the JVM, to allow for the garbage collector.
Does this help?
I tried:
using the empty() method(that's what I still use) ...
Wrong already. The empty() method tells you whether it is empty. It doesn't empty it. The method you are looking for is clear().
I'm checking my ram usage through good old task manager.
Wrong again. The Java garbage collector doesn't release memory back to the operating system.

What triggers garbage collection [duplicate]

This question already has answers here:
What triggers a full garbage collection in Java?
(2 answers)
Closed 9 years ago.
I know what all gets garbage collected. But what exactly tells JRE that it is time for garbage collection? Is it like some event or time interval?
HotSpot's garbage collection has grown into an exceedingly complex business, which even its creators struggle to understand in full detail. Therefore you can't be given a simple answer; some triggers are:
occupation of each object generation reaching a threshold;
a memory allocation request in a specific generation failing;
overall heap occupation reaching a threshold.
Note that you haven't even specified what kind of garbage collection you are interested in: there is a minor collection and a major collection, and technologically they are very different. You have also not specified which Garbage Collector you have in mind: HotSpot has four of them to choose from.
If you are a beginner with Java, the best advice to give is a) in day-to-day programming, don't worry about it; and b) if you want to learn, you'll have to dig deep.
object has null reference then it will garbage collected. but GC does not give guarantee like when it is done.
I thing as a good practices dont fully depend on GC,See THIS OR THIS
The JVM controls the Garbage collector, it decides when to run the Garbage Collector. It will run the GC when it realizes that the memory is running low or an object become eligible for GC when no live thread can access it.
But this behavior of JVM cannot be guaranteed, one can request the GC to happen from within the java program but there is no guarantee that this request will be taken care by JVM.
Garbage collection in java happen when JVM thinks it needs a garbage collection based on Java heap size.
But you can force GC to collect garbage using
System.gc ()
or
Runtime.gc ()
But it’s not guaranteed that garbage collection will happen.
Read more: http://javarevisited.blogspot.com/2011/04/garbage-collection-in-java.html#ixzz2Y9gKzQE1
You can use System.gc(); but there's no guarantee that it'll run (it just "suggests" to the GC to run).
The JVM has different implementations, including the GC which can be run in different modes, and has very complex algorithms which usually work pretty good, that said, if you have a special usage (or special events, like after a restart of a platform) - you can tune it using flags (like: minimum heap size, maximum heap size and etc), but even without doing so, the GC collects objects that have null reference (pointers that points to them) whenever there's a need to free memory from the heap.

System.gc does not clears in a single run. Uses 3 or more calls for clearing

I am testing the usage of Heap size in a java application running in JDK 1.6. I use the tool VisualVM to monitor the heap usage. I found the Maximum heap size usage of around 500 MB for a few mins. I used the option "Perform GC" which calls System.gc(). The first time i used it, the Maximum heap is reduced to 410MB, then once again I used it to get 130MB and the next time to 85MB. I made all the four calls next to next without any interval. Why does the call System.gc() does not collect all the Heap to 85MB at first time. Is there any other reason behind this. Or I should try with any other methods?
The System.gc() will return when all objects have been scanned once.
An object should be finalized() AFTER it has been collected. Most objects don't implement this method but for the ones which do, they are added to a queue to be cleaned up later. This means those objects cannot be cleaned up yet (not the queue nodes which hold them) i.e. the act of triggering a GC can increase memory consumption temporarily.
Additionally there are SoftReferences to objects which may or may not be cleaned up by a GC. The assumption is these should only be cleaned up if not much else was cleaned up.
In short, not all objects can be cleaned up in one cycle.
System.gc() requests the JVM to start garbage collection. If you are expecting that GC is invoked as soon as System.gc() then it is a wrong notion. Calling it multiple times will not help. It is not possible to map System.gc() with the actual garbage collection. Also no matter how many times you call System.gc(), JVM will do the GC only when it is ready to do so. What may be happening is that heap size is getting reduced even with the first System.gc() but not exactly as soon as you call it. Garbage collection related to your first System.gc() may be finishing in background and in parallel your code is reaching third System.gc() statement.
If you are pretty sure that only adding multiple System.gc() helps you reducing the heap size. Then you need to check what all objects are getting created in JVM in between first and last System.gc(). There may be other threads creating the objects.
One possible reason might be the use of java.lang.ref.Reference types. If the GC is going to break a "Reference" this will happen after the GC proper has completed. Any objects that become unreachable as a result are left for the next GC cycle to deal with.
Finalization works the same way. If an object requires finalization, it and all of the objects reachable from it (only) are likely to only be collectable in the next GC cycle.
Then there is the issue that the GC's algorithm for shrinking the heap is non-aggressive. According to the Java HotSpot VM Options page, the GC only shrinks the heap if more than 70% is free after garbage collection. However, it is not entirely clear if this refers to a full GC or not. So you could get the GC doing a partial GC and shrinking, and then a full GC and shrinking some more.
(Some people infer from the wording of the System.gc() javadocs that it will perform a full GC. However, I suspect that this is actually version / GC dependent.)
But to be honest this should all be moot. Trying to coerce an application into giving back as much memory is possible is pointless. The chances are that you are forcing it to throw away cached data. When the application gets active again it will start reloading its caches.

Forcing Java virtual machine to run garbage collector [duplicate]

This question already has answers here:
How to force garbage collection in Java?
(25 answers)
Closed 8 years ago.
I have a complex java application running on a large dataset. The application performs reasonably fast but as time goes it seems to eat lots of memory and slow down. Is there a way to run the JVM garbage collector without re-starting the application?
No, You cant force garbage collection.
Even using
System.gc();
You can just make a request for garbage collection but it depends on JVM to do it or not.
Also Garbage collector are smart enough to collect unused memory when required so instead of forcing garbage collection you should check if you are handling objects in a wrong way.
If you are handling objects in a wrong way (like keeping reference to unnecessary objects) there is hardly anything JVM can do to free the memory.
From Doc
Calling the gc method suggests that the Java Virtual Machine expend
effort toward recycling unused objects in order to make the memory
they currently occupy available for quick reuse. When control returns
from the method call, the Java Virtual Machine has made a best effort
to reclaim space from all discarded objects.
Open Bug regarding System.gc() documentation
The documentation for System.gc() is extremely misleading and fails to
make reference to the recommended practise of never calling
System.gc().
The choice of language leaves it unclear what the behaviour would be
when System.gc() is called and what external factors will influence
the behaviour.
Few useful link to visit when you think you should force JVM to free up some memory
1. How does garbage collection work
2. When does System.gc() do anything
3. Why is it bad practice to call System.gc()?
All says
1. You dont have control over GC in Java even System.gc() dont guarantee it.
2. Also its bad practise as forcing it may have adverse effect on performance.
3. Revisit your design and let JVM do his work :)
you should not relay on System.gc() - if you feel like you need to force GC to run it usually means that there is something wrong with your code/design. GC will run and clear your unused objects if they are ready to be created - please verify your design and think more about memory management, look as well for loops in object references.
The
System.gc()
call in java, suggest to the vm to run garbage collection. Though it doesn't guarantee that it will actually do it. Nevertheless the best solution you have. As mentioned in other responses jvisualvm utility (present in JDK since JDK 6 update 7), provides a garbage functionality as well.
EDIT:
your question open my appetite for the topic and I came across this resource:
oracle gc resource
The application performs reasonably fast but as time goes it seems to eat lots of memory and slow down.
These are a classic symptoms of a Java memory. It is likely that somewhere in your application there is a data structure that just keeps growing. As the heap gets close to full, the JVM spends an increasing proportion of its time running the GC in a (futile) attempt to claw back some space.
Forcing the GC won't fix this, because the GC can't collect the data structure. In fact forcing the GC to run just makes the application slower.
The cure for the problem is to find what is causing the memory leak, and fix it.
Performance gain/drop depends how often you need garbage collection and how much memory your jvm has and how much your program needs.
There is no certainity(its just a hint to the interpreter) of garbage collection when you call System.gc() but at least has a probability. With enough number of calls, you can achieve some statistically derived performance multiplier for only your system setup.
Below graph shows an example program's executions' consumptions and jvm was given only 1GB(no gc),1GB(gc),3GB(gc),3GB(no gc) heaps respectively to each trials.
At first, when jvm was given only 1GB memory while program needed 3.75GB, it took more than 50 seconds for the producer thread pool to complete their job because having less garbage management lead to poor object creation rate.
Second example is about %40 faster because System.gc() is called between each production of 150MB object data.
At third example, jvm is given 3GB memory space while keeping System.gc() on. More memory has given more performance as expected.
But when I turned System.gc() off at the same 3GB environment, it was faster!
Even if we cannot force it, we can have some percentage gain or drain of performance trying System.g() if we try long enough. At least on my windows-7 64 bit operating system with latest jvm .
Garbage collector runs automatically. You can't force the garbage collector.
I do not suggest that you do that but to force the garbage collector to run from within your java code you can just use all the available memory, this works because the garbage collector will run before the JVM throws OutOfMemoryError...
try {
List<Object> tempList = new ArrayList<Object>();
while (true) {
tempList.add(new byte[Integer.MAX_VALUE]);
}
} catch (OutOfMemoryError OME) {
// OK, Garbage Collector will have run now...
}
My answer is going to be different than the others but it will lead to the same point.
Explain:
YES it is possible to force the garbage collector with two methods used at the same time and in the same order this are:
System.gc ();
System.runFinalization ();
this two methods call will force the garbage collector to execute the finalise() method of any unreachable object and free the memory. however the performance of the software will down considerable this is because garbage runs in his own thread and to that one is not way to controlled and depending of the algorithm used by the garbage collector could lead to a unnecessary over processing, It is better if you check your code because it must be broken to you need use the garbage collector to work in a good manner.
NOTE: just to keep on mind this will works only if in the finalize method is not a reassignment of the object, if this happens the object will keep alive an it will have a resurrection which is technically possible.

When does Java's garbage collection free a memory allocation?

I have created an object in Java, Named FOO. FOO contains a large amount of data.. I don't know say for a ten mega byte text file that I have pulled into ram for manipulation.(This is just an example)
This is clearly a huge amount of space and I want to deallocate it from memory. I set FOO to NULL.
Will this free up that space in memory automatically?
or
Will the memory taken by the loaded text file be around until automatic garbage collection?
When you set the reference of any object to null, it becomes available for garbage collection. It still occupies the memory until the garbage collector actually runs. There are no guarantees regarding when GC will run except that it will definitely run and reclaim memory from unreachable objects before an OutOfMemoryException is thrown.
You can call System.gc() to request garbage collection, however, that's what it is - a request. It is upto GC's discretion to run.
Using a WeakReference can help in some cases. See this article by Brian Goetz.
Actually the object is not named FOO. FOO is the name of a variable which is not the object; the variable contains a reference to the object. There could be several distinct variables containing references to the same object.
The garbage collector works by automatically detecting unreachable objects: these are objects which the application cannot use anymore because it has irretrievably forgotten where they are (the application may possibly access any object for which it has a reference to, including the references stored in field in objects it can access, and so on).
When you set FOO = null, assuming that FOO contained at that point the last reachable reference to the object, then the memory is released immediately, in the following sense: at the very clock cycle at which null is set in FOO, the object becomes unreachable. Therefore, the garbage collector will notice that unreachable object and reclaim the corresponding memory block; that is, the GC will do that the next time it can be bothered to run. Of course, the actual bits which constitute the object may linger a bit in memory; but that block is nonetheless "free" since the memory allocator will automatically run the GC when free memory is tight. From the application point of view, the object is as good as dead and the corresponding memory is free since that memory will be reused the next time the application needs it. The whole thing is automatic.
Things are a bit more complex with regards to the operating system. If an unreachable object is free memory from the application point of view, it is still, as far as the OS is concerned, a block of RAM dedicated to the running process. That block of RAM may be given back to the OS only when the GC (which is, at the OS level, a part of the process) actually runs, notices that the object is unreachable, and condescends to give the block back to the OS. When the GC runs heavily depends on the GC technology and how the application allocates objects; also, some GC will never give back the block the OS at all (the GC knows that the block it free, the memory allocator will reuse it at will, but not other processes).
System.gc() is a hint to the VM, so that it runs the GC now. Formally, it is only a hint, and the VM is free to ignore it. In practice, it runs the GC, unless the VM was instructed not to obey such commands (with Sun's JVM, this is a matter of a specific command-line flag). Even if the GC runs, it does not necessarily give back the memory to the operating system. System.gc() is not terribly useful.
Setting foo = null; does not mean that foo will be garbage collected immediately. Instead, it will be collected when the GC next runs, if it can be. When foo is collected, any objects for which it holds the sole reference will also be eligible for collection and therefore collected.
Note that even calling System.gc() does not guarantee that that JVM will do it right away.
System.gc() is just a request and there is no guarantee that it's effect immediately.
There's no guarantee that JVM will do it right away, you can try to force it by using System.gc()
The garbage collector will free the memory after you "destroy" the reference. i.3 Setting the object reference to null. You can use forced garbage collection option but you should use it with care. The Garbage collector is designed to use an optimized schedule so calling the System.gc() may ruin the rhythem and possibly have less performance due to unnecessary task switching.
Alternatively you can think about a way that allows you to not to load large amounts of data into memory. If you can gain that by improving your code that would be much better.

Categories

Resources