Now, I met a strange case likes that:
public class SoftRefDemo {
private static List<SoftReference<String>> cache;
public static void main(String[] args) {
int total = 3000000;
cache = new ArrayList<SoftReference<String>>(total);
for (int i = 0; i < total; i++) {
cache.add(new SoftReference<String>("fafsfsfsdf" + i));
}
System.out.println(cache.size());
}
}
I have set the JVM setting:-Xms20m -Xmx40m. When I want to put many of SoftReference to cache, the JVM exit without any promption or exception. Actually, I am doubtful of the action of SoftReference, it's special object for JVM. Could anyone explains what's happen for this program?
Another two questions:
1. Does there has extra memory allocation method for those 'special reference instance' in JVM heap?
2. When does those reference instance can be freed when the instance that they pointer to has been freed? Thanks a lot!
When the OOM occurs, there always many of SoftReference instance has existing, can you help to explain this case?
Each instance of SoftReference occupies 24 bytes of heap-memory by itself (this is true for 32-bit Sun JVM, and may differ for others VMs). You are trying to store in the list 3'000'000 instances which means you will need at least ~70Mb of heap space just to store the SoftReference objects.
SoftReference object keeps a "soft reference" to the soft-reachable object (String in your case) and JVM spec guarantees those references will be cleared (i.e. String objects will be garbage collected) before the virtual machine throws an OutOfMemoryError. However JVM will NOT garbage-collect SoftReference objects since you keep strong references to them from your cache list. (So if you need SoftReference object to be removed from the heap - remove it from the cache list).
If I run this program with -mx40m
char[] chars = new char[4096];
List<SoftReference<String>> strings = new ArrayList<SoftReference<String>>();
do {
strings.add(new SoftReference<String>(new String(chars)));
} while(strings.get(0).get()!=null);
int nulls=0, set=0;
for (SoftReference<String> string : strings) {
if(string.get() == null) nulls++; else set++;
}
System.out.println("nulls= "+nulls+", was still set= "+set);
I get
nulls= 4618, was still set= 1
One of the problems with WeakReferences and SoftReferences is that they tend to all be cleared at once.
Try WeakReference, SoftReferences are only clear if it really has to. To see them clearer, try creating a large enough array to trigger an OutOfMemoryError.
All object are in the heap. Even static fields are wrapped in a pseudo object which is on the heap. (This later behaviour is not defined but it how I have seen it work)
The reference instance is freed after it has been discarded (like any other object)
Related
I'm reading the Effective Java book and its saying that eliminating obsolete reference is one of best way to avoid memory leaks. according to the below program, by doing -> elements[size] = null; its eliminating obsolete references in that program.
My problem here what is the advantage of doing elements[size] = null;. Any other program can use that freed memory location? Or is it garbage collected?
According to my understanding the array is already allocated the memory for its size. Even we do elements[size] = null; anyone can't use that freed memory location until you do elements = null;. Please someone tell me what is advantage of doing elements[size] = null; here.
public Object pop() {
if (size == 0)
throw new EmptyStackException();
Object result = elements[--size];
elements[size] = null; // Eliminate obsolete reference
return result;
}
My problem here what is the advantage of doing elements[size] = null;.
Here obsolete references refer to object references not required any longer for the program.
You want that unnecessary objects to be free to consume only memory that your program need. Generally it is done for the good working of the current application.
Any other program can use that freed memory location?
Theoretically yes but it also depends on the JVM memory options used. You don't generally focus on it.
elements[size] = null and elements = null; don't have at all the same intention and the same effects.
In the context of the book, elements is a structural intern of a class.
The idea is that some elements of the array may be stale and not required any longer after some removal operations.
The first one (elements[size] = null) will make the object of the array element located at the size index to be eligible to be GC if no other objects reference .
But the second one (elements = null) is much more. It will make all elements of the array to be eligible to be GC if no other objects reference it.
There are two cases we have to distinguish:
The outer object is "teared down" somehow, so it closes any open resource and also "voluntarily" releases all objects it had referred to. This s simply the explicit way of telling the jvm that the corresponding refence is "gone". You make it easier for the gc to understand: the corresponding object is eligible for garbage collection. Of course, that only has that effect if there are no other references to the same object elsewhere. And beyond: doing so isn't really required, the jvm/gc must of course be able to detect any eligible object all by itself.
But nullifying makes sense for refences that exist for longer periods of time, pointing to different objects over that time span. Like a container, such as the stack class in the underlying example. A container must forget about objects it referenced to when they get "removed". Otherwise you create a memory leak!
What happens here?
Let's imagine, elements is a 20-elements Object array (elements = new Object[20];), and has been filled with 18 BigInteger instances, the remaining two places being null.
So the heap now contains 18 BigInteger instances and a 20-elements Object[] array. The garbage collector won't reclaim any of these instances, and that's okay as you'll most probably use them later (via the pop() method).
Now you call the pop() method to get the BigInteger most recently added to the array. Let's assume you just want to print it and then forget it, so in your overall application that number isn't needed any more, and you'd expect the garbage collector to reclaim it. But that won't happen unless you do the null assignment
elements[size] = null; // Eliminate obsolete reference
Why that?
As long as you store the reference to an object in some accessible place, the garbage collector believes that you'll still need the object later.
As long as elements[17] still refers to the BigInteger, it can potentially be accessed by your program, so it can't be reclaimed. If elements[17] points to null, the BigInteger that used to be there isn't accessible via elements any more and can be reclaimed by the garbage collector (if no other part of your code still uses it).
Conclusion
It's only worth thinking about "obsolete references" if you have a long-living storage structure that contains fat objects, and you can tell at some point in time that you won't need one of the stored objects any more. As you won't need this object any more, you can now re-assign the storage with null, and then the GC no longer believes you still need the object and is able to reclaim the storage space.
I am reading "Item 6: Eliminate obsolete object references" of Effective Java second edition.
Below is the code snippet.
//Can you spot the "memory leak"?
public class Stack {
private Object[] elements;
private int size = 0;
private static final int DEFAULT_INITIAL_CAPACITY = 16;
public Stack() {
elements = new Object[DEFAULT_INITIAL_CAPACITY];
}
public void push(Object e) {
ensureCapacity();
elements[size++] = e;
}
public Object pop() {
if (size == 0)
throw new EmptyStackException();
return elements[--size];
}
/**
* Ensure space for at least one more element, roughly doubling the capacity
* each time the array needs to grow.
*/
private void ensureCapacity() {
if (elements.length == size)
elements = Arrays.copyOf(elements, 2 * size + 1);
}
}
As per this item, memory leak is because after popping, array index was not referenced to NULL like below:
public Object pop() {
if (size == 0)
throw new EmptyStackException();
Object result = elements[--size];
elements[size] = null; // Eliminate obsolete reference
return result;
}
My understanding have been that suppose for an given array, I have done elements[0] = new Object() and then I do this again elements[0] = new Object() then my first object would be eligible for garbage collection because 0th index of my array is no more pointing to it.
Is my understanding incorrect? If it is correct then how it is shown as memory leak in Effective Java.
You got most of it.
If you do:
elements[0] = someOtherObject;
then the other element stored at index 0 is no longer referenced and might be collected.
But the first pop() implementation keeps that reference in place - it only decreases the "counter" of stored elements. Therefore that object is still referenced - and won't be collected until a new object is added to the stack!
As the comment in the second version of pop() clearly states - the reference has to be eliminated to ensure that the stack doesn't keep a reference to that very object. The object is supposed to be popped - so the stack should not keep knowledge about that removed object!
And to confirm the commit: yes, when one pushes n object, then pushes n other objects, then you don't have a memory leak - because the underlying array references will all be updated and point to new objects then. And yes, if less than n objects get pushed after popping, stale references are kept and preventing garbage collection here.
The issue pertains to the fact that the array is still holding reference to objects which have only been logically popped off the array(decreasing the size counter). This means that the only way to ever get this memory back would be to garbage collect the entire stack by setting it to null.
You are correct with your case that if you just re-assigned to the nth index it would not be a leak, because you still expect that object to exist. However with pop, your aim is to decrease the size of the stack, which means any memory which was assigned for the top of the stack, should be collected after popping.
Quoting from Effective Java (emphasis mine)
If a stack grows and then shrinks, the objects that were popped off the stack will not be garbage collected, even if the program using the stack has no more references to them. This is because the stack maintains obsolete references to these objects. An obsolete reference is simply a reference that will never be dereferenced again. In this case, any references outside of the “active portion” of the element array are obsolete. The active portion consists of the elements whose index is less than size.
He refers to the references of the elements that are popped off.
But, you are right in your example, when you store the reference to a new Object at index 0, there is no reference to the first Object and hence it is eligible for Garbage creation.
But say,
You create five Objects (elements[0]... elements[4])
You pop three elements. This would leave your top variable (size here) pointing at index 2.
But still, you would have 5 active references which would prevent the last three objects from being garbage collected.
The term "Memory Leak" was borrowed from C and is often misused in Java. Memory Leak in C sense is a range of bytes allocated on heap that has no references in the code and thus can't be freed. For example:
// ...
char* leak = malloc(10); // Local reference to heap
return; // reference lost
In Java such leaks are impossible, since any lost reference is subject to GC.
There are situations, however, that can result in Java code using more memory than it should. Your code represents one of many possible examples of such behavior. In your case, as it was explained in previous answers, some elements of the stack will remain in the heap just because the array is holding references to objects that are no longer needed. In GC environments this is usually called "Lingering objects". A good way to spot memory usage problems in Java is to check heap in use after GC. If heap usage after GC is consistently going up, you probably have lingering objects or other memory allocation problems. For example, if heap in use is 1M after 1st GC, 2M after 2nd GC and 3M after 3d GC - you should probably use a Java Memory Profiler to pinpoint the problem. Note that in your example heap usage will not go up between GCs, but it will not go down either. If you assign null to unused objects, the heap usage will go down after GC if the stack shrinks.
With the following function:
Collection#clear
how can I attempt to reclaim memory that could be freed from an invocation? Code sample:
public class Foo
{
private static Collection<Bar> bars;
public static void main(String[] args){
bars = new ArrayList<Bar>();
for(int i = 0; i < 100000;i++)
{
bars.add(new Bar());
}
bars.clear();
//how to get memory back here
}
}
EDIT
What I am looking for is similar to how ArrayList.remove reclaims memory by copying the new smaller array.
It is more efficient to only reclaim memory when you need to. In this case it is much simpler/faster to let the GC do it asynchronous when there is a need to do. You can give the JVM a hint using System.gc() but this is likely to be slower and complicate your program.
how ArrayList.remove reclaims memory by copying the new smaller array.
It doesn't do this. It never shrinks the array, nor would you need to.
If you really need to make the collection smaller, which I seriously doubt, you can create a new ArrayList which has a copy of the elements you want to keep.
bars= null ;
would be the best. clear doesn't guarantee to release any memory, only to reset the logical contents to "empty".
In fact, bars= null ; doesn't guarantee that memory will be immediately released. However, it would make the object previously pointed by bars and all its dependents "ready for garbage collection" ("finalization", really, but let's keep this simple). If the JVM finds itself needing memory, it will collect these objects (other simplification here: this depends on the exact garbage collection algorithm the JVM is configured to use).
You can't.
At some point after there are no more references to the objects, the GC will collect them for you.
EDIT: To force the ArrayList to release its reference to the giant empty array, call trimToSize()
You can't force memory reclamation, that will happen when garbage collection occurs.
If you use clear() you will clear the references to objects that were contained in the collection. If there are no other references to those objects, then they will be reclaimed next time GC is run.
The collection itself (which just contains references, not the objects referred to), will not be resized. The only way to get back the storage used by the collection is to set the reference bars to null so it will eventually be reclaimed.
When does the object of type list, occupying memory, become eligible for garbage collection, Also where is the variable that holds reference to the list ? In the case of code below, there was no variable assigned to it.
CASE 1:
for (Integer i : returnList()) {
System.out.println(i);
}
In case of a code like:
CASE 2:
List list = returnList();
for (Integer i : list) {
System.out.println(i);
}
list = null;
We can take control of GC, Is there any ways to take care of that in the first case when no variable was assigned ?
To summarize:
What is the mechanism of referrence, without a reference variable to list is case 1?
Does list get eligible for GC'd when stack frame is popped ?
Any way to speed up the eligibility for GC'ing ?
What is the mechanism of referrence, without a reference variable to list is case 1?
There is an implicit reference to the list. This can be seen by understanding that enhanced for like that is translated into:
for(Iterator e = returnList().iterator(); e.hasNext(); ) {
Integer i = (Integer)e.Next();
System.out.println(i);
}
Here, e has a reference to an iterator on returnList, which itself has a reference to returnList. Thus, returnList is rooted as long as e is rooted which is only true while control is in the for loop. When control leaves the for body, e is eligible for collection, so returnList is eligible for collection.
Of course, all of this is assuming that
The owner of returnList isn't maintaining a reference to its return value.
The same list hasn't been returned to another caller and that other caller isn't maintaining a reference to the same list.
Does list get GC'd when stack frame is popped ?
Not necessarily. It will be eligible for collection when the JVM can determine that the referrent has no rooted references to it. Note that it does not necessarily immediately get collected.
Any way to speed up GC in case 1.
It can't be collected any sooner than control leaving the for loop. It might be collected after control leaves the for loop. Let the JVM worry about this.
Note that you can attempt a manual garbage collection via
System.gc();
but note that this might exhibit worse behavior because if it triggers a garbage collection, it might be a full garbage collection. Note that the JVM can ignore this request. You might be wasting a lot of CPU cycles here. Note that on a system with infinite memory, the garbage collector never needs to run. On such a system, requesting the garbage collector could be a complete waste of CPU cycles if the garbage collector obeys your request.
Let the JVM manage the garbage collections. The algorithms for it are highly tuned.
From SCJP page 256-257
The garbage collector does some magical, unknown operations, and when
it discovers an object that can't be reached by any live thread,it
will consider that object as eligible for deletion, and it might even
delete it at some point. (You guessed it; it also might not ever
delete it.) When we talk about reaching an object, we're really
talking about having a reachable reference variable that refers to the
object in question. If our Java program has a reference variable that
refers to an object, and that reference variable is available to a
live thread, then that object is considered reachable.
Setting an object to NULL might help to fasten when the GC deletes the object but it might as well not, you don't have control over that. The JVM will run GC when it senses that memory is running low and you can manually ask it to do so but nothing is guaranteed.
What is the mechanism of reference, without a reference variable to
list is case 1?
see next answer.
Does list get GC'd when stack frame is popped ?
The list is created on the heap - but if the only reference to it was on the stack it is eligible to be collected by GC. That doesn't mean that it'll happen any time soon though.
Any way to speed up GC in case 1.
You can't "speed up" GC, even by calling System.gc(); you're only "suggesting" that the GC can to do its work - again, it won't necessarily happen any time soon.
There's a lot of sense behind it too: say that your program has 2GB of memory to use and is currently using only 2KB - it does not justify GC stopping your program from running and clean the memory only because some objects are eligible for deletion.
You cannot take control of GC.It is managed by JVM. What you can manage is what objects should be available for garbage collection. Although , you can find out when garbage collector will run using finalize method. It is always called before an object is deleted
public class Demo{
static void create()
{
Demo o = new Demo();
}
public void finalize()
{
System.out.println("GC called");
}
public static void main (String ...ar)
{
for (long i=1;i<900000;i++) //Try changing values here
{
create();
}
}
}
Objects created inside the methods are available for GC when method is returned(just like local variables exist for duration of method).However if the method returns an object, it will not be avialble for garbage collection
public class Demo{
public void getDate()
{
Date o = new Date();
StringBuffer d = new StringBuffer(o.toString());
System.out.println(d);
return o;
}
public static void main (String ...ar)
{
Date x= getDate();
}
}
In the above code, object d is available for GC when the method returns. But object o will not be available for collection
I heard most elegant property of java is Garbage Collection
I wanna know does it guarantee that a program will not run out of memory?
No, it's always possible that you'll try to allocate more memory than is available.
Automatic garbage collection only means that garbage (i.e., unreferenced memory) is automatically collected (i.e., reclaimed for further use). If you keep references to it, it's not garbage, and not collected.
No it does not guarantee this. It is perfectly possible for a programmer to mistakingly create objects which never go out of scope, thus consuming more and more memory until all heap is exhausted.
It is the programmer's responsibility to ensure that objects no longer in use are no longer referenced by the application. That way the garbage collector can do its job and reclaim memory used by these objects.
Example
public class Main {
public static void main(String[] main) {
List<String> l = new LinkedList<String>();
// Enter infinite loop which will add a String to
// the list: l on each iteration.
do {
l.add(new String("Hello, World"));
} while(true);
}
}
No, there are still many ways to run out of memory. The garbage collector can only reclaim memory for objects that are no longer referenced - it is up to you to make sure that you are not referencing objects you don't need, or to use Soft References for objects you would like to have, but don't mind disappearing if memory gets tight.
To answer your question, NO. Garbage collection does not guarantee that a program will not run out of memory.
Consider object you don't want to
use any more are like garbage.
References to those objects will be
like having that garbage in your
house.
Garbage collection is like your
town's garbage truck that collects
garbage.
If you won't release those
references, it is like not taking
garbage out and soon your house will
be over filled with garbage as
garbage truck guys won't take out
garbage from your house.
Unreferenced objects will be garbage collected automatically by garbage collector. In java, most references to objects are released automatically once you come out of method.
Objects have reference to other objects, which in turn referr to other objects creating whole object graph. So as such object can be referenced by more than one object.
If object is having zero references,
it is eligible for garbage
collection.
Objects are allocated on heap.
Garbage collector runs from time to
time to delete unreferenced objects
from heap.
If you keep creating more objects on
heap without releasing you will
eventually get OutOfMemoryError
Example with garbage collection at work
public class TestGarbageNoError {
public static void main(String[] args) {
String hugeString;
for (int i = 0; i < Integer.MAX_VALUE; i++) {
System.out.println("i = " + i);
hugeString = getHugeString();
// At each iteration reference variable hugeString
// points to new String object. Hence there will be
// zero reference to previous string object and will
// eventually be garbage collected
}
}
public static String getHugeString() {
StringBuilder sb = new StringBuilder();
for (int x = 0; x < 5000000; x++) {
sb.append(x);
}
return sb.toString();
}
}
.
Example with memory leak at work
public class TestGarbageError {
public static void main(String[] args) {
Collection<String> memoryLeak = new ArrayList<String>();
for (int i = 0; i < Integer.MAX_VALUE; i++) {
System.out.println("i = " + i);
String hugeString = getHugeString();
memoryLeak.add(hugeString);
// At each iteration reference variable hugeString
// points to new String object. But all objects are added
// to memoryLeak Collection and will always have atleast one
// reference, i.e. from memoryLeak object. Hence this string
// objects will never be garbage collected and program will
// eventually run out of memory
}
}
public static String getHugeString() {
StringBuilder sb = new StringBuilder();
for (int x = 0; x < 5000000; x++) {
sb.append(x);
}
return sb.toString();
}
}
No. If you construct a lot of objects (millions) and keep a reference to them so they don't go out of scope (for example by adding them to an ArrayList), you could run out of addressable memory.
Absolutely not. Even in a garbage collected language like Java you can easily lose references, meaning objects will never get garbage collected.
Even then, you may simply instantiate (and keep reference to) too many objects for the system to handle.
How could anything ensure a program doesn't run out of memory short of arbitrarily deleting an item from memory to make room for new allocations?
Now, what if you are actually keeping a reference on (using) the thing randomly chosen to be evicted? You will soon have incorrect behavior.
No. The garbage collector, helps you to free unused memory automatically.
The way it works is, if an object reference can't be reached, the memory for that object may be garbage collected.
For instance:
public void test() {
Object o = new Object();
// the memory used by o may be garbage collected after this line
}
But if you never release object references, the garbage collector will never collect anything and a OutOfMemoryError will be thrown.
List list = ....
public void test() {
o = new Object();
list.add( o );
// the memory used by o WON'T be garbage collected after this line
// because its reference is used in the list.
}
If you use this several times:
while( true ) {
test();
}
The list will keep growing indefinitely until you run out of memory
No. Garbage collection only protects against one kind of memory leak. Specifically, the kind that occurs if you don't explicitly free up memory when your application no longer needs it. If your application holds references to unneeded objects (eg. evergrowing Lists), the garbage collector cannot clean them up and your application can still run out of memory.
No, not at all.
In languages without garbage collection, the programmer (or a library he uses) is responsible for making requests for memory and for returning the allocated memory for "recycling". there are no guarantees that the memory would be available when it is requested. However, if you never explicitly "recycle", there could be a situation where a request is rejected because no memory is available, but that if the memory was recycled that block could have been returned for this request.
Having automated garbage collection means that the system may recycle for you. As a result, certain requests would be filled using "recycled" memory. However, as with non-GC languages, some requests cannot be filled.
For instance, if your system has 1000 blocks available and you need 1500 at the same time, no GC in the world is going to help you because nothing is really available for recycling.
No, garbage collection cannot guarantee that your application will not run out of memory. It will not even guarantee that your application will not run out of memory when memory is available. If you come close to running out of memory or are allocating many object it can cause GC to thrash, throwing an out of memory exception. The program can also run out of memory when it has used all the physical memory (actual and virtual) or the program exceeds the maximum memory allowed by the JVM (see -Xmx).
However, it is guaranteed that before the JVM declares OutofMemoryException it will garbage collect all collectible references and consider to use the now free memory.