I am currently trying to diagnose a slow memory leak in my application. The facts I have so far are as follows.
I have a heap dump from a 4 day run of the application.
This heap dump contains ~800 WeakReference objects which point to objects (all of the same type, which I will call Foo for the purposes of this question) retaining 40mb of memory.
Eclipse Memory Analysis Tool shows that each of the Foo objects referred to by these WeakReferences is not referred to by any other objects. My expectation is that this should make these Foo objects Weakly Reachable and thus they should be collected at the next GC.
Each of these Foo objects has a timestamp which shows that they were allocated over the course of the 4 day run. I also have logs during this time which confirm that Garbage Collection was happening.
A huge number of Foo objects are being created by my application and only a very small fraction of them are ending up in this state within the heap dump. This suggests to me that the root cause is some sort of race condition.
My application uses JNI to call through to a native library. The JNI code calls NewGlobalRef 4 times during start of day initialisation to get references to Java classes which it uses.
What could possibly cause these Foo classes to not be collected despite only being referenced by WeakReferences (according to Eclipse Memory Analyser Tool)?
EDIT1:
#mindas
The WeakReference I am using is equivalent to the following example code.
public class FooWeakRef extends WeakReference<Foo>
{
public long longA;
public long longB;
public String stringA;
public FooWeakRef(Foo xiObject, ReferenceQueue<Foo> xiQueue)
{
super(xiObject, xiQueue);
}
}
Foo does not have a finalizer and any finalizer would not be a consideration so long as the WeakRefs have not been cleared. An object is not finalizable when it is weakly reachable. See this page for details.
#kasten The weakreferences are cleared before the object is finalizable. My heap dump shows that this has not happened.
#jarnbjo I refer to the WeakReference Javadoc:
"Suppose that the garbage collector determines at a certain point in time that an object is weakly reachable. At that time it will atomically clear all weak references to that object and all weak references to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references."
This suggests to me that the GC should be detecting the fact that my Foo objects are "Weakly reachable" and "At that time" clearing the weak references.
EDIT 2
#j flemm - I know that 40mb doesn't sound like much but I am worried that 40mb in 4 days means 4000mb in 100 days. All of the docs I have read suggest that objects which are weakly reachable should not hang around for several days. I am therefore interested in any other explanations about how an object could be strongly referenced without the reference showing up in a heap dump.
I am going to try allocating some large objects when some of these dangling Foo objects are present and see whether the JVM collects them. However, this test will take a couple of days to setup and complete.
EDIT 3
#jarnbjo - I understand that I have no guarantee about when the JDK will notice that an object is weakly reachable. However, I would expect that an application under heavy load for 4 days would provide enough opportunities for the GC to notice that my objects are weakly reachable. After 4 days I am strongly suspicious that the remaining weakly references objects have been leaked somehow.
EDIT 4
#j flemm - Thats really interesting! Just to clarify, are you saying that GC is happening on your app and is not clearing Soft/Weak refs? Can you give me any more details about what JVM + GC Config you are using? My app is using a memory bar at 80% of the heap to trigger GC. I was assuming that any GC of the old gen would clear Weak refs. Are you suggesting that a GC only collects Weak refs once the memory usage is above a higher threshold? Is this higher limit configurable?
EDIT 5
#j flemm - Your comment about clearing out WeakRefs before SoftRefs is consistent with the Javadoc which states:
SoftRef: "Suppose that the garbage collector determines at a certain point in time that an object is softly reachable. At that time it may choose to clear atomically all soft references to that object and all soft references to any other softly-reachable objects from which that object is reachable through a chain of strong references. At the same time or at some later time it will enqueue those newly-cleared soft references that are registered with reference queues."
WeakRef: "Suppose that the garbage collector determines at a certain point in time that an object is weakly reachable. At that time it will atomically clear all weak references to that object and all weak references to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references. At the same time it will declare all of the formerly weakly-reachable objects to be finalizable. At the same time or at some later time it will enqueue those newly-cleared weak references that are registered with reference queues."
For clarity, are you saying that the Garbage Collector runs when your app has more than 50% free memory and in this case it does not clear WeakRefs? Why would the GC run at all when your app has >50% free memory? I think your app is probably just generating a very low amount of garbage and when the collector runs it is clearing WeakRefs but not SoftRefs.
EDIT 6
#j flemm - The other possible explanation for your app's behaviour is that the young gen is being collected but that your Weak and Soft refs are all in the old gen and are only cleared when the old gen is being collected. For my app I have stats showing that the old gen is being collected which should mean that WeakRefs get cleared.
EDIT 7
I am starting a bounty on this question. I am looking for any plausible explanations for how WeakRefs could fail to be cleared while GC is happening. If the answer is that this is impossible I would ideally like to be pointed at the appropriate bits of OpenJDK which show WeakRefs being cleared as soon as an object is determined to be weakly reachable and that weak reachability is resolved every time GC runs.
I have finally got round to checking the Hotspot JVM source code and found the following code.
In referenceProcessor.cpp:
void ReferenceProcessor::process_discovered_references(
BoolObjectClosure* is_alive,
OopClosure* keep_alive,
VoidClosure* complete_gc,
AbstractRefProcTaskExecutor* task_executor) {
NOT_PRODUCT(verify_ok_to_handle_reflists());
assert(!enqueuing_is_done(), "If here enqueuing should not be complete");
// Stop treating discovered references specially.
disable_discovery();
bool trace_time = PrintGCDetails && PrintReferenceGC;
// Soft references
{
TraceTime tt("SoftReference", trace_time, false, gclog_or_tty);
process_discovered_reflist(_discoveredSoftRefs, _current_soft_ref_policy, true,
is_alive, keep_alive, complete_gc, task_executor);
}
update_soft_ref_master_clock();
// Weak references
{
TraceTime tt("WeakReference", trace_time, false, gclog_or_tty);
process_discovered_reflist(_discoveredWeakRefs, NULL, true,
is_alive, keep_alive, complete_gc, task_executor);
}
The function process_discovered_reflist has the following signature:
void
ReferenceProcessor::process_discovered_reflist(
DiscoveredList refs_lists[],
ReferencePolicy* policy,
bool clear_referent,
BoolObjectClosure* is_alive,
OopClosure* keep_alive,
VoidClosure* complete_gc,
AbstractRefProcTaskExecutor* task_executor)
This shows that WeakRefs are being unconditionally cleared by ReferenceProcessor::process_discovered_references.
Searching the Hotspot code for process_discovered_reference shows that the CMS collector (which is what I am using) calls this method from the following call stack.
CMSCollector::refProcessingWork
CMSCollector::checkpointRootsFinalWork
CMSCollector::checkpointRootsFinal
This call stack looks like it is invoked every time a CMS collection is run.
Assuming this is true, the only explanation for a long lived weakly referenced object would be either a subtle JVM bug or if the GC had not been run.
You might want to check if you have leaked classloader issue. More on this topic you could find in this blog post
You need to clarify on what is the link between Foo and WeakReference. The case
class Wrapper<T> extends WeakReference<T> {
private final T referent;
public Wrapper(T referent) {
super(t);
this.referent = referent;
}
}
is very different from just
class Wrapper<T> extends WeakReferece<T> {
public Wrapper(T referent) {
super(t);
}
}
or its inlined version, WeakReference<Foo> wr = new WeakReference<Foo>(foo).
So I assume your case is not like I described in my first code snippet.
As you have said you are working with JNI, you might want to check if you have any unsafe finalizers. Every finalizer should have finally block calling super.finalize() and it's easy to slip.
You probably need to tell us more about the nature of your objects to offer better ideas.
Try SoftReference instead. Javadoc says: All soft references to softly-reachable objects are guaranteed to have been cleared before the virtual machine throws an OutOfMemoryError.
WeakReference doesn't have such guarantees, which makes them more suitable for caches, but sometimes SoftReferences are better.
#iirekm No: WeakReferences are 'weaker' than SoftReferences, meaning that a WeakReference will always be garbage collected before a SoftReference.
More info in this post: Understanding Java's Reference classes: SoftReference, WeakReference, and PhantomReference
Edit: (after reading comments) Yes surely Weak References are 'Weaker' than SoftReferences, typo. :S
Here's some use cases to throw further light on the subject:
SoftReference: In-memory cache (Object stays alive until VM deems that there's not enough heap mem)
WeakReference: Auto-clearing Listeners (Object should be cleared on next GC cycle after deemed being Weakly reachable)
PhantomReference: Avoiding out-of-memory errors when handling unusually large objects (When scheduled in reference queue, we know that host object is to be cleared, safe to allocate another large object). Think of it as a finalize() alternative, without the ability to bring dead objects back to life (as you potentially could with finalize)
This being said, nothing prevents the VM (please correct me if I'm wrong) to let the Weakly reachable objects stay alive as long as it is not running out of memory (as in the orig. author's case).
This is the best resource I could find on the subject: http://www.pawlan.com/monica/articles/refobjs/
Edit 2: Added "to be" in front of cleared in PhantomRef
I am not acquainted with Java, but you may be using a generational garbage collector, which will keep your Foo and FooWeakRef objects alone (not collected) as long as
they passed in an older generation
there is enough memory to allocate new objects in younger generations
Does the log that indicates that garbage collection occurred discriminates between major and minor collections?
For non-believers who claim that weak references are cleared before soft references:
import java.lang.ref.Reference;
import java.lang.ref.ReferenceQueue;
import java.lang.ref.SoftReference;
import java.lang.ref.WeakReference;
import java.util.HashMap;
import java.util.Map;
public class Test {
/**
* #param args
*/
public static void main(String[] args) {
ReferenceQueue<Object> q = new ReferenceQueue<Object>();
Map<Reference<?>, String> referenceToId = new HashMap<Reference<?>, String>();
for(int i=0; i<100; ++i) {
Object obj = new byte [10*1024*1024]; // 10M
SoftReference<Object> sr = new SoftReference<Object>(obj, q);
referenceToId.put(sr, "soft:"+i);
WeakReference<Object> wr = new WeakReference<Object>(obj, q);
referenceToId.put(wr, "weak:"+i);
for(;;){
Reference<?> ref = q.poll();
if(ref == null) {
break;
}
System.out.println("cleared reference " + referenceToId.get(ref) + ", value=" + ref.get());
}
}
}
}
If your run it with either -client or -server, you'll see that soft references are always cleared before weak references, which also agrees with Javadoc: http://download.oracle.com/javase/1.4.2/docs/api/java/lang/ref/package-summary.html#reachability
Typically soft/weak references are used in connection with Maps to make kinds of caches. If keys in your Map are compared with == operator, (or unoverriden .equals from Object), then it's best to use Map which operates on SoftReference keys (eg from Apache Commons) - when the object 'disappears' no other object will ever be equal in the '==' sense to the old one. If keys of your Map are compared with advanced .equals() operator, like String or Date, many other objects may match to the 'disappearing' one, so it's better to use the standard WeakHashMap.
Related
This may be a very naive Question?
Suppose i have Class Something like this
class SlowConstructor {
private final int a;
private final String unReachableString;
public SlowConstructor(String random) {
unReachableString = "I am not reachable will GC will collect me " + random;
Thread.sleep(1000*3600); // ignoring Exception check for readbility
a = 100;
Thread.sleep(1000*3600);
}
}
So my question is if i create Many Objects of SlowConstructor (let say 50 in diff threads) and as you can see each Constructor will take two hours to complete. The String reference in SlowConstructor unReachableString is not reachable from any code for around two hours. If GC runs during this two hours will it not collect unReachableString ref ?. I assume it will not be Garbage Collected but then why? From where unReachableString is reachable ?
The String reference in SlowConstructor unReachableString is not reachable from any code for around two hours.
Incorrect. The SlowConstructor object is immediately reachable from the thread that is in the process of constructing it. So, therefore, is the string.
So that means that the String object won't be garbage collected before the constructor completes.
(And in fact, the string object corresponds to a String literal, and is therefore also reachable from the code (any code!) that assigns or applies a method to the literal.)
The concept of reachability includes any mechanism by which any current or future execution could use the object in question. That includes cases where the object hasn't been assigned to a named variable or array element ... yet.
As other have said GC is not going to affect a half-constructed object. But why? GC necessarily proceeds from a maximal set of root pointers. Anything that can be reached from these roots is "protected" from GC. This is either my marking as in mark-and-sweep collectors or by copying to a new active generation (arena) in a copying collector. Roots consist of the runtime stack, machine (virtual or physical) registers, and global pointers. When the constructor starts running, a pointer to the newly allocated record will be created. Either it will be a root or accessible from a root. So the GC will not collect it. Since the class instance under construction is accessible from a root, so is the string you're referring to. Therefore it can't be collected either.
So long as the threads weren't interrupted, your object will (eventually) instantiate, and (eventually) contain a value for unReachableString.
Strings are interned, and would be subject to garbage collection only if nothing referred to it - kind of like how garbage collection works now. The half-constructed object does refer to the interned string, so it would not be yet eligible for garbage collection.
I'm willing to bet that having fifty or so instances of this type floating around* wouldn't make a difference either - you then have fifty or so references to this string literal, and it wouldn't be yet eligible for garbage collection until these instances were eligible for garbage collection themselves.
*: OH GOD NO PLEASE DON'T DO THIS IN ACTUAL CODE PLEASE
It will not and should not be garbage collected. Sleeping thread is still a live thread.
Reachable in GC context means the following: if we go through the Stack will we find a reference pointing to this object (memory space) on the Heap.
In you case the answer is yes.
your logic is not correct, if thread is still alive it is in scope of method SlowConstructor. So JVM thinks that unReachableString string can be used so Garbacge Collection does not touch that reference.
According to the code you can assume that unReachableString is not used so it has to be Garbage Collected but JVM does not have intelligent logic to know the next. It just look at the scope of method and object reference.
Here is the code snippet.
method(){
String s1="abc";
String s2 = new String ("abc");
s1=null;
s2=null;
--------
---------
}
At the end is s1 & s2 objects are exists? How you will make sure these objects are garbage collected ?
Objects referenced to by s1 and s2 are eligible for garbage collection once s1=null and s2=null provided that no other references to that Object exists or when the method exits, provided that the Objects were only referenced by the local variables.An object once created uses some memory and the memory remains allocated till there are references for the use of the object.When there are no references for an object, it is assumed to be no longer needed and the memory occupied by the object *can be reclaimed.*An Object becomes eligible for Garbage collection or GC if its not reachable from any live threads or any static refrences in other words you can say that an object becomes eligible for garbage collection if its all references are null.
There are methods like System.gc() and Runtime.gc() which is used to send request of Garbage collection to JVM but it’s not guaranteed that garbage collection will happen. Java programmers can not force Garbage collection in Java; it will only trigger if JVM thinks it needs a garbage collection. Forced GC is sign of bad coding.Once should in turn always look to minimize creation of unnecessary objects and references to those objects.
They get garbage Collected after they go out of scope.
Unless you're actually having serious performance issues, I'd stop worrying about it so much and let the garbage collector do it's thing.
You should be careful though, there are some kinds of elements such as file streams, open sockets, and such that are not managed like that. you have to close those.
If the question is how to make sure, the answer is fairly simple. You can never make sure that any object will be garbage collected. Read this to understand what garbage collection really is and how to reason about it.
If the question is how to hint for a collection, then set all the references of unwanted objects to null and call System.gc(), which will request (not force) a collection. Nothing is guaranteed to be released using this method, but often it's the closest thing you can get.
If you want to do this specifically for strings, because they may contain sensitive data or something along these lines, use a char[] to store that data instead of a String, because you can change the primitive values of the array at will and erase them when you're done.
Garbage collector runs periodically(time period is JVM dependent). Java maintains table of objects and its references when reference is broken (probably by assigning null to reference) then on next execution of GC (garbage collector) object's having no reference will be deleted (If something goes wrong with GC then object will not garbage collected - very very rare condition), which is totally dependent on JVM. You can send request to JVM to run GC by using following code (Processing your request is once again JVM dependent):
Runtime.getRuntime().gc();
or
System.gc();
Programmer don't have to worry about the running GC mostly JVM will handle execution of GC. There are lots of enhancements made to the garbage collectors. Java (latest version) comes with G1(Garbage First) collector which is a server-style garbage collector which runs more effectively. G1 is a great replacement for CMS (Concurrent Mark-Sweep Collector).
If you want to know more about garbage collector then you should read the pages:
[http://docs.oracle.com/javase/7/docs/technotes/guides/vm/gc-ergonomics.html][1]
[http://docs.oracle.com/javase/7/docs/technotes/guides/vm/cms-6.html][2]
[http://docs.oracle.com/javase/7/docs/technotes/guides/vm/par-compaction-6.html][3]
String s2 = new String ("abc");
Here 'abc' will be created in regular, garbage collectible heap area.
So as soon as you make S2 null, this string object is eligible for garbage collection.
This is assuming that your programm do not have any other reference to this particular string object "abc".
String s1="abc";
In this case, "abc" will be created in special area of heap called literal pool or string pool. Making "abc" null does not make "abc" eligible for garbage collection since JVM will try to reuse this "abc" in future.
Baseline in this case is, normal garbage collection rules won't apply here.
Hope this helped. :-)
I have to assume that the following method doesn't leak memory:
public final void setData(final Integer p_iData)
{
data = p_iData;
}
Where data is a property of some class.
Every time the method gets called, a new Integer is replacing the currently existing data reference. So what's happening with the current/old data?
Java has to be doing something under the hood; otherwise we'd have to null-out any objects every time an object is assigned.
Simplistic explanation:
Periodically the garbage collector looks at all the objects in the system, and sees which aren't reachable any more from live references. It frees any objects which are no longer reachable.
Note that your method does not create a new Integer object at all. A reference to the same Integer object could be passed in time and time again, for example.
The reality of garbage collection is a lot more complicated than this:
Modern GCs tend to be generational, assuming that most objects are short-lived, so it doesn't need to check the whole (possibly large) heap as often; it can just check "recent" objects for liveness frequently
Objects can have finalizers - code to be run before they're garbage collected. This delays garbage collection of such objects by a cycle, and the object could even "resurrect" itself by making itself reachable
Modern GCs can collect in parallel, and have numerous tweaking options
Java is a garbage-collected language.
Once there are no more live references to an object, it becomes eligible for garbage collection. The collector runs from time to time and will reclaim the object's memory.
In a nutshell, your code is 100% correct and is not leaking memory.
It gets garbage collected eventually.
if there is no ther reference to data, the garbage collector of java will clean the old data up and free the memory
Actually, since Integer is an object not a primitive type, the line:
data = p_iData;
is updating a reference.
Now, the old object that this.data used to point to will be examined by the GC to determine if there are no more references to that object. If not, that object is destroyed and the memory is freed (at some later time)
If the object previously referenced by data is no longer referenced by any object structure that is referenced from any running thread it is eligible for garbage collecion. GC is performed by Java in the background to free the memory of unused objects.
i want to show one example to you
in some code :
int x;
x=10;
x=20;
initially i assigned x to 10
again x to 20
first reference memory will be handled by Java GC.
Java GC is a thread tht run continuously and checked unreferenced memory and clean it .
What's the difference between java.lang.ref.WeakReference and java.lang.ref.SoftReference ?
From Understanding Weak References, by Ethan Nicholas:
Weak references
A weak reference, simply put, is a
reference that isn't strong enough to
force an object to remain in memory.
Weak references allow you to leverage
the garbage collector's ability to
determine reachability for you, so you
don't have to do it yourself. You
create a weak reference like this:
WeakReference weakWidget = new WeakReference(widget);
and then
elsewhere in the code you can use
weakWidget.get() to get the actual
Widget object. Of course the weak
reference isn't strong enough to
prevent garbage collection, so you may
find (if there are no strong
references to the widget) that
weakWidget.get() suddenly starts
returning null.
...
Soft references
A soft reference is exactly like a
weak reference, except that it is less
eager to throw away the object to
which it refers. An object which is
only weakly reachable (the strongest
references to it are WeakReferences)
will be discarded at the next garbage
collection cycle, but an object which
is softly reachable will generally
stick around for a while.
SoftReferences aren't required to
behave any differently than
WeakReferences, but in practice softly
reachable objects are generally
retained as long as memory is in
plentiful supply. This makes them an
excellent foundation for a cache, such
as the image cache described above,
since you can let the garbage
collector worry about both how
reachable the objects are (a strongly
reachable object will never be removed
from the cache) and how badly it needs
the memory they are consuming.
And Peter Kessler added in a comment:
The Sun JRE does treat SoftReferences differently from WeakReferences. We attempt to hold on to object referenced by a SoftReference if there isn't pressure on the available memory. One detail: the policy for the "-client" and "-server" JRE's are different: the -client JRE tries to keep your footprint small by preferring to clear SoftReferences rather than expand the heap, whereas the -server JRE tries to keep your performance high by preferring to expand the heap (if possible) rather than clear SoftReferences. One size does not fit all.
Weak references are collected eagerly. If GC finds that an object is
weakly reachable (reachable only through weak references), it'll clear the
weak references to that object immediately. As such, they're good for
keeping a reference to an object for which your program also keeps
(strongly referenced) "associated information" somewere, like cached
reflection information about a class, or a wrapper for an object, etc.
Anything that makes no sense to keep after the object it is associated
with is GC-ed. When the weak reference gets cleared, it gets enqueued in a
reference queue that your code polls somewhere, and it discards the
associated objects as well. That is, you keep extra information about an
object, but that information is not needed once the object it refers to
goes away. Actually, in certain situations you can even subclass
WeakReference and keep the associated extra information about the object
in the fields of the WeakReference subclass. Another typical use of
WeakReference is in conjunction with Maps for keeping canonical instances.
SoftReferences on the other hand are good for caching external, recreatable resources
as the GC typically delays clearing them. It is guaranteed though that all
SoftReferences will get cleared before OutOfMemoryError is thrown, so they
theoretically can't cause an OOME[*].
Typical use case example is keeping a parsed form of a contents from a
file. You'd implement a system where you'd load a file, parse it, and keep
a SoftReference to the root object of the parsed representation. Next time
you need the file, you'll try to retrieve it through the SoftReference. If
you can retrieve it, you spared yourself another load/parse, and if the GC
cleared it in the meantime, you reload it. That way, you utilize free
memory for performance optimization, but don't risk an OOME.
Now for the [*]. Keeping a SoftReference can't cause an OOME in itself. If
on the other hand you mistakenly use SoftReference for a task a WeakReference is meant
to be used (namely, you keep information associated with an Object somehow
strongly referenced, and discard it when the Reference object gets
cleared), you can run into OOME as your code that polls the ReferenceQueue
and discards the associated objects might happen to not run in a timely
fashion.
So, the decision depends on usage
- if you're caching information that is expensive to construct, but
nonetheless reconstructible from other data, use soft references
- if you're keeping a reference to a canonical instance of some data, or
you want to have a reference to an object without "owning" it (thus
preventing it from being GC'd), use a weak reference.
In Java; order from strongest to weakest, there are: Strong, Soft, Weak and Phantom
A Strong reference is a normal reference that protects the referred object from collection by GC. i.e. Never garbage collects.
A Soft reference is eligible for collection by garbage collector, but probably won't be collected until its memory is needed. i.e. garbage collects before OutOfMemoryError.
A Weak reference is a reference that does not protect a referenced object from collection by GC. i.e. garbage collects when no Strong or Soft refs.
A Phantom reference is a reference to an object is phantomly referenced after it has been finalized, but before its allocated memory has been reclaimed.
Source
Analogy: Assume a JVM is a kingdom, Object is a king of the kingdom, and GC is an attacker of the kingdom who tries to kill the king(object).
When King is Strong, GC can not kill him.
When King is Soft, GC attacks him but King rule the kingdom with protection until resource are available.
When King is Weak, GC attacks him but rule the kingdom without protection.
When king is Phantom, GC already killed him but king is available via his soul.
Weak Reference
http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/ref/WeakReference.html
Principle: weak reference is related to garbage collection. Normally, object having one or more reference will not be eligible for garbage collection.
The above principle is not applicable when it is weak reference. If an object has only weak reference with other objects, then its ready for garbage collection.
Let's look at the below example: We have an Map with Objects where Key is reference a object.
import java.util.HashMap;
public class Test {
public static void main(String args[]) {
HashMap<Employee, EmployeeVal> aMap = new
HashMap<Employee, EmployeeVal>();
Employee emp = new Employee("Vinoth");
EmployeeVal val = new EmployeeVal("Programmer");
aMap.put(emp, val);
emp = null;
System.gc();
System.out.println("Size of Map" + aMap.size());
}
}
Now, during the execution of the program we have made emp = null. The Map holding the key makes no sense here as it is null. In the above situation, the object is not garbage collected.
WeakHashMap
WeakHashMap is one where the entries (key-to-value mappings) will be removed when it is no longer possible to retrieve them from the Map.
Let me show the above example same with WeakHashMap
import java.util.WeakHashMap;
public class Test {
public static void main(String args[]) {
WeakHashMap<Employee, EmployeeVal> aMap =
new WeakHashMap<Employee, EmployeeVal>();
Employee emp = new Employee("Vinoth");
EmployeeVal val = new EmployeeVal("Programmer");
aMap.put(emp, val);
emp = null;
System.gc();
int count = 0;
while (0 != aMap.size()) {
++count;
System.gc();
}
System.out.println("Took " + count
+ " calls to System.gc() to result in weakHashMap size of : "
+ aMap.size());
}
}
Output: Took 20 calls to System.gc() to result in aMap size of : 0.
WeakHashMap has only weak references to the keys, not strong references like other Map classes. There are situations which you have to take care when the value or key is strongly referenced though you have used WeakHashMap. This can avoided by wrapping the object in a WeakReference.
import java.lang.ref.WeakReference;
import java.util.HashMap;
public class Test {
public static void main(String args[]) {
HashMap<Employee, EmployeeVal> map =
new HashMap<Employee, EmployeeVal>();
WeakReference<HashMap<Employee, EmployeeVal>> aMap =
new WeakReference<HashMap<Employee, EmployeeVal>>(
map);
map = null;
while (null != aMap.get()) {
aMap.get().put(new Employee("Vinoth"),
new EmployeeVal("Programmer"));
System.out.println("Size of aMap " + aMap.get().size());
System.gc();
}
System.out.println("Its garbage collected");
}
}
Soft References.
Soft Reference is slightly stronger that weak reference. Soft reference allows for garbage collection, but begs the garbage collector to clear it only if there is no other option.
The garbage collector does not aggressively collect softly reachable objects the way it does with weakly reachable ones -- instead it only collects softly reachable objects if it really "needs" the memory. Soft references are a way of saying to the garbage collector, "As long as memory isn't too tight, I'd like to keep this object around. But if memory gets really tight, go ahead and collect it and I'll deal with that." The garbage collector is required to clear all soft references before it can throw OutOfMemoryError.
The only real difference between a soft reference and a weak reference is that
the garbage collector uses algorithms to decide whether or not to
reclaim a softly reachable object, but always reclaims a weakly
reachable object.
SoftReference is designed for caches. When it is found that a WeakReference references an otherwise unreachable object, then it will get cleared immediately. SoftReference may be left as is. Typically there is some algorithm relating to the amount of free memory and the time last used to determine whether it should be cleared. The current Sun algorithm is to clear the reference if it has not been used in as many seconds as there are megabytes of memory free on the Java heap (configurable, server HotSpot checks against maximum possible heap as set by -Xmx). SoftReferences will be cleared before OutOfMemoryError is thrown, unless otherwise reachable.
This article can be super helpful to understand strong, soft, weak and phantom references.
To give you a summary,
If you only have weak references to an object (with no strong references), then the object will be reclaimed by GC in the very next GC cycle.
If you only have soft references to an object (with no strong references), then the object will be reclaimed by GC only when JVM runs out of memory.
So you can say that, strong references have ultimate power (can never be collected by GC)
Soft references are powerful than weak references (as they can escape GC cycle until JVM runs out of memory)
Weak references are even less powerful than soft references (as they cannot excape any GC cycle and will be reclaimed if object have no other strong reference).
Restaurant Analogy
Waiter - GC
You - Object in heap
Restaurant area/space - Heap space
New Customer - New object that wants table in restaurant
Now if you are a strong customer (analogous to strong reference), then even if a new customer comes in the restaurant or what so ever happnes, you will never leave your table (the memory area on heap). The waiter has no right to tell you (or even request you) to leave the restaurant.
If you are a soft customer (analogous to soft reference), then if a new customer comes in the restaurant, the waiter will not ask you to leave the table unless there is no other empty table left to accomodate the new customer. (In other words the waiter will ask you to leave the table only if a new customer steps in and there is no other table left for this new customer)
If you are a weak customer (analogous to weak reference), then waiter, at his will, can (at any point of time) ask you to leave the restaurant :P
The six types of object reachability states in Java:
Strongly reachable objects - GC will not collect (reclaim the memory occupied by) this kind of object. These are reachable via a root node or another strongly reachable object (i.e. via local variables, class variables, instance variables, etc.)
Softly reachable objects - GC may attempt to collect this kind of object depending on memory contention. These are reachable from the root via one or more soft reference objects
Weakly reachable objects - GC must collect this kind of object. These
are reachable from the root via one or more weak reference objects
Resurrect-able objects - GC is already in the process of collecting these objects. But they may go back to one of the states - Strong/Soft/Weak by the execution of some finalizer
Phantomly reachable object - GC is already in the process of collecting these objects and has determined to not be resurrect-able by any finalizer (if it declares a finalize() method itself, then its finalizer will have been run). These are reachable from the root via one or more phantom reference objects
Unreachable object - An object is neither strongly, softly, weakly, nor phantom reachable, and is not resurrectable. These objects are ready for reclamation
For more details: https://www.artima.com/insidejvm/ed2/gc16.html « collapse
The Only Real Difference
Per the doc, loose WeakReferences must be cleared by a running GC.
Per the doc, loose SoftReferences must be cleared before OOM is thrown.
That's the only real difference. Everything else is not part of the contract. (I'll assume the latest docs are contractual.)
SoftReferences are useful. Memory-sensitive caches use SoftReferences, not WeakReferences.
The only proper use of WeakReference is to observe the GC run. You do this by creating a new WeakReference whose object immediately goes out of scope, then try to get null out of weak_ref.get(). When it's null, you learn that between this duration, the GC ran.
As for incorrect use of WeakReference, the list is endless:
a lousy hack to implement priority-2 softreference such that you don't have to write one, yet it doesn't work as expected because the cache would be cleared on every GC run, even when there is spare memory. See https://stackoverflow.com/a/3243242/632951 for phails. (Besides, what if you need more than 2 levels of cache priority? You'd still gotta need a real library for it.)
a lousy hack to associate data with an object of an existing class, yet it creates a memory leak (OutOfMemoryError) when your GC decides to take a break after your weakreferences are created. Besides, it's beyond ugly: A better approach is to use tuples.
a lousy hack to associate data with an object of an existing class, where the class has the nerve to make itself non-subclassable, and is used in an existing function code which you need to call. In such a case, the proper solution is to either edit the class and make it subclassable, or edit the function and make it take an interface instead of a class, or use an alternative function.
To give an in-action memory usage aspect, I did an experiment with Strong, Soft, Weak & Phantom references under heavy load with heavy objects by retaining them till end of program. Then monitored heap usage & GC behavior. These metrics may vary case by case basis but surely gives high level understanding. Below are findings.
Heap & GC Behavior under heavy load
Strong/Hard Reference - As program continued, JVM couldn't collect retained strong referenced object. Eventually ended up in "java.lang.OutOfMemoryError: Java heap space"
Soft Reference - As program continued, heap usage kept growing, but OLD gen GC happened hen it was nearing max heap. GC started bit later in time after starting program.
Weak Reference - As program started, objects started finalizing & getting collected almost immediately. Mostly objects got collected in young generation garbage collection.
Phantom Reference - Similar to weak reference, phantom referenced objects also started getting finalized & garbage collected immediately. There were no old generation GC & all objects were getting collected in young generation garbage collection itself.
You can get more in depth graphs, stats, observations for this experiment here.
One should be aware that a weakly referenced object will only get collected when it has ONLY weak reference(s). If it has so much as one strong reference, it does not get collected no matter how many weak references it has.
WeakReference: objects that are only weakly referenced are collected at every GC cycle (minor or full).
SoftReference: when objects that are only softly referenced are collected depends on:
-XX:SoftRefLRUPolicyMSPerMB=N flag (default value is 1000, aka 1 second)
Amount of free memory in the heap.
Example:
heap has 10MB of free space (after full GC);
-XX:SoftRefLRUPolicyMSPerMB=1000
Then object which is referenced only by SoftReference will be collected if last time when it was accessed is greater then 10 seconds.
Given an aggregation of class instances which refer to each other in a complex, circular, fashion: is it possible that the garbage collector may not be able to free these objects?
I vaguely recall this being an issue in the JVM in the past, but I thought this was resolved years ago. yet, some investigation in jhat has revealed a circular reference being the reason for a memory leak that I am now faced with.
Note: I have always been under the impression that the JVM was capable of resolving circular references and freeing such "islands of garbage" from memory. However, I am posing this question just to see if anyone has found any exceptions.
Only a very naive implementation would have a problem with circular references. Wikipedia has a good article on the different GC algorithms. If you really want to learn more, try (Amazon) Garbage Collection: Algorithms for Automatic Dynamic Memory Management . Java has had a good garbage collector since 1.2 and an exceptionally good one in 1.5 and Java 6.
The hard part for improving GC is reducing pauses and overhead, not basic things like circular reference.
The garbage collector knows where the root objects are: statics, locals on the stack, etc and if the objects aren't reachable from a root then they will be reclaimed. If they are reachable, then they need to stick around.
Ryan, judging by your comment to Circular References in Java, you fell into the trap of referencing objects from a class, which was probably loaded by the bootstrap/system classloader. Every class is referenced by the classloader that loaded the class, and can thus be garbage-collected only if the classloader is no longer reachable. The catch is that the bootstrap/system classloader is never garbage collected, therefore, objects reachable from classes loaded by the system classloader cannot be garbage-collected either.
The reasoning for this behavior is explained in JLS. For example, Third Edition 12.7 http://java.sun.com/docs/books/jls/third_edition/html/execution.html#12.7.
If I remember correctly, then according to the specifications, there are only guarantees about what the JVM can't collect (anything reachable), not what it will collect.
Unless you are working with real-time JVMs, most modern garbage collectors should be able to handle complex reference structures and identify "subgraphs" that can be eliminated safely. The efficiency, latency, and likelihood of doing this improve over time as more research ideas make their way into standard (rather than research) VMs.
No, at least using Sun's official JVM, the garbage collector will be able to detect these cycles and free the memory as soon as there are no longer any references from the outside.
The Java specification says that the garbage collector can garbage collect your object
ONLY If it is not reachable from any thread.
Reachable means there is a reference, or chain of references that leads from A to B,
and can go via C,D,...Z for all it cares.
The JVM not collecting things has not been a problem for me since 2000, but your mileage may vary.
Tip: Java serialization caches objects to make object mesh transfer efficient. If you have many large, transient objects, and all your memory is getting hogged, reset your serializer to clear it's cache.
A circular reference happens when one object refers to another, and that other one refers to the first object. For example:
class A {
private B b;
public void setB(B b) {
this.b = b;
}
}
class B {
private A a;
public void setA(A a) {
this.a = a;
}
}
public class Main {
public static void main(String[] args) {
A one = new A();
B two = new B();
// Make the objects refer to each other (creates a circular reference)
one.setB(two);
two.setA(one);
// Throw away the references from the main method; the two objects are
// still referring to each other
one = null;
two = null;
}
}
Java's garbage collector is smart enough to clean up the objects if there are circular references, but there are no live threads that have any references to the objects anymore. So having a circular reference like this does not create a memory leak.
Just to amplify what has already been said:
The application I've been working on for six years recently changed from Java 1.4 to Java 1.6, and we've discovered that we've had to add static references to things that we didn't even realize were garbage collectable before. We didn't need the static reference before because the garbage collector used to suck, and it is just so much better now.
Reference counting GCs are notorious for this issue. Notably, Suns JVM doesn't use a reference counting GC.
If the object can not be reach from the root of the heap (typically, at a minimum, through the classloaders if nothing else0, then the objects will be destroyed as they are not copied during a typical Java GC to the new heap.
The garbage collector is a very sophisticated piece of software -- it has been tested in a huge JCK test-suite. It is NOT perfect BUT there is a very good chance that as long as the java compiler(javac) will compile all of your classes and JVM will instantiate it, then you should be good.
Then again, if you are holding references to the root of this object graph, the memory will NOT be freed BUT if you know what you're doing, you should be OK.