I'm creating a service that will run constantly, each day at a specified time it will run the main body of the program.
Essentially:
while(true){
run();
Thread.sleep(day);
}
After a while, I'm getting OutOfMemoryHeapExceptions.
After reading about this a little I'm thinking its because any objects created inside the run() method will never be garbage collected.
Therefore I have done something like:
public void run(){
Object a = new Object();
a.doSomething();
a= null; //Wasn't here before
}
My question is, will this solve my problem? I'm under the impression that once an object is null, the object it previously referenced will be garbage collected? Also is this a good idea? Or should I look at doing something else?
Thanks
Adding a = null will almost certainly be insufficient to fix the problem (since a is about to go out of scope anyway).
My advice would be to use a memory profiler to pinpoint what's leaking and where.
I personally use YourKit. It's very good, but costs money (you can get a free evaluation).
Another recently-released tool is Plumbr. I am yet to try it, but the blurb says:
Try out our Java agent for timely discovery of memory leaks. We'll tell you what is leaking, where the leak originates from and where the leaked objects currently reside - well before the OutOfMemoryError!
That might indeed help, in some circumstances the GC algorithm needs a little help to perform, but it doesn't guarantee to solve your problems, merely delay them.
My advice:
Simulate the same behavior with a lower time period, so you can force the error to happen.
Run it with a profiler and see where all that memory is going, and work from there.
Your impression is incorrect. Objects created inside the run() method will be garbage collected provided they 1) go out of scope, and 2)have released any native or remote system resources they are using.
What functionality are you actually performing inside your run() method call? Are you reading files, making database calls, writing to sockets? Without knowing the details its very difficult to provide a better suggestion.
No. You don't need to set the variable to null. The VM knows that you exit that scope and that the variable a no longer exists, so it automatically decrements the reference count and your object is elegible for garbage collection if it had no other references.
The error is somewhere else.
Setting references to null depends if your object is still in scope in a long time consuming process, though theoretically it will mark the reference as null you cannot guarantee when it will be garbage collected.
You need to check if your objects are being held in long scope somewhere in your code.
Found a nice explanation of setting references to null : Does setting Java objects to null do anything anymore?
In order to corner out your issue you need to profile your application.
Searching SO gave so many pointers on Garbage Collection that I have decided to just place the search string here:
https://stackoverflow.com/search?q=Java+Garbage+collection+and+setting+references+to+null
http://java.sun.com/docs/books/performance/1st_edition/html/JPAppGC.fm.html
Local variables should be collected by GC. So, you don't need to put obj=null;. Because Object is also stored in Heap area.
You should get a memory dump and analyze that using tools like JConsole JVisualVM.
The scope of the run() method is left before the Thread.sleep(day); and thus any variables inside that method are destroyed. After that a won't exist any more and thus the object referenced by that variable might be eligible for garbage collection provided there's no other reference to it.
Analyzing a memory dump should allow you to find any references to those object if they still exist.
It might as well not be those objects but others that are kept alive and which eat up the memory. That depends on what you're actually doing and might be hard to analyze here. Thus look out for huge object graphs in terms of memory usage.
For instance, we had a problem with database connections that were created frequently (XA recovery mechanism) and we thought they'd be destroyed once the method scope is left. However, the server put those connections into a static list and never cleared it and thus we ended up with no memory really soon. What helped us identify that case was analyzing a memory dump. :)
In the short term a pragmatic approach to keeping your application stable is to exit the JVM after each execution. Use a batch scheduler (e.g. cron on *nix, at on Windows) to execute your application just once every day. Any memory leaks will be cleaned up when the JVM exists for sure. However you may have to be careful you're not leaving database connections open, etc.
This will give you time to troubleshoot and fix the underlying memory leak issues while keeping your production code running and not requiring support staff to restart servers, etc.
I'm assuming you're not running out of memory on a single execution
Related
Let's consider following code:
class Table {
private static int number_of_Tables=0;
public Table(){
++number_of_Tables;
}
public void finalize(){
--number_of_Tables;
}
public static int current_TableCount(){
return number_of_Tables;
}
}
What I want to achieve is that when Garbage Collector (GC) destroys the Object that the count of available Objects gets decreased by one.
But everyone here on topic of finalize() is saying that using this method is very bad because following could happen: even though there are no references pointing to the object the GC may not destroy it immediately because GC doesn't work around the clock i.e GC will be invoked after certain amount of object are there to be destroyed i.e at certain times GC will perform the cleanup, which means that even though the object is not available anymore my counter wouldn't decrease and I would give the false information upon invoking the method curret_TableCount()
What do people do instead, to solve this kind of a problem with certainty?
There must be some kind of solution in Java?
EDIT: I need to recognize when the object is not referenced anymore i.e during runtime there exists not even one pointer(reference) to the object when this is true, i would then decrese the number of that kind of objects by one.
…following could happen: even though there are no references pointing to the object the GC may not destroy it immediately because GC doesn't work around the clock
That’s correct. The garbage collector’s purpose is to manage memory and only to manage memory. As long as there are no memory needs, the garbage collector doesn’t need to run. It’s perfectly possible that an application runs completely without any gc cycle, when there is sufficient memory.
Further, there is no guaranty that a garbage collector run identifies all unreachable objects. It might stop its work when it identified enough reclaimable memory to allow the application to proceed.
This, however, is not the only issue. Often overlooked, the fact that the garbage collector only cares for memory needs, implies that an object may get collected even when being in use, when its memory is not needed anymore, which is possible with optimized code. This is not a theoretical issue. See for example this bug or that bug related to naive dependency on finalization, even in JDK code.
Note that even if finalize() happens to get invoked at the right time, it’s invoked by an unspecified thread, which requires using thread safe constructs.
What do people do instead, to solve this kind of a problem with certainty?
People usually don’t have that kind of problem. If you truly manage a non-memory resource, you should use an explicit cleanup action, i.e. a method like dispose() or close(), to be invoked after use. The straight-forward approach is to let the class implement AutoClosable (or a subtype of it) and use the try-with-resources statement.
Cleanup actions triggered by the garbage collector are only a last resort for dealing with scenarios where the explicit cleanup has been forgotten. As explained, implementing them needs special care.
In case of a counter maintained only for statistics, you may simply live with the fact that it is imprecise. Normally, you don’t need to know how many instances of a class exist. If you really need it, e.g. when trying to debug a memory leak, you can take a heap dump, a snapshot of all existing objects, and use a dedicated analysis tool.
I have a memory leak in Java in which I have 9600 ImapClients in my heap dump and only 7800 MonitoringTasks. This is a problem since every ImapClient should be owned by a MonitoringTask, so those extra 1800 ImapClients are leaked.
One problem is I can't isolate them in the heap dump and see what's keeping them alive. So far I've only been able to pinpoint them by using external evidence to guess at which ImapClients are dangling. I'm learning OQL which I believe can solve this but it's coming slowly, and it'll take a while before I can understand how to perform something recursive like this in a new query language.
Determining a leak exists is difficult, so here is my full situation:
this process was spewing OOMEs a week ago. I thought I fixed it and I'm trying to verify whether my fixed worked without waiting another full week to see if it spews OOMEs again.
This task creates 7000-9000 ImapClients on start then under normal operation connects and disconnects very few of them.
I checked another process running older pre-OOME code, and it showed numbers of 9000/9100 instead of 7800/9600. I do not know why old code will be different from new code but this is evidence of a leak.
The point of this question is so I can determine if there is a leak. There is a business rule that every ImapClient should be a referee of a MonitoringTask. If this query I am asking about comes up empty, there is not a leak. If it comes up with objects, together with this business rule, it is not only evidence of a leak but conclusive proof of one.
Your expectations are incorrect, there is no actual evidence of any leaks occuring
The Garbage Collector's goal is to free space when it is needed and
only then, anything else is a waste of resources. There is absolutely
no benefit in attempting to keep as much free space as possible
available all the time and only down sides.
Just because something is a candidate for garbage collection doesn't
mean it will ever actually be collected, and there is no way to
force garbage collection either.
I don't see any mention of OutOfMemoryError anywhere.
What you are concerned about you can't control, not directly anyway
What you should focus on is what in in your control, which is making sure you don't hold on to references longer than you need to, and that you are not duplicating things unnecessarily. The garbage collection routines in Java are highly optimized, and if you learn how their algorithms work, you can make sure your program behaves in the optimal way for those algorithms to work.
Java Heap Memory isn't like manually managed memory in other languages, those rules don't apply
What are considered memory leaks in other languages aren't the same thing/root cause as in Java with its garbage collection system.
Most likely in Java memory isn't consumed by one single uber-object that is leaking ( dangling reference in other environments ).
Intermediate objects may be held around longer than expected by the garbage collector because of the scope they are in and lots of other things that can vary at run time.
EXAMPLE: the garbage collector may decide that there are candidates, but because it considers that there is plenty of memory still to be had that it might be too expensive time wise to flush them out at that point in time, and it will wait until memory pressure gets higher.
The garbage collector is really good now, but it isn't magic, if you are doing degenerate things, it will cause it to not work optimally. There is lots of documentation on the internet about the garbage collector settings for all the versions of the JVMs.
These un-referenced objects may just have not reached the time that the garbage collector thinks it needs them to for them to be expunged from memory, or there could be references to them held by some other object ( List ) for example that you don't realize still points to that object. This is what is most commonly referred to as a leak in Java, which is a reference leak more specifically.
I don't see any mention of OutOfMemoryError
You probably don't have a problem in your code, the garbage collection system just might not be getting put under enough pressure to kick in and deallocate objects that you think it should be cleaning up. What you think is a problem probably isn't, not unless your program is crashing with OutOfMemoryError. This isn't C, C++, Objective-C, or any other manual memory management language / runtime. You don't get to decide what is in memory or not at the detail level you are expecting you should be able to.
Check your code for finalizers, especially anything relating to IMapclient.
It could be that your MonitoringTasks are being easily collected whereas your IMapclient's are finalized, and therefore stay on the heap (though dead) until the finalizer thread runs.
The obvious answer is to add a WeakHashMap<X, Object> (and Y) to your code -- one tracking all instances of X and another tracking all instances of Y (make them static members of the class and insert every object into the map in the constructor with a null 'value'). Then you can at any time iterate over these maps to find all live instances of X and Y and see which Xs are not referenced by Ys. You might want to trigger a full GC first, to ignore objects that are dead and not yet collected.
In my application, I would like to load some amount of data into memory when first needed and keep it there in case another part of the application wants to use it. The same data would be accessed from a couple of different Activity'es, but by far not all the user could interact with. So, when not working with the relevant part of my application, I would like Android to feel free to discard the data, reloading them again at need. Note that it is unpredictable for me what the user will do, so I want Android to free the data only if hasn't been used for some time. What is a good approach to doing this?
I thought of creating a class that would be only used statically, loading the data in its static initialisation block. However, I am not sure if Dalvik would ever discard any static data stored this way. I have read something on class loaders but I have no idea what loader is used in loading my class and how it could potentially become discarded. Perhaps someone does...?
Another way I came up with is using weak reference to keep an instance of the data-holding class (non-static, obviously) but here I am afraid that the GC could decide it's useless when no Activity is currently actively operating it, even when memory is no concern at that moment. (In that case, I would like to keep the data loaded.)
The loading of my data is costly. I want, if possible, to destroy it only when the system is running out of memory or when the application exits.
It sounds like SoftReferences are what you need. These are cleared at the garbage collector's discretion when it detects that there is a memory shortfall.
If you read the class javadoc, it gives some hints about how to prevent recently used cache entries from being reclaimed.
For the record, classloaders won't help you manage instances of a class. But making the cache a static should allow the cached objects to be discarded if the cache class gets unloaded.
FOLLOWUP
My data is a solid block that would be represented by a single object.
This rather changes things. If you have a single object to cache, then LRU makes no sense. Basically it sounds like you want to hang onto the object as long as possible ... without trigger OOMEs by hanging onto it too long. This is kind of hard. Indeed, doing a perfect job is going to entail correctly predicting what the user is going to do ... which is clearly impossible.
Possibly the best strategy is to make use of the reference enqueing mechanism, and implement the queue processor to make an "intelligent" choice between letting the object die or recreating the soft link. The "intelligence" might entail looking at how much free memory there is, and / or how long it was since the object was last used. But beware!! If you get this wrong you can cause OOMEs or cause the platform to spend lots of time thrashing the garbage collector.
If I set up the cache to hold 1 object, it would be equivalent to a hard reference, wouldn't it?
Nope. If you use a SoftReference the GC will break the reference if it is running out of memory.
You can use SoftReferences. Take a look at:
http://docs.oracle.com/javase/6/docs/api/java/lang/ref/SoftReference.html
With SoftReferences you can achieve what you need:
" I want, if possible, to destroy it only when the system is running out of memory or when the application exits."
Take a look at
SoftReference gets garbage collected too early
You can also look into LruCache if your looking to cache some data in memory your app.
http://developer.android.com/reference/android/support/v4/util/LruCache.html
For a longer lived disk based cache take a look at Android Objects Cache
You can find the DiskLruCache source at https://github.com/JakeWharton/DiskLruCache/
Is there a way to "clean up" objects and other variables you create? Or are they automatically disposed of or do I have this whole concept wrong? What is the proper way to go about doing this? I am trying to avoid the GC as much as possible.
The only way to cleanup in an GC language with no memory management is the GC . You can force GC but its not recommended , the GC is pretty good , to be more proactive set objects to null for the GC to clean up.
Addition:
Also try to make objects as local as possible , that way they are GCed as they scope out.
Calling System.gc() will force Garbage Collection to happen.
There is a system counting references to objects you create. If you are looping a lot and creating lots of objects you will create periods of time where they pile up. The system will collect the garbage when your processor is not doing anything, or it will wait till you need more free memory before collection occurs. If you have been processing for some time, you will experience hiccups in your performance due to Garbage Collection happening during your processes.
Please view this page and search for "Garbage Collection"
http://developer.android.com/guide/practices/design/performance.html
NOTE: Anything created with an Application Context will live until the end of the application execution. Anything created with an Activity Context will live until the end of the activity. This two situations can cause memory leaks!
For a more complete answer specific to Android:
Make sure you review the application lifecycle for android. It will help you avoid activity leaks in Android.
For the most part they are cleaned up as long as you do not maintain a reference to the object (variable). Something's like cursor's and bitmap's though need to be closed before they can be deleted to prevent memory leaks.
I don't think you have to worry about the GC as long as your object creation is not over the top. Note: GC is a part of java. You can't avoid it.
Addendum 1: If you really are that worried about it, you could reuse variables. That way you keep object creation to a minimum, but in so doing you will lose that variable and will be unable to store a wide range of data.
Android's activities have onDestroy() method. You can use this method to close open connections or dialogs or close some pending tasks.
You could also read about Java GC to get a more proper understanding of it. I would recommend SCJP book, Garbage collection chapter. It explains well when an object becomes eligible for garbage collection.
I have a Swing browser application with a bug that as I add/remove to/from the GUI the memory isn't released for those objects and I'm trying to track down what is holding onto them. Problem is I don't know how to tell when something has actually be fully released from memory.
Is there a way to tell if an object has been released from memory? I'm used to Objective-C where there are several ways to tell.
Thanks
You can't really do it in Java. All the answers mentioning finalizers are really not what you're after.
The best you can do is enqueue a PhantomReference in a ReferenceQueue and poll until you get the reference.
final ReferenceQueue rq = new ReferenceQueue();
final PhantomReference phantom = new PhantomReference( referenceToObjectYouWantToTrack, rq );
You want to read Peter Kofler's answer here (it explains what a PhantomReference is):
Have you ever used Phantom reference in any project?
Very interesting read here:
http://www.kdgregory.com/index.php?page=java.refobj
Basically, I'm using a PhantomReference in a project where a very special kind of cache needs to be computed once, when the software is installed. To efficiently compute this (disk-based) cache, a gigantic amount of memory is needed (the more the better). I'm using a PhantomReference to track "nearly exactly" when that gigantic amount of memory is released.
There are multiple ways of detecting memory leaks. Here the three I'm currently thinking of:
Attaching a Profiler to your application (Netbeans or Eclipse TPTP should be useful here)
Making a heap dump and analyze (with Eclipse Memory Analyzer) what instances of a class are held in memory by which other instances.
Attaching VisualVM to track Garbage Collection status.
Edit:
As NoozNooz42 has pointed out, a PhantomReference can do everything a finalizer can, without the problems finalizers present. So, I encourage using PhantomReferences over extending finalize(). I am keeping my original post in tact, since I think Java programmers should at least know that finalize() exists.
Original Post:
Every Java class has a finalize() method that gets run when no other Objects hold a reference to that class. You can extend this method like so:
protected void finalize() throws Throwable {
try {
// Do whatever you want
} finally {
super.finalize();
}
}
By doing so, you can figure out if anything holds a reference to the Objects in question. Just make sure you always call super.finalize() if you use this approach.
Reference on finalization:
http://java.sun.com/developer/technicalArticles/javase/finalization/
Try YourKit. It's a Java profiler which can show you your memory usage and what the heck is going on. IIRC, the free trial can integrate with Eclipse and has all of the paid version's features (it just has a time limit).