I have a problem of memory usage in my application, it start some task that full the memory. I expect that, when the task finish, the gc will clean the memory, but it never happen.
I wish to say that the code is witten very bad (like my English :( ). This makes some extraction from db and it make a massive usage of variables, store they in lists and at the end write all values to a file.
First of all I tried to write values to file frequently, but nothing change. I read that an object became eligible for the gc only if it has no reference to nothing, so, it is possible to see (maybe in debug) the tree reference of an object? In this way i can nullify the parent object at the end of the task. I know that the solution is to re-write the code, but it is very complex and we don't have much effort to do that, so we need a temporary solution.
Thank you
Bye
Related
I have some doubts about the garbage collector and how I can clear memory in Java.
I have a program that writes a binary search tree to a file and I made a function that inserts an element and another that removes an element, but in the method that removes I put the elements that I remove in a space in the file that I call "empty blocks" (which is a stack). In the C language there is a method that freed the memory that was free(), in Java there is the garbage collector that is at the discretion of Java. How can I free the memory of these blocks in the file (elements excluded).
Is there a way to free the memory of an element on file in Java (the element is of type int)?
I put the elements that I remove in a space in the file that I call “empty blocks ”(Which is a stack)
Whatever data structure you use to track your data will be in an object of some class.
When that object no longer has any references pointing to it, that object becomes a candidate for garbage collection. No need for you to do anything except not hang on to any reference longer than needed.
The garbage collector may clear the unneeded object immediately, or may clear it later. Either way, we as Java programmers do not care. Eventually the memory will be freed up.
If the reference variable pointing to an object is a local variable, that reference is dropped when the local variable goes out of scope.
If the reference variable is a member field on another object, the
object in question will be released when the other object becomes
garbage.
If the reference variable is static, you should assign null explicitly to let the referenced object become garbage. In Java, static variables stay in memory throughout the execution run of your app.
In the first two cases, you can release the object sooner by setting the reference variable to null. Generally this is not needed, but doing so may be wise if a large amount of memory is at stake. Ditto if other precious resources are being needlessly held.
Is there a way to free the memory of an element on file in Java (the element is of type int)?
Your question is really hard to understand, but I think you are asking about freeing up disk blocks in a data structure stored in a file1.
There is no Java support for this. If you write a data structure to a file, the problem of reclaiming space in the file is yours, not Java's. Indeed, I don't think that a typical OS will allow you to (literally) free disk blocks in the middle of a file2.
There may be 3rd-party libraries that support this kind of thing, but I don't have the background knowledge to make a recommendation.
If I have correctly understood what you are asking, your discussion of C's malloc / free versus Java's garbage collection is only peripherally relevant. Both of these schemes are for managing memory, not space in a random access file. Now you could conceivably implement similar schemes for managing space in a file, but you would need to take account of the different characteristics of memory and disk I/O. (Even if you are mapping the file into memory.)
1 - If you are actually talking about managing objects in heap memory in Java, your best bet is to just let the garbage collector deal with it; see Basil's answer. There are also 3rd-party libraries for storing objects in off-heap memory, but it is unclear if they would help you. I understand that such libraries typically leave it to the programmer to decide when to free an object. (They are not garbage collected.)
2 - It would be a bad idea. If the disk blocks thus freed were then used in a different file, you would get a lot of file fragmentation. That would be bad for file I/O performance.
In my application, I would like to load some amount of data into memory when first needed and keep it there in case another part of the application wants to use it. The same data would be accessed from a couple of different Activity'es, but by far not all the user could interact with. So, when not working with the relevant part of my application, I would like Android to feel free to discard the data, reloading them again at need. Note that it is unpredictable for me what the user will do, so I want Android to free the data only if hasn't been used for some time. What is a good approach to doing this?
I thought of creating a class that would be only used statically, loading the data in its static initialisation block. However, I am not sure if Dalvik would ever discard any static data stored this way. I have read something on class loaders but I have no idea what loader is used in loading my class and how it could potentially become discarded. Perhaps someone does...?
Another way I came up with is using weak reference to keep an instance of the data-holding class (non-static, obviously) but here I am afraid that the GC could decide it's useless when no Activity is currently actively operating it, even when memory is no concern at that moment. (In that case, I would like to keep the data loaded.)
The loading of my data is costly. I want, if possible, to destroy it only when the system is running out of memory or when the application exits.
It sounds like SoftReferences are what you need. These are cleared at the garbage collector's discretion when it detects that there is a memory shortfall.
If you read the class javadoc, it gives some hints about how to prevent recently used cache entries from being reclaimed.
For the record, classloaders won't help you manage instances of a class. But making the cache a static should allow the cached objects to be discarded if the cache class gets unloaded.
FOLLOWUP
My data is a solid block that would be represented by a single object.
This rather changes things. If you have a single object to cache, then LRU makes no sense. Basically it sounds like you want to hang onto the object as long as possible ... without trigger OOMEs by hanging onto it too long. This is kind of hard. Indeed, doing a perfect job is going to entail correctly predicting what the user is going to do ... which is clearly impossible.
Possibly the best strategy is to make use of the reference enqueing mechanism, and implement the queue processor to make an "intelligent" choice between letting the object die or recreating the soft link. The "intelligence" might entail looking at how much free memory there is, and / or how long it was since the object was last used. But beware!! If you get this wrong you can cause OOMEs or cause the platform to spend lots of time thrashing the garbage collector.
If I set up the cache to hold 1 object, it would be equivalent to a hard reference, wouldn't it?
Nope. If you use a SoftReference the GC will break the reference if it is running out of memory.
You can use SoftReferences. Take a look at:
http://docs.oracle.com/javase/6/docs/api/java/lang/ref/SoftReference.html
With SoftReferences you can achieve what you need:
" I want, if possible, to destroy it only when the system is running out of memory or when the application exits."
Take a look at
SoftReference gets garbage collected too early
You can also look into LruCache if your looking to cache some data in memory your app.
http://developer.android.com/reference/android/support/v4/util/LruCache.html
For a longer lived disk based cache take a look at Android Objects Cache
You can find the DiskLruCache source at https://github.com/JakeWharton/DiskLruCache/
Read in some blog that GC in Android happens on main(UI) thread, this may create sluggishness in UI screen depending on the frequency of GC execution.
Hence I was wondering will it be a good idea if I manually release objects(by assigning null value) which has no further use for me.
This way we may avoid multiple execution of GC in the application.
Please share your thoughts.
Thanks,
sku
There's no such thing as "manually releasing objects" -- at least not in any way that's meaningful to GC. An object doesn't immediately get freed/collected/whatever when you lose all references to it; it just becomes eligible for collection. GC is what actually does the releasing of the object, and it does so when it feels like doing so.
The only real way to keep the GC from working so hard is to create fewer objects, particularly temporary objects. Less garbage == less collection.
Releasing (dereferencing) objects for which you have no further use is always a good idea. You can also use SoftReference, WeakReference and/or WeakHashMap to help the GC pick up stuff that you don't mind going away if the system needs space.
There's more information about Android's GC system here.
I am working on an application which reads in a huge amount of data from a database into a Map<String,Map<String,Map<String,String>>>, processes it, and writes the processed reports to a spreadsheet using an in-house xml writer. The whole run can take about 12 hours.
I'm finding I'm getting
Exception in thread "CursorController-Thread-0" java.lang.OutOfMemoryError: Java heap space
at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:45)
at java.lang.StringBuilder.<init>(StringBuilder.java:68)
When I attempt to write this jumbo file. For this reason I think it would be best to write each Map<String,Map<String,String>> (notice that's a layer deeper)as it's finished processing.
My question is, how can I make sure that the Map<String,Map<String,String>> is not retained in memory after I write it, since the Map>> will still contain it?
Once you're done with the Map<String,Map<String,String>> mapped to by the key "key" you simply do
hugeMap.remove("key");
This will "null" out the entry in the hugeMap and make the Map<String,Map<String,String>> eligible for garbage collection (i.e., never be part of causing a heap space out of memory).
I would choose a different solution for this kind of problem. 12 hours for processing the source data is heavy.
Have you considered any scalable solutions? For e.g. Hadoop?
Use map.remove(key) method on your Map>>. You can call from time to time System.gc(); to force garbage collection.
You can keep the Map> that was written in the outer map if you want to keep your structure. but probably you should clear its contents so that it is empty. also, make sure that when you are processing it and writing it, you don't keep references to its members (mappings) anywhere before clearing the content. please see the following post for picking the approach that suits your needs best Using .clear() or letting the GC take care of it
You can't.
The garbage collector runs whenever it likes and frees whatever it likes.
That said, it is worth trying that after you delete all references to the data you no longer need, call to System.gc().
Anyway, you have written that the out of memory error is while writting the data. Maybe you have a memory leak there.
Is there a way to "clean up" objects and other variables you create? Or are they automatically disposed of or do I have this whole concept wrong? What is the proper way to go about doing this? I am trying to avoid the GC as much as possible.
The only way to cleanup in an GC language with no memory management is the GC . You can force GC but its not recommended , the GC is pretty good , to be more proactive set objects to null for the GC to clean up.
Addition:
Also try to make objects as local as possible , that way they are GCed as they scope out.
Calling System.gc() will force Garbage Collection to happen.
There is a system counting references to objects you create. If you are looping a lot and creating lots of objects you will create periods of time where they pile up. The system will collect the garbage when your processor is not doing anything, or it will wait till you need more free memory before collection occurs. If you have been processing for some time, you will experience hiccups in your performance due to Garbage Collection happening during your processes.
Please view this page and search for "Garbage Collection"
http://developer.android.com/guide/practices/design/performance.html
NOTE: Anything created with an Application Context will live until the end of the application execution. Anything created with an Activity Context will live until the end of the activity. This two situations can cause memory leaks!
For a more complete answer specific to Android:
Make sure you review the application lifecycle for android. It will help you avoid activity leaks in Android.
For the most part they are cleaned up as long as you do not maintain a reference to the object (variable). Something's like cursor's and bitmap's though need to be closed before they can be deleted to prevent memory leaks.
I don't think you have to worry about the GC as long as your object creation is not over the top. Note: GC is a part of java. You can't avoid it.
Addendum 1: If you really are that worried about it, you could reuse variables. That way you keep object creation to a minimum, but in so doing you will lose that variable and will be unable to store a wide range of data.
Android's activities have onDestroy() method. You can use this method to close open connections or dialogs or close some pending tasks.
You could also read about Java GC to get a more proper understanding of it. I would recommend SCJP book, Garbage collection chapter. It explains well when an object becomes eligible for garbage collection.