I'm about to start using a 3rd party closed-source library to load a bunch of data and wanted to check how fast it is and how much memory it requires to load one 'set' of data. I wrote this simple harness to invoke a data load to time it and used YourKit to have a quick look at memory usage and delve in to the CPU time.
I'm running it on Windows 7, using Eclipse on JDK8 with no VM args.
public static void main(String[] args){
long start = System.currentTimeMillis();
// There are a few more calls involved, but not much
BlackBoxDataProvider bd = new BlackBoxDataProvider("c:\\thedata");
BlackBoxData = bd.loadTheData();
System.out.println(System.currentTimeMillis() - start + "ms");
// Keep the application alive so I can have a quick look at memory usage
while(true) {
Thread.sleep(1000);
}
}
Here's the YourKit snapshot of memory after the load is complete:
I then used YourKit to "Force" Garbage Collection and this happened:
Obviously it's not a real life scenario because I'm stuck inside the main method, on the main thread, so some of my references won't be cleaned up, but I can't figure out why the memory allocation would keep increasing.
Every time I click 'Force System GC', the allocation increases. I got up to 11.9GB before it stopped increasing.
Why is this happening?
The System.gc() will return when all objects have been scanned once. If objects implementing the finalize() method are added to a queue, to be cleaned up later. This means those objects cannot be cleaned up yet (not the queue nodes which hold them) i.e. the act of triggering a GC can increase memory consumption temporarily. This is what might be happening in your case.
In short, not all objects can be cleaned up in one cycle.
Related
This question already has answers here:
How to make a long time Full GC in Java manually
(2 answers)
Closed 1 year ago.
How do I drive the garbage collection activity to some significant level, say, 10% or more, preferrably without running into an out-of-memory condition?
I have been trying to build code that does this, but I'm not getting anywhere near 10%.
What approaches are there?
I tried a pool of randomly-sized blocks which are being replaced in random order, with newly created randomly-sized-again blocks; this is giving me ca. 20% CPU and 0.6%GC in VisualVM, slightly varying with pool and block sizes.
You might want to take a look here to get few ideas.
Basically the technique used in above example is to create fragmentation of Java heap memory as objects are added and removed from the LinkedHashMap being used
as a cache.
Running on my local with 300m max memory to JVM (java -Xmx300m -jar gcstress.jar) I was able to generate 20% consistent CPU usage for garbage collection.
You can do a humongous allocation (assuming G1GC with defaults):
public class Del {
public static void main(String[] args) {
for(int i=0;i<100_000;++i) {
System.out.println(allocate());
}
}
private static int allocate() {
int [] x = ThreadLocalRandom.current().ints(1024 * 1024, 10, 10_000_000).toArray();
return Arrays.hashCode(x);
}
}
You can constrain the heap and also enable GC logs to see how bad is G1 trying to cope with the constant allocations:
java -Xmx100m -Xms100m "-Xlog:gc*=info" Del.java
Running this on my machine shows that the CPU is occupied, constantly, from that java process, because of constant GC activity.
One way to cause the GC to spend a lot of time is to almost fill up the heap and then trigger repeated garbage collections by allocating and discarding1 lots of temporary objects.
A typical generational GC spends most of its time tracing and moving non-garbage objects from one space to another. When the heap is nearly full of non-garbage objects, and you trigger the GC repeatedly, it does a lot of work for very little gain (in terms of space reclaimed).
Another way (assuming that explicit GC has not been disabled) is to repeatedly call System.gc().
1 - That is, not keeping a reference to the object so that it is almost immediately unreachable.
[ONLY for debugging] Reduce the -XX:NewSize JVM parameter to a smaller size to trigger GC. This is for older GCs.
You can call System.gc() in program. Read here: Why it is bad to call System.gc()
I have an JavaFX application, which includes a background thread used for data processing and show the result in the user interface.
I created the following code for data processing:
public static void runningThread(){
long startTime = java.lang.System.nanoTime();
WSN wsn = new WSN(100, 100, 30, 60, 200);
wsn.initializeNodePosition();
wsn.alphaNodesDead = wsn.nodeNumber/2;
BaseStation BS = new BaseStation(); //the BS is created
BS.x = 125;
BS.y = 50;
BS.maxRadius = 65;
BS.energyModel = new NOEnergyModel();
wsn.BS = BS;
BS.wsn = wsn;
Thread queryThread = new Thread() {
public void run() {
System.out.println("Start");
for(int m=0; m<1000; m++){
System.out.println(m);
wsn.protocol = new HEED(wsn);
wsn.generateHomogeneousWSN(HEEDNODE.class, new MITModel(), new SimpleAggregation());
wsn.protocol.setRadiusAndNeighbors();
boolean running = true;
while(running){
wsn.roundPerformed++;
wsn.protocol.election_cluster_formation(); //cluster formation
wsn.defineStandardCHRouting(); //defines the routing at CH level
wsn.protocol.runRound();
System.out.println(wsn.roundPerformed);
if(wsn.deadNodeList.size() >= wsn.alphaNodesDead){
long stopTime = java.lang.System.nanoTime();
System.out.println("end: " + (stopTime-startTime) / 1000000000.0);
running = false;
}
}
}
}
};
queryThread.start();
}
The problem is after I run the application and click "Start" button to run the "runningThread()" function, the consuming of memory and CPU are higher and higher, when it reaches more than 2GB of memory and 90% of CPU, the "for(int m=0; m<1000; m++)" loop become very slow. I am clearing all the object before each single loop starts.
Does JVM will reclaims memory automatically for reuse once any of the object lose all the reference to it ?
The memoryleak can be anywhere in your code. Use VisualVM profiler or a build-in profiler in your IDE to find it.
The symptoms you describe are strongly suggestive of a memory leak:
Ever increasing memory utilization
The application gets slower and slower because the GC takes longer and runs more and more often.
Eventually, you get an OutOfMemoryError and the application crashes.
A memory leak typically happens when an application creates more and more objects which cannot be garbage collected because they are still reachable.
The general solution to this is to find the cause of the memory leak (which is typically a bug) and fix it. There are lots of tools for finding memory leaks in Java programs, and StackOverflows on how to do it. on how to do it.
Forcing the GC to run won't help. The GC is already running ... and is not able to reclaim objects because they are still reachable.
Increasing the heap size probably won't help either. It just puts off the inevitable slowdown to later in your application's run.
Does JVM will reclaims memory automatically for reuse once any of the object lose all the reference to it ?
Yes.
And you DON'T need to force the GC to run.
The fact that objects are not being reclaimed implies that they are still reachable. In other words, your code that is supposed to ensure that objects become unreachable is not working.
It is 'undefined' as to when the JVM garbage collection kicks in.
I think your major problem is that your inner while loop is continually testing without a pause - try adding Thread.sleep(100) inside the while loop.
The JVM will reclaim memory for unused objects, but it won't do it until the GC process runs. I would take a guess that runRound is allocating a lot of objects internally. Chances are the GC won't have run by the next iteration of your loop, so new memory is allocated, then again, and again, etc.
Eventually it will hit whatever the JVM's ceiling is (-Xmx parameter) and then the GC will start to kick in a lot more aggressively to free up unused objects, but you don't really have a lot of control over when it happens.
Using a tool such as VisualVM will help you identify what your problem is. It will show you the difference between bytes allocated, and bytes actually used. You can also see when the GC process occurs. VisualVM is included with the JDK.
The other alternative is that runRound is allocating objects that it's keeping some sort of global reference to. Again, careful use of VisualVM should let you identify this.
I am trying to do some memory analysis using visualvm.
I have written a basic code that runs an infinite loop to add objects to List.
package home.always.learning.java;
import java.util.ArrayList;
import java.util.List;
public class Heaper {
private static List<PersonDetails> listObj = new ArrayList<PersonDetails>();
private static final String nameConst = "Tarun Trehan";
public static void main(String[] args)throws Exception{
personListCreation();
}
public static void personListCreation() throws Exception
{
int i = 0;
while(true)
{
System.out.println("Looping to create person list...");
i++;
listObj.add(new PersonDetails(nameConst+i));
//Thread.sleep(1000L);
}
}
}
As expected, memory shoots up and is visible in visualvm 1st snapshot attached i.e. Memory_Shoots.JPG.
However, if i remove the following comment from the code and allow the program to sleep for 1 second; the result is different:
Thread.sleep(1000L);
The memory increases but then stabilizes and it goes on. Please refer 2nd snapshot attached i.e. Memory_Stabilizes.JPG
I am unable to understand this behavior?
Can you please provide your inputs.
Looking at a slightly modified version of the code (fixed number of iterations, added a 5s pause after start to allow my IDE to connect to visualvm, removed the system.out.print for speed, etc) it seems your culprit is garbage collection:
With 100L sleep: (ran for 5:18)
With 10L sleep: (ran for 5:01)
With 2L sleep: (ran for 4:57)
With 1L sleep: (ran for 6:36)
With 0L sleep: (ran for 0:23)
So basically what we find is that the used heap will rise slowly until the Eden space is filled (causing an allocation failure), moving the older members of eden (almost all of them, looking at the space usage) to survivor 0/1, and some of those to the old gen. The difference between sleep and no sleep may very well be a difference between the relative frequencies of minor collections and major collections.
There are a couple things that are worth thinking about:
How big are these PersonDetail objects?
How much memory is required to create a PersonDetail object, that isn't part of the object?
This second value can be a lot of thing. The array list will produce some garbage every so often. The process of creating the string "nameConst+i" will result in some garbage objects.
Suppose the answer is that a PersonDetail is small but it takes a medium amount of memory to make one. Then the JVM is going to have a bunch of trash to throw away (garbage collect) for every PersonDetail you make. As you create PersonDetails the garbage will pile up and eventually the JVM will collect it. The garbage collection step will find a lot of free memory since most of the allocations are short lived objects. This will could result in a graph like the one in your second picture, a saw tooth, where the memory is used then lots of garbage is collected.
Now imagine that you create a bunch of objects quickly (without the sleep statement). Now the total memory used is going to go up quickly both through garbage and through the objects you are holding in the list. As you get to the max memory the garbage collections will have to happen more frequently. Kind of like the top graph.
I am not sure that is what is happening, but one thing you can do to look at it is to look at the sampling in VisualVM and see how many objects are created by class, and more importantly how much memory they take up. Compare the amount used by the PersonDetails to the other stuff that is just garbage to be cleaned when needed.
I've a very simple class which has one integer variable. I just print the value of variable 'i' to the screen and increment it, and make the thread sleep for 1 second. When I run a profiler against this method, the memory usage increases slowly even though I'm not creating any new variables. After executing this code for around 16 hours, I see that the memory usage had increased to 4 MB (initially 1 MB when I started the program). I'm a novice in Java. Could any one please help explain where am I going wrong, or why the memory usage is gradually increasing even when there are no new variables created? Thanks in advance.
I'm using netbeans 7.1 and its profiler to view the memory usage.
public static void main(String[] args)
{
try
{
int i = 1;
while(true)
{
System.out.println(i);
i++;
Thread.sleep(1000);
}
}
catch(InterruptedException ex)
{
System.out.print(ex.toString());
}
}
Initial memory usage when the program started : 1569852 Bytes.
Memory usage after executing the loop for 16 hours : 4095829 Bytes
It is not necessarily a memory leak. When the GC runs, the objects that are allocated (I presume) in the System.out.println(i); statement will be collected. A memory leak in Java is when memory fills up with useless objects that can't be reclaimed by the GC.
The println(i) is using Integer.toString(int) to convert the int to a String, and that is allocating a new String each time. That is not a leak, because the String will become unreachable and a candidate for GC'ing once it has been copied to the output buffer.
Other possible sources of memory allocation:
Thread.sleep could be allocating objects under the covers.
Some private JVM thread could be causing this.
The "java agent" code that the profiler is using to monitor the JVM state could be causing this. It has to assemble and send data over a socket to the profiler application, and that could well involve allocating Java objects. It may also be accumulating stuff in the JVM's heap or non-heap memory.
But it doesn't really matter so long as the space can be reclaimed if / when the GC runs. If it can't, then you may have found a JVM bug or a bug in the profiler that you are using. (Try replacing the loop with one very long sleep and see if the "leak" is still there.) And it probably doesn't matter if this is a slow leak caused by profiling ... because you don't normally run production code with profiling enabled for that long.
Note: calling System.gc() is not guaranteed to cause the GC to run. Read the javadoc.
I don't see any memory leak in this code. You should see how Garbage collector in Java works and at its strategies. Very basically speaking GC won't clean up until it is needed - as indicated in particular strategy.
You can also try to call System.gc().
The objects are created probably in the two Java Core functions.
It's due to the text displayed in the console, and the size of the integer (a little bit).
Java print functions use 8-bit ASCII, therefor 56000 prints of a number, at 8 bytes each char will soon rack up memory.
Follow this tutorial to find your memory leak: Analyzing Memory Leak in Java Applications using VisualVM. You have to make a snapshot of your application at the start and another one after some time. With VisualVM you can do this and compare these to snapshots.
Try setting the JVM upper memory limit so low that the possible leak will cause it to run out of memory.
If the used memory hits that limit and continues to work away happily then garbage collection is doing its job.
If instead it bombs, then you have a real problem...
This does not seem to be leak as the graphs of the profiler also tell. The graph drops sharply after certain intervals i.e. when GC is performed. It would have been a leak had the graph kept climbing steadily. The heap space remaining after that must be used by the thread.sleep() and also (as mentioned in one of answers above) from the some code of the profiler.
You can try running VisualVM located at %JAVA_HOME%/bin and analyzing your application therein. It also gives you the option of performing GC at will and many more options.
I noted that the more features of VisualVM I used more memory was being consumed (upto 10MB). So this increase, it has to be from your profiler as well but it still is not a leak as space is reclaimed on GC.
Does this occur without the printlns? In other words, perhaps keeping the printlns displayed on the console is what is consuming the memory.
I have this class and I'm testing insertions with different data distributions. I'm doing this in my code:
...
AVLTree tree = new AVLTree();
//insert the data from the first distribution
//get results
...
tree = new AVLTree();
//inser the data from the next distribution
//get results
...
I'm doing this for 3 distributions. Each one should be tested an average of 14 times, and the 2 lowest/highest values removed from to compute the average. This should be done 2000 times, each time for 1000 elements. In other words, it goes 1000, 2000, 3000, ..., 2000000.
The problem is, I can only get as far as 100000. When I tried 200000, I ran out of heap space. I increased the available heap space with -Xmx in the command line to 1024m and it didn't even complete the tests with 200000. I tried 2048m and again, it wouldn't work.
What I'm thinking is that the garbage collector isn't getting rid of the old trees once I do tree = new AVL Tree(). But why? I thought that the elements from the old trees would no longer be accessible and their memory would be cleaned up.
The garbage collector should have no trouble cleaning up your old tree objects, so I can only assume there's some other allocation that you're doing that's not being cleaned up.
Java has a good tool to watch the GC in progress (or not in your case), JVisualVM, which comes with the JDK.
Just run that and it will show you which objects are taking up the heap, and you can both trigger and see the progress of GC's. Then you can target those for pools so they can be re-used by you, saving the GC the work.
Also look into this option, which will probably stop the error you're getting that stops the program, and you program will finish, but it may take a long time because your app will fill up the heap then run very slowly.
-XX:-UseGCOverheadLimit
Which JVM you are using and what JVM parameters you have used to configure GC?
Your explaination shows there is a memory leak in your code. If you have any tool like jprofiler then use it to find out where is the memory leak.
There's no reason those trees shouldn't be collected, although I'd expect that before you ran out of memory you should see long pauses as the system ran a full GC. As it's been noted here that that's not what you're seeing, you could try running with flags like -XX:-PrintGC, -XX:-PrintGCDetails,-XX:-PrintGCTimeStamps to give you some more information on exactly what's going on, along with perhaps some sort of running count of roughly where you are. You could also explicitly tell the garbage collector to use a different garbage-collection algorithm.
However, it still seems unlikely to me. What other code is running? is it possible there's something in the AVLTree class itself that's keeping its instances from being GC'd? What about manually logging the finalize() on that class to insure that (some of them, at least) are collectible (e.g. make a few and manually call System.gc())?
GC params here, a nice ref on garbage collection from sun here that's well worth reading.
The Java garbage collector isn't guaranteed to garbage collect after each object's refcount becomes zero. So if you're writing code that is only creating and deleting a lot of objects, it's possible to expend all of the heap space before the gc has a chance to run. Alternatively, Pax's suggestion that there is a memory leak in your code is also a strong possibility.
If you are only doing benchmarking, then you may want to use the java gc function (in the System class I think) between tests, or even re-run you program for each distribution.
We noticed this in a server product. When making a lot of tiny objects that quickly get thrown away, the garbage collector can't keep up. The problem is more pronounced when the tiny objects have pointers to larger objects (e.g. an object that points to a large char[]). The GC doesn't seem to realize that if it frees up the tiny object, it can then free the larger object. Even when calling System.gc() directly, this was still a huge problem (both in 1.5 and 1.6 VMs)!
What we ended up doing and what I recommend to you is to maintain a pool of objects. When your object is no longer needed, throw it into the pool. When you need a new object, grab one from the pool or allocate a new one if the pool is empty. This will also save a small amount of time over pure allocation because Java doesn't have to clear (bzero) the object.
If you're worried about the pool getting too large (and thus wasting memory), you can either remove an arbitrary number of objects from the pool on a regular basis, or use weak references (for example, using java.util.WeakHashMap). One of the advantages of using a pool is that you can track the allocation frequency and totals, and you can adjust things accordingly.
We're using pools of char[] and byte[], and we maintain separate "bins" of sizes in the pool (for example, we always allocate arrays of size that are powers of two). Our product does a lot of string building, and using pools showed significant performance improvements.
Note: In general, the GC does a fine job. We just noticed that with small objects that point to larger structures, the GC doesn't seem to clean up the objects fast enough especially when the VM is under CPU load. Also, System.gc() is just a hint to help schedule the finalizer thread to do more work. Calling it too frequently causes a significant performance hit.
Given that you're just doing this for testing purposes, it might just be good housekeeping to invoke the garbage collector directly using System.gc() (thus forcing it to make a pass). It won't help you if there is a memory leak, but if there isn't, it might buy you back enough memory to get through your test.