I have a loop that spawns a lot of threads. These threads contains, among other things, 2 (massive) StringBuilder objects. These threads then run and do their thing.
However, I noticed that after a certain amount of threads, I get strange crashes. I know this is because of these StringBuilder, because when I reduce their initial capacity, I can start a lot more threads. Now for these StringBuilders, they are create like this in the constructor of the thread object:
StringBuilder a = new StringBuilder(30000);
StringBuilder b = new StringBuilder(30000);
The point where it generally crashes is around 550 threads, which results in a little bit more than 62MB. Combined with the rest of the program the memory in use is most likely 64MB, which I read online somewhere was the defaulf size of the JVM memory allocation pool. I don't know whether this is true or not.
Now, is there something I am doing wrong, that somehow because of the design, I am allocating memory the wrong way? Or is this the only way and should I tell the JVM to increase its memory pool? Or something else entirely?
Also, please do not tell me to set a lower capacity, I know these StringBuilders automatically increase their capacity when needed, but I would like to have a solution to this problem.
Use the -Xmx JVM option to increase the Java maximum heap size.
Before J2SE 5.0, the default maximum heap size was 64MB. You can override this default using the -Xmx command-line option.
If you are storing a lot of information in a StringBuilder, are you going to reference back to it at some point? If not then just write it to another medium (DB, File etc). The program has a finite amount of resources, it can't hold all of the state of an entire system at once. -Xmx will give you more space for storage in the memory, however it won't make your storage ability infinite.
Consider using a ThreadPoolExecutor, and set the pool size to the number of CPUs on your machine. Creating more threads than CPUs is just adding overhead.
ExecutorService service = Executors.newFixedThreadPool(cpuCount))
Also, you can reduce memory usage by writing your strings to files instead of keeping them in-memory with StringBuilders.
Assume 1MB per thread. That's the RAM cost of creating each one, over and above the memory allocated by its process.
As Gregory said, give the jvm, some options like -Xmx.
Also consider using a ThreadPool or Executor to ensure that only a given amount of threads are running simultaneously. That way the amount of memory can be kept limited without slowdown (as your processor is not capable of running 550 threads at the same time anyway).
And when you're using an Executor don't create the StringBuilders in the constructor, but in the run method.
You can use a FileWriter to output text to a file, then pull it back in with a FileReader. That way you'll only need to store the filename in memory, rather then the entire contents of the string.
To cut down on threads you can use an ExecutorService, or simply use a few threads that read out of a queue.
My guess is that with a little tinkering you can probably get your program down to not needing much memory at all.
Related
I implemented a heuristic in Java that solves an optimization problem for a given input. The heuristic can run for thousands of iterations and create lots of objects of varying complexity.
In order to test it, I have thousands of test inputs. My main method takes all inputs and sequentially starts the heuristic for each input in a loop. The results are stored in a separate file for each input.
When I run the program, it always stops after producing 218 or 219 and throws an "OutOfMemoryError". Once it says Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded and once Exception in thread "main" java.lang.OutOfMemoryError: Java heap space.
My guess is, the program creates too many objects over time until it runs out of memory when computing the 218th or 219th input. Every instance is computed in an independent run. Hence, it should solve the problem to clear the memory and getting rid of all created objects after the result for an input is stored and before the next input is parsed. Is that correct? I heard using System.gc() is bad practice, but what else would you recommend in my case?
Edit:
To specify what I want: Instead of pressing "start" for each input, I implemented the loop to do that for me. However, it seems like it doesn't behave the same way and it keeps old objects from previous runs. Can I change my java code in such a way that it behaves similar to starting the program anew for each input? Or do I have to use a shell skript that starts my heuristic for each input separatly to make it work?
I have never used any JVM parameters and it seems to me like they don't really tackle the problem.
Resolved: There was in fact a memory leak that I discovered and fixed. No System.gc() needed. Thanks for helping anyways!
Yes leave GC handling with JVM. You need to follow some of the steps mentioned below in order:
Increase your heap size using Xmx... parameter
Set proper GC algorithm and parameters. If you have already have GC parameters try to tune the parameters
Try using -XX:+HeapDumpOnOutOfMemoryError and -XX:HeapDumpPath=<path for heap dump> option when you start your JVM, so you get heap dump when your jvm runs OOM. By using the heap dump, you could use profilers like jprofiler/yourkit/jvisualvm etc to investigate memory leaks and then rectify the same.
First, when you start a JVM to run your tests, disable the GC overhead limit:
-XX:-UseGCOverheadLimit
I recommend this because you already know you're purposefully stressing the garbage collector, and you don't want it to warn you about GC overhead.
Second, take a look at how you can break up your tests better, in such a way that you're allowing objects from the previous test to be garbage collected. Don't keep active pointers to large structures of objects after each test completes.
Third, if you still need more memory due to exceeding Java heap space, use:
-Xms<size> set initial Java heap size
-Xmx<size> set maximum Java heap size
If you know you'll be using the memory anyhow, it works best to set both of these to the same value, which prevents thrashing during execution.
Don't bother explicitly calling System.gc(), it's ultimately pointless because garbage collection is always going to happen when it's necessary.
Fourth, another JVM setting which could be useful in your circumstances:
-XX:NewRatio=<n> Ratio of old/new generation sizes. The default value is 2.
It's normally not recommended to set this lower than 2 (2/3 old, 1/3 new), but in your situation I might suggest you try setting this to 1 (1/2 old, 1/2 new).
See also GC overhead limit exceeded and check out Java HotSpot VM Options.
Give this a try:
http://javaandroidandrest.blogspot.de/2012/06/wait-for-jvm-garbage-collector.html
From the site:
Using functions like System.gc(); or Runtime.getRuntime().gc(); only suggest to the JVM that you want to run the garbage collector.
I found a way on the internet not to force the grabage collector but to wait until the garbage collector runs.
I have an application that produces large results objects and puts them in a queue. Multiple worker threads create the results objects and queue them, and a single writer thread de-queues the objects, converts them to CSV, and writes them to disk. Due to both I/O and the size of the results objects, writing the results takes far longer than generating them. This application is not a server, it is simply a command-line app that runs through a large batch of requests and finishes.
I would like to decrease the overall memory footprint of the application. Using a heap analysis tool (IBM HeapAnalyzer), I am finding that just before the program terminates, most of the large results objects are still on the heap, even though they were de-queued and have no other references to them. That is, they are all root objects. They take up the majority of the heap space.
To me, this means that they made it into tenured heap space while they were still in the queue. As no full GC is ever triggered during the run, that is where they remain. I realize that they should be tenured, otherwise I'd be copying them back and forth within the Eden spaces while they are still in the queue, but at the same time I wish there was something I could do to facilitate getting rid of them after de-queueing, short of calling System.gc().
I realize one way of getting rid of them would be to simply shrink the maximum heap size and trigger a full GC. However the inputs to this program vary considerably in size and I would prefer to have one -Xmx setting for all runs.
Added for Clarification: this is all an issue because there is also a large memory overhead in Eden for actually writing the object out (mostly String instances, which also appear as roots in the heap analysis). There are frequent minor GC's in Eden as a result. These would be less frequent if the result objects were not hanging around in the tenured space. The argument could be made that my real problem is the output overhead in Eden, and I am working on that, but wanted to pursue this tenured issue at the same time.
As I research this, are there any particular garbage collector settings or programmatic approaches I should be focusing on? Note I am using JDK 1.8.
Answer Update: #maaartinus made some great suggestions that helped me avoid queueing (and thus tenuring) the large objects in the first place. He also suggested bounding the queue, which would surely cut down on the tenuring of what I am now queueing instead (the CSV byte[] representations of the results objects). The right mix of thread count and queue bounds will definitely help, though I have not tried this as the problem basically disappeared by finding a way to not tenure the big objects in the first place.
I'm sceptical concerning a GC-related solution, but it looks like you're creating a problem you needn't to have:
Multiple worker threads create the results objects and queue them, and a single writer...
... writing the results takes far longer than generating them ...
So it looks like it should actually be the other way round: single producer and many consumers to keep the game even.
Multiple writers mightn't give you much speed up, but I'd try it, if possible. The number of producers doesn't matter much as long as you use a bounded queue for their results (I'm assuming they have no substantially sized input as you haven't mentioned it). This bounded queue could also ensure that the objects get never too old.
In any case, you can use multiple to CSV converters, so effectively replacing a big object by a big String or byte[], or ByteBuffer, or whatever (assuming you want to do the conversion in memory). The nice thing about the buffer is that you can recycle it (so the fact that it gets tenured is no problem anymore).
You could also use some unmanaged memory, but I really don't believe it's necessary. Simply bounding the queue should be enough, unless I'm missing something.
And by the way, quite often the cheapest solution is to buy more RAM. Really, one hour of work is worth a couple of gigabytes.
Update
how much should I be worried about contention between multiple writer threads, since they would all be sharing one thread-safe Writer?
I can imagine two kinds of problems:
Atomicity: While synchronizations ensures that each executed operations happens atomically, it doesn't mean that the output makes any sense. Imagine multiple writers, each of them generating a single CSV and the resulting file should contain all the CSVs (in any order). Using a PrintWriter would keep each line intact, but it'd intermix them.
Concurrency: For example, a FileWriter performs the conversion from chars to bytes, which may in this context end up in a synchronized block. This could reduce parallelism a bit, but as the IO seems to be the bottleneck, I guess, it doesn't matter.
Assume a multithreaded application scenario, in which every thread acquires some data (one or more files) from the network, performs some processing and then saves the results on the hard disk of the hosting machine.
In such a scenario, there is always the possibility that the disk space is exhausted, leading to unexpected service behavior (e.g., a system crash).
To avoid a case like that, it would be helpful if Java provided a means of reserving hard disk space, but, as verified in an earlier question, such an option is not available and even if it were, it could lead to inefficient allocation (e.g., in the case of a decompressing application, which does not know beforehand the total size of the decompressed data).
So, an alternative could be to make "virtual disk space reservations", e.g. by keeping in memory a static registry of the free space and having each thread request capacity from the registry before proceeding.
Are there any better alternatives, or improvements to this approach?
Is there any (preferably open source) Java library that implements such functionality?
An abstract way to implement this might be to use a constant or user inputted value of how much disk space the multi-threaded application is allowed to use, save this as variable and have synchronized get and set methods that will get the value of allowed space, allocate it space for the thread (as much is needed but not more then available), and then minus that from the total so other threads may see a decreased 'disk space' and once a thread has finished and data is deleted re-add the value so the used 'disk space' becomes usable to other threads?
EDIT:
it could lead to inefficient allocation (e.g., in the case of a decompressing application, which does not know beforehand the total size of the decompressed data).
If this occurs and the thread 'sees' (through a constant check while extracting file) that it has reached its limit of 'disk space', it could then request for more space allocation if available, or be put back into a queue until the needed space has been free'd up by other threads
Couldn't you also simply have an alert that triggers off after >XX% of the given partition is used up? That way your admin has time to go in and remove/copy data off or add additional storage to that mount point?
We have a swing based application that does complex processing on data. One of the prerequisites for our software is that any given column cannot have too many unique values. If the number is numeric, the user would need to discretize the data before they could from our tool.
Unfortunately, the algorithms we are using are combinatorially expensive in memory depending on the number of unique values per column. Right now with the wrong dataset, the app would run out of memory very quickly. Before doing one of these operations that would run out of memory, we should be able to calculate roughly how much memory the operation will need. It would be nice if we could check how much memory the app currently is using, estimate if the app is going to run out of memory, and show an error message accordingly rather than running out of memory. Using java.lang.Runtime, we can find the free memory, total memory, and max memory, but is this really helpful? Even if it appears we won't have enough heap space, it could be that if we wait 30 milliseconds the garbage collector will run, and suddenly we have more than enough heap space to run our operation. Is there anyway to really predict if we are going to run out of memory?
I have done something similar for a database application where the number of rows that were loaded could not be estimated. So in the loop that processes the result set I'm calling a "MemorWatcher" method that would check the memory that was free.
If the available memory goes under a certain threshold the watcher would force a garbage collection and re-check. If there still wasn't enough memory the watcher method signals this to the caller with an exception. The caller can gracefully recover from that exception - as opposed to the OutOfMemoryException which sometimes leaves Swing totally unstable.
I don't have expertise on this, but I feel you can take an extra step of bytecode analysis using ASM to preempt bugs like null pointer exception, out of memory exception etc.
Unless you run your application with the maximum amount of memory you need from the outset (using -Xms) I don't think you can achieve anything useful, since other applications will be able to consume memory before your app needs it.
Have you considered using Soft/WeakReferences, and letting garbage collection reap objects that you could possible recalculate/regenerate on the fly ?
I have this class and I'm testing insertions with different data distributions. I'm doing this in my code:
...
AVLTree tree = new AVLTree();
//insert the data from the first distribution
//get results
...
tree = new AVLTree();
//inser the data from the next distribution
//get results
...
I'm doing this for 3 distributions. Each one should be tested an average of 14 times, and the 2 lowest/highest values removed from to compute the average. This should be done 2000 times, each time for 1000 elements. In other words, it goes 1000, 2000, 3000, ..., 2000000.
The problem is, I can only get as far as 100000. When I tried 200000, I ran out of heap space. I increased the available heap space with -Xmx in the command line to 1024m and it didn't even complete the tests with 200000. I tried 2048m and again, it wouldn't work.
What I'm thinking is that the garbage collector isn't getting rid of the old trees once I do tree = new AVL Tree(). But why? I thought that the elements from the old trees would no longer be accessible and their memory would be cleaned up.
The garbage collector should have no trouble cleaning up your old tree objects, so I can only assume there's some other allocation that you're doing that's not being cleaned up.
Java has a good tool to watch the GC in progress (or not in your case), JVisualVM, which comes with the JDK.
Just run that and it will show you which objects are taking up the heap, and you can both trigger and see the progress of GC's. Then you can target those for pools so they can be re-used by you, saving the GC the work.
Also look into this option, which will probably stop the error you're getting that stops the program, and you program will finish, but it may take a long time because your app will fill up the heap then run very slowly.
-XX:-UseGCOverheadLimit
Which JVM you are using and what JVM parameters you have used to configure GC?
Your explaination shows there is a memory leak in your code. If you have any tool like jprofiler then use it to find out where is the memory leak.
There's no reason those trees shouldn't be collected, although I'd expect that before you ran out of memory you should see long pauses as the system ran a full GC. As it's been noted here that that's not what you're seeing, you could try running with flags like -XX:-PrintGC, -XX:-PrintGCDetails,-XX:-PrintGCTimeStamps to give you some more information on exactly what's going on, along with perhaps some sort of running count of roughly where you are. You could also explicitly tell the garbage collector to use a different garbage-collection algorithm.
However, it still seems unlikely to me. What other code is running? is it possible there's something in the AVLTree class itself that's keeping its instances from being GC'd? What about manually logging the finalize() on that class to insure that (some of them, at least) are collectible (e.g. make a few and manually call System.gc())?
GC params here, a nice ref on garbage collection from sun here that's well worth reading.
The Java garbage collector isn't guaranteed to garbage collect after each object's refcount becomes zero. So if you're writing code that is only creating and deleting a lot of objects, it's possible to expend all of the heap space before the gc has a chance to run. Alternatively, Pax's suggestion that there is a memory leak in your code is also a strong possibility.
If you are only doing benchmarking, then you may want to use the java gc function (in the System class I think) between tests, or even re-run you program for each distribution.
We noticed this in a server product. When making a lot of tiny objects that quickly get thrown away, the garbage collector can't keep up. The problem is more pronounced when the tiny objects have pointers to larger objects (e.g. an object that points to a large char[]). The GC doesn't seem to realize that if it frees up the tiny object, it can then free the larger object. Even when calling System.gc() directly, this was still a huge problem (both in 1.5 and 1.6 VMs)!
What we ended up doing and what I recommend to you is to maintain a pool of objects. When your object is no longer needed, throw it into the pool. When you need a new object, grab one from the pool or allocate a new one if the pool is empty. This will also save a small amount of time over pure allocation because Java doesn't have to clear (bzero) the object.
If you're worried about the pool getting too large (and thus wasting memory), you can either remove an arbitrary number of objects from the pool on a regular basis, or use weak references (for example, using java.util.WeakHashMap). One of the advantages of using a pool is that you can track the allocation frequency and totals, and you can adjust things accordingly.
We're using pools of char[] and byte[], and we maintain separate "bins" of sizes in the pool (for example, we always allocate arrays of size that are powers of two). Our product does a lot of string building, and using pools showed significant performance improvements.
Note: In general, the GC does a fine job. We just noticed that with small objects that point to larger structures, the GC doesn't seem to clean up the objects fast enough especially when the VM is under CPU load. Also, System.gc() is just a hint to help schedule the finalizer thread to do more work. Calling it too frequently causes a significant performance hit.
Given that you're just doing this for testing purposes, it might just be good housekeeping to invoke the garbage collector directly using System.gc() (thus forcing it to make a pass). It won't help you if there is a memory leak, but if there isn't, it might buy you back enough memory to get through your test.