Related
I need to find out when I'm really close to the OutOfMemoryError so I can flush results to file and call runtime.gc();. My code is something like this:
Runtime runtime = Runtime.getRuntime();
...
if ((1.0 * runtime.totalMemory() / runtime.maxMemory()) > 0.9) {
... flush results to file ...
runtime.gc();
}
Is there a better way to do this? Can someone give me a hand please?
EDIT
I understood that I am playing with fire this way so I reasoned to a more solid and simple way of determining when I've had enough. I am currently working with the Jena model so I do a simple check: if the model has more than 550k statements then I flush so I don't run any risks.
First: if you want to determine if you're close to OutOfMemoryError, then what all you have to do is to compare the current memory with the max memory used by JVM, and that what you already did.
Second: You want to flush results to file, am wondering why you want to do that just if you close to OutOfMemoryError, you simply can use something like a FileWriter which has a buffer, so if the buffer got filled it will flush the results automatically.
Third: don't ever call the GC explicitly, its a bad practice, optimize your JVM memory arguments instead:
-Xmx -> this param to set the max memory that the JVM can allocate
-Xms -> the init memory that JVM will allocate on the start up
-XX:MaxPermSize= -> this for the max Permanent Generation memory
Also
-XX:MaxNewSize= -> this need to be 40% from your Xmx value
-XX:NewSize= -> this need to be 40% from your Xmx value
These will speed up the GC.
And -XX:+UseConcMarkSweepGC to enable using CMS for the old space.
This seems to work:
public class LowMemoryDetector {
// Use a soft reference to some memory - will be held onto until GC is nearly out of memory.
private final SoftReference<byte[]> buffer;
// The queue that watches for the buffer to be discarded.
private final ReferenceQueue<byte[]> queue = new ReferenceQueue<>();
// Have we seen the low condition?
private boolean seenLow = false;
public LowMemoryDetector(int bufferSize) {
// Make my buffer and add register the queue for it to be discarded to.
buffer = new SoftReference(new byte[bufferSize], queue);
}
/**
* Please be sure to create a new LMD after it returns true.
*
* #return true if a memory low condition has been detected.
*/
public boolean low () {
// Preserve that fact that we've seen a low.
seenLow |= queue.poll() != null;
return seenLow;
}
}
private static final int OneMeg = 0x100000;
public void test() {
LowMemoryDetector lmd = new LowMemoryDetector(2*OneMeg);
ArrayList<char[]> eatMemory = new ArrayList<>();
int ate = 0;
while ( !lmd.low() ) {
eatMemory.add(new char[OneMeg]);
ate += 1;
}
// Let it go.
eatMemory = null;
System.out.println("Ate "+ate);
}
it prints
Ate 1070
for me.
Use a buffer size of something larger than the largest allocation unit you are using. It needs to be big enough so that any allocation request would be satisfied if the buffer was freed.
Please remember that on a 64bit JVM it is potentially possible that you are running with many tb of memory. This approach would almost certainly encounter many difficulties in this case.
I'm creating and inserting fairly light weight Person objects which have one field- age in Drool's working memory. But even after removing facts, heap size is not reducing. Sample code- (using Drools 6.0.0.CR5 from maven)
long numOfFacts=1000000;
long heapSize = Runtime.getRuntime().totalMemory();
System.out.println("Heapsize before insertion: "+heapSize);
System.out.println("Inserting objects");
ArrayList<FactHandle> factHandles = new ArrayList<FactHandle>(100);
for (int i = 0; i < numOfFacts; i++) {
Person person = new Person();
person.setAge(randomGenerator.nextInt(100));
FactHandle factHandle = wkmem.insert(person);
factHandles.add(factHandle);
}
long heapSizeAfter = Runtime.getRuntime().totalMemory();
System.out.println("Heapsize after insertion: "+heapSizeAfter);
long endTimeInsert = System.currentTimeMillis();
long elTime= endTimeInsert-startTimeInsert;
System.out.println("Time it took to insert " +numOfFacts+" objects :"+elTime+" milliseconds");
long startTime = System.currentTimeMillis();
System.out.println("Number of facts: " + wkmem.getFactCount());
wkmem.fireAllRules();
long stopTime = System.currentTimeMillis();
long elapsedTime = stopTime - startTime;
System.out.println("Time it took for evaluation: " + elapsedTime);
for(int i=0;i<numOfFacts;i++){
wkmem.retract(factHandles.get(i));
}
long heapSizeAfterRemoval = Runtime.getRuntime().totalMemory();
System.out.println("Heapsize after removal of facts: "+heapSizeAfterRemoval);
The output of code is-
Heapsize before insertion: 158138368
Inserting objects
Heapsize after insertion: 746717184
Time it took to insert 1000000 objects :5372 milliseconds
Number of facts: 1000000
Time it took for evaluation: 839
Heapsize after removal of facts: 792002560
Why is that heapsize has in fact increased?
As mentioned in Peter Lawrey's answer, you're not going to to see heap size reduced in the middle of a method. Unless perhaps GC just happens to kick in at that very moment. To test for that, you need to have a long-running application and connect to it with something such as JConsole or use a profiler of some sort.
However, it is worth noting that the way you are retracting is not reliable and will result in memory leaks in some cases. The truth is that in some cases Drools will generate FactHandles internally, so that after retracting all facts associated with your own fact handle references, there may well be more sitting in working memory. If I remember right, these keep hold of references to your facts, which prevents those objects from being garbage collected. Therefore it's a lot safer to just retract all fact handles:
public void retractAll() {
for (FactHandle handle : ksession.getFactHandles()) {
retract(handle);
}
}
... or retract all FactHandles for a filter:
public void retractAll(ObjectFilter filter) {
for (FactHandle handle : ksession.getFactHandles(filter)) {
retract(handle);
}
}
I discovered this the hard way ... my retraction code made the same assumption as yours originally. :)
The heap size always stays the same or increases, until the GC needs to run or decides to run (for concurrent collectors)
Collecting memory is expensive so it only does it when it has to, not when it might.
Why is that heapsize has in fact increased?
Removing objects can do some work which can end up creating temporary objects.
Basically you should only look at memory consumption after a Full GC, anything else is on a least effort basis.
Consider the following two segments of code in Java,
Integer x=new Integer(100);
Integer y=x;
Integer z=x;
System.out.println("Used memory (bytes): " +
(Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory()));
In which the memory usage was when tested on my system : Used memory (bytes): 287848
and
int a=100;
int b=a;
int c=a;
System.out.println("Used memory (bytes): " +
(Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory()));
In which the memory usage was when tested on my system : Used memory (bytes): 287872
and the following
Integer x=new Integer(100);
System.out.println("Used memory (bytes): " +
(Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory()));
and
int a=100;
System.out.println("Used memory (bytes): " +
(Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory()));
in both of the above cases, the memory usage was exactly the same when tested on my system : Used memory (bytes): 287872
The statement
System.out.println("Used memory (bytes): " +
(Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory()));
will display the total memory currently in use [Total available memory-Currently free available memory], (in bytes).
I have alternatively verified through the above mentioned methods that in the first case the memory usage (287848) was lower than the second one (287872) while in the rest of the two cases it was exactly the same (287872). Of course and obviously, it should be such because in the very first case, y and z contain a copy of the reference held in x and they all (x, y and z) point to the same/common object (location) means that the first case is better and more appropriate than the second one and in the rest of the two cases, there are equivalent statements with exactly the same memory usage (287872). If it is so, then the use of primitive data types in Java should be useless and avoidable though they were basically designed for better memory usage and more CPU utilization. still why do primitive data types in Java exist?
A question somewhat similar to this one was already posted here but it did not have such a scenario.
That question is here.
I wouldn't pay attention to Runtime.freeMemory -- it's very ambiguous (does it include unused stack space? PermGen space? gaps between heap objects that are too small to be used?), and giving any precise measurement without halting all threads is impossible.
Integers are necessarily less space efficient than ints, because just the reference to the Integer takes 32 bits (64 for a 64-bit JVM without compressed pointers).
If you really want to test it empirically, have many threads recurse deeply and then wait. As in
class TestThread extends Thread {
private void recurse(int depth) {
int a, b, c, d, e, f, g;
if (depth < 100)
recurse(depth + 1);
for (;;) try {
Thread.sleep(Long.MAX_VALUE);
} catch (InterruptedException e) {}
}
#Override public void run() {
recurse(0);
}
public static void main(String[] _) {
for (int i = 0; i < 500; ++i)
new TestThread().start();
}
}
For a start, an Integer wraps an int, therefore Integer has to be at least as big as int.
From the docs (I really doubt this is necessary):
The Integer class wraps a value of the primitive type int in an
object. An object of type Integer contains a single field whose type
is int.
So obviously a primitive int is still being used.
Not only that but objects have more overhead, and the most obvious one is that when you're using objects your variable contains a reference to it:
Integer obj = new Integer(100);
int prim = 100;
ie. obj stores a reference to an Integer object, which contains an int, whereas prim stores the value 100. That there's enough to prove that using Integer over int brings with it more overhead. And there's more overhead than just that.
The wrapper contains a primitive as a field, but it causes additional overhead because it's an object. The reference takes up space as well, but your example isn't really designed to show this.
The tests you designed aren't really well-suited for a precise measurement, but since you used them, try this example instead:
public static void main(String[] args) {
int numInts = 100000;
Integer[] array = new Integer[numInts];
// int[] array = new int[numInts];
for(int i = 0; i < numInts; i++){
array[i] = i; //put some real data into the arrays using auto-boxing if needed
}
System.out.println("Used memory (bytes): " +
(Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory()));
}
Now try it again but uncomment the primitive line and comment out the wrapper line. You should see that the wrapper takes up much more memory
If your first example, you have the equivalent to 1 integer, and 2 pointers.
Because Integer is an Object, it has pointer properties, and contains functions.
By using int instead of Integer, you are copying the value 3 times.
You have a difference in 24 bytes, which is used for storing the headers and values of your extra 2 ints. Although I wouldn't trust your test: the JVM can be somewhat random, and it's garbage collection is quite dynamic. As far as required memory for a single Integer vs int, Integer will take up more space because it is an Object, and thus contains more information.
Runtime.getRuntime().freeMemory() : getting delta on this does not give you the correct statistics as there are many moving parts like garbage collection and other threads.
Integer takes more memory than int primitive.
Your test case is too simple to be of any conclusive result.
Any test case that takes less than 5 seconds doesn't mean anything.
You need to at least do something with these objects you are creating. The JVM can simply look at your code and just not do anything because your objects aren't ever used, and you exit. (Can't say for certain what the JVM interpreter does, but the JIT will use escape analysis to optimize your entire testcase into nothing)
First of all, if you're looking for memory effectiveness, primitives are smaller because they are what size they are. The wrapper objects are objects, and need to be garbage collected. They have tons of fields within them that you can use, those fields are stored somewhere...
Primitives aren't "designed" to be more effective. Wrapper objects were designed to be more feature friendly. You need primitives, because how else are you going to store a number?
If you really wan't to see the memory difference, take a real application. If you want to write it yourself, go ahead but it'll take some time. Use some text editor and search and replace every single int declaration with Integer, and long with Long, etc. Then take a look at the memory footprint. I wouldn't be surprised if you see your computer explode.
From a programming point of view, you need to use primitives when necessary, and wrapper objects when necessary. When its applicable to do both, it's your preference. Trust me, there aren't that many.
http://www.javaspecialists.eu/archive/Issue193.html
This might help you understand/explore things a little bit more. An excellent article! Cheers!
If you look at the source code of java.lang.Integer, the value is stored as an int.
private int value;
Your test is not valid, that's all there is to it.
Proof:
when you run these Tests you'll get an AssertionError in second Test (because memory gets lower, even if you stop resetting memory-field). Once you try this tests with 10.000 loops you'll get at both StackOverflowError.
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;
import org.junit.Test;
public class TestRedundantIntegers {
private long memory;
#Test
public void whenRecursiveIntIsSet() {
memory = Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory();
recurseInt(0, 100);
}
private void recurseInt(int depth, int someInt) {
int x = someInt;
assertThat(memory,is(Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory()));
memory=Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory();
if (depth < 1000)
recurseInt(depth + 1, x);
}
#Test
public void whenRecursiveIntegerIsSet() {
memory = Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory();
recurseInteger(0, new Integer(100));
}
private void recurseInteger(int depth, Integer someInt) {
Integer x = someInt;
assertThat(memory,is(Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory()));
memory=Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory();
if (depth < 1000)
recurseInt(depth + 1, x);
}
}
As for "where and when": use the non-primitive types where an Object is required, and the primitives everywhere else. For example, the types of a generic can't be primitive, so you can't use primitives with them. Even before generics were introduced, things like HashSet and HashMap couldn't store primitives.
Is there any built-in method in Java to find the size of any datatype?
Is there any way to find size?
No. There is no such method in the standard Java SE class library.
The designers' view is that it is not needed in Java, since the language removes the need for an application1 to know about how much space needs to be reserved for a primitive value, an object or an array with a given number of elements.
You might think that a sizeof operator would be useful for people that need to know how much space their data structures take. However you can also get this information and more, simply and reliably using a Java memory profiler, so there is no need for a sizeof method.
Previous commenters made the point that sizeof(someType) would be more readable than 4. If you accept that readability argument, then the remedy is in your hands. Simply define a class like this ...
public class PrimitiveSizes {
public static int sizeof(byte b) { return 1; }
public static int sizeof(short s) { return 2; }
// etcetera
}
... and statically import it ...
import static PrimitiveSizes.*;
Or define some named constants; e.g.
public static final int SIZE_OF_INT = 4;
Or (Java 8 and later) use the Integer.BYTES constant, and so on.
Why haven't the Java designers implemented this in standard libraries? My guess is that:
they don't think there is a need for it,
they don't think there is sufficient demand for it, and
they don't think it is worth the effort.
There is also the issue that the next demand would be for a sizeof(Object o) method, which is fraught with technical difficulties.
The key word in the above is "they"!
1 - A programmer may need to know in order to design space efficient data structures. However, I can't imagine why that information would be needed in application code at runtime via a method call.
From the article in JavaWorld
A superficial answer is that Java does not provide anything like C's sizeof(). However,
let's consider why a Java programmer might occasionally want it.
A C programmer manages most datastructure memory allocations himself,
and sizeof() is indispensable for knowing memory block sizes to
allocate. Additionally, C memory allocators like malloc() do almost
nothing as far as object initialization is concerned: a programmer
must set all object fields that are pointers to further objects. But
when all is said and coded, C/C++ memory allocation is quite
efficient.
By comparison, Java object allocation and construction are tied
together (it is impossible to use an allocated but uninitialized
object instance). If a Java class defines fields that are references
to further objects, it is also common to set them at construction
time. Allocating a Java object therefore frequently allocates numerous
interconnected object instances: an object graph. Coupled with
automatic garbage collection, this is all too convenient and can make
you feel like you never have to worry about Java memory allocation
details.
Of course, this works only for simple Java applications. Compared with
C/C++, equivalent Java datastructures tend to occupy more physical
memory. In enterprise software development, getting close to the
maximum available virtual memory on today's 32-bit JVMs is a common
scalability constraint. Thus, a Java programmer could benefit from
sizeof() or something similar to keep an eye on whether his
datastructures are getting too large or contain memory bottlenecks.
Fortunately, Java reflection allows you to write such a tool quite
easily.
Before proceeding, I will dispense with some frequent but incorrect
answers to this article's question. Fallacy: Sizeof() is not needed
because Java basic types' sizes are fixed
Yes, a Java int is 32 bits in all JVMs and on all platforms, but this
is only a language specification requirement for the
programmer-perceivable width of this data type. Such an int is
essentially an abstract data type and can be backed up by, say, a
64-bit physical memory word on a 64-bit machine. The same goes for
nonprimitive types: the Java language specification says nothing about
how class fields should be aligned in physical memory or that an array
of booleans couldn't be implemented as a compact bitvector inside the
JVM. Fallacy: You can measure an object's size by serializing it into
a byte stream and looking at the resulting stream length
The reason this does not work is because the serialization layout is
only a remote reflection of the true in-memory layout. One easy way to
see it is by looking at how Strings get serialized: in memory every
char is at least 2 bytes, but in serialized form Strings are UTF-8
encoded and so any ASCII content takes half as much space
The Java Native Access library is typically used for calling native shared libraries from Java. Within this library there exist methods for determining the size of Java objects:
The getNativeSize(Class cls) method and its overloads will provide the size for most classes.
Alternatively, if your classes inherit from JNA's Structure class the calculateSize(boolean force) method will be available.
You can do bit manipulations like below to obtain the size of primitives:
public int sizeofInt() {
int i = 1, j = 0;
while (i != 0) {
i = (i<<1); j++;
}
return j;
}
public int sizeofChar() {
char i = 1, j = 0;
while (i != 0) {
i = (char) (i<<1); j++;
}
return j;
}
As mentioned here, there are possibilities to get the size of primitive types through their wrappers.
e.g. for a long this could be Long.SIZE / Byte.SIZE from java 1.5 (as mentioned by zeodtr already) or Long.BYTES as from java 8
There is a contemporary way to do that for primitives. Use BYTES of types.
System.out.println("byte " + Byte.BYTES);
System.out.println("char " + Character.BYTES);
System.out.println("int " + Integer.BYTES);
System.out.println("long " + Long.BYTES);
System.out.println("short " + Short.BYTES);
System.out.println("double " + Double.BYTES);
System.out.println("float " + Float.BYTES);
It results in,
byte 1
char 2
int 4
long 8
short 2
double 8
float 4
You can use Integer.SIZE / 8, Double.SIZE / 8, etc. for primitive types from Java 1.5.
The Instrumentation class has a getObjectSize() method however, you shouldn't need to use it at runtime. The easiest way to examine memory usage is to use a profiler which is designed to help you track memory usage.
EhCache provides a SizeOf class that will try to use the Instrumentation agent and will fall back to a different approach if the agent is not loaded or cannot be loaded (details here).
Also see the agent from Heinz Kabutz.
I decided to create an enum without following the standard Java conventions. Perhaps you like this.
public enum sizeof {
;
public static final int FLOAT = Float.SIZE / 8;
public static final int INTEGER = Integer.SIZE / 8;
public static final int DOUBLE = Double.SIZE / 8;
}
Try java.lang.Instrumentation.getObjectSize(Object). But please be aware that
It returns an implementation-specific approximation of the amount of storage consumed by the specified object. The result may include some or all of the object's overhead, and thus is useful for comparison within an implementation but not between implementations. The estimate may change during a single invocation of the JVM.
There's a class/jar available on SourceForge.net that uses Java instrumentation to calculate the size of any object. Here's a link to the description: java.sizeOf
Just some testing about it:
public class PrimitiveTypesV2 {
public static void main (String[] args) {
Class typesList[] = {
Boolean.class , Byte.class, Character.class, Short.class, Integer.class,
Long.class, Float.class, Double.class, Boolean.TYPE, Byte.TYPE, Character.TYPE,
Short.TYPE, Integer.TYPE, Long.TYPE, Float.TYPE, Double.TYPE
};
try {
for ( Class type : typesList ) {
if (type.isPrimitive()) {
System.out.println("Primitive type:\t" + type);
}
else {
boolean hasSize = false;
java.lang.reflect.Field fields[] = type.getFields();
for (int count=0; count<fields.length; count++) {
if (fields[count].getName().contains("SIZE")) hasSize = true;
}
if (hasSize) {
System.out.println("Bits size of type " + type + " :\t\t\t" + type.getField("SIZE").getInt(type) );
double value = type.getField("MIN_VALUE").getDouble(type);
long longVal = Math.round(value);
if ( (value - longVal) == 0) {
System.out.println("Min value for type " + type + " :\t\t" + longVal );
longVal = Math.round(type.getField("MAX_VALUE").getDouble(type));
System.out.println("Max value for type " + type + " :\t\t" + longVal );
}
else {
System.out.println("Min value for type " + type + " :\t\t" + value );
value = type.getField("MAX_VALUE").getDouble(type);
System.out.println("Max value for type " + type + " :\t\t" + value );
}
}
else {
System.out.println(type + "\t\t\t type without SIZE field.");
}
} // if not primitive
} // for typesList
} catch (Exception e) {e.printStackTrace();}
} // main
} // class PrimitiveTypes
Not sure for older versions, but since version 1.8 java sdk provides the .BYTES properties for boxed Objects of primitive types.
BYTES ( = SIZE / Byte.size )
import java.util.*;
import java.lang.*;
import java.io.*;
// The main method must be in a class named "Main".
class Main {
public static void main(String[] args) {
System.out.println("size of Integer: " + Integer.BYTES);
System.out.println("size of Character: " + Character.BYTES);
System.out.println("size of Short: " + Short.BYTES);
System.out.println("size of Long: " + Long.BYTES);
System.out.println("size of Double: " + Double.BYTES);
System.out.println("size of Float: " + Float.BYTES);
}
}
Here's a fiddle: https://www.mycompiler.io/view/0N19Y6cWL8F
I don't think it is in the java API. but most datatypes which have a number of elements in it, have a size() method. I think you can easily write a function to check for size yourself?
yes..in JAVA
System.out.println(Integer.SIZE/8); //gives you 4.
System.out.println(Integer.SIZE); //gives you 32.
//Similary for Byte,Long,Double....
I have a problem with my Java progam suddenly exiting, without any exception thrown or the program finishing normally.
I'm writing a program to solve Project Euler's 14th problem. This is what I got:
private static final int INITIAL_CACHE_SIZE = 30000;
private static Map<Long, Integer> cache = new HashMap<Long, Integer>(INITIAL_CACHE_SIZE);
public void main(String... args) {
long number = 0;
int maxSize = 0;
for (long i = 1; i <= TARGET; i++) {
int size = size(i);
if (size > maxSize) {
maxSize = size;
number = i;
}
}
}
private static int size(long i) {
if (i == 1L) {
return 1;
}
final int size = size(process(i)) + 1;
return size;
}
private static long process(long n) {
return n % 2 == 0 ? n/2 : 3*n + 1;
}
This runs fine, and finishes correctly in about 5 seconds when using a TARGET of 1 000 000.
I wanted to optimize by adding a cache, so I changed the size method to this:
private static int size(long i) {
if (i == 1L) {
return 1;
}
if (cache.containsKey(i)) {
return cache.get(i);
}
final int size = size(process(i)) + 1;
cache.put(i, size);
return size;
}
Now when I run it, it simply stops (process exits) when I get to 555144. Same number every time. No exception, error, Java VM crash or anything is thrown.
Changing the cache size doesn't seem to have any effect either, so how could the cache
introduction cause this error?
If I enforce the cache size to be not just initial, but permanent like so:
if (i < CACHE_SIZE) {
cache.put(i, size);
}
the bug no longer occurs.
Edit: When I set the cache size to like 2M, the bug starts showing again.
Can anyone reproduce this, and maybe even provide a suggestion as to why it happens?
This is simply an OutOfMemoryError that is not being printed. The program runs fine if I set a high heap size, otherwise it exits with an unlogged OutOfMemoryError (easy to see in a Debugger, though).
You can verify this and get a heap dump (as well as printout that an OutOfMemoryError occurred) by passing this JVM arg and re-running your program:
-XX:+HeapDumpOnOutOfMemoryError
With this it will then print out something to this effect:
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid4192.hprof ...
Heap dump file created [91901809 bytes in 4.464 secs]
Bump up your heap size with, say, -Xmx200m and you won't have an issue - At least for TARGET=1000000.
It sounds like the JVM itself crashes (that is the first thought when your program dies without a hint of an exception anyway). The first step in such a problem is to upgrade to the latest revision for your platform. The JVM should dump the heap to a .log file in the directory where you started the JVM, assuming your user level has access rights to that directory.
That being said, some OutOfMemory errors don't report in the main thread, so unless you do a try/catch (Throwable t) and see if you get one, it is hard to be sure you aren't actually just running out of memory. The fact that it only uses 100MB could just mean that the JVM isn't configured to use more. That can be changed by changing the startup options to the JVM to -Xmx1024m to get a Gig of memory, to see if the problem goes anywhere.
The code for doing the try catch should be something like this:
public static void main(String[] args) {
try {
MyObject o = new MyObject();
o.process();
} catch (Throwable t) {
t.printStackTrace();
}
}
And do everything in the process method and do not store your cache in statics, that way if the error happens at the catch statement the object is out of scope and can be garbage collected, freeing enough memory to allow the printing of the stack trace. No guarantees that that works, but it gives it a better shot.
One significant difference between the two implmentations of size(long i) is in the amount of objects you are creating.
In the first implementation, there are no Objects being created. In the second you are doing an awful lot of autoboxing, creating a new Long for each access of your cache, and putting in new Longs and new Integers on each modification.
This would explain the increase in memory usage, but not the absence of an OutOfMemoryError. Increasing the heap does allows it to complete for me.
From this Sun aritcle:
The performance ... is likely to be poor, as it boxes or unboxes on every get or set operation. It is plenty fast enough for occasional use, but it would be folly to use it in a performance critical inner loop.
If your java process suddenly crashes it could be some resource got maxed out. Like memory. You could try setting a higher max heap
Do you see a Heap Dump being generated after the crash? This file should be in the current directory for your JVM, that's where I would look for more info.
I am getting an OutOfMemory error on cache.put(i, size);
To get the error run your program in eclipse using debug mode it will appear in the debug window. It does not produce a stack trace in the console.
The recursive size() method is probably not a good place to do the caching. I put a call to cache.put(i, size); inside the main()'s for-loop and it works much more quickly. Otherwise, I also get an OOM error (no more heap space).
Edit: Here's the source - the cache retrieval is in size(), but the storing is done in main().
public static void main(String[] args) {
long num = 0;
int maxSize = 0;
long start = new Date().getTime();
for (long i = 1; i <= TARGET; i++) {
int size = size(i);
if (size >= maxSize) {
maxSize = size;
num = i;
}
cache.put(i, size);
}
long computeTime = new Date().getTime() - start;
System.out.println(String.format("maxSize: %4d on initial starting number %6d", maxSize, num));
System.out.println("compute time in milliseconds: " + computeTime);
}
private static int size(long i) {
if (i == 1l) {
return 1;
}
if (cache.containsKey(i)) {
return cache.get(i);
}
return size(process(i)) + 1;
}
Note that by removing the cache.put() call from size(), it does not cache every computed size, but it also avoids re-caching a previously computed size. This does not affect the hashmap operations, but like akf points out, it avoids the autoboxing/unboxing operations which is where your heap killer is coming from. I also tried a "if (!containsKey(i)) { cache.put() etc" in size() but that unfortunately also runs out of memory.