Related
What is a StackOverflowError, what causes it, and how should I deal with them?
Parameters and local variables are allocated on the stack (with reference types, the object lives on the heap and a variable in the stack references that object on the heap). The stack typically lives at the upper end of your address space and as it is used up it heads towards the bottom of the address space (i.e. towards zero).
Your process also has a heap, which lives at the bottom end of your process. As you allocate memory, this heap can grow towards the upper end of your address space. As you can see, there is a potential for the heap to "collide" with the stack (a bit like tectonic plates!!!).
The common cause for a stack overflow is a bad recursive call. Typically, this is caused when your recursive functions doesn't have the correct termination condition, so it ends up calling itself forever. Or when the termination condition is fine, it can be caused by requiring too many recursive calls before fulfilling it.
However, with GUI programming, it's possible to generate indirect recursion. For example, your app may be handling paint messages, and, whilst processing them, it may call a function that causes the system to send another paint message. Here you've not explicitly called yourself, but the OS/VM has done it for you.
To deal with them, you'll need to examine your code. If you've got functions that call themselves then check that you've got a terminating condition. If you have, then check that when calling the function you have at least modified one of the arguments, otherwise there'll be no visible change for the recursively called function and the terminating condition is useless. Also mind that your stack space can run out of memory before reaching a valid terminating condition, thus make sure your method can handle input values requiring more recursive calls.
If you've got no obvious recursive functions then check to see if you're calling any library functions that indirectly will cause your function to be called (like the implicit case above).
To describe this, first let us understand how local variables and objects are stored.
Local variable are stored on the stack:
If you looked at the image you should be able to understand how things are working.
When a function call is invoked by a Java application, a stack frame is allocated on the call stack. The stack frame contains the parameters of the invoked method, its local parameters, and the return address of the method. The return address denotes the execution point from which, the program execution shall continue after the invoked method returns. If there is no space for a new stack frame then, the StackOverflowError is thrown by the Java Virtual Machine (JVM).
The most common case that can possibly exhaust a Java application’s stack is recursion. In recursion, a method invokes itself during its execution. Recursion is considered as a powerful general-purpose programming technique, but it must be used with caution, to avoid StackOverflowError.
An example of throwing a StackOverflowError is shown below:
StackOverflowErrorExample.java:
public class StackOverflowErrorExample {
public static void recursivePrint(int num) {
System.out.println("Number: " + num);
if (num == 0)
return;
else
recursivePrint(++num);
}
public static void main(String[] args) {
StackOverflowErrorExample.recursivePrint(1);
}
}
In this example, we define a recursive method, called recursivePrint that prints an integer and then, calls itself, with the next successive integer as an argument. The recursion ends until we pass in 0 as a parameter. However, in our example, we passed in the parameter from 1 and its increasing followers, consequently, the recursion will never terminate.
A sample execution, using the -Xss1M flag that specifies the size of the thread stack to equal to 1 MB, is shown below:
Number: 1
Number: 2
Number: 3
...
Number: 6262
Number: 6263
Number: 6264
Number: 6265
Number: 6266
Exception in thread "main" java.lang.StackOverflowError
at java.io.PrintStream.write(PrintStream.java:480)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
at java.io.PrintStream.write(PrintStream.java:527)
at java.io.PrintStream.print(PrintStream.java:669)
at java.io.PrintStream.println(PrintStream.java:806)
at StackOverflowErrorExample.recursivePrint(StackOverflowErrorExample.java:4)
at StackOverflowErrorExample.recursivePrint(StackOverflowErrorExample.java:9)
at StackOverflowErrorExample.recursivePrint(StackOverflowErrorExample.java:9)
at StackOverflowErrorExample.recursivePrint(StackOverflowErrorExample.java:9)
...
Depending on the JVM’s initial configuration, the results may differ, but eventually the StackOverflowError shall be thrown. This example is a very good example of how recursion can cause problems, if not implemented with caution.
How to deal with the StackOverflowError
The simplest solution is to carefully inspect the stack trace and
detect the repeating pattern of line numbers. These line numbers
indicate the code being recursively called. Once you detect these
lines, you must carefully inspect your code and understand why the
recursion never terminates.
If you have verified that the recursion
is implemented correctly, you can increase the stack’s size, in
order to allow a larger number of invocations. Depending on the Java
Virtual Machine (JVM) installed, the default thread stack size may
equal to either 512 KB, or 1 MB. You can increase the thread stack
size using the -Xss flag. This flag can be specified either via the
project’s configuration, or via the command line. The format of the
-Xss argument is:
-Xss<size>[g|G|m|M|k|K]
If you have a function like:
int foo()
{
// more stuff
foo();
}
Then foo() will keep calling itself, getting deeper and deeper, and when the space used to keep track of what functions you're in is filled up, you get the stack overflow error.
Stack overflow means exactly that: a stack overflows. Usually there's a one stack in the program that contains local-scope variables and addresses where to return when execution of a routine ends. That stack tends to be a fixed memory range somewhere in the memory, therefore it's limited how much it can contain values.
If the stack is empty you can't pop, if you do you'll get stack underflow error.
If the stack is full you can't push, if you do you'll get stack overflow error.
So stack overflow appears where you allocate too much into the stack. For instance, in the mentioned recursion.
Some implementations optimize out some forms of recursions. Tail recursion in particular. Tail recursive routines are form of routines where the recursive call appears as a final thing what the routine does. Such routine call gets simply reduced into a jump.
Some implementations go so far as implement their own stacks for recursion, therefore they allow the recursion to continue until the system runs out of memory.
Easiest thing you could try would be to increase your stack size if you can. If you can't do that though, the second best thing would be to look whether there's something that clearly causes the stack overflow. Try it by printing something before and after the call into routine. This helps you to find out the failing routine.
A stack overflow is usually called by nesting function calls too deeply (especially easy when using recursion, i.e. a function that calls itself) or allocating a large amount of memory on the stack where using the heap would be more appropriate.
Like you say, you need to show some code. :-)
A stack overflow error usually happens when your function calls nest too deeply. See the Stack Overflow Code Golf thread for some examples of how this happens (though in the case of that question, the answers intentionally cause stack overflow).
A StackOverflowError is a runtime error in Java.
It is thrown when the amount of call stack memory allocated by the JVM is exceeded.
A common case of a StackOverflowError being thrown, is when the call stack exceeds due to excessive deep or infinite recursion.
Example:
public class Factorial {
public static int factorial(int n){
if(n == 1){
return 1;
}
else{
return n * factorial(n-1);
}
}
public static void main(String[] args){
System.out.println("Main method started");
int result = Factorial.factorial(-1);
System.out.println("Factorial ==>"+result);
System.out.println("Main method ended");
}
}
Stack trace:
Main method started
Exception in thread "main" java.lang.StackOverflowError
at com.program.stackoverflow.Factorial.factorial(Factorial.java:9)
at com.program.stackoverflow.Factorial.factorial(Factorial.java:9)
at com.program.stackoverflow.Factorial.factorial(Factorial.java:9)
In the above case, it can be avoided by doing programmatic changes.
But if the program logic is correct and it still occurs then your stack size needs to be increased.
StackOverflowError is to the stack as OutOfMemoryError is to the heap.
Unbounded recursive calls result in stack space being used up.
The following example produces StackOverflowError:
class StackOverflowDemo
{
public static void unboundedRecursiveCall() {
unboundedRecursiveCall();
}
public static void main(String[] args)
{
unboundedRecursiveCall();
}
}
StackOverflowError is avoidable if recursive calls are bounded to prevent the aggregate total of incomplete in-memory calls (in bytes) from exceeding the stack size (in bytes).
The most common cause of stack overflows is excessively deep or infinite recursion. If this is your problem, this tutorial about Java Recursion could help understand the problem.
Here is an example of a recursive algorithm for reversing a singly linked list. On a laptop (with the specifications 4 GB memory, Intel Core i5 2.3 GHz CPU 64 bit and Windows 7), this function will run into StackOverflow error for a linked list of size close to 10,000.
My point is that we should use recursion judiciously, always taking into account of the scale of the system.
Often recursion can be converted to iterative program, which scales better. (One iterative version of the same algorithm is given at the bottom of the page. It reverses a singly linked list of size 1 million in 9 milliseconds.)
private static LinkedListNode doReverseRecursively(LinkedListNode x, LinkedListNode first){
LinkedListNode second = first.next;
first.next = x;
if(second != null){
return doReverseRecursively(first, second);
}else{
return first;
}
}
public static LinkedListNode reverseRecursively(LinkedListNode head){
return doReverseRecursively(null, head);
}
Iterative Version of the Same Algorithm:
public static LinkedListNode reverseIteratively(LinkedListNode head){
return doReverseIteratively(null, head);
}
private static LinkedListNode doReverseIteratively(LinkedListNode x, LinkedListNode first) {
while (first != null) {
LinkedListNode second = first.next;
first.next = x;
x = first;
if (second == null) {
break;
} else {
first = second;
}
}
return first;
}
public static LinkedListNode reverseIteratively(LinkedListNode head){
return doReverseIteratively(null, head);
}
The stack has a space limit that depends on the operating system. The normal size is 8 MB (in Ubuntu (Linux), you can check that limit with $ ulimit -u and it can be checked in other OS similarly). Any program makes use of the stack at runtime, but to fully know when it is used you need to check the assembly language. In x86_64 for example, the stack is used to:
Save the return address when making a procedure call
Save local variables
Save special registers to restore them later
Pass arguments to a procedure call (more than 6)
Other: random unused stack base, canary values, padding, ... etc.
If you don't know x86_64 (normal case) you only need to know when the specific high-level programming language you are using compile to those actions. For example in C:
(1) → a function call
(2) → local variables in function calls (including main)
(3) → local variables in function calls (not main)
(4) → a function call
(5) → normally a function call, it is generally irrelevant for a stack overflow.
So, in C, only local variables and function calls make use of the stack. The two (unique?) ways of making a stack overflow are:
Declaring too large local variables in main or in any function that it's called in (int array[10000][10000];)
A very deep or infinite recursion (too many function calls at the same time).
To avoid a StackOverflowError you can:
check if local variables are too big (order of 1 MB) → use the heap (malloc/calloc calls) or global variables.
check for infinite recursion → you know what to do... correct it!
check for normal too deep recursion → the easiest approach is to just change the implementation to be iterative.
Notice also that global variables, include libraries, etc... don't make use of the stack.
Only if the above does not work, change the stack size to the maximum on the specific OS. With Ubuntu for example: ulimit -s 32768 (32 MB). (This has never been the solution for any of my stack overflow errors, but I also don't have much experience.)
I have omitted special and/or not standard cases in C (such as usage of alloc() and similar) because if you are using them you should already know exactly what you are doing.
In a crunch, the below situation will bring a stack overflow error.
public class Example3 {
public static void main(String[] args) {
main(new String[1]);
}
}
A simple Java example that causes java.lang.StackOverflowError because of a bad recursive call:
class Human {
Human(){
new Animal();
}
}
class Animal extends Human {
Animal(){
super();
}
}
public class Test01 {
public static void main(String[] args) {
new Animal();
}
}
Many answers to this question are good. However, I would like to take a slightly different approach and give some more insight into how memory works and also a (simplified) visualization to better understand StackOverflow errors. This understanding does not only apply to Java but all processes alike.
On modern systems all new processes get their own virtual address space (VAS). In essence VAS is an abstraction layer provided by the operating system on top of physical memory in order to ensure processes do not interfere with each others memory. It's the kernels job to then map the virtual addresses provided to to the actual physical addresses.
VAS can be divided into a couple of sections:
In order to let the CPU know what it is supposed to do machine instructions must be loaded into memory. This is usually referred to as the code or text segment and of static size.
On top of that one can find the data segment and heap. The data segment is of fixed size and contains global or static variables.
As a program runs into special conditions it may need to additionally allocate data, which is where the heap comes into play and is therefore able to dynamically grow in size.
The stack is located on the other side of the virtual address space and (among other things) keeps track of all function calls using a LIFO data structure. Similar to the heap a program may need additional space during runtime to keep track of new function calls being invoked. Since the stack is located on the other side of the VAS it is growing into the opposite direction i.e. towards the heap.
TL;DR
This is where the StackOverflow error comes into play.
Since the stack grows down (towards the heap) it may so happen that at some point in time it cannot grow further as it would overlap with the heap address space. Once that happens the StackOverflow error occurs.
The most common reason as to why this happens is due to a bug in the program making recursive calls that do not terminate properly.
Note that on some systems VAS may behave slightly different an can be divided into even more segments, however, this general understanding applies to all UNIX systems.
Here's an example
public static void main(String[] args) {
System.out.println(add5(1));
}
public static int add5(int a) {
return add5(a) + 5;
}
A StackOverflowError basically is when you try to do something, that most likely calls itself, and goes on for infinity (or until it gives a StackOverflowError).
add5(a) will call itself, and then call itself again, and so on.
This is a typical case of java.lang.StackOverflowError... The method is recursively calling itself with no exit in doubleValue(), floatValue(), etc.
File Rational.java
public class Rational extends Number implements Comparable<Rational> {
private int num;
private int denom;
public Rational(int num, int denom) {
this.num = num;
this.denom = denom;
}
public int compareTo(Rational r) {
if ((num / denom) - (r.num / r.denom) > 0) {
return +1;
} else if ((num / denom) - (r.num / r.denom) < 0) {
return -1;
}
return 0;
}
public Rational add(Rational r) {
return new Rational(num + r.num, denom + r.denom);
}
public Rational sub(Rational r) {
return new Rational(num - r.num, denom - r.denom);
}
public Rational mul(Rational r) {
return new Rational(num * r.num, denom * r.denom);
}
public Rational div(Rational r) {
return new Rational(num * r.denom, denom * r.num);
}
public int gcd(Rational r) {
int i = 1;
while (i != 0) {
i = denom % r.denom;
denom = r.denom;
r.denom = i;
}
return denom;
}
public String toString() {
String a = num + "/" + denom;
return a;
}
public double doubleValue() {
return (double) doubleValue();
}
public float floatValue() {
return (float) floatValue();
}
public int intValue() {
return (int) intValue();
}
public long longValue() {
return (long) longValue();
}
}
File Main.java
public class Main {
public static void main(String[] args) {
Rational a = new Rational(2, 4);
Rational b = new Rational(2, 6);
System.out.println(a + " + " + b + " = " + a.add(b));
System.out.println(a + " - " + b + " = " + a.sub(b));
System.out.println(a + " * " + b + " = " + a.mul(b));
System.out.println(a + " / " + b + " = " + a.div(b));
Rational[] arr = {new Rational(7, 1), new Rational(6, 1),
new Rational(5, 1), new Rational(4, 1),
new Rational(3, 1), new Rational(2, 1),
new Rational(1, 1), new Rational(1, 2),
new Rational(1, 3), new Rational(1, 4),
new Rational(1, 5), new Rational(1, 6),
new Rational(1, 7), new Rational(1, 8),
new Rational(1, 9), new Rational(0, 1)};
selectSort(arr);
for (int i = 0; i < arr.length - 1; ++i) {
if (arr[i].compareTo(arr[i + 1]) > 0) {
System.exit(1);
}
}
Number n = new Rational(3, 2);
System.out.println(n.doubleValue());
System.out.println(n.floatValue());
System.out.println(n.intValue());
System.out.println(n.longValue());
}
public static <T extends Comparable<? super T>> void selectSort(T[] array) {
T temp;
int mini;
for (int i = 0; i < array.length - 1; ++i) {
mini = i;
for (int j = i + 1; j < array.length; ++j) {
if (array[j].compareTo(array[mini]) < 0) {
mini = j;
}
}
if (i != mini) {
temp = array[i];
array[i] = array[mini];
array[mini] = temp;
}
}
}
}
Result
2/4 + 2/6 = 4/10
Exception in thread "main" java.lang.StackOverflowError
2/4 - 2/6 = 0/-2
at com.xetrasu.Rational.doubleValue(Rational.java:64)
2/4 * 2/6 = 4/24
at com.xetrasu.Rational.doubleValue(Rational.java:64)
2/4 / 2/6 = 12/8
at com.xetrasu.Rational.doubleValue(Rational.java:64)
at com.xetrasu.Rational.doubleValue(Rational.java:64)
at com.xetrasu.Rational.doubleValue(Rational.java:64)
at com.xetrasu.Rational.doubleValue(Rational.java:64)
at com.xetrasu.Rational.doubleValue(Rational.java:64)
Here is the source code of StackOverflowError in OpenJDK 7.
What is a StackOverflowError, what causes it, and how should I deal with them?
Parameters and local variables are allocated on the stack (with reference types, the object lives on the heap and a variable in the stack references that object on the heap). The stack typically lives at the upper end of your address space and as it is used up it heads towards the bottom of the address space (i.e. towards zero).
Your process also has a heap, which lives at the bottom end of your process. As you allocate memory, this heap can grow towards the upper end of your address space. As you can see, there is a potential for the heap to "collide" with the stack (a bit like tectonic plates!!!).
The common cause for a stack overflow is a bad recursive call. Typically, this is caused when your recursive functions doesn't have the correct termination condition, so it ends up calling itself forever. Or when the termination condition is fine, it can be caused by requiring too many recursive calls before fulfilling it.
However, with GUI programming, it's possible to generate indirect recursion. For example, your app may be handling paint messages, and, whilst processing them, it may call a function that causes the system to send another paint message. Here you've not explicitly called yourself, but the OS/VM has done it for you.
To deal with them, you'll need to examine your code. If you've got functions that call themselves then check that you've got a terminating condition. If you have, then check that when calling the function you have at least modified one of the arguments, otherwise there'll be no visible change for the recursively called function and the terminating condition is useless. Also mind that your stack space can run out of memory before reaching a valid terminating condition, thus make sure your method can handle input values requiring more recursive calls.
If you've got no obvious recursive functions then check to see if you're calling any library functions that indirectly will cause your function to be called (like the implicit case above).
To describe this, first let us understand how local variables and objects are stored.
Local variable are stored on the stack:
If you looked at the image you should be able to understand how things are working.
When a function call is invoked by a Java application, a stack frame is allocated on the call stack. The stack frame contains the parameters of the invoked method, its local parameters, and the return address of the method. The return address denotes the execution point from which, the program execution shall continue after the invoked method returns. If there is no space for a new stack frame then, the StackOverflowError is thrown by the Java Virtual Machine (JVM).
The most common case that can possibly exhaust a Java application’s stack is recursion. In recursion, a method invokes itself during its execution. Recursion is considered as a powerful general-purpose programming technique, but it must be used with caution, to avoid StackOverflowError.
An example of throwing a StackOverflowError is shown below:
StackOverflowErrorExample.java:
public class StackOverflowErrorExample {
public static void recursivePrint(int num) {
System.out.println("Number: " + num);
if (num == 0)
return;
else
recursivePrint(++num);
}
public static void main(String[] args) {
StackOverflowErrorExample.recursivePrint(1);
}
}
In this example, we define a recursive method, called recursivePrint that prints an integer and then, calls itself, with the next successive integer as an argument. The recursion ends until we pass in 0 as a parameter. However, in our example, we passed in the parameter from 1 and its increasing followers, consequently, the recursion will never terminate.
A sample execution, using the -Xss1M flag that specifies the size of the thread stack to equal to 1 MB, is shown below:
Number: 1
Number: 2
Number: 3
...
Number: 6262
Number: 6263
Number: 6264
Number: 6265
Number: 6266
Exception in thread "main" java.lang.StackOverflowError
at java.io.PrintStream.write(PrintStream.java:480)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
at java.io.PrintStream.write(PrintStream.java:527)
at java.io.PrintStream.print(PrintStream.java:669)
at java.io.PrintStream.println(PrintStream.java:806)
at StackOverflowErrorExample.recursivePrint(StackOverflowErrorExample.java:4)
at StackOverflowErrorExample.recursivePrint(StackOverflowErrorExample.java:9)
at StackOverflowErrorExample.recursivePrint(StackOverflowErrorExample.java:9)
at StackOverflowErrorExample.recursivePrint(StackOverflowErrorExample.java:9)
...
Depending on the JVM’s initial configuration, the results may differ, but eventually the StackOverflowError shall be thrown. This example is a very good example of how recursion can cause problems, if not implemented with caution.
How to deal with the StackOverflowError
The simplest solution is to carefully inspect the stack trace and
detect the repeating pattern of line numbers. These line numbers
indicate the code being recursively called. Once you detect these
lines, you must carefully inspect your code and understand why the
recursion never terminates.
If you have verified that the recursion
is implemented correctly, you can increase the stack’s size, in
order to allow a larger number of invocations. Depending on the Java
Virtual Machine (JVM) installed, the default thread stack size may
equal to either 512 KB, or 1 MB. You can increase the thread stack
size using the -Xss flag. This flag can be specified either via the
project’s configuration, or via the command line. The format of the
-Xss argument is:
-Xss<size>[g|G|m|M|k|K]
If you have a function like:
int foo()
{
// more stuff
foo();
}
Then foo() will keep calling itself, getting deeper and deeper, and when the space used to keep track of what functions you're in is filled up, you get the stack overflow error.
Stack overflow means exactly that: a stack overflows. Usually there's a one stack in the program that contains local-scope variables and addresses where to return when execution of a routine ends. That stack tends to be a fixed memory range somewhere in the memory, therefore it's limited how much it can contain values.
If the stack is empty you can't pop, if you do you'll get stack underflow error.
If the stack is full you can't push, if you do you'll get stack overflow error.
So stack overflow appears where you allocate too much into the stack. For instance, in the mentioned recursion.
Some implementations optimize out some forms of recursions. Tail recursion in particular. Tail recursive routines are form of routines where the recursive call appears as a final thing what the routine does. Such routine call gets simply reduced into a jump.
Some implementations go so far as implement their own stacks for recursion, therefore they allow the recursion to continue until the system runs out of memory.
Easiest thing you could try would be to increase your stack size if you can. If you can't do that though, the second best thing would be to look whether there's something that clearly causes the stack overflow. Try it by printing something before and after the call into routine. This helps you to find out the failing routine.
A stack overflow is usually called by nesting function calls too deeply (especially easy when using recursion, i.e. a function that calls itself) or allocating a large amount of memory on the stack where using the heap would be more appropriate.
Like you say, you need to show some code. :-)
A stack overflow error usually happens when your function calls nest too deeply. See the Stack Overflow Code Golf thread for some examples of how this happens (though in the case of that question, the answers intentionally cause stack overflow).
A StackOverflowError is a runtime error in Java.
It is thrown when the amount of call stack memory allocated by the JVM is exceeded.
A common case of a StackOverflowError being thrown, is when the call stack exceeds due to excessive deep or infinite recursion.
Example:
public class Factorial {
public static int factorial(int n){
if(n == 1){
return 1;
}
else{
return n * factorial(n-1);
}
}
public static void main(String[] args){
System.out.println("Main method started");
int result = Factorial.factorial(-1);
System.out.println("Factorial ==>"+result);
System.out.println("Main method ended");
}
}
Stack trace:
Main method started
Exception in thread "main" java.lang.StackOverflowError
at com.program.stackoverflow.Factorial.factorial(Factorial.java:9)
at com.program.stackoverflow.Factorial.factorial(Factorial.java:9)
at com.program.stackoverflow.Factorial.factorial(Factorial.java:9)
In the above case, it can be avoided by doing programmatic changes.
But if the program logic is correct and it still occurs then your stack size needs to be increased.
StackOverflowError is to the stack as OutOfMemoryError is to the heap.
Unbounded recursive calls result in stack space being used up.
The following example produces StackOverflowError:
class StackOverflowDemo
{
public static void unboundedRecursiveCall() {
unboundedRecursiveCall();
}
public static void main(String[] args)
{
unboundedRecursiveCall();
}
}
StackOverflowError is avoidable if recursive calls are bounded to prevent the aggregate total of incomplete in-memory calls (in bytes) from exceeding the stack size (in bytes).
The most common cause of stack overflows is excessively deep or infinite recursion. If this is your problem, this tutorial about Java Recursion could help understand the problem.
Here is an example of a recursive algorithm for reversing a singly linked list. On a laptop (with the specifications 4 GB memory, Intel Core i5 2.3 GHz CPU 64 bit and Windows 7), this function will run into StackOverflow error for a linked list of size close to 10,000.
My point is that we should use recursion judiciously, always taking into account of the scale of the system.
Often recursion can be converted to iterative program, which scales better. (One iterative version of the same algorithm is given at the bottom of the page. It reverses a singly linked list of size 1 million in 9 milliseconds.)
private static LinkedListNode doReverseRecursively(LinkedListNode x, LinkedListNode first){
LinkedListNode second = first.next;
first.next = x;
if(second != null){
return doReverseRecursively(first, second);
}else{
return first;
}
}
public static LinkedListNode reverseRecursively(LinkedListNode head){
return doReverseRecursively(null, head);
}
Iterative Version of the Same Algorithm:
public static LinkedListNode reverseIteratively(LinkedListNode head){
return doReverseIteratively(null, head);
}
private static LinkedListNode doReverseIteratively(LinkedListNode x, LinkedListNode first) {
while (first != null) {
LinkedListNode second = first.next;
first.next = x;
x = first;
if (second == null) {
break;
} else {
first = second;
}
}
return first;
}
public static LinkedListNode reverseIteratively(LinkedListNode head){
return doReverseIteratively(null, head);
}
The stack has a space limit that depends on the operating system. The normal size is 8 MB (in Ubuntu (Linux), you can check that limit with $ ulimit -u and it can be checked in other OS similarly). Any program makes use of the stack at runtime, but to fully know when it is used you need to check the assembly language. In x86_64 for example, the stack is used to:
Save the return address when making a procedure call
Save local variables
Save special registers to restore them later
Pass arguments to a procedure call (more than 6)
Other: random unused stack base, canary values, padding, ... etc.
If you don't know x86_64 (normal case) you only need to know when the specific high-level programming language you are using compile to those actions. For example in C:
(1) → a function call
(2) → local variables in function calls (including main)
(3) → local variables in function calls (not main)
(4) → a function call
(5) → normally a function call, it is generally irrelevant for a stack overflow.
So, in C, only local variables and function calls make use of the stack. The two (unique?) ways of making a stack overflow are:
Declaring too large local variables in main or in any function that it's called in (int array[10000][10000];)
A very deep or infinite recursion (too many function calls at the same time).
To avoid a StackOverflowError you can:
check if local variables are too big (order of 1 MB) → use the heap (malloc/calloc calls) or global variables.
check for infinite recursion → you know what to do... correct it!
check for normal too deep recursion → the easiest approach is to just change the implementation to be iterative.
Notice also that global variables, include libraries, etc... don't make use of the stack.
Only if the above does not work, change the stack size to the maximum on the specific OS. With Ubuntu for example: ulimit -s 32768 (32 MB). (This has never been the solution for any of my stack overflow errors, but I also don't have much experience.)
I have omitted special and/or not standard cases in C (such as usage of alloc() and similar) because if you are using them you should already know exactly what you are doing.
In a crunch, the below situation will bring a stack overflow error.
public class Example3 {
public static void main(String[] args) {
main(new String[1]);
}
}
A simple Java example that causes java.lang.StackOverflowError because of a bad recursive call:
class Human {
Human(){
new Animal();
}
}
class Animal extends Human {
Animal(){
super();
}
}
public class Test01 {
public static void main(String[] args) {
new Animal();
}
}
Many answers to this question are good. However, I would like to take a slightly different approach and give some more insight into how memory works and also a (simplified) visualization to better understand StackOverflow errors. This understanding does not only apply to Java but all processes alike.
On modern systems all new processes get their own virtual address space (VAS). In essence VAS is an abstraction layer provided by the operating system on top of physical memory in order to ensure processes do not interfere with each others memory. It's the kernels job to then map the virtual addresses provided to to the actual physical addresses.
VAS can be divided into a couple of sections:
In order to let the CPU know what it is supposed to do machine instructions must be loaded into memory. This is usually referred to as the code or text segment and of static size.
On top of that one can find the data segment and heap. The data segment is of fixed size and contains global or static variables.
As a program runs into special conditions it may need to additionally allocate data, which is where the heap comes into play and is therefore able to dynamically grow in size.
The stack is located on the other side of the virtual address space and (among other things) keeps track of all function calls using a LIFO data structure. Similar to the heap a program may need additional space during runtime to keep track of new function calls being invoked. Since the stack is located on the other side of the VAS it is growing into the opposite direction i.e. towards the heap.
TL;DR
This is where the StackOverflow error comes into play.
Since the stack grows down (towards the heap) it may so happen that at some point in time it cannot grow further as it would overlap with the heap address space. Once that happens the StackOverflow error occurs.
The most common reason as to why this happens is due to a bug in the program making recursive calls that do not terminate properly.
Note that on some systems VAS may behave slightly different an can be divided into even more segments, however, this general understanding applies to all UNIX systems.
Here's an example
public static void main(String[] args) {
System.out.println(add5(1));
}
public static int add5(int a) {
return add5(a) + 5;
}
A StackOverflowError basically is when you try to do something, that most likely calls itself, and goes on for infinity (or until it gives a StackOverflowError).
add5(a) will call itself, and then call itself again, and so on.
This is a typical case of java.lang.StackOverflowError... The method is recursively calling itself with no exit in doubleValue(), floatValue(), etc.
File Rational.java
public class Rational extends Number implements Comparable<Rational> {
private int num;
private int denom;
public Rational(int num, int denom) {
this.num = num;
this.denom = denom;
}
public int compareTo(Rational r) {
if ((num / denom) - (r.num / r.denom) > 0) {
return +1;
} else if ((num / denom) - (r.num / r.denom) < 0) {
return -1;
}
return 0;
}
public Rational add(Rational r) {
return new Rational(num + r.num, denom + r.denom);
}
public Rational sub(Rational r) {
return new Rational(num - r.num, denom - r.denom);
}
public Rational mul(Rational r) {
return new Rational(num * r.num, denom * r.denom);
}
public Rational div(Rational r) {
return new Rational(num * r.denom, denom * r.num);
}
public int gcd(Rational r) {
int i = 1;
while (i != 0) {
i = denom % r.denom;
denom = r.denom;
r.denom = i;
}
return denom;
}
public String toString() {
String a = num + "/" + denom;
return a;
}
public double doubleValue() {
return (double) doubleValue();
}
public float floatValue() {
return (float) floatValue();
}
public int intValue() {
return (int) intValue();
}
public long longValue() {
return (long) longValue();
}
}
File Main.java
public class Main {
public static void main(String[] args) {
Rational a = new Rational(2, 4);
Rational b = new Rational(2, 6);
System.out.println(a + " + " + b + " = " + a.add(b));
System.out.println(a + " - " + b + " = " + a.sub(b));
System.out.println(a + " * " + b + " = " + a.mul(b));
System.out.println(a + " / " + b + " = " + a.div(b));
Rational[] arr = {new Rational(7, 1), new Rational(6, 1),
new Rational(5, 1), new Rational(4, 1),
new Rational(3, 1), new Rational(2, 1),
new Rational(1, 1), new Rational(1, 2),
new Rational(1, 3), new Rational(1, 4),
new Rational(1, 5), new Rational(1, 6),
new Rational(1, 7), new Rational(1, 8),
new Rational(1, 9), new Rational(0, 1)};
selectSort(arr);
for (int i = 0; i < arr.length - 1; ++i) {
if (arr[i].compareTo(arr[i + 1]) > 0) {
System.exit(1);
}
}
Number n = new Rational(3, 2);
System.out.println(n.doubleValue());
System.out.println(n.floatValue());
System.out.println(n.intValue());
System.out.println(n.longValue());
}
public static <T extends Comparable<? super T>> void selectSort(T[] array) {
T temp;
int mini;
for (int i = 0; i < array.length - 1; ++i) {
mini = i;
for (int j = i + 1; j < array.length; ++j) {
if (array[j].compareTo(array[mini]) < 0) {
mini = j;
}
}
if (i != mini) {
temp = array[i];
array[i] = array[mini];
array[mini] = temp;
}
}
}
}
Result
2/4 + 2/6 = 4/10
Exception in thread "main" java.lang.StackOverflowError
2/4 - 2/6 = 0/-2
at com.xetrasu.Rational.doubleValue(Rational.java:64)
2/4 * 2/6 = 4/24
at com.xetrasu.Rational.doubleValue(Rational.java:64)
2/4 / 2/6 = 12/8
at com.xetrasu.Rational.doubleValue(Rational.java:64)
at com.xetrasu.Rational.doubleValue(Rational.java:64)
at com.xetrasu.Rational.doubleValue(Rational.java:64)
at com.xetrasu.Rational.doubleValue(Rational.java:64)
at com.xetrasu.Rational.doubleValue(Rational.java:64)
Here is the source code of StackOverflowError in OpenJDK 7.
What is a StackOverflowError, what causes it, and how should I deal with them?
Parameters and local variables are allocated on the stack (with reference types, the object lives on the heap and a variable in the stack references that object on the heap). The stack typically lives at the upper end of your address space and as it is used up it heads towards the bottom of the address space (i.e. towards zero).
Your process also has a heap, which lives at the bottom end of your process. As you allocate memory, this heap can grow towards the upper end of your address space. As you can see, there is a potential for the heap to "collide" with the stack (a bit like tectonic plates!!!).
The common cause for a stack overflow is a bad recursive call. Typically, this is caused when your recursive functions doesn't have the correct termination condition, so it ends up calling itself forever. Or when the termination condition is fine, it can be caused by requiring too many recursive calls before fulfilling it.
However, with GUI programming, it's possible to generate indirect recursion. For example, your app may be handling paint messages, and, whilst processing them, it may call a function that causes the system to send another paint message. Here you've not explicitly called yourself, but the OS/VM has done it for you.
To deal with them, you'll need to examine your code. If you've got functions that call themselves then check that you've got a terminating condition. If you have, then check that when calling the function you have at least modified one of the arguments, otherwise there'll be no visible change for the recursively called function and the terminating condition is useless. Also mind that your stack space can run out of memory before reaching a valid terminating condition, thus make sure your method can handle input values requiring more recursive calls.
If you've got no obvious recursive functions then check to see if you're calling any library functions that indirectly will cause your function to be called (like the implicit case above).
To describe this, first let us understand how local variables and objects are stored.
Local variable are stored on the stack:
If you looked at the image you should be able to understand how things are working.
When a function call is invoked by a Java application, a stack frame is allocated on the call stack. The stack frame contains the parameters of the invoked method, its local parameters, and the return address of the method. The return address denotes the execution point from which, the program execution shall continue after the invoked method returns. If there is no space for a new stack frame then, the StackOverflowError is thrown by the Java Virtual Machine (JVM).
The most common case that can possibly exhaust a Java application’s stack is recursion. In recursion, a method invokes itself during its execution. Recursion is considered as a powerful general-purpose programming technique, but it must be used with caution, to avoid StackOverflowError.
An example of throwing a StackOverflowError is shown below:
StackOverflowErrorExample.java:
public class StackOverflowErrorExample {
public static void recursivePrint(int num) {
System.out.println("Number: " + num);
if (num == 0)
return;
else
recursivePrint(++num);
}
public static void main(String[] args) {
StackOverflowErrorExample.recursivePrint(1);
}
}
In this example, we define a recursive method, called recursivePrint that prints an integer and then, calls itself, with the next successive integer as an argument. The recursion ends until we pass in 0 as a parameter. However, in our example, we passed in the parameter from 1 and its increasing followers, consequently, the recursion will never terminate.
A sample execution, using the -Xss1M flag that specifies the size of the thread stack to equal to 1 MB, is shown below:
Number: 1
Number: 2
Number: 3
...
Number: 6262
Number: 6263
Number: 6264
Number: 6265
Number: 6266
Exception in thread "main" java.lang.StackOverflowError
at java.io.PrintStream.write(PrintStream.java:480)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
at java.io.PrintStream.write(PrintStream.java:527)
at java.io.PrintStream.print(PrintStream.java:669)
at java.io.PrintStream.println(PrintStream.java:806)
at StackOverflowErrorExample.recursivePrint(StackOverflowErrorExample.java:4)
at StackOverflowErrorExample.recursivePrint(StackOverflowErrorExample.java:9)
at StackOverflowErrorExample.recursivePrint(StackOverflowErrorExample.java:9)
at StackOverflowErrorExample.recursivePrint(StackOverflowErrorExample.java:9)
...
Depending on the JVM’s initial configuration, the results may differ, but eventually the StackOverflowError shall be thrown. This example is a very good example of how recursion can cause problems, if not implemented with caution.
How to deal with the StackOverflowError
The simplest solution is to carefully inspect the stack trace and
detect the repeating pattern of line numbers. These line numbers
indicate the code being recursively called. Once you detect these
lines, you must carefully inspect your code and understand why the
recursion never terminates.
If you have verified that the recursion
is implemented correctly, you can increase the stack’s size, in
order to allow a larger number of invocations. Depending on the Java
Virtual Machine (JVM) installed, the default thread stack size may
equal to either 512 KB, or 1 MB. You can increase the thread stack
size using the -Xss flag. This flag can be specified either via the
project’s configuration, or via the command line. The format of the
-Xss argument is:
-Xss<size>[g|G|m|M|k|K]
If you have a function like:
int foo()
{
// more stuff
foo();
}
Then foo() will keep calling itself, getting deeper and deeper, and when the space used to keep track of what functions you're in is filled up, you get the stack overflow error.
Stack overflow means exactly that: a stack overflows. Usually there's a one stack in the program that contains local-scope variables and addresses where to return when execution of a routine ends. That stack tends to be a fixed memory range somewhere in the memory, therefore it's limited how much it can contain values.
If the stack is empty you can't pop, if you do you'll get stack underflow error.
If the stack is full you can't push, if you do you'll get stack overflow error.
So stack overflow appears where you allocate too much into the stack. For instance, in the mentioned recursion.
Some implementations optimize out some forms of recursions. Tail recursion in particular. Tail recursive routines are form of routines where the recursive call appears as a final thing what the routine does. Such routine call gets simply reduced into a jump.
Some implementations go so far as implement their own stacks for recursion, therefore they allow the recursion to continue until the system runs out of memory.
Easiest thing you could try would be to increase your stack size if you can. If you can't do that though, the second best thing would be to look whether there's something that clearly causes the stack overflow. Try it by printing something before and after the call into routine. This helps you to find out the failing routine.
A stack overflow is usually called by nesting function calls too deeply (especially easy when using recursion, i.e. a function that calls itself) or allocating a large amount of memory on the stack where using the heap would be more appropriate.
Like you say, you need to show some code. :-)
A stack overflow error usually happens when your function calls nest too deeply. See the Stack Overflow Code Golf thread for some examples of how this happens (though in the case of that question, the answers intentionally cause stack overflow).
A StackOverflowError is a runtime error in Java.
It is thrown when the amount of call stack memory allocated by the JVM is exceeded.
A common case of a StackOverflowError being thrown, is when the call stack exceeds due to excessive deep or infinite recursion.
Example:
public class Factorial {
public static int factorial(int n){
if(n == 1){
return 1;
}
else{
return n * factorial(n-1);
}
}
public static void main(String[] args){
System.out.println("Main method started");
int result = Factorial.factorial(-1);
System.out.println("Factorial ==>"+result);
System.out.println("Main method ended");
}
}
Stack trace:
Main method started
Exception in thread "main" java.lang.StackOverflowError
at com.program.stackoverflow.Factorial.factorial(Factorial.java:9)
at com.program.stackoverflow.Factorial.factorial(Factorial.java:9)
at com.program.stackoverflow.Factorial.factorial(Factorial.java:9)
In the above case, it can be avoided by doing programmatic changes.
But if the program logic is correct and it still occurs then your stack size needs to be increased.
StackOverflowError is to the stack as OutOfMemoryError is to the heap.
Unbounded recursive calls result in stack space being used up.
The following example produces StackOverflowError:
class StackOverflowDemo
{
public static void unboundedRecursiveCall() {
unboundedRecursiveCall();
}
public static void main(String[] args)
{
unboundedRecursiveCall();
}
}
StackOverflowError is avoidable if recursive calls are bounded to prevent the aggregate total of incomplete in-memory calls (in bytes) from exceeding the stack size (in bytes).
The most common cause of stack overflows is excessively deep or infinite recursion. If this is your problem, this tutorial about Java Recursion could help understand the problem.
Here is an example of a recursive algorithm for reversing a singly linked list. On a laptop (with the specifications 4 GB memory, Intel Core i5 2.3 GHz CPU 64 bit and Windows 7), this function will run into StackOverflow error for a linked list of size close to 10,000.
My point is that we should use recursion judiciously, always taking into account of the scale of the system.
Often recursion can be converted to iterative program, which scales better. (One iterative version of the same algorithm is given at the bottom of the page. It reverses a singly linked list of size 1 million in 9 milliseconds.)
private static LinkedListNode doReverseRecursively(LinkedListNode x, LinkedListNode first){
LinkedListNode second = first.next;
first.next = x;
if(second != null){
return doReverseRecursively(first, second);
}else{
return first;
}
}
public static LinkedListNode reverseRecursively(LinkedListNode head){
return doReverseRecursively(null, head);
}
Iterative Version of the Same Algorithm:
public static LinkedListNode reverseIteratively(LinkedListNode head){
return doReverseIteratively(null, head);
}
private static LinkedListNode doReverseIteratively(LinkedListNode x, LinkedListNode first) {
while (first != null) {
LinkedListNode second = first.next;
first.next = x;
x = first;
if (second == null) {
break;
} else {
first = second;
}
}
return first;
}
public static LinkedListNode reverseIteratively(LinkedListNode head){
return doReverseIteratively(null, head);
}
The stack has a space limit that depends on the operating system. The normal size is 8 MB (in Ubuntu (Linux), you can check that limit with $ ulimit -u and it can be checked in other OS similarly). Any program makes use of the stack at runtime, but to fully know when it is used you need to check the assembly language. In x86_64 for example, the stack is used to:
Save the return address when making a procedure call
Save local variables
Save special registers to restore them later
Pass arguments to a procedure call (more than 6)
Other: random unused stack base, canary values, padding, ... etc.
If you don't know x86_64 (normal case) you only need to know when the specific high-level programming language you are using compile to those actions. For example in C:
(1) → a function call
(2) → local variables in function calls (including main)
(3) → local variables in function calls (not main)
(4) → a function call
(5) → normally a function call, it is generally irrelevant for a stack overflow.
So, in C, only local variables and function calls make use of the stack. The two (unique?) ways of making a stack overflow are:
Declaring too large local variables in main or in any function that it's called in (int array[10000][10000];)
A very deep or infinite recursion (too many function calls at the same time).
To avoid a StackOverflowError you can:
check if local variables are too big (order of 1 MB) → use the heap (malloc/calloc calls) or global variables.
check for infinite recursion → you know what to do... correct it!
check for normal too deep recursion → the easiest approach is to just change the implementation to be iterative.
Notice also that global variables, include libraries, etc... don't make use of the stack.
Only if the above does not work, change the stack size to the maximum on the specific OS. With Ubuntu for example: ulimit -s 32768 (32 MB). (This has never been the solution for any of my stack overflow errors, but I also don't have much experience.)
I have omitted special and/or not standard cases in C (such as usage of alloc() and similar) because if you are using them you should already know exactly what you are doing.
In a crunch, the below situation will bring a stack overflow error.
public class Example3 {
public static void main(String[] args) {
main(new String[1]);
}
}
A simple Java example that causes java.lang.StackOverflowError because of a bad recursive call:
class Human {
Human(){
new Animal();
}
}
class Animal extends Human {
Animal(){
super();
}
}
public class Test01 {
public static void main(String[] args) {
new Animal();
}
}
Many answers to this question are good. However, I would like to take a slightly different approach and give some more insight into how memory works and also a (simplified) visualization to better understand StackOverflow errors. This understanding does not only apply to Java but all processes alike.
On modern systems all new processes get their own virtual address space (VAS). In essence VAS is an abstraction layer provided by the operating system on top of physical memory in order to ensure processes do not interfere with each others memory. It's the kernels job to then map the virtual addresses provided to to the actual physical addresses.
VAS can be divided into a couple of sections:
In order to let the CPU know what it is supposed to do machine instructions must be loaded into memory. This is usually referred to as the code or text segment and of static size.
On top of that one can find the data segment and heap. The data segment is of fixed size and contains global or static variables.
As a program runs into special conditions it may need to additionally allocate data, which is where the heap comes into play and is therefore able to dynamically grow in size.
The stack is located on the other side of the virtual address space and (among other things) keeps track of all function calls using a LIFO data structure. Similar to the heap a program may need additional space during runtime to keep track of new function calls being invoked. Since the stack is located on the other side of the VAS it is growing into the opposite direction i.e. towards the heap.
TL;DR
This is where the StackOverflow error comes into play.
Since the stack grows down (towards the heap) it may so happen that at some point in time it cannot grow further as it would overlap with the heap address space. Once that happens the StackOverflow error occurs.
The most common reason as to why this happens is due to a bug in the program making recursive calls that do not terminate properly.
Note that on some systems VAS may behave slightly different an can be divided into even more segments, however, this general understanding applies to all UNIX systems.
Here's an example
public static void main(String[] args) {
System.out.println(add5(1));
}
public static int add5(int a) {
return add5(a) + 5;
}
A StackOverflowError basically is when you try to do something, that most likely calls itself, and goes on for infinity (or until it gives a StackOverflowError).
add5(a) will call itself, and then call itself again, and so on.
This is a typical case of java.lang.StackOverflowError... The method is recursively calling itself with no exit in doubleValue(), floatValue(), etc.
File Rational.java
public class Rational extends Number implements Comparable<Rational> {
private int num;
private int denom;
public Rational(int num, int denom) {
this.num = num;
this.denom = denom;
}
public int compareTo(Rational r) {
if ((num / denom) - (r.num / r.denom) > 0) {
return +1;
} else if ((num / denom) - (r.num / r.denom) < 0) {
return -1;
}
return 0;
}
public Rational add(Rational r) {
return new Rational(num + r.num, denom + r.denom);
}
public Rational sub(Rational r) {
return new Rational(num - r.num, denom - r.denom);
}
public Rational mul(Rational r) {
return new Rational(num * r.num, denom * r.denom);
}
public Rational div(Rational r) {
return new Rational(num * r.denom, denom * r.num);
}
public int gcd(Rational r) {
int i = 1;
while (i != 0) {
i = denom % r.denom;
denom = r.denom;
r.denom = i;
}
return denom;
}
public String toString() {
String a = num + "/" + denom;
return a;
}
public double doubleValue() {
return (double) doubleValue();
}
public float floatValue() {
return (float) floatValue();
}
public int intValue() {
return (int) intValue();
}
public long longValue() {
return (long) longValue();
}
}
File Main.java
public class Main {
public static void main(String[] args) {
Rational a = new Rational(2, 4);
Rational b = new Rational(2, 6);
System.out.println(a + " + " + b + " = " + a.add(b));
System.out.println(a + " - " + b + " = " + a.sub(b));
System.out.println(a + " * " + b + " = " + a.mul(b));
System.out.println(a + " / " + b + " = " + a.div(b));
Rational[] arr = {new Rational(7, 1), new Rational(6, 1),
new Rational(5, 1), new Rational(4, 1),
new Rational(3, 1), new Rational(2, 1),
new Rational(1, 1), new Rational(1, 2),
new Rational(1, 3), new Rational(1, 4),
new Rational(1, 5), new Rational(1, 6),
new Rational(1, 7), new Rational(1, 8),
new Rational(1, 9), new Rational(0, 1)};
selectSort(arr);
for (int i = 0; i < arr.length - 1; ++i) {
if (arr[i].compareTo(arr[i + 1]) > 0) {
System.exit(1);
}
}
Number n = new Rational(3, 2);
System.out.println(n.doubleValue());
System.out.println(n.floatValue());
System.out.println(n.intValue());
System.out.println(n.longValue());
}
public static <T extends Comparable<? super T>> void selectSort(T[] array) {
T temp;
int mini;
for (int i = 0; i < array.length - 1; ++i) {
mini = i;
for (int j = i + 1; j < array.length; ++j) {
if (array[j].compareTo(array[mini]) < 0) {
mini = j;
}
}
if (i != mini) {
temp = array[i];
array[i] = array[mini];
array[mini] = temp;
}
}
}
}
Result
2/4 + 2/6 = 4/10
Exception in thread "main" java.lang.StackOverflowError
2/4 - 2/6 = 0/-2
at com.xetrasu.Rational.doubleValue(Rational.java:64)
2/4 * 2/6 = 4/24
at com.xetrasu.Rational.doubleValue(Rational.java:64)
2/4 / 2/6 = 12/8
at com.xetrasu.Rational.doubleValue(Rational.java:64)
at com.xetrasu.Rational.doubleValue(Rational.java:64)
at com.xetrasu.Rational.doubleValue(Rational.java:64)
at com.xetrasu.Rational.doubleValue(Rational.java:64)
at com.xetrasu.Rational.doubleValue(Rational.java:64)
Here is the source code of StackOverflowError in OpenJDK 7.
The 2 following versions of the same function (which basically tries to recover a password by brute force) do not give same performance:
Version 1:
private static final char[] CHARS = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789".toCharArray();
private static final int N_CHARS = CHARS.length;
private static final int MAX_LENGTH = 8;
private static char[] recoverPassword()
{
char word[];
int refi, i, indexes[];
for (int length = 1; length <= MAX_LENGTH; length++)
{
refi = length - 1;
word = new char[length];
indexes = new int[length];
indexes[length - 1] = 1;
while(true)
{
i = length - 1;
while ((++indexes[i]) == N_CHARS)
{
word[i] = CHARS[indexes[i] = 0];
if (--i < 0)
break;
}
if (i < 0)
break;
word[i] = CHARS[indexes[i]];
if (isValid(word))
return word;
}
}
return null;
}
Version 2:
private static final char[] CHARS = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789".toCharArray();
private static final int N_CHARS = CHARS.length;
private static final int MAX_LENGTH = 8;
private static char[] recoverPassword()
{
char word[];
int refi, i, indexes[];
for (int length = 1; length <= MAX_LENGTH; length++)
{
refi = length - 1;
word = new char[length];
indexes = new int[length];
indexes[length - 1] = 1;
while(true)
{
i = refi;
while ((++indexes[i]) == N_CHARS)
{
word[i] = CHARS[indexes[i] = 0];
if (--i < 0)
break;
}
if (i < 0)
break;
word[i] = CHARS[indexes[i]];
if (isValid(word))
return word;
}
}
return null;
}
I would expect version 2 to be faster, as it does (and that is the only difference):
i = refi;
...as compare to version 1:
i = length -1;
However, it's the opposite: version 1 is faster by over 3%!
Does someone knows why? Is that due to some optimization done by the compiler?
Thank you all for your answers that far.
Just to add that the goal is actually not to optimize this piece of code (which is already quite optimized), but more to understand, from a compiler / CPU / architecture perspective, what could explain such performance difference.
Your answers have been very helpful, thanks again!
Key
It is difficult to check this in a micro-benchmark because you cannot say for sure how the code has been optimised without reading the machine code generated, even then the CPU can do plenty of tricks to optimise it future eg. it turns the x86 code in RISC style instructions to actually execute.
A computation takes as little as one cycle and the CPU can perform up to three of them at once. An access to L1 cache takes 4 cycles and for L2, L3, main memory it takes 11, 40-75, 200 cycles.
Storing values to avoid a simple calculation is actually slower in many cases. BTW using division and modulus is quite expensive and caching this value can be worth it when micro-tuning your code.
The correct answer should be retrievable by a deassembler (i mean .class -> .java converter),
but my guess is that the compiler might have decided to get rid of iref altogether and decided to store length - 1 an auxiliary register.
I'm more of a c++ guy, but I would start by trying:
const int refi = length - 1;
inside the for loop. Also you should probably use
indexes[ refi ] = 1;
Comparing running times of codes does not give exact or quarantine results
First of all, it is not the way comparing performances like this. A running time analysis is needed here. Both 2 codes have same loop structure and their running time are the same. You may have different running times when you run codes. However, they mostly differ with cache hits, I/O times, thread & process schedules. There is no quarantine that code is always completed in a exact time.
However, there is still differences in your code, to understand the difference you should look into your CPU architecture. I can explain it according to x86 architecture basically.
What happens behind the scenes?
i = refi;
CPU takes refi and i to its registers from ram. there is 2 access to ram if the values in not in the cache. and value of i will be written to the ram. However, it always takes different times according to thread & process schedules. Furrhermore, if the values are in virtual memory it wil take longer time.
i = length -1;
CPU also access i and length from ram or cache. there is same number of accesses. In addition, there is a subtraction here which means extra CPU cycles. That is why you think this one take longer time to complete. It is expected, but the issues that i mentioned above explain why this take longer time.
Summation
As i explain this is not the way of comparing performance. I think, there is no real difference between these codes. There are lots of optimizations inside CPU and also in compiler. You can see optimized codes if you decompile .class files.
My advice is it is better to minimize BigO running time analysis. If you find better algorithms it is the best way of optimizing codes. In case you still have bottlenecks in your code, you may try micro-benchmarking.
See also
Analysis of algorithms
Big O notation
Microprocessor
Compiler optimization
CPU Scheduling
To start with, you can't really compare the performance by just running your program - micro benchmarking in Java is complicated.
Also, a subtraction on modern CPUs can take as little as a third of a clock cycle on average. On a 3GHz CPU, that is 0.1 nanoseconds. And nothing tells you that the subtraction actually happens as the compiler might have modified the code.
So:
You should try to check the generated assembly code.
If you really care about the performance, create an appropriate micro-benchmark.
I have a problem with my Java progam suddenly exiting, without any exception thrown or the program finishing normally.
I'm writing a program to solve Project Euler's 14th problem. This is what I got:
private static final int INITIAL_CACHE_SIZE = 30000;
private static Map<Long, Integer> cache = new HashMap<Long, Integer>(INITIAL_CACHE_SIZE);
public void main(String... args) {
long number = 0;
int maxSize = 0;
for (long i = 1; i <= TARGET; i++) {
int size = size(i);
if (size > maxSize) {
maxSize = size;
number = i;
}
}
}
private static int size(long i) {
if (i == 1L) {
return 1;
}
final int size = size(process(i)) + 1;
return size;
}
private static long process(long n) {
return n % 2 == 0 ? n/2 : 3*n + 1;
}
This runs fine, and finishes correctly in about 5 seconds when using a TARGET of 1 000 000.
I wanted to optimize by adding a cache, so I changed the size method to this:
private static int size(long i) {
if (i == 1L) {
return 1;
}
if (cache.containsKey(i)) {
return cache.get(i);
}
final int size = size(process(i)) + 1;
cache.put(i, size);
return size;
}
Now when I run it, it simply stops (process exits) when I get to 555144. Same number every time. No exception, error, Java VM crash or anything is thrown.
Changing the cache size doesn't seem to have any effect either, so how could the cache
introduction cause this error?
If I enforce the cache size to be not just initial, but permanent like so:
if (i < CACHE_SIZE) {
cache.put(i, size);
}
the bug no longer occurs.
Edit: When I set the cache size to like 2M, the bug starts showing again.
Can anyone reproduce this, and maybe even provide a suggestion as to why it happens?
This is simply an OutOfMemoryError that is not being printed. The program runs fine if I set a high heap size, otherwise it exits with an unlogged OutOfMemoryError (easy to see in a Debugger, though).
You can verify this and get a heap dump (as well as printout that an OutOfMemoryError occurred) by passing this JVM arg and re-running your program:
-XX:+HeapDumpOnOutOfMemoryError
With this it will then print out something to this effect:
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid4192.hprof ...
Heap dump file created [91901809 bytes in 4.464 secs]
Bump up your heap size with, say, -Xmx200m and you won't have an issue - At least for TARGET=1000000.
It sounds like the JVM itself crashes (that is the first thought when your program dies without a hint of an exception anyway). The first step in such a problem is to upgrade to the latest revision for your platform. The JVM should dump the heap to a .log file in the directory where you started the JVM, assuming your user level has access rights to that directory.
That being said, some OutOfMemory errors don't report in the main thread, so unless you do a try/catch (Throwable t) and see if you get one, it is hard to be sure you aren't actually just running out of memory. The fact that it only uses 100MB could just mean that the JVM isn't configured to use more. That can be changed by changing the startup options to the JVM to -Xmx1024m to get a Gig of memory, to see if the problem goes anywhere.
The code for doing the try catch should be something like this:
public static void main(String[] args) {
try {
MyObject o = new MyObject();
o.process();
} catch (Throwable t) {
t.printStackTrace();
}
}
And do everything in the process method and do not store your cache in statics, that way if the error happens at the catch statement the object is out of scope and can be garbage collected, freeing enough memory to allow the printing of the stack trace. No guarantees that that works, but it gives it a better shot.
One significant difference between the two implmentations of size(long i) is in the amount of objects you are creating.
In the first implementation, there are no Objects being created. In the second you are doing an awful lot of autoboxing, creating a new Long for each access of your cache, and putting in new Longs and new Integers on each modification.
This would explain the increase in memory usage, but not the absence of an OutOfMemoryError. Increasing the heap does allows it to complete for me.
From this Sun aritcle:
The performance ... is likely to be poor, as it boxes or unboxes on every get or set operation. It is plenty fast enough for occasional use, but it would be folly to use it in a performance critical inner loop.
If your java process suddenly crashes it could be some resource got maxed out. Like memory. You could try setting a higher max heap
Do you see a Heap Dump being generated after the crash? This file should be in the current directory for your JVM, that's where I would look for more info.
I am getting an OutOfMemory error on cache.put(i, size);
To get the error run your program in eclipse using debug mode it will appear in the debug window. It does not produce a stack trace in the console.
The recursive size() method is probably not a good place to do the caching. I put a call to cache.put(i, size); inside the main()'s for-loop and it works much more quickly. Otherwise, I also get an OOM error (no more heap space).
Edit: Here's the source - the cache retrieval is in size(), but the storing is done in main().
public static void main(String[] args) {
long num = 0;
int maxSize = 0;
long start = new Date().getTime();
for (long i = 1; i <= TARGET; i++) {
int size = size(i);
if (size >= maxSize) {
maxSize = size;
num = i;
}
cache.put(i, size);
}
long computeTime = new Date().getTime() - start;
System.out.println(String.format("maxSize: %4d on initial starting number %6d", maxSize, num));
System.out.println("compute time in milliseconds: " + computeTime);
}
private static int size(long i) {
if (i == 1l) {
return 1;
}
if (cache.containsKey(i)) {
return cache.get(i);
}
return size(process(i)) + 1;
}
Note that by removing the cache.put() call from size(), it does not cache every computed size, but it also avoids re-caching a previously computed size. This does not affect the hashmap operations, but like akf points out, it avoids the autoboxing/unboxing operations which is where your heap killer is coming from. I also tried a "if (!containsKey(i)) { cache.put() etc" in size() but that unfortunately also runs out of memory.