Develop Reference

Why does JVM need a maximum memory capacity? [duplicate]

In the spirit of question Java: Why does MaxPermSize exist?, I'd like to ask why the Oracle JVM uses a fixed upper limit for the size of its memory allocation pool.
The default is 1/4 of your physical RAM (with upper & lower limit); as a consequence, if you have a memory-hungry application you have to manually change the limit (parameter -Xmx), or your app will perform poorly, possible even crash with an OutOfMemoryError.
Why does this fixed limit even exist? Why does the JVM not allocate memory as needed, like native programs do on most operating systems?
This would solve a whole class of common problems with Java software (just Google to see how many hints there are on the net on solving problems by setting -Xmx).
Edit:
Some answers point out that this will protect the rest of the system from a Java program with a run-away memory leak; without the limit this would bring the whole system down by exhausting all memory. This is true. However, it is equally true for any other program, and modern OSes already let you limit the maximum memory for a programm (Linux ulimit, Windows "Job Objects"). So this does not really answer the question, which is "Why does the JVM do it differently from most other programs / runtime environments?".

Why does this fixed limit even exist? Why does the JVM not allocate memory as needed, like native programs do on most operating systems?
The reason is NOT that the GC needs to know before hand what the maximum heap size can be. The JVM is clearly capable of expanding its heap ... up to the maximum ... and I'm sure it would be a relatively small change to remove that maximum. (After all, other Java implementations do this.) And it would equally be possible to have a simple way to say "use as much memory as you like" to the JVM.
I'm sure that the real reason is to protect the host operating system against the effects of faulty Java applications using all available memory. Running with an unbounded heap is potentially dangerous.
Basically, many operating systems (e.g. Windows, Linux) suffer serious performance degradation if some application tries to use all available memory. On Linux for example, the system may thrash badly, resulting in everything on the system running incredibly slowly. In the worst case, the system won't be able to start new processes, and existing processes may start crashing when the operating system refuses their (legitimate) requests for more memory. Often, the only option is to reboot.
If the JVM ran with an unbounded heap by default, any time someone ran a Java program with a storage leak ... or that simply tried to use too much memory ... they would risk bringing down the entire operating system.
In summary, having a default heap bound is a good thing because:
it protects the health of your system,
it encourages developers / users to think about memory usage by "hungry" applications, and
it potentially allows GC optimizations. (As suggested by other answers: it is plausible, but I cannot confirm this.)
EDIT
In response to the comments:
It doesn't really matter why Sun's JVMs live within a bounded heap, where other applications don't. They do, and advantages of doing so are (IMO) clear. Perhaps a more interesting question is why other managed languages don't put a bound on their heaps by default.
The -Xmx and ulimit approaches are qualitatively different. In the former case, the JVM has full knowledge of the limits it is running under and gets a chance to manage its memory usage accordingly. In the latter case, the first thing a typical C application knows about it is when a malloc call fails. The typical response is to exit with an error code (if the program checks the malloc result), or die with a segmentation fault. OK, a C application could in theory keep track of how much memory it has used, and try to respond to an impending memory crisis. But it would be hard work.
The other thing that is different about Java and C/C++ applications is that the former tend to be both more complicated and longer running. In practice, this means that Java applications are more likely to suffer from slow leaks. In the C/C++ case, the fact that memory management is harder means that developers don't attempt to build single applications of that complexity. Rather, they are more likely to build (say) a complex service by having a listener process fork of child processes to do stuff ... and then exit. This naturally mitigates the effect of memory leaks in the child process.
The idea of a JVM responding "adaptively" to requests from the OS to give memory back is interesting. But there is a BIG problem. In order to give a segment of memory back, the JVM first has to clear out any reachable objects in the segment. Typically that means running the garbage collector. But running the garbage collector is the last thing you want to do if the system is in a memory crisis ... because it is pretty much guaranteed to generate a burst of virtual memory paging.

Hm, I'll try summarizing the answers so far.
There is no technical reason why the JVM needs to have a hard limit for its heap size. It could have been implemented without one, and in fact many other dynamic languages do not have this.
Therefore, giving the JVM a heap size limit was simply a design decision by the implementors. Second-guessing why this was done is a bit difficult, and there may not be a single reason. The most likely reason is that it helps protect a system from a Java program with a memory leak, which might otherwise exhaust all RAM and cause other apps to crash or the system to thrash.
Sun could have omitted the feature and simply told people to use the OS-native resource limiting mechanisms, but they probably wanted to always have a limit, so they implemented it themselves.
At any rate, the JVM needs to be aware of any such limit (to adapt its GC strategy), so using an OS-native mechanism would not have saved much programming effort.
Also, there is one reason why such a built-in limit is more important for the JVM than for a "normal" program without GC (such as a C/C++ program):
Unlike a program with manual memory management, a program using GC does not really have a well-defined memory requirement, even with fixed input data. It only has a minimum requirement, i.e. the sum of the sizes of all objects that are actually live (reachable) at a given point in time. However, in practice a program will need additional memory to hold dead, but not yet GCed objects, because the GC cannot collect every object right away, as that would cause too much GC overhead. So GC only kicks in from time to time, and therefore some "breathing room" is required on the heap, where dead objects can await the GC.
This means that the memory required for a program using GC is really a compromise between saving memory and having good througput (by letting the GC run less often). So in some cases it may make sense to set the heap limit lower than what the JVM would use if it could, so save RAM at the expense of performance. To do this, there needs to be a way to set a heap limit.

I think part of it has to do with the implementation of the Garbage Collector (GC). The GC is typically lazy, meaning it will only start really trying to reclaim memory internally when the heap is at its maximum size. If you didn't set an upper limit, the runtime would happily continue to inflate until it used every available bit of memory on your system.
That's because from the application's perspective, it's more performant to take more resources than exert effort to use the resources you already have to full utilization. This tends to make sense for a lot of (if not most) uses of Java, which is a server setting where the application is literally the only thing that matters on the server. It tends to be slightly less ideal when you're trying to implement a client in Java, which will run amongst dozens of other applications at the same time.
Remember that with native programs, the programmer typically requests but also explicitly cleans up resources. That isn't typically true with environments who do automatic memory management.

It is due to the design of the JVM. Other JVM's (like the one from Microsoft and some IBM ones) can use all the memory available in the system if needed, without an arbitrary limit.
I believe it allows for GC-optimizations.

I think that the upper limit for memory is is linked to the fact that JVM is a VM.
As any physical machine has a given (fixed) ammount of RAM so the VM has one.
The maximal size makes the JVM easier to manage by the operating system and ensures some performance gains(less swapping).
Sun' JVM also works in quite limited hardware architecture(embedded ARM systems) and there the management of resources is crucial.

One answer that no-one above gave is that the JVM uses both heap and non-heap memory pools. Putting an upper limit on the heap defines not only how much memory is available for the heap memory pools, but it also defines how much memory is available for NON-HEAP usages. I suppose that the JVM could just allocate non-heap at the top of virtual memory and heap at the bottom of virtual memory and grow both toward each other.
Non-heap memory includes the DLLs or SOs that comprise the JVM and any native code being used as well as compiled Java code, thread stacks, native objects, PermGen (meta-data about compiled classes), among other uses. I've seen Java programs crash because so much memory was given to the heap that the application ran out of non-heap memory. This is where I learned that it can be important to reserve memory for non-heap usages by not setting the heap to be too large.
This makes a much bigger difference in a 32-bit world where an application often has only 2GB of virtual address space than it does in a 64-bit world, of course.

Would it not make more sense to separate the upper bound that triggers GC and the maximum that can be allocated ? Once the memory allocated hits the upper-bound, GC can kick in and release some memory to the free pool.
sort of like how I clean my desk that I share with my co-worker. I have a large desk, and my threshold of how much junk I can tolerate on the table is much less than the size of my desk. I don't need to have fill up every available inch before I garbage collect.
I could also return some of the desk space that I using to my co-worker, who is sharing my desk....I understand jvms don't return memory back to the system after they've allocated it to themselves, but it does not have to be that way no ?

It does allocate memory as needed, up to -Xmx ;)
One reason I can think of is that once the JVM allocates an amount of memory for its heap, it will never let it go. So if your heap has no upper bound, the JVM may just grab all the free memory on the system and then never let it go.
The upper bound also tells the JVM when it needs to do a full garbage collection. If your app is still under the upper bound, the JVM will postpone garbage collection and let the memory footprint of your application grow.
Native programs can die due to out of memory errors as well since native applications also have a memory limit: the memory available on the system - the memory already held by other applications.
The JVM also needs a contiguous block of system memory in order for garbage collection to be performed efficiently.
EDIT
Contiguous memory claim or here
The JVM will apparently let some memory go, but it is rare with the default configuration.

How can I disable Java garbage collector?

We have a PHP webapp that calls a Java binary to produce a PDF report (with JasperReports). The Java binary outputs the PDF to standard output and exits; the PHP then sends the PDF to browser. This Java command lasts about 3 to 6 seconds, and I think when it lasts 6 second it's because the GC kicks in. I would like to disable the GC because anyway when the command exits all memory is returned.
I would like to know how to disable it for Java 1.4.2 and for Java 1.6.0 because we are currently testing both JVM to see which performs faster..

It sounds like you are trying to save time, but going about it the wrong way. The time saved in disabling garbage collection would be trivial (for a single task) compared to the time taken to launch and shutdown the java process. You might want to consider having a java process launch that you can ask multiple times to do the work you require if run-time performance is your goal.

There is no way to disable garbage collection entirely. Garbage collection is only run when the JVM runs out of space, so you could give the program more memory. Add these command line options to the Java command
-Xmx256M -Xms256M
This gives the program 256Mb of ram (the default is 64Mb). Garbage collection will not take 3 seconds for a default size JVM though, so you might want to investigate more closely what the program is doing. Yourkit profiler is very useful for figuring out what is taking a long time.

GC only kicks in when JVM is short on memory, so you either GC or die. Try turning on verbose GC and see if it actually takes significant amount of time.
java -verbose:gc

Java 11 comes with an no-op garbage collector.
It can be enabled by the -XX:+UseEpsilonGC option at JVM start.
According to the JEP decription one of its goals is to make certain short-lived jobs more efficient, what might be you use case:
Extremely short lived jobs. A short-lived job might rely on exiting quickly to free the resources (e.g. heap memory). In this case, accepting the GC cycle to futilely clean up the heap is a waste of time, because the heap would be freed on exit anyway. Note that the GC cycle might take a while, because it would depend on the amount of live data in the heap, which can be a lot.

Java 11 gives you the binary option to either have Java GC on, or have Java GC turned off. To turn off Java GC you use the Epsilon Garbage Collector which must be turned off on the command line. On Java 11, use the following two JVM arguments:
-XX:+UnlockExperimentalVMOptions -XX:+UseEpsilonGC
Without the UnlockExperimentalVMOptions argument, the JVM will fail to start, so make sure it's included.
Can't Stop Java GC
Unfortunately, if you're not using Epsilon GC, there is no way to disable, stop or prevent garbage collection from happening. Just like not being able to trigger GC, you can't stop Java GC either. The algorithms are non-deterministic. Only the JVM can control when they occur.

You can use the -Xmx option to set the maximum heap size; using a larger heap should prevent the VM from runnning out of memory and, thereby, requiring garbage collection so soon.

Are you sure that it is garbage collection causing the slowdown? Have you run java with -verbose:gc to see what is happening?
You cannot disable garbage collection on the JVM. You could however look at tuning the garbage collector for better performance.

Contrary to what everyone else has said, there is a way to suspend GC, though it's very convoluted.
If you call a native function via JNI, in between the native code calling GetPrimitiveArrayCritical and ReleasePrimitiveArrayCritical, GC is suspended. It has to do this because it is the mechanism used for sharing memory between Java and native code, and GC may move objects around in memory.
So to utilise this you'd need to create a JNI function which calls the former, then polls a flag written to that critical array (which could just be a byte[1]) waiting till it sees the flag before calling the latter. When the Java code wishes to suspend GC it would call the JNI method and when it wishes to resume GC, set the aforementioned flag (I believe reads/writes to the critical array are volatile, so the native code would see the flag immediately).
Now I'm not saying this is a good idea, and certainly not the correct solution to the OP's problem. But if you absolutely had to temporarily suspend GC for some reason (perhaps you wish to manipulate raw memory via sun.misc.Unsafe and needed to ensure objects were not moved about by GC whilst doing so), that's how you could achieve it.

As everyone as said you can't disable GC in the JVM, which makes sense right, because if you could there'd be memory leaks due to java not having an explicit way for the developer to delete the heap data.
Do you have access to the source of this java binary? If so it might be worth analysing it and seeing if there's any bottle-necks which could be better written to cut down on GC activity. This could be done with most java profilers, like JProbe for example.

To avoid garbage collector release a variable or property from any object,
you must set this property (released by gc) as static in your class
it was my solution.
Example:
private static String myProperty;

Java memory usages

I cannot understand the Java memory usage. I have an application which is executed with maximum memory size set to 256M. Yet, at some point in time I can see that according to the task manager it takes up to 700MB!
Needless to say, all the rest of the applications are a bit unresponsive when this happens as they are probably swapped out.
It's JDK 1.6 on WinXP. Any ideas ?

The memory configured is available to the application. It won't include
the JVM size
the jars/libs loaded in
native libraries and related allocated memory
which will result in a much bigger image. Note that due to how the OS and the JVM work that 700Mb may be shared between multiple JVMs (due to shared binary images, shared libraries etc.)

The amount you specify with -Xmx is only for the user accessible heap - the space in which you create runtime objects dynamically.
The Java process will usea lot more space for its own needs, including the JVM, the program and other libraries, constants pool, etc.
In addition, because of the way the garbage collection system works, there may be more memory allocated than what is currently in the heap - it just hasn't been reclaimed yet.
All that being said, setting your program to a maximal heap of 256MB is really lowballing it on a modern system. For heavy programs you can usually request at least 1GB of heap.
As you mentioned, one possible cause of slowness is that some of the memory allocated to Java gets swapped off to disk. In that case, the program would indeed start churning the disk, so don't go overboard if you have little physical memory available. On Linux, you can get page miss stats for a process, I am sure there's a similar way on windows.

The -Xmx option only limits the java heap size. In addition to the heap, java will allocate memory for other things, including a stack for each thread (2kB by default, set by -Xss), the PermGenSpace, etc.
So, depending on how many threads you launch, the number of classes your application loads, and some other factors, you may use a lot more memory than expected.
Also, as pointed out, the Windows task manager may take the virtual memory into account.

You mean the heap right? As far as i know there are two things to take care. The Xms option which sets an initial java heap size and the Xmx option which sets the maximum java heap space. If the heap memory is overreaching the Xmx value there should be an OutOfMemoryException.

What about the virtual pages it's taking up. I think Windows shows you the full set of everything aggregated.

How to help java tomcat processes to regain used memory?

We are running an web application that is using Java 64bit 5 gigs of -Xmx of maximum heap size. We have no control over the java code. We can only tweak configuration parameters. The situation that we are facing is that the java processes after it takes the full heap allocated at start up, it starts acting very responding very slow to web site requests. My guess is that is waiting for the GC to collect unused memory objects.
The image below will show you a image of top in linux that shows the critical situation of the processes.
top image of java process http://cp.images.s3.amazonaws.com/ForumImages/java-gc-issue.jpg
Is there any way, we can help java regain the used memory inside the allocated space.
EDIT 1:
I used some of the answers below to be able to get to the answer of my question. Since my question was too difficult to answer, and it turned out to be a discussion. I will post how I was able to monitor the GC cycles and I will pick the answer with more votes. I used jconsole through real vnc viewer to be able to hook from my windows machine to my linux machine running tomcat.
I used this parameters to start the java processes:
-Djava.awt.headless=true -server -Xms512m -Xmx5120m -Dbuild.compiler.emacs=true -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=4999 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false
This is the type of output I got, from jconsole through vnc viewer.
GC Sample Image http://cp.images.s3.amazonaws.com/ForumImages/Sample_GC_Image.jpg

I'd recommend that you not guess. Get some data to see exactly what's going on. You can use visual gc to see what's happening.
If it's the perm space that's being filled up, there won't be much you can do.
Which JVM? If it's 5 or higher there are additional parameters besides just max heap size you can adjust. Check out http://blog.springsource.com/2008/10/14/optimising-and-tuning-apache-tomcat-part-2/

It sounds like you have a memory leak if your application is getting progressively slower. The GC will always start to clean up unused objects as soon as it needs to. If you add -verbosegc you will be able to see how often a GC is performed and much memory is free after a GC. If the heap is more than 80% used you either have to increase the max memory or fix the program so it doesn't use so much.
Can you do a numactl --hardware ? I suggest you not use more than 80% of one memory bank or your GC times will increases dramatically.

Sounds like you need to get details of what's using up the stack. For that I recommend JMAP (http://java.sun.com/j2se/1.5.0/docs/tooldocs/share/jmap.html) which you can run on the process ID (PID) to see what's using memory. Take JMAP snapshots several times when the application is running and see what classes are not freeing up the stack.
Cheers,
-Richard

Try running the app with the -verbose:gc -Xloggc:/path/to/where/you/want/gc.log parameters, and study the resulting gc.log; it should tell you how much time is being spent in garbage collection. Or, as Duffymo suggests above, use visualGC to give you the same data.
Make sure you're using an appropriate colllector - you probably want either the parallel or low-pause (CMS) collectors, assuming you're on java 5.
Have a read of Sun's GC tuning document to see what else you can tweak. On occasions I have found very large heaps to be counter-productive (assuming the application doesn't actually need all that space); more frequent, smaller collections can sometimes end up less disruptive than occasional massive ones.

Which heap size do you prefer?

I know there is no "right" heap size, but which heap size do you use in your applications (application type, jdk, os)?
The JVM Options -Xms (initial/minimum) and -Xmx (maximum) allow for controlling the heap size. What settings make sense under which circumstances? When are the defaults appropriate?

You have to try your application and see how it performs. for example, I used to always run IDEA out of the box until I've got this new job where I work on this huge monolithic project. IDEA was running very slow and regularly throwing out of memory errors when compiling the full project.
first thing I did is to ramp up the heap to 1 gig. this got rid of the out of memory issues but it was still slow. I also noticed IDEA was regularly freezing for 10 seconds or so after which the used memory was cut in half only to ramp up again and , and that triggered the garbage collection idea. I now use it with -Xms512m, -Xmx768m but, I also added -Xincgc, to activate incremental garbage collection
As a result, I've got my old IDEA back: it runs smooth, doesn't freeze anymore and never uses more than 600m of heap.
For your application you have to use a similar approach. try to determine the typical memory usage and tune your heap for the application to run well in those conditions. But also let advanced users tune the setting, to address out of the ordinary data loads.

It depends on the application type. A desktop application is much different than a web application. An application server is much different than a standalone application.
It also depends on the JVM that you are using. JDK5 and later 6 include enhancements that help understand how to tune your application.
Heap size is important, but its also important to know how it plays with the garbage collector.
JDK1.4 Garbage Collector Tuning
JDK5 Garbage Collector Tuning
JDK6 Garbage Collector Tuning

Actually I always considered it very strange that Java limits the heap size. A native application can usually use as much heap as it wants, until it runs out of virtual address space. The only reason to limit the heap in Java seems the garbage collector, which has a certain kind of "laziness" and may not garbage collect objects, unless there is a necessity to do so. That means if you choose the heap too big, your app constantly uses more memory than is really necessary.
However, Sun has improved the GC a lot over the years and to emulate the behavior of a native C app, I would set the initial heap size to 32 MB (for small programs) or 64 MB (for bigger ones) and the maximum to something between 1-2 GB. If your app really needs over a 1 GB of memory, it is most likely broken (unless you deal with data objects that large), but I see no reason why your app should be killed, just because it goes over a certain heap size.
Of course, this is referring to normal PCs. If you create Java code for mobile phones or other limited devices, you should probably adopt the initial and maximum heap size to the limitations of that device.

Typically i try not to use heaps which are larger than 1GB.
It will cost you on major garbage collections.
Sometime it is better to split your application to a few JVM on the same machine and not you large heap sizes.
Major collection with a large heap size can take >10 mintues (on unoptimized GC applications).

This is entirely dependent on your application and any hardware limitations you may have. There is no one size fits all.
jmap can be used to have a look at what heap you are actually using and is a good starting point for right-sizing the heap.

You need to spend quite some time in JConsole or visualvm to get a clear picture on what the plateau memory usage is. Wait until everything is stable and you see the characteristic sawtooth curve of heap memory usage. The peaks should be your 70-80% heap, depending on what garbage collector you use.
Most garbage collectors trigger full GCs when heap usage reaches a certain percentage. This percentage is from 60% to 80% of max heap, depending on what strategy is involved.

1.3Gb for a heavy GUI application.
Unfortunately on Linux the JVM seems to pre-request 1.3G of virtual memory in that situation, which looks bad even if it's not needed (and causes a lot of confused grumbling from users)

On my most memory intensive app:
-Xms250M -Xmx1500M -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC