just my complete Linux box crashed with OOM (OOM Killer Process killed the wrong processes), due to a java application consumed too much memory and there was no memroy left.
My question is, if I use the JVM Paramter -XmX, does this limit Java to no more use Memory as specified by the -XmX option? Or said differently, If I do NOT specify the -XmX than java might allocate more and more memory with the result my linux box is crahsing itself with OOM?
Thank you very much!
Jens
The default maximum for Java 6 is 1/4 of the main memory. This can mean the total virtual memory of your applications can exceed the main memory and swap space.
Given the cost of memory (8 GB costs less than £40) you should buy more memory. However, an alternative is to use less memory or increase the swap space, so you are less likely to run out.
There's a default maximum heap size (used to be 64M, I think it's 128M now.) The -Xmx parameter changes that maximum size. Oracle's JVMs will never allocate a larger heap than specified in that parameter.
That's not to say that -Xmx gives the total amount of RAM used by the JVM; it'll actually use more than that. Some is for the executable code of the JVM implementation itself; there's also memory used for the "permgen" area, and possibly memory-mapped buffers for other purposes. But Oracle's JVMs, in any event, will not grow their RAM usage without bound; there's always an upper limit.
Now, why doesn't your Linux box have more swap space? It's cheap, and it would prevent this sort of thing from happening in the first place.
Related
I know that the -Xms flag of JVM process is to allow the JVM process to use a specific amount of memory to initialize its process. And in regard to performance of a Java application, it is often recommended to set the same values to both -Xms and -Xmx when starting the application, like -Xms2048M -Xmx2048M.
I'm curious whether the -Xms and -Xmx flags mean that the JVM process makes a reservation for the specific amount of memory to prevent other processes in the same machine from using it.
Is this right?
Xmx merely reserves virtual address space.
Xms actually allocates (commits) it but does not necessarily prefault it.
How operating systems respond to allocations varies.
Windows does allow you to reserve very large chunks of address space (Xmx) but will not allow overcommit (Xms). The limit is defined by swap + physical. The exception are large pages (which need to be enabled with a group policy setting), which will limit it by physical ram.
Linux behavior is more complicated, it depends on the vm.overcommit_memory and related sysctls and various flags passed to the mmap syscall, which to some extent can be controlled by JVM configuration flags. The behavior can range from a) Xms can exceed total ram + swap to b) Xmx is capped by available physical ram.
Short answer: Depends on the OS, though it's definitely a NO in all popular operating systems.
I'll take the example of Linux's memory allocation terminology here.
-Xms and -Xmx specify the minimum and maximum size of JVM heap. These sizes reflect VIRTUAL MEMORY allocations which can be physical mapped to pages in RAM called the RESIDENT SIZE of the process at any time.
When the JVM starts, it'll allocate -Xms amount of virtual memory. This can be mapped to resident memory (physical memory) once you dynamically create more objects on heap. This operation will not require JVM requesting any new allocation from the OS, but will increase you RAM utilization, because those virtual pages will now actually have corresponding physical memory allocation too. However, once your process tries to create more objects on heap after consuming all its Xms allocation on RAM, it has to request the OS for more virtual memory from the OS, which may/may not also be mapped to physical memory later depending on when you need it. The limit for this is your -Xmx allocation.
Note that this is all possible because the memory in linux is shared. So, even if a process allocates memory beforehand, what it gets is virtual memory which is just an addressable contiguous fictional allocation that may or may not be mapped to real physical pages depending on the demand. Read this answer for a short description of how memory management works in popular operating systems. Here is a much detailed (slightly outdated but very useful) information on how Linux's memory management works.
Also note that, these flags only affect heap sizes. The resident memory size that you will see will be larger than the current JVM heap size. More specifically, the memory consumed by a JVM is equals to its HEAP SIZE plus DIRECT MEMORY which reflects things coming from method stacks, native buffer allocations etc.
Does JVM process makes a reservation for the specific amount of memory?
Yes, the JVM reserves the memory specified by Xms at the start and might reserve upto Xmx but the reservation need not be in the physical memory, it can also be in the swap. The JVM pages will be swaped in and out of memory as needed.
Why is it recommended to have same value for Xms and Xmx?
Note: Setting Xms and Xmx is generally recommended for production systems where the machines are dedicated for a single application (or there aren't many applications competing for system resources). This does not generalize it is good everywhere.
Avoids Heap Size:
The JVM starts with the heap size specified by the Xms value initially. When the heap is exhausted due to allocation of objects by the application. The JVM starts increasing the heap. Each time the JVM increases the heap size it must ask the operating system for additional memory. This is a time consuming operation and results in increased gc pause times and inturn the response times for the requests.
Applications Behaviour In the Long Run:
Even though I cannot generalize, many applications over the long run eventually grow to the maximum heap value. This is another reason to start of with maximum memory instead of growing the heap over time and creating unnecessary overhead of heap resize. It is like asking the application to take up the memory at the start itself which it will eventually take.
Number of GCs::
Starting off with small heap sizes results in garbage collection more often. Bigger heap sizes reduce the number of gcs that happen because more memory is available to object allocation. However it must be noted that increased heap sizes increases gc pause times. This is an advantage only if your garbage collection has been tuned properly and the pause times don't increase significantly with increase in heap sizes.
One more reason for doing this is servers generally come with large amounts of memory, So why not use the resources available?
Suppose the maximum size of a JVM heap is 2GB (-Xmx2048m -Xms100m), we find that the peak used usage of this heap is 1GB and the peak committed usage is 1.2GB after it finishes. So, my question is whether the free space (2GB - 1.2GB) can be consumed by other applications while the JVM is running.
I think the free space cannot be used by others but I'm not sure currently: The operating system reserves 2GB free space before the JVM runs. The reserved space may not be consumed by other applications though the JVM cannot use it up.
JVM checks whether OS has enough address space for -Xmx, but OS won't actually allocate the memory until that much is requested by JVM. JVM will only reserve -Xms memory but can extend upto -Xmx provided that much memory is available.
Runtime.getRuntime().totalMemory() will return the current size of the heap, which will not exceed the maximum size specified on the command line.
That is approximately the amount of memory assigned to the JVM (not including non-heap JVM memory) by the operating system. Other memory is free for use by other applications.
Of course that is grossly oversimplified -- total system memory is total physical memory + total available swap, with other complications (e.g. Linux makes promises of memory to processes but doesn't actually commit it to that process unless it is touched, also simplified). In any case though, the short answer is: Yes, you specify a maximum size on the command line, but the current size is what is allocated to the JVM; the rest is available for other applications.
The memory seen by Java process is virtual memory. The operating system doesn't need to really reserved 2GB free physical memory for Java process.
java -Xms is apparently not having an affect on the amount of memory the java process consumes during a run.
I have an app that consumes about 1Gb from the system point of view. I tried setting -Xms2048m (and -Xmx4096m) and I see absolutely no change in memory consumption.
The hotspot docs claim the heap size is bounded below by the Xms value or the default.
The only thing I can think of is maybe the process cannot grab a contiguous block of memory, so it grabbed all it could and then will allocate more later, or maybe windows is not letting it have that much memory to start with. (64-bit windows 7)
(I don't need this for anything, it is just something curious I noticed)
The default memory usage windows task manager shows you is not what's allocated in the processes virtual memory space. It's how much that process has actually written into the virtual space that has had to be mapped onto real memory. If you enable the column for 'Commit Size' in your task manager that will show what is actually considered "used" from the perspective of your processes's virtual address space. (roughly Xms + permsize + size of VM and system stuff itself.)
For Java 1 try with -ms and -mx
Since Java2 you can use -Xms and -Xmx
My experience is, that -ms and `-mx works also in Java2. See http://www.devx.com/tips/Tip/5578
The JVM need a continuous region of memory for the heap. This means it allocates the maximum size as virtual memory on startup. This is not as bad as it sounds as the OS only allocates main memory to the application as it uses it, (not when it allocates virtual memory)
If you look at the amount of memory used in a tool like VisualVM, you can find that even with overhead of 150 - 500 MB, the size is less than the minimum size. This is because Java doesn't just use the minimum size if it doesn't have a use for it.
Instead the minimum size is the point below which it makes only minor attempts to clean up memory. (You may see it perform minor GCs) In most cases this means the application will use the minimum size very quickly. However, a "hello world" program will not use the minimum size.
maybe windows is not letting it have that much memory to start with
The JVM will fail to start if it cannot allocate the maximum size as a continuous block. (This was a common problem on 32-bit Window, such that the limit could be 1.5 GB or as low as 1.2 GB)
I have set the default memory limit of Java Virtual Machine while running Java Application like this...
java -mx128m ClassName
I Know this will set maximum memory allocation pool to 128MB, but I don't know what the benefit is, of specifying this memory limit of JVM?
Please enlighten me in this issue...
On Sun's 1.6 JVM, on a server-class machine (meaning one with 2 CPUs and at least 2GB of physical memory) the default maximum heap size is the smaller of 1/4th of the physical memory or 1GB. Using -Xmx lets you change that.
Why would you want to limit the amount of memory Java uses? Two reasons.
Firstly, Java's automatic memory management tends to grab as much memory from the operating system as possible, and then manage it for the benefit of the program. If you are running other programs on the same machine as your Java program, then it will grab more than its fair share of memory, putting pressure on them. If you are running multiple copies of your Java program, they will compete with each other, and you may end up with some instances being starved of memory. Putting a cap on the heap size lets you manage this - if you have 32 GB of RAM, and are running four processes, you can limit each heap to about 8 GB (a bit less would be better), and be confident they will all get the right amount of memory.
Secondly (another aspect of the first, really), if a process grabs more memory than the operating system can supply from physical memory, it uses virtual memory, which gets paged out to disk. This is very slow. Java can reduce its memory usage by making its garbage collector work harder. This is also slow - but not as slow as going to disk. So, you can limit the heap size to avoid the Java process being paged, and so improve performance.
There will be a default heap size limit defined for the JVM. This setting lets you override it, usually so that you can specify that you want more memory to be allocated to the java process.
This sets the maximum Heap Size. The total VM might be larger
There is always a limit because this parameter has a default value (at least for the Oracle/Sun VM)
So the benefit might either be: you can give the memory to the app that it actually needs in order to work (efficiently) or if you come from the other direction: (somewhat) limit the maximum memory used in order to manage the distribution of resources among different applications on one machine.
There already has been a question about java and memory SO: Java memory explained
A very nice article about Java memory is found here. It gives an overview of the memory, how it is used, how it is cleaned and how it can be measured.
The defaults of the memory are (prior java 6):
-Xms size in bytes Sets the initial size of the Java heap. The
default size is 2097152 (2MB). The values must be a multiple of, and
greater than, 1024 bytes (1KB). (The -server flag increases the
default size to 32M.)
-Xmn size in bytes Sets the initial Java heap size for the Eden
generation. The default value is 640K. (The -server flag increases
the default size to 2M.)
-Xmx size in bytes Sets the maximum size to which the Java heap can
grow. The default size is 64M. (The -server flag increases the
default size to 128M.) The maximum heap limit is about 2 GB (2048MB).
Another source (here) states that in Java 6 the default heap size depends on the amount of system memory.
I assume this should help avoid high memory consumption (due to bugs or due to many allocations and deallocations). You would use this if you design for a low-memory system (such as an old computer with little amounts of RAM, mobile phones, etc.).
Alternatively, use this to increase the default memory limit, if it is not enough for you and you are getting OutOfMemoryExceptions for normal behavior.
You can control the maximum heap size in java using the -Xmx option.
We are experiencing some weird behavior on Windows with this switch. We run some very beefy servers (think 196gb ram). Windows version is Windows Server 2008R2
Java version is 1.6.0_18, 64-Bit (obviously).
Anyway, we were having some weird bugs where processes were quitting with out of memory exceptions even though the process was using much less memory than specified by the -Xmx setting.
So we wrote simple program that would allocate a 1GB byte array each time one pressed the enter key, and initialize the byte array to random values (to prevent any memory compression etc).
Basically, whats happening is that if we run the program with -Xmx35000m (roughly 35 gb) we get an out of memory exception when we hit 25 GB of process space (using windows task manager to measure). We hit this after allocating 24 GB worth of 1 GB blocks, BTW, so that checks out.
Simply specifying a larger value for -Xmx option makes the program work fine to larger amounts of ram.
So, what is going on? Is -Xmx just "off". BTW: We need to specify -Xmx55000m to get a 35 GB process space...
Any ideas on what is going on?
Is their a bug in the Windows JVM?
Is it safe to simply set the -Xmx option bigger, even though there is a disconnect between the -Xmx option and what is going on process wise?
Theory #1
When you request a 35Gb heap using -Xmx35000m, what you are actually saying is that to allow the total space used for the heap to be 35Gb. But the total space consists of the Tenured Object space (for objects that survive multiple GC cycles), the Eden space for newly created objects, and other spaces into which objects will be copied during garbage collection.
The issue is that some of the spaces are not and cannot be used for allocating new objects. So in effect, you "lose" a significant percent of your 35Gb to overheads.
There are various -XX options that can be used to tweak the sizes of the respective spaces, etc. You might try fiddling with them to see if they make a difference. Refer to this document for more information. (The commonly used GC tuning options are listed in section 8. The -XX:NewSpace option looks promising ...)
Theory #2
This might be happening because you are allocating huge objects. IIRC, objects above a certain size can be allocated directly into the Tenured Object space. In your (highly artificial) benchmark, this might result in the JVM not putting stuff into the Eden space, and therefore being able to use less of the total heap space than is normal.
As an experiment, try changing your benchmark to allocate lots of small objects, and see if it manages to use more of the available space before OOME-ing.
Here are some other theories that I would discount:
"You are running into OS-imposed limits." I would discount this, since you said that you can get significantly greater memory utilization by increasing the -Xmx... setting.
"The Windows task manager is reporting bogus numbers." I would discount this because the numbers reported roughly match the 25Gb that you think your application had managed to allocate.
"You are losing space to other things; e.g. the permgen heap." AFAIK, the permgen heap size is controlled and accounted independently of the "normal" heaps. Other non-heap memory usage is either a constant (for the app) or dependent on the app doing specific things.
"You are suffering from heap fragmentation." All of the JVM garbage collectors are "copying collectors", and this family of collectors has the property that heap nodes are automatically compacted.
"JVM bug on Windows." Highly unlikely. There must be tens of thousands of 64bit Java on Windows installations that maximize the heap size. Someone else would have noticed ...
Finally, if you are NOT doing this because your application requires you to allocate memory in huge chunks, and hang onto it "for ever" ... there's a good chance that you are chasing shadows. A "normal" large-memory application doesn't do this kind of thing, and the JVM is tuned for normal applications ... not anomalous ones.
And if your application really does behave this way, the pragmatic solution is to just set the -Xmx... option larger, and only worry if you start running into OS-level issues.
To get a feeling for what exactly you are measuring you should use some different tools:
the Windows Task Manager (I only know Windows XP, but I heard rumours that the Task Manager has improved since then.)
procexp and vmmap from Sysinternals
jconsole from the JVM (you are using the SunOracle HotSpot JVM, aren't you?)
Now you should answer the following questions:
What does jconsole say about the used heap size? How does that differ from procexp?
Does the value from procexp change if you fill the byte arrays with non-zero numbers instead of keeping them at 0?
did you try turning on the verbose output for the GC to find out why the last allocation fails. is it because the OS fails to allocate a heap beyond 25GB for the native JVM process or is it because the GC is hitting some sort of limit on the maximum memory it can manage. I would recommend you also connect to the command line process using jconsole to see what the status of the heap is just before the allocation failure. Also tools like the sysinternals process explorer might give better details as where the failure is occurring if it is in the jvm process.
Since the process is dying at 25GB and you have a generational collector maybe the rest of the generations are consuming 10GB. I would recommend you install JDK 1.6_u24 and use jvisualvm with the visualGC plugin to see what the GC is doing especially factor in the size of all the generations to see how the 35GB heap is being chopped up into different regions by the GC / VM memory manager.
see this link if you are not familiar with Generational GC http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html#generation_sizing.total_heap
I assume this has to do with fragmenting the heap. The free memory is probably not available as a single contiguous free area and when you try to allocate a large block this fails because the requested memory cannot be allocated in a single piece.
The memory displayed by windows task manager is the total memory allocated to the process which includes memory for code, stack, perm gen and heap.
The memory you measure using your click program is the amount of heap jvm makes available to running jvm programs.
Natrually the total allocated memory to JVM by windows should be greater than what JVM makes available to your program as heap memory.