Based on the description of metaspace, it only uses the native memory (no paging).
Since the class metadata is allocated out of native memory, the max available space is the total available system memory.
I found above two explanation in the internet.
I have one question.
The so-called native memory is located in jvm process? The native memory size = java process memory size - heap size, right? If that, why they said the max available space is the total available system memory since the maximum size of 32-bit java process is limited only to about 2G ?
it only uses the native memory (no paging).
This memory can be swapped, as required.
The so-called native memory is located in jvm process?
Native memory is in the JVM process.
The native memory size = java process memory size - heap size, right?
Native memory is all the memory the native code can see. You might want to exclude the heap.
If that, why they said the max available space is the total available system memory
This is true if you don't have an OS or architecural limitations such as
the maximum size of 32-bit java process is limited only to about 2G ?
The maximum is 4 GB, but on different OSes, portions of virtual memory are used by the OS. On Windows XP, you have only 1.2 - 1.5 GB. On Some UNIXes a 32-bit process can use 3.0 - 3.5 GB
Related
1) Ours is a Spring Boot/Java 8 application that we run using
xms = 256 MB, xmx = 2 GB
2) Our release engineers are running TOP command on the unix server where the
application is running and are reporting that application is using 3.5 GB
3) When I profile our application's production JVM instance using VisualVM,
I see the used heap size shows only about 1.4 GB
Any thoughts on why the memory consumption numbers are so different between #2 and #3, above?
Thanks for your feedback.
The -Xmx parameter only sets a maximum size for the Java heap. Any memory outside the Java heap is not limited/controlled by -Xmx.
Examples of non heap memory usage include thread stacks, direct memory and perm gen.
The total virtual memory used (as reported by top) is the sum of heap usage (which you have capped by using -Xmx) and non heap usage (which cannot be capped by -Xmx).
So, the numbers in #2 and #3 are not comparable because they are not measurements of the same thing.
They will never be the same but if you want to bring them closer to each other (or at least have more control over the amount of virtual memory used) then you might consider using ...
-XX:MaxPermSize to limit perm gen size
-XX:MaxHeapFreeRatio to facilitate more aggressive heap shrinkage
I would like to find out how much physical memory does JVM uses under the following situation:
java -Xms2G -Xmx2G -XX:PermSize512M -version
I assume it is using min and max of 2GB + 512 MB - total of 2.5GB physical memory used?
When will swap space come into play for memory usage?
Ref 1: https://plumbr.eu/outofmemoryerror/java-heap-space
Ref 2: https://plumbr.eu/outofmemoryerror/out-of-swap-space
How much physical memory does JVM uses based on the arguments given?
The (virtual) memory footprint varies significantly depending on a number of factors in addition to the arguments.
I assume it is using min and max of 2GB + 512 MB - total of 2.5GB physical memory used?
First of all, the memory usage is likely to be larger than that. Additional memory occupied by:
the (loaded) executable and shared libraries / DLLs comprising the JVM native codebase,
off-heap memory segments that hold thread stacks, and
off-heap memory used for direct-mapped files and other things, allocated by the JVM, application and 3rd-party native libraries.
Second, what we are talking about here is virtual memory usage, not physical memory usage. The physical memory usage will be less than the virtual memory.
Note that the actual physical memory usage typically depends on the overall system demand for memory, and is liable to fluctuate as system deals with competing demands by "paging" your JVM in and out.
When will swap space come into play for memory usage?
Swap space is typically allocated as "backing" for the JVM processes virtual memory pages. It "comes into play", if and when the operating system needs to take physical memory pages away from the JVM to give to other applications. The JVM state in those pages are written to swap space, and then the pages are loaded with another application's state (e.g. from that application's swap space) and then mapped into the address space.
For more information, read the Wikipedia page on virtual memory.
I want to chose a value for maximum JVM heap size. I have some questions about it.
1) Is it usually related with the available physical memory size on the system? If so, is there any formula about this?
2) For the 32-bit JVM, can I set a value larger than 4GB even if the physical memory is very large?
3) Do I need to consider some impact of OS (e.g. Windows, Linux)?
Thanks
1.) Yes amount of memory you can allocate to JVM is related to physical memory available on your system. There is no fixed formula, however there is a rule of thumb that you can use to decide the memory allocation size for your process. There is memory recommendation for different Operating Systems. So (PhysicalMemory - Recommended OS memory) can be allocated to the java process. It varies for 32-bit vs 64-bit OS. If you don't have any other process then this formula will work fine. However if you have other memory consuming processes then you need to take into account memory allocation of those processes as well.
e.g. Windows 7 has following recommended requirement 1 gigabyte (GB) RAM (32-bit) or 2 GB RAM (64-bit). So, if you have 4GB physical memory then you can allocate 2GB to your process assuming 2 GB is used by OS.
2.) For 32-bit JVM you can not set the value larger than 4GB. In fact if your total physical memory is 4GB. You cannot allocate more than 2GB to your Java process. Checkout this answer for more detailed explanation
3.) Yes you should consider OS impact. Check the recommended memory configuration for different Operations Systems and decide memory for your Java process.
Hope this helps.
Suppose the maximum size of a JVM heap is 2GB (-Xmx2048m -Xms100m), we find that the peak used usage of this heap is 1GB and the peak committed usage is 1.2GB after it finishes. So, my question is whether the free space (2GB - 1.2GB) can be consumed by other applications while the JVM is running.
I think the free space cannot be used by others but I'm not sure currently: The operating system reserves 2GB free space before the JVM runs. The reserved space may not be consumed by other applications though the JVM cannot use it up.
JVM checks whether OS has enough address space for -Xmx, but OS won't actually allocate the memory until that much is requested by JVM. JVM will only reserve -Xms memory but can extend upto -Xmx provided that much memory is available.
Runtime.getRuntime().totalMemory() will return the current size of the heap, which will not exceed the maximum size specified on the command line.
That is approximately the amount of memory assigned to the JVM (not including non-heap JVM memory) by the operating system. Other memory is free for use by other applications.
Of course that is grossly oversimplified -- total system memory is total physical memory + total available swap, with other complications (e.g. Linux makes promises of memory to processes but doesn't actually commit it to that process unless it is touched, also simplified). In any case though, the short answer is: Yes, you specify a maximum size on the command line, but the current size is what is allocated to the JVM; the rest is available for other applications.
The memory seen by Java process is virtual memory. The operating system doesn't need to really reserved 2GB free physical memory for Java process.
I have set the default memory limit of Java Virtual Machine while running Java Application like this...
java -mx128m ClassName
I Know this will set maximum memory allocation pool to 128MB, but I don't know what the benefit is, of specifying this memory limit of JVM?
Please enlighten me in this issue...
On Sun's 1.6 JVM, on a server-class machine (meaning one with 2 CPUs and at least 2GB of physical memory) the default maximum heap size is the smaller of 1/4th of the physical memory or 1GB. Using -Xmx lets you change that.
Why would you want to limit the amount of memory Java uses? Two reasons.
Firstly, Java's automatic memory management tends to grab as much memory from the operating system as possible, and then manage it for the benefit of the program. If you are running other programs on the same machine as your Java program, then it will grab more than its fair share of memory, putting pressure on them. If you are running multiple copies of your Java program, they will compete with each other, and you may end up with some instances being starved of memory. Putting a cap on the heap size lets you manage this - if you have 32 GB of RAM, and are running four processes, you can limit each heap to about 8 GB (a bit less would be better), and be confident they will all get the right amount of memory.
Secondly (another aspect of the first, really), if a process grabs more memory than the operating system can supply from physical memory, it uses virtual memory, which gets paged out to disk. This is very slow. Java can reduce its memory usage by making its garbage collector work harder. This is also slow - but not as slow as going to disk. So, you can limit the heap size to avoid the Java process being paged, and so improve performance.
There will be a default heap size limit defined for the JVM. This setting lets you override it, usually so that you can specify that you want more memory to be allocated to the java process.
This sets the maximum Heap Size. The total VM might be larger
There is always a limit because this parameter has a default value (at least for the Oracle/Sun VM)
So the benefit might either be: you can give the memory to the app that it actually needs in order to work (efficiently) or if you come from the other direction: (somewhat) limit the maximum memory used in order to manage the distribution of resources among different applications on one machine.
There already has been a question about java and memory SO: Java memory explained
A very nice article about Java memory is found here. It gives an overview of the memory, how it is used, how it is cleaned and how it can be measured.
The defaults of the memory are (prior java 6):
-Xms size in bytes Sets the initial size of the Java heap. The
default size is 2097152 (2MB). The values must be a multiple of, and
greater than, 1024 bytes (1KB). (The -server flag increases the
default size to 32M.)
-Xmn size in bytes Sets the initial Java heap size for the Eden
generation. The default value is 640K. (The -server flag increases
the default size to 2M.)
-Xmx size in bytes Sets the maximum size to which the Java heap can
grow. The default size is 64M. (The -server flag increases the
default size to 128M.) The maximum heap limit is about 2 GB (2048MB).
Another source (here) states that in Java 6 the default heap size depends on the amount of system memory.
I assume this should help avoid high memory consumption (due to bugs or due to many allocations and deallocations). You would use this if you design for a low-memory system (such as an old computer with little amounts of RAM, mobile phones, etc.).
Alternatively, use this to increase the default memory limit, if it is not enough for you and you are getting OutOfMemoryExceptions for normal behavior.