I am playing around with Java, parameter -Xms and class Runtime.
I see that JVM allocates about 15% of the memory that is available - regardless how much it is.
Runtime rt Runtime.getRuntime ();
long total = rt.totalMemory ()
long free = rt.freeMemory ();
That is the case if I start my programm with paramter
-Xms4000m
as well as with
-Xms90m
I see the difference of Xms in the result of totalMemory.
I understand the strategy of allocating more from the beginning to avoid expensive reallocation.
But I do not know if that is in place here AND how I can meter the real memory-consumption of my program.
If your're using the Oracle JVM, you can use the command line tool jmap or the GUI tool jvisualvm to measure the heap consumption of your Java program.
Get the PID of the application and you can use top command in the below way to print out the statistics.
top -b | grep PID
You can see how the memory is utilized over time.
Related
I am getting the following error on execution of a multi-threading program
java.lang.OutOfMemoryError: Java heap space
The above error occured in one of the threads.
Upto my knowledge, Heap space is occupied by instance variables only. If this is correct, then why this error occurred after running fine for sometime as space for instance variables are alloted at the time of object creation.
Is there any way to increase the heap space?
What changes should I made to my program so that It will grab less heap space?
If you want to increase your heap space, you can use java -Xms<initial heap size> -Xmx<maximum heap size> on the command line. By default, the values are based on the JRE version and system configuration. You can find out more about the VM options on the Java website.
However, I would recommend profiling your application to find out why your heap size is being eaten. NetBeans has a very good profiler included with it. I believe it uses the jvisualvm under the hood. With a profiler, you can try to find where many objects are being created, when objects get garbage collected, and more.
1.- Yes, but it pretty much refers to the whole memory used by your program.
2.- Yes see Java VM options
-Xms<size> set initial Java heap size
-Xmx<size> set maximum Java heap size
Ie
java -Xmx2g assign 2 gigabytes of ram as maximum to your app
But you should see if you don't have a memory leak first.
3.- It depends on the program. Try spot memory leaks. This question would be to hard to answer. Lately you can profile using JConsole to try to find out where your memory is going to
You may want to look at this site to learn more about memory in the JVM:
http://developer.streamezzo.com/content/learn/articles/optimization-heap-memory-usage
I have found it useful to use visualgc to watch how the different parts of the memory model is filling up, to determine what to change.
It is difficult to determine which part of memory was filled up, hence visualgc, as you may want to just change the part that is having a problem, rather than just say,
Fine! I will give 1G of RAM to the JVM.
Try to be more precise about what you are doing, in the long run you will probably find the program better for it.
To determine where the memory leak may be you can use unit tests for that, by testing what was the memory before the test, and after, and if there is too big a change then you may want to examine it, but, you need to do the check while your test is still running.
You can get your heap memory size through below programe.
public class GetHeapSize {
public static void main(String[] args) {
long heapsize = Runtime.getRuntime().totalMemory();
System.out.println("heapsize is :: " + heapsize);
}
}
then accordingly you can increase heap size also by using:
java -Xmx2g
http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html
To increase the heap size you can use the -Xmx argument when starting Java; e.g.
-Xmx256M
Upto my knowledge, Heap space is occupied by instance variables only. If this is correct, then why this error occurred after running fine for sometime as space for instance variables are alloted at the time of object creation.
That means you are creating more objects in your application over a period of time continuously. New objects will be stored in heap memory and that's the reason for growth in heap memory.
Heap not only contains instance variables. It will store all non-primitive data types ( Objects). These objects life time may be short (method block) or long (till the object is referenced in your application)
Is there any way to increase the heap space?
Yes. Have a look at this oracle article for more details.
There are two parameters for setting the heap size:
-Xms:, which sets the initial and minimum heap size
-Xmx:, which sets the maximum heap size
What changes should I made to my program so that It will grab less heap space?
It depends on your application.
Set the maximum heap memory as per your application requirement
Don't cause memory leaks in your application
If you find memory leaks in your application, find the root cause with help of profiling tools like MAT, Visual VM , jconsole etc. Once you find the root cause, fix the leaks.
Important notes from oracle article
Cause: The detail message Java heap space indicates object could not be allocated in the Java heap. This error does not necessarily imply a memory leak.
Possible reasons:
Improper configuration ( not allocating sufficiant memory)
Application is unintentionally holding references to objects and this prevents the objects from being garbage collected
Applications that make excessive use of finalizers. If a class has a finalize method, then objects of that type do not have their space reclaimed at garbage collection time. If the finalizer thread cannot keep up, with the finalization queue, then the Java heap could fill up and this type of OutOfMemoryError exception would be thrown.
On a different note, use better Garbage collection algorithms ( CMS or G1GC)
Have a look at this question for understanding G1GC
In most of the cases, the code is not optimized. Release those objects which you think shall not be needed further. Avoid creation of objects in your loop each time. Try to use caches. I don't know how your application is doing. But In programming, one rule of normal life applies as well
Prevention is better than cure. "Don't create unnecessary objects"
Local variables are located on the stack. Heap space is occupied by objects.
You can use the -Xmx option.
Basically heap space is used up everytime you allocate a new object with new and freed some time after the object is no longer referenced. So make sure that you don't keep references to objects that you no longer need.
No, I think you are thinking of stack space. Heap space is occupied by objects. The way to increase it is -Xmx256m, replacing the 256 with the amount you need on the command line.
To avoid that exception, if you are using JUnit and Spring try adding this in every test class:
#DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS)
I have tried all Solutions but nothing worked from above solutions
Solution: In My case I was using 4GB RAM and due to that RAM usage comes out 98% so the required amount if Memory wasn't available. Please do look for this also.If such issue comes upgrade RAM and it will work fine.
Hope this will save someone Time
In netbeans, Go to 'Run' toolbar, --> 'Set Project Configuration' --> 'Customise' --> 'run' of its popped up windo --> 'VM Option' --> fill in '-Xms2048m -Xmx2048m'. It could solve heap size problem.
I am having some memory issues with my application and need help understanding these statistics.
Unix 'top' shows these stats for my process-
VSZ: 37.4g
RSS: 20.0g
So, this means 20g is currently swapped in for the process and in use.
However, when I print stats from within my application using Runtime class, I get this:
Runtime.totalMemory() : 9.8G
Runtime.freeMemory() : 3.6G
Runtime.maxMemory() : 14.3G
Why doesn't [Runtime.totalMemory() - Runtime.freeMemory()] match RSS? This is the memory currently in use by the process. There is a huge difference between the two numbers.
Also, does the runtime give back the unused memory (Runtime.freeMemory()) back to OS for use by other processes?
Note that my applications are running in a peer to peer GemFire caching system set up with shared and replicated caches. I need to optimize the application to reduce memory footprint.
Runtime.totalMemory shows currently available memory. Java allocates memory lazily.
Also, does the runtime give back the unused memory (Runtime.freeMemory()) back to OS for use by other processes?
No. If Java allocated memory (totalMemory) it is in java process now.
RSS: 20.0g
14.3G
As already mentioned Java uses memory besides heap.
Also Gemfire uses off heap memory (check this).
Try to look at them in VisualVM-Buffer Monitor.
What is your infrastracture (OS, VMs)?
If you can't use standard tools, you probably should write own serviceability agent using JMX (for example)
UPDATE
As per doc, by default Gemfire uses JVM Heap.
Ok
Why is RSS consistently showing 20G for a process that is using only ~10G of heap memory.
If you still ask, I would provide more details. What are java memory usage?
Heap memory (Xmx);
Stack memory (each thread has own stack ThreadStackSize, VMThreadStackSize);
MaxPermSize or MaxMetaspaceSize (MaxPermGenSize, MaxMetaspaceSize);
Direct byte buffers (MaxDirectMemorySize);
Memory Pools (Par Eden Space, Par Survivor Space, CMS Old Gen, Metaspace/PermGen, Code Cache, Compressed Class Space (CompressedClassSpaceSize));
String table (where all String.intern() will be, StringTableSize);
PerfDataMemorySize;
MarkStackSize;
CompilerThreadStackSize;
Constant Pool;
I likely missed something;
To see default values, you can run:
java -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version
To see which options for your particular process, you can run:
jps -lvm
Is there an command line tool in unix to determine the rest usage?
For some of them yes. For some of them no.
For direct memory you could try sun.misc.SharedSecrets.getJavaNioAccess().getDirectBufferPool().getMemoryUsed().
With cmd, for example:
jcmd <pid> VM.native_memory baseline
But it depends on running settings (NativeMemoryTracking). You could read how to enable Native Memory tracking here.
In linux you also could use:
pmap -x <pid>
In conclusion this is probably no matter, because your task:
Note that my applications are running in a peer to peer GemFire caching system set up with shared and replicated caches. I need to optimize the application to reduce memory footprint.
And you can not impact on native memory usage. I suppose look at jmap util, which can show you class histogram. You should check what has big size in GemFire and review those objects, probably you store in cache data, which are not should be there. I mean in my practice for optimizing cache I review object and fields and see what fields are really frequent, what are not. Another approach is check your serialization mechanism and object layouts.
As I mentioned you can use serviceability agents:
import com.sun.tools.attach.VirtualMachine;
import sun.tools.attach.HotSpotVirtualMachine;
import java.io.InputStream;
public class JMemoryMain {
public static void main(String[] args) throws Exception {
final int pid = JMemoryMainUtils.getPid(args);
final HotSpotVirtualMachine vm = (HotSpotVirtualMachine) VirtualMachine.attach(String.valueOf(pid));
final byte[] buffer = new byte[1024];
try (InputStream is = vm.heapHisto()) {
for (int read; (read = is.read(buffer)) > 0;) {
System.out.write(buffer, 0, read);
}
}
vm.detach();
}
}
You need tools.jar from JDK in dependencies to run this. It could print you class histogram too, but sometimes works, when jmap does not.
Also, when you should wait a lot of time, while histogram is calculated, you could use VM.getSystemDictionary() and find only your classes.
And, it would be usefull, if you can not enable NMT, because of overhead.
public class Test {
public static void main(String[] args) {
long heapsize = Runtime.getRuntime().totalMemory();
System.out.println("heapsize is :: " + heapsize/(1024*1024));
}
}
The output is:
heapsize is :: 245
Does this mean my runtime has only 245M memory? My computer has 8GB memory. I doubt output this is correct, since the running of Eclipse alone will consume a lot more than 245M.
In Eclipse, I click Windows->Preferences->Java->Installed JREs, and set Default JVM arguments as follows:
-ea -Xms256m -Xmx4096M
Then run the test again. It still prints out the same number, 245. How could this happen?
Edited: from Java doc for Runtime.getRuntime().totalMemory():
Returns the total amount of memory in the Java virtual machine.
The value returned by this method may vary over time, depending on the
host environment.
Your program doesn't run in Eclipse's heap space. Eclipse spawns off a separate JVM for your program.
Runtime.totalMemory() does indeed return the current heap size.
The -Xms argument specifies the initial heap size. Java will expand if it cannot free up enough memory through garbage collection until it reaches the maximum, as set by -Xmx. At this point, the JVM will exit with an OutOfMemoryError.
Java memory management is a complex topic, involving garbage collection, moving objects from nursery to tenured space, etc.
I have an applet and I need to increase his memory. If I pass this parameter:
-Xmx=1024m
and execute this code:
rt = Runtime.getRuntime()
rt.maxMemory()/1024/1024
It returns 989M
If I instead pass:
-Xmx=2048m
It returns 154M
Why is this, and how can I increase the maximum available memory for my Applet?
Xmx option you provide to limit JVM for max memory , now it is upto JVM how much memory it usages at Runtime. When you observed once it was 989M and the other time 154M, even it can go beyond also till it reaches Max Limit. You can not control how much memory applet will use other than giving max limit , JVM will control that.
I am getting the following error on execution of a multi-threading program
java.lang.OutOfMemoryError: Java heap space
The above error occured in one of the threads.
Upto my knowledge, Heap space is occupied by instance variables only. If this is correct, then why this error occurred after running fine for sometime as space for instance variables are alloted at the time of object creation.
Is there any way to increase the heap space?
What changes should I made to my program so that It will grab less heap space?
If you want to increase your heap space, you can use java -Xms<initial heap size> -Xmx<maximum heap size> on the command line. By default, the values are based on the JRE version and system configuration. You can find out more about the VM options on the Java website.
However, I would recommend profiling your application to find out why your heap size is being eaten. NetBeans has a very good profiler included with it. I believe it uses the jvisualvm under the hood. With a profiler, you can try to find where many objects are being created, when objects get garbage collected, and more.
1.- Yes, but it pretty much refers to the whole memory used by your program.
2.- Yes see Java VM options
-Xms<size> set initial Java heap size
-Xmx<size> set maximum Java heap size
Ie
java -Xmx2g assign 2 gigabytes of ram as maximum to your app
But you should see if you don't have a memory leak first.
3.- It depends on the program. Try spot memory leaks. This question would be to hard to answer. Lately you can profile using JConsole to try to find out where your memory is going to
You may want to look at this site to learn more about memory in the JVM:
http://developer.streamezzo.com/content/learn/articles/optimization-heap-memory-usage
I have found it useful to use visualgc to watch how the different parts of the memory model is filling up, to determine what to change.
It is difficult to determine which part of memory was filled up, hence visualgc, as you may want to just change the part that is having a problem, rather than just say,
Fine! I will give 1G of RAM to the JVM.
Try to be more precise about what you are doing, in the long run you will probably find the program better for it.
To determine where the memory leak may be you can use unit tests for that, by testing what was the memory before the test, and after, and if there is too big a change then you may want to examine it, but, you need to do the check while your test is still running.
You can get your heap memory size through below programe.
public class GetHeapSize {
public static void main(String[] args) {
long heapsize = Runtime.getRuntime().totalMemory();
System.out.println("heapsize is :: " + heapsize);
}
}
then accordingly you can increase heap size also by using:
java -Xmx2g
http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html
To increase the heap size you can use the -Xmx argument when starting Java; e.g.
-Xmx256M
Upto my knowledge, Heap space is occupied by instance variables only. If this is correct, then why this error occurred after running fine for sometime as space for instance variables are alloted at the time of object creation.
That means you are creating more objects in your application over a period of time continuously. New objects will be stored in heap memory and that's the reason for growth in heap memory.
Heap not only contains instance variables. It will store all non-primitive data types ( Objects). These objects life time may be short (method block) or long (till the object is referenced in your application)
Is there any way to increase the heap space?
Yes. Have a look at this oracle article for more details.
There are two parameters for setting the heap size:
-Xms:, which sets the initial and minimum heap size
-Xmx:, which sets the maximum heap size
What changes should I made to my program so that It will grab less heap space?
It depends on your application.
Set the maximum heap memory as per your application requirement
Don't cause memory leaks in your application
If you find memory leaks in your application, find the root cause with help of profiling tools like MAT, Visual VM , jconsole etc. Once you find the root cause, fix the leaks.
Important notes from oracle article
Cause: The detail message Java heap space indicates object could not be allocated in the Java heap. This error does not necessarily imply a memory leak.
Possible reasons:
Improper configuration ( not allocating sufficiant memory)
Application is unintentionally holding references to objects and this prevents the objects from being garbage collected
Applications that make excessive use of finalizers. If a class has a finalize method, then objects of that type do not have their space reclaimed at garbage collection time. If the finalizer thread cannot keep up, with the finalization queue, then the Java heap could fill up and this type of OutOfMemoryError exception would be thrown.
On a different note, use better Garbage collection algorithms ( CMS or G1GC)
Have a look at this question for understanding G1GC
In most of the cases, the code is not optimized. Release those objects which you think shall not be needed further. Avoid creation of objects in your loop each time. Try to use caches. I don't know how your application is doing. But In programming, one rule of normal life applies as well
Prevention is better than cure. "Don't create unnecessary objects"
Local variables are located on the stack. Heap space is occupied by objects.
You can use the -Xmx option.
Basically heap space is used up everytime you allocate a new object with new and freed some time after the object is no longer referenced. So make sure that you don't keep references to objects that you no longer need.
No, I think you are thinking of stack space. Heap space is occupied by objects. The way to increase it is -Xmx256m, replacing the 256 with the amount you need on the command line.
To avoid that exception, if you are using JUnit and Spring try adding this in every test class:
#DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS)
I have tried all Solutions but nothing worked from above solutions
Solution: In My case I was using 4GB RAM and due to that RAM usage comes out 98% so the required amount if Memory wasn't available. Please do look for this also.If such issue comes upgrade RAM and it will work fine.
Hope this will save someone Time
In netbeans, Go to 'Run' toolbar, --> 'Set Project Configuration' --> 'Customise' --> 'run' of its popped up windo --> 'VM Option' --> fill in '-Xms2048m -Xmx2048m'. It could solve heap size problem.