I have 16GB RAM on my Linux machine and have set the maximum java heap memory to 4GB using -Xmx4096m argument. But I am getting the following error when i start my process.
Invalid maximum heap size: -Xmx4096m The specified size exceeds the
maximum representable size. Could not create the Java virtual machine.
It works fine when i set the value to 2048m.
Is there any other configuration parameter that i need to change to increase the heap size ?
Thanks in advance!
its not only about how much RAM you have
on a 32 bit machine max heap available is 1628MB
on a 64 bit machine max heap available is 2^64 (theoretically) but there are limitations
What OS are you using?
Take a look at the Oracle Hotspot FAQ. Look out for the following section:
Why can't I get a larger heap with the 32-bit JVM?
If you are using a 32-bit system, lower it to 1.6G. For 64-bit systems, check the supported systems list in the link provided.
Try -Xmx4066M
It's a bit less than 4G
after typing to command prompt (cmd)
java -XX:+PrintFlagsFinal -version | findstr /i "HeapSize PermSize ThreadStackSize"
Shows me following result
in order to get from bit's to Mb you have to
divide it by 1024^2=1'048'576 bits/mb
uintx HeapSizePerGCThread = 87241520 bit (83,1999... MiB)
uintx InitialHeapSize = 268435456 bit (256 MiB)
uintx LargePageHeapSizeThreshold = 134217728 bit (128 MiB)
uintx MaxHeapSize = 4263510016 bit (4066 MiB)<4G
When I've tryed changing -Xmx4G and display this is showed me MaxHeapSize =0...
But when i do -XmX4066M it works for me.
My problem is with InitalHeapSize I've Tryed changing -Xms2G or -Xms2000M and it doesnt work for me or it refreshes every time...
After I make a new project
public class MaxMemory {
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
Runtime rt = Runtime.getRuntime();
long totalMem = rt.totalMemory();
long maxMem = rt.maxMemory();
long freeMem = rt.freeMemory();
long usedMem= totalMem-freeMem;
double megs = 1048576.0;
System.out.println ("Max Memory: " + maxMem + " (" + (maxMem/megs) + " MiB)");
System.out.println ("Total Memory: " + totalMem + " (" + (totalMem/megs) + " MiB)");
System.out.println ("Used: " + usedMem + " (" + (usedMem/megs) + " MiB)");
System.out.println ("Free Memory: " + freeMem + " (" + (freeMem/megs) + " MiB)");
}
it throws me following resoult in one of my programs:
Max Memory: 259522560 (247.5 MiB)
Total Memory: 259522560 (247.5 MiB)
Used: 246496784 (235.07765197753906 MiB)
Free Memory: 13025776 (12.422348022460938 MiB)
and i may not increase this "Max Memory" which i think is InitialHeapSize
And I've tried to change it in
Controll Panel > java control panel > java (tab) > view (button) >
RuntimeParameter (block) > -Xms2000M
, of course after going out I click apply but Nothing happened... still run out of my memory ;'[
Related
With jdk 8 these were my steps to figure out how much memory is being consumed while a process is running:
/usr/java/latest/bin>: ./jps
27116 Main
7591 Jps
2879 AmbusProcessor
Then picked up process id to check the status of the heap :
/usr/java/latest/bin>: ./jmap -heap 2879
Attaching to process ID 2879, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 24.45-b08
using thread-local object allocation.
Parallel GC with 13 thread(s)
Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 68719476736 (65536.0MB)
NewSize = 1310720 (1.25MB)
MaxNewSize = 17592186044415 MB
OldSize = 5439488 (5.1875MB)
NewRatio = 2
SurvivorRatio = 8
PermSize = 21757952 (20.75MB)
MaxPermSize = 134217728 (128.0MB)
G1HeapRegionSize = 0 (0.0MB)
Heap Usage:
PS Young Generation
Eden Space:
capacity = 15427698688 (14713.0MB)
used = 9122094480 (8699.507217407227MB)
free = 6305604208 (6013.492782592773MB)
59.128031111311266% used
From Space:
capacity = 2062024704 (1966.5MB)
used = 813973552 (776.2656707763672MB)
free = 1248051152 (1190.2343292236328MB)
39.474481097196396% used
To Space:
capacity = 1944059904 (1854.0MB)
used = 0 (0.0MB)
free = 1944059904 (1854.0MB)
0.0% used
PS Old Generation
capacity = 8520204288 (8125.5MB)
used = 6649238896 (6341.208358764648MB)
free = 1870965392 (1784.2916412353516MB)
78.04083882548333% used
PS Perm Generation
capacity = 31981568 (30.5MB)
used = 16156728 (15.408256530761719MB)
free = 15824840 (15.091743469238281MB)
50.518873871349896% used
6141 interned Strings occupying 609896 bytes.
I am looking for a way to read the status that should look like the above example, However, not finding an option like that with openjdk 11. I have tried all possible option on jmap on openjdk11.
Is there a way still to get that kind of status reading with openjdk11?
I am trying to dump exactly when the outOfMemory is happening
For newer Java versions like JDK 11 you can use this command:
jhsdb jmap --heap --pid <pid>
Output is almost the same as with Java 8 and jmap.
There are some JVM options that may help you by logging GC info:
-XX:+PrintGCDetails
-XX:+PrintGCDateStamps
-XX:+PrintHeapAtGC
-XX:+PrintTenuringDistribution
-XX:+PrintGCApplicationStoppedTime
-XX:+HeapDumpOnOutOfMemoryError
-XX:+UseGCLogFileRotation
-XX:NumberOfGCLogFiles=10
-XX:GCLogFileSize=10M
The file created by HeapDumpOnOutOfMemoryError can be analyzed with tools shipped with the JDK. See also https://docs.oracle.com/javase/7/docs/webnotes/tsg/TSG-VM/html/clopts.html
edit: There's also some official docs about debugging memory issues in Java 11: https://docs.oracle.com/en/java/javase/11/troubleshoot/troubleshoot-memory-leaks.html
I have configured 16GB heap in my elastic search node and the node has 36GB
in it, the elastic search java process is consuming 95% of it. If we put together the heap and non heap memory, together they are not 95%. I want to control the memory usage and don't want the system memory usage to go beyond 90%.
Following is the results of ps aux |grep elasticsearch
103 3242 55.0 95.4 208321876 36468416 ? Sl 00:16 778:35 /usr/bin/java -Xms16g -Xmx16g -XX:MaxDirectMemorySize=16g -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-5.1.1.jar:/usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -d -p /var/run/elasticsearch/elasticsearch.pid -Edefault.path.logs=/var/log/elasticsearch -Edefault.path.data=/var/lib/elasticsearch -Edefault.path.conf=/etc/elasticsearch
Following is the results of jmap -heap
root#ice-bsd-none-551475:~# jmap -heap 3242
Attaching to process ID 3242, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.5-b02
using parallel threads in the new generation.
using thread-local object allocation.
Concurrent Mark-Sweep GC
Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 17179869184 (16384.0MB)
NewSize = 959643648 (915.1875MB)
MaxNewSize = 959643648 (915.1875MB)
OldSize = 16220225536 (15468.8125MB)
NewRatio = 2
SurvivorRatio = 8
MetaspaceSize = 21807104 (20.796875MB)
CompressedClassSpaceSize = 1073741824 (1024.0MB)
MaxMetaspaceSize = 17592186044415 MB
G1HeapRegionSize = 0 (0.0MB)
Heap Usage:
New Generation (Eden + 1 Survivor Space):
capacity = 863698944 (823.6875MB)
used = 89336768 (85.19818115234375MB)
free = 774362176 (738.4893188476562MB)
10.343507841547158% used
Eden Space:
capacity = 767754240 (732.1875MB)
used = 82442520 (78.6233139038086MB)
free = 685311720 (653.5641860961914MB)
10.738139329585467% used
From Space:
capacity = 95944704 (91.5MB)
used = 6894248 (6.574867248535156MB)
free = 89050456 (84.92513275146484MB)
7.18564726615864% used
To Space:
capacity = 95944704 (91.5MB)
used = 0 (0.0MB)
free = 95944704 (91.5MB)
0.0% used
concurrent mark-sweep generation:
capacity = 16220225536 (15468.8125MB)
used = 2346399531588267400 (2.237700969303386E12MB)
free = 15354485090581 MB
1.4465887212113964E10% used
31656 interned Strings occupying 4070216 bytes.
Inspite of the total memory from jmap -heap being less than 20GB, the total memory utilisation of the java process is 95.4% from the results of the top command.
I checked the Direct Memory used and Mapped memory used from jconsole
Direct memory used = 10798837846(10GB)
Mapped Memory used = 166282997194(166GB)
I want to reduce the total memory usage on the machine and control the total memory of this elastic search process.
Is there any way I can log native memory usage from Java, i.e., either native memory directly or the total memory the process is using (e.g., ask the OS)?
I'd like to run this on user's machines behind the scenes, so the NativeMemoryTracking command line tool isn't really the most appealing option. I already log free/max/total heap sizes.
Background: A user of my software reported an exception (below) and I have no idea why. My program does use SWIG'd native code, but it's a simple API, I don't think it has a memory leak, and wasn't on the stacktrace (or run immediately before the error). My log indicated there was plenty of heap space available when the error occurred. So I'm really at a loss for how to track this down.
java.lang.OutOfMemoryError: null
at java.io.RandomAccessFile.writeBytes0(Native Method) ~[na:1.7.0_45]
at java.io.RandomAccessFile.writeBytes(Unknown Source) ~[na:1.7.0_45]
at java.io.RandomAccessFile.write(Unknown Source) ~[na:1.7.0_45]
The error occurred on Windows (7 or 10)(?) from within webstart, configured with these parameters:
<java href="http://java.sun.com/products/autodl/j2se" initial-heap-size="768m" java-vm-args="" max-heap-size="900m" version="1.7+"/>
If you want tp track down the JVM memory on your certain method or lines of code you can use the Runtime API.
Runtime runtime = Runtime.getRuntime();
NumberFormat format = NumberFormat.getInstance();
long maxMemory = runtime.maxMemory();
long allocatedMemory = runtime.totalMemory();
long freeMemory = runtime.freeMemory();
System.out.println("free memory: " + format.format(freeMemory / 1024));
System.out.println("allocated memory: " + format.format(allocatedMemory / 1024));
System.out.println("max memory: " + format.format(maxMemory / 1024));
System.out.println("total free memory: " + format.format((freeMemory + (maxMemory - allocatedMemory)) / 1024));
I ended up using this code which asks the OS for RSS and Peak memory usage. It was straightforward for me to add since I already have a SWIG module set up. The code might not be threadsafe since I hit a random malloc exception when I was testing, meaning I'm not sure I want to keep it in there.
I'm really surprised the JVM doesn't provide a way to do this. Please someone let me know if there's a way.
Here is a snippet of code that sets the string equal to the amount of memory used(mb)/total memory(mb) You can then use this to log however you want!
Runtime instance = Runtime.getRuntime();
String mem = "Memory Used: "+ (instance.totalMemory() - instance.freeMemory()) / mb +"MB ("+
(int)((instance.totalMemory() - instance.freeMemory())*1.0/instance.totalMemory()*100.0)+"%)"
public final void writeBytes(String s) throws IOException {
int len = s.length();
byte[] b = new byte[len];
s.getBytes(0, len, b, 0);
writeBytes(b, 0, len);
}
Looking at the source, it is possible that a sufficiently large String would have caused out of memory error. I suspect that your heap log was done before this happened which explains the free heap space you saw. I suggest you verify if this is the case and if yes, limit the String size and/or increase the heap size.
I have a java program, which i ran and figured out its process id with jps.
How can i see what is the value of -Xms and -Xmx variable for this java process ?
Try
jcmd <PID> VM.command_line
jcmd <PID> VM.flags
you can use jps and do it from the command line:
jps # shows pids
jps -v <pid> # shows params
jps -v <localhost:pid> # the host must be indicated
if this is not enough you can do it programmatically inside the program to check the maximum amount of memory that the Java virtual machine will attempt to use:
Runtime.getRuntime().maxMemory()
and you can also use use the class MemoryUsage to get the initial, used and max ammount that can be used for memory management.
MemoryMXBean memoryBean = ManagementFactory.getMemoryMXBean();
memoryBean.getHeapMemoryUsage().getMax()
memoryBean.getHeapMemoryUsage().getUsed()
memoryBean.getHeapMemoryUsage().getInit()
I think jmap command will give you everything you want.
usage: jmap -heap {pid}
root#BobServerStation:/usr/local $ jmap -heap 3280
Attaching to process ID 3280, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 24.65-b04
using thread-local object allocation.
Parallel GC with 8 thread(s)
Heap Configuration:
MinHeapFreeRatio = 0
MaxHeapFreeRatio = 100
MaxHeapSize = 4116709376 (3926.0MB)
NewSize = 1310720 (1.25MB)
MaxNewSize = 17592186044415 MB
OldSize = 5439488 (5.1875MB)
NewRatio = 2
SurvivorRatio = 8
PermSize = 21757952 (20.75MB)
MaxPermSize = 85983232 (82.0MB)
G1HeapRegionSize = 0 (0.0MB)
Heap Usage:
PS Young Generation
Eden Space:
capacity = 65011712 (62.0MB)
used = 42273152 (40.3148193359375MB)
free = 22738560 (21.6851806640625MB)
65.0239021547379% used
From Space:
capacity = 10485760 (10.0MB)
used = 10479760 (9.994277954101562MB)
free = 6000 (0.0057220458984375MB)
99.94277954101562% used
To Space:
capacity = 10485760 (10.0MB)
used = 0 (0.0MB)
free = 10485760 (10.0MB)
0.0% used
PS Old Generation
capacity = 171442176 (163.5MB)
used = 376368 (0.3589324951171875MB)
free = 171065808 (163.1410675048828MB)
0.21953057805332568% used
PS Perm Generation
capacity = 22020096 (21.0MB)
used = 15401488 (14.688003540039062MB)
free = 6618608 (6.3119964599609375MB)
69.94287400018601% used
8464 interned Strings occupying 699456 bytes.
As the title suggests I'm getting this error inside a thread.
The offending LOCs looks like this:
for (int i = 0; i < objectListSize; i++) {
logger.INFO("Loop repeat: "+i+" ...", true);
final Double discreteScore = sp.getDouble(superPeerSocket);
final int expectedObjectIDs = sp.getInteger(superPeerSocket);
final String discreteObjects[] = new String[expectedObjectIDs];
for ( int j = 0; j < expectedObjectIDs; j++)
discreteObjects[j] = sp.getString(superPeerSocket);
htPlus.attachInitialDiscreteList2L1(discreteScore, discreteObjects);
}
The final String discreteObjects[] declaration is where I get the error. I am running this code inside a thread. I have two threads currently active when I get this. I also tried using the MAT tool from eclipse. here is a link with some chart files inside:
PLC chart files (dropbox URL)
If anyone has any idea for this problem I would be grateful.
P.S.: I am thinking to remove the loop although it just fails in the first loop pass.
(I get this in the output when the program fails)
Expected data size: 10
Repeat: 0 ...
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid3793.hprof ...
Heap dump file created [1404020 bytes in 0.015 secs]
Exception in thread "1" java.lang.OutOfMemoryError: Java heap space
at planetlab.app.factory.Worker$15.run(Worker.java:796)
at java.lang.Thread.run(Thread.java:662)
Something irrelevant:
What's with the code not properly formatted/intended error when making posts in stack overflow? It took me 15 minutes to figure out what to do :# :S :#
Every Java program runs in a sandbox. While your OS might have 10 GB of RAM available to it your app might only have 128 MB.
You need to make sure you app has enough ram allocated to the JVM by using the -Xms -Xmx arguments. -Xms indicates the minimum and -Xmx the maximum
It's been suggested in the comments that your expectedObjectIDs seem kinda high. I would certainly check that out. However, you can use the following code to get an idea as you to memory usage and available memory. Using that info you can adjust your -Xms -Xmx accordingly.
Good luck!
Runtime runtime = Runtime.getRuntime();
long maxMemory = runtime.maxMemory();
long allocatedMemory = runtime.totalMemory();
long freeMemory = runtime.freeMemory();
System.out.println("free memory: " + freeMemory / 1024);
System.out.println("allocated memory: " + allocatedMemory / 1024);
System.out.println("max memory: " + maxMemory /1024);
System.out.println("total free memory: " +
(freeMemory + (maxMemory - allocatedMemory)) / 1024);