Relationship between java memory allocation needs and available memory/swap - java

when starting a new java process we are getting an error which sounds like this:
"Could not reserve enough space for 16775168KB object heap"
Looking at the log file produced by JVM we see however the following:
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 32744 bytes for ChunkPool::allocate
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
At the same time at the bottom of the log file there is a statement:
Memory: 4k page, physical 67098420k(52119824k free), swap 86034256k(8132400k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (25.45-b02) for windows-amd64 JRE (1.8.0_45-b15), built on Apr 30 2015 12:40:44 by "java_re" with MS VC++ 10.0 (VS2010)
How can the java process fail to allocate 1GB of memory if at the same time there is a bigger amount of memory available for both the physical memory and the swap space?
Thanks a lot
Boris
Excerpt from the log file:
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 32744 bytes for ChunkPool::allocate
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Use 64 bit Java on a 64 bit OS
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (allocation.cpp:273), pid=17164, tid=21188
#
# JRE version: Java(TM) SE Runtime Environment (8.0_45-b15) (build 1.8.0_45-b15)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.45-b02 mixed mode windows-amd64 compressed oops)
# Failed to write core dump.
#
...
VM Arguments:
java_command: com.uniper.rest.bridge.ExternalBridge -Xmx128m
java_class_path (initial): D:\Progs\ENDUR-~1\EN57BD~1\bin\otk\openjvs...
Launcher Type: SUN_STANDARD
Environment Variables:
JAVA_HOME=D:\Progs\Endur-64bit\Endur_V15_0_05052016EON_07072016_1104\bin\olf_dependencies\java\jdk1.8.0_45\
...
--------------- S Y S T E M ---------------
OS: Windows Server 2008 R2 , 64 bit Build 7601 (6.1.7601.23807)
CPU:total 8 (4 cores per cpu, 2 threads per core) family 6 model 26 stepping 5, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, ht, tsc, tscinvbit, tscinv
Memory: 4k page, physical 67098420k(52119824k free), swap 86034256k(8132400k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (25.45-b02) for windows-amd64 JRE (1.8.0_45-b15), built on Apr 30 2015 12:40:44 by "java_re" with MS VC++ 10.0 (VS2010)

Related

Why can java -Xms exceed available memory when in docker container?

I am running a java application within a docker container and java allows me to specify a -Xms size well in excess of the memory available to the container. Why is this the case?
When running outside of a container I get the expected result:
$ free -g; java -Xms100g -version
total used free shared buff/cache available
Mem: 31 14 17 0 0 17
Swap: 28 0 28
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f5f60a00000, 71583137792, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 71583137792 bytes for committing reserved memory.
But when I run the same command from within a docker container
$ free -g; java -Xms100g -version
total used free shared buff/cache available
Mem: 4 0 3 0 0 3
Swap: 0 0 0
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
The command somehow succeeds. Checking the JVM parameters, it doesn't look like the container is overriding JVM parameters. It looks like java thinks it successfully allocated a heap size 25 time larger than the available memory:
java -XX:+PrintFlagsFinal -Xms100g -version 2>&1 | grep InitialHeapSize
uintx InitialHeapSize := 107374182400 {product}
Why does this happen?
Both systems are Ubuntu 18.04.1 LTS (WSL vs FROM ubuntu:18.04) and java 1.8.0_222. Docker desktop 2.1.0.3 (38240). Win10 host OS.

Creating dump file in tomcat folder

When I stop the tomcat service, it creates huge hs_err_pid6636.mdmp and hs_err_pid6636.log files.
Full product version :
java version "1.8.0_40"
Java(TM) SE Runtime Environment (build 1.8.0_40-b25)
Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
Full OS version :
Microsoft Windows [Version 6.3.9600]
Extra relevant system configuration :
Windows 2012 R2 standard,Oracle 12c ,Tomcat 8.
Dump file location: C:\Program Files\Apache Software Foundation\Tomcat 8.0\hs_err_pid1832.mdmp
In Log file:
Expected versus actual behavior :
should not generate error files(hs_err_pid1832.mdmp,hs_err_pid1832.log) in tomcat.
Error messages/stack traces that occur :
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000000742f3643, pid=1832, tid=5720
#
# JRE version: Java(TM) SE Runtime Environment (8.0_40-b25) (build 1.8.0_40-b25)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.40-b25 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# V [jvm.dll+0x2d3643]
#
# Core dump written. Default location: C:\Program Files\Apache Software Foundation\Tomcat 8.0\hs_err_pid1832.mdmp
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
.
.
.
.
OS: Windows Server 2012 R2 , 64 bit Build 9600 (6.3.9600.17056)
CPU:total 1 (1 cores per cpu, 1 threads per core) family 6 model 45 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, aes, clmul, tsc, tscinvbit, tscinv
Memory: 4k page, physical 8388152k(4594380k free), swap 9698872k(5318672k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (25.40-b25) for windows-amd64 JRE (1.8.0_40-b25), built on Feb 10 2015 21:53:57 by "java_re" with MS VC++ 10.0 (VS2010)
time: Tue Apr 04 07:57:24 2017
elapsed time: 4117 seconds (0d 1h 8m 37s)

elastic search and logstash giving memory errors

I have elasticsearch 2.2.0 and logstash 2.1 installed in my system.
When i do a config test for logstash with:
sudo service logstash configtest
It gives following errors:
There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 55545856 bytes for committing reserved memory.
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Use 64 bit Java on a 64 bit OS
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (os_linux.cpp:2638), pid=7887, tid=140476386535168
#
# JRE version: OpenJDK Runtime Environment (8.0_71-b15) (build 1.8.0_71-b15)
# Java VM: OpenJDK 64-Bit Server VM (25.71-b15 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
What is the best way to get rid of this memory error, as there a are lot of prescribed solutions?
Also,
I have bootstrap.mlockall : true in elasticsearch.yml configured.
Can we configure elasticsearch bit differently?

Matlab:fatal error on Java Runtime Environment

I just download the matlab 2013b and once I start it,
it will automatically close once the main interface have been loaded.
In the workspace of Matlab, i find a log file and is posted below with only the infomation i believe is important.
I have tried to search for some information ,updated my java to the latest version, and re-install the whole matlab, even uninstall the older version
but all seem do nothing on the error, what would be the possible solution?
Thank in advances.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000000770f004f, pid=9108, tid=5584
#
# JRE version: 7.0_11-b21
# Java VM: Java HotSpot(TM) 64-Bit Server VM (23.6-b04 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# C 0x00000000770f004f
[error occurred during error reporting (printing problematic frame), id 0xc0000005]
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# If you would like to submit a bug report, please visit:
# http://bugreport.sun.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
--------------- T H R E A D ---------------
Current thread (0x0000000064006800): JavaThread "FileDecorationCache request queue" daemon [_thread_in_native, id=5584, stack(0x0000000100000000,0x0000000104000000)]
siginfo: ExceptionCode=0xc0000005, reading address 0x0000000003ffd5d0
> --------------- S Y S T E M ---------------
>
> OS: Windows 7 , 64 bit Build 7601 Service Pack 1
>
> CPU:total 4 (2 cores per cpu, 2 threads per core) family 6 model 42
> stepping 7, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1,
> sse4.2, popcnt, avx, ht, tsc, tscinvbit
>
> Memory: 4k page, physical 6127920k(3409508k free), swap
> 12253980k(9240140k free)
>
> vm_info: Java HotSpot(TM) 64-Bit Server VM (23.6-b04) for
> windows-amd64 JRE (1.7.0_11-b21), built on Jan 12 2013 02:38:42 by
> "java_re" with unknown MS VC++:1600
>
> time: Wed Apr 23 18:14:36 2014 elapsed time: 83 seconds
Found A solution from another site.
The problem can be solved by disable your anti-virus software.
In my case, I am using avast, and once your disable it, there will be no more error
I had the same problem. I confirm the origin of the problem.
I work with Matlab2010a, Windows 10 (64 bits) and Kapersky.
I disabled the antivirus and it works.
Cheers

Why is my JBoss wrapper application JVM restarted?

My OS version is Windows 7 64 bit and the JDK is 32 bit version. I started my JBoss Wrapper Application successfully, but after it ran for a while the JVM failed and restarted.
The message in the JVM dump log is:
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 543672 bytes for Chunk::new
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Use 64 bit Java on a 64 bit OS
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (allocation.cpp:328), pid=5480, tid=4740
#
# JRE version: 7.0_05-b05
# Java VM: Java HotSpot(TM) Server VM (23.1-b03 mixed mode windows-x86 )
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
I deploy 3 wrapper applications on my computer. Each of them set to a maximum JVM heap size of 700Mb.
Please help me review this problem. thanks. My questions are:
How can I know current JVM allocated size?
What is the reason for this problem?
How can fix it? Someone recommended me to use the 64 bit JDK. Is it necessary?
If you use 32-bit JDK, the maximum heap size we can set and still have the JVM start up is about 1.2 GB.For larger heaps, we need to run a 64-bit JDK. To run 64-bit JDK, you’d also need a64-bitoperating system running on a server that has a64-bit` CPU.
Downloaded the JDK 64 bit version
Set the JAVA_OPTS to
JAVA_OPTS=-Xms1024m -Xmx1024m -XX:MaxPermSize=256m
and refer this link.
Also this is good article about memory.
32-bit JVM are limited to a 2GB heap maximum (-Xmx). In some operating systems, much less than that.
A 64-bit JVM will not have this limitation.
In Windows, you can follow your JVM's memory consumption with
Task Manager->Processes.

Categories

Resources