We are periodically runnign jmap -heap command to monitor a tomcat application. However this is first time when we got OutOfMemoryError.
When we monitor the memory at server level there is around 110MB available. Since it is just monitoring can we ignore this time assuming no harm has been done to the application and the server was low on memory when this command run?
$ ./jmap -heap 13511
Attaching to process ID 13511, please wait...
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.tools.jmap.JMap.runTool(JMap.java:179)
at sun.tools.jmap.JMap.main(JMap.java:110)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:597)
at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.<init>(LinuxDebuggerLocal.java:210)
at sun.jvm.hotspot.bugspot.BugSpotAgent.setupDebuggerLinux(BugSpotAgent.java:816)
at sun.jvm.hotspot.bugspot.BugSpotAgent.setupDebugger(BugSpotAgent.java:518)
at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:493)
at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:332)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:163)
at sun.jvm.hotspot.tools.HeapSummary.main(HeapSummary.java:39)
UPDATE: We tried to run jmap -heap again and got java.lang.OutOfMemoryError: Cannot create GC thread. Out of system resources.
So, the problem seems to be at OS resource level.
$ ./jmap -heap 13511
#
# A fatal error has been detected by the Java Runtime Environment:
#
# java.lang.OutOfMemoryError: Cannot create GC thread. Out of system resources.
#
# Internal Error (gcTaskThread.cpp:38), pid=13162, tid=1104300352
# Error: Cannot create GC thread. Out of system resources.
#
# JRE version: 6.0_18-b07
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.0-b13 mixed mode linux-amd64 )
# An error report file with more information is saved as:
# /data01/home/s618199/JAVA/jdk1.6.0_18/bin/hs_err_pid13162.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
Aborted
This is not a memory problem (even though it is reported as one) but an operating system resource problem: You can not create any more threads. And it seems to affect the jmap program and not Tomcat.
Related
I'm trying to do a simulation in Cooja simulator, Contiki-ng using cooja motes and CSMA and the energest model.
Even though the simulation works well in Z1 mote with cooja motes the simulation stops and I get a null pointer error:
[15:19:20 - Thread-1] [Simulation.java:308] [FATAL] - Simulation stopped due to error: null
java.lang.NullPointerException: null
at org.contikios.cooja.contikimote.ContikiMoteType.getCoreMemory(ContikiMoteType.java:824) ~[cooja.jar:?]
at org.contikios.cooja.contikimote.ContikiMote.execute(ContikiMote.java:149) ~[cooja.jar:?]
at org.contikios.cooja.motes.AbstractWakeupMote$1.execute(AbstractWakeupMote.java:51) ~[cooja.jar:?]
at org.contikios.cooja.Simulation.run(Simulation.java:296) ~[cooja.jar:?]
at java.lang.Thread.run(Thread.java:829) ~[?:?]
When I try to do the same simulation with TSCH, 6TiSCH 6top and add the energest model I get this error:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f893449310d, pid=723817, tid=724266
#
# JRE version: OpenJDK Runtime Environment 18.9 (11.0.15+10) (build 11.0.15+10-LTS)
# Java VM: OpenJDK 64-Bit Server VM 18.9 (11.0.15+10-LTS, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# V [libjvm.so+0xa5710d] Klass::search_secondary_supers(Klass*) const+0xd
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e" (or dumping to usr/contiki-ng/tools/cooja/build/core.723817)
#
# If you would like to submit a bug report, please visit:
# https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-11-openjdk
a quick search says it's a bug, but as it only occurs when I add the energy model I'm not sure!
I was having problems configuring my Raspberry Pi 4, as I explained in my previous question (JavaFX in RaspberryPi 4 with Liberica (Cannot open display ,Cannot create resource)), so I decided to start over.
I followed the steps on the Gluon tutorial (https://docs.gluonhq.com/#_javafx_on_embedded), (thanks to #José Pereda) but in the end, when I have to execute the line:
sudo -E java -Dglass.platform=gtk --module-path /opt/javafx-sdk-17/lib --add-modules javafx.controls -cp dist/. hellofx.HelloFX
It shows me this error:
Exception in thread "main" java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at java.base/sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:1051)
Caused by: java.lang.UnsupportedOperationException: Unable to open DISPLAY
at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication.lambda$new$6(GtkApplication.java:173)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication.<init>(GtkApplication.java:171)
at javafx.graphics/com.sun.glass.ui.gtk.GtkPlatformFactory.createApplication(GtkPlatformFactory.java:41)
at javafx.graphics/com.sun.glass.ui.Application.run(Application.java:145)
at javafx.graphics/com.sun.javafx.tk.quantum.QuantumToolkit.startup(QuantumToolkit.java:281)
at javafx.graphics/com.sun.javafx.application.PlatformImpl.startup(PlatformImpl.java:288)
at javafx.graphics/com.sun.javafx.application.PlatformImpl.startup(PlatformImpl.java:160)
at javafx.graphics/com.sun.javafx.application.LauncherImpl.startToolkit(LauncherImpl.java:658)
at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:409)
at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:363)
... 5 more
I have activated X11Forwarding in /etc/ssh/sshd_config.
I remember this error from other Attempts to work with JavaFX, so I tried to change from -Dglass.platform=gtk to -Dglass.platform=Monocle, but then It shows this fail:
Exception in thread "Event Thread" java.lang.UnsatisfiedLinkError: 'void com.sun.glass.ui.monocle.DispmanScreen.wrapNativeSymbols()'
at javafx.graphics/com.sun.glass.ui.monocle.DispmanScreen.wrapNativeSymbols(Native Method)
at javafx.graphics/com.sun.glass.ui.monocle.DispmanScreen.<init>(DispmanScreen.java:34)
at javafx.graphics/com.sun.glass.ui.monocle.DispmanPlatform.createScreen(DispmanPlatform.java:38)
at javafx.graphics/com.sun.glass.ui.monocle.NativePlatform.getScreen(NativePlatform.java:129)
at javafx.graphics/com.sun.glass.ui.monocle.MonocleApplication.staticScreen_getScreens(MonocleApplication.java:222)
at javafx.graphics/com.sun.glass.ui.Screen.initScreens(Screen.java:412)
at javafx.graphics/com.sun.glass.ui.Application.lambda$run$1(Application.java:153)
at javafx.graphics/com.sun.glass.ui.monocle.RunnableProcessor.runLoop(RunnableProcessor.java:92)
at javafx.graphics/com.sun.glass.ui.monocle.RunnableProcessor.run(RunnableProcessor.java:51)
at java.base/java.lang.Thread.run(Thread.java:834)
I'm using a 5.0 inch screen (http://www.lcdwiki.com/5inch_HDMI_Display-B) with an adapter from hdmi to microhdmi.
After Comments things
Enabling X11 Forwarding,same errors trying with sudo -E java -Dglass.platform=gtk --module-path /opt/javafx-sdk-17/lib --add-modules javafx.controls export DISPLAY='IP:0.0' -cp dist/. hellofx.HelloFX
then trying with:
sudo -E java -Degl.displayid=/dev/dri/card1 -Dembedded=monocle -Dglass.platform=Monocle -Dmonocle.platform=EGL -Dmonocle.egl.lib=/opt/javafx-sdk-17/lib/libgluon_drm-1.1.0.so -Dprism.verbose=true --module-path /opt/javafx-sdk-17/lib --add-modules javafx.controls -cp dist/. hellofx.HelloFX
I get:
Prism pipeline init order: es2 sw
Using Double Precision Marlin Rasterizer
Using dirty region optimizations
Using system sized mask for primitives
Not forcing power of 2 sizes for textures
Using hardware CLAMP_TO_ZERO mode
Opting in for HiDPI pixel scaling
Prism pipeline name = com.sun.prism.es2.ES2Pipeline
Loading ES2 native library ... prism_es2_monocle
succeeded.
GLFactory using com.sun.prism.es2.MonocleGLFactory
[GluonDRM] use GPU at /dev/dri/card1 and display id -1
S0 NOT defined:
[GluonDRM] Device /dev/dri/card1 could be opened, but has no drm capabilities.eglinit failed
eglbind ok
eglCreateWindowSurface called
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0xb6dbd830, pid=904, tid=921
#
# JRE version: OpenJDK Runtime Environment (11.0.9.1+1) (build 11.0.9.1+1-post-Raspbian-1deb10u2)
# Java VM: OpenJDK Server VM (11.0.9.1+1-post-Raspbian-1deb10u2, mixed mode, g1 gc, linux-)
# Problematic frame:
# C [libc.so.6+0x77830] strlen+0x0
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/pi/Downloads/samples/CommandLine/Modular/CLI/hellofx/hs_err_pid904.log
#
# If you would like to submit a bug report, please visit:
# Unknown
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Abortado
I used ulimit -c unlimited, then used same command, getting:
Prism pipeline init order: es2 sw
Using Double Precision Marlin Rasterizer
Using dirty region optimizations
Using system sized mask for primitives
Not forcing power of 2 sizes for textures
Using hardware CLAMP_TO_ZERO mode
Opting in for HiDPI pixel scaling
Prism pipeline name = com.sun.prism.es2.ES2Pipeline
Loading ES2 native library ... prism_es2_monocle
succeeded.
GLFactory using com.sun.prism.es2.MonocleGLFactory
[GluonDRM] use GPU at /dev/dri/card1 and display id -1
S0 NOT defined:
[GluonDRM] Device /dev/dri/card1 could be opened, but has no drm capabilities.eglinit failed
eglbind ok
eglCreateWindowSurface called
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0xb6dcc830, pid=923, tid=940
#
# JRE version: OpenJDK Runtime Environment (11.0.9.1+1) (build 11.0.9.1+1-post-Raspbian-1deb10u2)
# Java VM: OpenJDK Server VM (11.0.9.1+1-post-Raspbian-1deb10u2, mixed mode, g1 gc, linux-)
# Problematic frame:
# C [libc.so.6+0x77830] strlen+0x0
#
# Core dump will be written. Default location: /home/pi/Downloads/samples/CommandLine/Modular/CLI/hellofx/core
#
# An error report file with more information is saved as:
# /home/pi/Downloads/samples/CommandLine/Modular/CLI/hellofx/hs_err_pid923.log
#
# If you would like to submit a bug report, please visit:
# Unknown
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Abortado
I am getting below error while restarting Jenkins.
INFO: Ready to receive data from Gerrit: Gerrit-QA
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000007b1480000, 246939648, 0) failed; error='C
annot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 246939648 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /usr/share/tomcat6/hs_err_pid29653.log
How I resolve this ?
I am working on a Spring-MVC application in which I am computing the statistics every night. The problem is, yesterdays computation failed and I have this error and an hs_err_something.log file. The file basically says Out of memory error, but our servers have 32GB ram and quite lot of disk space too. Also, the server is kind of relaxed in night. Why I am I getting this error. I will post contents of relevant code.
StatisticsServiceImpl :
#Override
#Scheduled(cron = "0 2 2 * * ?")
public void computeStatisticsForAllUsers() {
// One of the count as part of statistics
int groupNotesCount = this.groupNotesService.getNoteCountForUser(person.getUsername());
}
GroupNotesDAOImpl :
#Override
public int getNoteCountForUser(String noteCreatorEmail) {
Session session = this.sessionFactory.getCurrentSession();
Query query = session.createQuery("select count(*) from GroupNotes as gn where gn.noteCreatorEmail=:noteCreatorEmail");
query.setParameter("noteCreatorEmail", noteCreatorEmail);
return new Integer(String.valueOf(query.uniqueResult()));
}
Error log :
Aug 05, 2015 2:02:02 AM org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already. Could not load gn. The eventual following stack trace is cause
d by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no function
al impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1612)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
at com.journaldev.spring.dao.GroupNotesDAOImpl.getNoteCountForUser(GroupNotesDAOImpl.java:359)
hs_err.log file :
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 741867520 bytes for committing reserved memory.
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Use 64 bit Java on a 64 bit OS
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (os_linux.cpp:2673), pid=20080, tid=140319513569024
#
# JRE version: Java(TM) SE Runtime Environment (8.0_45-b14) (build 1.8.0_45-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.45-b02 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
What should I do. Any help would be nice. Thanks a lot.
I am using Jboss 4.2.1 GA in my windows7 system. When I run my application I am getting the following error.
2015-03-10 10:33:22,413 ERROR [STDERR] Exception in thread "Thread-30"
2015-03-10 10:33:23,061 ERROR [STDERR] java.lang.OutOfMemoryError: PermGen space
2015-03-10 10:33:23,061 ERROR [STDERR] Exception in thread "Initialize"
2015-03-10 10:33:23,061 DEBUG [org.jboss.cache.interceptors.TxInterceptor] Running rollback phase
2015-03-10 10:33:23,061 ERROR [STDERR] java.lang.OutOfMemoryError: PermGen space
2015-03-10 10:33:23,061 DEBUG [org.jboss.cache.interceptors.TxInterceptor] Finished local commit/rollback method for GlobalTransaction:<null>:9
2015-03-10 10:33:23,061 DEBUG [org.jboss.cache.interceptors.TxInterceptor] Finished rollback phase
2015-03-10 10:33:23,062 ERROR [org.jboss.ejb.plugins.LogInterceptor] Unexpected Error in method: public abstract java.util.Collection com.agnosys.ejb.GroupManagerLocal.loadGroup() throws com.agnosys.ejb.GroupException
java.lang.OutOfMemoryError: PermGen space
2015-03-10 10:33:24,288 ERROR [STDERR] Exception in thread "Thread-32"
2015-03-10 10:33:24,289 ERROR [STDERR] java.lang.OutOfMemoryError: PermGen space
2015-03-10 10:33:42,515 ERROR [STDERR] Exception in thread "ScannerThread"
2015-03-10 10:33:42,515 ERROR [STDERR] java.lang.OutOfMemoryError: PermGen space
2015-03-10 10:33:47,766 ERROR [STDERR] Exception in thread "ScannerThread"
2015-03-10 10:33:47,767 ERROR [STDERR] java.lang.OutOfMemoryError: PermGen space
when I search it in google, I found the following changes in run.conf file.
## -*- shell-script -*- ######################################################
## ##
## JBoss Bootstrap Script Configuration ##
## ##
##############################################################################
### $Id: run.conf 62747 2007-05-02 17:43:36Z dimitris#jboss.org $
#
# This file is optional; it may be removed if not needed.
#
#
# Specify the maximum file descriptor limit, use "max" or "maximum" to use
# the default, as queried by the system.
#
# Defaults to "maximum"
#
#MAX_FD="maximum"
#
# Specify the profiler configuration file to load.
#
# Default is to not load profiler configuration file.
#
#PROFILER=""
#
# Specify the location of the Java home directory. If set then $JAVA will
# be defined to $JAVA_HOME/bin/java, else $JAVA will be "java".
#
#JAVA_HOME="/opt/java/jdk"
#
# Specify the exact Java VM executable to use.
#
#JAVA=""
#
# Specify options to pass to the Java VM.
#
if [ "x$JAVA_OPTS" = "x" ]; then
JAVA_OPTS="-Xms256m -Xmx512m -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -XX:PermSize=128m -XX:MaxPermSize=512m -XX:+UseConcMarkSweepGC -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled "
fi
# Sample JPDA settings for remote socket debuging
#JAVA_OPTS="$JAVA_OPTS -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n"
# Sample JPDA settings for shared memory debugging
#JAVA_OPTS="$JAVA_OPTS -Xrunjdwp:transport=dt_shmem,server=y,suspend=n,address=jboss"
Still I am getting the same error. I don't know what is the issue.
Help me.
Thanks..
Various IDEs with specify JAVA_OPTS, especially when in debug mode, so a more robust development solution is to add MaxPermSize to all startup configurations.
if [ "x$JAVA_OPTS" = "x" ]; then
JAVA_OPTS="-Xms256m -Xmx512m -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -XX:PermSize=128m -XX:+UseConcMarkSweepGC -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled "
fi
# Sample JPDA settings for remote socket debuging
#JAVA_OPTS="$JAVA_OPTS -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n"
# Sample JPDA settings for shared memory debugging
#JAVA_OPTS="$JAVA_OPTS -Xrunjdwp:transport=dt_shmem,server=y,suspend=n,address=jboss"
# Enforce larger MaxPermSize
JAVA_OPTS="$JAVA_OPTS -XX:MaxPermSize=1024m"