On starting JbossAS 5.1 server on Linux:
26204 jboss 20 0 4874m 1.3g 12m S 144.0 11.4 1:45.50 java
This is before any class-loading.
It starts with minimum 1g (RES) memory. How can i reduce this?
Is there any-way we can suppress memory usages?
inside your %JAVA_HOME%\bin directory (the linux equivelant)
Check the run.conf file for:
if [ "x$JAVA_OPTS" = "x" ]; then
JAVA_OPTS="-Xms128m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000"
fi
I am running 5.1.0.GA on a very old PC, and having has lots of memory errors during start up i removed this specification from the JAVA_OPTS spec. I did this in windows so the batch syntax is different, but essentially, i just removed this option completely. It stopped the server from moaning about memory, but i don't know if you can use these options to restrict the memory usage further.
Not really an answer, but you might find it helps
Related
I have two machines, On both machines I have installed solr-8.9.0. i have updated the solr core, with same csv file having size as 546MB.
Machine 1 configuration is as follows-
open file limit is 1024.
Max Processes Limit is 31168.
Memory allocated to solr-8.9.0 is 512MB. This can be checked using following command 'ps -ef|grep solr'
solr 4955 6959 1 Jul19 pts/0 00:54:51 /home/solr/installer/jdk/jdk1.8.0_66/bin/java -server -Xms512m -Xmx512m -XX:+UseG1GC -XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=250 -XX:+UseLargePages -XX:+AlwaysPreTouch -XX:+ExplicitGCInvokesConcurrent -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/home/solr/Lucene/solr/solr-8.9.0/server/logs/solr_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M -Dsolr.jetty.inetaccess.includes= -Dsolr.jetty.inetaccess.excludes= -Dsolr.log.dir=/home/solr/Lucene/solr/solr-8.9.0/server/logs -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -XX:-OmitStackTraceInFastThrow -XX:OnOutOfMemoryError=/home/solr/Lucene/solr/solr-8.9.0/bin/oom_solr.sh 8983 /home/solr/Lucene/solr/solr-8.9.0/server/logs -Djetty.home=/home/solr/Lucene/solr/solr-8.9.0/server -Dsolr.solr.home=/home/solr/Lucene/solr/solr-8.9.0/server/solr -Dsolr.data.home= -Dsolr.install.dir=/home/solr/Lucene/solr/solr-8.9.0 -Dsolr.default.confdir=/home/solr/Lucene/solr/solr-8.9.0/server/solr/configsets/_default/conf -Xss256k -Dsolr.log.muteconsole -jar start.jar --module=http
To load a csv file following command has been used.
curl 'http://localhost:8983/solr/comparison/update?commit=true&separator=|' --data-binary #/home/solr/data_546MB_7fields.csv -H 'Content-type:application/csv'
processor model name : Intel(R) Core(TM) i7-7700 CPU # 3.60GHz
Number of processor - 8
RAM - 8GB
On loading of file having 546M, solr-8.9.0 is taking 5 minutes 7 seconds.
Machine-2 configuration is as follows-
open file limit is 4096.
Max Processes Limit is 9216.
Memory allocated to solr-8.9.0 is 32GB. This can be checked using following command 'ps -ef|grep solr'
solr 28821 1 62 11:44 pts/1 00:16:28 /home/solr/installer/jdk1.8.0_66/bin/java -server -Xms10240m -Xmx32768m -XX:+UseG1GC -XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=250 -XX:+UseLargePages -XX:+AlwaysPreTouch -XX:+ExplicitGCInvokesConcurrent -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/home/solr/installer/solr/solr-8.9.0/server/logs/solr_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=9 -XX:GCLogFileSize=20M -Dsolr.jetty.inetaccess.includes= -Dsolr.jetty.inetaccess.excludes= -Dsolr.log.dir=/home/solr/installer/solr/solr-8.9.0/server/logs -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -XX:-OmitStackTraceInFastThrow -XX:OnOutOfMemoryError=/home/solr/installer/solr/solr-8.9.0/bin/oom_solr.sh 8983 /home/solr/installer/solr/solr-8.9.0/server/logs -Djetty.home=/home/solr/installer/solr/solr-8.9.0/server -Dsolr.solr.home=/home/solr/installer/solr/solr-8.9.0/server/solr -Dsolr.data.home= -Dsolr.install.dir=/home/solr/installer/solr/solr-8.9.0 -Dsolr.default.confdir=/home/solr/installer/solr/solr-8.9.0/server/solr/configsets/_default/conf -Xss256k -Dsolr.log.muteconsole -jar start.jar --module=http
processor model name : Intel(R) Xeon(R) CPU E5-2697 v2 # 2.70GHz
number of processor - 48
RAM - 128GB
On loading of file having size as 546M, solr-8.9.0 is taking 9 minutes 28 seconds.
Why solr running on machine-2 is taking 1.65 times of machine-1 for loading a csv file, If fact i thought machine-2 shall take less time as open file limit and memory allocated to solr is higher than than machine-1.
It is really awful.Im pretty new to this, so i have tried to post everything you need to understand my problem. Your help is much appreciated.
Is there an important reason to stay on the old Java 8 (and even than this old version update 66, based on the directory name?) you are missing out on a lot of improvements in later versions.
Without the results from profilers like: JDK Flight Recorder or Linux perf (with e.g. flamegraphs as visualisation) I am left to guess which of the parameters you provide may be causing a slowdown.
Try and remove the option -XX:+AlwaysPreTouch with this large heap size: -Xmx32768m because in Java 8 this was single-threaded for the G1 collector.
Also consider decreasing the max heap size if you don't need all that memory to allow for optimizations like compressed pointers (oops), the less memory you use the faster your code runs (unless the the heap is too full and you are frequently running Full-GC, or other issues such as https://plumbr.io/handbook/gc-tuning-in-practice/premature-promotion )
If you need to stay on Java 8 you should at least make sure you use the latest version (not update 66 from 5 years ago) and maybe not on G1GC that was still somewhat buggy in older releases, and instead use the default, which should be -XX:+UseParallelGC if you don't care about pause times. (If you do have interactive users that care about pause-times you may want to compare other options like CMS or the experimental Shenandoah).
To find out how much heap memory your application requires you may want to analyse your solr_gc.log with tools like: https://gceasy.io/
Sources:
https://www.elastic.co/blog/a-heap-of-trouble
https://access.redhat.com/solutions/2685771
Pretouching JVM heap is slow
I have this VM with tomcat, java, and grails in it. I've been getting permgen errors so I looked around and found the solution:
set JAVA_OPTS="-Xms256m -Xmx1024m -XX:PermSize=512m -XX:MaxPermSize=512m"
I use SSH to access the vm and type the arguments above. I suppose that would fix the problem. Thing is, I wanted to make sure that I did it correctly. So I searched again on how I could check the current permSize and this is the solution I got:
jinfo -flag MaxPermSize 6444
6444 is the pid, and as a response, I got this.
-XX:MaxPermSize=85983232
Question: Is the value of the maxPermSize in bytes? because, if it is, then that would mean that the java_opts command didn't work. I am expecting to get 512m but 85983232 bytes = 82 mb.. Or am I seeing it wrong..? Can anybody enlighten me on this? D:
You have to change the values in the CATALINA_OPTS option defined in the Tomcat Catalina start file. To increase the PermGen memory change the value of the MaxPermSize variable, otherwise change the value of the Xmx variable.
Linux & Mac OS: Open or create setenv.sh file placed in the "bin" directory. You have to apply the changes to this line:
export CATALINA_OPTS="$CATALINA_OPTS -server -Xms256m -Xmx1024m -XX:PermSize=512m -XX:MaxPermSize=512m"
Windows:
Open or create the setenv.bat file placed in the "bin" directory:
set CATALINA_OPTS=-server -Xms256m -Xmx1024m -XX:PermSize=512m -XX:MaxPermSize=512m
Don't put the environment configuration in catalina.bat/catalina.sh. Instead you should create a new file in CATALINA_BASE\bin\setenv.bat to keep your customizations separate of tomcat installation.
So you are doing the right thing concerning "-XX:MaxPermSize=512m": it is indeed the correct syntax. You could try to set these options directly to the Catalyna server files so they are used on server start.
Maybe this post will help you!
How to make sure that Tomcat6 reads CATALINA_OPTS on Windows?
Completely removed from java 8 +
Partially removed from java 7 (interned Strings for example)
source
I cant seem to run java at all in a Docker container on my server. Even when issuing java -version, I get the following error.
root#86088d679103:/# java -version
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000035ce1000000, 2555904, 1) failed; error='Operation not permitted' (errno=1)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 2555904 bytes for committing reserved memory.
# An error report file with more information is saved as:
# //hs_err_pid17.log
According to this, java can't map 2.5Mb of space for reserved memory? This does not seem right...
I have the full log included at the end, but for the sake of some extra information, my system is reporting the following:
root#86088d679103:/# uname -m
x86_64
root#86088d679103:/# free -mh
total used free shared buffers cached
Mem: 15G 9.7G 5.8G 912K 148M 8.9G
-/+ buffers/cache: 639M 14G
Swap: 15G 0B 15G
Can anyone point me in the right direction?
Full Log: https://gist.github.com/KayoticSully/e206c44681ce261674ba
Update
#Yobert nailed the problem and I highly suggest you read through the comments and chat log. Good info in there.
For those who want the final command that made Java work: setfattr -n user.pax.flags -v "mr" /usr/bin/java
If your distro does not have setfattr installed by default it should be included in the installable package attr through paceman, apt-get, etc.
I had this same problem when using a Grsec enabled kernel. For java to play nice, I had to disable MPROTECT on the java binary. You can use the paxctl utility for this:
paxctl -m /usr/lib/jvm/java-7-openjdk/jre/bin/java
You'll need to do paxctl -c on the binary first if you've never used it on that binary before:
paxctl -c /usr/lib/jvm/java-7-openjdk/jre/bin/java
More information about paxctl can be found at: http://en.wikibooks.org/wiki/Grsecurity/Additional_Utilities
I had the same problem when running Docker on Alpine Linux, after enabling PaX soft mode it worked:
sysctl -w kernel.pax.softmode=1
Soft mode will disable most PaX features by default, therefore it is not recommended to enable it. The proper way is to use paxctl, as already mentioned above.
Also have a look here:
https://en.wikibooks.org/wiki/Grsecurity/Appendix/Grsecurity_and_PaX_Configuration_Options#Support_soft_mode
This happened to me as well ,
We reduced the RAM size on our VM and after a couple of days started getting this error and service did not came up for ever.
Solution :: We reduced the heap size of the application or service having this issue and the service came up fine again.
I installed the jasperserver(version: 5.6) in Windows 7 through the official document, use an exsiting Tomcat(version: 7) and an existing PostgreSQL(version: 9.2.8), I checked the installation.log file in JasperServer's installation path, and everything is OK.
But when I starting the Tomcat by double click 'Tomcat.exe' and it always says the following error:
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "RMI TCP Connection(idle)"
I searched for these problem and some says to configure the JVM options in %CATALINA_HOME%\bin\setenv.bat or %CATALINA_HOME\bin\%catalina.bat, but I can't find any one of them. I think this is because I'm using an exsiting Tomcat and the installation of JasperServer modified some files in %CATALINA_HOME%. So I searched 'setenv.bat' from %JasperServer% and find it in %JasperServer%\scripts. Here is my configuration:
set JAVA_OPTS=%JAVA_OPTS% -Xms2048m -Xmx3072m -XX:PermSize=1024m -XX:MaxPermSize=2048m -Xss2m -XX:+UseCompressedOoops -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
Also, I did followed the official document here, and nothing helps, Error exists.
Is there anyone who met the same problem? What should I do? I really need help. Thanks.
It seems like the memory parameters you have specified have not been picked up? Check it out, for example by looking at the process table similar to following example:
ivos-mbp:demo ivomagi$ jps
1562 start.jar
1572 Jps
my-mbp:demo me$ ps axu 1562
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
me 1562 0.2 2.8 2965576 230784 s001 S 12:17PM 0:22.62 /usr/bin/java -Dcom.sun.management.jmxremote -Xmx168m
In case you do not see the specified -Xmx3072m in the printout, you need to doublecheck where from the configuration is loaded.
Please, set CATALINA_OPTS instead of JAVA_OPS. Tomcat refers to CATALINA_OPTS on the start:
set "CATALINA_OPTS= -Xms2048m -Xmx3072m -XX:PermSize=1024m -XX:MaxPermSize=2048m -Xss2m -XX:+UseCompressedOoops -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled"
I have the following problem:
$ java -jar program.jar
Error occurred during initialization of VM
Could not reserve enough space for code cache
The amount of memory that is seen by the system seems sufficient:
$ free -m
total used free shared buffers cached
Mem: 5959 640 5318 0 0 390
-/+ buffers/cache: 249 5710
Swap: 4099 0 4099
I tried lowering heap settings as low as 16mb, but it didn't help:
$ java -Xmx16m -Xms16m -jar program.jar
Error occurred during initialization of VM
Could not reserve enough space for code cache
What can be wrong? How can I debug this?
EDIT:
Forgot to include version - I use Sun Java 7u15, 64 bit. I actually can't get it to display a version string, because "java -version" fails with the same error.
Also, for some reason, if I use OpenJDK 6u27, it starts up - but since the app expects java 7, it doesn't work.
Seems that my java was running on Gentoo Hardened Linux, and kernel had PaX enabled. To disable pax memory control, I used the following command:
paxctl -c -m /path/to/your/java/bin/java
After that, everything was smooth.
You can set these sysctl variables (in BSD it is in /etc/sysctl.conf)
security.pax.mprotect.enabled=0
security.pax.mprotect.global=0
security.pax.mprotect.ptrace=0
than.... /etc/rc.d/sysctl restart
do the "trick"