I have a BI application (looker) runs on a linux VM.
tobe able to restart the service, I need to clear the existing java process.
In below screenshot, after run below script, there is a java process, but not showing in the list when I run jps script. What's the reason? and how can I properly terminate this java process?
ps aux | grep java
Have you tried these ?
killall java
or
kill $(pidof java)
As you can see from your image, the process id is changing each time 9287 / 9304 and represents | grep java - and not a java VM!
A common fix is to filter the ps results for not matching | grep -v, such as:
ps aux | grep java | grep -v --regexp=grep.\*java
If there are results above you could append commands to read the process ids and kill command:
kill -TERM $(ps aux | grep java | grep -v --regexp=grep.\*java | awk '{print $2}')
Note: the above will kill all processes with "java" in name so is not very useful if there are multiple java services for same account. You may need to add filter for specific Java VMs.
This question might be a bit similar to this or this question.
How do I stop a particular .jar file from running in Mac OS? When I open up the activity monitor, it does not show the process listed. But I am sure it is running because when I visit the localhost (I have developed a .jar from a Spring Boot application), I can still see the welcome message!
Could you please let me know how I could stop a particular .jar file from executing? Thanks.
You can use jps to get the pid (and jar name), awk to parse the pid and then kill it. Like, (with bash or similar)
kill -9 $(jps | grep -i "thejar.jar" | awk '{print $1}')
or
kill -9 `jps | grep -i "thejar.jar" | awk '{print $1}'`
The -i option to grep makes it case insensitive. Omit if that is not needed.
Step1: ps -aux | grep xxx, xxx is the name of .jar
Step2: kill -9 pid, pid you can get from the first command.
I have a java app on my (Ubuntu) server. If I do this, then it starts correctly:
/usr/bin/java -cp /home/jenkins/veta/lily.jar com.sugarapp.lily.Main
but I don't know how to get its PID. I don't know how to stop it with an init.d script.
I have a different app, written in Clojure, and for it I was able to write an init.d script that works great. So I tried to refashion that init.d script for my Java app, and this is what I got:
WORK_DIR="/home/jenkins/veta"
NAME="lily"
JAR="lily.jar"
USER="jenkins"
DAEMON="/usr/bin/java"
DAEMON_ARGS=" -cp /home/jenkins/veta/lily.jar com.sugarapp.lily.Main"
start () {
echo "Starting lily..."
if [ ! -f $WORK_DIR/lily.pid ]; then
/sbin/start-stop-daemon --start --verbose --background --chdir $WORK_DIR --exec $DAEMON --pidfile $WORK_DIR/lily.pid --chuid "$USER" --make-pidfile -- $DAEMON_ARGS
else
echo "lily is already running..."
fi
}
stop () {
echo "Stopping lily..."
/sbin/start-stop-daemon --stop --exec $DAEMON --pidfile $WORK_DIR/lily.pid
rm $WORK_DIR/lily.pid
}
But this doesn't work. Although the PID in $WORK_DIR/lily.pid changes every time I run the script, no process with that PID ever seems to run. If I try:
ps aux | grep java
I don't see this app, nor if I try using the PID.
So is there a way I can use the first command, but somehow capture the PID, so I can store it for later?
I just want a reliable way to stop and start this app. That can be by PID or some other factor. I'm open to suggestions.
UPDATE:
Maybe my question is unclear? Something like jps or ps will give me too many answers. If I do something like "ps aux | grep java" I'll see that there are 5 different java apps running on the server. The start-stop-daemon won't know which PID belongs to this particular app, nor can I figure out what I should feed into my init.d script.
If your system has jdk installed there is an utility called jps which resides in jdk/bin. It will display the list of running java process. Make use of it.
If jdk is not installed in your machine then you have to grep the java process from ps -eaf command.
If you want the pid from the command line, this might work:
myCommand & echo $!
Which I copied from the accepted response to a very similar topic in ServerFault: https://serverfault.com/a/205504
We have an open beta of an app which occasionally causes the heapspace to overflow. The JVM reacts by going on a permanent vacation.
To analyze this I would like to peek into the memory at the point where it failed. Java does not want me to do this. The process is still in memory but it doesn't seem to be recognized as a java process.
The server in question is a debian Lenny server, Java 6u14
/opt/jdk/bin# ./jmap -F -dump:format=b,file=/tmp/apidump.hprof 11175
Attaching to process ID 11175, please wait...
sun.jvm.hotspot.debugger.NoSuchSymbolException: Could not find symbol "gHotSpotVMTypeEntryTypeNameOffset" in any of the known library names (libjvm.so, libjvm_g.so, gamma_g)
at sun.jvm.hotspot.HotSpotTypeDataBase.lookupInProcess(HotSpotTypeDataBase.java:390)
at sun.jvm.hotspot.HotSpotTypeDataBase.getLongValueFromProcess(HotSpotTypeDataBase.java:371)
at sun.jvm.hotspot.HotSpotTypeDataBase.readVMTypes(HotSpotTypeDataBase.java:102)
at sun.jvm.hotspot.HotSpotTypeDataBase.<init>(HotSpotTypeDataBase.java:85)
at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:568)
at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:494)
at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:332)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:163)
at sun.jvm.hotspot.tools.HeapDumper.main(HeapDumper.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.tools.jmap.JMap.runTool(JMap.java:179)
at sun.tools.jmap.JMap.main(JMap.java:110)
Debugger attached successfully.
sun.jvm.hotspot.tools.HeapDumper requires a java VM process/core!
The solution was very simple. I was running the jmap as root, but I had to run it as the user who started the jvm. I will now go hide my head in shame.
I was running the jmap and the application with the same user and still get the error.
The solution was run that comand before the jmap
echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
Than is just use jmap and will works fine
jmap -heap 17210
If someone tries to get Heap Dump of Java application in Docker container.
This is the only solution that worked for me:
docker exec <container-name> jcmd 1 GC.heap_dump /tmp/docker.hprof
It basically dumps the heap of process with pid=1 using jcmd
See https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr006.html
Future Googlers:
This could also happen if you installed the JDK while the process you're trying to jmap was running.
If that's the case, restart the java process.
Follow the below steps to take the thread and Heap dumps from a docker container
Run the below command to bash into the container. Please change the CONTAINER_NAME appropriately
docker exec -it CONTAINER_NAME bash
Then type jps to find the all the Java application details and extract the PID for your application
jps
Then run the below command to get the thread dump. Please change the PID appropriately
jstack PID > threadDump.tdump
Then run the below command to get the Heap dump. Please change the PID appropriately
jmap -dump:live,format=b,file=heapDump.hprof PID
Then exit from the docker container and download the threadDump.tdump and heapDump.hprof from the docker container by running the below command. Please change the CONTAINER_NAME appropriately
sudo docker cp CONTAINER_NAME:threadDump.tdump .
sudo docker cp CONTAINER_NAME:heapDump.hprof .
What happens if you just run
./jmap -heap 11175
And are you sure the application JVM is identical to the JMAP JVM? (same version, etc)
You need to use the jmap that comes with the JVM.
I got the same jmap error on a linux machine that have two different OpenJdks installed. First I installed OpenJDK 1.6 and after that OpenJDK 1.7.
A call of ...
/usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -XshowSettings:properties -version
# produce the following output ...
...
java.library.path = /usr/java/packages/lib/amd64
/usr/lib/x86_64-linux-gnu/jni
/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu
/usr/lib/jni
/lib
/usr/lib
...
java version "1.7.0_65"
With including '/usr/lib' every with OpenJDK 1.7.* started program includes the libraries of the first installed JDK (in my case OpenJDK 1.6.*). So the jmap versions of Java6 and Java7 failed.
After I changed the start for the Java7 programms with included OpenJDK 1.7 libraries ...
/usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -Djava.library.path=/usr/lib/jvm/java- \
7-openjdk-amd64/jre/lib/amd64/server:/usr/java/packages/lib/amd64: \
/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/ \
x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib ...
I was able access proccess with the Java 7 version of the jmap program. But it needs a sudo to run.
I have the same problem, I'm trying to find a memory leak in a process running inside a Docker container. I wasn't able to use jmap, instead I used this:
jcmd <pid> GC.class_histogram
This gives you a list of the objects in the memory. And from the Oracle documentation:
It is recommended to use the latest utility, jcmd instead of jmap utility for enhanced diagnostics and reduced performance overhead. https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks004.html
1.Execute "Docker ps", will give the container Id of all services and collect the container id foe TSC.
2.Execute "docker exec -it CONTAINER_ID bash" (replace CONTAINER_ID with TSC Container id)
3.Bash will come and then execute the "jps" on bash, that will give you the PID for process(it will be 1 for jar)
4.Execute the "jstack PID > threadDump.tdump"(replace PID with process id received in step 3, it should be 1)
5.Execute the "jmap -dump:format=b,file=heapDump.hprof PID"(replace PID with process id received in step 3, it should be 1)
6.Then we have to exit the bash using "exit" command
7.Execute "sudo docker cp CONTAINER_ID:heapDump.hprof ." from ec2 command line, that will copy the dump file on ec2 machine present working directory.
8.Execute "sudo docker cp CONTAINER_ID:threadDump.tdump ." from ec2 command line, that will copy the dump file on ec2 machine present working directory.
When none of these work or if you don't want to change sensitive OS flags such as ptrace_scope:
Either you can use jconsole/jvisualvm to trigger heap dumps or run any JMX client directly from console as follows as you are doing it locally on the machine that needs the dump and so is faster:
echo 'jmx_invoke -m com.sun.management:type=HotSpotDiagnostic dumpHeap heapdump-20160309.hprof false' | java -jar jmxsh.jar -h $LOCALHOST_OR_IP -p $JMX_PORT
I used the wget https://github.com/davr/jmxsh/raw/master/jmxsh.jar for this example.
What worked for me was to simply issue the command with sudo as in:
sudo jmap -heap 21797
In my case it is not as simple as check the user :(
I have a script called collectd-java which invokes jstat and jmap. I've checked by top that such script is launched, as expected, by the user owning the JVM. However, jstat gives me what I need and jmap can't attach. Here is the script - the echo stuff is just the format I need to present the values:
HOSTNAME="${COLLECTD_HOSTNAME:-localhost}"
INTERVAL="${COLLECTD_INTERVAL:-60}"
MAIN_CLASS="my.fully.qualified.MainClass"
PID=$(pgrep -f ${MAIN_CLASS})
get_jstat_classloaderdata() {
VALUE=`jstat -class $PID 1 1 | awk '{print $1}' | grep -vi loaded`
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-java_classloader_loaded\" interval=$INTERVAL N:$VALUE"
VALUE=`jstat -class $PID 1 1 | awk '{print $2}' | grep -vi bytes`
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-java_classloader_bytesload\" interval=$INTERVAL N:$VALUE"
VALUE=`jstat -class $PID 1 1 | awk '{print $3}' | grep -vi unload`
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-java_classloader_unloaded\" interval=$INTERVAL N:$VALUE"
VALUE=`jstat -class $PID 1 1 | awk '{print $4}' | grep -vi bytes`
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-java_classloader_bytesunload\" interval=$INTERVAL N:$VALUE"
VALUE=`jstat -class $PID 1 1 | awk '{print $5}' | grep -vi time`
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-java_classloader_time\" interval=$INTERVAL N:$VALUE"
}
get_jmap_heapdata() {
VALUE=$(jmap -heap ${PID} | grep MinHeapFreeRatio |awk '{print $3}')
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-jmap_minheapfreeratio\" interval=$INTERVAL N:$VALUE"
VALUE=$(jmap -heap ${PID} | grep MaxHeapFreeRatio|awk '{print $3}')
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-jmap_maxheapfreeratio\" interval=$INTERVAL N:$VALUE"
VALUE=$(jmap -heap ${PID} | grep MaxHeapSize|awk '{print $3}')
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-jmap_maxheapsize\" interval=$INTERVAL N:$VALUE"
}
##Do it
get_jmap_heapdata
get_jstat_classloaderdata
Jstat succeeds and jmap fails. Does anyone understands it ?
Not sure why a plain "jmap " fails when I docker exec -it into my container running centos7 systemd and a java service, but below jmap options worked for me. Thanks:
https://dkbalachandar.wordpress.com/2016/07/05/thread-dump-from-a-docker-container/
[root#b29924306cfe /]# jmap 170
Attaching to process ID 170, please wait...
Error attaching to process: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 170: Operation not permitted
sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 170: Operation not permitted
[root#b29924306cfe /]# jmap -dump:live,format=b,file=heapDump.hprof 170
Dumping heap to /heapDump.hprof ...
Heap dump file created
If you are doing local development I want to know how we can stop/kill Jetty webserver? I have to close my Eclipse IDE whenever I need to do that. Is there any other way to do it? Or we have to kill the process?
Thanks.
You should see Jetty instance(s) running in one of the Console. Just press the Red Button.
Jetty can be stopped in the following ways:
You can click the "red" button in the Eclipse console window
You can type "q" then hit enter in the console window (or "r" for restart) (This may only work for Run Jetty Run)
If both of those fail, then if you're on linux or osx, you can open a terminal window and type the following to find the process id of Jetty and kill it. The process id is the 2nd column shown in the results, and you'll need to be careful that you're not seeing some other process here as well:
ps aux | grep jetty | grep java
then run the following to kill the process
kill process_id_from_above_call
You can run the above 2 commands in one shot by doing this:
kill $(ps aux | grep jetty | grep java | awk '{print $2}')
but be sure you're matching Jetty before you do this, as you may kill something you don't want to kill if it doesn't match correctly ;-) Note that if it still isn't killed by the above, you can force it to be killed by doing this:
kill -9 $(ps aux | grep jetty | grep java | awk '{print $2}')