Develop Reference

kill a java process in Linux

I have a BI application (looker) runs on a linux VM.
tobe able to restart the service, I need to clear the existing java process.
In below screenshot, after run below script, there is a java process, but not showing in the list when I run jps script. What's the reason? and how can I properly terminate this java process?
ps aux | grep java

Have you tried these ?
killall java
or
kill $(pidof java)

As you can see from your image, the process id is changing each time 9287 / 9304 and represents | grep java - and not a java VM!
A common fix is to filter the ps results for not matching | grep -v, such as:
ps aux | grep java | grep -v --regexp=grep.\*java
If there are results above you could append commands to read the process ids and kill command:
kill -TERM $(ps aux | grep java | grep -v --regexp=grep.\*java | awk '{print $2}')
Note: the above will kill all processes with "java" in name so is not very useful if there are multiple java services for same account. You may need to add filter for specific Java VMs.

Unix shells: How to ensure command works against most common shells?

I have some java code which uses jsch to connect to our unix servers and run below command.
ls -lt /tempdirectory/ grep -v '^do | grep "$"(date '+%b %e')" | head
-1 | awk '{print %9}'
Above command shows all files in the /tempdirectory in long format (the first pipe removes directories). From the result it then filters for all files which were created today (I.e. it greps for "Oct 16"). Since I did ls -lt the files are ordered by time, so when I pipe against head -1 I will get the latest file today. And then the last pipe will print the filename out
On a normal putty terminal, once I switch to bash shell, and run above command, it will correctly print out the latest file today.
But the default shell seems to be Bourne (-sh) which the above command will fail (and hence my code will fail)
Is there a way to create above command that is safe for common shells? I think the main shells I've seen at work are:
Bourne (sh)
Bourne again (bash)
Korn (ksh)

This is where you should be using the find(1) command instead:
$ find /tmpdir -type f -ctime -1
gets all the files from /tmpdir created (ctime) today.
If you want all the files then you don't need any more modifiers. If you want to limit further, see the GNU Find manpage at https://linux.die.net/man/1/find.
The find(1) command is independent of which shell you have access to on the remote machine.

The code that you posted
ls -lt / grep -v '^do | grep "$"(date '+%b %e')" | head -1 |awk'{print %9}'
has syntax errors and cannot be run in any shell
Fixed it to
ls -lt /tempdirectory/ | grep -v '^d' | grep "$(date '+%b %e')" | head -1 | awk '{print $9}'
It is ANSI compatible and runs in sh, bash and zsh (what I have tested).

As per comments, I found the issue. Bourne cannot handle $(...) hence I needed to replace this with --> ` <--- (not the regular apostrophes)
Reference:
https://www.unix.com/shell-programming-and-scripting/188983-syntax-error-line-24-unexpected.html
So the command becomes
ls -lt /tempdirectory/ | grep -v '^d' | grep "date '+%b %e'" | head -1 | awk '{print $9}'
I dont whether this will work on many different shells but it seems to work on sh as well as bash
For some reason StackOverflow is not showing that apostrophe properly so I will clarify that I added the weird other apostrophe ` after the first double quote and before the last double quote in:
"{here}date '+%b %e'{here}"
(Also based on previous comments, it sounds like this is not just a matter of what shell you are using when determining command compatibility - your underlying OS may also have an impact. In my case, using uname -a shows i am using sunOS)

Jmap can't connect to make a dump

We have an open beta of an app which occasionally causes the heapspace to overflow. The JVM reacts by going on a permanent vacation.
To analyze this I would like to peek into the memory at the point where it failed. Java does not want me to do this. The process is still in memory but it doesn't seem to be recognized as a java process.
The server in question is a debian Lenny server, Java 6u14
/opt/jdk/bin# ./jmap -F -dump:format=b,file=/tmp/apidump.hprof 11175
Attaching to process ID 11175, please wait...
sun.jvm.hotspot.debugger.NoSuchSymbolException: Could not find symbol "gHotSpotVMTypeEntryTypeNameOffset" in any of the known library names (libjvm.so, libjvm_g.so, gamma_g)
at sun.jvm.hotspot.HotSpotTypeDataBase.lookupInProcess(HotSpotTypeDataBase.java:390)
at sun.jvm.hotspot.HotSpotTypeDataBase.getLongValueFromProcess(HotSpotTypeDataBase.java:371)
at sun.jvm.hotspot.HotSpotTypeDataBase.readVMTypes(HotSpotTypeDataBase.java:102)
at sun.jvm.hotspot.HotSpotTypeDataBase.<init>(HotSpotTypeDataBase.java:85)
at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:568)
at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:494)
at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:332)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:163)
at sun.jvm.hotspot.tools.HeapDumper.main(HeapDumper.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.tools.jmap.JMap.runTool(JMap.java:179)
at sun.tools.jmap.JMap.main(JMap.java:110)
Debugger attached successfully.
sun.jvm.hotspot.tools.HeapDumper requires a java VM process/core!

The solution was very simple. I was running the jmap as root, but I had to run it as the user who started the jvm. I will now go hide my head in shame.

I was running the jmap and the application with the same user and still get the error.
The solution was run that comand before the jmap
echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
Than is just use jmap and will works fine
jmap -heap 17210

If someone tries to get Heap Dump of Java application in Docker container.
This is the only solution that worked for me:
docker exec <container-name> jcmd 1 GC.heap_dump /tmp/docker.hprof
It basically dumps the heap of process with pid=1 using jcmd
See https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr006.html

Future Googlers:
This could also happen if you installed the JDK while the process you're trying to jmap was running.
If that's the case, restart the java process.

Follow the below steps to take the thread and Heap dumps from a docker container
Run the below command to bash into the container. Please change the CONTAINER_NAME appropriately
docker exec -it CONTAINER_NAME bash
Then type jps to find the all the Java application details and extract the PID for your application
jps
Then run the below command to get the thread dump. Please change the PID appropriately
jstack PID > threadDump.tdump
Then run the below command to get the Heap dump. Please change the PID appropriately
jmap -dump:live,format=b,file=heapDump.hprof PID
Then exit from the docker container and download the threadDump.tdump and heapDump.hprof from the docker container by running the below command. Please change the CONTAINER_NAME appropriately
sudo docker cp CONTAINER_NAME:threadDump.tdump .
sudo docker cp CONTAINER_NAME:heapDump.hprof .

What happens if you just run
./jmap -heap 11175
And are you sure the application JVM is identical to the JMAP JVM? (same version, etc)

You need to use the jmap that comes with the JVM.

I got the same jmap error on a linux machine that have two different OpenJdks installed. First I installed OpenJDK 1.6 and after that OpenJDK 1.7.
A call of ...
/usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -XshowSettings:properties -version
# produce the following output ...
...
java.library.path = /usr/java/packages/lib/amd64
/usr/lib/x86_64-linux-gnu/jni
/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu
/usr/lib/jni
/lib
/usr/lib
...
java version "1.7.0_65"
With including '/usr/lib' every with OpenJDK 1.7.* started program includes the libraries of the first installed JDK (in my case OpenJDK 1.6.*). So the jmap versions of Java6 and Java7 failed.
After I changed the start for the Java7 programms with included OpenJDK 1.7 libraries ...
/usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -Djava.library.path=/usr/lib/jvm/java- \
7-openjdk-amd64/jre/lib/amd64/server:/usr/java/packages/lib/amd64: \
/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/ \
x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib ...
I was able access proccess with the Java 7 version of the jmap program. But it needs a sudo to run.

I have the same problem, I'm trying to find a memory leak in a process running inside a Docker container. I wasn't able to use jmap, instead I used this:
jcmd <pid> GC.class_histogram
This gives you a list of the objects in the memory. And from the Oracle documentation:
It is recommended to use the latest utility, jcmd instead of jmap utility for enhanced diagnostics and reduced performance overhead. https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks004.html

1.Execute "Docker ps", will give the container Id of all services and collect the container id foe TSC.
2.Execute "docker exec -it CONTAINER_ID bash" (replace CONTAINER_ID with TSC Container id)
3.Bash will come and then execute the "jps" on bash, that will give you the PID for process(it will be 1 for jar)
4.Execute the "jstack PID > threadDump.tdump"(replace PID with process id received in step 3, it should be 1)
5.Execute the "jmap -dump:format=b,file=heapDump.hprof PID"(replace PID with process id received in step 3, it should be 1)
6.Then we have to exit the bash using "exit" command
7.Execute "sudo docker cp CONTAINER_ID:heapDump.hprof ." from ec2 command line, that will copy the dump file on ec2 machine present working directory.
8.Execute "sudo docker cp CONTAINER_ID:threadDump.tdump ." from ec2 command line, that will copy the dump file on ec2 machine present working directory.

When none of these work or if you don't want to change sensitive OS flags such as ptrace_scope:
Either you can use jconsole/jvisualvm to trigger heap dumps or run any JMX client directly from console as follows as you are doing it locally on the machine that needs the dump and so is faster:
echo 'jmx_invoke -m com.sun.management:type=HotSpotDiagnostic dumpHeap heapdump-20160309.hprof false' | java -jar jmxsh.jar -h $LOCALHOST_OR_IP -p $JMX_PORT
I used the wget https://github.com/davr/jmxsh/raw/master/jmxsh.jar for this example.

What worked for me was to simply issue the command with sudo as in:
sudo jmap -heap 21797

In my case it is not as simple as check the user :(
I have a script called collectd-java which invokes jstat and jmap. I've checked by top that such script is launched, as expected, by the user owning the JVM. However, jstat gives me what I need and jmap can't attach. Here is the script - the echo stuff is just the format I need to present the values:
HOSTNAME="${COLLECTD_HOSTNAME:-localhost}"
INTERVAL="${COLLECTD_INTERVAL:-60}"
MAIN_CLASS="my.fully.qualified.MainClass"
PID=$(pgrep -f ${MAIN_CLASS})
get_jstat_classloaderdata() {
VALUE=`jstat -class $PID 1 1 | awk '{print $1}' | grep -vi loaded`
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-java_classloader_loaded\" interval=$INTERVAL N:$VALUE"
VALUE=`jstat -class $PID 1 1 | awk '{print $2}' | grep -vi bytes`
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-java_classloader_bytesload\" interval=$INTERVAL N:$VALUE"
VALUE=`jstat -class $PID 1 1 | awk '{print $3}' | grep -vi unload`
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-java_classloader_unloaded\" interval=$INTERVAL N:$VALUE"
VALUE=`jstat -class $PID 1 1 | awk '{print $4}' | grep -vi bytes`
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-java_classloader_bytesunload\" interval=$INTERVAL N:$VALUE"
VALUE=`jstat -class $PID 1 1 | awk '{print $5}' | grep -vi time`
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-java_classloader_time\" interval=$INTERVAL N:$VALUE"
}
get_jmap_heapdata() {
VALUE=$(jmap -heap ${PID} | grep MinHeapFreeRatio |awk '{print $3}')
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-jmap_minheapfreeratio\" interval=$INTERVAL N:$VALUE"
VALUE=$(jmap -heap ${PID} | grep MaxHeapFreeRatio|awk '{print $3}')
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-jmap_maxheapfreeratio\" interval=$INTERVAL N:$VALUE"
VALUE=$(jmap -heap ${PID} | grep MaxHeapSize|awk '{print $3}')
echo "PUTVAL \"$HOSTNAME/exec-cecoco/gauge-jmap_maxheapsize\" interval=$INTERVAL N:$VALUE"
}
##Do it
get_jmap_heapdata
get_jstat_classloaderdata
Jstat succeeds and jmap fails. Does anyone understands it ?

Not sure why a plain "jmap " fails when I docker exec -it into my container running centos7 systemd and a java service, but below jmap options worked for me. Thanks:
https://dkbalachandar.wordpress.com/2016/07/05/thread-dump-from-a-docker-container/
[root#b29924306cfe /]# jmap 170
Attaching to process ID 170, please wait...
Error attaching to process: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 170: Operation not permitted
sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process: ptrace(PTRACE_ATTACH, ..) failed for 170: Operation not permitted
[root#b29924306cfe /]# jmap -dump:live,format=b,file=heapDump.hprof 170
Dumping heap to /heapDump.hprof ...
Heap dump file created

Shell script to stop a java program

Is there a way to stop a java program running using a shell script by knowing the name alone.I am using ksh shell

following up on Mnementh' suggestion:
this should do the job
jps -l | grep org.example.MyMain | cut -d ' ' -f 1 | xargs -rn1 kill
jps -l: list java process with "full package name for the application's main class or the full path name to the application's JAR file."
grep: choose the process you like
cut -d -' ' -f 1: split the output in columns using delimiter ' ' and print only the first one (the pid)
xargs -rn1 kill: execute kill for each PID (if any)
note that you must run jps and xargs with the same user (or root) as you're running the process

Add a unique property to the JVM to identify it easily, e.g. for test.class
java -Duniquename=1 test
To kill it:
ps ax | grep uniquename | grep -v grep | awk '{print $1}' | xargs kill

You can use jps identifying the process-id associated with the name of the started java-program (jps is a process-manager for java-programs). With this id you can kill the process normally.

You can use pkill:
pkill your_java_program_name
This would work if you run only one instance of the your program is running.

you can use -o option of ps to format your output,
ps -eo cmd,pid | awk '!/awk/&&/mycommand/{cmd="kill -9 "$2;system(cmd)}'

How to capture the result of a system call in a shell variable?

We want to build a script that run every night (kills and restart a java process). For that we need to capture the process number (since there could be more than one java process running). The command below is basically what we will use to obtain the processes number, probably with a regexp at the end of the grep. Unless any better suggestions comes up.
root#ps -e |grep 'java'
18179 pts/0 00:00:43 java
We want to know how to parse the output above and get it into a shell variable so we can use the kill command as below.
kill -9 ${processid}
wait 10
Note1: The reason we cannot rely on the normal service stop command is because the processes sometimes does not want to die. And we have to use the kill command manually.

There are a couple of options to solve this. If you're using bash, then the shell variable '$!' will contain the PID of the last forked-off child process. So, after you start your Java program, do something like:
echo $! > /var/run/my-process.pid
Then, after your init script stops the Java process:
# Get the pidfile.
pid=$(cat /var/run/my-process.pid)
# Wait ten seconds to stop our process.
for count in $(1 2 3 4 5 6 7 8 9 10); do
sleep 1
cat "/proc/$pid/cmdline" 2>/dev/null | grep -q java
test $? -ne 0 && pid="" && break
done
# If we haven't stopped, kill the process.
if [ ! -z "$pid" ]; then
echo "Not stopping; terminating with extreme prejudice."
kill -9 $pid
fi
Make sure to remove the pidfile when you're done.

ps aux | grep java | awk '{print $1}' | xargs kill -9
Here's an explanation:
ps aux gives you a listing of all processes
grep java gives you all of the processes whose names and command line arguments contain the string "java"
awk '{print $1}' parses the output of the grep command into columns by whitespace and re-prints only the first column
xargs kill -9 passes each of the results of the awk command as parameters to a kill -9 command

I realize this is old, but what about:
pidof java | xargs kill

You can easily get the PID or list of PIDs into a variable using backticks and cut (or awk if preferred) to retrieve only the PID field:
[user#host ~]$ ps -e | grep java | cut -d' ' -f1
12812
12870
13008
13042
13060
Note in the above example I have multiple Java processes running hence the multiple values. If you save this into a variable like so:
JAVA_PROCS=`ps -e | grep java | cut -d' ' -f1`
You can iterate through the processes to kill them if desired:
for proc in $JAVA_PROCS; do
kill -9 $proc;
done
Of course, if you're only retrieving one process, then there's no need to iterate and you can just run it as:
kill -9 $JAVA_PROCS

If you do what you suggest, you may end up capturing the grep itself and killing that (since your grep command contains the java string that you are searching for). You can work around this by excluding grep (by using another grep!):
pid=`ps -e | fgrep java | fgrep -v grep | awk '{print $1}'`
# check pid has a value
# kill $pid
You might also like ps -e -opid,args.
A better alternative is to use pgrep(1) or pkill(1) if your system has them. No more pipes, seds, awks, cuts, xargs:
pkill -9 java

I use something like this:
kill $(ps -A | grep java | cut -b 1-5)

killing it:
ps -e | grep java | cut -f1 -d' ' | xargs kill -9
storing PID on variable:
export JAVAPID=`ps -e | grep 'java' | cut -f1 -d' '`
checking that it worked:
echo $JAVAPID