I have some java code which uses jsch to connect to our unix servers and run below command.
ls -lt /tempdirectory/ grep -v '^do | grep "$"(date '+%b %e')" | head
-1 | awk '{print %9}'
Above command shows all files in the /tempdirectory in long format (the first pipe removes directories). From the result it then filters for all files which were created today (I.e. it greps for "Oct 16"). Since I did ls -lt the files are ordered by time, so when I pipe against head -1 I will get the latest file today. And then the last pipe will print the filename out
On a normal putty terminal, once I switch to bash shell, and run above command, it will correctly print out the latest file today.
But the default shell seems to be Bourne (-sh) which the above command will fail (and hence my code will fail)
Is there a way to create above command that is safe for common shells? I think the main shells I've seen at work are:
Bourne (sh)
Bourne again (bash)
Korn (ksh)
This is where you should be using the find(1) command instead:
$ find /tmpdir -type f -ctime -1
gets all the files from /tmpdir created (ctime) today.
If you want all the files then you don't need any more modifiers. If you want to limit further, see the GNU Find manpage at https://linux.die.net/man/1/find.
The find(1) command is independent of which shell you have access to on the remote machine.
The code that you posted
ls -lt / grep -v '^do | grep "$"(date '+%b %e')" | head -1 |awk'{print %9}'
has syntax errors and cannot be run in any shell
Fixed it to
ls -lt /tempdirectory/ | grep -v '^d' | grep "$(date '+%b %e')" | head -1 | awk '{print $9}'
It is ANSI compatible and runs in sh, bash and zsh (what I have tested).
As per comments, I found the issue. Bourne cannot handle $(...) hence I needed to replace this with --> ` <--- (not the regular apostrophes)
Reference:
https://www.unix.com/shell-programming-and-scripting/188983-syntax-error-line-24-unexpected.html
So the command becomes
ls -lt /tempdirectory/ | grep -v '^d' | grep "date '+%b %e'" | head -1 | awk '{print $9}'
I dont whether this will work on many different shells but it seems to work on sh as well as bash
For some reason StackOverflow is not showing that apostrophe properly so I will clarify that I added the weird other apostrophe ` after the first double quote and before the last double quote in:
"{here}date '+%b %e'{here}"
(Also based on previous comments, it sounds like this is not just a matter of what shell you are using when determining command compatibility - your underlying OS may also have an impact. In my case, using uname -a shows i am using sunOS)
Related
I have a BI application (looker) runs on a linux VM.
tobe able to restart the service, I need to clear the existing java process.
In below screenshot, after run below script, there is a java process, but not showing in the list when I run jps script. What's the reason? and how can I properly terminate this java process?
ps aux | grep java
Have you tried these ?
killall java
or
kill $(pidof java)
As you can see from your image, the process id is changing each time 9287 / 9304 and represents | grep java - and not a java VM!
A common fix is to filter the ps results for not matching | grep -v, such as:
ps aux | grep java | grep -v --regexp=grep.\*java
If there are results above you could append commands to read the process ids and kill command:
kill -TERM $(ps aux | grep java | grep -v --regexp=grep.\*java | awk '{print $2}')
Note: the above will kill all processes with "java" in name so is not very useful if there are multiple java services for same account. You may need to add filter for specific Java VMs.
I want to get video duration with help of ffmpeg:
String command = "ffmpeg -i /home/user/Videos/my-video.mp4 2>&1 | grep Duration | awk '{print $2}' | tr -d ,"
Runtime.getRuntime().exec(cmdarray);
But i always get
java.io.IOException: Cannot run program "ffmpeg -i /home/user/Videos/my-video.mp4 2>&1 | grep Duration | awk '{print $2}' | tr -d ,": error=2, No such file or directory
If I run this command from terminal - all is ok
You've got several issues here. Firstly as #joy points out there could be a problem with the Path used by Java to locate the command so Java may not be finding a command called "ffmpeg". Fixing the Path used for launching your VM should resolve that, or just insert the fully qualified pathname to "ffmpeg".
Secondly: you are trying to run a terminal / shell command. The "|" pipes are normally interpretted correctly by a terminal / shell which breaks the chain into sub-processes linking stdout/stdin. But Java is being asked to run "ffmpeg" passing in some arguments containing "|" which would not be handled as you wish by "ffmpeg".
Check the shell you use:
echo $SHELL
Let's say that printed /bin/bash - you can fix by getting Java to launch the shell and make that interpret the pipe command:
String[] command = new String[] { "/bin/bash", "-c", "ffmpeg -i /home/user/Videos/my-video.mp4 2>&1 | grep Duration | awk '{print $2}' | tr -d ," };
Runtime.getRuntime().exec(cmdarray);
Most likely the path isn't the same when you run from terminal vs when you run from Java. 1. you can try using the full path of ffmpeg (run "which ffmpeg" in terminal). 2. perhaps ffmpeg is an alias in your .profile file in that case you can try to source(load) your .profile file first before executing the command in Java.
I am running a job in Jenkins(Jenkins is an open source continuous integration tool) by executing a list of shell commands. One of the command is to run a Java program which does some data validation. If meets with invalid date, the Java program will exit with a none-zero exit code so that Jenkins can discover that this time the build fails.
Unfortunately the Java program prints too much log to stdout and stderr, only a few of them are useful. Since the Java program can not be modified, I decide to filter the output with grep. So I wrote the shell as:
java -cp $CLASSPATH MetaValidatorMain | grep -v "useless keyword1"| grep -v "useless keyword2"
But the problem is that, after the execution of the line of shell, the parent process(Jenkins) got exit code of grep indead of java, so that Jenkins could not determine whether the build was success.
I also tried this:
(java -cp $CLASSPATH MetaValidatorMain || exit 1) | grep -v "useless keyword1"| grep -v "useless keyword2"
also did not work.
Could anyone tell me how could I write the line of shell to filter output and obtain the right exit code at the same.
thx
Bit of a long way round, but you could redirect the program output to a file, capture the return and then grep the output file for the content you want:
java -cp $CLASSPATH MetaValidatorMain > /tmp/outfile.txt 2>&1
RETURN_CODE=$?
grep -v "useless keyword1" /tmp/outfile | grep -v "useless keyword2"
exit RETURN_CODE
There are 3 ways of doing this. However your current setup should work. The reason here being that the grep won't match anything if the command fails, so grep will return with status 1 (unless the program always shows that text no matter what).
Pipefail
The first way is to set the pipefail option. This is the simplest and what it does is basically set the exit status $? to the exit code of the last program to exit non-zero (or zero if all exited successfully).
# false | true; echo $?
0
# set -o pipefail
# false | true; echo $?
1
$PIPESTATUS
Bash also has a variable called $PIPESTATUS which contains the exit status of all the programs in the last command.
# true | true; echo "${PIPESTATUS[#]}"
0 0
# false | true; echo "${PIPESTATUS[#]}"
1 0
# false | true; echo "${PIPESTATUS[0]}"
1
# true | false; echo "${PIPESTATUS[#]}"
0 1
You can use the 3rd command example to get the specific value in the pipeline that you need.
This solution might not be available though. I think $PIPESTATUS might have been added in a fairly recent version of bash, and your OS may not have it.
Separate executions
This is the most unwieldy of the solutions. Run each command separately and capture the status
# java -cp $CLASSPATH MetaValidatorMain > /tmp/outfile.txt 2>&1
# RETURN_CODE=$?
# grep -v "useless keyword1" /tmp/outfile | grep -v "useless keyword2"
# exit RETURN_CODE
Is there a way to stop a java program running using a shell script by knowing the name alone.I am using ksh shell
following up on Mnementh' suggestion:
this should do the job
jps -l | grep org.example.MyMain | cut -d ' ' -f 1 | xargs -rn1 kill
jps -l: list java process with "full package name for the application's main class or the full path name to the application's JAR file."
grep: choose the process you like
cut -d -' ' -f 1: split the output in columns using delimiter ' ' and print only the first one (the pid)
xargs -rn1 kill: execute kill for each PID (if any)
note that you must run jps and xargs with the same user (or root) as you're running the process
Add a unique property to the JVM to identify it easily, e.g. for test.class
java -Duniquename=1 test
To kill it:
ps ax | grep uniquename | grep -v grep | awk '{print $1}' | xargs kill
You can use jps identifying the process-id associated with the name of the started java-program (jps is a process-manager for java-programs). With this id you can kill the process normally.
You can use pkill:
pkill your_java_program_name
This would work if you run only one instance of the your program is running.
you can use -o option of ps to format your output,
ps -eo cmd,pid | awk '!/awk/&&/mycommand/{cmd="kill -9 "$2;system(cmd)}'
We want to build a script that run every night (kills and restart a java process). For that we need to capture the process number (since there could be more than one java process running). The command below is basically what we will use to obtain the processes number, probably with a regexp at the end of the grep. Unless any better suggestions comes up.
root#ps -e |grep 'java'
18179 pts/0 00:00:43 java
We want to know how to parse the output above and get it into a shell variable so we can use the kill command as below.
kill -9 ${processid}
wait 10
Note1: The reason we cannot rely on the normal service stop command is because the processes sometimes does not want to die. And we have to use the kill command manually.
There are a couple of options to solve this. If you're using bash, then the shell variable '$!' will contain the PID of the last forked-off child process. So, after you start your Java program, do something like:
echo $! > /var/run/my-process.pid
Then, after your init script stops the Java process:
# Get the pidfile.
pid=$(cat /var/run/my-process.pid)
# Wait ten seconds to stop our process.
for count in $(1 2 3 4 5 6 7 8 9 10); do
sleep 1
cat "/proc/$pid/cmdline" 2>/dev/null | grep -q java
test $? -ne 0 && pid="" && break
done
# If we haven't stopped, kill the process.
if [ ! -z "$pid" ]; then
echo "Not stopping; terminating with extreme prejudice."
kill -9 $pid
fi
Make sure to remove the pidfile when you're done.
ps aux | grep java | awk '{print $1}' | xargs kill -9
Here's an explanation:
ps aux gives you a listing of all processes
grep java gives you all of the processes whose names and command line arguments contain the string "java"
awk '{print $1}' parses the output of the grep command into columns by whitespace and re-prints only the first column
xargs kill -9 passes each of the results of the awk command as parameters to a kill -9 command
I realize this is old, but what about:
pidof java | xargs kill
You can easily get the PID or list of PIDs into a variable using backticks and cut (or awk if preferred) to retrieve only the PID field:
[user#host ~]$ ps -e | grep java | cut -d' ' -f1
12812
12870
13008
13042
13060
Note in the above example I have multiple Java processes running hence the multiple values. If you save this into a variable like so:
JAVA_PROCS=`ps -e | grep java | cut -d' ' -f1`
You can iterate through the processes to kill them if desired:
for proc in $JAVA_PROCS; do
kill -9 $proc;
done
Of course, if you're only retrieving one process, then there's no need to iterate and you can just run it as:
kill -9 $JAVA_PROCS
If you do what you suggest, you may end up capturing the grep itself and killing that (since your grep command contains the java string that you are searching for). You can work around this by excluding grep (by using another grep!):
pid=`ps -e | fgrep java | fgrep -v grep | awk '{print $1}'`
# check pid has a value
# kill $pid
You might also like ps -e -opid,args.
A better alternative is to use pgrep(1) or pkill(1) if your system has them. No more pipes, seds, awks, cuts, xargs:
pkill -9 java
I use something like this:
kill $(ps -A | grep java | cut -b 1-5)
killing it:
ps -e | grep java | cut -f1 -d' ' | xargs kill -9
storing PID on variable:
export JAVAPID=`ps -e | grep 'java' | cut -f1 -d' '`
checking that it worked:
echo $JAVAPID