Develop Reference

Python Subprocess call to invoke java jar files with JAVA_OPTS

I am trying to make a subprocess call from my python script to a java class built inside a jar. I am running the python code on a docker container in AWS Batch. I set the CLASSPATH environment variable in the Dockerfile to include the directory containing the jar file.
ENV CLASSPATH /path/to/dir/containing/jar/file
When I pass the entire command with arguments as a string to subprocess, it works fine.
runnable_command = "java $JAVA_OPTS " \
"RunCommand " \
"-b arg_b"
sp = subprocess.Popen(runnable_command,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
close_fds=True,
shell=True,
universal_newlines=True,
env=os.environ)
result, stderr_data = sp.communicate()
print(result)
But for that I had to make the variable "shell=True" which has a security risk. So I modified the variable 'shell=False" and I pass in the command and arguments as a list to the subprocess. This also works fine.
runnable_command = ["/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.x86_64/jre/bin/java", "$JAVA_OPTS", "<ClassName>"]
However, I am setting JAVA_OPTS environment variable in the Dockerfile to pass the log4j configuration file to JVM.
ENV JAVA_OPTS="-Dlog4j.configurationFile=/opt/amazon/lib/log4j2.xml"
This is important because I want to pipe the logs from this java script to my python script.
When I add JAVA_OPTS to the command, it fails with the following error:
runnable_command = ["/usr/lib/jvm/java-1.8.0-jdk-1.8.0.252.b09-2.x86_64/jre/bin/java", "$JAVA_OPTS", "<ClassName>"]
I am not able to pass JAVA_OPTS to the list of args in the subprocess command. It fails to find the log4j.xml file. I followed this question from stackoverflow but it fails with the same error, even after adding the JAVA_OPTS to the "env" argument.
'Error: Could not find or load main class $JAVA_OPTS\n'
Also, when I pass the arguments as a list, I am not able to run 'java' but I am forced to pass the absolute path of java executable.
Can someone help me with the following questions?
How can i pass the log4j configuration to this java command?
Why am I having to pass the absolute path to java command when running subprocess with a list and not a string?

How to find file path to a bash file when the folder is renamed or moved to a new drive?

I've made an executable jar file for a terminal game that can be opened by typing java -jar name.jar in the Terminal.
Then I made a .sh file inside the same folder as the jar file to open it by double-clicking the .sh. I asked how to do this here, where people told me to use the following code in the .sh.
#! /bin/bash
DIR=$(dirname "$0")
java -jar "$DIR/game.jar"
This worked for a while, but when I renamed the folder, I realised if I move the folder to a pen drive the whole thing stops working and I get this in the Terminal.
Error: Unable to access jarfile /Volumes/Hard
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
[Process completed]
So how to find the file path to the folder the .sh and the jar are in, regardless of where it is, what its name is and what drive it is on?
Also, I'm using MacOS Mojave 10.14.4 if that's of any importance.

The error looks like the path does contain spaces, like probably /Volumes/Hard Drive/Users/something. The solution is to quote the command substitution.
Tangentially, don't use upper case for your private variable names.
But of course, the variable isn't really necessary here, either.
#!/bin/sh
java -jar "$(dirname "$0")/game.jar"
Nothing in this script uses Bash syntax, so it's more portable (as well as often slightly faster) to use sh in the shebang. Perhaps see also Difference between sh and bash

You can store the full path of the working directory using the environement variable $PWD, like in this example (done in 5min, it is just to show you how it is works) :
#!/bin/bash
DIR=$PWD
gamePath='java -jar '$DIR'/game.jar'
echo $gamePath
Wherever I will execute this script, it will shows up the working directory even if I change the name of the parent. Let me show you :
You can see that $PWD environnment variable works great.
Now, I will change the directory name from TestFolder to TestFolderRenamed and execute the script again :
So, in your case, change your code as following :
#! /bin/bash
DIR=$PWD
java -jar "$DIR/game.jar"
It should works.

How to run jar file on shell script in linux terminal?

I made a java project.
The project is....output log message and system.out.println message. just simple.
So I changed into a jar file(the name is LinuxSample.jar).
and I wrote a shell script to run this jar file.
Look at this shell script. (speakee is package name and PrintLinux is main class name)
#!bin/bash
CLASSPATH=/home/tangooc/TANGOOC/test/libs/*
CLASSPATH="${CLASSPATH};/home/tangooc/TANGOOC/test/linux/LinuxSample.jar"
java speakee.PrintLinux
this jar file and this shell script work in Window.
but linux didn't work. I don't know why
this is error message.
Could not find or load main class

Hi Best way to run a java application is to set CLASS_PATH and PATH variable first. If your current jar file depends on external jar files you will face lots of problem. Better set your path variable like below and run the application:-
#!/usr/bin/ksh
export PATH=/usr/java/bin:$PATH
# =/usr/java/bin is your java bin folder
#set environment variable CP with all the jar libraries
CP=/home/flussi/xmlEncoder/encoder.jar
CP=${CP}:/other/jar/somejar.jar
java -Xmx256M -classpath "$CP" "com.myproj.Example"

I made it
I changed the shell script.
CLASSPATH=/home/tangooc/TANGOOC/test/client/LinuxSample.jar
LIB_TOTAL=/home/tangooc/TANGOOC/test/libs/*
echo ${LIB_TOTAL}
echo ${CLASSPATH}
java -cp ${LIB_TOTAL}:${CLASSPATH} speakee.PrintLinux
also there is another way.
CLASSPATH=/home/tangooc/TANGOOC/test/client/LinuxSample.jar
CLASSPATH=${CLASSPATH}:/home/tangooc/TANGOOC/test/libs/*
echo ${CLASSPATH}
java -cp ${CLASSPATH} speakee.PrintLinux
If the someone like me change the shell script.
and check a line, a line, a line...

How to use a .jar from Command Line without typing full path

I downloaded example.jar and I can type java -jar example.jar from within the directory where it is located and it works.
The problem is that I need to be able to call it from elsewhere without typing the full path. Is it possible?
I tried adding it to $CLASSPATH like this:
export CLASSPATH=$CLASSPATH:/Path/to/Directory:/Path/to/Directory/example.jar with no success.

Yes. Option 1. Using the CLASSPATH you have set, however you would have to specify the fully qualified main-class from the jar
java com.mypackage.MyMain
As long as com.mypackage.MyMain is on the CLASSPATH and contains a valid main method, that will run it.
Option 2. Create a bash shell script to run it (note that this is really providing the full path to the java command)
#!/usr/bin/env bash
export JARFILE="/Path/to/Directory/example.jar"
java -jar $JARFILE

Define hadoop FS path inside shell script or perl script

How can I define my path for HDFS inside my shell or perl script so that it picks the input files stored in hdfs and executes the script . It executes correctly under the local file system, but i may need in hdfs
For example , I have part of the script below defined for executing in local path
Define names of folders to be watched
$folderRoot = **'/home/local'**;
A Java pgm
$oscmd = "java -classpath **/home/local**";
#print "forking Java PGM [$thisFile] [$oscmd]\n";
$oscmdResult = `$oscmd`;
print "$oscmdResult\n";
How to define the HDFS path inside Shell or perl script
How to define in the Java classpath in Java Pgm for HDFS so when the shell script is called it invokes the Java pgm as well
My objective : the perl/shell script needs to pick the input files in HDFS and execute it successfully

Mount HDFS into some place and then define a path to this place.