I'm trying to do some automation on MacOS with Java.
no problems when running the commands manually from a terminal
i assume it works because of <user.home>/.zprofile
the commands are not found when trying to execute them via ProcessBuilder
How can I execute commands with the same environment as if running a zsh terminal manually?
import java.io.BufferedReader;
import java.io.InputStreamReader;
public class Test {
public static void main(String[] args) throws Exception {
// these commands work
run("/bin/sh", "-c", "echo $PATH");
run("/bin/bash", "-c", "echo $PATH");
run("/bin/zsh", "-c", "echo $PATH");
// these commands all work when I run them manually in a terminal
// but fail here with "zsh:1: command not found: ..."
run("/bin/zsh", "-c", "node -v");
run("/bin/zsh", "-c", "npm -v");
}
private static void run(String... command) throws Exception {
ProcessBuilder processBuilder = new ProcessBuilder();
processBuilder.redirectErrorStream(true);
processBuilder.command(command);
Process process = processBuilder.start();
try(BufferedReader br = new BufferedReader(new InputStreamReader(process.getInputStream()))) {
for(String line = br.readLine(); line != null; line = br.readLine()) {
System.out.println(line);
}
}
System.out.println("return value: " + process.waitFor());
}
}
Output:
/usr/bin:/bin:/usr/sbin:/sbin
return value: 0
/usr/bin:/bin:/usr/sbin:/sbin
return value: 0
/usr/bin:/bin:/usr/sbin:/sbin
return value: 0
zsh:1: command not found: node
return value: 127
zsh:1: command not found: npm
return value: 127
After reading way too many articles about shells, and studying shell init diagrams, I decided to go with Zsh.
The reason was this blog post, which indicated that Zsh seems to have at least one init file that is executed for all possible shell variants (login, non-login, interactive, non-interactive etc).
I moved all my environment setup (PATH and LANG) to /etc/zshenv, deleted /etc/zprofile and all ~/.z* files.
I also changed the shell for both root and my user to Zsh (for the user this can also be done via system preferences):
dscl . -delete /Users/root UserShell && dscl . -create /Users/root UserShell /bin/zsh && dscl . -read /Users/root UserShell
dscl . -delete /Users/reto UserShell && dscl . -create /Users/reto UserShell /bin/zsh && dscl . -read /Users/reto UserShell
Now I get the same environment for:
SSH as root
SSH as user
Terminal.app
Processes started from Java
And pretty much everything else so far
So far so good. Test program output:
/usr/bin:/bin:/usr/sbin:/sbin
return value: 0
/usr/bin:/bin:/usr/sbin:/sbin
return value: 0
/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
return value: 0
v14.17.0
return value: 0
6.14.13
return value: 0
I am trying to execute git clone in Java using Java's Runtime.getRuntime().exec() in Linux and the interpreter is /bin/bash. However, I get "no such file or directory" in error stream. I searched the stackoverflow and found no answer can solve my problem. Here is my program in Test.java:
import java.io.*;
public class Test {
public static void main(String[] args) throws IOException, InterruptedException {
String version = "10.1.1";
String repo_url = "https://github.com/postcss/postcss-url";
String directory = "./tmp";
String cmd = "\"/usr/bin/git clone --branch " + version + " " + repo_url + " --depth=1 " + directory + "\"";
// String cmd = "git -h";
String interpreter = "/bin/bash";
cmd = " -c "+ cmd;
System.out.println(interpreter + cmd);
Process process = Runtime.getRuntime().exec(new String[]{ interpreter, cmd });
print(process.getInputStream());
print(process.getErrorStream());
process.waitFor();
int exitStatus = process.exitValue();
System.out.println("exit status: " + exitStatus);
File[] files = new File(directory).listFiles();
System.out.println("number of files in the directory: " + files.length);
}
public static void print(InputStream input) {
new Thread(new Runnable() {
#Override
public void run() {
BufferedReader bf = new BufferedReader(new InputStreamReader(input));
String line = null;
try {
while ((line = bf.readLine()) != null) {
System.out.println(line);
}
} catch (IOException e) {
System.out.println("IOException");
}
}
}).start();
}
}
./tmp is surely an empty directory. I use javac Test.java to compile the code and then run java Test. Besides, I tried sudo java Test and got the same result. I get output like this:
/bin/bash -c "/usr/bin/git clone --branch 10.1.1 https://github.com/postcss/postcss-url --depth=1 ./tmp"
exit status: 127
/bin/bash: -c "/usr/bin/git clone --branch 10.1.1 https://github.com/postcss/postcss-url --depth=1 ./tmp": No such file or directory
Exception in thread "main" java.lang.NullPointerException
at Test.main(Test.java:18)
When I use "git -h" or "ls", it works just fine. But, this command /bin/bash -c "/usr/bin/git clone --branch 10.1.1 https://github.com/postcss/postcss-url --depth=1 ./tmp" works in shell but failed in Java. How can I solve this problem?
You have to pass -c as a separate parameter, and you shouldn't add literal double quotes to the command:
new String[]{ "bash", "-c", "git clone ..." }
This is because spaces and quotes are shell syntax, and Runtime.exec doesn't invoke one to run the command (which happens to be a shell invocation, but that's unrelated)
The program I'm working on uses ADB (Android Debug Bridge) to send files to my phone:
for (String s : files)
String cmd = "adb -s 0123456789ABCDEF push " + s + " /mnt/sdcard/" + s;
try {
InputStream is = Runtime.getRuntime().exec(cmd).getInputStream();
while (is.read() != -1) {}
} catch (IOException e) {
e.printStackTrace();
}
I want the program to wait until ADB finished the transmission, but ADB runs as a daemon and therefore never finishes. But the program continues immideately and somehow the files aren't sent to my phone (no exceptions in log). When I run the command from console, it's working without problems.
What am I doing wrong? How do I send files via ADB correctly?
NOTE: the is.read() == -1 won't work, because the ADB daemon writes all output to the system standard output. I've tried forwarding it into a textfile. It stayed empty and the output was still written to the terminal instead
EDIT: Reading the ErrorStream of the ADB process returned the adb help for each adb push-command. Again: The exact commands (copied from Eclipse console) work in a terminal
EDIT 2: Using a ProcessBuilder instead of RUntime.getRuntime.exec() resulted in the following error:
java.io.IOException: Cannot run program "adb -s 0123456789ABCDEF push "inputfile "outputfile""": error=2, File or directory not found
at the ProcessBuilder's start()-method
The same happens when using an absolute path for ADB (/usr/bin/adb). The inputfile and outputfile Strings are also absolute paths, like /home/sebastian/testfile and definitely exist. When running the commands from terminal (string "cmd" printed, copy&paste), evreything still works fine.
I finally got it working:
ProcessBuilder pb = new ProcessBuilder("adb", "-s", "0123456789ABCDEF", "push", inputfile, outputfile);
Process pc = pb.start();
pc.waitFor();
System.out.println("Done");
I don't know what problems ProcessBuilder has with spaces in a string, but finally, it's working...
I've solved in this way:
public class Utils {
private static final String[] WIN_RUNTIME = { "cmd.exe", "/C" };
private static final String[] OS_LINUX_RUNTIME = { "/bin/bash", "-l", "-c" };
private Utils() {
}
private static <T> T[] concat(T[] first, T[] second) {
T[] result = Arrays.copyOf(first, first.length + second.length);
System.arraycopy(second, 0, result, first.length, second.length);
return result;
}
public static List<String> runProcess(boolean isWin, String... command) {
System.out.print("command to run: ");
for (String s : command) {
System.out.print(s);
}
System.out.print("\n");
String[] allCommand = null;
try {
if (isWin) {
allCommand = concat(WIN_RUNTIME, command);
} else {
allCommand = concat(OS_LINUX_RUNTIME, command);
}
ProcessBuilder pb = new ProcessBuilder(allCommand);
pb.redirectErrorStream(true);
Process p = pb.start();
p.waitFor();
BufferedReader in = new BufferedReader(new InputStreamReader(p.getInputStream()));
String _temp = null;
List<String> line = new ArrayList<String>();
while ((_temp = in.readLine()) != null) {
System.out.println("temp line: " + _temp);
line.add(_temp);
}
System.out.println("result after command: " + line);
return line;
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
}
If you don't need env variables in your .bash_profile cut "-l" parameter.
I have a Mac but it should work on Linux also.
public static void adbpush() {
System.out.println("adb push....");
String[] aCommand = new String[] { adbPath, "push", inputFile(String),OutputDirectory };
try {
// Process process = new ProcessBuilder(aCommand).start();
Process process = Runtime.getRuntime().exec(aCommand);
process.waitFor(3, TimeUnit.SECONDS);
System.out.println("file pushed");
} catch (Exception e) {
e.printStackTrace();
}
}
It will be better to give full path for ADB execution: like this $ANDROID_HOME/platform-tools/adb devices
This is the full code you can use:
String cmd = "$ANDROID_HOME/platform-tools/adb devices";
ProcessBuilder processBuilder = new ProcessBuilder();
if (Config.osName.contains("Windows"))
processBuilder.command("cmd.exe", "/c", cmd);
else
processBuilder.command("bash", "-c", cmd);
Process process = processBuilder.start();
Process process = Runtime.getRuntime().exec("su");
DataOutputStream os = new DataOutputStream(process.getOutputStream());
string cmd = "/system/bin/input keyevent 23\n";
os.writeBytes(cmd);
the phone must be rooted. here I have executed adb command "input keyevent 23". remember when you execute adb command through su you does not need to add "adb shell input keyevent 23"
I need launch sh script from jenkins, it is simple, but my script change symlink for JAVA_HOME, in fact im switching between JDK versions using sh script. It works when Im launching job without jenkins(job writen on bash), but it does not working under jenkins... Jenkins remember JAVA_HOME after start and use this path... how can I change JAVA_HOME from sh script under jenkins ? may be from script invoke jenkins reload config if it's possible... thx for any help!
Just try as below ;
public static void execShellCmd(String cmd) {
try {
Runtime runtime = Runtime.getRuntime();
Process process = runtime.exec(new String[] { "/bin/bash", "-c", cmd });
int exitValue = process.waitFor();
System.out.println("exit value: " + exitValue);
BufferedReader buf = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line = "";
while ((line = buf.readLine()) != null) {
System.out.println("exec response: " + line);
}
} catch (Exception e) {
System.out.println(e);
}
}
For more details : Shell Script Running with java
Problem solved!!! soulution is simple, I have sh script that was launched in jenkins job like this:
./MY_SCRIPT.sh
And after that script launched under jenkins and I had problems with switching JAVA_HOME.
All is need to do launch script like this:
sh MY_SCRIPT.sh
and it will launched smt like from the system.
It is quite simple to run a Unix command from Java.
Runtime.getRuntime().exec(myCommand);
But is it possible to run a Unix shell script from Java code? If yes, would it be a good practice to run a shell script from within Java code?
You should really look at Process Builder. It is really built for this kind of thing.
ProcessBuilder pb = new ProcessBuilder("myshellScript.sh", "myArg1", "myArg2");
Map<String, String> env = pb.environment();
env.put("VAR1", "myValue");
env.remove("OTHERVAR");
env.put("VAR2", env.get("VAR1") + "suffix");
pb.directory(new File("myDir"));
Process p = pb.start();
You can use Apache Commons exec library also.
Example :
package testShellScript;
import java.io.IOException;
import org.apache.commons.exec.CommandLine;
import org.apache.commons.exec.DefaultExecutor;
import org.apache.commons.exec.ExecuteException;
public class TestScript {
int iExitValue;
String sCommandString;
public void runScript(String command){
sCommandString = command;
CommandLine oCmdLine = CommandLine.parse(sCommandString);
DefaultExecutor oDefaultExecutor = new DefaultExecutor();
oDefaultExecutor.setExitValue(0);
try {
iExitValue = oDefaultExecutor.execute(oCmdLine);
} catch (ExecuteException e) {
System.err.println("Execution failed.");
e.printStackTrace();
} catch (IOException e) {
System.err.println("permission denied.");
e.printStackTrace();
}
}
public static void main(String args[]){
TestScript testScript = new TestScript();
testScript.runScript("sh /root/Desktop/testScript.sh");
}
}
For further reference, An example is given on Apache Doc also.
I think you have answered your own question with
Runtime.getRuntime().exec(myShellScript);
As to whether it is good practice... what are you trying to do with a shell script that you cannot do with Java?
I would say that it is not in the spirit of Java to run a shell script from Java. Java is meant to be cross platform, and running a shell script would limit its use to just UNIX.
With that said, it's definitely possible to run a shell script from within Java. You'd use exactly the same syntax you listed (I haven't tried it myself, but try executing the shell script directly, and if that doesn't work, execute the shell itself, passing the script in as a command line parameter).
Yes it is possible to do so. This worked out for me.
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import org.omg.CORBA.portable.InputStream;
public static void readBashScript() {
try {
Process proc = Runtime.getRuntime().exec("/home/destino/workspace/JavaProject/listing.sh /"); //Whatever you want to execute
BufferedReader read = new BufferedReader(new InputStreamReader(
proc.getInputStream()));
try {
proc.waitFor();
} catch (InterruptedException e) {
System.out.println(e.getMessage());
}
while (read.ready()) {
System.out.println(read.readLine());
}
} catch (IOException e) {
System.out.println(e.getMessage());
}
}
Here is my example. Hope it make sense.
public static void excuteCommand(String filePath) throws IOException{
File file = new File(filePath);
if(!file.isFile()){
throw new IllegalArgumentException("The file " + filePath + " does not exist");
}
if(isLinux()){
Runtime.getRuntime().exec(new String[] {"/bin/sh", "-c", filePath}, null);
}else if(isWindows()){
Runtime.getRuntime().exec("cmd /c start " + filePath);
}
}
public static boolean isLinux(){
String os = System.getProperty("os.name");
return os.toLowerCase().indexOf("linux") >= 0;
}
public static boolean isWindows(){
String os = System.getProperty("os.name");
return os.toLowerCase().indexOf("windows") >= 0;
}
Yes, it is possible and you have answered it! About good practises, I think it is better to launch commands from files and not directly from your code. So you have to make Java execute the list of commands (or one command) in an existing .bat, .sh , .ksh ... files.
Here is an example of executing a list of commands in a file MyFile.sh:
String[] cmd = { "sh", "MyFile.sh", "\pathOfTheFile"};
Runtime.getRuntime().exec(cmd);
To avoid having to hardcode an absolute path, you can use the following method that will find and execute your script if it is in your root directory.
public static void runScript() throws IOException, InterruptedException {
ProcessBuilder processBuilder = new ProcessBuilder("./nameOfScript.sh");
//Sets the source and destination for subprocess standard I/O to be the same as those of the current Java process.
processBuilder.inheritIO();
Process process = processBuilder.start();
int exitValue = process.waitFor();
if (exitValue != 0) {
// check for errors
new BufferedInputStream(process.getErrorStream());
throw new RuntimeException("execution of script failed!");
}
}
As for me all things must be simple.
For running script just need to execute
new ProcessBuilder("pathToYourShellScript").start();
The ZT Process Executor library is an alternative to Apache Commons Exec. It has functionality to run commands, capturing their output, setting timeouts, etc.
I have not used it yet, but it looks reasonably well-documented.
An example from the documentation: Executing a command, pumping the stderr to a logger, returning the output as UTF8 string.
String output = new ProcessExecutor().command("java", "-version")
.redirectError(Slf4jStream.of(getClass()).asInfo())
.readOutput(true).execute()
.outputUTF8();
Its documentation lists the following advantages over Commons Exec:
Improved handling of streams
Reading/writing to streams
Redirecting stderr to stdout
Improved handling of timeouts
Improved checking of exit codes
Improved API
One liners for quite complex use cases
One liners to get process output into a String
Access to the Process object available
Support for async processes ( Future )
Improved logging with SLF4J API
Support for multiple processes
This is a late answer. However, I thought of putting the struggle I had to bear to get a shell script to be executed from a Spring-Boot application for future developers.
I was working in Spring-Boot and I was not able to find the file to be executed from my Java application and it was throwing FileNotFoundFoundException. I had to keep the file in the resources directory and had to set the file to be scanned in pom.xml while the application was being started like the following.
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<includes>
<include>**/*.xml</include>
<include>**/*.properties</include>
<include>**/*.sh</include>
</includes>
</resource>
</resources>
After that I was having trouble executing the file and it was returning error code = 13, Permission Denied. Then I had to make the file executable by running this command - chmod u+x myShellScript.sh
Finally, I could execute the file using the following code snippet.
public void runScript() {
ProcessBuilder pb = new ProcessBuilder("src/main/resources/myFile.sh");
try {
Process p;
p = pb.start();
} catch (IOException e) {
e.printStackTrace();
}
}
Hope that solves someone's problem.
Here is an example how to run an Unix bash or Windows bat/cmd script from Java. Arguments can be passed on the script and output received from the script. The method accepts arbitrary number of arguments.
public static void runScript(String path, String... args) {
try {
String[] cmd = new String[args.length + 1];
cmd[0] = path;
int count = 0;
for (String s : args) {
cmd[++count] = args[count - 1];
}
Process process = Runtime.getRuntime().exec(cmd);
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
try {
process.waitFor();
} catch (Exception ex) {
System.out.println(ex.getMessage());
}
while (bufferedReader.ready()) {
System.out.println("Received from script: " + bufferedReader.readLine());
}
} catch (Exception ex) {
System.out.println(ex.getMessage());
System.exit(1);
}
}
When running on Unix/Linux, the path must be Unix-like (with '/' as separator), when running on Windows - use '\'. Hier is an example of a bash script (test.sh) that receives arbitrary number of arguments and doubles every argument:
#!/bin/bash
counter=0
while [ $# -gt 0 ]
do
echo argument $((counter +=1)): $1
echo doubling argument $((counter)): $(($1+$1))
shift
done
When calling
runScript("path_to_script/test.sh", "1", "2")
on Unix/Linux, the output is:
Received from script: argument 1: 1
Received from script: doubling argument 1: 2
Received from script: argument 2: 2
Received from script: doubling argument 2: 4
Hier is a simple cmd Windows script test.cmd that counts number of input arguments:
#echo off
set a=0
for %%x in (%*) do Set /A a+=1
echo %a% arguments received
When calling the script on Windows
runScript("path_to_script\\test.cmd", "1", "2", "3")
The output is
Received from script: 3 arguments received
It is possible, just exec it as any other program. Just make sure your script has the proper #! (she-bang) line as the first line of the script, and make sure there are execute permissions on the file.
For example, if it is a bash script put #!/bin/bash at the top of the script, also chmod +x .
Also as for if it's good practice, no it's not, especially for Java, but if it saves you a lot of time porting a large script over, and you're not getting paid extra to do it ;) save your time, exec the script, and put the porting to Java on your long-term todo list.
I think with
System.getProperty("os.name");
Checking the operating system on can manage the shell/bash scrips if such are supported.
if there is need to make the code portable.
String scriptName = PATH+"/myScript.sh";
String commands[] = new String[]{scriptName,"myArg1", "myArg2"};
Runtime rt = Runtime.getRuntime();
Process process = null;
try{
process = rt.exec(commands);
process.waitFor();
}catch(Exception e){
e.printStackTrace();
}
Just the same thing that Solaris 5.10 it works like this ./batchstart.sh there is a trick I don´t know if your OS accept it use \\. batchstart.sh instead. This double slash may help.
for linux use
public static void runShell(String directory, String command, String[] args, Map<String, String> environment)
{
try
{
if(directory.trim().equals(""))
directory = "/";
String[] cmd = new String[args.length + 1];
cmd[0] = command;
int count = 1;
for(String s : args)
{
cmd[count] = s;
count++;
}
ProcessBuilder pb = new ProcessBuilder(cmd);
Map<String, String> env = pb.environment();
for(String s : environment.keySet())
env.put(s, environment.get(s));
pb.directory(new File(directory));
Process process = pb.start();
BufferedReader inputReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
BufferedWriter outputReader = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
BufferedReader errReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
int exitValue = process.waitFor();
if(exitValue != 0) // has errors
{
while(errReader.ready())
{
LogClass.log("ErrShell: " + errReader.readLine(), LogClass.LogMode.LogAll);
}
}
else
{
while(inputReader.ready())
{
LogClass.log("Shell Result : " + inputReader.readLine(), LogClass.LogMode.LogAll);
}
}
}
catch(Exception e)
{
LogClass.log("Err: RunShell, " + e.toString(), LogClass.LogMode.LogAll);
}
}
public static void runShell(String path, String command, String[] args)
{
try
{
String[] cmd = new String[args.length + 1];
if(!path.trim().isEmpty())
cmd[0] = path + "/" + command;
else
cmd[0] = command;
int count = 1;
for(String s : args)
{
cmd[count] = s;
count++;
}
Process process = Runtime.getRuntime().exec(cmd);
BufferedReader inputReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
BufferedWriter outputReader = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
BufferedReader errReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
int exitValue = process.waitFor();
if(exitValue != 0) // has errors
{
while(errReader.ready())
{
LogClass.log("ErrShell: " + errReader.readLine(), LogClass.LogMode.LogAll);
}
}
else
{
while(inputReader.ready())
{
LogClass.log("Shell Result: " + inputReader.readLine(), LogClass.LogMode.LogAll);
}
}
}
catch(Exception e)
{
LogClass.log("Err: RunShell, " + e.toString(), LogClass.LogMode.LogAll);
}
}
and for usage;
ShellAssistance.runShell("", "pg_dump", new String[]{"-U", "aliAdmin", "-f", "/home/Backup.sql", "StoresAssistanceDB"});
OR
ShellAssistance.runShell("", "pg_dump", new String[]{"-U", "aliAdmin", "-f", "/home/Backup.sql", "StoresAssistanceDB"}, new Hashmap<>());