Develop Reference

Difference when executing linux command from terminal and java runtime process

I'm looking a way to write running log of python which is executed by java app via script.
Let's say my script is:
import time
for x in range(120):
print("Running ", x)
time.sleep(1)
Here is my current solution:
Trigger script using java
String cmd = "python script.py";
var process = Runtime.getRuntime().exec(cmd, null, new File(sandboxPath));
Write log to new file:
String traceLogCmd = String.format("strace -p %s -s 9999 -e trace=write -o output.txt", process.pid());
Runtime.getRuntime().exec(traceLogCmd, null, new File(sandboxPath));
Now the problem is output.txt only has content whenever the python script is done executing so that I cannot tailf the output file.
Meanwhile if I execute python script.py and strace command dirrectly from terminal, the output is exactly what I expected.
Can someone correct me if I did something wrong or have a another way to get python log?
Thanks in advance.

Use ProcessBuilder instead of Runtime.exec(). More details: When Runtime.exec() won't
The following code will append to StringBuilder object output of the script sb.append(line);. It would not be difficult to write that content to a file.
Process p = new ProcessBuilder("sh", "-c", "python", "path-to-your-script").start();
String result = getCommandResult(p.getInputStream());
private static String getCommandResult(InputStream stream) throws IOException {
StringBuilder sb = new StringBuilder();
try (InputStreamReader isr = new InputStreamReader(stream);
BufferedReader in = new BufferedReader(isr)) {
String line;
while ((line = in.readLine()) != null) {
sb.append(line);
}
}
return sb.toString().trim();
}

Runtime.getRuntime().exec() on Android doesn't work [duplicate]

Consider the following code:
String commandf = "ls /etc | grep release";
try {
// Execute the command and wait for it to complete
Process child = Runtime.getRuntime().exec(commandf);
child.waitFor();
// Print the first 16 bytes of its output
InputStream i = child.getInputStream();
byte[] b = new byte[16];
i.read(b, 0, b.length);
System.out.println(new String(b));
} catch (IOException e) {
e.printStackTrace();
System.exit(-1);
}
The program's output is:
/etc:
adduser.co
When I run from the shell, of course, it works as expected:
poundifdef#parker:~/rabbit_test$ ls /etc | grep release
lsb-release
The internets tell me that, due to the fact that pipe behavior isn't cross-platform, the brilliant minds who work in the Java factory producing Java can't guarantee that pipes work.
How can I do this?
I am not going to do all of my parsing using Java constructs rather than grep and sed, because if I want to change the language, I'll be forced to re-write my parsing code in that language, which is totally a no-go.
How can I make Java do piping and redirection when calling shell commands?

Write a script, and execute the script instead of separate commands.
Pipe is a part of the shell, so you can also do something like this:
String[] cmd = {
"/bin/sh",
"-c",
"ls /etc | grep release"
};
Process p = Runtime.getRuntime().exec(cmd);

I ran into a similar problem in Linux, except it was "ps -ef | grep someprocess".
At least with "ls" you have a language-independent (albeit slower) Java replacement. Eg.:
File f = new File("C:\\");
String[] files = f.listFiles(new File("/home/tihamer"));
for (String file : files) {
if (file.matches(.*some.*)) { System.out.println(file); }
}
With "ps", it's a bit harder, because Java doesn't seem to have an API for it.
I've heard that Sigar might be able to help us:
https://support.hyperic.com/display/SIGAR/Home
The simplest solution, however, (as pointed out by Kaj) is to execute the piped command as a string array. Here is the full code:
try {
String line;
String[] cmd = { "/bin/sh", "-c", "ps -ef | grep export" };
Process p = Runtime.getRuntime().exec(cmd);
BufferedReader in =
new BufferedReader(new InputStreamReader(p.getInputStream()));
while ((line = in.readLine()) != null) {
System.out.println(line);
}
in.close();
} catch (Exception ex) {
ex.printStackTrace();
}
As to why the String array works with pipe, while a single string does not... it's one of the mysteries of the universe (especially if you haven't read the source code). I suspect that it's because when exec is given a single string, it parses it first (in a way that we don't like). In contrast, when exec is given a string array, it simply passes it on to the operating system without parsing it.
Actually, if we take time out of busy day and look at the source code
(at http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/lang/Runtime.java#Runtime.exec%28java.lang.String%2Cjava.lang.String[]%2Cjava.io.File%29), we find that is exactly what is happening:
public Process [More ...] exec(String command, String[] envp, File dir)
throws IOException {
if (command.length() == 0)
throw new IllegalArgumentException("Empty command");
StringTokenizer st = new StringTokenizer(command);
String[] cmdarray = new String[st.countTokens()];
for (int i = 0; st.hasMoreTokens(); i++)
cmdarray[i] = st.nextToken();
return exec(cmdarray, envp, dir);
}

Create a Runtime to run each of the process. Get the OutputStream from the first Runtime and copy it into the InputStream from the second one.

#Kaj accepted answer is for linux. This is the equivalent one for Windows:
String[] cmd = {
"cmd",
"/C",
"dir /B | findstr /R /C:"release""
};
Process p = Runtime.getRuntime().exec(cmd);

Run exec file using Java on Mac

I need to start a server using bash, so I had created an UNIX shell , but I am not able to execute it with Java from Eclipse.
I tried the following code which doesn't work :
Process proc = Runtime.getRuntime().exec(./startServer);
Here is content of the startServer file :
#!/bin/bash
cd /Users/sujitsoni/Documents/bet/client
npm start

You can try the following two options.
Option 1
Process proc = Runtime.getRuntime().exec("/bin/bash", "-c", "<Abosulte Path>/startServer");
Option 2
ProcessBuilder pb = new ProcessBuilder("/bin/bash", "-c", "<Absolute Path>/startServer");
pb.directory(new File("<Absolute Path>"));
Process proc = pb.start();

A couple Of things can go wrong:
The path to the file you have given might be wrong for eclipse it can take relative path but from the command line, it will take the absolute path.
error=13, Permission denied - If the script file doesn't have required permissions. In your scenario, that might not the case as you are not getting any error.
At last, you are executing the script by java program so the output of your script will not be printed out. In your scenario, this might be the case. You need to capture the output of script from BufferedReade and print it. ( In your case server might have started but you are not seeing the logs/output of the script.
See the code sample below for printing output.
public static void main(String[] args) throws IOException, InterruptedException {
Process proc = Runtime.getRuntime().exec("./startServer");
proc.waitFor();
StringBuffer output = new StringBuffer();
BufferedReader reader = new BufferedReader(new InputStreamReader(proc.getInputStream()));
String line = "";
while ((line = reader.readLine()) != null) {
output.append(line + "\n");
}
System.out.println(output);
}

How to get the output of a piped bash command in Java [duplicate]

Consider the following code:
String commandf = "ls /etc | grep release";
try {
// Execute the command and wait for it to complete
Process child = Runtime.getRuntime().exec(commandf);
child.waitFor();
// Print the first 16 bytes of its output
InputStream i = child.getInputStream();
byte[] b = new byte[16];
i.read(b, 0, b.length);
System.out.println(new String(b));
} catch (IOException e) {
e.printStackTrace();
System.exit(-1);
}
The program's output is:
/etc:
adduser.co
When I run from the shell, of course, it works as expected:
poundifdef#parker:~/rabbit_test$ ls /etc | grep release
lsb-release
The internets tell me that, due to the fact that pipe behavior isn't cross-platform, the brilliant minds who work in the Java factory producing Java can't guarantee that pipes work.
How can I do this?
I am not going to do all of my parsing using Java constructs rather than grep and sed, because if I want to change the language, I'll be forced to re-write my parsing code in that language, which is totally a no-go.
How can I make Java do piping and redirection when calling shell commands?

Write a script, and execute the script instead of separate commands.
Pipe is a part of the shell, so you can also do something like this:
String[] cmd = {
"/bin/sh",
"-c",
"ls /etc | grep release"
};
Process p = Runtime.getRuntime().exec(cmd);

I ran into a similar problem in Linux, except it was "ps -ef | grep someprocess".
At least with "ls" you have a language-independent (albeit slower) Java replacement. Eg.:
File f = new File("C:\\");
String[] files = f.listFiles(new File("/home/tihamer"));
for (String file : files) {
if (file.matches(.*some.*)) { System.out.println(file); }
}
With "ps", it's a bit harder, because Java doesn't seem to have an API for it.
I've heard that Sigar might be able to help us:
https://support.hyperic.com/display/SIGAR/Home
The simplest solution, however, (as pointed out by Kaj) is to execute the piped command as a string array. Here is the full code:
try {
String line;
String[] cmd = { "/bin/sh", "-c", "ps -ef | grep export" };
Process p = Runtime.getRuntime().exec(cmd);
BufferedReader in =
new BufferedReader(new InputStreamReader(p.getInputStream()));
while ((line = in.readLine()) != null) {
System.out.println(line);
}
in.close();
} catch (Exception ex) {
ex.printStackTrace();
}
As to why the String array works with pipe, while a single string does not... it's one of the mysteries of the universe (especially if you haven't read the source code). I suspect that it's because when exec is given a single string, it parses it first (in a way that we don't like). In contrast, when exec is given a string array, it simply passes it on to the operating system without parsing it.
Actually, if we take time out of busy day and look at the source code
(at http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/lang/Runtime.java#Runtime.exec%28java.lang.String%2Cjava.lang.String[]%2Cjava.io.File%29), we find that is exactly what is happening:
public Process [More ...] exec(String command, String[] envp, File dir)
throws IOException {
if (command.length() == 0)
throw new IllegalArgumentException("Empty command");
StringTokenizer st = new StringTokenizer(command);
String[] cmdarray = new String[st.countTokens()];
for (int i = 0; st.hasMoreTokens(); i++)
cmdarray[i] = st.nextToken();
return exec(cmdarray, envp, dir);
}

Create a Runtime to run each of the process. Get the OutputStream from the first Runtime and copy it into the InputStream from the second one.

#Kaj accepted answer is for linux. This is the equivalent one for Windows:
String[] cmd = {
"cmd",
"/C",
"dir /B | findstr /R /C:"release""
};
Process p = Runtime.getRuntime().exec(cmd);

How can I execute unix sort command (/usr/bin/sort) from Java?

I am hoping to leverage the unix sort command to sort a large text file in Java. I've tried executing sort with the process builder, but with no luck. However when I print the exact command it is going to execute and copy and paste it into the terminal, it works fine.
So far I've tried executing with /bin/sh -c "", making sure the directory the input file is and where the output file will be is fully permissioned (chmod 777) but with no luck.
Here is the code (if it looks funny, note is using some functions found in Guava)
File inputFile = new File(inputFileName);
//build the command (optional number of sort columns)
List<String> command = new LinkedList<String>();
command.addAll(ImmutableList.<String>of("sort","-t"+delimiter));
for (int i : sortFieldPositions) {
command.add("-k"+i+","+i);
}
command.addAll(ImmutableList.<String>of(inputFileName,">",outputFileName));
//for debugging: output the command that will be executed
System.out.println("Executing: "+Joiner.on(" ").join(command));
//construct and start the process
Process process = new ProcessBuilder(command).redirectErrorStream(true).directory(inputFile.getParentFile()).start();
//for debugging: save process output
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
StringBuilder outputStringBuilder = new StringBuilder();
for (String line; (line = bufferedReader.readLine()) != null; /*reading taking place in check */) {
System.out.println("FROM PROCESS: "+line);
outputStringBuilder.append(line);
}
bufferedReader.close();
if (process.exitValue() != 0) {
//something went wrong
throw new RuntimeException("Error code "+process.exitValue()+" executing command: "+Joiner.on(" ").join(command)+"\n"+outputStringBuilder.toString());
}
Unfortunately this does not work, with the following output:
Executing: sort -t, -k2,2 -k1,1 /tmp/java/TestDataSorterImporterInput.txt /tmp/java/TestDataSorterImporterOutput.txt
FROM PROCESS: sort: stat failed: >: No such file or directory
Edit: It may be helpful to note that if I remove saving the output (> outputfile) from the command, then the command executes without complaint and the sorted version appears in the output from the Processes' input stream)

It is the shell that knows how to perform output redirection. The sort program cannot do it on its own. So if you want redirection, you need to do /bin/sh -c ... to let she shell into the loop.
(You write that you have tried this, but something else must have gone wrong with that).

Try this:
String whatever = "filename";
Process p = Runtime.getRuntime().exec("sort -t -k2 2 -k1 1 " + whatever);
See this site.

Process process = new ProcessBuilder("/bin/bash", "-c", "sort -t'|' -k2").start();