Develop Reference

Java Runtime.exec() works for some command but not others [duplicate]

This question already has an answer here:
Why does Runtime.exec(String) work for some but not all commands?
(1 answer)
Closed 10 months ago.
I am doing an exercise related to Runtime.exec(), I understand that Runtime.exec is not a shell interpreter, that's why I execute "bash -c 'command'" instead, but for some reason, I can execute commands like ls bash -c 'ls' but not echo or redirection or multiple commands. These does not work:
bash -c 'echo 1234'
bash -c 'ls > abc'
bash -c 'ls;id'
bash -c 'ls -al'
Here is my java code:
import java.util.*;
import java.io.*;
public class runtime {
public static void main(String args[]) throws Exception{
String cmd = args[0];
System.out.println(cmd);
Process p = Runtime.getRuntime().exec(cmd);
OutputStream os = p.getOutputStream();
InputStream in = p.getInputStream();
DataInputStream dis = new DataInputStream(in);
String disr = dis.readLine();
while ( disr != null ) {
System.out.println("Out: " + disr);
disr = dis.readLine();
}
}
}
I run the above commands with the syntax:
java runtime "bash -c 'command'"
This works:
$ java runtime "bash -c 'ls'"
bash -c 'ls'
Out: Main.class
Out: Main.java
Out: runtime.class
Out: runtime.java
I am using openjdk 11.0.15 on Ubuntu 20.04 and zsh.
Can anyone tell me why Runtime doesn't work in this case? Thank you!

Because of shell parsing.
These are all concepts that the OS just does not have:
The concept of 'every space separates one argument from another (and the command from the list of arguments'). The concept that a single string can possibly run anything, in fact; at the OS level that's just not what the 'run this thing' API looks like. When you type commands on the command prompt, your shell app is 'interpreting' these strings into a command execution request.
The concept of figuring out that bash means /bin/bash, i.e. $PATH resolution.
The concept that *.txt is supposed to refer to all files that ends in .txt.
The concept that $FOO should be replaced with the value of the environment variable 'FOO'
The concept that ; separates 2 commands, and it's supposed to run both.
The concept that single and double quotes escape things. "Escape things" implies that things can cause interpretation to happen. The OS interprets nothing, therefore there's nothing to escape. Obvious, then, that the OS doesn't know what ' is, or ".
That >foo means: Please set the standard output of the spawned process such that it sends it all to file 'foo'.
In windows shells, that # in front of the command means 'do not echo the command itself'. This and many other things are shellisms: /bin/bash does that, maybe /bin/zsh does something else. Windows's built in shell thing definitely is quite different from bash!
Instead, an OS simply wants you to provide it a full path to an executable along with a list of strings, and pick targets for standard in, standard out, and standard err. It does no processing on any of that, just starts the process you named, and passes it the strings verbatim.
You're sort of half there, as you already figured out that e.g. ls >foo cannot work if you execute it on its own, but it can work if you tell bash to execute it. As ALL of that stuff in the above list? Your shell does that.
It gets more complicated: Turning *.txt into foo.txt bar.txt is a task of bash and friends, e.g. if you attempted to run: ls '*.txt' it does not work. But on windows, it's not the shell's job; the shell just passes it verbatim to dir, and it is the command's job to undo it. What a mess, right? Executing things is hard!
So, what's wrong here? Two things:
Space splitting isn't working out.
Quote application isn't being done.
When you write:
bash -c 'ls >foo'
in a bash shell, bash has to first split this up, into a command, and a list of arguments. Bash does so as follows:
Command: bash
arg1: -c
arg2: ls >foo
It knows that ls >foo isn't 2 arguments because, effectively, "space" is the bash operator for: "... and here is the next argument", and with quotes (either single or double), the space becomes a literal space instead of the '... move to next argument ...' operator.
In your code, you ask bash to run java, and then java to run bash. So, bash first does:
cmd: java
arg1: bash -c 'ls >foo'
With the same logic at work. Your java app then takes that entire string (that's args[0]: "bash -c 'ls >foo'"), and you then feed it to a method you should never use: Runtime.exec(). Always use ProcessBuilder, and always use the list-based form of it.
Given that you're using the bad method, you're now asking java to do this splitting thing. After all, if you just tell the OS verbatim, please run "bash -c 'ls >foo'", the OS dutifully reports: "I'm sorry, but file ./bash -c ;ls >foo' does not exist", because it does not processing at all". This is unwieldy so java's exec() method is a disaster you should never use: Because people are confused about this, it tries to do some extremely basic processing, except every OS and shell is different, java does not know this, so it does a really bad job at it.
Hence, do not use it.
In this case, java doesn't realize that those quotes mean it shouldn't split, so java tells the OS:
Please run:
cmd: /bin/bash (java DOES do path lookup; but you should avoid this, do not use relative path names, you should always write them out in full)
arg1: -c
arg2: 'ls
arg3: >foo'
and now you understand why this is just going completely wrong.
Instead, you want java to tell the OS:
cmd: /bin/bash
arg1: -c
arg2: ls >foo
Note: ls >foo needs to be one argument, and NOTHING in the argument should be containing quotes, anywhere. The reason you write:
/bin/bash -c 'ls >foo'
In bash, is because you [A] want bash not to treat that space between ls and >foo as an arg separator (you want ls >foo to travel to /bin/bash as one argument), and [B] that you want >foo to just be sent to the bash you're spawning and not to be treated as 'please redirect the output to file foo' at the current shell level.
Runtime.exec isn't a shell, so the quotes stuff? Runtime.exec has no idea.
This means more generally your plan of "I shall write an app where the entire argument you pass to it is just run" is oversimplified and can never work unless you write an entire quotes and escaper analyser for it.
An easy way out is to take the input, write it out to a shell script on disk, set the exec flag on it, and always run /bin/bash -c /path/to/script-you-just-wrote, sidestepping any need to attempt to parse anything in java: Let bash do it.
The ONE weird bizarro thing I cannot explain, is that literally passing 'ls' to /bin/bash -c, with quotes intact, DOES work and runs ls as normal, but 'ls *' does not, perhaps because now bash thinks you want executable file /bin/ls * which obviously does not exist (a star cannot be in a file name, or at least, that's not the ls executable, and it's not an alias for the ls built-in). At any rate, you want to pass ls without the quotes.
Let's try it!
import java.util.*;
import java.io.*;
public class runtime {
public static void main(String args[]) throws Exception{
ProcessBuilder pb = new ProcessBuilder();
pb.command("/bin/bash", "-c", "echo 1234");
// pb.command("/bin/bash", "-c", "'echo 1234'");
Process p = pb.start();
OutputStream os = p.getOutputStream();
InputStream in = p.getInputStream();
DataInputStream dis = new DataInputStream(in);
String disr = dis.readLine();
while ( disr != null ) {
System.out.println("Out: " + disr);
disr = dis.readLine();
}
int errCode = p.waitFor();
System.out.println("Process exit code: " + errCode);
}
}
The above works fine. Replace the .command line with the commented out variant and you'll notice it does not work at all, and you get an error. On my mac I get a '127' error; perhaps this is bash reporting back: I could not find the command you were attempting to execute. 0 is what you're looking for when you invoke waitFor: That's the code for 'no errors'.

Cannot run 'python -c ".." ' command from Java Process builder and Linux

I was developing my spring boot server on Windows. Now I have upgraded to Ubuntu 20.04.
the project executes a python script which should return a result as a txt file with this command:
python3 -c "from main import *;main(function,'/tmp/execution12480676806364930620/executionResponse.txt')"
Thanks to this code:
List<String> items = Arrays.asList(project.getExecutorType().buildAndGetExecutionCommandByProject(project));
ProcessBuilder pb = new ProcessBuilder(items);
pb.directory(new File(project.getPath()));
Process p = pb.start();
p.waitFor(5, TimeUnit.SECONDS);
when I print in the console the array passed in the Item variable:
[python3, -c, "from main import *;main(function, '/tmp/execution12480676806364930620/executionResponse.txt')"]
and the path of the array passed in pb.directory :
/tmp/execution12480676806364930620
My problem is that the project is not running and returning nothing.
when i go to the folder and run the same command from terminal everything works.
And that on windows 10 this same process worked fine.
Looking at similar issues I modified my code like this but it doesn't change anything:
List<String> items = Arrays.asList(project.getExecutorType().buildAndGetExecutionCommandByProject(project));
ProcessBuilder pb = new ProcessBuilder();
pb.command(items);
pb.redirectErrorStream(true);
pb.directory(new File(project.getPath()));
Process p = pb.start();
p.waitFor(5, TimeUnit.SECONDS);
What am I doing wrong?
Edit :
My command for read outputs :
private String inputStreamToString(InputStream inputStream){
return new BufferedReader(
new InputStreamReader(inputStream, StandardCharsets.UTF_8)).lines()
.collect(Collectors.joining("\n"));
}
And I call it like that :
System.out.println(this.inputStreamToString(p.getErrorStream()));
System.out.println(this.inputStreamToString(p.getInputStream()));
What works, when I just run "python main.py" I get the errors and print them out.
I can easily add the command at the end of the main file but I don't understand why the python -c "..." is not working? I am not receiving any errors ... I manage several languages ​​and this could be a problem for me later

Aha! On closer inspection I think you're right it's not executing anything (and thus not producing any output either normal or error for you to see).
You don't show how the array of strings is created, but your printout suggests you have actually put quotemarks in the third string. That's wrong. When you give the shell command line python -c "import this; dothat" the shell uses the quotemarks to control parsing of this command line, but it does not pass them to the python process; the args passed to the python process (shown vertically for clarity, and omitting the argv[0]=program used in C but omitted in Java) are actually
-c
import this; dothat
If you pass an argument actually containing quotemarks like
-c
"import this; dothat"
then python doesn't execute the commands import and dothat; instead it evaluates the string literal "import this; dothat" and (since it isn't running interactively) discards the result.
Try not including, or removing, the " at the beginning and end. But leave the ' inside the string value because you do want python to receive those.

Back to back commands in mac terminal via java Runtime

Basically I have 2 commands I need to execute via a java program the way you would if you were just typing it into terminal.
so like
cd /Users/nameOfUser/Desktop/someFolder/someSubFolder
and then another command I want to execute within that directory. Currently I am doing this:
Process navigate = Runtime.getRuntime().exec("cd /Users/nameOfUser/Desktop/someFolder/someSubFolder");
Process doSomething = Runtime.getRuntime().exec("commandInThatDirectory");
Which doesn't work, it doesn't throw an exception but the second process doesn't seem to take place in the directory specified before it. I am new to processes and runtimes so please bear with me :P.
Is their a way to execute the commands back to back within the same instance of terminal or at least a format for 1 command where you can specify the directory for another command to take place in? I'm a linux user so I don't know mac terminal very well sorry.

It can be done something like this. you can run any command by by placing a semicolon between the commands.
public class Main {
public static void main(String[] args) throws IOException {
ProcessBuilder pb1 = new ProcessBuilder(
"bash",
"-c",
"cd /Users/nameOfUser/Desktop/someFolder/someSubFolder;commandInThatDirectory");
pb1.redirectErrorStream(true);
Process p = pb1.start();
}
}

servlet multiple exec commands

I am writing a servlet to run terminal commands, I have to run three commands one after the another on the same terminal.
When I use exec as shown below I am able to get the result for the single command,
Runtime rt = Runtime.getRuntime();
process proc = rt.exec("zsh");
but when I attempt to run,
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec("zsh", "source ./myenv/bin/activate", "python runner.py");
it shows a warning (doesn't execute) as follows,
The method exec(String, String[], File) in the type Runtime is not applicable for the arguments (String, String, String).
I've also tried running three differed exec commands but they do not happen in the same sequence and on the same terminal, what should I do to make the above three commands to run on the same terminal sequentially and give me the output after the third command?
The three commands I intent to run are,
1. zsh
2. source ./myenv/bin/activate
3. python runner.py
all the three must be executed one after the other in the same sequence.

exec takes a String[] not a varargs list so you need to run:
Process proc = rt.exec(new String[] {"zsh", "source ./myenv/bin/activate",
"python runner.py"}, null, new File("parentdirectoryofmyenv"));
As shown you also need to include the working directory for the process or it won't be able to find "./myenv".
EDIT:
Simplifying this further. I am assuming that "source ./myenv/bin/activate" is setting the environment for the process. This can be done as follows:
Process proc = rt.exec(
new String[] {"python", "runner.py"},
new String[] {"ENV1=VAL1", "ENV2=VAL2"},
new File("parentdirectoryofrunnerpy"));

Execute Unix system command from JAVA problem

I am facing a weird issue with executing a system command from JAVA code.
Actually i want to get the Mac OSX system information from my JAVA App.
For that im using
Runtime.getRuntime().exec("system_profiler -detailLevel full");
This is working fine.If i print the output,it is cool.
But i want to write this information to a plist file for future use.For that im using the -xml argument of system_profiler.like,
String cmd = "system_profiler -detailLevel full -xml > "+System.getProperty( "user.home" )+"/sysinfo.plist";
Process p = Runtime.getRuntime().exec(cmd);
Basically this should create a plist file in the current users home directory.
But this seems to be not writing anything to file.
Am i missing something here ?

My Java is more than rusty, so please be gentle. ;-)
Runtime.exec() does not automatically use the shell to execute the command you passed, so the IO redirection is not doing anything.
If you just use:
"/bin/sh -c system_profiler -detailLevel full > path/file.plist"
Then the string will be tokenized into:
{ "/bin/sh", "-c", "system_profiler", "-detailLevel", "full", ">", "path/file.plist" }
Which also wouldn't work, because -c only expects a single argument.
Try this instead:
String[] cmd = { "/bin/sh", "-c", "system_profiler -detailLevel full > path/file.plist" };
Process p = Runtime.getRuntime.exec(cmd);
Of course, you could also just read the output of your Process instance using Process.getInputStream() and write that into the file you want; thus skip the shell, IO redirection, etc. altogether.

Christian.K is absolutely correct. Here is a complete example:
public class Hello {
static public void main (String[] args) {
try {
String[] cmds = {
"/bin/sh", "-c", "ls -l *.java | tee tmp.out"};
Process p = Runtime.getRuntime().exec (cmds);
p.waitFor ();
System.out.println ("Done.");
}
catch (Exception e) {
System.out.println ("Err: " + e.getMessage());
}
}
}
If you weren't using a pipe (|) or redirect (>), then you'd be OK with String cmd = "ls -l *.java", as in your original command.
If you actually wanted to see any of the output in your Java console window, then you'd ALSO need to call Process.getInputStream().
Here's a good link:
Running system commands in Java applications