Develop Reference

Java Runtime.exec() works for some command but not others [duplicate]

This question already has an answer here:
Why does Runtime.exec(String) work for some but not all commands?
(1 answer)
Closed 10 months ago.
I am doing an exercise related to Runtime.exec(), I understand that Runtime.exec is not a shell interpreter, that's why I execute "bash -c 'command'" instead, but for some reason, I can execute commands like ls bash -c 'ls' but not echo or redirection or multiple commands. These does not work:
bash -c 'echo 1234'
bash -c 'ls > abc'
bash -c 'ls;id'
bash -c 'ls -al'
Here is my java code:
import java.util.*;
import java.io.*;
public class runtime {
public static void main(String args[]) throws Exception{
String cmd = args[0];
System.out.println(cmd);
Process p = Runtime.getRuntime().exec(cmd);
OutputStream os = p.getOutputStream();
InputStream in = p.getInputStream();
DataInputStream dis = new DataInputStream(in);
String disr = dis.readLine();
while ( disr != null ) {
System.out.println("Out: " + disr);
disr = dis.readLine();
}
}
}
I run the above commands with the syntax:
java runtime "bash -c 'command'"
This works:
$ java runtime "bash -c 'ls'"
bash -c 'ls'
Out: Main.class
Out: Main.java
Out: runtime.class
Out: runtime.java
I am using openjdk 11.0.15 on Ubuntu 20.04 and zsh.
Can anyone tell me why Runtime doesn't work in this case? Thank you!

Because of shell parsing.
These are all concepts that the OS just does not have:
The concept of 'every space separates one argument from another (and the command from the list of arguments'). The concept that a single string can possibly run anything, in fact; at the OS level that's just not what the 'run this thing' API looks like. When you type commands on the command prompt, your shell app is 'interpreting' these strings into a command execution request.
The concept of figuring out that bash means /bin/bash, i.e. $PATH resolution.
The concept that *.txt is supposed to refer to all files that ends in .txt.
The concept that $FOO should be replaced with the value of the environment variable 'FOO'
The concept that ; separates 2 commands, and it's supposed to run both.
The concept that single and double quotes escape things. "Escape things" implies that things can cause interpretation to happen. The OS interprets nothing, therefore there's nothing to escape. Obvious, then, that the OS doesn't know what ' is, or ".
That >foo means: Please set the standard output of the spawned process such that it sends it all to file 'foo'.
In windows shells, that # in front of the command means 'do not echo the command itself'. This and many other things are shellisms: /bin/bash does that, maybe /bin/zsh does something else. Windows's built in shell thing definitely is quite different from bash!
Instead, an OS simply wants you to provide it a full path to an executable along with a list of strings, and pick targets for standard in, standard out, and standard err. It does no processing on any of that, just starts the process you named, and passes it the strings verbatim.
You're sort of half there, as you already figured out that e.g. ls >foo cannot work if you execute it on its own, but it can work if you tell bash to execute it. As ALL of that stuff in the above list? Your shell does that.
It gets more complicated: Turning *.txt into foo.txt bar.txt is a task of bash and friends, e.g. if you attempted to run: ls '*.txt' it does not work. But on windows, it's not the shell's job; the shell just passes it verbatim to dir, and it is the command's job to undo it. What a mess, right? Executing things is hard!
So, what's wrong here? Two things:
Space splitting isn't working out.
Quote application isn't being done.
When you write:
bash -c 'ls >foo'
in a bash shell, bash has to first split this up, into a command, and a list of arguments. Bash does so as follows:
Command: bash
arg1: -c
arg2: ls >foo
It knows that ls >foo isn't 2 arguments because, effectively, "space" is the bash operator for: "... and here is the next argument", and with quotes (either single or double), the space becomes a literal space instead of the '... move to next argument ...' operator.
In your code, you ask bash to run java, and then java to run bash. So, bash first does:
cmd: java
arg1: bash -c 'ls >foo'
With the same logic at work. Your java app then takes that entire string (that's args[0]: "bash -c 'ls >foo'"), and you then feed it to a method you should never use: Runtime.exec(). Always use ProcessBuilder, and always use the list-based form of it.
Given that you're using the bad method, you're now asking java to do this splitting thing. After all, if you just tell the OS verbatim, please run "bash -c 'ls >foo'", the OS dutifully reports: "I'm sorry, but file ./bash -c ;ls >foo' does not exist", because it does not processing at all". This is unwieldy so java's exec() method is a disaster you should never use: Because people are confused about this, it tries to do some extremely basic processing, except every OS and shell is different, java does not know this, so it does a really bad job at it.
Hence, do not use it.
In this case, java doesn't realize that those quotes mean it shouldn't split, so java tells the OS:
Please run:
cmd: /bin/bash (java DOES do path lookup; but you should avoid this, do not use relative path names, you should always write them out in full)
arg1: -c
arg2: 'ls
arg3: >foo'
and now you understand why this is just going completely wrong.
Instead, you want java to tell the OS:
cmd: /bin/bash
arg1: -c
arg2: ls >foo
Note: ls >foo needs to be one argument, and NOTHING in the argument should be containing quotes, anywhere. The reason you write:
/bin/bash -c 'ls >foo'
In bash, is because you [A] want bash not to treat that space between ls and >foo as an arg separator (you want ls >foo to travel to /bin/bash as one argument), and [B] that you want >foo to just be sent to the bash you're spawning and not to be treated as 'please redirect the output to file foo' at the current shell level.
Runtime.exec isn't a shell, so the quotes stuff? Runtime.exec has no idea.
This means more generally your plan of "I shall write an app where the entire argument you pass to it is just run" is oversimplified and can never work unless you write an entire quotes and escaper analyser for it.
An easy way out is to take the input, write it out to a shell script on disk, set the exec flag on it, and always run /bin/bash -c /path/to/script-you-just-wrote, sidestepping any need to attempt to parse anything in java: Let bash do it.
The ONE weird bizarro thing I cannot explain, is that literally passing 'ls' to /bin/bash -c, with quotes intact, DOES work and runs ls as normal, but 'ls *' does not, perhaps because now bash thinks you want executable file /bin/ls * which obviously does not exist (a star cannot be in a file name, or at least, that's not the ls executable, and it's not an alias for the ls built-in). At any rate, you want to pass ls without the quotes.
Let's try it!
import java.util.*;
import java.io.*;
public class runtime {
public static void main(String args[]) throws Exception{
ProcessBuilder pb = new ProcessBuilder();
pb.command("/bin/bash", "-c", "echo 1234");
// pb.command("/bin/bash", "-c", "'echo 1234'");
Process p = pb.start();
OutputStream os = p.getOutputStream();
InputStream in = p.getInputStream();
DataInputStream dis = new DataInputStream(in);
String disr = dis.readLine();
while ( disr != null ) {
System.out.println("Out: " + disr);
disr = dis.readLine();
}
int errCode = p.waitFor();
System.out.println("Process exit code: " + errCode);
}
}
The above works fine. Replace the .command line with the commented out variant and you'll notice it does not work at all, and you get an error. On my mac I get a '127' error; perhaps this is bash reporting back: I could not find the command you were attempting to execute. 0 is what you're looking for when you invoke waitFor: That's the code for 'no errors'.

Runtime.getRuntime().exec how to run goto commands

I'm trying to run small command-line code in java application to delete itself. So the command would keep running and keeps trying to delete the file, and once the application is closed it would be deleted.
I've tired passing the command into Runtime.getRuntime().exec but still unable to have it work.
eg:
SET file="program.jar"
goto :d
:r
timeout /t 1
:d
del %file%
if exist %file% goto :r
Tried doing this which obviously looks wrong. I've tired using ; instead of && but doesn't work either.
Runtime.getRuntime().exec("cmd /c \"SET file=\"program.jar\" && goto :d && :r && timeout /t 1 && :d && del %file% && if exist %file% goto :r\"");
Works perfectly well in a .bat file but how do i implement it into java .exec() method.
I could just run the .bat file but I would want all the code be contained inside java.

First, you can't use labels an gotos in a cmd one-liner. Use some kind of while TRUE loop instead. In cmd terms: for /L %a in (1 0 2) do ...
Second, you need to apply Delayed Expansion. Proof:
==> SET "_file=init.ext"
==> SET "_file=prog.jar" & for /L %a in (1 1 2) do #echo %_file% %time% & timeout /T 3 1>NUL
init.ext 8:05:55,33
init.ext 8:05:55,33
==> set _file
_file=prog.jar
In above example, %_file% and %time% variables are expanded in parse time.On the other side, with delayed expansion enabled: !_file! and !time! variables are expanded in execution time:
==> SET "_file=init.ext"
==> cmd /E /V /C SET "_file=prog.jar" ^& for /L %a in (1 1 2) do #echo !_file! !time! ^& timeout /T 3 1^>NUL
prog.jar 8:08:55,42
prog.jar 8:08:58,18
Hence, your one-liner could look as follows (verified from Windows cmd CLI for a locked file; loops until unlocked):
cmd /E /V /C SET "_file=program.jar" ^& for /L %a in (1 0 2) do if exist "!_file!" (del "!_file!" ^& timeout /T 2) else (exit /B)

Running multiple cmd.exe processes from a batch file in Java

I'm currently revising a small application that I wrote as a demonstration for a security course I'm taking. The prof loved it and has requested some modifications that I'm unable to figure out. I would like to add the ability to pass in as a parameter to the batch file a few variables from my code (i.e. the number of cmd.exe instances to run and in the code below, the message variable).
Using an answer found at Launching multiple shell prompts from a single batch file among other helpful SO threads, here's an example of the code I've put together:
The batch file:
#echo off
if not "%1" == "" goto :%1
SET message="The random number is: "
SET /a rand=%RANDOM%
start "Job 1" "%~dpfx0" job1
start "Job 2" "%~dpfx0" job2
start "Job 3" "%~dpfx0" job3
goto :eof
:job1
call :showMessage %message% %rand%
:job2
call :showMessage %message% %rand%
:job3
call :showMessage %message% %rand%
:showMessage
echo %~1 %~2
This works great to spawn 3 instances of cmd.exe. I'm stumped, though, as to how I can dynamically choose the number of jobs to run and how to pass in a variable to the batch file with the above configuration. I figure I need a FOR loop, but I'm uncertain how to do it either with the above setup or a new way entirely.
I'm 110% sure the line from the batch file "if not "%1" == "" goto :%1" is absolutely necessary as without it, the system will continuously open cmd.exe processes (experimentation resulted in several hard resets). It interferes with passing in parameters to the batch file, but I am not able to find a way around using it.
For reference:
The batch file is executed in Java via the following code:
public class CmdProcess{
Runtime rt;
public cmdProcess() {
rt = Runtime.getRuntime();
}
public void startSpawning() {
try {
rt.exec("cmd.exe /C example.bat");
} catch (Exception e) {
e.printStackTrace();
}
}
}
I call this class from the gui code with the following listener:
private void setupListeners() {
class spawnButtonListener implements ActionListener {
public void actionPerformed(ActionEvent ae) {
// Fire off 3 cmd.exe processes that show a message
CmdProcess proc = new CmdProcess ();
proc.startSpawning();
// Increment the local counter and display on the client
processCounter += 3;
processCounterTextField.setText(Integer.toString(processCounter));
}
}
tl;dr: I need to know if it's possible with DOS magic in a batch file executed with Java to pass in the parameters necessary to run any number of cmd.exe instances that will each perform the same command.
EDIT TO INCLUDE SOLUTION:
Based on input from #JosefZ, here is a solution that works the way I need it to:
#echo off
if not %1 == "" (
set /A noiter=%1
) Else (
set /A noiter=1
)
SET message="The random number is: "
SET /a rand=%RANDOM%
if %noiter% == 0 (
call :showMessage %message% %rand%
) Else (
For /L %%G IN (1, 1, %noiter%) do (
start "Job %%G" "%~dpfx0" job%%G
set /a rand=%RANDOM%
:job%%G
call :showMessage %message% %rand%
)
)
)
:showMessage
echo %~1 %~2
If I wanted to pass in additional parameters, I would handle them in the first if block...
#echo off
if not %1 == "" (
rem passed in parameters
set /A noiter=%1
set /A message=%2
set /A rand=%3
) Else (
rem default parameters
set /A noiter=1
set message="The random number is: "
set /a rand=%RANDOM%
)

You know we can call a script with a parameter. Let's check this parameter as follows: if it's a numeric value, consider this is instances count of cmd.exe to be launched and remember it to a numeric environment variable: set /A "noiter=%1"; otherwise it's simply a flag to end. Here's the code:
if not "%1" == "" (
set /A "noiter=%1"
) Else (
set /A "noiter=1"
)
Sample an effect of set /A "noiter=text" yoursef. Now we can test value of the noiter variable:
if "%noiter%" == "0" (
call :showMessage %message% %rand%
) Else (
rem loop here
)
Finally, substitute next code snippet in place of above's rem loop here:
rem loop from 1 stepping 1 to "noiter"
For /L %%G IN (1, 1, %noiter%) do (
rem for testing purposes only: echo our command line first
echo start "Job %%G" "%~dpnx0" job%%G
rem for testing purposes only: simulate real call
set /a "rand=%RANDOM%"
call :showMessage %message% %rand%
)
This is a raw blueprint only; there are still some issues: e.g. %random% tends to stay constant :-)
So, learn more on set command, setlocal, enabledelayedexpansion etc. etc.

Batch File: "Environment Variable Not Defined"

I am working with a Java program that calls an external batch file and passes in an array of commands. I have a loop in the batch file that looks like this:
set paramCount=0
for %%x in (%*) do (
set /A paramCount+=1
set list[!paramCount!]=%%x
)
The parameters are a bunch of directories, stored as strings, like this:
String[] commands = {"cmd.exe",
"/C",
"C:\Users\user\Documents",
"C:\Users\user\Pictures",
"C:\Users\user\Videos"}
As you can see, my for loop is supposed to loop through the list of the arguments passed to the batch file (%*) and emulate an array within the script (Since the first two elements in the command array are used to start the command process, that only leaves the directories to be looped through). The program was working just fine until the other day, when I suddenly started getting an error saying the following:
Environment variable list[ not defined
I had not made any changes to the batch file at all and it seemed to stop working for no reason. In case it's necessary information, I'm using a process builder to run the process:
ProcessBuilder pb = new ProcessBuilder(commands);
Process p = pb.start();
Supposedly using this syntax for an array in a batch file is okay, so I'm not sure why it doesn't accept it. I appreciate any help you can provide in this matter. I've run into a lot of roadblocks with this program, and while I've been able to solve 90% of them, the remaining 10% have begun to drive me crazy! Thanks!
EDIT:
I have re-written the loop and added in some echo commands to make debugging easier. When I run the batch file, though, nothing prints to the screen as a result of the echo but I still get the same error:
#echo off
setlocal enableDelayedExpansion
set paramCount=0
for %%x in (%*) do (
echo !paramCount!
echo %%x
set list[!paramCount!]=%%x
set /A paramCount=paramCount+1
)
I also forgot to mention that the program works fine when I run the Java from Eclipse; it calls the batch files correctly and all works as expected. I don't get the error until I export the project to a runnable JAR and try running that.
EDIT 2:
After going through my batch file code again (I wrote it a while ago), I found only one line that looks like it might cause this problem. The odd thing is that I used an almost identical code example I found somewhere else to model it, and it worked for a long time without ever giving the error. It's a loop, designed to loop through the elements of the list "array" created in the first loop:
for /F "tokens=2 delims==" %%d in ('set list[') do (
set /A paramCount+=1
set _dir=%%d
set _dir=!_dir:"=!
if NOT "%%d"=="nul" set "dirs[!paramCount!]=!_dir!"
)
As you can see, the first line has a segment that says set list[, which looks odd to me. However, as I mentioned, it worked fine for quite a while.

The code you posted cannot give that error message. There must be some other code in your script that causes that error.
Only a SET statement without an = could give that error. After parsing and expansion, the offending statement must look like one of the following:
set list[
set "list[
set "list["
Actually, you could get that error in the presence of an = if there are quotes that are misplaced. For example, the following will give that error because all text after the last " is ignored:
set "list["1]=value
The 1]=value appears after the last " and is ignored, leaving set "list[".
You might consider enabling ECHO so that you can pinpoint exactly where the error message is occurring. Then you need to figure out what conditions could lead to the error at that point.
Update in response to "Edit 2:" in question
That IN() clause in the newly posted code is almost assuredly the point where the error message is generated. It indicates that the list "array" is not defined when the FOR /F loop is executed. The question is, why not? Things to look for:
Is something preventing the earlier FOR loop that defines the list "array" from running?
Are you sure the parameters are getting passed properly to the script? If there are no parameters, then there will not be an array.
Is something undefining the array before the 2nd FOR loop has a chance to execute?
I suggest you put in some diagnostic ECHO statements to help debug. For example, ECHO BEFORE 1ST LOOP immediately before the loop that defines the array to make sure the loop is being reached. Or ECHO ARGS = %* at the beginning of the script to make sure the parameters are getting passed properly. Happy sleuthing :-)

I tried the code bellow and it worked for me when calling from command line.
#echo off
setlocal enabledelayedexpansion
set paramCount=0
for %%x in (%*) do (
set /A paramCount=!paramCount!+1
set list[!paramCount!]=%%x
)
echo !list[1]!
echo !list[2]!
echo !list[3]!

I mentioned it in the comments but I figured I'd post an answer, for anyone else who's wondering:
As previously mentioned, the program was running fine in Eclipse, the batch files were being called and ran as expected. However, prior to making some changes to the batch files, I had created .exe's from them, and apparently was running the .exe's instad of the .bat's. It was a very stupid mistake on my part, and the problem was caused by some errors in the previous version of the batch file. If I remember correctly, that error was because the "array element" was empty. So I ended up doing a check to make sure it wasn't empty before operating on it. The following is the code i am currently using, and it works as expected:
#echo off
setlocal ENABLEDELAYEDEXPANSION
set /A paramCount=-3
for %%x in (%*) do (
set list[!paramCount!]=%%x
set /A paramCount=paramCount+1
)
set argA=%list[-3]%
set argB=%list[-2]%
set argC=%list[-1]%
for /F "tokens=2 delims==" %%a in ('set list[-') do SET "%%a="
set list[-3]=""
set list[-2]=""
set list[-1]=""
set paramCount=0
for /F "tokens=2 delims==" %%d in ('set list[') do (
if not "%%d"=="" (
set _dir=%%d
set _dir=!_dir:"=!
if NOT "%%d"=="nul" set "dirs[!paramCount!]=!_dir!"
)
set /A paramCount+=1 )
Thanks for all the answers, folks!

Windows batch file to run jar file multiple times

Id like to make a batch file that runs a jar X amount of times from user input. I have looked for how to handle user input, but I am not totally sure.
In this loop I'd like to increase the parameters I am sending to the jar.
As of now, I do not know
manipulate the variables in the for loop, numParam, strParam
So, when i run this little bat file from the command line, I get am able to do user input, but once it gets to the for loop, it spits out "The syntax of the command is incorrect
So far I have the following
#echo off
echo Welcome, this will run Lab1.jar
echo Please enter how many times to run the program
:: Set the amount of times to run from user input
set /P numToRun = prompt
set numParam = 10000
set strParam = 10000
:: Start looping here while increasing the jar pars
:: Loop from 0 to numToRun
for /L %%i in (1 1 %numToRun%) do (
java -jar Lab1.jar %numParam% %strParam%
)
pause
#echo on
Any suggestion would be helpful
Edit:
With the recent change, it doesnt seem to run my jar file. or at least doesnt seem to run my test echo program. It seems that my user input variable is not being set to what I have inputted, it stays at 0

If you read the documentation (type help for or for /? from the command line) then you will see the correct syntax for executing a FOR loop a fixed number of times.
for /L %%i in (1 1 %numToRun%) do java -jar Lab1.jar %numParam% %strParam%
If you want to use multiple lines, then you must either use line continuation
for /L %%i in (1 1 %numToRun%) do ^
java -jar Lab1.jar %numParam% %strParam%
or parentheses
for /L %%i in (1 1 %numToRun%) do (
java -jar Lab1.jar %numParam% %strParam%
REM parentheses are more convenient for multiple commands within the loop
)

What happened was for my last issues was something with how the variables expanded. This was actually answer at dreamincode.net: Here
Final code:
#echo off
echo Welcome, this will run Lab1.jar
:: Set the amount of times to run from user input
set /P numToRun= Please enter how many times to run the program:
set /a numParam = 1000
set /a strParam = 1000
setlocal enabledelayedexpansion enableextensions
:: Start looping here while increasing the jar pars
:: Loop from 0 to numToRun
for /L %%i in (1 1 %numToRun%) do (
set /a numParam = !numParam! * 2
set /a strParam = !strParam! * 2
java -jar Lab1.jar !numParam! !strParam!
:: The two lines below are used for testing
echo %numParam% !numParam!
echo %strParam% !strParam!
)
#echo on