I would like to run a python command within Java.
The format of the command is as follows:
python abc.py TAG < xyz.txt
When I use Java's Process:
Process p=Runtime.getRuntime().exec("python abc.py TAG < xyz.txt");
I get an incorrect argument error because the < xyz.txt is being interpreted as arguments to abc.py as opposed to the terminal meaning of piping xyz.txt.
Is there any way to run a java command that has the piping functionality?
exec() does not involve a shell. You can't do things like that.
You either need to exec() a shell and have it run the python process as you're trying, or you need to exec() the python process, open and read the xyz.txt file with Java, then send the data to the exec'd process' standard in (You can get that stream from the Process object).
Related
Is this a simple and good way to execute a Shell command via Java?
Runtime.getRuntime().exec( some command );
Or is this bad practice?
It depends.
The original purpose and basic functionality of a Unix shell is to let you run programs, optionally passing them arguments. For example the command ls runs the ls program, and the command grep foo bar runs the grep program with the arguments foo and bar. If your command only runs a (fixed) program with fixed if any arguments, Runtime.exec can do it. There are two subcases:
the overloads taking a String parse the line into 'words' (program name and arguments) using any whitespace; this is essentially the same as the default parsing (with no quoting) done by standard shells.
if you need any different parsing, for example if your command would use any quoting in shell, you must do that parsing yourself and pass the results to one of the overloads taking a String[].
But note that when you run a program from an interactive shell -- one using a terminal or equivalent (sometimes called a console) for input and output -- the program's input and output default to that terminal. The I/O for a program run by Runtime.exec is always pipes from and to the Java process, and some programs behave differently when their input and/or output is/are pipe(s) -- or file(s) -- instead of a terminal. Plus you must write code to send (write) any desired input and receive (read) any output. Of course, shells can be and sometimes are run without a terminal too.
However, shells can be and routinely are used to do much much more than the basics:
shell can execute commands with contents different from the input by variable (formally parameter) substitution (possibly with modification/editing), command substitution, process substitution, special notations like squiggle and bang, and filename expansion aka 'globbing' (so called because in the early versions of Unix it was done by a separate program named glob). Runtime.exec doesn't do these, although you can write Java code to produce the same resulting command execution by very different means.
shell executes some commands directly in the shell rather than by running a program, because these commands affect the shell process itself,
like cd umask ulimit exec source/. eval exit alias/unalias, or variables in the shell like set shift unset export local readonly declare typeset let read readarray/mapfile,
or child process like jobs fg bg, or special parsing like [[ ]] and (( )) (in some shells). These are called 'builtin' and Runtime.exec can't do them,
with two partial exceptions: it can run a program with a different working directory and/or env var settings, equivalent to having previously executed cd or export or equivalent.
Shell also often has builtins that duplicate, or modify, a 'normal' program; these commonly include test/[ echo printf kill time. Runtime.exec can only do the program version, not the builtin version.
shell has control structures (compound commands) like if/then/else/elif/fi and while/for/do/done and trap && || ( ) { }. Runtime.exec can't do these, although in some cases you could use Java logic to produce the same results.
shell can also have user-defined functions and aliases that can be used as commands; Runtime.exec does not.
shell can redirect the I/O of programs it runs, including forming pipes. Runtime.exec can't do these, but see below.
Since 1.5, Java also has ProcessBuilder, which provides the same functionality and more, in a more flexible and arguably clearer API, and thus is generally recommended instead. ProcessBuilder does support redirecting I/O for the program it runs, including using the terminal/console if the JVM was run on/from one (which is not always the case), and since 9 it can build a pipeline. It does not have the word-splitting functionality of Runtime.exec(String) but you can easily get the same result with string.split("[ \t]+") or in most cases just " +".
Note shell is itself a program, so you can use either Runtime.exec or ProcessBuilder to run a shell and pass it a command, either as an argument using option -c (on standard shells at least) or as input, and unsurprisingly this shell command can do anything a shell command can do.
But this can be a portability issue because different systems may have different shells, although any system claiming Unix certification or POSIX conformance must have a shell named sh that meets certain minimum requirements.
The actual shell used on different systems might be any of bash dash ksh ash or even more. OTOH this is true for other programs as well; some programs that typically differ significantly on different systems are awk sed grep and anything to do with administration like netstat.
A few of the existing Qs that show shell commands that don't work in Runtime.exec at least as-is:
a command for sherlock.py is interpreted differently from linux command line and java process api
Execute shell script multiple commands in one line using Process Builder in Java (Unix)
Check in Java if a certain application is in focus
Problem in executing command on AIX through Java
ProcessBuilder doesn't recognise embedded command
File not Found when executing a python scipt from java
Java system command to load sqlite3 db from file fails
Curl To Download Image In JAVA
Keytool command does not work when invoked with Java
use javap from within a java program on all the files
Using SSMTP and ProcessBuilder
Process Builder Arguments
Whitespace in bash path with java
java.lang.Runtime exception "Cannot run program"
Why does Runtime.exec(String) work for some but not all commands?
How to save Top command output in a text or csv file in java?
Execute bash-command in Java won't give a return
Using Java's Runtime.getRuntime().exec I get error with some commands, success with others -- how can this be determined?
Java and exec command - pipe multiple commands
Java exec() does not return expected result of pipes' connected commands
How to make pipes work with Runtime.exec()?
How to use Pipe Symbol through exec in Java
In Runtime.getRuntime().exec() getting error: /bin/bash: No such file or directory
Java exec linux command
How to use pipes in a java Runtime.exec
Java Runtime.getRuntime().exec and the vertical bar
Whenever I execute terminal command from code it gives "cannot run program" error=2 No such file or directory
Command line proccess read linux in java
Java Command line system call does not work properly
I'm trying to open gnome-terminal to execute a command from Java by the following code:
Runtime.getRuntime().exec(new String[]{"gnome-terminal", "-e", "command"});
But this code will show terminal for user, how I can open gnome-terminal in background?
You don't need to open a terminal visually to execute a command. Shell interpreter is a program which you can invoke without the terminal window. What terminal window does is it just sends what you type on the terminal, interacts with the underlying shell interpreter and gives you the results. When you need to run a OS specific command you would want to directly call the shell interpreter you want to with(i.e bash, csh, ksh) and get the result from within the Java program.
Working with commands and reading output from them is not that easy sometimes. So I would suggest you to have a look at Apache Exec which will ease reading data from externally invoked program.
I am trying to convert various xls files into csv. when I execute the following command in the terminal it works fine
libreoffice --headless --convert-to csv --outdir
/Data/edennis/ /Data/edennis/2013-10/*.xls
but when I try with runtime exec it does not.
Research I've done:
According to this thread Java Runtime exec() behavior cannot execute system commands like echo, but libreoffice is not a system command, isn't it an executable program ?
Java runtime execThis thread recommends to use processBuilder, but not sure if this is what I would need to do in my case.
According to the Java Doc:
EXEC:
Executes the specified string command in a separate process with the
specified environment.
First, there is no reason why Runtime.exec should not be able to run /bin/echo (if available).
Second, yes, use ProcessBuilder.
Third, your problems stem from using shell syntax for file patterns like *.xls. Runtime.exec calls the program you specify, not a shell that would do filename expansion. If you need to do filename expansion, run a shell like:
"sh -c libreoffice --blabla *.xls"
I am trying to build a very simple python script to automate minifying/combining some css/js assets.
I am not sure how to properly handle the minification step. I use yui-compressor and usually call the jar directly from the command line.
Assuming the build script is in the same directory as rhino js.jar and yui-compressor.jar, I'd be able to compress a css/js file like so:
java -cp js.jar -jar yuicompressor-2.4.4.jar -o css/foo.min.css css/foo.css
Calling that from the terminal works fine, but in the python build file, it does not
eg, os.system("...")
The exit status being returned is 0, and no output is being returned from the command (for example, when using os.popen() instead of os.system())
I'm guessing it has something to do with paths, perhaps with java not resolving properly when calling to os.system()… any ideas?
Thanks for any help
I have a somewhat similar case, when I want a python program to build up some commands and then run them, with the output going to the user who fired off the script. The code I use is:
import subprocess
def run(cmd):
call = ["/bin/bash", "-c", cmd]
ret = subprocess.call(call, stdout=None, stderr=None)
if ret > 0:
print "Warning - result was %d" % ret
run("javac foo.java")
run("javac bar.java")
In my case, I want all commands to run error or not, which is why I don't have an exception raised on error. Also, I want any messages printed straight to the terminal, so I have stdout and stderr be None which causes them to not go to my python program. If your needs are slightly different for errors and messages, take a look at the http://docs.python.org/library/subprocess.html documentation for how to tweak what happens.
(I ask bash to run my command for me, so that I get my usual path, quoting etc)
os.system should return 0 when the command executes correctly. 0 is the standard return code for success.
Does it print output when run from the command line?
Why would you want to do this in Python? For tasks like this, especially Java, you are better off using Apache Ant. Write commands in xml and then ant runs for you.
I have my minecraft_server.jar file as my source. I want to run that and take the output of it to my .jar application. How?
Use Runtime.exec to start the process and use Process.getOutputStream on the returned process object to access the output of that process:
Process proc = Runtime.getRuntime().exec("java -jar minecraft_server.jar");
OutputStream out = proc.getOutputStream();
/* read output using out */
The simple solution is system dependent. For example if you are using a Linux / Unix shell:
java -jar minecraft_server.jar | java -jar my_app.jar
You could also modify your application so that it launched the "minecraft server" application as an external command and read its output. But it is probably not worth the effort. (Hint, #casablanca's answer leaves out a lot of tedious detail such as handling exceptions, dealing with the standard error stream, dealing with return codes, the pathname of the java command, etc)