Below is a python script that executes a linux bash command "echo Hello World > ./output"
import os
os.system("bash -c \"echo Hello World > ./output\"");
I am trying to do the same with Java. Below is my best effort, following the instructions I found here: Want to invoke a linux shell command from Java
import java.io.IOException;
public class callCommand {
public static void main(String[] args) {
try {
Process p = Runtime.getRuntime().exec(
new String[]{"bash","-c",
"\"echo Hello World > ./output\""});
} catch(IOException e) {
e.printStackTrace();
}
}
}
It compiles without issue, and runs without complaint, but no output file is generated.
The extra quotes around echo ... should be removed:
Process p = Runtime.getRuntime().exec(new String[]{
"bash", "-c",
"echo Hello World > ./output"
});
The python version needs extra quotes to tell the underlying system that echo Hello World > ./output is a single argument. The java version explicitly specifies arguments as separate strings, so it doesn't need those quotes.
Also, your version doesn't "run without complaint", you just don't see the complaints, because you don't read the error stream of the created process.
The standard input, output and error streams to/from a system process started from Java are accessed through the methods getOutputStream(), getInputStream() and getErrorStream() of Process.
I recommend you to get the error output produced by your system process:
Process p = Runtime.getRuntime().exec(...);
InputStream input=p.getErrorStream();
do
{
n=input.read(...);
}
while (n>=0);
Be careful: For your actual problem, this would be enough. But for a process which produces a longer error/output, you need to perform the reading of the standard error/output in a separate thread. If not, the system process would block when the error/output buffer is full, and wait till it is externally consumed, and if you place the reading loop just after the process is executed, it will never execute and so, the program will get into a deadlock.
Related
I'm running a jar file from another jar like here somebody answers but waiting for the process.
Process proc = Runtime.getRuntime().exec("java -jar A.jar" + stringParams);
try {
proc.waitFor();
} catch (InterruptedException e) {
e.printStackTrace();
}
InputStream in = proc.getInputStream();
InputStream err = proc.getErrorStream();
My problem comes when i have no feedback on the status of the program that is called, but i don't want my program continues beyond those lines. I would need the standard and error outputs but the results are shown when the execution is over. Is there any way of executing and getting those streams while the jar is still running?
Buffered/unbuffered
It seems like an issue with buffered output.
Executed process (in this case java -jar <path>) buffers output and writes it only when it's done (in big chunks, we don't like that!)
So one way to go is execute process through unbuffering (very hacky tools):
unbuffered <command>
stdbuf -i0 -o0 -e0 <command>
use terminal emulation
Hacking
stdbuf is part of GNU tools.
https://www.gnu.org/software/coreutils/manual/html_node/stdbuf-invocation.html
unbuffered is part of expect package.
https://wiki.tcl.tk/3548
The key thing is making the program thinking that it's in interactive mode (like you are launching it from console).
The first two options are very hacky and do not work in all cases (idk if java command works with them?)
Emulation
The third option is most promising.
We launch a program (terminal emulator) that emulates interactive terminal making program think it's working in real active session!
Pty4j
You might use pty4j too:
From there: https://github.com/traff/pty4j
// The command to run in a PTY...
String[] cmd = { "java", "-jar", "path_to_jar" };
// The initial environment to pass to the PTY child process...
String[] env = { "TERM=xterm" };
PtyProcess pty = PtyProcess.exec(cmd, env);
OutputStream os = pty.getOutputStream();
InputStream is = pty.getInputStream();
// ... work with the streams ...
// wait until the PTY child process terminates...
int result = pty.waitFor();
// free up resources.
pty.close();
Zt-exec
Maybe it's worth trying zt-exec?
I have no idea how it executes commands.
But it may be it (I didn't test that).
Using https://github.com/zeroturnaround/zt-exec
new ProcessExecutor().command("java", "-jar path_to_jar")
.redirectOutput(new LogOutputStream() {
#Override
protected void processLine(String line) {
...
}
})
.execute();
That should work, but I didn't test that.
In general, there are no ways to nicely resolve your problem.
Depending on what platforms you want to target consider using unbuffered, stdbuff or the (slowest) terminal emulation...
Please let me know if that helps and good luck! :)
This has got to be one of the strangest things I have ever observed. Consider the following Java program:
import java.io.IOException;
public class StrangeError {
public static void main(String[] args) {
try {
Process process = new ProcessBuilder(
"cmd",
"/c",
"\"C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\vcvarsall.bat\" amd64 && set"
).start();
process.waitFor();
} catch (IOException|InterruptedException e) {
System.out.println(e.getMessage());
}
}
}
I compiled it with javac StrangeError.java, copied it to my server running Windows Server 2012 R2, and ran it with java StrangeError.
Here's where things start to get weird. The program hangs, waiting for the process it spawned to finish. This is not the expected behavior, since the vcvarsall.bat script should complete immediately as well as set.
So I started playing around and discovered the following:
Removing set causes vcvarsall.bat to terminate
Removing vcvarsall.bat causes set to terminate
Replacing && with || causes everything to terminate correctly
Copying vcvarsall.bat to a location on the desktop and changing the path causes everything to terminate correctly
A nearly equivalent program works fine in Go using the same commands
I get this output if I run everything in WinDbg and interrupt the process after it hangs
This does not appear to be reproducible with vcvarsall.bat from MSVC2013 but is also reproducible with MSVC2015 on Windows 10
What on earth is wrong with the original program? If I copy and paste the entire command (cmd /c "C:\...) into Start->Run, it immediately launches cmd and terminates, as expected.
Is this a bug with Java? Is this a bug with Windows?
Is this a bug with Java? Is this a bug with Windows?
It's a bug in your code. :-)
By default, a child process created using a ProcessBuilder object has output redirected to a pipe, the parent end of which can be obtained using Process.getInputStream() and which is not automatically drained if your code does not make use of it.
Since your code simply calls .waitFor without making any provision to drain the pipe, it will deadlock as soon as the pipe's buffer overflows. I believe the default buffer size is 4,096 bytes. On my machine, the output of the command you're running is 5,192 bytes, but this will vary depending on the original contents of the environment block. (From the sounds of it, the output length in your environment is borderline, only just above the limit, so that even small changes like changing the version of VS make a difference.)
One of the many possible solutions, depending on what you're actually trying to do, is to tell Java not to pipe the child's output:
import java.io.IOException;
public class StrangeError {
public static void main(String[] args) {
try {
ProcessBuilder processb = new ProcessBuilder(
"cmd",
"/c",
"\"C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\vcvarsall.bat\" amd64 && set"
);
processb.redirectOutput(ProcessBuilder.Redirect.INHERIT);
Process process = processb.start();
process.waitFor();
} catch (IOException|InterruptedException e) {
System.out.println(e.getMessage());
}
}
}
Not possible to read standard input and output error inside the same ProcessBuilder.
So you need to create two ProcessBuilder
Process process1 = new ProcessBuilder(
"cmd",
"/c",
"\"C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\vcvarsall.bat\",
"amd64");
Process process2 = new ProcessBuilder(
"cmd",
"/c",
"set");
process1.start();
if (process1.waitFor() == 0) {
process2.start();
if (process2.waitFor() == 0) {
// Successfull execution
}
}
And one thing : I don't think it is a good practice to do shell/batch launches with Java (or another language). Maybe you should use a script (shell, batch, python, perl...) to control standard input/output streams.
In "C", I can run a long blocking process in the background (AND HAVE IT CONTINUE TO RUN) after the starting process has exited.
void main(void)
{
system("some_long_blocking_process &");
exit();
}
// "some_long_blocking_process" is still running here (DESIRED BEHAVIOR)
Java's getRuntime().exec() DOESN'T have this behavior. Instead, "some_long_blocking_process" ends immediately when the Java process ends.
Anyone know how I can recapture this behavior in Java?
I am using Java 1.4 (No process builder)
I specifically am looking to start the long blocking process and to exit immediately (no "waitFor(), etc.)
Things I have already tried (the process runs correctly, but I still get the same undesired behavior)
adding "nohup" and run in foreground ("nohup some_long_process")
adding "nohup" and running in background ("nohup some_long_process &")
run in foreground ("some_long_process")
run in background ("some_long_process &")
THANKS!
Thanks to all the suggestions... I've decided to use jtahlborn's answer (it worked for me)
try this:
String[] cmd = {"/bin/sh", "-c", "yourcommand args"};
Process p = Runtime.getRuntime().exec(cmd);
when redirect stream to /dev/null:
String[] cmd = {"/bin/sh", "-c", "yourcommand args > /dev/null 2>&1 &"};
Process p = Runtime.getRuntime().exec(cmd);
the only way we were able to achieve this with java was to add another layer of script. you need a simple wrapper script which invokes the app you actually want to run, e.g.:
runner.sh:
#!/bin/sh
nohup "$#" > /dev/null 2>&1 &
then invoke "/bin/sh runner.sh the real command" from your java program.
EDIT:
Have your tried this?
Runtime.getRuntime().exec("/bin/sh -c /usr/X11/bin/xterm &")
This worked for me on MacOS.
Previous answer (JDK 1.5, apologies for not reading the question correctly):
To execute a process without waiting you can use the ProcessBuilder
ProcessBuilder pb = new ProcessBuilder("/usr/X11/bin/xterm");
pb.start();
Your problem is probably due to the trailing &. Try removing it.
getRuntime().exec() is more similar to fork() and exec() than system().
system() passes the command to the shell, and it's Bash that understands that the trailing ampersand means to run the process in the background.
getRuntime().exec() parses the command using a StringTokenizer to parse the command, and doesn't do anything with the trailing ampersand. That's simply passed as the first argument to your some_long_blocking_process, which may exit out immediately on the unknown error.
Have you tried spawning a new Thread to run the executable? Try:
new Thread(new Runnable() {
#Override
public void run() {
try {
Runtime.getRuntime().exec(<your exec>);
} catch (IOException e) {
e.printStackTrace();
}
}
}).run();
This way, the main process won't shutdown until the exec has finished running (and the thread has finished).
Also, don't manually call exit() in your application unless you have some overwhelming reason to--the JVM does a good job of detecting when the application has finished on its own. This way, you won't force close threads that are running in the background.
I am working on a program written in Java which, for some actions, launches external programs using user-configured command lines. Currently it uses Runtime.exec() and does not retain the Process reference (the launched programs are either a text editor or archive utility, so no need for the system in/out/err streams).
There is a minor problem with this though, in that when the Java program exits, it doesn't really quit until all the launched programs are exited.
I would greatly prefer it if the launched programs were completely independent of the JVM which launched them.
The target operating system is multiple, with Windows, Linux and Mac being the minimum, but any GUI system with a JVM is really what is desired (hence the user configurability of the actual command lines).
Does anyone know how to make the launched program execute completely independently of the JVM?
Edit in response to a comment
The launch code is as follows. The code may launch an editor positioned at a specific line and column, or it may launch an archive viewer. Quoted values in the configured command line are treated as ECMA-262 encoded, and are decoded and the quotes stripped to form the desired exec parameter.
The launch occurs on the EDT.
static Throwable launch(String cmd, File fil, int lin, int col) throws Throwable {
String frs[][]={
{ "$FILE$" ,fil.getAbsolutePath().replace('\\','/') },
{ "$LINE$" ,(lin>0 ? Integer.toString(lin) : "") },
{ "$COLUMN$",(col>0 ? Integer.toString(col) : "") },
};
String[] arr; // array of parsed tokens (exec(cmd) does not handle quoted values)
cmd=TextUtil.replace(cmd,frs,true,"$$","$");
arr=(String[])ArrayUtil.removeNulls(TextUtil.stringComponents(cmd,' ',-1,true,true,true));
for(int xa=0; xa<arr.length; xa++) {
if(TextUtil.isQuoted(arr[xa],true)) {
arr[xa]=TextDecode.ecma262(TextUtil.stripQuotes(arr[xa]));
}
}
log.println("Launching: "+cmd);
Runtime.getRuntime().exec(arr);
return null;
}
This appears to be happening only when the program is launched from my IDE. I am closing this question since the problem exists only in my development environment; it is not a problem in production. From the test program in one of the answers, and further testing I have conducted I am satisfied that it is not a problem that will be seen by any user of the program on any platform.
There is a parent child relation between your processes and you have to break that.
For Windows you can try:
Runtime.getRuntime().exec("cmd /c start editor.exe");
For Linux the process seem to run detached anyway, no nohup necessary.
I tried it with gvim, midori and acroread.
import java.io.IOException;
public class Exec {
public static void main(String[] args) {
try {
Runtime.getRuntime().exec("/usr/bin/acroread");
} catch (IOException e) {
e.printStackTrace();
}
System.out.println("Finished");
}
}
I think it is not possible to to it with Runtime.exec in a platform independent way.
for POSIX-Compatible system:
Runtime.getRuntime().exec(new String[]{"/bin/sh", "-c", "your command"}).waitFor();
I have some observations that may help other people facing similar issue.
When you use Runtime.getRuntime().exec() and then you ignore the java.lang.Process handle you get back (like in the code from original poster), there is a chance that the launched process may hang.
I have faced this issue in Windows environment and traced the problem to the stdout and stderr streams. If the launched application is writing to these streams, and the buffer for these stream fills up then the launched application may appear to hang when it tries to write to the streams. The solutions are:
Capture the Process handle and empty out the streams continually - but if you want to terminate the java application right after launching the process then this is not a feasible solution
Execute the process call as cmd /c <<process>> (this is only for Windows environment).
Suffix the process command and redirect the stdout and stderr streams to nul using 'command > nul 2>&1'
It may help if you post a test section of minimal code needed to reproduce the problem. I tested the following code on Windows and a Linux system.
public class Main {
/**
* #param args the command line arguments
*/
public static void main(String[] args) throws Exception {
Runtime.getRuntime().exec(args[0]);
}
}
And tested with the following on Linux:
java -jar JustForTesting.jar /home/monceaux/Desktop/__TMP/test.sh
where test.sh looks like:
#!/bin/bash
ping -i 20 localhost
as well as this on Linux:
java -jar JustForTesting.jar gedit
And tested this on Windows:
java -jar JustForTesting.jar notepad.exe
All of these launched their intended programs, but the Java application had no problems exiting. I have the following versions of Sun's JVM as reported by java -version :
Windows: 1.6.0_13-b03
Linux: 1.6.0_10-b33
I have not had a chance to test on my Mac yet. Perhaps there is some interaction occuring with other code in your project that may not be clear. You may want to try this test app and see what the results are.
You want to launch the program in the background, and separate it from the parent. I'd consider nohup(1).
I suspect this would require a actual process fork. Basically, the C equivalent of what you want is:
pid_t id = fork();
if(id == 0)
system(command_line);
The problem is you can't do a fork() in pure Java. What I would do is:
Thread t = new Thread(new Runnable()
{
public void run()
{
try
{
Runtime.getRuntime().exec(command);
}
catch(IOException e)
{
// Handle error.
e.printStackTrace();
}
}
});
t.start();
That way the JVM still won't exit, but no GUI and only a limited memory footprint will remain.
I tried everything mentioned here but without success. Main parent Java process can't quit until the quit of subthread even with cmd /c start and redirecting streams tu nul.
Only one reliable solution for me is this:
try {
Runtime.getRuntime().exec("psexec -i cmd /c start cmd.cmd");
}
catch (Exception e) {
// handle it
}
I know that this is not clear, but this small utility from SysInternals is very helpful and proven. Here is the link.
One way I can think of is to use Runtime.addShutdownHook to register a thread that kills off all the processes (you'd need to retain the process objects somewhere of course).
The shutdown hook is only called when the JVM exits so it should work fine.
A little bit of a hack but effective.
I want to exceute a simple command which works from the shell but doesn't work from Java.
This is the command I want to execute, which works fine:
soffice -headless "-accept=socket,host=localhost,port=8100;urp;"
This is the code I am excecuting from Java trying to run this command:
String[] commands = new String[] {"soffice","-headless","\"-accept=socket,host=localhost,port=8100;urp;\""};
Process process = Runtime.getRuntime().exec(commands)
int code = process.waitFor();
if(code == 0)
System.out.println("Commands executed successfully");
When I run this program I get "Commands executed successfully".
However the process is not running when the program finishes.
Is it possible that the JVM kills the program after it has run?
Why doesn't this work?
I'm not sure if I'm not mistaken, but as far as I see you're generating the commands but never passing them to the "execute" method... you're executing "".
Try using Runtime.getRuntime().exec(commands) =)
I would like to say how I solved this.
I created a sh script that basically run the command of soffice for me.
Then from Java I just run the script, and it works fine, like this:
public void startSOfficeService() throws InterruptedException, IOException {
//First we need to check if the soffice process is running
String commands = "pgrep soffice";
Process process = Runtime.getRuntime().exec(commands);
//Need to wait for this command to execute
int code = process.waitFor();
//If we get anything back from readLine, then we know the process is running
BufferedReader in = new BufferedReader(new InputStreamReader(process.getInputStream()));
if (in.readLine() == null) {
//Nothing back, then we should execute the process
process = Runtime.getRuntime().exec("/etc/init.d/soffice.sh");
code = process.waitFor();
log.debug("soffice script started");
} else {
log.debug("soffice script is already running");
}
in.close();
}
I also kill the soffice process by calling this method:
public void killSOfficeProcess() throws IOException {
if (System.getProperty("os.name").matches(("(?i).*Linux.*"))) {
Runtime.getRuntime().exec("pkill soffice");
}
}
Note that this only works in Linux.
I believe you aren't handling quoting correctly. The original sh command line includes double quotes to prevent the shell interpreting the semicolons. The shell strips them off before the soffice process sees them.
In your Java code the shell will never see the arguments, so the extra double quotes (escaped with backslashes) are not needed - and they are probably confusing soffice.
Here's the code with the extra quotes stripped out (and a semicolon thrown in)
String[] commands = new String[] {"soffice","-headless","-accept=socket,host=localhost,port=8100;urp;"};
Process process = Runtime.getRuntime().exec(commands);
int code = process.waitFor();
if(code == 0)
System.out.println("Commands executed successfully");
(Disclaimer: I don't know Java, and I haven't tested this!)
"/Applications/OpenOffice.org\ 2.4.app/Contents/MacOS/soffice.bin -headless -nofirststartwizard -accept='socket,host=localhost,port=8100;urp;StartOffice.Service'"
or simply escaping the quotes will work as well. We feed a command like this to an ant script that ultimately ends up in an exec call like you have above. I would also recommend restarting the process every 500 or so conversions because OOO does not properly free memory (depending on what version you are running).