Develop Reference

Why in order to execute an external code I have to re-run the main application? [duplicate]

I have the next code:
Process p = Runtime.getRuntime().exec(args);
and I want my program to wait for the Runtime.getRuntime().exec(args); to finish cause it last 2-3sec and then to continue.
Ideas?

use Process.waitFor():
Process p = Runtime.getRuntime().exec(args);
int status = p.waitFor();
From JavaDoc:
causes the current thread to wait, if necessary, until the process represented by this Process object has terminated. This method returns immediately if the subprocess has already terminated. If the subprocess has not yet terminated, the calling thread will be blocked until the subprocess exits.

Here is a sample code:
Process proc = Runtime.getRuntime().exec(ANonJava.exe#);
InputStream in = proc.getInputStream();
byte buff[] = new byte[1024];
int cbRead;
try {
while ((cbRead = in.read(buff)) != -1) {
// Use the output of the process...
}
} catch (IOException e) {
// Insert code to handle exceptions that occur
// when reading the process output
}
// No more output was available from the process, so...
// Ensure that the process completes
try {
proc.waitFor();
} catch (InterruptedException) {
// Handle exception that could occur when waiting
// for a spawned process to terminate
}
// Then examine the process exit code
if (proc.exitValue() == 1) {
// Use the exit value...
}
You can find more on this site: http://docs.rinet.ru/JWP/ch14.htm

Java exec method, how to handle streams correctly

What is the proper way to produce and consume the streams (IO) of external process from Java? As far as I know, java end input streams (process output) should be consumed in threads parallel to producing the process input due the possibly limited buffer size.
But I'm not sure if I eventually need to synchronize with those consumer threads, or is it enough just to wait for process to exit with waitFor method, to be certain that all the process output is actually consumed? I.E is it possible, even if the process exits (closes it's output stream), there is still unread data in the java end of the stream? How does the waitFor actually even know when the process is done? For the process in question, EOF (closing the java end of it's input stream) signals it to exit.
My current solution to handle the streams is following
public class Application {
private static final StringBuffer output = new StringBuffer();
private static final StringBuffer errOutput = new StringBuffer();
private static final CountDownLatch latch = new CountDownLatch(2);
public static void main(String[] args) throws IOException, InterruptedException {
Process exec = Runtime.getRuntime().exec("/bin/cat");
OutputStream procIn = exec.getOutputStream();
InputStream procOut = exec.getInputStream();
InputStream procErrOut = exec.getErrorStream();
new Thread(new StreamConsumer(procOut, output)).start();
new Thread(new StreamConsumer(procErrOut, errOutput)).start();
PrintWriter printWriter = new PrintWriter(procIn);
printWriter.print("hello world");
printWriter.flush();
printWriter.close();
int ret = exec.waitFor();
latch.await();
System.out.println(output.toString());
System.out.println(errOutput.toString());
}
public static class StreamConsumer implements Runnable {
private InputStream input;
private StringBuffer output;
public StreamConsumer(InputStream input, StringBuffer output) {
this.input = input;
this.output = output;
}
#Override
public void run() {
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
String line;
try {
while ((line = reader.readLine()) != null) {
output.append(line + System.lineSeparator());
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
try {
reader.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
latch.countDown();
}
}
}
}
}
Is it necessary to use the latch here, or does the waitFor implicate all the output is already consumed? Also, if the output doesn't end/contain new line, will the readLine miss the output, or still read all that is left? Does reading null mean process has closed it's end of the stream - is there any other scenario where null could be read?
What is the correct way to handle streams, could I do something better than in my example?

waitFor signals that the process ended, but you cannot be sure the threads which collect strings from its stdout and stderr finished also, so using a latch is a step in the right direction, but not an optimal one.
Instead of waiting for the latch, you can wait for the threads directly:
Thread stdoutThread = new Thread(new StreamConsumer(procOut, output)).start();
Thread stderrThread = ...
...
int ret = exec.waitFor();
stdoutThread.join();
stderrThread.join();
BTW, storing lines in StringBuffers is useless work. Use ArrayList<String> instead, put lines there without any conversion, and finally retrieve them in a loop.

Your appapproach is right, but is't better to remove CountDownLatch and use ThreadPool, and not create new Thread directly. From ThreadPool you will get two futures, which you can wait after to completion.
But I'm not sure if I eventually need to synchronize with those consumer threads, or is it enough just to wait for process to exit with waitFor method, to be certain that all the process output is actually consumed? I.E is it possible, even if the process exits (closes it's output stream), there is still unread data in the java end of the stream?
Yes, this situation may occurs. Termination and reading IO streams is unrelated processes.

How to get the output from a process and set a timeout?

How can I obtain a process' output while setting a timeout value?
I am currently using Apache Commons IO utils to create a string from the process' standard and error outputs.
The code below, as is (with the comments), works fine for processes that terminate. However, if the process doesn't terminate, the main thread doesn't terminate either!
If I uncomment out the commented code and instead comment out process.waitfor(), the method will properly destroy non terminating processes. However, for terminating processes, the output isn't properly obtained. It appears that once waitFor is completed, I cannot get the process' input and error streams?
Finally, if I attempt to move the commented section to where process.waitFor() currently is, remove process.waitFor() and uncomment the commented section, then for non terminating processes, the main thread also won't stop. This is because the process.waitFor(15, ...) will never be reached.
private static Outputs runProcess(String command) throws Exception {
Process process = Runtime.getRuntime().exec(command);
// if (!process.waitFor(15, TimeUnit.SECONDS)) {
// System.out.println("Destroy");
// process.destroy();
// }
// Run and collect the results from the standard output and error output
String stdStr = IOUtils.toString(process.getInputStream());
String errStr = IOUtils.toString(process.getErrorStream());
process.waitFor();
return new Outputs(stdStr, errStr);
}

As #EJP suggested, You can use different threads to capture the streams or use ProcessBuilder or redirect to a file from your command.
Here are 3 approaches that I feel you can use.
Using different threads for Streams.
Process process = Runtime.getRuntime().exec("cat ");
ExecutorService newFixedThreadPool = Executors.newFixedThreadPool(2);
Future<String> output = newFixedThreadPool.submit(() -> {
return IOUtils.toString(process.getInputStream());
});
Future<String> error = newFixedThreadPool.submit(() -> {
return IOUtils.toString(process.getErrorStream());
});
newFixedThreadPool.shutdown();
// process.waitFor();
if (!process.waitFor(3, TimeUnit.SECONDS)) {
System.out.println("Destroy");
process.destroy();
}
System.out.println(output.get());
System.out.println(error.get());
Using ProcessBuilder
ProcessBuilder processBuilder = new ProcessBuilder("cat")
.redirectError(new File("error"))
.redirectOutput(new File("output"));
Process process = processBuilder.start();
// process.waitFor();
if (!process.waitFor(3, TimeUnit.SECONDS)) {
System.out.println("Destroy");
process.destroy();
}
System.out.println(FileUtils.readFileToString(new File("output")));
System.out.println(FileUtils.readFileToString(new File("error")));
Use a redirection operator in your command to redirect Output & Error to a file and then Read from File.
Here is very good blog which explains different ways of handling Runtime.Exec

This is a slightly adjusted version of Kishore Bandi's first solution which uses separate thread to capture output.
It has been simplified, uses no external libraries and have more robust termination code.
Process process = new ProcessBuilder("cat", "file.txt")
.redirectErrorStream(true)
.start();
System.out.println("Output: " + waitForOuput(process, Duration.ofSeconds(10)));
/**
* Waits {#code timeout} time for the output of
* {#code process.getInputStream()}. Returns when the process is terminated.
* Throws on non-zero exit value.
*/
public static String waitForOuput(Process process, Duration timeout) throws InterruptedException, TimeoutException {
ExecutorService pool = Executors.newFixedThreadPool(1);
Future<String> outputFuture = pool.submit(() -> new String(process.getInputStream().readAllBytes()));
pool.shutdown();
try {
String output = outputFuture.get(timeout.toMillis(), TimeUnit.MILLISECONDS);
if (process.exitValue() != 0) {
throw new IllegalStateException("Command " + process.info().command()
+ " returned exit value " + process.exitValue());
}
return output;
} catch (ExecutionException | TimeoutException | InterruptedException ex) {
process.destroyForcibly();
outputFuture.cancel(true);
process.waitFor(1, TimeUnit.SECONDS); // Give process time to die
if (ex instanceof InterruptedException intEx) {
throw intEx;
} else if (ex instanceof TimeoutException timeEx) {
throw timeEx;
} else {
throw new IllegalStateException(ex);
}
}
}

Run a process asynchronously and read from stdout and stderr

I have some code that runs a process and reads from the stdout and stderr asynchronously and then handles when the process completes. It looks something like this:
Process process = builder.start();
Thread outThread = new Thread(() -> {
try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()))) {
// Read stream here
} catch (Exception e) {
}
});
Thread errThread = new Thread(() -> {
try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getErrorStream()))) {
// Read stream here
} catch (Exception e) {
}
});
outThread.start();
errThread.start();
new Thread(() -> {
int exitCode = -1;
try {
exitCode = process.waitFor();
outThread.join();
errThread.join();
} catch (Exception e) {
}
// Process completed and read all stdout and stderr here
}).start();
My issue is with the fact that I am using 3 threads to achieve this asynchronous "run-and-get-output" task - I don't know why, but I feel it doesn't feel right using 3 threads. I could allocate the threads out of a thread pool, but that would still be blocking those threads.
Is there anything I can do, maybe with NIO, to reduce this to fewer (1?) thread? Anything I can think of will be constantly spinning a thread (unless I add a few sleeps), which I don't really want to do either...
NOTE: I do need to read as I go (rather than when the process has stopped) and I do need to separate stdin from stderr so can't do a redirect.

Since you've specified that you need to read the output as you go, there is no non-multi-threaded solution.
You can reduce the number of threads to one beyond your main thread though:
Process process = builder.start();
Thread errThread = new Thread(() -> {
try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getErrorStream()))) {
// Read stream here
} catch (Exception e) {
}
});
errThread.start();
try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()))) {
// Read stream here
} catch (Exception e) {
}
// we got an end of file, so there can't be any more input. Now we need to wait for stderr/process exit.
int exitCode = -1;
try {
exitCode = process.waitFor();
errThread.join();
} catch (Exception e) {
}
// Process completed
If you truely don't need to deal with the error/output until after the process ends, you can simplify it a bit and only use your main thread like this:
File stderrFile = File.createTempFile("tmpErr", "out");
File stdoutFile = File.createTempFile("tmpStd", "out");
try {
ProcessBuilder builder = new ProcessBuilder("ls /tmp");
Process p = builder.start();
int exitCode = -1;
boolean done = false;
while (!done) {
try {
exitCode = p.waitFor();
done = true;
} catch (InterruptedException ie) {
System.out.println("Interrupted waiting for process to exit.");
}
}
BufferedReader err = new BufferedReader(new FileReader(stderrFile));
BufferedReader in = new BufferedReader(new FileReader(stdoutFile));
....
} finally {
stderrFile.delete();
stdoutFile.delete();
}
This is probably not a good idea if you generate a lot of output from the process you are calling as it could run out of disk space... but it'll likely be slightly faster since it doesn't have to spin up another Thread.

Assuming you don't mind the input and error streams to be merged, you could only use one thread with:
builder.redirectErrorStream(true); //merge input and error streams
Process process = builder.start();
Thread singleThread = new Thread(() -> {
int exitCode = -1;
//read from the merged stream
try (BufferedReader reader =
new BufferedReader(new InputStreamReader(process.getInputStream()))) {
String line;
//read until the stream is exhausted, meaning the process has terminated
while ((line = reader.readLine()) != null) {
System.out.println(line); //use the output here
}
//get the exit code if required
exitCode = process.waitFor();
} catch (Exception e) { }
}).start();

Have a look at the ExecHelper from OstermillerUtils.
The idea is that the thread waiting for the process to complete, does not just wait but reads input from stdout and stderr if there is input available and regurarly checks if the process has finished.
If you do not do any heavy processing with the input from stdout and stderr, you might not need an extra thread to handle the input. Just copy ExecHelper and add some extra functions/methods to process any new input. I've done this before to show the process output while the process is running, it is not difficult to do (but I lost the source code).
If you do need a separate thread for processing the input, make sure to synchronize the output and error StringBuffers when these buffers are updated or read.
Another thing you might want to consider is adding an abort time-out. It is a little bit harder to implement but was very valuable to me: if a process takes too much time, the process gets destroyed which in turn ensures nothing remains hanging. You can find an old (outdated?) example this gist.

You'll have to compromise. Here are your options:
A. You can do it with 2 threads (instead of 3):
First thread:
read from stdout until readline returns null
call Process.waitFor()
join Thread#2
Second thread:
reads from stderr until readline returns null
B. Merge streams and use Debian's annotate-output to discriminate the 2 streams
http://manpages.debian.org/cgi-bin/man.cgi?query=annotate-output&sektion=1
C. If it's a short-living process just wait for the end of it
D. If it's a long-living process then you can spin between readers with some sleep in between.

Check if process is running on windows/linux

i have a process
Runtime rt = Runtime.getRuntime() ;
Process p = rt.exec(filebase+port+"/hlds.exe +ip "+ip+" +maxplayers "+players+ " -game cstrike -console +port "+port+" -nojoy -noipx -heapsize 250000 +map de_dust2 +servercfgfile server.cfg +lservercfgfile +mapcyclefile mapcycle.txt +motdfile motd.txt +logsdir logs -zone 2048",null, new File(filebase+port)) ;
i want to keep a check on this process whether its running or has crashed in case of crash want to restart it, this Process can have multiple instance available depending upon the port
Can i trace this thing on Linux as well as on windows? Read some articles on it but this 1 is bit different, since it involves multiple occurrences and have to check on some particular process only

boolean isRunning(Process process) {
try {
process.exitValue();
return false;
} catch (Exception e) {
return true;
}
}
See http://docs.oracle.com/javase/6/docs/api/java/lang/Process.html#exitValue()

You can do a p.waitFor() so the thread that executed the statement waits till the process is complete. You can then do the cleanup/restart logic right after, as that code will get executed when the process dies. However I am not sure how this would work if the process hangs instead of dying, but this could be worth a try. By the way I would recommend using Java Service Wrapper and supervisord in your case if this is something you're going to do on production.

As of Java 8 you can do:
process.isAlive()

For pre Java 8 code, I'm using reflection with a fallback to catching IllegalThreadStateException. The reflection will only work on instances of ProcessImpl, but as that's what's returned by ProcessBuilder it's usually the case for me.
public static boolean isProcessIsAlive(#Nullable Process process) {
if (process == null) {
return false;
}
// XXX replace with Process.isAlive() in Java 8
try {
Field field;
field = process.getClass().getDeclaredField("handle");
field.setAccessible(true);
long handle = (long) field.get(process);
field = process.getClass().getDeclaredField("STILL_ACTIVE");
field.setAccessible(true);
int stillActive = (int) field.get(process);
Method method;
method = process.getClass().getDeclaredMethod("getExitCodeProcess", long.class);
method.setAccessible(true);
int exitCode = (int) method.invoke(process, handle);
return exitCode == stillActive;
} catch (Exception e) {
// Reflection failed, use the backup solution
}
try {
process.exitValue();
return false;
} catch (IllegalThreadStateException e) {
return true;
}
}

Java 5 and on have a way to handle this using java.util.concurrent.Future.
A Future represents the result of an asynchronous computation. Methods are provided to check if the computation is complete, to wait for its completion, and to retrieve the result of the computation. The result can only be retrieved using method get when the computation has completed, blocking if necessary until it is ready. Cancellation is performed by the cancel method. Additional methods are provided to determine if the task completed normally or was cancelled. Once a computation has completed, the computation cannot be cancelled. If you would like to use a Future for the sake of cancellability but not provide a usable result, you can declare types of the form Future and return null as a result of the underlying task.

public class ProcessEndNotifier extends Thread
{
Process process;
MyClass classThatNeedsToBeNotified;
public ProcessEndNotifier(MyClass classThatNeedsToBeNotified, Process process)
{
this.process = process;
this.classThatNeedsToBeNotified = classThatNeedsToBeNotified;
}
#Override
public void run()
{
try {
process.waitFor();
}
catch (InterruptedException e) {
classThatNeedsToBeNotified.processEnded();
}
classThatNeedsToBeNotified.processEnded();
}
}
Now you can know if a process in running like this:
public class MyClass
{
boolean isProcessRunning;
public static void main(String[]args)
{
Process process = Runtime.getRuntime().exec("foo -bar");
isProcessRunning = true;
new ProcessEndNotifier(this, process).run();
}
public void processEnded()
{
isProcessRunning = false;
// Or just do stuff here!
}
}

In java 9, you can use the isAlive() method to check if a process is stil running like this:
Runtime rt = Runtime.getRuntime() ;
Process p = rt.exec(filebase+port+"/hlds.exe +ip "+ip+" +maxplayers "+players+ " -game cstrike -console +port "+port+" -nojoy -noipx -heapsize 250000 +map de_dust2 +servercfgfile server.cfg +lservercfgfile +mapcyclefile mapcycle.txt +motdfile motd.txt +logsdir logs -zone 2048",null, new File(filebase+port)) ;
boolean isRunning = p.toHandle.isAlive();