There's a command, that runs an exe, it makes an output and when this command ends, then I should run the upload(); method.
run.command(exe_path, txt_path);
run.start();
TimeUnit.SECONDS.sleep(20);
upload();
How can I replace the static sleep method?
Using a ProcessBuilder, you should be able to call Process.waitFor() to wait for the command to finish.
Process p = new ProcessBuilder("foo.exe", "arg1").start();
// handle any input and output
InputStream stdout = p.getInputStream();
OutputStream stdin = p.getOutputStream();
InputStream stderr = p.getErrorStream();
int ret = p.waitFor(); // <- wait for exit
Related
I have a java Spring REST API with a controller that runs a linux command with the ProcessBuilder class
. The command is a generated 'find' command
The problem is that I found a lot of unterminated processes in the hosting server after fiew days of use. I don't know why they still there and not ended or destroyed . (I checked with a ps -ef command)
Here is my runCmd function:
public static final BufferedReader runCmd(String cmd) throws IOException, InterruptedException {
ProcessBuilder processBuilder = new ProcessBuilder();
processBuilder.command("bash", "-c", cmd);
processBuilder.redirectErrorStream(true);
Process process = processBuilder.start();
BufferedReader output = new BufferedReader(new InputStreamReader(process.getInputStream()));
int ret = process.waitFor();
return output;
}
Is there a way to make sur that there is no more process left behind ?
UPDATE
The problem comes only from commands with a very large output stream (std output) Thanks for the hint #DuncG
As this output is important, I can't ignore it. I have to find a way to consume it.
Any Idea on how to do it with Runnable Threads ?
Thanks
Are your commands generating a lot of output? The cause of the zombies may be simply that cmd has written a lot of output the STDOUT and the stream is blocking in the BufferedReader.
You can test if this the case by adding redirect to null - just append " > /dev/null" the end of cmd. This discards the sub-process output and means the BufferedReader is not full of unread data / blocking the sub-process.
processBuilder.command("bash", "-c", cmd + " > /dev/null");
If that fixes the zombie problem you can revert the redirect and make ProcessBuilder redirect output to files (before calling start()), or you'll need to add a thread to consume the IO as it is generated.
Path tmpdir = Path.of(System.getProperty("java.io.tmpdir"));
Path out = tmpdir.resolve("stdout.log");
processBuilder.redirectErrorStream(true);
processBuilder.redirectOutput(out.toFile());
At the end you should return the out file for caller to check, or could return Files.newBufferedReader(out).
If you don't use the redirect to file as above, this will store using thread to capture the output into memory buffer. Note you'd need to duplicate for STDERR too if not redirecting ERR->OUT:
Process p = pb.start();
ByteArrayOutputStream stdout = new ByteArrayOutputStream(8192);
new Thread(() -> copy(p.getInputStream(), stdout), "STDOUT").start();
int rc = p.waitFor();
byte[] sour = stdout.toByteArray()
Using method:
private static void copy(InputStream in, OutputStream buf)
{
try(var autoClose = in; var autoClose2 = buf)
{
in.transferTo(buf);
}
catch(IOException io)
{
throw new UncheckedIOException(io);
}
}
Process process = Runtime.getRuntime().exec("tasklist");
BufferedReader reader =
new BufferedReader(new InputStreamReader(process.getInputStream()));
process.waitFor();
There are many reasons that waitFor() doesn't return.
But it usually boils down to the fact that the executed command doesn't quit.
This, again, can have many reasons.
One common reason is that the process produces some output and you don't read from the appropriate streams. This means that the process is blocked as soon as the buffer is full and waits for your process to continue reading. Your process in turn waits for the other process to finish (which it won't because it waits for your process, ...). This is a classical deadlock situation.
You need to continually read from the processes input stream to ensure that it doesn't block.
There's a nice article that explains all the pitfalls of Runtime.exec() and shows ways around them called "When Runtime.exec() won't" (yes, the article is from 2000, but the content still applies!)
It appears you are not reading the output before waiting for it to finish. This is fine only if the output doesn't fill the buffer. If it does, it will wait until you read the output, catch-22.
Perhaps you have some errors which you are not reading. This would case the application to stop and waitFor to wait forever. A simple way around this is to re-direct the errors to the regular output.
ProcessBuilder pb = new ProcessBuilder("tasklist");
pb.redirectErrorStream(true);
Process process = pb.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line;
while ((line = reader.readLine()) != null)
System.out.println("tasklist: " + line);
process.waitFor();
Also from Java doc:
java.lang
Class Process
Because some native platforms only provide limited buffer size for standard input and
output streams, failure to promptly write the input stream or read the output stream of
the subprocess may cause the subprocess to block, and even deadlock.
Fail to clear the buffer of input stream (which pipes to the output stream of subprocess)
from Process may lead to a subprocess blocking.
Try this:
Process process = Runtime.getRuntime().exec("tasklist");
BufferedReader reader =
new BufferedReader(new InputStreamReader(process.getInputStream()));
while ((reader.readLine()) != null) {}
process.waitFor();
I would like to add something to the previous answers but since I don't have the rep to comment, I will just add an answer. This is directed towards android users which are programming in Java.
Per the post from RollingBoy, this code almost worked for me:
Process process = Runtime.getRuntime().exec("tasklist");
BufferedReader reader =
new BufferedReader(new InputStreamReader(process.getInputStream()));
while ((reader.readLine()) != null) {}
process.waitFor();
In my case, the waitFor() was not releasing because I was executing a statement with no return ("ip adddr flush eth0"). An easy way to fix this is to simply ensure you always return something in your statement. For me, that meant executing the following: "ip adddr flush eth0 && echo done". You can read the buffer all day, but if there is nothing ever returned, your thread will never release its wait.
Hope that helps someone!
There are several possibilities:
You haven't consumed all the output on the process's stdout.
You haven't consumed all the output on the process's stderr.
The process is waiting for input from you and you haven't provided it, or you haven't closed the process's stdin.
The process is spinning in a hard loop.
As others have mentioned you have to consume stderr and stdout.
Compared to the other answers, since Java 1.7 it is even more easy. You do not have to create threads yourself anymore to read stderr and stdout.
Just use the ProcessBuilder and use the methods redirectOutput in combination with either redirectError or redirectErrorStream.
String directory = "/working/dir";
File out = new File(...); // File to write stdout to
File err = new File(...); // File to write stderr to
ProcessBuilder builder = new ProcessBuilder();
builder.directory(new File(directory));
builder.command(command);
builder.redirectOutput(out); // Redirect stdout to file
if(out == err) {
builder.redirectErrorStream(true); // Combine stderr into stdout
} else {
builder.redirectError(err); // Redirect stderr to file
}
Process process = builder.start();
For the same reason you can also use inheritIO() to map Java console with external app console like:
ProcessBuilder pb = new ProcessBuilder(appPath, arguments);
pb.directory(new File(appFile.getParent()));
pb.inheritIO();
Process process = pb.start();
int success = process.waitFor();
You should try consume output and error in the same while
private void runCMD(String CMD) throws IOException, InterruptedException {
System.out.println("Standard output: " + CMD);
Process process = Runtime.getRuntime().exec(CMD);
// Get input streams
BufferedReader stdInput = new BufferedReader(new InputStreamReader(process.getInputStream()));
BufferedReader stdError = new BufferedReader(new InputStreamReader(process.getErrorStream()));
String line = "";
String newLineCharacter = System.getProperty("line.separator");
boolean isOutReady = false;
boolean isErrorReady = false;
boolean isProcessAlive = false;
boolean isErrorOut = true;
boolean isErrorError = true;
System.out.println("Read command ");
while (process.isAlive()) {
//Read the stdOut
do {
isOutReady = stdInput.ready();
//System.out.println("OUT READY " + isOutReady);
isErrorOut = true;
isErrorError = true;
if (isOutReady) {
line = stdInput.readLine();
isErrorOut = false;
System.out.println("=====================================================================================" + line + newLineCharacter);
}
isErrorReady = stdError.ready();
//System.out.println("ERROR READY " + isErrorReady);
if (isErrorReady) {
line = stdError.readLine();
isErrorError = false;
System.out.println("ERROR::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::" + line + newLineCharacter);
}
isProcessAlive = process.isAlive();
//System.out.println("Process Alive " + isProcessAlive);
if (!isProcessAlive) {
System.out.println(":::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Process DIE " + line + newLineCharacter);
line = null;
isErrorError = false;
process.waitFor(1000, TimeUnit.MILLISECONDS);
}
} while (line != null);
//Nothing else to read, lets pause for a bit before trying again
System.out.println("PROCESS WAIT FOR");
process.waitFor(100, TimeUnit.MILLISECONDS);
}
System.out.println("Command finished");
}
I think I observed a similar problem: some processes started, seemed to run successfully but never completed. The function waitFor() was waiting forever except if I killed the process in Task Manager.
However, everything worked well in cases the length of the command line was 127 characters or shorter. If long file names are inevitable you may want to use environmental variables, which may allow you keeping the command line string short. You can generate a batch file (using FileWriter) in which you set your environmental variables before calling the program you actually want to run.
The content of such a batch could look like:
set INPUTFILE="C:\Directory 0\Subdirectory 1\AnyFileName"
set OUTPUTFILE="C:\Directory 2\Subdirectory 3\AnotherFileName"
set MYPROG="C:\Directory 4\Subdirectory 5\ExecutableFileName.exe"
%MYPROG% %INPUTFILE% %OUTPUTFILE%
Last step is running this batch file using Runtime.
Here is a method that works for me.
NOTE: There is some code within this method that may not apply to you, so try and ignore it. For example "logStandardOut(...), git-bash, etc".
private String exeShellCommand(String doCommand, String inDir, boolean ignoreErrors) {
logStandardOut("> %s", doCommand);
ProcessBuilder builder = new ProcessBuilder();
StringBuilder stdOut = new StringBuilder();
StringBuilder stdErr = new StringBuilder();
boolean isWindows = System.getProperty("os.name").toLowerCase().startsWith("windows");
if (isWindows) {
String gitBashPathForWindows = "C:\\Program Files\\Git\\bin\\bash";
builder.command(gitBashPathForWindows, "-c", doCommand);
} else {
builder.command("bash", "-c", doCommand);
}
//Do we need to change dirs?
if (inDir != null) {
builder.directory(new File(inDir));
}
//Execute it
Process process = null;
BufferedReader brStdOut;
BufferedReader brStdErr;
try {
//Start the command line process
process = builder.start();
//This hangs on a large file
// https://stackoverflow.com/questions/5483830/process-waitfor-never-returns
//exitCode = process.waitFor();
//This will have both StdIn and StdErr
brStdOut = new BufferedReader(new InputStreamReader(process.getInputStream()));
brStdErr = new BufferedReader(new InputStreamReader(process.getErrorStream()));
//Get the process output
String line = null;
String newLineCharacter = System.getProperty("line.separator");
while (process.isAlive()) {
//Read the stdOut
while ((line = brStdOut.readLine()) != null) {
stdOut.append(line + newLineCharacter);
}
//Read the stdErr
while ((line = brStdErr.readLine()) != null) {
stdErr.append(line + newLineCharacter);
}
//Nothing else to read, lets pause for a bit before trying again
process.waitFor(100, TimeUnit.MILLISECONDS);
}
//Read anything left, after the process exited
while ((line = brStdOut.readLine()) != null) {
stdOut.append(line + newLineCharacter);
}
//Read anything left, after the process exited
while ((line = brStdErr.readLine()) != null) {
stdErr.append(line + newLineCharacter);
}
//cleanup
if (brStdOut != null) {
brStdOut.close();
}
if (brStdErr != null) {
brStdOut.close();
}
//Log non-zero exit values
if (!ignoreErrors && process.exitValue() != 0) {
String exMsg = String.format("%s%nprocess.exitValue=%s", stdErr, process.exitValue());
throw new ExecuteCommandException(exMsg);
}
} catch (ExecuteCommandException e) {
throw e;
} catch (Exception e) {
throw new ExecuteCommandException(stdErr.toString(), e);
} finally {
//Log the results
logStandardOut(stdOut.toString());
logStandardError(stdErr.toString());
}
return stdOut.toString();
}
Asynchronous reading of stream combined with avoiding Wait with a timeout will solve the problem.
You can find a page explaining this here http://simplebasics.net/.net/process-waitforexit-with-a-timeout-will-not-be-able-to-collect-the-output-message/
public static void main(String[] args) throws PyException, IOException, InterruptedException
these should be the exceptions thrown
I'm posting requests to a Servlet. In that Servlet, I have a code which executes terminal and running some commands there. I want my code to wait the terminal process before it response my requests. How can I manage it?
Below you can see my code:
Process send = Runtime.getRuntime().exec(new String[] {"sh", "-c", fileNameShort + ".jar /tmp/"+ fileNameShort +".class; /home/ubuntu/android-x86.rund.sh -cp /tmp/"+ fileNameShort +".jar " + fileNameShort});
You can use Process#waitFor():
System.out.println("Waiting for process...");
send.waitFor();
Process process = Runtime.getRuntime().exec("tasklist");
BufferedReader reader =
new BufferedReader(new InputStreamReader(process.getInputStream()));
BufferedReader reader =
new BufferedReader(new InputStreamReader(process.getErrorStream()));
System.out.println("Waiting for process completion..");
int exitValue = process.waitFor();
System.out.println("Process is done..");
Method is waitFor() enables you to wait for the process to complete.
ErrorStream and InoutStream enables to get the output & error of the execution respectively
exitValue indicates manner of completion of the process. exitValue 0 is for normal completion
I am a pretty new guy to the world of threading, have been trying to solve this problem for a week now.
The run method in the Thread class is not being called for some reason, I dont know why ( but would love to know)
ProcessBuilder processBuilder = new ProcessBuilder();
processBuilder.command("/bin/sh", "-c", "echo \"w30000001z,none,16488,,181075\nw30000001z,none,16488,,181082\n\" | /home/beehive/bin/exec/tableSize");
Process process = processBuilder.start();
process.waitFor();
InputStream stdin = process.getInputStream();
OutputStream stdout = process.getOutputStream();
InputStream stderr = process.getErrorStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(stdin));
BufferedReader error = new BufferedReader(new InputStreamReader(stderr));
StreamGobbler errorStream = new StreamGobbler(process.getErrorStream(), "ERROR");
StreamGobbler outputStream = new StreamGobbler(process.getInputStream(), "OUTPUT");
errorStream.start();
outputStream.start();
errorStream.join();
outputStream.join();
tableSize is a python executable which takes input through its stdin, processes it and outputs a few lines of text. I need to collect this output and do some further processing on it.
There is a separate thread to process the output on the inputStream and errorStream. This thread class is as shown below.
/* StreamGobbler.java */
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.PrintWriter;
class StreamGobbler extends Thread
{
InputStream is;
String type;
OutputStream os;
StreamGobbler(InputStream is, String type)
{
this(is, type, null);
}
StreamGobbler(InputStream is, String type, OutputStream redirect)
{
this.is = is;
this.type = type;
this.os = redirect;
}
public void run()
{
try
{
PrintWriter pw = null;
if (os != null)
pw = new PrintWriter(os);
System.out.println("Ajay" + type);
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line=null;
while ( (line = br.readLine()) != null)
{
if (pw != null)
pw.println(line);
System.out.println(type + ">" + line);
}
if (pw != null)
pw.flush();
} catch (IOException ioe)
{
ioe.printStackTrace();
}
}
}
For some reason, the run method in the StreamGobbler class is not being called. However the constructor of this class is being executed.
Any help would be much appreciated.
You need to start the stream gobblers before you call process.waitFor().
The stream gobblers read and discard the external processes output / error streams so that the external process doesn't block while trying to write. But the way you have it, the process has to have finished writing (and exited) before you attempt to gobble the output.
Result - deadlock ... if the external process writes too much to its output streams.
Actually, I think that the primary problem is that your Java code does not write anything to inputStream. So, the external process just sits there waiting for input that never arrives ... and process.waitFor() waits for the external process. Deadlock.
The correct order of things should be:
build the process and start it.
build the gobblers and start them.
open the input stream.
write stuff to input stream.
close input stream.
call process.waitFor() and check the processes return code.
look at the output captured by the gobblers
You need consume the STDOUT and/or STDERR before Process.waitFor(), here's an excellent article:
http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html?page=2
1) Try removing the waitFor() if it is not required. It only makes the executing thread wait until the process execution is completed.
I need to start external executable in such way that user can interact with program that was just started.
For example in OpenSuse Linux there is a package manager - Zypper. You can start zypper in command mode and give commands like install, update, remove, etc. to it.
I would like to run it from Java code in a way user could interact with it: input commands and see output and errors of the program he started.
Here is a Java code I tried to use:
public static void main(String[] args) throws IOException, InterruptedException {
Process proc = java.lang.Runtime.getRuntime().exec("zypper shell");
InputStream stderr = proc.getInputStream();
InputStreamReader isr = new InputStreamReader(stderr);
BufferedReader br = new BufferedReader(isr);
String line = null;
char ch;
while ( (ch = (char)br.read()) != -1)
System.out.print(ch);
int exitVal = proc.waitFor();
System.out.println("Process exitValue: " + exitVal);
}
But unfortunately I can only see it's output:
zypper>
but no matter what I write, my input doesn't affect program that was started.
How can I do what want to?
You need to get an output stream in order to write to the process:
OutputStream out = proc.getOuptutStream();
This output stream is piped into the standard input stream of the process, so you can just write to it (perhaps you want to wrap it in a PrintWriter first) and the data will be sent to the process' stdin.
Note that it might also be convenient to get the error stream (proc.getErrorStream) in order to read any error output that the process writes to its stderr.
API reference:
http://download.oracle.com/javase/6/docs/api/java/lang/Process.html
Seems like the converting inside the while condition fails in your example, this seems to work better (I don't run Suse so I haven't tried with Zypper):
public static void main(String[] args) throws IOException, InterruptedException
{
//Process proc = java.lang.Runtime.getRuntime().exec("zypper shell");
Process proc = java.lang.Runtime.getRuntime().exec("ping -t localhost");
InputStream stderr = proc.getInputStream();
InputStreamReader isr = new InputStreamReader(stderr);
BufferedReader br = new BufferedReader(isr);
int i;
while ( (i = br.read()) != -1)
{
System.out.print((char) i);
}
int exitVal = proc.waitFor();
System.out.println("Process exitValue: " + exitVal);
}
I recently wrapped Google Closure Compiler into a .jar-file which is extracted and used in a Process. This compiler only talks via System.in/out/err. There's a big "gotcha" in connecting pipes together, which is just briefly mentioned in the Process javadoc.
"...failure to promptly write the
input stream or read the output stream
of the subprocess may cause the
subprocess to block, and even
deadlock."
On Mac OS X the buffer is 16k, and if you don't read it promptly as suggested, the process deadlocks. My only solution to this problem ATM, is a rather nasty busy wait.
https://github.com/algesten/googccwrap/blob/master/src/main/java/googccwrap/GoogleClosureCompilerWrapper.java