Question : I want to execute java jar file from webservice in spring boot project. and want to monitor it.
Problem : I am able to execute this process but problem is that process did not processed further.
I want to know why this process is waiting and why its not processed. how can i monitor its progress.
its get processed on following conditions:
Once i stop the spring boot project or tomcat.
Its get processed if i remove process.waitFor();
I tried the solution from this, that is execute the process from another thread.
My web service call
#RequestMapping(value="/startAnalysis", method=RequestMethod.POST)
public String startAnalysis() {
List<String> cmd = new ArrayList<>();
cmd.add("java");
cmd.add("-jar");
cmd.add("test.jar");
try {
//Process p = Runtime.getRuntime().exec(cmd.toArray(new String[0]));
//ProcMon procMon = new ProcMon(cmd);
//Thread t = new Thread(procMon);
//t.setName("procMon");
//t.start();
ProcessBuilder processBuilder = new ProcessBuilder(cmd);
Process process = processBuilder.start();
process.waitFor();
} catch (Exception e) {
e.printStackTrace();
}
return "success";
}
Often the process waits for it's output to be consumed by the calling process. Sometimes this simply means the output gets displayed on the terminal but in this case you might need to read the inputstream until it blocks. If you don't know how the process is supposed to behave you might just want to read the process.getInputStream() in a separate thread. It's also possible your process is waiting for something to be written to the process.getOutputStream() in some states.
Either you need to check the documentation of the jar, or try it by executing it straight from a command prompt/shell and see how it behaves. Then you can change your application to read the output as you expect it to behave.
In lots of applications the output is most easily read line-by-line:
final String EXPECTED_OUTPUT = "Hello World";
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String firstLine = reader.readLine();
if (!firstLine.equals(EXPECTED_OUTPUT)) {
// handle unexpected situation
}
// maybe handle some more output
// or send something to the process.getOutputStream() in response
// and finally wait for the application to exit when it should be done
process.waitFor();
Please find an elaborate reasoning of why a call to process.waitFor() would not return at all in some cases, here.
Alternatively, see if its possible for you to use the alternate version of waitFor() method where we provide the timeOut value.
Related
I've been trying to use Java's ProcessBuilder to launch an application in Linux that should run "long-term". The way this program runs is to launch a command (in this case, I am launching a media playback application), allow it to run, and check to ensure that it hasn't crashed. For instance, check to see if the PID is still active, and then relaunch the process, if it has died.
The problem I'm getting right now is that the PID remains alive in the system, but the GUI for the application hangs. I tried shifting the ProcessBuilder(cmd).start() into a separate thread, but that doesn't seem to be solving anything, as I hoped it would have.
Basically the result is that, to the user, the program APPEARS to have crashed, but killing the Java process that drives the ProcessBuilder.start() Process actually allows the created Process to resume its normal behavior. This means that something in the Java application is interfering with the spawned Process, but I have absolutely no idea what, at this point. (Hence why I tried separating it into another thread, which didn't seem to resolve anything)
If anyone has any input/thoughts, please let me know, as I can't for the life of me think of how to solve this problem.
Edit: I have no concern over the I/O stream created from the Process, and have thus taken no steps to deal with that--could this cause a hang in the Process itself?
If the process writes to stderr or stdout, and you're not reading it - it will just "hang" , blocking when writing to stdout/err. Either redirect stdout/err to /dev/null using a shell or merge stdout/err with redirectErrorStream(true) and spawn another thread that reads from stdout of the process
You want the trick?
Don't start your process from ProcessBuilder.start(). Don't try to mess with stream redirection/consumption from Java (especially if you give no s**t about it ; )
Use ProcessBuilder.start() to start a little shell script that gobbles all the input/output streams.
Something like that:
#!/bin/bash
nohup $1 >/dev/null 2>error.log &
That is: if you don't care about stdout and still want to log stderr (do you?) to a file (error.log here).
If you don't even care about stderr, just redirect it to stdout:
#!/bin/bash
nohup $1 >/dev/null 2>1 &
And you call that tiny script from Java, giving it as an argument the name of the process you want to run.
If a process running on Linux that is redirecting both stdout and stderr to /dev/null still produce anything then you've got a broken, non-compliant, Linux install ;)
In other word: the above Just Works [TM] and get rid of the problematic "you need to consume the streams in this and that order bla bla bla Java-specific non-sense".
The thread running the process may block if it does not handle the output. This can be done by spawning a new thread that reads the output of the process.
final ProcessBuilder builder = new ProcessBuilder("script")
.redirectErrorStream(true)
.directory(workDirectory);
final Process process = builder.start();
final StringWriter writer = new StringWriter();
new Thread(new Runnable() {
public void run() {
IOUtils.copy(process.getInputStream(), writer);
}
}).start();
final int exitValue = process.waitFor();
final String processOutput = writer.toString();
Just stumbled on this after I had a similar issue. Agreeing with nos, you need to handle the output. I had something like this:
ProcessBuilder myProc2 = new ProcessBuilder(command);
final Process process = myProc2.start();
and it was working great. The spawned process even did output some output but not much. When I started to output a lot more, it appeared my process wasn't even getting launched anymore. I updated to this:
ProcessBuilder myProc2 = new ProcessBuilder(command);
myProc2.redirectErrorStream(true);
final Process process = myProc2.start();
InputStream myIS = process.getInputStream();
String tempOut = convertStreamToStr(myIS);
and it started working again. (Refer to this link for convertStreamToStr() code)
Edit: I have no concern over the I/O stream created from the Process, and have thus taken no steps to deal with that--could this cause a hang in the Process itself?
If you don't read the output streams created by the process then it is possible that the application will block once the application's buffers are full. I've never seen this happen on Linux (although I'm not saying that it doesn't) but I have seen this exact problem on Windows. I think this is likely related.
JDK7 will have builtin support for subprocess I/O redirection:
http://download.oracle.com/javase/7/docs/api/java/lang/ProcessBuilder.html
In the meantime, if you really want to discard stdout/stderr, it seems best (on Linux) to invoke ProcessBuilder on a command that looks like:
["/bin/bash", "-c", "exec YOUR_COMMAND_HERE >/dev/null 2>&1"]
Another solution is to start the process with Redirect.PIPE and close the InputStream like this:
ProcessBuilder builder = new ProcessBuilder(cmd);
builder.redirectOutput(Redirect.PIPE);
builder.redirectErrorStream(true); // redirect the SysErr to SysOut
Process proc = builder.start();
proc.getInputStream().close(); // this will close the pipe and the output will "flow"
proc.waitFor(); //wait
I tested this in Windows and Linux, and works!
In case you need to capture stdout and stderr and monitor the process then using Apache Commons Exec helped me a lot.
I believe the problem is the buffering pipe from Linux itself.
Try to use stdbuf with your executable
new ProcessBuilder().command("/usr/bin/stdbuf","-o0","*executable*","*arguments*");**
The -o0 says not to buffer the output.
The same goes to -i0 and -e0 if you want to unbuffer the input and error pipe.
you need to read the output before waiting to finish the cycle. You will not be notified If the output doesn't fill the buffer. If it does, it will wait until you read the output.
Suppose you have some errors or responses regarding your command which you are not reading. This would cause the application to stop and waitFor to wait forever. A simple way around is to re-direct the errors to the regular output.
I was spent 2 days on this issue.
public static void exeCuteCommand(String command) {
try {
boolean isWindows = System.getProperty("os.name").toLowerCase().startsWith("windows");
ProcessBuilder builder = new ProcessBuilder();
if (isWindows) {
builder.command("cmd.exe", "/c", command);
} else {
builder.command("sh", "-c", command);
}
Process process = builder.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line;
while ((line = reader.readLine()) != null)
System.out.println("Cmd Response: " + line);
process.waitFor();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Recently I am trying to write an java application to call SCM.exe to execute the code loading job. However, after I successfully execute the SCM load command via java, I found that I actually cannot really download the code (as using the command line, the password need to be entered after execute the SCM load command). May I know how can I enter this password just after I use the process to run the SCM in java? How can I get the output of the command line and enter something into the command line?
Thanks a million,
Eric
Since I don't know what exactly SCM.exe in your case is, I'm answering only what deals with the input/output redirection requirements in an abstract sense. I assume further that you are calling SCM.exe with whatever parameters it needs through System("...") and this is where you are unable pass any further input (stdin of the called process).
You need, instead, to be able to, upon receiving the request for password, pass it to the stdin of the other process, which is solved by using pipes in the classical sense (since you are presumably on Windows, YMMV). More generally, you are dealing with a very simple case of IPC.
In Java you may find an adequate solution by using ProcessBuilder [1] (never did myself, though -- i'd use things a lot simpler than java for this purpose, but I digress...).
An outline of a solution would be:
call the process, having its input and output being handled as output/input streams from your caller java process.
read the output of your process until you are queried the password
write the password
carry on as needed.
If you need further clarification you may need to give more details about your scenario.
[1] http://docs.oracle.com/javase/7/docs/api/java/lang/ProcessBuilder.html
public class test {
public static void main(String[] args){
try {
System.out.println("");
String commands = "C:/swdtools/IBM/RAD8/scmtools/eclipse/scm.exe load -d C:/users/43793207/test -i test -r eric-repo";
// load -d C:/users/43793207/test -i test -r eric-repo
test test=new test();
test.execCommand(commands);
} catch (Exception e) {
e.printStackTrace();
}
}
public void execCommand(String commands){
//ProcessBuilder pb = new ProcessBuilder(Command);
//pb.start();
String line;
try {
//Process pp = Runtime.getRuntime().exec(commands);
System.out.println(commands);
Process process = new ProcessBuilder(commands).start();
InputStream is = process.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
while ((line = br.readLine()) != null) {
System.out.println(line);
}
} catch (IOException e) {
e.printStackTrace();
}
}
i want to open an external app using java.
Process p = Runtime.getRuntime().exec("/Users/kausar/myApp");
this runs the process as i can see in activity monitor.
Now the file i run is actually console app which then takes commands and gives response based on those commands.
for example if i go to terminal and put the same
Kausars-MacBook-Air:~ kausar$ /Users/kausar/myApp
myApp>
Now i can give commands to app as for example
myApp> SHOW 'Hi There'
These are commands taken as keyboard input in the console app, these are not parameters. I have seen different approaches with parameters. I tried the following as well but couldnt get it to work.
String res;
String cmnd = "SHOW \'Hi There\'";
OutputStream stdin = null;
InputStream stdout = null;
stdout = p.getInputStream();
stdin = p.getOutputStream();
stdin.write(cmnd.getBytes());
stdin.flush();
p.waitFor();
BufferedReader input = new BufferedReader(
new InputStreamReader(stdout));
while ((res = input.readLine()) != null) {
System.out.println(res)
}
input.close();
p.destroy();
Its displaying nothing while the same procedure with "/bin/bash -c ls" works just fine.
please help!
Of hand I would say the problem is with p.*wait*For()
Exactly what object and when to usee notify() or notifyAll() call to wake up the object thread would be something like on stdout and maybe a restructure of the process.
note: an interesting feature is the class field in BufferedReader called "lock", the api docs do mention some way of structuring your program so it can be notified.
I've got to write down some java code that runs an external process.
This process gets an input line from stdin and gives an output line on stdout.
The code I wrote is as follows
try
{
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec("./" + args[0]);
Process proc2 = rt.exec(c);
int exitVal = proc2.exitValue();
System.out.println("Process exitValue: " + exitVal);
}
catch (Throwable t){
}
Where args[0] is the external process and c is the input (String) I need to feed to the process. If it was running correctly, I would get an exitVal=0, instead I get nothing. Also, what I really want to print is the output of the external process (which would be "6" in the given example).
You must use getInputStream() and getOutputStream() on the process to communicate it, see the javadocs.
If you do not read all data from the process, it might hang and even deadlock. For a short introduction on common problems, see this blog post. At the very least, you should be reading all data from the process output.
You cannot "get nothing". Method exitValue() returns primitive type, therefore it even cannot be null. But this method can stuck however. This happens because your process is still running.
Is is possible that it is running because you did not "feed" it? In this case the only thing you have to do is to call proc2.getOutputStream() and write what you need to this stream. If you want to read the process' output use proc2.getInputStream() and read from the stream.
BTW, take a look on ProcessBuilder. It provides better, more "object oriented" API thant simple Runtime.exec().
You can feed the program input with '<' redirection:
{
Process p = Runtime.getRuntime().exec("wc < pom.xml");
p.waitFor();
System.out.println(p.exitValue());
BufferedReader bri = new BufferedReader
(new InputStreamReader(p.getInputStream()));
String line;
while ((line = bri.readLine()) != null) {
System.out.println(line);
}
bri.close();
}
I'm launching wkhtmltopdf from within my Java app (part of a Tomcat server, running in debug mode within Eclipse Helios on Win7 64-bit): I'd like to wait for it to complete, then Do More Stuff.
String cmd[] = {"wkhtmltopdf", htmlPathIn, pdfPathOut};
Process proc = Runtime.getRuntime().exec( cmd, null );
proc.waitFor();
But waitFor() never returns. I can still see the process in the Windows Task Manager (with the command line I passed to exec(): looks fine). AND IT WORKS. wkhtmltopdf produces the PDF I'd expect, right where I'd expect it. I can open it, rename it, whatever, even while the process is still running (before I manually terminate it).
From the command line, everything is fine:
c:\wrk>wkhtmltopdf C:\Temp\foo.html c:\wrk\foo.pdf
Loading pages (1/6)
Counting pages (2/6)
Resolving links (4/6)
Loading headers and footers (5/6)
Printing pages (6/6)
Done
The process exits just fine, and life goes on.
So what is it about runtime.exec() that's causing wkhtmltopdf to never terminate?
I could grab proc.getInputStream() and look for "Done", but that's... vile. I want something that is more general.
I've calling exec() with and without a working directory. I've tried with and without an empty "env" array. No joy.
Why is my process hanging, and what can I do to fix it?
PS: I've tried this with a couple other command line apps, and they both exhibit the same behavior.
Further exec woes.
I'm trying to read standard out & error, without success. From the command line, I know there's supposed to be something remarkably like my command line experience, but when I read the input stream returned by proc.getInputStream(), I immediately get an EOL (-1, I'm using inputStream.read()).
I checked the JavaDoc for Process, and found this
The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the [b]subprocess to block, and even deadlock[/b].
Emphasis added. So I tried that. The first 'read()' on the Standard Out inputStream blocked until I killed the process...
WITH WKHTMLTOPDF
With the generic command line ap & no params so it should "dump usage and terminate", it sucks out the appropriate std::out, then terminates.
Interesting!
JVM version issue? I'm using 1.6.0_23. The latest is... v24. I just checked the change log and don't see anything promising, but I'll try updating anyway.
Okay. Don't let the Input Streams fill or they'll block. Check. .close() can also prevent this, but isn't terribly bright.
That works in general (including the generic command line apps I've tested).
In specific however, it falls down. It appears that wkhtmltopdf is using some terminal manipulation/cursor stuff to do an ASCII-graphic progress bar. I believe this is causing the inputStream to immediately return EOF rather than giving me the correct values.
Any ideas? Hardly a deal-breaker, but it would definitely be Nice To Have.
I had the same exact issue as you and I solved it. Here are my findings:
For some reason, the output from wkhtmltopdf goes to STDERR of the process and NOT STDOUT. I have verified this by calling wkhtmltopdf from Java as well as perl
So, for example in java, you would have to do:
//ProcessBuilder is the recommended way of creating processes since Java 1.5
//Runtime.getRuntime().exec() is deprecated. Do not use.
ProcessBuilder pb = new ProcessBuilder("wkhtmltopdf.exe", htmlFilePath, pdfFilePath);
Process process = pb.start();
BufferedReader errStreamReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
//not "process.getInputStream()"
String line = errStreamReader.readLine();
while(line != null)
{
System.out.println(line); //or whatever else
line = reader.readLine();
}
On a side note, if you spawn a process from java, you MUST read from the stdout and stderr streams (even if you do nothing with it) because otherwise the stream buffer will fill and the process will hang and never return.
To futureproof your code, just in case the devs of wkhtmltopdf decide to write to stdout, you can redirect stderr of the child process to stdout and read only one stream like this:
ProcessBuilder pb = new ProcessBuilder("wkhtmltopdf.exe", htmlFilePath, pdfFilePath);
pb.redirectErrorStream(true);
Process process = pb.start();
BufferedReader inStreamReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
Actually, I do this in all the cases where I have to spawn an external process from java. That way I don't have to read two streams.
You should also read the streams of the spawned process in different threads if you dont want your main thread to block, since reading from streams is blocking.
Hope this helps.
UPDATE: I raised this issue in the project page and was replied that this is by design because wkhtmltopdf supports giving the actual pdf output in STDOUT. Please see the link for more details and java code.
A process has 3 streams: input, output and error. you can read both output and error stream at the same time using separate processes. see this question and its accepted answer and also this one for example.
You should read from the streams in a different thread.
final Semaphore semaphore = new Semaphore(numOfThreads);
final String whktmlExe = tmpwhktmlExePath;
int doccount = 0;
try{
File fileObject = new File(inputDir);
for(final File f : fileObject.listFiles()) {
if(f.getAbsolutePath().endsWith(".html")) {
doccount ++;
if(doccount >500 ) {
LOG.info(" done with conversion of 1000 docs exiting ");
break;
}
System.out.println(" inside for before "+semaphore.availablePermits());
semaphore.acquire();
System.out.println(" inside for after "+semaphore.availablePermits() + " ---" +f.getName());
new java.lang.Thread() {
public void run() {
try {
String F_ = f.getName().replaceAll(".html", ".pdf") ;
ProcessBuilder pb = new ProcessBuilder(whktmlExe , f.getAbsolutePath(), outPutDir + F_ .replaceAll(" ", "_") );//"wkhtmltopdf.exe", htmlFilePath, pdfFilePath);
pb.redirectErrorStream(true);
Process process = pb.start();
BufferedReader errStreamReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line = errStreamReader.readLine();
while(line != null)
{
System.err.println(line); //or whatever else
line = errStreamReader.readLine();
}
System.out.println("after completion for ");
} catch (Exception e) {
e.printStackTrace();
}finally {
System.out.println(" in finally releasing ");
semaphore.release();
}
}
}.start();
}
}
}catch (Exception ex) {
LOG.error(" *** Error in pdf generation *** ", ex);
}
while (semaphore.availablePermits() < numOfThreads) {//till all threads finish
LOG.info( " Waiting for all threads to exit "+ semaphore.availablePermits() + " --- " +( numOfThreads - semaphore.availablePermits()));
java.lang.Thread.sleep(10000);
}