I have a BAT file, which creates a number of csv files by reading DB tables. bcp.exe is used for this purpose, thus, for each CSV created from a table, there's a separate bcp.exe call. All these are found in the BAT file, which I invoke using Runtime.exec().
Now the issue I face is random. It can't be recreated in developer environment, but occurs once in a while in the production environment.
Sometimes after the BAT file is executed, only few of the CSV files have been created, and the rest is missing. But when you re-execute the same, you get all the CSVs.
Here's the code:
String command = "cmd /C " + batFilePath + " " + batParams;
LOGGER.info("Executing : " + command);
Runtime rt = Runtime.getRuntime();
Process process = rt.exec(command);
process.getInputStream();
is = process.getInputStream();
isr = new InputStreamReader(is);
br = new BufferedReader(isr);
String line;
while ((line = br.readLine()) != null) {
LOGGER.info(line);
}
Would really appreciate it if anyone can enlighten me on how this might happen, since I am all at sea regarding this.
Thanks in advance,
-Raj.
Just a couple of points.
The first is that I've never understood why Java insists on getting the process's output stream with getInputStream - that's just bizarre. But that's just me ranting, there's not much you can do about that :-)
Secondly, I'm not sure why you have a "naked" process.getInputStream(); in your code. I don't think it's bad but it seems unnecessary.
Thirdly (and, to be honest, this is the only one I think may help), you need to debug the batch file itself rather than your Java code.
This can be done with the following two suggestions.
First, get the error stream and look at it. It's quite possible that cmd is delivering error information which you're just ignoring.
Secondly, change the batch file to output copious amounts of debug statements, one after each line if necessary. This will hopefully pinpoint the problem down to a specific place in the batch file.
If it only happens in production (and intermittently), that's harder, but we generally find that our customers are more than willing to accept debug-style temporary patches so we can collect the information to fix the problems they're seeing.
Output from a batch file which is simply logged is also a low-risk change. Some debug code is not so low-risk and we have to test that very thoroughly before involving the customer production systems. Some will refuse point blank, a not-unwise position to take.
It might be that you are exiting your input stream code before the batch script has completed executing.
After:
Process process = rt.exec(command);
you should probably add:
process.waitFor();
If this is the case, then you could verify it in your developer environment by deliberately slowing down your batch script and checking whether you experience the problem. Try sticking something like this:
PING 1.1.1.1 -n 1 -w 5000 > NUL
into your batch file. It will pause your script for 5 seconds.
Related
I use the following code, for redirecting the output of a process I launch from my Java app:
ProcessBuilder builder = new ProcessBuilder("MyProcess.exe");
builder.redirectOutput(Redirect.INHERIT);
builder.redirectErrorStream(true);
Now, this works fine when I run the code from eclipse - I can see the output in Eclipse's console.
Yet when I create a jar file and run it from a cmd window, e.g. java -jar MyJar.jar, it doesn't print the output of the process. What could be the reason for this?
I know I'm late in answering, but I came across this question before coming across the answer, and wanted to save anybody else in the same boat some searching.
This is actually a known bug for Windows: https://bugs.openjdk.java.net/browse/JDK-8023130
You can get around it by redirecting the streams yourself:
Process p = pb.start();
BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = null;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
p.waitFor();
br.close();
It may be, that process is printing an error and exiting for some reason. So, the actual output goes into Err stream and not into the Out stream. Your code redirects Out stream only, so important process error information may be lost. I would suggest to inherit both Out and Err streams using this code:
ProcessBuilder builder = new ProcessBuilder("MyProcess.exe");
builder.inheritIO();
One more reason to redirect both streams is related to the output buffering for child process. If parent process (your java application) is not reading or redirecting standard streams (Out and Err) of the child process, then the latter may be blocked after a while, unable to make any further progress.
It definitely wouldn't hurt to have possible errors in the output anyway.
I'm launching wkhtmltopdf from within my Java app (part of a Tomcat server, running in debug mode within Eclipse Helios on Win7 64-bit): I'd like to wait for it to complete, then Do More Stuff.
String cmd[] = {"wkhtmltopdf", htmlPathIn, pdfPathOut};
Process proc = Runtime.getRuntime().exec( cmd, null );
proc.waitFor();
But waitFor() never returns. I can still see the process in the Windows Task Manager (with the command line I passed to exec(): looks fine). AND IT WORKS. wkhtmltopdf produces the PDF I'd expect, right where I'd expect it. I can open it, rename it, whatever, even while the process is still running (before I manually terminate it).
From the command line, everything is fine:
c:\wrk>wkhtmltopdf C:\Temp\foo.html c:\wrk\foo.pdf
Loading pages (1/6)
Counting pages (2/6)
Resolving links (4/6)
Loading headers and footers (5/6)
Printing pages (6/6)
Done
The process exits just fine, and life goes on.
So what is it about runtime.exec() that's causing wkhtmltopdf to never terminate?
I could grab proc.getInputStream() and look for "Done", but that's... vile. I want something that is more general.
I've calling exec() with and without a working directory. I've tried with and without an empty "env" array. No joy.
Why is my process hanging, and what can I do to fix it?
PS: I've tried this with a couple other command line apps, and they both exhibit the same behavior.
Further exec woes.
I'm trying to read standard out & error, without success. From the command line, I know there's supposed to be something remarkably like my command line experience, but when I read the input stream returned by proc.getInputStream(), I immediately get an EOL (-1, I'm using inputStream.read()).
I checked the JavaDoc for Process, and found this
The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the [b]subprocess to block, and even deadlock[/b].
Emphasis added. So I tried that. The first 'read()' on the Standard Out inputStream blocked until I killed the process...
WITH WKHTMLTOPDF
With the generic command line ap & no params so it should "dump usage and terminate", it sucks out the appropriate std::out, then terminates.
Interesting!
JVM version issue? I'm using 1.6.0_23. The latest is... v24. I just checked the change log and don't see anything promising, but I'll try updating anyway.
Okay. Don't let the Input Streams fill or they'll block. Check. .close() can also prevent this, but isn't terribly bright.
That works in general (including the generic command line apps I've tested).
In specific however, it falls down. It appears that wkhtmltopdf is using some terminal manipulation/cursor stuff to do an ASCII-graphic progress bar. I believe this is causing the inputStream to immediately return EOF rather than giving me the correct values.
Any ideas? Hardly a deal-breaker, but it would definitely be Nice To Have.
I had the same exact issue as you and I solved it. Here are my findings:
For some reason, the output from wkhtmltopdf goes to STDERR of the process and NOT STDOUT. I have verified this by calling wkhtmltopdf from Java as well as perl
So, for example in java, you would have to do:
//ProcessBuilder is the recommended way of creating processes since Java 1.5
//Runtime.getRuntime().exec() is deprecated. Do not use.
ProcessBuilder pb = new ProcessBuilder("wkhtmltopdf.exe", htmlFilePath, pdfFilePath);
Process process = pb.start();
BufferedReader errStreamReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
//not "process.getInputStream()"
String line = errStreamReader.readLine();
while(line != null)
{
System.out.println(line); //or whatever else
line = reader.readLine();
}
On a side note, if you spawn a process from java, you MUST read from the stdout and stderr streams (even if you do nothing with it) because otherwise the stream buffer will fill and the process will hang and never return.
To futureproof your code, just in case the devs of wkhtmltopdf decide to write to stdout, you can redirect stderr of the child process to stdout and read only one stream like this:
ProcessBuilder pb = new ProcessBuilder("wkhtmltopdf.exe", htmlFilePath, pdfFilePath);
pb.redirectErrorStream(true);
Process process = pb.start();
BufferedReader inStreamReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
Actually, I do this in all the cases where I have to spawn an external process from java. That way I don't have to read two streams.
You should also read the streams of the spawned process in different threads if you dont want your main thread to block, since reading from streams is blocking.
Hope this helps.
UPDATE: I raised this issue in the project page and was replied that this is by design because wkhtmltopdf supports giving the actual pdf output in STDOUT. Please see the link for more details and java code.
A process has 3 streams: input, output and error. you can read both output and error stream at the same time using separate processes. see this question and its accepted answer and also this one for example.
You should read from the streams in a different thread.
final Semaphore semaphore = new Semaphore(numOfThreads);
final String whktmlExe = tmpwhktmlExePath;
int doccount = 0;
try{
File fileObject = new File(inputDir);
for(final File f : fileObject.listFiles()) {
if(f.getAbsolutePath().endsWith(".html")) {
doccount ++;
if(doccount >500 ) {
LOG.info(" done with conversion of 1000 docs exiting ");
break;
}
System.out.println(" inside for before "+semaphore.availablePermits());
semaphore.acquire();
System.out.println(" inside for after "+semaphore.availablePermits() + " ---" +f.getName());
new java.lang.Thread() {
public void run() {
try {
String F_ = f.getName().replaceAll(".html", ".pdf") ;
ProcessBuilder pb = new ProcessBuilder(whktmlExe , f.getAbsolutePath(), outPutDir + F_ .replaceAll(" ", "_") );//"wkhtmltopdf.exe", htmlFilePath, pdfFilePath);
pb.redirectErrorStream(true);
Process process = pb.start();
BufferedReader errStreamReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line = errStreamReader.readLine();
while(line != null)
{
System.err.println(line); //or whatever else
line = errStreamReader.readLine();
}
System.out.println("after completion for ");
} catch (Exception e) {
e.printStackTrace();
}finally {
System.out.println(" in finally releasing ");
semaphore.release();
}
}
}.start();
}
}
}catch (Exception ex) {
LOG.error(" *** Error in pdf generation *** ", ex);
}
while (semaphore.availablePermits() < numOfThreads) {//till all threads finish
LOG.info( " Waiting for all threads to exit "+ semaphore.availablePermits() + " --- " +( numOfThreads - semaphore.availablePermits()));
java.lang.Thread.sleep(10000);
}
I'm trying to launch an external program from my java swing app using this:
Process proc = Runtime.getRuntime().exec(cmd);
But the external program never actually gets launched until I close out of my java app...everytime.
It waits to launch only after I have closed out.
the external program I am trying to run is an exe that takes arguments so:
cmd = "externalProgram.exe -v --fullscreen --nowing";
What could possibly be wrong here.
Funny enough it works as expected if i try something simple like:
Process proc = Runtime.getRuntime().exec("notepad.exe");
You may need to read from the process's standard output, or close the standard input, before it will proceed. For reading the output, the problem is that the buffer can get full, blocking the program; for closing the input, the problem is that some programs will try to read data from there if it's available, waiting to do so. One or both of these tricks is very likely to straighten things out for you.
You may also read the error output stream to check it the program is actually being unsuccessfully executed
String cmd = "svn.exe";
Process proc = Runtime.getRuntime().exec(cmd);
BufferedReader reader = new BufferedReader(new InputStreamReader(proc.getErrorStream()));
String line = null;
while((line=reader.readLine())!=null){
System.out.println(line);
}
reader.close();
My console shows
Type 'svn help' for usage.
Which evidently shows the program was executed by Java.
I am using Java on Windows XP and want to be able to send commands to another program such as telnet.
I do not want to simply execute another program. I want to execute it, and then send it a sequence of commands once it's running.
Here's my code of what I want to do, but it does not work:
(If you uncomment and change the command to "cmd" it works as expected. Please help.)
This is a simplified example. In production there will be many more commands sent, so please don't suggest calling "telnet localhost".
try
{
Runtime rt = Runtime.getRuntime();
String command = "telnet";
//command = "cmd";
Process pr = rt.exec(command);
BufferedReader processOutput = new BufferedReader(new InputStreamReader(pr.getInputStream()));
BufferedWriter processInput = new BufferedWriter(new OutputStreamWriter(pr.getOutputStream()));
String commandToSend = "open localhost\n";
//commandToSend = "dir\n" + "exit\n";
processInput.write(commandToSend);
processInput.flush();
int lineCounter = 0;
while(true)
{
String line = processOutput.readLine();
if(line == null) break;
System.out.println(++lineCounter + ": " + line);
}
processInput.close();
processOutput.close();
pr.waitFor();
}
catch(Exception x)
{
x.printStackTrace();
}
That looks OK, as it won't be producing that much output, but you should really read and write in separate threads so it doesn't fill up the buffer and block waiting you to read before you reach the next step.
So if it's reaching the point where you flush the command you send to it, find out whether the Windows telnet client supports receiving commands from standard input rather than a console by piping the text you're sending to its standard input to it in a command prompt.
For example, echo dir c:\ | cmd causes cmd to run, list the c: drive contents and exit, much the same behaviour as if you typed dir c:\ into the console. But echo open localhost | telnet causes telnet to clear the screen then exit, rather than behaving the same way as if you typed it into the console. As telnet needs to mask user input for passwords, it's quite likely that it's using the console API rather than reading from standard input. It's help doesn't list any command arguments to tell it to read from standard input, so maybe you need to use a telnet implementation which is better suited to scripting.
It's not directly an answer to your question, but...
Instead of using Runtime.exec() you should use a ProcessBuilder and redirect stderr to stdout (ProcessBuilder.redirectErrorStream(true)). Otherwise your process could block if it writes something to stderr (Windows doesn't like it when the output of a process isn't read).
If you want to control a telnet session programatically from Java, you might be able to use this Java telnet library... you can do the same things (open connections, send username/password, send commands and receive results) but without actually spawning a separate process.
You may take a look at the Telnet Ant task you can call it directly in your code with out having to use a build.xml file.
You can also take a look at the source code and see how they do it.
I am developing an application in Spring Web MVC where i need to execute some of the linux script..
I am using tomcat version 5.5 for running my project in linux..
My code is looking like this :
Process proc = runtime.exec("sudo cp /var/tmp/mailserverfiles/editinterface.txt /etc/sysconfig/network-scripts/editinterface.txt");
InputStream inputstream = proc.getInputStream();
InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
String line;
while ((line = bufferedreader.readLine()) != null) {
System.out.println("\nOUTPUT = " + line);
}
System.out.print("\nbefore execute6");
try {
if (proc.waitFor() != 0) {
System.err.println("\nexit value = " + proc.exitValue());
}
} catch (InterruptedException e) {
System.err.println("\nERROR = " + e);
}
Here i want to cp a particular file from one location to another using linux script..
But when i am executing this part, i am getting
exit value = 1
as a output.. I have also tried to put this script into .sh file and try to execute that shell script here from Java Code, but i am getting same result..
Can anybody tell me, what should be the reason for this ?
Thanks in advance..
I would guess that sudo is expecting an interactive terminal in order to ask for a password. Since there is no interactive terminal, it prints an error message to stderr and exits with an exit code of 1. You are not reading the error stream, so you won't see any message that it might print.
You will definitely want to read the error stream in any case. Doing so now will help you diagnose what is going wrong at this point.
I assume the user that Tomcat is running under has unrestricted access to sudo? And that it's not being prompted for a password?
It is possible that your search path is weird and that "cp" and "sudo" are not found when you try to execute the command.
Here are some things you could try to track down your problem(s):
Try running the "cp" command without "sudo".
Try giving the full pathname of the command(s). This will avoid search path problems.
By default "sudo" logs failed commands using syslog(3). See if you can find traces in the corresponding logfiles.
Assuming you can run your command from a command line, logged in as the tomcat user - try
ProcessBuilder pb = new ProcessBuilder("/usr/bin/sudo", "cp",
"/var/tmp/mailserverfiles/editinterface.txt",
"/etc/sysconfig/network-scripts/editinterface.txt");
pb.redirectErrorStream(true);
Process proc = pb.start();
... rest of code as before
if things still fail, start debugging. strace should be helpful. e.g. run this shell script
from your java application, and figure out where things fail in the /tmp/trace.txt file:
#!/bin/sh
strace -f sudo cp /var/tmp/mailserverfiles/editinterface.txt /etc/sysconfig/network-scripts/editinterface.txt >/tmp/trace.txt 2>&1
Whilst not directly answering your question, the following will help. You need to read stdout and stderr (to capture all process output), and do this concurrently to prevent blocking of the spawned process. See this answer for more info.