Develop Reference

Tracking the progress of wget (bash) with SwingWorker / Swing

I am trying to make a gui, whereby users are able to download files. Currently I am able to call the wget command through a process, but I am struggling to use it along with swingworker.
How would I go about tracking the progress of the downloading and updating a gui simultaneously?
Currently I have tried using this method:
ShellProcess.command("wget --progress=dot "+_url);
Where command is the method that creates the process:
InputStream stdout = _process.getInputStream();
BufferedReader stdoutBuffered =new BufferedReader(new InputStreamReader(stdout));
String line = null;
String output ="";
try {
while ((line = _stdoutBuffered.readLine()) != null ) {
// System.out.println(line);
output+=(line+" ");
System.out.println(line +" SHELL");
_progress++;
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
_progress = 0;
return output;
}
I am trying to count the amount of lines outputted as "wget --progress=dot" should output a line for every percent of progress. But this does not seem to work.
My doInBackground method inside the swingworker looks like this:
#Override
protected Integer doInBackground() throws Exception {
// Start
download .command("wget "+_url);
ShellProcess.command("wget --progress=dot "+_url);
int progress = 0;
while (progress<101){
progress = ShellProcess.getProgress() %100 ;
publish(ShellProcess.getOutput());
setProgress(progress);
}
return 1;
}
Any help would be appreciated.

In this complete example, the background method of a SwingWorker starts a ProcessBuilder. Standard output and error streams are combined for display in a text component. Substitute your wget command to see the effect. Experiment with --progress=bar and reading a character at a time.
ProcessBuilder pb = new ProcessBuilder("wget", "--progress=dot", url);

You don't really need SwingWorker for this. Just download in a separate Thread (don't do it in the EDT), and every time you encounter a new dot line output from wget, update a GUI component (a progress bar for example), but do this update in the EDT e.g. with SwingUtilities.invokeLater():
JProgressBar progressBar = ...; // Initialize and add progress bar to your GUI
...
// In your separate download thread:
final AtomicInteger percent = new AtomicInteger();
while ((line = _stdoutBuffered.readLine()) != null ) {
if (".".equals(line)) {
// A new percent was completed, update the progressbar:
percent.incrementAndGet();
SwingUtilities.invokeLater(new Runnable() {
#Override
public void run() {
progressBar.setValue(percent.get());
}
});
}
}

Reading InputStream from a batch file's process skips to next line

I am trying to run a batch file with Runtime.exec() and then output its InputStream into a JTextArea. What I have works, but only partially. What happens is the batch file runs, but if it executes a command other than something like "echo" that command immediately terminates and the next line executes. For example, let's say I try to run a simple batch file like this:
#echo off
echo hello. waiting 5 seconds.
timeout /t 5 /nobreak > NUL
echo finished. goodbye.
The batch file executes, and the JTextArea says
hello. waiting 5 seconds.
finished. goodbye.
but it doesn't wait for 5 seconds in the middle.
I can't figure out why it's doing this. Here's what I use to run the batch file and read its InputStream.
private class ScriptRunner implements Runnable {
private final GUI.InfoGUI gui; // the name of my GUI class
private final String script;
public ScriptRunner(final GUI.InfoGUI gui, final File script) {
this.gui = gui;
this.script = script.getAbsolutePath();
}
#Override
public void run() {
try {
final Process p = Runtime.getRuntime().exec(script);
StreamReader output = new StreamReader(p.getInputStream(), gui);
Thread t = new Thread(output);
t.start();
int exit = p.waitFor();
output.setComplete(true);
while (t.isAlive()) {
sleep(500);
}
System.out.println("Processed finished with exit code " + exit);
} catch (final Exception e) {
e.printStackTrace();
}
}
}
private class StreamReader implements Runnable {
private final InputStream is;
private final GUI.InfoGUI gui;
private boolean complete = false;
public StreamReader(InputStream is, GUI.InfoGUI gui) {
this.is = is;
this.gui = gui;
}
#Override
public void run() {
BufferedReader in = new BufferedReader(new InputStreamReader(is));
try {
while (!complete || in.ready()) {
while (in.ready()) {
gui.setTextAreaText(in.readLine() + "\n");
}
sleep(250);
}
} catch (final Exception e) {
e.printStackTrace();
} finally {
try {
in.close();
} catch (final Exception e) {
e.printStackTrace();
}
}
public void setComplete(final boolean complete) {
this.complete = complete;
}
}
public void sleep(final long ms) {
try {
Thread.sleep(ms);
} catch (final InterruptedException ie) {
}
}
I know my code is pretty messy, and I'm sure it contains grammatical errors.
Thanks for anything you can do to help!

You're creating a Process but you're not reading from its standard error stream. The process might be writing messages to its standard error to tell you that there's a problem, but if you're not reading its standard error, you won't be able to read these messages.
You have two options here:
Since you already have a class that reads from a stream (StreamReader), wire up another one of these to the process's standard error stream (p.getErrorStream()) and run it in another Thread. You'll also need to call setComplete on the error StreamReader when the call to p.waitFor() returns, and wait for the Thread running it to die.
Replace your use of Runtime.getRuntime().exec() with a ProcessBuilder. This class is new in Java 5 and provides an alternative way to run external processes. In my opinion its most significant improvement over Runtime.getRuntime().exec() is the ability to redirect the process's standard error into its standard output, so you only have one stream to read from.
I would strongly recommend going for the second option and choosing to redirect the process's standard error into its standard output.
I took your code and replaced the line
final Process p = Runtime.getRuntime().exec(script);
with
final ProcessBuilder pb = new ProcessBuilder(script);
pb.redirectErrorStream(true);
final Process p = pb.start();
Also, I don't have your GUI code to hand, so I wrote the output of the process to System.out instead.
When I ran your code, I got the following output:
hello. waiting 5 seconds.
ERROR: Input redirection is not supported, exiting the process immediately.
finished. goodbye.
Processed finished with exit code 0
Had you seen that error message, you might have twigged that something was up with the timeout command.
Incidentally, I noticed in one of your comments that none of the commands suggested by ughzan worked. I replaced the timeout line with ping -n 5 127.0.0.1 > NUL and the script ran as expected. I couldn't reproduce a problem with this.

The problem is definitely in timeout.exe. If you add echo %errorlevel% after line with timeout, you will see that it returns 1 if running from java. And 0 if running in usual way. Probably, it requires some specific console functionality (i.e. cursor positioning) that is suppressed when running from java process.
Is there anything I can do to get this to work while running from Java
If you don't need ability to run any batch file then consider to replace timeout with ping. Otherwise... I've tried to run batch file with JNA trough Kernel32.CreateProcess and timeout runs fine. But then you need to implement reading of process output trough native calls also.
I hope someone will suggest better way.

The ready method only tells if the stream can guarantee that something can be read immediately, without blocking. You can't really trust it because always returning false is a valid implementation. Streams with buffers may return true only when they have something buffered. So I suspect your problem is here:
while (!complete || in.ready()) {
while (in.ready()) {
gui.setTextAreaText(in.readLine() + "\n");
}
sleep(250);
}
It should rather read something like this:
String line;
while (!complete || (line=in.readLine()) != null) {
gui.setTextAreaText(line + "\n");
}

It's probably because your "timeout ..." command returned with an error.
Three ways to test it:
Check if the "timeout ..." command works in the Windows command prompt.
Replace "timeout ..." in the script with "ping -n 5 127.0.0.1 > NUL" (it essentially does the same thing)
Remove everything but "timeout /t 5 /nobreak > NUL" from your script. The process should return with an error (1) if the timeout failed because it is the last command executed.

Unable to perform any action before Process.Runtime.exec statement line

my very first post here after long time of searching but yet get an answer regarding this issue, please help me in this issue.
I am using Netbean 6.9.1 to build a Java application which massive call to few different external program, therefore I used process and runtime function to call for external program.
The whole application process is separated into few stages and I wish to inform the user till which stage the application is currently running by updating the GUI textarea, the code is showed as below:
public void executeCommand (String cmd, File path)
{
try
{
****areaOutput.setText("Executing audio decoding, please wait till process is done\n");****
btnTranscribe.setEnabled(false);
areaOutput.setEditable(false);
areaOutput.setEnabled(false);
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(cmd , null, path);
InputStream stderr = proc.getErrorStream();
InputStreamReader isr = new InputStreamReader(stderr);
BufferedReader br = new BufferedReader(isr);
String line = null;
System.out.println("<ERROR>");
while ( (line = br.readLine()) != null)
System.out.println(line);
System.out.println("</ERROR>");
int exitVal = proc.waitFor();
System.out.println("Process exitValue: " + exitVal);
areaOutput.append("\n\nConversion is done, processing with features extraction....");
} catch (Throwable t)
{
t.printStackTrace();
}
}
As showed in the code above, I wish to set the Textarea and disable some button before executing the command, but when the application ran, all of these line seems unable to work and nothing is changed at the application itself till the command is finish executed, any solution to run the pre-command code execute first before the .exec() start to run?
I appreciate your great help and advice regarding this issue.
Best regards,
Striky
P/S:
hi there, I have make a Thread class for this CmdExec in order to execute cmd in different thread:
public class CmdExec extends Thread
{
private String cmd;
private File path;
public CmdExec() {
}
public CmdExec(String cmd, File path) {
this.cmd = cmd;
this.path = path;
}
public void run(){
try
{
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(cmd , null, path);
InputStream stderr = proc.getErrorStream();
InputStreamReader isr = new InputStreamReader(stderr);
BufferedReader br = new BufferedReader(isr);
String line = null;
System.out.println("<ERROR>");
while ( (line = br.readLine()) != null)
System.out.println(line);
System.out.println("</ERROR>");
int exitVal = proc.waitFor();
System.out.println("Process exitValue: " + exitVal);
} catch (Throwable t)
{
t.printStackTrace();
}
}
}
and in order to call to this class,
CmdExec tryDemo = new CmdExec();
tryDemo = new CmdExec(strSegment, fSegment);
tryDemo.run();
is used to start the thread, but I failed to put SwingUtilities.invokeLater in any part of these process, it simply won't run the tryDemo.run() because it is void...
Also, may I know so far am I doing right?? Very thank you for your kind help regarding this issue
P/S 2: I have just added another runnable code (so threads for process executing, runnable to GUI update) for GUI update command as below:
Runnable doWorkRunnable = new Runnable() {
public void run() {
System.out.println("hello world");
btnTranscribe.setEnabled(false);
areaOutput.setEditable(false);
areaOutput.setEnabled(false);
areaOutput.setText("Performing segmentation, please wait till process is done\n"); }
};
and I used SwingUtilies.invokeLater before the execution of process as below:
SwingUtilities.invokeLater(doWorkRunnable);
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(cmd , null, path);
But all these failed, am I get the wrong sequence for the GUI and process thread coordination?

you are executing this work on the EDT (the thread which updates the gui). so, the gui cannot update until all this work finishes. what you want to do is run a separate thread which does all the work and periodically calls SwingUtilities.invokeLater with a status update.

Try to put sleep before execute method. To verify what is happening.

Runtime exec and a custom built RTF editor

I have a class that manages the creation of RTF documents and a method in that class that calls the RTF editor with a XML file for display.
All but one user can access this editor without any issues. This one user consistently runs into an issue where their application just hangs. There are no errors in any logs. Normally this kind of problem is easily identified, reproduced and corrected, however, I can't for the life of my reproduce it so my attempts at debugging are failing.
Basically the code is as follows:
int exitVal = CUBSRTFEditor.runRTFEditor("c:\\tmp\\control"+ap_doc_id+".xml", xml,"I:\\AppealsLetters.exe /process \"c:\\tmp\\control"+ap_doc_id+".xml\"");
public static int runRTFEditor(String xmlLocation, String xmlContent, String executePath)
{
int exitVal = 0;
createLocalFile(xmlLocation, xmlContent);
try
{
System.out.println("executePath must = "+executePath);
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(executePath);
System.out.println("after executePath runs");
//exhaust that stream before waiting for the process to exit
InputStream inputstream = proc.getInputStream();
InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
// read the ls output
String line;
while ((line = bufferedreader.readLine())!= null)
{
System.out.println(line);
}
exitVal = proc.waitFor();
}
catch (Throwable t)
{
t.printStackTrace();
}
CUBSRTFEditor.deleteTempFile(xmlLocation);
return exitVal;
}
The last output is the first System.out. When I take the xml file and execute this on any other PC it executes without issue. I see no useful info in proc.getErrorStream() or proc.getOutputStream().
The JDK's Javadoc documentation on this problem (exec hanging):
Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.
I try exhausting that stream before waiting for the process to exit and that does not seem to help as it never seems to get to that point (the second System.out is not displayed)
Have I implemented this incorrectly? Am I missing something important? Any ideas on how to get more info out of the process would be great.
I am stuck....

Runtime.exec() is a deceptively nasty little spud to work with. I found this article (old, but still relevant) to be quite helpful. You can always skip to Page 4 for some highly gankable sample code. :-)
At a glance, your code needs to handle both proc.getOutputStream() and proc.getErrorStream(), which is a good reason to handle those streams in separate threads.

I wanted to update this because the change went into Production today and worked. Based off of BlairHippo's suggestions I got it to work with an anonymous inner class to create a separate thread to exhaust both the Error and Input streams.
new Thread(new Runnable(){
public void run()
{
try
{
BufferedReader br = new BufferedReader(new InputStreamReader(proc.getErrorStream()));
String line;
while ((line = br.readLine())!= null)
{
System.out.println(line);
}
}
catch (Throwable t)
{
t.printStackTrace();
}
}
}).start();

Using a thread to capture process output

I am using a thread to capture stream output from a process, and then outputting that stream to the eclipse console. The question I have is when to terminate the thread that is doing the stream output.
Thread t = new Thread(new Runnable(){
private boolean isProcessDone(Process p)
{
//not sure what to do here
}
public void run()
{
Process p = Runtime.getRuntime().exec("executable with output");
BufferedReader input = new BufferedReader(new InputStreamReader(p.getInputStream()));
BufferedReader error = new BufferedReader (new InputStreamReader(p.getErrorStream()));
while ( !isProcessDone(p) ) {
String line;
if( (line = input.readLine()) != null )
{
System.out.println(line);
}
if( (line = error.readLine()) != null )
{
System.out.println(line);
}
}
input.close();
error.close();
}
});
t.start();
My question is what belongs in the isProcessDone() function. The example I am basing this off of uses the stream's ready() function, but I am unclear whether this will work for programs that either std::err and std::out, but not both. I also tried using
try{
p.exitValue();
return true;
}catch(IllegalThreadStateException e){}
return false;
but then the thread finishes before the while loop has a chance to act on the streams, and the output is lost.

You need to use Process.waitFor() to wait for process completion.
Additionally, you need to consume stdout and stderr concurrently in order to avoid blocking and a possible process hang. Consequently you need two threads to read these streams, and to continue reading whilst the streams are available.
See this Javaworld article for more info and a StreamGobbler implementation to consume the stdout/err.

You can use VerboseProcess from jcabi-log (I'm a developer):
String name = new VerboseProcess(
new ProcessBuilder("executable with output")
).stdout();
The only dependency you need:
<dependency>
<groupId>com.jcabi</groupId>
<artifactId>jcabi-log</artifactId>
<version>0.7.5</version>
</dependency>

You will need to have two threads. One to handle the I/O and another to wait for process completion (Process.waitFor()) and set a flag telling the I/O thread to quit when it exhausts the data.

You need to process reading the output in a separate thread, theres an example here