Develop Reference

PrintWriter.print blocks when the string is too long

I'm writing a wrapper program in java that's just supposed to pass arguments to other processes by writing to their standard in streams, and reading the response from their standard out streams. However, when the String I try to pass in is too large, PrintWriter.print simply blocks. No error, just freezes. Is there a good workaround for this?
Relevant code
public class Wrapper {
PrintWriter writer;
public Wrapper(String command){
start(command);
}
public void call(String args){
writer.println(args); // Blocks here
writer.flush();
//Other code
}
public void start(String command) {
try {
ProcessBuilder pb = new ProcessBuilder(command.split(" "));
pb.redirectErrorStream(true);
process = pb.start();
// STDIN of the process.
writer = new PrintWriter(new OutputStreamWriter(process.getOutputStream(), "UTF-8"));
} catch (Exception e) {
e.printStackTrace();
System.out.println("Process ended catastrophically.");
}
}
}
If I try using
writer.print(args);
writer.print("\n");
it can handle a larger string before freezing, but still ultimately locks up.
Is there maybe a buffered stream way to fix this? Does print block on the processes stream having enough space or something?
Update
In response to some answers and comments, I've included more information.
Operating System is Windows 7
BufferedWriter slows the run time, but didn't stop it from blocking eventually.
Strings could get very long, as large as 100,000 characters
The Process input is consumed, but by line i.e Scanner.nextLine();
Test code
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeoutException;
import ProcessRunner.Wrapper;
public class test {
public static void main(String[] args){
System.out.println("Building...");
Wrapper w = new Wrapper("java echo");
System.out.println("Calling...");
String market = "aaaaaa";
for(int i = 0; i < 1000; i++){
try {
System.out.println(w.call(market, 1000));
} catch (InterruptedException | ExecutionException
| TimeoutException e) {
System.out.println("Timed out");
}
market = market + market;
System.out.println("Size = " + market.length());
}
System.out.println("Stopping...");
try {
w.stop();
} catch (IOException e) {
e.printStackTrace();
System.out.println("Stop failed :(");
}
}
}
Test Process:
You have to first compile this file, and make sure the .class is in the same folder as the test .class file
import java.util.Scanner;
public class echo {
public static void main(String[] args){
while(true){
Scanner stdIn = new Scanner(System.in);
System.out.println(stdIn.nextLine());
}
}
}

I suspect that what is happening here is that the external process is writing to its standard output. Since your Java code doesn't read it, it eventually fills the external process's standard out (or err) pipe. That blocks the external process, which means that it can read from its input pipe .... and your Java process freezes.
If this is the problem, then using a buffered writer won't fix it. You either need to read the external processes output or redirect it to a file (e.g. "/dev/null" on Linux)

Writing to any pipe or socket by any means in java.io blocks if the peer is slower reading than you are writing.
Nothing you can do about it.

Percentage not updating on stdout when running sdelete with ProcessBuilder

There is program named "sdelete" that clearly delete disk free space, from sysinternals. If you execute with sdelete.exe -c, it will clear all your free space and it shows up the percentage about progress to stdout.
So I tried to make code using this:
public class deleteTest {
public static void main(String[] arg){
ProcessBuilder p = new ProcessBuilder("sdelete.exe","-c");
try {
Process pGet = p.start();
p.redirectError(p.redirectInput());
InputStream is = pGet.getInputStream();
int ch;
while((ch = is.read()) != -1){
System.out.print((char)ch);
}
} catch (IOException e) {
e.printStackTrace();
}catch (Exception e){
e.printStackTrace();
}
}
}
but it doesn't show up the percentage, it just keeps showing Zeroing free space on C:\: 0%. The process doesn't hang or freeze, but when it finishes all the percentage show up at once!
How can I fix this?

I fear that you're going to complete miss the error stream and its messages with that set up. Consider changing
p.redirectError(p.redirectInput());
to:
p.redirectErrorStream(true);
This way, if your process building code is not correct, you'll at least see why.

Reading InputStream from a batch file's process skips to next line

I am trying to run a batch file with Runtime.exec() and then output its InputStream into a JTextArea. What I have works, but only partially. What happens is the batch file runs, but if it executes a command other than something like "echo" that command immediately terminates and the next line executes. For example, let's say I try to run a simple batch file like this:
#echo off
echo hello. waiting 5 seconds.
timeout /t 5 /nobreak > NUL
echo finished. goodbye.
The batch file executes, and the JTextArea says
hello. waiting 5 seconds.
finished. goodbye.
but it doesn't wait for 5 seconds in the middle.
I can't figure out why it's doing this. Here's what I use to run the batch file and read its InputStream.
private class ScriptRunner implements Runnable {
private final GUI.InfoGUI gui; // the name of my GUI class
private final String script;
public ScriptRunner(final GUI.InfoGUI gui, final File script) {
this.gui = gui;
this.script = script.getAbsolutePath();
}
#Override
public void run() {
try {
final Process p = Runtime.getRuntime().exec(script);
StreamReader output = new StreamReader(p.getInputStream(), gui);
Thread t = new Thread(output);
t.start();
int exit = p.waitFor();
output.setComplete(true);
while (t.isAlive()) {
sleep(500);
}
System.out.println("Processed finished with exit code " + exit);
} catch (final Exception e) {
e.printStackTrace();
}
}
}
private class StreamReader implements Runnable {
private final InputStream is;
private final GUI.InfoGUI gui;
private boolean complete = false;
public StreamReader(InputStream is, GUI.InfoGUI gui) {
this.is = is;
this.gui = gui;
}
#Override
public void run() {
BufferedReader in = new BufferedReader(new InputStreamReader(is));
try {
while (!complete || in.ready()) {
while (in.ready()) {
gui.setTextAreaText(in.readLine() + "\n");
}
sleep(250);
}
} catch (final Exception e) {
e.printStackTrace();
} finally {
try {
in.close();
} catch (final Exception e) {
e.printStackTrace();
}
}
public void setComplete(final boolean complete) {
this.complete = complete;
}
}
public void sleep(final long ms) {
try {
Thread.sleep(ms);
} catch (final InterruptedException ie) {
}
}
I know my code is pretty messy, and I'm sure it contains grammatical errors.
Thanks for anything you can do to help!

You're creating a Process but you're not reading from its standard error stream. The process might be writing messages to its standard error to tell you that there's a problem, but if you're not reading its standard error, you won't be able to read these messages.
You have two options here:
Since you already have a class that reads from a stream (StreamReader), wire up another one of these to the process's standard error stream (p.getErrorStream()) and run it in another Thread. You'll also need to call setComplete on the error StreamReader when the call to p.waitFor() returns, and wait for the Thread running it to die.
Replace your use of Runtime.getRuntime().exec() with a ProcessBuilder. This class is new in Java 5 and provides an alternative way to run external processes. In my opinion its most significant improvement over Runtime.getRuntime().exec() is the ability to redirect the process's standard error into its standard output, so you only have one stream to read from.
I would strongly recommend going for the second option and choosing to redirect the process's standard error into its standard output.
I took your code and replaced the line
final Process p = Runtime.getRuntime().exec(script);
with
final ProcessBuilder pb = new ProcessBuilder(script);
pb.redirectErrorStream(true);
final Process p = pb.start();
Also, I don't have your GUI code to hand, so I wrote the output of the process to System.out instead.
When I ran your code, I got the following output:
hello. waiting 5 seconds.
ERROR: Input redirection is not supported, exiting the process immediately.
finished. goodbye.
Processed finished with exit code 0
Had you seen that error message, you might have twigged that something was up with the timeout command.
Incidentally, I noticed in one of your comments that none of the commands suggested by ughzan worked. I replaced the timeout line with ping -n 5 127.0.0.1 > NUL and the script ran as expected. I couldn't reproduce a problem with this.

The problem is definitely in timeout.exe. If you add echo %errorlevel% after line with timeout, you will see that it returns 1 if running from java. And 0 if running in usual way. Probably, it requires some specific console functionality (i.e. cursor positioning) that is suppressed when running from java process.
Is there anything I can do to get this to work while running from Java
If you don't need ability to run any batch file then consider to replace timeout with ping. Otherwise... I've tried to run batch file with JNA trough Kernel32.CreateProcess and timeout runs fine. But then you need to implement reading of process output trough native calls also.
I hope someone will suggest better way.

The ready method only tells if the stream can guarantee that something can be read immediately, without blocking. You can't really trust it because always returning false is a valid implementation. Streams with buffers may return true only when they have something buffered. So I suspect your problem is here:
while (!complete || in.ready()) {
while (in.ready()) {
gui.setTextAreaText(in.readLine() + "\n");
}
sleep(250);
}
It should rather read something like this:
String line;
while (!complete || (line=in.readLine()) != null) {
gui.setTextAreaText(line + "\n");
}

It's probably because your "timeout ..." command returned with an error.
Three ways to test it:
Check if the "timeout ..." command works in the Windows command prompt.
Replace "timeout ..." in the script with "ping -n 5 127.0.0.1 > NUL" (it essentially does the same thing)
Remove everything but "timeout /t 5 /nobreak > NUL" from your script. The process should return with an error (1) if the timeout failed because it is the last command executed.

How to write a byte array to OutputStream of process builder (Java)

byte[] bytes = value.getBytes();
Process q = new ProcessBuilder("process","arg1", "arg2").start();
q.getOutputStream().write(bytes);
q.getOutputStream().flush();
System.out.println(q.getInputStream().available());
I'm trying to stream file contents to an executable and capture the output but the output(InputStream) is always empty. I can capture the output if i specify the the file location but not with streamed input.
How might I overcome this?

Try wrapping your streams with BufferedInputStream() and BufferedOutputStream():
http://download.oracle.com/javase/6/docs/api/java/lang/Process.html#getOutputStream%28%29
Implementation note: It is a good idea for the output stream to be buffered.
Implementation note: It is a good idea for the input stream to be buffered.
Even with buffered streams, it is still possible for the buffer to fill if you're dealing with large amounts of data, you can deal with this by starting a separate thread to read from q.getInputStream(), so you can still be reading from the process while writing to the process.

Perhaps the program you execute only starts its work when it detects the end of its input data. This is normally done by waiting for an EOF (end-of-file) symbol. You can send this by closing the output stream to the process:
q.getOutputStream().write(bytes);
q.getOutputStream().close();
Try this together with waiting for the process.

I dont know if something else may also be wrong here, but the other process ("process") does not even have time to respond, you are not waiting for it (the method available() does not block). To try this out you can first insert a sleep(2000) after the flush(), and if that works you should switch to query'ing q.getInputStream().available() multiple times with short pauses in between.

I think, you have to wait, until the process finished.
I implemented something like this this way:
public class ProcessReader {
private static final int PROCESS_LOOP_SLEEP_MILLIS = 100;
private String result;
public ProcessReader(Process process) {
BufferedReader resultReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
StringBuilder resultOutput = new StringBuilder();
try {
while (!checkProcessTerminated(process, resultReader, resultOutput)) {
}
} catch (Exception ex1) {
throw new RuntimeException(ex1);
}
result = resultOutput.toString();
}
public String getResult(){
return result;
}
private boolean checkProcessTerminated(Process process, BufferedReader resultReader, StringBuilder resultOutput) throws Exception {
try {
int exit = process.exitValue();
return true;
} catch (IllegalThreadStateException ex) {
Thread.sleep(PROCESS_LOOP_SLEEP_MILLIS);
} finally {
while (resultReader.ready()) {
String out = resultReader.readLine();
resultOutput.append(out).append("\n");
}
}
return false;
}
}
I just removed now some specific code, that you dont need, but it should work, try it.
Regards

Calling a shell script from java hangs

So I'm trying to execute a shell script which produces a lot of output(in 100s of MBs) from a Java file.
This hangs the process and never completes.
However, within the shell script, if I redirect the output of the script to some log file or /dev/null Java file executes and completes in a jiffy.
Is it because of amount of data that the Java program never completes?
If so, is there any documentation as such? or is there any limit on the amount of data(documented)?
Here's how you can simulate this scenario.
Java file will look like:
import java.io.InputStream;
public class LotOfOutput {
public static void main(String[] args) {
String cmd = "sh a-script-which-outputs-huuggee-data.sh";
try {
ProcessBuilder pb = new ProcessBuilder("bash", "-c", cmd);
pb.redirectErrorStream(true);
Process shell = pb.start();
InputStream shellIn = shell.getInputStream();
int shellExitStatus = shell.waitFor();
System.out.println(shellExitStatus);
shellIn.close();
} catch (Exception ignoreMe) {
}
}
}
The script 'a-script-which-outputs-huuggee-data.sh' may look like:
#!/bin/sh
# Toggle the line below
exec 3>&1 > /dev/null 2>&1
count=1
while [ $count -le 1000 ]
do
cat some-big-file
((count++))
done
echo
echo Yes I m done
Free beer for the right answer. :)

It's because you're not reading from the Process' output.
As per the class' Javadocs, if you don't do this then you may end up with a deadlock; the process fills its IO buffer and waits for the "shell" (or listening process) to read from it and empty it. Meanwhile your process, which should be doing this, is blocking waiting for the process to exit.
You'll want to call getInputStream() and read from that reliably (perhaps from another thread) to stop the process blocking.
Also take a look at Five Java Process Pitfalls and When Runtime.exec() Won't - both informative articles about common problems with Process.

You're never reading the input stream, so it's probably blocking because the input buffer is full.

The input/output buffer have a limited size (depending on the operating system). If I remember correctly this wasn't big or Windows XP at least. Try creating a thread that reads the InputStream as fast as possible.
Something along these lines:
class StdInWorker
implements Worker
{
private BufferedReader br;
private boolean run = true;
private int linesRead = 0;
private StdInWorker (Process prcs)
{
this.br = new BufferedReader(
new InputStreamReader(prcs.getInputStream()));
}
public synchronized void run ()
{
String in;
try {
while (this.run) {
while ((in = this.br.readLine()) != null) {
this.buffer.add(in);
linesRead++;
}
Thread.sleep(50);
}
}
catch (IOException ioe) {
ioe.printStackTrace();
}
catch (InterruptedException ie) {}
}
}
}