Develop Reference

Java: is there a way to run a system command and print the output during execution?

I have a python script and it takes a long time to finish. I would like to run it from Java, but also output the script's output while it is executing, so that I can tell if it is properly running.
I've searched and only found examples where we output the output after the system command has finished, rather than during its execution.
Any way to do it while the script is running?
Here's what I have
public void doSomething() throws IOException {
String[] callAndArgs = {"python", "/hi.py"};
Process p = Runtime.getRuntime().exec(callAndArgs);
BufferedReader stdInput = new BufferedReader(new InputStreamReader(p.getInputStream()));
BufferedReader stdError = new BufferedReader(new InputStreamReader(p.getErrorStream()));
String s;
while ((s = stdInput.readLine()) != null) {
System.out.println(s);
}
while ((s = stdError.readLine()) != null) {
System.out.println(s);
}
}

i managed to get it working like this (Note it requires java7):
package test;
import java.lang.ProcessBuilder.Redirect;
public class Test {
public static void main(String... args) throws Exception {
ProcessBuilder pb = new ProcessBuilder("python","/home/foobar/Programming/test/src/test/test.py");
pb.redirectOutput(Redirect.INHERIT);
Process p = pb.start();
p.waitFor();
}
}
python (note i flush on python to make it work using sys.stdout.flush())
import time,sys
c =0
while c<=50:
time.sleep(1)
print("----")
c = c +1
sys.stdout.flush()
Note if you don't want to flush in a loop you can use this:
ProcessBuilder pb = new ProcessBuilder("python","-u","/home/foobar/Programming/NetBeansProjects/test/src/test/test.py");
Redirect.INHERIT
Indicates that subprocess I/O source or destination will be the same as those of the current process. This is the normal behavior of most operating system command interpreters (shells).

I've searched and only found examples where we output the output after
the system command has finished, rather than during its execution.
That's weird, because your example should be dumping the output as the command is executing.
Instead of using BufferedReader, you could try reading directly from the InputStream instead as the required conditions for readLine might not be being met until after the process exits.
I'd also recommend that you use a ProcessBuilder over Process directly, as, apart from anything else, it allows you to redirect the output from the error stream into the input stream, allowing you to read just one stream instead of two...
This might also be an issue with Python and how it flushes it output buffers...
For example, rather then waiting for the BufferedReader to decide when to return, try printing each character from the stream as it occurs/is reported
ProcessBuilder pb = new ProcessBuilder("test.py");
pb.redirectError();
Process p = pb.start();
InputStream is = null;
try {
is = p.getInputStream();
int in = -1;
while ((in = is.read()) != -1) {
System.out.print((char)in);
}
} finally {
try {
is.close();
} catch (Exception e) {
}
}
Update
Doing a little reading, Python seems to be buffering its out out before sending it to the stdout. I don't think you can fix this on the a Java side, but need to alter either the way Python is run or the script works.
See How to flush output of Python print? for more details

I'm suspecting that you are writing to stderr, which you can't see because you are blocking on stdin. Use a ProcessBuilder instead of doing exec. This way, you can redirect stderr and stdin into a single stream.
Here is an example:
import java.io.*;
public class Test {
public static void main(String... args) throws IOException {
ProcessBuilder pb =
new ProcessBuilder("test.py");
pb.redirectErrorStream(true);
Process proc = pb.start();
Reader reader = new InputStreamReader(proc.getInputStream());
BufferedReader bf = new BufferedReader(reader);
String s;
while ((s = bf.readLine()) != null) {
System.out.println(s);
}
}
}
Alternatively you can spawn threads to read from stdin/stderr respectively.
Another thing to look for is output buffering by python. You can see if this is the cause by doing:
import sys
sys.stdout.flush()
after you write to stdout

Don't use #readLine as the conditional in your while loop. Instead wrap your inputStream in a scanner and use #hasNextLine()
Scanner in = new Scanner(p.getInputStream());
while (in.hasNextLine()) {
System.out.println(in.nextLine());
}

Interacting with a command line process

I need to interact with a command line process, e.g. diskpart on windows. The problem: input.readLine() in the following sample leads to a blocking while.
public static void main(String[] args) throws IOException
{
ProcessBuilder processBuilder = new ProcessBuilder("C:\\Windows\\system32\\diskpart.exe");
processBuilder.redirectErrorStream(true);
Process process = processBuilder.start();
input = new BufferedReader(new InputStreamReader(process.getInputStream()));
output = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
// read #1 code position
String line = null;
while((line = input.readLine())!= null)
System.out.println(line);
// code position #2
System.out.println("This line is never executed");
output.write("list disk" + System.lineSeparator());
output.flush(); // important
}
The output (from read #1 code position) is
Microsoft DiskPart-Version 6.1.7601
Copyright (C) 1999-2008 Microsoft Corporation.
Auf Computer: MYPC
This is correct, however after that nothing happens, e.g. code position #2
System.out.println("This line is never executed");
is never reached. Can anyone tell me, why and how to fix this? Thanks!
Update:
Trying to read byte by byte also seems not to work? ):
InputStreamReader input = new InputStreamReader(process.getInputStream());
int mychar = -1;
while((mychar = input.read()) != -1)
System.out.println(mychar);
System.out.println("This line is never executed");

Because the next thing Diskpart does is show the prompt, which doesn't include a newline:
Microsoft DiskPart version 6.1.7601
Copyright (C) 1999-2008 Microsoft Corporation.
On computer: PCNAME
DISKPART> _
So your code sits there waiting for the newline, which never appears.
You need to change your code to send the "list disk" command at the right time.

Diskpart has an interactive console that requires input from the user. Attempting to read its output like this:
while((line = input.readLine())!= null)
System.out.println(line);
will cause you to wait indefinitely as the application itself requires input.
You need to wait for input first from the windows command so you need to add CMD /C to your command.
As diskpart is interactive, you could try running your list command as a script, so you would have instead:
String[] command = {"CMD", "/C", "C:\\Windows\\system32\\diskpart.exe", "/s", "diskpart.txt"};
ProcessBuilder processBuilder = new ProcessBuilder(command);
with diskpart.txt containing:
list disk
I recommend you getting this working in a standard batch file first though to check that the output is correct.

Reading InputStream from Java Interactive Process using ProcessBuilder

I am trying to run an interactive executable from Java application using ProcessBuilder; it's supposed to take input, produce output and then wait for the next input. The main problem here with Input/Output streams. I send an input and get nothing. Here is the code:
private static Process process;
private static BufferedReader result;
private static PrintWriter input;
process = new ProcessBuilder("compile-lm", lmFile.toString(), " --score yes").redirectErrorStream(true).start();
input = new PrintWriter(new OutputStreamWriter(process.getOutputStream()), true);
input.println(message);
System.out.println(message);
result = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line = new String();
while ((line = result.readLine()) != null)
{
/* Some processing for the read line */
System.out.println("output:\t" + line);
}

I have tried your code it works fine there is no problem with the code. I think that the problem with the command that you are trying to execute ( it returns nothing ). try to change parameters or even change the entire command to test. and if you can execute the comand in other place ( terminal for example try it and see the output with the same parameters )

I have used a similar setup many times over but can not find a working copy right now :( My first instinct though is to move the line where you initialise the reader (result variable) to before the one where you send the command out to the process (input.println(message)).

Try closing the output stream to the process. Basically you're at the mercy of whatever buffering is happening in the output side of the child process.

I want realtime output of my Runtime.getRuntime().exec()

public static void executeCommand(String cmd) {
try {
Process process = Runtime.getRuntime().exec(cmd, null,
new File("/usr/hadoop-0.20.2/"));
InputStream stdin = process.getInputStream();
InputStreamReader isr = new InputStreamReader(stdin);
BufferedReader br = new BufferedReader(isr);
String line;
System.out.println("<output></output>");
while ((line = br.readLine()) != null)
System.out.println(line);
InputStreamReader esr = new InputStreamReader(
process.getErrorStream());
BufferedReader errorReader = new BufferedReader(esr);
String lineError;
while ((lineError = errorReader.readLine()) != null)
System.out.println(lineError);
process.waitFor();
System.out.println("");
} catch (Exception e) {
e.printStackTrace();
}
}
Here's my code for executing a command named 'cmd'. But I cannot get realtime output through this code. The output comes out when the command finishes. I want realtime output. Is there a way to do this?

The issue you describe is most likely caused by the application you called: many applications use unbuffered I/O when connected to a terminal, but bufferen I/O when connected to a pipe. So your cmd may simply decide not to write its output in small bits, but instead in huge chunks. The proper fix is to adjust the command, to flush its output at the appropriate times. There is little you can do about this on the Java side. See also this answer.

I think you need to have a thread for handling the output.
You should try first with the cmd which run for a while
Last time, when I try with wvdial command (this wvdial will not finish until we stop it), I need a thread to read the output of wvdial

Actually, the problem is that Process.getInputStream() returns a BufferedReader.
So, even if the called subprocess flushes all its output, a read in the calling Java program will only get it if the buffer is full.

Reading InputStream from Java Process

Im trying to open a terminal console, and be able to read / write commands to it.
I read some questions like:
Java Process with Input/Output Stream
With that was able build a little app that opens the terminal and pass commands to the console and print the result back, it works well with any system comand like browsing folders, deleting files and stuff like that.
The problem I have is that I need to load another java program from that console and read its output but that program uses java.util.logging.Logger to send most of its output and for some reason my launching app can't read what Logger prints.
Basically Im trying to build like a wrapper for another java app, because I want to interact with it but cant modify it.
Thanks for your help.
EDIT
Here is the code, but its basically taken from another questions, also as I said it works for things in the "normal" stdout, but not for the output Logger prints to the console.
package launcher;
import java.io.*;
import java.util.Scanner;
public class Launcher {
public static void main(String[] args) throws IOException {
String line;
Scanner scan = new Scanner(System.in);
Process process = Runtime.getRuntime().exec("/bin/bash");
OutputStream stdin = process.getOutputStream();
InputStream stderr = process.getErrorStream();
InputStream stdout = process.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(stdout));
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(stdin));
while (scan.hasNext()) {
String input = scan.nextLine();
if (input.trim().equals("exit")) {
writer.write("exit\n");
} else {
writer.write("((" + input + ") && echo --EOF--) || echo --EOF--\n");
}
writer.flush();
line = reader.readLine();
while (line != null && !line.trim().equals("--EOF--")) {
System.out.println("Stdout: " + line);
line = reader.readLine();
}
if (line == null) {
break;
}
}
}
}

Without seeing any code/config, I would guess that either the logger is configured to write to stderr (System.err) and you're only reading stdout (System.out), or else the logger is configured to write to a file.

Per dty's answer, I think by default java.util.logging uses stderr, so you should redirect stderr to stdout like this:
ProcessBuilder builder = new ProcessBuilder("ls", "-l"); // or whatever your command is
builder.redirectErrorStream(true);
Process proc = builder.start();
FWIW in my experience, you'd be better off trying to use the other Java program by starting its main method in your own program, than trying to wrestle with input/output streams etc., but that depends on what the other program does.