I am trying to run an interactive executable from Java application using ProcessBuilder; it's supposed to take input, produce output and then wait for the next input. The main problem here with Input/Output streams. I send an input and get nothing. Here is the code:
private static Process process;
private static BufferedReader result;
private static PrintWriter input;
process = new ProcessBuilder("compile-lm", lmFile.toString(), " --score yes").redirectErrorStream(true).start();
input = new PrintWriter(new OutputStreamWriter(process.getOutputStream()), true);
input.println(message);
System.out.println(message);
result = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line = new String();
while ((line = result.readLine()) != null)
{
/* Some processing for the read line */
System.out.println("output:\t" + line);
}
I have tried your code it works fine there is no problem with the code. I think that the problem with the command that you are trying to execute ( it returns nothing ). try to change parameters or even change the entire command to test. and if you can execute the comand in other place ( terminal for example try it and see the output with the same parameters )
I have used a similar setup many times over but can not find a working copy right now :( My first instinct though is to move the line where you initialise the reader (result variable) to before the one where you send the command out to the process (input.println(message)).
Try closing the output stream to the process. Basically you're at the mercy of whatever buffering is happening in the output side of the child process.
Related
I am unfamiliar of the way how to use Jython to call a Python app from my Java (Spring Boot) Application, so I usually use the following method to retrieve the json response from the python app: (the Java app is running on a CentOS7 environment)
StringBuffer output = new StringBuffer();
Process p;
try {
p = Runtime.getRuntime().exec("python test.py");
p.waitFor();
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = "";
while ((line = reader.readLine()) != null) {
log.debug(line + "\n");
output.append(line + "\n");
}
} catch (Exception e) {
e.printStackTrace();
}
return output.toString();
That is helpful when I call any python application, which gives me back nothing but a single line, like {"status":"ok"}
But if it gives me back multiple lines, or an exception after giving me back the json answer I expect, my Java application returns an empty string, like if it wouldn't get any response back from the Python app.
Though when I run the same command via terminal, I get the multiple line answers.
So I wonder if the issue is with my code? Am I missing something to see here which obstructs me to have multiple lines from the answer? I need the answer regardless of how many lines I get back.
Your methode has nothing to do with Jython or Python in general. You are just starting a new Process and reading its standard output.
It just happens that in your case this is a python app and the output should be json (but could be anything).
If I understand you correctly, you only want to "accept" one line json outputs from your python process. Try this:
public String getOutputFromProcess() {
//Use StringBuilder instead of Buffer if you dont need the thread safety
StringBuilder output = new StringBuilder();
StringBuilder error = new StringBuilder();
//Removed error handling for simplicity
Process p = Runtime.getRuntime().exec("python test.py");
p.waitFor(); //Maybe this needs to be moved after the reading part
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
//You also need to read the standard error output
BufferedReader stdError = new BufferedReader(new InputStreamReader(p.getErrorStream()));
String line = "";
int counter = 0;
while ((line = reader.readLine()) != null) {
output.append(line + "\n");
counter++;
}
while ((line = stdError.readLine()) != null) {
error.append(line + "\n");
}
//Check if we have read more than one line
//If yes return empty string or null etc.
if(counter > 1) return "";
//Here you should check if this is a valid json string
return output.toString();
}
In general I would suggest you have a look at Jython and call the python function directly. If you want to use your methode, have a look at ProcessBuilder.
So apparently the solution in my case was to create a shell script serving as a bridge between the Java and the Python app.
For some reason this python app I want to use simply doesn't return with any answer when there would be multiple lines.
There are several workaround on this, probably the best to go with is Jython, as #HectorLector advised.
Also it works when I create a shell script which calls the Python app, and my Java app calls the shell script instead of the Python file.
Another solution is to run the command with an additional > output.txt
in the command, which will make sure that the called process' output will flow into the specified file. Then later the application can retrieve the data from that file, and delete it when it is not necessary anymore.
public static void executeCommand(String cmd) {
try {
Process process = Runtime.getRuntime().exec(cmd, null,
new File("/usr/hadoop-0.20.2/"));
InputStream stdin = process.getInputStream();
InputStreamReader isr = new InputStreamReader(stdin);
BufferedReader br = new BufferedReader(isr);
String line;
System.out.println("<output></output>");
while ((line = br.readLine()) != null)
System.out.println(line);
InputStreamReader esr = new InputStreamReader(
process.getErrorStream());
BufferedReader errorReader = new BufferedReader(esr);
String lineError;
while ((lineError = errorReader.readLine()) != null)
System.out.println(lineError);
process.waitFor();
System.out.println("");
} catch (Exception e) {
e.printStackTrace();
}
}
Here's my code for executing a command named 'cmd'. But I cannot get realtime output through this code. The output comes out when the command finishes. I want realtime output. Is there a way to do this?
The issue you describe is most likely caused by the application you called: many applications use unbuffered I/O when connected to a terminal, but bufferen I/O when connected to a pipe. So your cmd may simply decide not to write its output in small bits, but instead in huge chunks. The proper fix is to adjust the command, to flush its output at the appropriate times. There is little you can do about this on the Java side. See also this answer.
I think you need to have a thread for handling the output.
You should try first with the cmd which run for a while
Last time, when I try with wvdial command (this wvdial will not finish until we stop it), I need a thread to read the output of wvdial
Actually, the problem is that Process.getInputStream() returns a BufferedReader.
So, even if the called subprocess flushes all its output, a read in the calling Java program will only get it if the buffer is full.
I need your suggestions and guidance in following task.
I am using libdmtx which comes with a command line utility which reads the image files for ECC200 Data Matrix barcodes, reads their contents, and writes the decoded messages to standard output.
I want to use this command line utility in my java program on linux platform. I amd using ubuntu linux. I have installed the libdmtx on my linux machine. and when I invoke the command
dmtxread -n /home/admin/ab.tif
on linux terminal it gives the decoded value of barcode in image immediately.
when I am going to invoke this command using my java program the code stuks in execution of the command and dotn gives output.
it looks like the program is processing or got hang.
Following is my java code which invokes the following command
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.InputStreamReader;
public class Classtest {
public static void getCodes(){
try
{
Process p;
String command[]=new String[3];
command[0]="dmtxread";
command[1]="-n";
command[2]="/home/admin/ab.tif";
System.out.println("Command : "+command[0]+command[1]+command[2]);
p=Runtime.getRuntime().exec(command); //I think hangs over here.
BufferedReader reader=new BufferedReader(new InputStreamReader(p.getErrorStream()));
String line=reader.readLine();
if(line==null){
reader=new BufferedReader(new InputStreamReader(p.getInputStream()));
line=reader.readLine();
System.out.print("Decoded :- "+line);
}else{
System.out.print("Error :- "+line);
}
System.out.println(p.waitFor());
}catch(IOException e1) {
e1.getMessage();
e1.printStackTrace();
}catch(InterruptedException e2) {
e2.getMessage();
e2.printStackTrace();
}
}
public static void main(String args[]){
getCodes();
}
}
Please tell me friends where my code is going wrong.
I refered to following article but dint get any help
http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html?page=1
Please guide me friends!
Thank you!
Here is the new code in which I used the ProcessBuilder Class this code also giving the same output as above code that is it hangs at the line
Process process = pb.start();
public class Test {
public static void main(final String[] args) throws IOException, InterruptedException {
//Build command
List<String> commands = new ArrayList<String>();
commands.add("dmtxread");
commands.add("-n");
commands.add("/home/admin/ab.tif");
System.out.println(commands);
//Run macro on target
ProcessBuilder pb = new ProcessBuilder(commands);
pb.redirectErrorStream(true);
Process process = pb.start();
//Read output
StringBuilder out = new StringBuilder();
BufferedReader br = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line = null, previous = null;
while ((line = br.readLine()) != null){
System.out.println(line);
}
//Check result
if (process.waitFor() == 0)
System.out.println("Success!");
System.exit(0);
//Abnormal termination: Log command parameters and output and throw ExecutionException
System.err.println(commands);
System.err.println(out.toString());
System.exit(1);
}
}
Please guide me to solve this problem.
Thanks You!
The readLine blocks until it receives a new line from the error stream. So, if there is no output, your program won't get past the first readLine.
For simplicity I would recommend you use a ProcessBuilder instead of Runtime.exec(), which lets you merge the two InputStreams as follows:
ProcessBuilder builder = new ProcessBuilder(cmd,arg0,arg1);
builder.redirectErrorStream(true);
Process process = builder.start();
So, now you can just read from one.
Alternatively you can use separate threads to consume the two InputStreams.
Hope that helps
Your stream-consumption code is very confused. You try to read a single line from the stderr, then abandon that reader, then try to read a single line from the stdout.
If the program doesn't print anything to stderr, you'll hang at line 2.
If the program sends too much stuff to stderr so it fills its buffer, then the program itself will block and your Java will block at waitFor.
Both of these apply to stdout.
The proper way to consume the process's output streams is covered in detail in that article you have linked. Take that advice, nobody can give you better advice than that.
I am not sure what exactly happens with your program and where does it hang (you could use a debugger or trace output to check that), but here is the possible scenario:
Imagine that the program wants to output 2 lines of text. Or only one line but into stderr. Your code reads only 1 line fro stdout and than waits for the process to exit. This means that the child program may wait for the reader to read the next line, so it waits in write until someone unblocks the pipe -- forever.
When you run dmtxread from command line, there is no blocking on output pipe, so the program runs just finely.
Im trying to open a terminal console, and be able to read / write commands to it.
I read some questions like:
Java Process with Input/Output Stream
With that was able build a little app that opens the terminal and pass commands to the console and print the result back, it works well with any system comand like browsing folders, deleting files and stuff like that.
The problem I have is that I need to load another java program from that console and read its output but that program uses java.util.logging.Logger to send most of its output and for some reason my launching app can't read what Logger prints.
Basically Im trying to build like a wrapper for another java app, because I want to interact with it but cant modify it.
Thanks for your help.
EDIT
Here is the code, but its basically taken from another questions, also as I said it works for things in the "normal" stdout, but not for the output Logger prints to the console.
package launcher;
import java.io.*;
import java.util.Scanner;
public class Launcher {
public static void main(String[] args) throws IOException {
String line;
Scanner scan = new Scanner(System.in);
Process process = Runtime.getRuntime().exec("/bin/bash");
OutputStream stdin = process.getOutputStream();
InputStream stderr = process.getErrorStream();
InputStream stdout = process.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(stdout));
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(stdin));
while (scan.hasNext()) {
String input = scan.nextLine();
if (input.trim().equals("exit")) {
writer.write("exit\n");
} else {
writer.write("((" + input + ") && echo --EOF--) || echo --EOF--\n");
}
writer.flush();
line = reader.readLine();
while (line != null && !line.trim().equals("--EOF--")) {
System.out.println("Stdout: " + line);
line = reader.readLine();
}
if (line == null) {
break;
}
}
}
}
Without seeing any code/config, I would guess that either the logger is configured to write to stderr (System.err) and you're only reading stdout (System.out), or else the logger is configured to write to a file.
Per dty's answer, I think by default java.util.logging uses stderr, so you should redirect stderr to stdout like this:
ProcessBuilder builder = new ProcessBuilder("ls", "-l"); // or whatever your command is
builder.redirectErrorStream(true);
Process proc = builder.start();
FWIW in my experience, you'd be better off trying to use the other Java program by starting its main method in your own program, than trying to wrestle with input/output streams etc., but that depends on what the other program does.
I have the following code example below. Whereby you can enter a command to the bash shell i.e. echo test and have the result echo'd back. However, after the first read. Other output streams don't work?
Why is this or am I doing something wrong? My end goal is to created a Threaded scheduled task that executes a command periodically to /bash so the OutputStream and InputStream would have to work in tandem and not stop working. I have also been experiencing the error java.io.IOException: Broken pipe any ideas?
Thanks.
String line;
Scanner scan = new Scanner(System.in);
Process process = Runtime.getRuntime ().exec ("/bin/bash");
OutputStream stdin = process.getOutputStream ();
InputStream stderr = process.getErrorStream ();
InputStream stdout = process.getInputStream ();
BufferedReader reader = new BufferedReader (new InputStreamReader(stdout));
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(stdin));
String input = scan.nextLine();
input += "\n";
writer.write(input);
writer.flush();
input = scan.nextLine();
input += "\n";
writer.write(input);
writer.flush();
while ((line = reader.readLine ()) != null) {
System.out.println ("Stdout: " + line);
}
input = scan.nextLine();
input += "\n";
writer.write(input);
writer.close();
while ((line = reader.readLine ()) != null) {
System.out.println ("Stdout: " + line);
}
Firstly, I would recommend replacing the line
Process process = Runtime.getRuntime ().exec ("/bin/bash");
with the lines
ProcessBuilder builder = new ProcessBuilder("/bin/bash");
builder.redirectErrorStream(true);
Process process = builder.start();
ProcessBuilder is new in Java 5 and makes running external processes easier. In my opinion, its most significant improvement over Runtime.getRuntime().exec() is that it allows you to redirect the standard error of the child process into its standard output. This means you only have one InputStream to read from. Before this, you needed to have two separate Threads, one reading from stdout and one reading from stderr, to avoid the standard error buffer filling while the standard output buffer was empty (causing the child process to hang), or vice versa.
Next, the loops (of which you have two)
while ((line = reader.readLine ()) != null) {
System.out.println ("Stdout: " + line);
}
only exit when the reader, which reads from the process's standard output, returns end-of-file. This only happens when the bash process exits. It will not return end-of-file if there happens at present to be no more output from the process. Instead, it will wait for the next line of output from the process and not return until it has this next line.
Since you're sending two lines of input to the process before reaching this loop, the first of these two loops will hang if the process hasn't exited after these two lines of input. It will sit there waiting for another line to be read, but there will never be another line for it to read.
I compiled your source code (I'm on Windows at the moment, so I replaced /bin/bash with cmd.exe, but the principles should be the same), and I found that:
after typing in two lines, the output from the first two commands appears, but then the program hangs,
if I type in, say, echo test, and then exit, the program makes it out of the first loop since the cmd.exe process has exited. The program then asks for another line of input (which gets ignored), skips straight over the second loop since the child process has already exited, and then exits itself.
if I type in exit and then echo test, I get an IOException complaining about a pipe being closed. This is to be expected - the first line of input caused the process to exit, and there's nowhere to send the second line.
I have seen a trick that does something similar to what you seem to want, in a program I used to work on. This program kept around a number of shells, ran commands in them and read the output from these commands. The trick used was to always write out a 'magic' line that marks the end of the shell command's output, and use that to determine when the output from the command sent to the shell had finished.
I took your code and I replaced everything after the line that assigns to writer with the following loop:
while (scan.hasNext()) {
String input = scan.nextLine();
if (input.trim().equals("exit")) {
// Putting 'exit' amongst the echo --EOF--s below doesn't work.
writer.write("exit\n");
} else {
writer.write("((" + input + ") && echo --EOF--) || echo --EOF--\n");
}
writer.flush();
line = reader.readLine();
while (line != null && ! line.trim().equals("--EOF--")) {
System.out.println ("Stdout: " + line);
line = reader.readLine();
}
if (line == null) {
break;
}
}
After doing this, I could reliably run a few commands and have the output from each come back to me individually.
The two echo --EOF-- commands in the line sent to the shell are there to ensure that output from the command is terminated with --EOF-- even in the result of an error from the command.
Of course, this approach has its limitations. These limitations include:
if I enter a command that waits for user input (e.g. another shell), the program appears to hang,
it assumes that each process run by the shell ends its output with a newline,
it gets a bit confused if the command being run by the shell happens to write out a line --EOF--.
bash reports a syntax error and exits if you enter some text with an unmatched ).
These points might not matter to you if whatever it is you're thinking of running as a scheduled task is going to be restricted to a command or a small set of commands which will never behave in such pathological ways.
EDIT: improve exit handling and other minor changes following running this on Linux.
I think you can use thread like demon-thread for reading your input and your output reader will already be in while loop in main thread so you can read and write at same time.You can modify your program like this:
Thread T=new Thread(new Runnable() {
#Override
public void run() {
while(true)
{
String input = scan.nextLine();
input += "\n";
try {
writer.write(input);
writer.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
} );
T.start();
and you can reader will be same as above i.e.
while ((line = reader.readLine ()) != null) {
System.out.println ("Stdout: " + line);
}
make your writer as final otherwise it wont be able to accessible by inner class.
You have writer.close(); in your code. So bash receives EOF on its stdin and exits. Then you get Broken pipe when trying to read from the stdoutof the defunct bash.