In the while loop below I want to read only the newest line from Process p's output, ignoring anything else that entered the buffer while the loop was sleeping. How do I do that?
String s;
Runtime r = Runtime.getRuntime();
Process p = r.exec("SomeContinuousProgram");
myInput = new BufferedReader(new InputStreamReader(p.getInputStream()));
while (true){
if ((s = myInput.readLine()) != null) {
System.out.println(s);
}
Thread.sleep(sleep);
}
You can't "skip" to the newest line that was written from the process. You have to read all lines that came before it.
Split the program into 2 threads. The main thread will read from the BufferedReader and will keep track of what the newest line is. The other thread will sleep, and then display the newest line.
while (true){
if ((s = myInput.readLine()) != null) {
System.out.println(s);
}
This code doesn't make any sense. If readLine() returns null, it is the end of the stream, and the only sensible course is to close the stream and exit the reading loop.
Related
I have a weird problem when trying to execute a shell command from within a java program. Since there exist thousands of websites that explain how to do it I used the following recommended code:
public String executeShellCommand (String command)
{
try
{
StringBuffer sb = new StringBuffer();
String line = "";
Process p = Runtime.getRuntime().exec(command);
BufferedReader reader = new BufferedReader(new InputStreamReader(
p.getInputStream()));
while ((line = reader.readLine()) != null)
sb.append(line + "\n");
p.waitFor();
return sb.toString();
}
catch (Exception e)
{
e.printStackTrace();
}
return null;
}
Acutally, when I try to execute for instance ls -aF is works fine and I get some output as a result. Therefore I'm pretty sure that the above code is, in principal, correct. However, I got another program I'd like to run and that produces a file as an output. I would like to execute it the above way but it never is executed and no output file is generated. Also I do not get any error, warnings or whatsoever in java. When copy and pasting the actual command argument string into the console the execution of the programm/command directly in the shell works fine and the output file is generated. So the command I pass to the method is also correct.
Are there additional things I need to pay attention to when trying to execute a shell command from within java?
UPDATE: I modified my code according to the suggestions. However, it is still hanging:
public String executeShellCommand(List<String> command, String logfile, boolean waitForProcess) { try {
ProcessBuilder pb = new ProcessBuilder(command);
System.out.println("pb.toString() = " + pb.toString());
Process p = pb.start();
System.out.println("2");
BufferedReader err = new BufferedReader(new InputStreamReader(p.getErrorStream()));
BufferedReader out = new BufferedReader(new InputStreamReader(p.getInputStream()));
System.out.println("3");
StringBuilder errSb = new StringBuilder();
StringBuilder outSb = new StringBuilder();
String line;
System.out.println("4");
while ((line = err.readLine()) != null) { // <--- code hangs here
errSb.append(line + "\n");
System.out.println("errSb = " + errSb.toString());
}
System.out.println("4a");
while ((line = out.readLine()) != null) {
outSb.append(line + "\n");
System.out.println("outSb = " + outSb.toString());
}
System.out.println("5");
if(waitForProcess) {
System.out.println("Wait for process");
p.waitFor();
} else {
System.out.println("Sleep 5000");
Thread.sleep(5000);
}
System.out.println("6");
//Log result to file
if(logfile != null) {
OutputStreamWriter outWriter = new OutputStreamWriter(new FileOutputStream(logfile));
outWriter.write(errSb.toString());
outWriter.close();
}
return errSb.toString();
} catch(Exception e) { e.printStackTrace(); } return null; }
This will block if your command writes too many characters to stderr. Like for sdtout, Java redirect stderr through a pipe, and if you do not read the pipe, it can fill up and block (size of the pipe is probably less than 256 bytes). To avoid that, you need to read from the Process.getErrorStream(), preferable from another thread as the main thread is busy reading from the Process.getInputStream().
A simpler way to avoid that is to use the ProcessBuilder class instead of Runtime.exec() and ProcessBuilder.redirectErrorStream(true) so that both stdout and stderr are merged into the Process.getInputStream()
As per Process javadoc :
Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, or even deadlock.
You are calling p.waitFor(). If we carefully read the waitFor() documentation:
Causes the current thread to wait, if necessary, until the process represented by this Process object has terminated.
You are waiting for a process which hangs, because its error stream and output stream are never read.
What you should do, is to read these streams:
p.start();
BufferedReader err= new BufferedReader(new InputStreamReader(p.getErrorStream()));
BufferedReader out = new BufferedReader(new InputStreamReader(p.getOutputStream()));
StringBuilder errSb = new StringBuilder();
StringBuilder outSb = new Stringbuilder();
String line;
while ((line = err.readLine()) != null) {
errSb.append(line);
}
while ((line = out.readLine()) != null) {
outSB.append(line);
}
int retCode = p.waitFor(); //0 for success
System.out.println(retCode);
System.err.println(errSB.toString());
You should always read the error stream when calling external programs via the Process class, else you may find yourself in this odd situation where a process hangs forever. (well until someone else -the operating system, another application, etc- kills it, more exactly).
I've also noticed that you use the Runtime.getRuntime() which is not the recommended way to run external programs, starting with java 1.5, as per javadoc:
As of 1.5, ProcessBuilder.start() is the preferred way to create a Process.
ProcessBuilder pb = new ProcessBuilder("ls" , "-aF");
Process p = pb.start();
I've a Java program to call a python script. I've used exec method. Please find the code snippet below:
Python program (which is to gather a portion of text from wikipedia), when run separately, gives me proper output. When called from Java, I'm not getting the complete output from python program.
I checked the status of BufferedReader Object using ready() method ( as explained here, and the code entered into infinite loop.
I think others also have faced similar problems-https://stackoverflow.com/a/20661352/3409074
Can anyone help me?
public String enhanceData(String name,String entity) {
String s = null;
StringBuffer output = new StringBuffer();
try{
String command="python C://enhancer.py "+name+" "+entity;
Process p = Runtime.getRuntime().exec(command);
BufferedReader stdError=new BufferedReader(new InputStreamReader(p.getErrorStream()));
BufferedReader stdInput = new BufferedReader(new InputStreamReader(p.getInputStream()));
while ((s = stdInput.readLine()) != null) {
System.out.println(s);
output.append(s);
}
The while loop condition has actually already read a line so you are double reading it for every time in the loop.
while ((s = stdInput.readLine()) != null) {
//s=stdInput.readLine(); <- don't need this
System.out.println(s);
output.append(s);
}
/Nick
I need to be able to read each line of the file for multiple arguments, hence the for loop. After the first one, it does not seem to be reading them anymore, seems to skip the try statement. Any ideas? I'm sure Its something silly I am missing but have been playing about with it and unfortunately time is not on my side.
for (int j = 0; j < ags.length; j++){
try{
String nameFromFile = null;
BufferedReader InputReader = new BufferedReader(new InputStreamReader(System.in));
while ((nameFromFile = InputReader.readLine()) != null) {
// Do stuff
} catch (IOException e) {
System.out.println(e.getMessage());
}
}
You appear to have two sources you want to compare System.in and args I suggest you read these individually and then compare them.
Set<String> fromInt = new HashSet<>();
try (BufferedReader br = new BufferedReader(new InputStreamReader(System.in))) {
for(String line; (line = br.readLine()) != null;)
fromIn.add(normalise(line));
}
// compare argsList with fromIn.
e.g.
for(String arg: args) {
if (fromIn.contains(normalise(arg))) {
// something
} else {
// something else
}
}
I need to be able to read each line of the file
What file? You're reading from System.in:
BufferedReader InputReader = new BufferedReader(new InputStreamReader(System.in));
Your code will block at this line until you enter something at the console.
You do not read a file, bu the System.in stream.
Every stream has an internal pointer, so the stream nows, which line was read at last.
If the System stream was read once, the pointer is pointing to the end of the stream.
As long as the stream is not reset, the read command will not return anything.
try
InputStream.reset()
or even better, only read the Stream once and cache the result! This is faster and safe, because the Stream input can change during iteration.
Your code will never exit from while loop.
while ((nameFromFile = InputReader.readLine()) != null)
In above loop it will print only one time and at the end of the file it will not be out of the while loop . That's why you are getting only one time output. Since it is not exited from while loop it does not go back into for loop. readLine() return the string and it is terminated by "\n" or "\r\n". Change as below and you will be able to read as ags.length
while ((nameFromFile = InputReader.readLine())=="\n")
I have following piece of code :
fis = new FileInputStream(new File(st[0]));
br = new BufferedReader(new InputStreamReader(fis));
while(fis.available()!=-1)
{
System.out.println(br.readLine());
System.out.println(fis.available());
}
The first println statement prints whole of my file but alongside second println statement always shows 0. why when there is actual content to read, is it showing 0 ?
and what should i put as end condition over here.
You want to stop when readLine() returns null, something like this:
String sCurrentLine;
br = new BufferedReader(new FileReader("C:\\testing.txt"));
while ((sCurrentLine = br.readLine()) != null) {
System.out.println(sCurrentLine);
}
The first println statement prints whole of my file but alongside second println statement always shows 0.
You're checking available() twice. After you've read some data, it's no longer available to read, so the available() value printed is different to the one used for the loop condition above.
Secondly, you're reading from the BufferedReader, which does its own buffering of the data from the input stream. That means it's wrong to then sneak around the reader's back to call the available method of the underlying input stream!
Try this:
for (;;) {
String line = br.readLine();
if (line == null) break;
System.out.println(line);
}
availabe() is returning the amount of bytes that can be read for that InputStream when it is not blocking. your readLine() is blocking that InputStream.
I have the following code example below. Whereby you can enter a command to the bash shell i.e. echo test and have the result echo'd back. However, after the first read. Other output streams don't work?
Why is this or am I doing something wrong? My end goal is to created a Threaded scheduled task that executes a command periodically to /bash so the OutputStream and InputStream would have to work in tandem and not stop working. I have also been experiencing the error java.io.IOException: Broken pipe any ideas?
Thanks.
String line;
Scanner scan = new Scanner(System.in);
Process process = Runtime.getRuntime ().exec ("/bin/bash");
OutputStream stdin = process.getOutputStream ();
InputStream stderr = process.getErrorStream ();
InputStream stdout = process.getInputStream ();
BufferedReader reader = new BufferedReader (new InputStreamReader(stdout));
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(stdin));
String input = scan.nextLine();
input += "\n";
writer.write(input);
writer.flush();
input = scan.nextLine();
input += "\n";
writer.write(input);
writer.flush();
while ((line = reader.readLine ()) != null) {
System.out.println ("Stdout: " + line);
}
input = scan.nextLine();
input += "\n";
writer.write(input);
writer.close();
while ((line = reader.readLine ()) != null) {
System.out.println ("Stdout: " + line);
}
Firstly, I would recommend replacing the line
Process process = Runtime.getRuntime ().exec ("/bin/bash");
with the lines
ProcessBuilder builder = new ProcessBuilder("/bin/bash");
builder.redirectErrorStream(true);
Process process = builder.start();
ProcessBuilder is new in Java 5 and makes running external processes easier. In my opinion, its most significant improvement over Runtime.getRuntime().exec() is that it allows you to redirect the standard error of the child process into its standard output. This means you only have one InputStream to read from. Before this, you needed to have two separate Threads, one reading from stdout and one reading from stderr, to avoid the standard error buffer filling while the standard output buffer was empty (causing the child process to hang), or vice versa.
Next, the loops (of which you have two)
while ((line = reader.readLine ()) != null) {
System.out.println ("Stdout: " + line);
}
only exit when the reader, which reads from the process's standard output, returns end-of-file. This only happens when the bash process exits. It will not return end-of-file if there happens at present to be no more output from the process. Instead, it will wait for the next line of output from the process and not return until it has this next line.
Since you're sending two lines of input to the process before reaching this loop, the first of these two loops will hang if the process hasn't exited after these two lines of input. It will sit there waiting for another line to be read, but there will never be another line for it to read.
I compiled your source code (I'm on Windows at the moment, so I replaced /bin/bash with cmd.exe, but the principles should be the same), and I found that:
after typing in two lines, the output from the first two commands appears, but then the program hangs,
if I type in, say, echo test, and then exit, the program makes it out of the first loop since the cmd.exe process has exited. The program then asks for another line of input (which gets ignored), skips straight over the second loop since the child process has already exited, and then exits itself.
if I type in exit and then echo test, I get an IOException complaining about a pipe being closed. This is to be expected - the first line of input caused the process to exit, and there's nowhere to send the second line.
I have seen a trick that does something similar to what you seem to want, in a program I used to work on. This program kept around a number of shells, ran commands in them and read the output from these commands. The trick used was to always write out a 'magic' line that marks the end of the shell command's output, and use that to determine when the output from the command sent to the shell had finished.
I took your code and I replaced everything after the line that assigns to writer with the following loop:
while (scan.hasNext()) {
String input = scan.nextLine();
if (input.trim().equals("exit")) {
// Putting 'exit' amongst the echo --EOF--s below doesn't work.
writer.write("exit\n");
} else {
writer.write("((" + input + ") && echo --EOF--) || echo --EOF--\n");
}
writer.flush();
line = reader.readLine();
while (line != null && ! line.trim().equals("--EOF--")) {
System.out.println ("Stdout: " + line);
line = reader.readLine();
}
if (line == null) {
break;
}
}
After doing this, I could reliably run a few commands and have the output from each come back to me individually.
The two echo --EOF-- commands in the line sent to the shell are there to ensure that output from the command is terminated with --EOF-- even in the result of an error from the command.
Of course, this approach has its limitations. These limitations include:
if I enter a command that waits for user input (e.g. another shell), the program appears to hang,
it assumes that each process run by the shell ends its output with a newline,
it gets a bit confused if the command being run by the shell happens to write out a line --EOF--.
bash reports a syntax error and exits if you enter some text with an unmatched ).
These points might not matter to you if whatever it is you're thinking of running as a scheduled task is going to be restricted to a command or a small set of commands which will never behave in such pathological ways.
EDIT: improve exit handling and other minor changes following running this on Linux.
I think you can use thread like demon-thread for reading your input and your output reader will already be in while loop in main thread so you can read and write at same time.You can modify your program like this:
Thread T=new Thread(new Runnable() {
#Override
public void run() {
while(true)
{
String input = scan.nextLine();
input += "\n";
try {
writer.write(input);
writer.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
} );
T.start();
and you can reader will be same as above i.e.
while ((line = reader.readLine ()) != null) {
System.out.println ("Stdout: " + line);
}
make your writer as final otherwise it wont be able to accessible by inner class.
You have writer.close(); in your code. So bash receives EOF on its stdin and exits. Then you get Broken pipe when trying to read from the stdoutof the defunct bash.