In the following program am giving name as "don" so the command will search activedirectory
with all the names starting with don (like donald etc). But the line2 variable becomes null after the assignment from reader object and it never goes into the loop. What am i doing wrong? FYI: the command works when i give it on the command line.
try {
Process p = Runtime.getRuntime().exec(
"dsquery user -name " + name + "* -limit 200|dsget user -samid -display");
p.waitFor();
BufferedReader reader = new BufferedReader(
new InputStreamReader(p.getInputStream()));
String line2 = reader.readLine();
HashMap<String,String> hmap = new HashMap<String,String>();
while (line2 != null) {
line2 = line2.trim();
if (line2.startsWith("dsget")||line2.startsWith("samid")) {
continue;
}
String[] arr = line2.split(" ",1);
hmap.put(arr[0].toLowerCase(),arr[1].toLowerCase());
line2 = reader.readLine();
}
reader.close();
line2 = reader.readLine();
}
If I am not mistaken, the pipe (or redirection) requires to launch the programs with cmd.exe.
Something like:
Process p = Runtime.getRuntime().exec("cmd /c dsquery user -name " + name + "* -limit 200|dsget user -samid -display");
I can see at least some possible problems:
1) as PhiLho wrote: pipe and redirection is done by the shell (sh, bash,... or cmd.exe on Windows). You must handle it in the Java code or run your commands in a shell.
2) after calling waitFor() the Thread is blocked until the process terminates, the process only terminates if you "consume" it's InputStream. This is not happening since waitFor() is still waiting... Better to read and process the InputStream in an additional Thread (or call waitFor after reading the InputStream).
3) reading after closing (2 last lines) should throw an Exception.
Reading the ErrorStream could help find some errors, and checking the return of waitFor is also indicated.
EDIT:
actually there should be some Exceptions being throw by that code.
Are the Exceptions being reported (printStackTrace) or just ignored?
Related
I can run this command from the command line without any problem (the validation script executes):
c:/Python27/python ../feedvalidator/feedvalidator/src/demo.py https://das.dynalias.org:8080/das_core/das/2.16.840.1.113883.4.349/1012581676V377802/otherAdminData/careCoordinators
and from java if I leave off the URL parameter and just do:
String[] args1 = {"c:/Python27/python", "../feedvalidator/feedvalidator/src/demo.py" };
Runtime r = Runtime.getRuntime();
Process p = r.exec(args1);
it works fine. If I use certain URLs for a parameter such as:
String[] args1 = {"c:/Python27/python", "../feedvalidator/feedvalidator/src/demo.py" , "http://www.intertwingly.net/blog/index.atom"};
// or
String[] args1 = {"c:/Python27/python", "../feedvalidator/feedvalidator/src/demo.py" , "http://www.cnn.com"};
it also works fine.
But if I use this particular URL https://das.dynalias.org:8080/das_core/das/2.16.840.1.113883.4.349/1012581676V377802/otherAdminData/careCoordinators, then the script just hangs (java waits for the process to finish). I’m not sure why it works from the command line for that URL but not from a java program. I tried adding quotes to surround the URL parameter but that didn’t work either. I don’t see any character in the URL that I think need to be escaped.
Full Code:
String urlToValidate = "https://das.dynalias.org:8080/das_core/das/2.16.840.1.113883.4.349/1012581676V377802/otherAdminData/careCoordinators";
String[] args1 = {"c:/Python27/python", "C:/Documents and Settings/vhaiswcaldej/DAS_Workspace/feedvalidator/feedvalidator/src/demo.py", urlToValidate };
System.out.println(args1[0] + " " + args1[1] + " " + args1[2]);
Runtime r = Runtime.getRuntime();
Process p = r.exec(args1);
BufferedReader br = new BufferedReader(new InputStreamReader(
p.getInputStream()));
int returnCode = p.waitFor();
System.out.println("Python Script or OS Return Code: " + Integer.toString(returnCode));
if (returnCode >= 2) {
.out.println("OS Error: Unable to Find File or other OS error.");
}
String line = "";
while (br.ready()) {
String str = br.readLine();
System.out.println(str);
if (str.startsWith("line")) {
//TODO: Report this error back to test tool.
//System.out.println("Error!");
}
}
You need to drain the output and error streams of the process, or else it will block when the executed program produces output.
From the Process documentation:
Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.
People usually got caught by exec routine hangs in Java. I was cought by that once too. The problem is that the process you are trying to execute may (depending on lot of things) either first write to stdOut or stdErr. If you handle them in wrong order exec will hang. To handle this properly always you must create 2 threads to read stdErr and stdOut simulteneously. Sth like:
Process proc = Runtime.getRuntime().exec( cmd );
// handle process' stdout stream
Thread out = new StreamHandlerThread( stdOut, proc.getInputStream() );
out.start();
// handle process' stderr stream
Thread err = new StreamHandlerThread( stdErr, proc.getErrorStream() );
err.start();
exitVal = proc.waitFor(); // InterruptedException
...
out.join();
err.join();
Read (and close) p.getInputStream() and p.getErrorStream().
For example:
// com.google.common.io.CharStreams
CharStreams.toString(new InputStreamReader(p.getInputStream()));
CharStreams.toString(new InputStreamReader(p.getErrorStream()));
I would like to read and process the output of tcpdump line by line as connections come in. So far I am using the Process class. I also want to run as a non-root user, so I have configured sudo:
user1 ALL= NOPASSWD:/usr/sbin/sudo
I am running on RHEL 6.4.
My Java code looks like this:
String[] tcpdumpCmd = {"/usr/bin/sudo", "-n", "/usr/sbin/tcpdump", "-i", "eth0", "port 8561"};
Process p = new ProcessBuilder(tcpdumpCmd).start();
BufferedReader in = new BufferedReader(new InputStreamReader(p.getInputStream()));
tcpdumpOut = null;
tcpdumpLineCnt = 0;
while ((tcpdumpOut = in.readLine()) != null ) {
System.out.println(tcpdumpOut);
tcpdumpLineCnt++;
}
System.out.println("Output lines: " + tcpdumpLineCnt+"\nCommand exit code: " + p.exitValue());
The command works fine, so it is obvious to me what is happening because when I Ctrl-C I get the output printed. So it doing a sort of "buffering" until I interrupt it (at which point the remaining code isn't executed, so I don't get to the RC part.)
I assume that either my problem is in how I handle the output in my while loop, or it just cannot be done in this way. So I kindly ask if anyone has any advice for me.
I need to interact with a command line process, e.g. diskpart on windows. The problem: input.readLine() in the following sample leads to a blocking while.
public static void main(String[] args) throws IOException
{
ProcessBuilder processBuilder = new ProcessBuilder("C:\\Windows\\system32\\diskpart.exe");
processBuilder.redirectErrorStream(true);
Process process = processBuilder.start();
input = new BufferedReader(new InputStreamReader(process.getInputStream()));
output = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
// read #1 code position
String line = null;
while((line = input.readLine())!= null)
System.out.println(line);
// code position #2
System.out.println("This line is never executed");
output.write("list disk" + System.lineSeparator());
output.flush(); // important
}
The output (from read #1 code position) is
Microsoft DiskPart-Version 6.1.7601
Copyright (C) 1999-2008 Microsoft Corporation.
Auf Computer: MYPC
This is correct, however after that nothing happens, e.g. code position #2
System.out.println("This line is never executed");
is never reached. Can anyone tell me, why and how to fix this? Thanks!
Update:
Trying to read byte by byte also seems not to work? ):
InputStreamReader input = new InputStreamReader(process.getInputStream());
int mychar = -1;
while((mychar = input.read()) != -1)
System.out.println(mychar);
System.out.println("This line is never executed");
Because the next thing Diskpart does is show the prompt, which doesn't include a newline:
Microsoft DiskPart version 6.1.7601
Copyright (C) 1999-2008 Microsoft Corporation.
On computer: PCNAME
DISKPART> _
So your code sits there waiting for the newline, which never appears.
You need to change your code to send the "list disk" command at the right time.
Diskpart has an interactive console that requires input from the user. Attempting to read its output like this:
while((line = input.readLine())!= null)
System.out.println(line);
will cause you to wait indefinitely as the application itself requires input.
You need to wait for input first from the windows command so you need to add CMD /C to your command.
As diskpart is interactive, you could try running your list command as a script, so you would have instead:
String[] command = {"CMD", "/C", "C:\\Windows\\system32\\diskpart.exe", "/s", "diskpart.txt"};
ProcessBuilder processBuilder = new ProcessBuilder(command);
with diskpart.txt containing:
list disk
I recommend you getting this working in a standard batch file first though to check that the output is correct.
I can run this command from the command line without any problem (the validation script executes):
c:/Python27/python ../feedvalidator/feedvalidator/src/demo.py https://das.dynalias.org:8080/das_core/das/2.16.840.1.113883.4.349/1012581676V377802/otherAdminData/careCoordinators
and from java if I leave off the URL parameter and just do:
String[] args1 = {"c:/Python27/python", "../feedvalidator/feedvalidator/src/demo.py" };
Runtime r = Runtime.getRuntime();
Process p = r.exec(args1);
it works fine. If I use certain URLs for a parameter such as:
String[] args1 = {"c:/Python27/python", "../feedvalidator/feedvalidator/src/demo.py" , "http://www.intertwingly.net/blog/index.atom"};
// or
String[] args1 = {"c:/Python27/python", "../feedvalidator/feedvalidator/src/demo.py" , "http://www.cnn.com"};
it also works fine.
But if I use this particular URL https://das.dynalias.org:8080/das_core/das/2.16.840.1.113883.4.349/1012581676V377802/otherAdminData/careCoordinators, then the script just hangs (java waits for the process to finish). I’m not sure why it works from the command line for that URL but not from a java program. I tried adding quotes to surround the URL parameter but that didn’t work either. I don’t see any character in the URL that I think need to be escaped.
Full Code:
String urlToValidate = "https://das.dynalias.org:8080/das_core/das/2.16.840.1.113883.4.349/1012581676V377802/otherAdminData/careCoordinators";
String[] args1 = {"c:/Python27/python", "C:/Documents and Settings/vhaiswcaldej/DAS_Workspace/feedvalidator/feedvalidator/src/demo.py", urlToValidate };
System.out.println(args1[0] + " " + args1[1] + " " + args1[2]);
Runtime r = Runtime.getRuntime();
Process p = r.exec(args1);
BufferedReader br = new BufferedReader(new InputStreamReader(
p.getInputStream()));
int returnCode = p.waitFor();
System.out.println("Python Script or OS Return Code: " + Integer.toString(returnCode));
if (returnCode >= 2) {
.out.println("OS Error: Unable to Find File or other OS error.");
}
String line = "";
while (br.ready()) {
String str = br.readLine();
System.out.println(str);
if (str.startsWith("line")) {
//TODO: Report this error back to test tool.
//System.out.println("Error!");
}
}
You need to drain the output and error streams of the process, or else it will block when the executed program produces output.
From the Process documentation:
Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.
People usually got caught by exec routine hangs in Java. I was cought by that once too. The problem is that the process you are trying to execute may (depending on lot of things) either first write to stdOut or stdErr. If you handle them in wrong order exec will hang. To handle this properly always you must create 2 threads to read stdErr and stdOut simulteneously. Sth like:
Process proc = Runtime.getRuntime().exec( cmd );
// handle process' stdout stream
Thread out = new StreamHandlerThread( stdOut, proc.getInputStream() );
out.start();
// handle process' stderr stream
Thread err = new StreamHandlerThread( stdErr, proc.getErrorStream() );
err.start();
exitVal = proc.waitFor(); // InterruptedException
...
out.join();
err.join();
Read (and close) p.getInputStream() and p.getErrorStream().
For example:
// com.google.common.io.CharStreams
CharStreams.toString(new InputStreamReader(p.getInputStream()));
CharStreams.toString(new InputStreamReader(p.getErrorStream()));
I have the following code example below. Whereby you can enter a command to the bash shell i.e. echo test and have the result echo'd back. However, after the first read. Other output streams don't work?
Why is this or am I doing something wrong? My end goal is to created a Threaded scheduled task that executes a command periodically to /bash so the OutputStream and InputStream would have to work in tandem and not stop working. I have also been experiencing the error java.io.IOException: Broken pipe any ideas?
Thanks.
String line;
Scanner scan = new Scanner(System.in);
Process process = Runtime.getRuntime ().exec ("/bin/bash");
OutputStream stdin = process.getOutputStream ();
InputStream stderr = process.getErrorStream ();
InputStream stdout = process.getInputStream ();
BufferedReader reader = new BufferedReader (new InputStreamReader(stdout));
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(stdin));
String input = scan.nextLine();
input += "\n";
writer.write(input);
writer.flush();
input = scan.nextLine();
input += "\n";
writer.write(input);
writer.flush();
while ((line = reader.readLine ()) != null) {
System.out.println ("Stdout: " + line);
}
input = scan.nextLine();
input += "\n";
writer.write(input);
writer.close();
while ((line = reader.readLine ()) != null) {
System.out.println ("Stdout: " + line);
}
Firstly, I would recommend replacing the line
Process process = Runtime.getRuntime ().exec ("/bin/bash");
with the lines
ProcessBuilder builder = new ProcessBuilder("/bin/bash");
builder.redirectErrorStream(true);
Process process = builder.start();
ProcessBuilder is new in Java 5 and makes running external processes easier. In my opinion, its most significant improvement over Runtime.getRuntime().exec() is that it allows you to redirect the standard error of the child process into its standard output. This means you only have one InputStream to read from. Before this, you needed to have two separate Threads, one reading from stdout and one reading from stderr, to avoid the standard error buffer filling while the standard output buffer was empty (causing the child process to hang), or vice versa.
Next, the loops (of which you have two)
while ((line = reader.readLine ()) != null) {
System.out.println ("Stdout: " + line);
}
only exit when the reader, which reads from the process's standard output, returns end-of-file. This only happens when the bash process exits. It will not return end-of-file if there happens at present to be no more output from the process. Instead, it will wait for the next line of output from the process and not return until it has this next line.
Since you're sending two lines of input to the process before reaching this loop, the first of these two loops will hang if the process hasn't exited after these two lines of input. It will sit there waiting for another line to be read, but there will never be another line for it to read.
I compiled your source code (I'm on Windows at the moment, so I replaced /bin/bash with cmd.exe, but the principles should be the same), and I found that:
after typing in two lines, the output from the first two commands appears, but then the program hangs,
if I type in, say, echo test, and then exit, the program makes it out of the first loop since the cmd.exe process has exited. The program then asks for another line of input (which gets ignored), skips straight over the second loop since the child process has already exited, and then exits itself.
if I type in exit and then echo test, I get an IOException complaining about a pipe being closed. This is to be expected - the first line of input caused the process to exit, and there's nowhere to send the second line.
I have seen a trick that does something similar to what you seem to want, in a program I used to work on. This program kept around a number of shells, ran commands in them and read the output from these commands. The trick used was to always write out a 'magic' line that marks the end of the shell command's output, and use that to determine when the output from the command sent to the shell had finished.
I took your code and I replaced everything after the line that assigns to writer with the following loop:
while (scan.hasNext()) {
String input = scan.nextLine();
if (input.trim().equals("exit")) {
// Putting 'exit' amongst the echo --EOF--s below doesn't work.
writer.write("exit\n");
} else {
writer.write("((" + input + ") && echo --EOF--) || echo --EOF--\n");
}
writer.flush();
line = reader.readLine();
while (line != null && ! line.trim().equals("--EOF--")) {
System.out.println ("Stdout: " + line);
line = reader.readLine();
}
if (line == null) {
break;
}
}
After doing this, I could reliably run a few commands and have the output from each come back to me individually.
The two echo --EOF-- commands in the line sent to the shell are there to ensure that output from the command is terminated with --EOF-- even in the result of an error from the command.
Of course, this approach has its limitations. These limitations include:
if I enter a command that waits for user input (e.g. another shell), the program appears to hang,
it assumes that each process run by the shell ends its output with a newline,
it gets a bit confused if the command being run by the shell happens to write out a line --EOF--.
bash reports a syntax error and exits if you enter some text with an unmatched ).
These points might not matter to you if whatever it is you're thinking of running as a scheduled task is going to be restricted to a command or a small set of commands which will never behave in such pathological ways.
EDIT: improve exit handling and other minor changes following running this on Linux.
I think you can use thread like demon-thread for reading your input and your output reader will already be in while loop in main thread so you can read and write at same time.You can modify your program like this:
Thread T=new Thread(new Runnable() {
#Override
public void run() {
while(true)
{
String input = scan.nextLine();
input += "\n";
try {
writer.write(input);
writer.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
} );
T.start();
and you can reader will be same as above i.e.
while ((line = reader.readLine ()) != null) {
System.out.println ("Stdout: " + line);
}
make your writer as final otherwise it wont be able to accessible by inner class.
You have writer.close(); in your code. So bash receives EOF on its stdin and exits. Then you get Broken pipe when trying to read from the stdoutof the defunct bash.