I'm trying to create a Thread that keeps netsh windows command-line tool open so I can execute netsh commands without open it every single time.
The thing is, once I've created the Thread, just the first command call works... the subsequent calls seems to have no effect.
Here is my code:
public class NetshThread implements Runnable{
private static Process netshProcess = null;
private static BufferedInputStream netshInStream = null;
private static BufferedOutputStream netshOutStream = null;
public BufferedReader inPipe = null;
public void run(){
startNetsh();
}
public void startNetsh(){
try {
netshProcess = Runtime.getRuntime().exec("netsh");
netshInStream = new BufferedInputStream(netshProcess.getInputStream());
netshOutStream = new BufferedOutputStream(netshProcess.getOutputStream());
inPipe = new BufferedReader(new InputStreamReader(netshInStream));
} catch (IOException e) {
e.printStackTrace();
}
}
public void executeCommand(String command){
System.out.println("Executing: " + command);
try {
String str = "";
netshOutStream.write(command.getBytes());
netshOutStream.close();
while ((str = inPipe.readLine()) != null) {
System.out.println(str);
}
} catch (IOException e) {
e.printStackTrace();
}
}
public void closeNetsh(){
executeCommand("exit");
}
public static void main(String[] args){
NetshThread nthread = new NetshThread();
nthread.run();
String command = "int ip set address " +
"\"Local Area Connection 6\" static .69.69.69 255.255.255.0";
nthread.executeCommand(command);
command = "int ip set address " +
"\"Local Area Connection 6\" static 69.69.69.69 255.255.255.0";
nthread.executeCommand(command);
System.out.println("*** DONE ***");
}
}
Thank you!!! =)
Update 1:
Ok... I'm now using a PrintWriter instead... so I think I don't need to flush anything anymore, since the constructor is:
new PrintWriter(netshOutStream, true); (just like Mr. Shiny told me)...
Suppose I decide to break the while loop when the first output line is available... I doesn't work either... the next command wont be executed.... My code now looks like:
import java.io.*;
public class NetshThread implements Runnable{
private static Process netshProcess = null;
private static BufferedInputStream netshInStream = null;
private static BufferedOutputStream netshOutStream = null;
public BufferedReader inPipe = null;
private PrintWriter netshWriter = null;
public void run(){
startNetsh();
}
public void startNetsh(){
try {
netshProcess = Runtime.getRuntime().exec("netsh");
netshInStream = new BufferedInputStream(netshProcess.getInputStream());
netshOutStream = new BufferedOutputStream(netshProcess.getOutputStream());
netshWriter = new PrintWriter(netshOutStream, true);
inPipe = new BufferedReader(new InputStreamReader(netshInStream));
} catch (IOException e) {
e.printStackTrace();
}
}
public void executeCommand(String command){
System.out.println("Executing: " + command);
try {
String str = "";
netshWriter.println(command);
while ((str = inPipe.readLine()) != null) {
System.out.println(str);
break;
}
} catch (IOException e) {
e.printStackTrace();
}
}
public void closeNetsh(){
executeCommand("exit");
}
public static void main(String[] args){
NetshThread nthread = new NetshThread();
Thread xs = new Thread(nthread);
xs.run();
String command = "int ip set address " +
"\"Local Area Connection 6\" static .69.69.69 255.255.255.0";
nthread.executeCommand(command);
command = "int ip set address " +
"\"Local Area Connection 6\" static 69.69.69.69 255.255.255.0";
nthread.executeCommand(command);
System.out.println("*** DONE ***");
}
}
and the output I get:
Executing: int ip set address "Local
Area Connection 6" static .69.69.69
255.255.255.0 netsh>.69.69.69 is not an acceptable value for addr.
Executing: int ip set address "Local
Area Connection 6" static 69.69.69.69
Why the second command is not executed???
255.255.255.0
* DONE *
Update 2:
Everything seemed to work just fine until a teacher tried my app in a spanish-windows enviroment....
my code looks like this:
Scanner fi = new Scanner(netshProcess.getInputStream());
public void executeCommand(String command) {
System.out.println("Executing: " + command);
String str = "";
netshWriter.println(command);
fi.skip("\\s*");
str = fi.nextLine();
System.out.println(str);
}
and what i need is to somehow set the netshWriter encoding to the windows default.
Can anyone know who to do this?
You are closing the output stream.
You need to move the stream processing into separate threads. What's happening is that inPipe.readLine() is blocking waiting for netsh to return data. Apache has a package that deals with process handling. I'd look at using that instead of rolling your own (http://commons.apache.org/exec/)
This seems wrong in many ways.
First, why a Runnable object? This isn't ever passed to a Thread anywhere. The only thread you're creating isn't a java thread, it is an OS process created by exec().
Second, you need a way to know when netsh is done. Your loop that reads the output of netsh will just run forever because readLine will only return null when netsh closes its standard out (which is never, in your case). You need to look for some standard thing that netsh prints when it is done processing your request.
And as others mentioned, close is bad. Use a flush. And hope netsh uses a flush back to you...
I'd try:
PrintWriter netshWriter = new PrintWriter(netshOutputStream, true); // auto-flush writer
netshWriter.println(command);
No close()ing the stream, flush the stream automatically, and uses a writer to send character data rather than relying on the platforms "native character set".
You do definitely need to remove the close, else you'll never be able to execute another command. When you say "it won't work" once the close() call removed, do you mean no commands are processed?
Chances are that after you send the bytes for the command, you need to send some kind of confirmation key for the process to start, well, processing it. If you'd normally enter this from the keyboard it might be as simple as a carriage return, otherwise it might need to be a Ctrl-D or similar.
I'd try replacing the close() line with
netshOutStream.write('\n');
and see if that works. Depending on the software you might need to change the character(s) you send to signify the end of the command, but this general approach should see you through.
EDIT:
It would also be prudent to call
netshOutStream.flush();
after the above lines; without the flush there's no guarantee that your data will be written and in fact, since you're using a BufferedInputStream I'm 99% sure that nothing will be written until the stream is flushed. Hence why the code afterwards blocks, as you're waiting for a response while the process has not seen any input yet either and is waiting for you to send it some.
I've used scanner instead of BufferedReader, just because I like it. So this code works:
Scanner fi = new Scanner(netshProcess.getInputStream());
public void executeCommand(String command) {
System.out.println("Executing: " + command);
String str = "";
netshWriter.println(command);
fi.skip("\\s*");
str = fi.nextLine();
System.out.println(str);
}
It executes both commands.
Related
I'm having trouble with ProcessBuilder not running a command on the server.
Early in my project I use Runtime.exec() just to retrieve output from a program which works fine:
private List<SatelliteCode> getSatelliteCodes() {
List<SatelliteCode> codes = new ArrayList<>();
Runtime runtime = Runtime.getRuntime();
String[] commands = { "w_scan", "-s?" };
Process process;
try {
process = runtime.exec(commands);
BufferedReader error = new BufferedReader(new InputStreamReader(process.getErrorStream()));
String s = error.readLine(); // discard first line
while ((s = error.readLine()) != null) {
s = s.trim();
int i = s.indexOf('\t'); // separated by a tab!?!?
codes.add(new SatelliteCode(s.substring(0, i), s.substring(i)));
}
} catch (IOException e) {
e.printStackTrace();
}
return codes;
}
Running this in the terminal works fine and I get all the output I need:
w_scan -fs -cGB -sS19E2 > channels.conf
However, the server needs to grab the ongoing output from the 'process.getErrorStream()' to display in the web interface. What is actually happening is the ProcessBuilder is failing and returning an exit code of 1.
The function that initialises the ProcessBuilder and to start the scan running is [EDIT 1]:
private static StringBuilder scan_error_output = null;
#Override
public boolean startSatelliteScan(String user, String country_code, String satellite_code) {
UserAccountPermissions perm = validateUserEdit(user);
if (perm == null) return false;
Shared.writeUserLog(user, Shared.getTimeStamp() +
": DVB satellite scan started " +
country_code + " - " + satellite_code +
System.lineSeparator() + System.lineSeparator());
scan_error_output = new StringBuilder();
new ScanThread(country_code, satellite_code).start();
// write out country code and satellite code to prefs file
BufferedWriter bw = null;
try {
bw = new BufferedWriter(new FileWriter(satellite_last_scan_codes));
bw.write(country_code); bw.newLine();
bw.write(satellite_code); bw.newLine();
bw.close();
} catch (IOException e) {
e.printStackTrace();
}
return true;
}
That will then run two other threads on the server, one that will run the scan itself and wait for it to finish so that it can get the final scan data. And the other which constantly updates the output from the std error stream which is then polled at intervals from the clients browser. This is much like showing the ongoing output from the terminal.
The scan thread (which fails to start the process) [EDIT 1]:
private static class ScanThread extends Thread {
private String cc, sc;
public ScanThread(String country_code, String satellite_code) {
cc = country_code;
sc = satellite_code;
}
public void run() {
ProcessBuilder pb = new ProcessBuilder("/usr/bin/w_scan",
"-fs", "-c" + cc, "-s" + sc);
pb.redirectOutput(new File(satellite_scan_file));
Process process;
try {
System.out.println("Scan thread started");
process = pb.start();
IOScanErrorOutputHandler error_output_handler = new IOScanErrorOutputHandler(process.getErrorStream());
error_output_handler.start();
int result = process.waitFor();
System.out.println(cc + " - " + sc + " - " +
"Process.waitFor() result " + result);
} catch (IOException e) {
System.out.println(e.getMessage());
e.printStackTrace();
} catch (InterruptedException e) {
System.out.println(e.getMessage());
e.printStackTrace();
}
System.out.println("Scan thread finished");
}
}
The error output stream thread which captures the output which obviously doesn't start due to the scan thread failing:
private static class IOScanErrorOutputHandler extends Thread {
private InputStream inputStream;
IOScanErrorOutputHandler(InputStream inputStream) {
this.inputStream = inputStream;
}
public void run() {
Scanner br = null;
try {
System.out.println("Scan thread Error IO capture running");
br = new Scanner(new InputStreamReader(inputStream));
String line = null;
while (br.hasNextLine()) {
line = br.nextLine();
scan_error_output.append(line + System.getProperty("line.separator"));
}
} finally {
br.close();
}
System.out.println("Scan thread Error IO capture finished");
scan_error_output = null;
}
}
And the server function which returns the std error output progress:
#Override
public String pollScanResult(String user) {
if (validateUserEdit(user) == null) return null;
StringBuilder sb = scan_error_output; // grab instance
if (sb == null) return null;
return sb.toString();
}
As mentioned above, Runtime.exec() works fine, but the ProcessBuilder is failing.
NB: I'm on Linux Mint 18.1, using Apache Tomcat 8 as the server, linux default JDK 8 and GWT 2.7 [Correction from 2.8] in Eclipse Neon.
Can anyone see what I am doing wrong?
Many thanks in advance...
[EDIT 1]
Whilst developing this on another machine, Linux Mint 17.2, JDK 8 and Apache Tomcat 7, for DVB-T, this method worked fine and polling for the scan output showed up in the client's browser.
The ProcessBuilder.start still returns 1 and an empty file is created for the output scan file.
[EDIT 2]
It appears that the reason the ProcessBuilder is failing is because the user 'tomcat8' doesn't have permissions to run 'w_scan'. 'w_scan' works from the terminal, but not from the tomcat server. Somehow I've got to fix that now.
[SOLUTIONS]
After being put in the right direction by VGR for getting the error stream from the ProcessBuilder, I started digging further and found I was getting:
main:3909: FATAL: failed to open '/dev/dvb/adapter0/frontend0': 13 Permission denied
Apache tomcat 8 didn't have permission to access the DVB-S frontend to run a scan. This was fixed in two ways:
1 - 03catalina.policy I added the extra permissions (whether they made a difference I do not know).
grant codeBase "file:/dev/dvb/-" {
permission java.io.FilePermission "file:/dev/dvb/-", "read, write";
permission java.security.AllPermission;
};
2 - The dvb frontends belong to the 'video' group. So I needed to add the user tomcat8 to that group.
usermod -a -G video tomcat8
All works for now...
You are not doing the same thing with ProcessBuilder that you’re doing with Runtime.exec, so I don't know why you think ProcessBuilder is the problem.
You have a few problems with how you’re writing the command’s output to a file.
First, the presence of ">", satellite_scan_temp_file in your ProcessBuilder command is incorrect. Output redirection is not part of any command; it is handled by a shell. But when you run with Runtime.exec or ProcessBuilder, you are not running in a shell, you are executing the process directly. Neither w_scan nor any other command considers > a special character.
The correct way to redirect to a file is with the redirectOutput method:
ProcessBuilder pb = new ProcessBuilder(
"/usr/bin/w_scan", "-fs", "-s" + satellite_code, "-c" + country_code);
pb.redirectOutput(new File(satellite_scan_temp_file));
Second, your ScanThread code is ignoring the (currently incorrect) redirect, and is attempting to read the command’s output. But there is no output, because you are redirecting it all to a file.
Once you are properly redirecting output to a file, you can remove your BufferedReader and BufferedWriter loops completely.
Finally, it is worth noting that the error output you captured probably told you that > is not a valid argument to the w_scan process.
I'm writing a wrapper program in java that's just supposed to pass arguments to other processes by writing to their standard in streams, and reading the response from their standard out streams. However, when the String I try to pass in is too large, PrintWriter.print simply blocks. No error, just freezes. Is there a good workaround for this?
Relevant code
public class Wrapper {
PrintWriter writer;
public Wrapper(String command){
start(command);
}
public void call(String args){
writer.println(args); // Blocks here
writer.flush();
//Other code
}
public void start(String command) {
try {
ProcessBuilder pb = new ProcessBuilder(command.split(" "));
pb.redirectErrorStream(true);
process = pb.start();
// STDIN of the process.
writer = new PrintWriter(new OutputStreamWriter(process.getOutputStream(), "UTF-8"));
} catch (Exception e) {
e.printStackTrace();
System.out.println("Process ended catastrophically.");
}
}
}
If I try using
writer.print(args);
writer.print("\n");
it can handle a larger string before freezing, but still ultimately locks up.
Is there maybe a buffered stream way to fix this? Does print block on the processes stream having enough space or something?
Update
In response to some answers and comments, I've included more information.
Operating System is Windows 7
BufferedWriter slows the run time, but didn't stop it from blocking eventually.
Strings could get very long, as large as 100,000 characters
The Process input is consumed, but by line i.e Scanner.nextLine();
Test code
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeoutException;
import ProcessRunner.Wrapper;
public class test {
public static void main(String[] args){
System.out.println("Building...");
Wrapper w = new Wrapper("java echo");
System.out.println("Calling...");
String market = "aaaaaa";
for(int i = 0; i < 1000; i++){
try {
System.out.println(w.call(market, 1000));
} catch (InterruptedException | ExecutionException
| TimeoutException e) {
System.out.println("Timed out");
}
market = market + market;
System.out.println("Size = " + market.length());
}
System.out.println("Stopping...");
try {
w.stop();
} catch (IOException e) {
e.printStackTrace();
System.out.println("Stop failed :(");
}
}
}
Test Process:
You have to first compile this file, and make sure the .class is in the same folder as the test .class file
import java.util.Scanner;
public class echo {
public static void main(String[] args){
while(true){
Scanner stdIn = new Scanner(System.in);
System.out.println(stdIn.nextLine());
}
}
}
I suspect that what is happening here is that the external process is writing to its standard output. Since your Java code doesn't read it, it eventually fills the external process's standard out (or err) pipe. That blocks the external process, which means that it can read from its input pipe .... and your Java process freezes.
If this is the problem, then using a buffered writer won't fix it. You either need to read the external processes output or redirect it to a file (e.g. "/dev/null" on Linux)
Writing to any pipe or socket by any means in java.io blocks if the peer is slower reading than you are writing.
Nothing you can do about it.
Sorry for this odd-sounding title...
I have the following situation: I want my Java program to interact with an external console. In order to "send" the individual commands to that console, I need to simulate what would be an "enter key pressed" on a normal console. To clarify what I want, imagine mysql had no other API and I would need to interact via console. Although this is not my actual problem, it is close enough.
I have the following code:
String command = "/usr/local/mysql/bin/mysql";
Process child = Runtime.getRuntime().exec(command);
StreamGobbler gobbler = new StreamGobbler(child.getInputStream());
gobbler.start();
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(child.getOutputStream()));
out.write("help");
// here enter key needs to be pressed
out.flush();
// out.close();
If the call to out.close() is executed, everything is fine. But of course, this way I can only send a single command, which is not what I want. But if out.close() is omitted, the other program never executes the command. My guess is that it still waits for the command to "finish", which on a normal console would be done by pressing enter. out.write(System.getProperty("line.separator")); and out.newLine(); (which are the same) do not solve the problem, neither does out.write("\r\n"); and out.write((char) 26); (EOF).
Of course, it might be, that I am doing it completely wrong (i.e., wrong approach). Then I would appreciate a pointer into the right direction...
Any help on this highly appreciated.
The following code works fine on both Windows 7 using Java 1.6.0_23 and on Ubuntu 8.04 using Java 1.6.0_22:
public class Laj {
private static class ReadingThread extends Thread {
private final InputStream inputStream;
private final String name;
public ReadingThread(InputStream inputStream, String name) {
this.inputStream = inputStream;
this.name = name;
}
public void run() {
try {
BufferedReader in = new BufferedReader(
new InputStreamReader(inputStream));
for (String s = in.readLine(); s != null; s = in.readLine()) {
System.console().writer().println(name + ": " + s);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
public static void main(String[] args) throws Exception {
String command = "psql -U archadm arch";
final Process child = Runtime.getRuntime().exec(command);
new ReadingThread(child.getInputStream(), "out").start();
new ReadingThread(child.getErrorStream(), "err").start();
BufferedWriter out = new BufferedWriter(
new OutputStreamWriter(child.getOutputStream()));
out.write("\\h");
out.newLine();
out.flush();
out.write("\\q");
out.newLine();
out.flush();
}
}
newLine() is the same as writing the platform line separator. As one would expect, it prints help preceded with "out: ", then exits. If I don't send "\q", it doesn't exit (obviously) but still prints help. Using "\r\n" or "\r" instead of the platform line separator doesn't look like a good idea to me, because such command-line utilities will usually detect that they don't get input from the terminal and assume it is in the native text format (think "psql < script.sql"). Good software should properly detect and accept all reasonable line endings though.
What about out.write((char) 13)? See this Wikipedia article. I don't have enough code to test this for you.
You also might want to try looking at this API
http://download.oracle.com/javase/6/docs/api/java/io/Console.html
From my experience, I've never tried doing anything more than running one process from the Process API. It seems like you want to enter multiple commands I think this API might let you do that.
EDIT: Found a tutorial on it to help you further.
http://download.oracle.com/javase/tutorial/essential/io/cl.html
Hope this helps,
I have some issues regarding ProcessBuilder.
The program is basically a simple wrapper invoking a command line script.
When running the script on its own via the terminal, the memory consumption stays below 2G.
When running the script via the java wrapper, the memory consumption explodes and even 8G is quickly filled up, resulting in out-of-memory errors.
The code to launch the process is simply:
public static int execute(String command) throws IOException
{
System.out.println("Executing: " + command);
ProcessBuilder pb = new ProcessBuilder(command.split(" +"));
Process p = pb.start();
// display any output in stderr or stdout
StreamConsumer stderr = new StreamConsumer(p.getErrorStream(), "stderr");
StreamConsumer stdout = new StreamConsumer(p.getInputStream(), "stdout");
new Thread(stderr).start();
new Thread(stdout).start();
try {
return p.waitFor();
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
The StreamConsumer class is simply a class which consumes the stdout/stderr streams and display them on the console.
...the question is: why on earth does the memory consumption explode?
Regards,
Arnaud
Edit:
Whether I use ProcessBuilder or
Runtime.getRuntime.exec(...), the
result is the same.
The memory bursts tend to appear during unix 'sort' invoked by the
shell script called:
sort big-text-file > big-text-file.sorted
Edit 2 on request of Jim Garrison:
Ok, here is the StreamConsumer class which I omitted because it is rather simple:
class StreamConsumer implements Runnable
{
InputStream stream;
String descr;
StreamConsumer(InputStream stream, String descr) {
this.stream = stream;
this.descr = descr;
}
#Override
public void run()
{
String line;
BufferedReader brCleanUp =
new BufferedReader (new InputStreamReader (stream));
try {
while ((line = brCleanUp.readLine ()) != null)
System.out.println ("[" + descr + "] " + line);
brCleanUp.close();
} catch (IOException e) {
// TODO: handle exception
}
}
}
if you change your command like this :
sort -o big-text-file.sorted big-text-file
is it always the same ?
Maybe its because those StreamConsumer threads are not daemons so they don't die and get garbage collected when your processes return? You could try:
//...
final StreamConsumer stderr = new StreamConsumer(p.getErrorStream(), "stderr");
final StreamConsumer stdout = new StreamConsumer(p.getInputStream(), "stdout");
final Thread stderrThread = new Thread(stderr);
final Thread stdoutThread = new Thread(stdout);
stderrThread.setDaemon(true);
stdoutThread.setDaemon(true);
stderrThread.start();
stdoutThread.start();
//...
Is this behavior happening for single invocation or after doing this many times?
im trying to come up with a design for a wrapper for use when invoking command line utilities in java. the trouble with runtime.exec() is that you need to keep reading from the process' out and err streams or it hangs when it fills its buffers. this has led me to the following design:
public class CommandLineInterface {
private final Thread stdOutThread;
private final Thread stdErrThread;
private final OutputStreamWriter stdin;
private final History history;
public CommandLineInterface(String command) throws IOException {
this.history = new History();
this.history.addEntry(new HistoryEntry(EntryTypeEnum.INPUT, command));
Process process = Runtime.getRuntime().exec(command);
stdin = new OutputStreamWriter(process.getOutputStream());
stdOutThread = new Thread(new Leech(process.getInputStream(), history, EntryTypeEnum.OUTPUT));
stdOutThread.setDaemon(true);
stdOutThread.start();
stdErrThread = new Thread(new Leech(process.getErrorStream(), history, EntryTypeEnum.ERROR));
stdErrThread.setDaemon(true);
stdErrThread.start();
}
public void write(String input) throws IOException {
this.history.addEntry(new HistoryEntry(EntryTypeEnum.INPUT, input));
stdin.write(input);
stdin.write("\n");
stdin.flush();
}
}
And
public class Leech implements Runnable{
private final InputStream stream;
private final History history;
private final EntryTypeEnum type;
private volatile boolean alive = true;
public Leech(InputStream stream, History history, EntryTypeEnum type) {
this.stream = stream;
this.history = history;
this.type = type;
}
public void run() {
BufferedReader reader = new BufferedReader(new InputStreamReader(stream));
String line;
try {
while(alive) {
line = reader.readLine();
if (line==null) break;
history.addEntry(new HistoryEntry(type, line));
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
my issue is with the Leech class (used to "leech" the process' out and err streams and feed them into history - which acts like a log file) - on the one hand reading whole lines is nice and easy (and what im currently doing), but it means i miss the last line (usually the prompt line). i only see the prompt line when executing the next command (because there's no line break until that point).
on the other hand, if i read characters myself, how can i tell when the process is "done" ? (either complete or waiting for input)
has anyone tried something like waiting 100 millis since the last output from the process and declaring it "done" ?
any better ideas on how i can implement a nice wrapper around things like runtime.exec("cmd.exe") ?
Use PlexusUtils it is used by Apache Maven 2 to execute all external processes.
I was looking for the same thing myself, and I found a Java port of Expect, called ExpectJ. I haven't tried it yet, but it looks promising
I would read the input in with the stream and then write it into a ByteArrayOutputStream. The byte array will continue to grow until there are no longer any available bytes to read. At this point you will flush the data to history by converting the byte array into a String and splitting it on the platform line.separator. You can then iterate over the lines to add history entries. The ByteArrayOutputStream is then reset and the while loop blocks until there is more data or the end of stream is reached (probably because the process is done).
public void run() {
ByteArrayOutputStream out = new ByteArrayOutputStream();
int bite;
try {
while((bite = stream.read()) != -1) {
out.write(bite);
if (stream.available() == 0) {
String string = new String(out.toByteArray());
for (String line : string.split(
System.getProperty("line.separator"))) {
history.addEntry(new HistoryEntry(type, line));
}
out.reset();
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
This will make sure you pick up that last line of input and it solves your problem of knowing when the stream is ended.