I'm writing a wrapper program in java that's just supposed to pass arguments to other processes by writing to their standard in streams, and reading the response from their standard out streams. However, when the String I try to pass in is too large, PrintWriter.print simply blocks. No error, just freezes. Is there a good workaround for this?
Relevant code
public class Wrapper {
PrintWriter writer;
public Wrapper(String command){
start(command);
}
public void call(String args){
writer.println(args); // Blocks here
writer.flush();
//Other code
}
public void start(String command) {
try {
ProcessBuilder pb = new ProcessBuilder(command.split(" "));
pb.redirectErrorStream(true);
process = pb.start();
// STDIN of the process.
writer = new PrintWriter(new OutputStreamWriter(process.getOutputStream(), "UTF-8"));
} catch (Exception e) {
e.printStackTrace();
System.out.println("Process ended catastrophically.");
}
}
}
If I try using
writer.print(args);
writer.print("\n");
it can handle a larger string before freezing, but still ultimately locks up.
Is there maybe a buffered stream way to fix this? Does print block on the processes stream having enough space or something?
Update
In response to some answers and comments, I've included more information.
Operating System is Windows 7
BufferedWriter slows the run time, but didn't stop it from blocking eventually.
Strings could get very long, as large as 100,000 characters
The Process input is consumed, but by line i.e Scanner.nextLine();
Test code
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeoutException;
import ProcessRunner.Wrapper;
public class test {
public static void main(String[] args){
System.out.println("Building...");
Wrapper w = new Wrapper("java echo");
System.out.println("Calling...");
String market = "aaaaaa";
for(int i = 0; i < 1000; i++){
try {
System.out.println(w.call(market, 1000));
} catch (InterruptedException | ExecutionException
| TimeoutException e) {
System.out.println("Timed out");
}
market = market + market;
System.out.println("Size = " + market.length());
}
System.out.println("Stopping...");
try {
w.stop();
} catch (IOException e) {
e.printStackTrace();
System.out.println("Stop failed :(");
}
}
}
Test Process:
You have to first compile this file, and make sure the .class is in the same folder as the test .class file
import java.util.Scanner;
public class echo {
public static void main(String[] args){
while(true){
Scanner stdIn = new Scanner(System.in);
System.out.println(stdIn.nextLine());
}
}
}
I suspect that what is happening here is that the external process is writing to its standard output. Since your Java code doesn't read it, it eventually fills the external process's standard out (or err) pipe. That blocks the external process, which means that it can read from its input pipe .... and your Java process freezes.
If this is the problem, then using a buffered writer won't fix it. You either need to read the external processes output or redirect it to a file (e.g. "/dev/null" on Linux)
Writing to any pipe or socket by any means in java.io blocks if the peer is slower reading than you are writing.
Nothing you can do about it.
Related
I create class to execute CMD command continuously It is working fine for the first iteration in the below code, but the problem is the process is died after one iteration is done
class CommandLine{
Process Handle ;
OutputStreamWriter writer;
Scanner getCommand;
Socket socket;
public CommandLine(Socket socket) throws IOException {
this.socket = socket;
}
public void executeCommand() {
try {
getCommand = new Scanner(socket.getInputStream()).useDelimiter("\\A");
Handle = new ProcessBuilder("cmd.exe").redirectErrorStream(true).start();
while(getCommand.hasNextLine()) {
try(PrintWriter stdin = new PrintWriter(Handle.getOutputStream())) {
stdin.write(getCommand.nextLine()+System.lineSeparator());
stdin.flush();
}
if(Handle.getInputStream().read()>0) {
Scanner result = new Scanner(Handle.getInputStream()).useDelimiter("\\A");
while(result.hasNextLine()) {
System.out.print(result.nextLine()+"\n");
}
}
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
thx for response
You need to re-organise your code. The sub-process dies because you've got a try with resources block inside the loop:
try(PrintWriter stdin = new PrintWriter(Handle.getOutputStream())) {
stdin.write(getCommand.nextLine()+System.lineSeparator());
stdin.flush();
}
The above means STDIN of the sub-process ends after one line, and so does the CMD.EXE.
Also note that just moving PrintWriter stdin part outside the loop isn't enough. You won't be able to reliably supply the STDIN and read STDOUT in same loop as the STDOUT might be many lines of input and block the process when you write STDIN.
The fix is easy: follow #VGR suggestion and replace .redirectErrorStream(true) by either .redirectOutput(ProcessBuilder.Redirect.INHERIT) or .inheritIO()
which will mean that you don't need to read getInputStream(). Alternatively use a background thread for your code either for the write to STDIN or for read from getInputStream() / STDOUT.
In my Java application, I need to execute some scripts as subprocesses and monitor the output on stdout from Java so that I can react when necessary to some output.
I am using apache commons-exec to spawn the subprocess and redirect stdout of the executed script to an input stream.
The problem that I am having is that when reading from the stream, the Java process is blocked until the subprocess is finished execution. I cannot wait until the end of the subprocess to react to the output, but I need to read it asynchronously as it becomes available.
Below is my Java code:
public class SubProcessReact {
public static class LogOutputStreamImpl extends LogOutputStream {
#Override
protected void processLine(String line, int logLevel) {
System.out.println("R: " + line);
}
}
public static void main (String[] args) throws IOException, InterruptedException {
CommandLine cl = CommandLine.parse("python printNumbers.py");
DefaultExecutor e = new DefaultExecutor();
ExecuteStreamHandler sh = new PumpStreamHandler(new LogOutputStreamImpl());
e.setStreamHandler(sh);
Thread th = new Thread(() -> {
try {
e.execute(cl);
} catch (IOException e1) {
e1.printStackTrace();
}
});
th.start();
}
}
For this example, the subprocess is a python script which counts upwards with a one second delay between outputs so that I can verify that the Java code is responding as data comes in.
Python Code:
import time
for x in range(0,10):
print x
time.sleep(1)
I would expect LogOutputStreamImpl to print each line as it comes, but what is actually happening is that it reading the stream blocks until the subprocess is completed, and then all of the output is printed.
Is there something I could do to make this work as I intend?
Why use a third-party library to do something Java SE already does well? Personally, I prefer to depend on as few external libraries as possible, in order to make my programs easily portable and to reduce the points of failure:
ProcessBuilder builder = new ProcessBuilder("python", "printNumbers.py");
builder.inheritIO().redirectOutput(ProcessBuilder.Redirect.PIPE);
Process process = builder.start();
try (BufferedReader reader = new BufferedReader(
new InputStreamReader(process.getInputStream()))) {
reader.lines().forEach(line -> System.out.println("R: " + line));
}
process.waitFor();
I am trying to read three files using thread and then pass on the content to the writer class to write it to another file. The thread associated with the first file(which has line break in it) is returning back after every line break. Can anyone please tell me why is this happening. I will be pasting my code of the reader class.
package filehandling;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
public class FileReading extends Thread{
BufferedReader fis;
int count=0;
FileWriting fw;
String str1, str2;
String ssr;
public FileReading(String str) throws IOException
{
//getting filename
File f= new File(str);
String strin;
strin= f.getName();
System.out.println(".." + strin);
//splitting filename to get the initial name
String stra[]= new String[2];
stra= strin.split("\\.");
str1= stra[0];
str2= stra[1];
System.out.println("extension name :" + str2);
System.out.println("filename :" + str1);
//associating file to input stream
fis= new BufferedReader(new FileReader(f));
}
public void run()
{
try
{
while((ssr=fis.readLine())!=null)
{
//file contents
System.out.println(ssr);
//writer thread
fw= new FileWriting(str1,ssr);
fw.start();
//assigning thread time to read,else next thread comes in
join(1000);
}
}
catch(Exception e)
{
System.out.println("exception : " + e.getMessage() + e.getStackTrace());
}
}
}
There is no sense in starting multiple threads within the process of reading a single file. Reading from a file data stream has no opportunity for parallel execution. But you can read three independent files in parallel by using three threads using an ordinary loop within each thread.
There is another misconception; you seem to think that you have to assign time to the threads. That’s wrong, you don’t need to think about that and you can’t do it the way you tried. When you start three threads, each of them reading and writing a file, all of them will go to sleep when no data are available and proceed on new data. The operating system will assign them CPU time appropriately.
Since you don’t have given a write part I can’t give you a code example for your task, but here is a simple example of reading three files in parallel and getting their contents back as a String:
import java.io.BufferedReader;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.*;
public class ReadFile implements Callable<String>
{
public static void main(String[] args) throws InterruptedException
{
// enter the three file names here
String[] file={
"",
"",
""};
ExecutorService executorService = Executors.newFixedThreadPool(file.length);
List<Callable<String>> jobs=new ArrayList<>(file.length);
for(String f:file) jobs.add(new ReadFile(f));
List<Future<String>> all = executorService.invokeAll(jobs);
System.out.println("all files have been read in");
int i=0; for(Future<String> f:all) { i++;
System.out.println("file "+i+": ");
try
{
String contents = f.get();
System.out.println(contents);
} catch(ExecutionException ex)
{
ex.getCause().printStackTrace();
}
}
}
final String fileName;
public ReadFile(String file) {
fileName=file;
}
public String call() throws Exception {
String newLine=System.getProperty("line.separator");
StringBuilder sb=new StringBuilder();
try(BufferedReader r=new BufferedReader(new FileReader(fileName))) {
for(;;) {
String line=r.readLine();
if(line==null) break;
sb.append(line).append(newLine);
}
}
return sb.toString();
}
}
The line executorService.invokeAll will invoke the call methods of all provided ReadFile instances, each of them in another Thread. These threads read their files in a loop, becoming blocked whenever the I/O system has no new data for them, giving the other threads a chance to proceed. However, don’t expect three threads to run significantly faster the one thread processing the files one after another. The limiting factor is the I/O speed of your harddrive/SSD/etc. You have the most chances of getting more speed by multiple threads when all files lie on different devices.
Things will look different if the threads are not just reading or writing but also performing some computations.
In java, I am trying to read the output from a consoleprogram I wrote in C.
This program is continuously printing its current progress into stdout using printf().
If I run that program in a console, everything is fine, I am seeing the output.
I am now trying to run it from inside java, which also runs fine, the process is starting and calculating, but the whole output however will be read in huge blocks at once (There is no output for some seconds and then everything appears at once).
I assume there is some kind of a buffer in between that must be filled.
In order to draw a progressbar and work with other parameters the program is printing it is neccessary to read from the stdout fluidly and not everything at once.
I already read about this in Questions like Problem reading InputStream from Java Process (Runtime.getRuntime().exec() or ProcessBuilder), but this did not help me very much as I followed the tips in these questions.
Below is the code I am currently trying to use.
public static void main(String[] args)
{
ProcessBuilder builder = new ProcessBuilder("render.exe", "1920", "1080", "10000", "0", "0", "1", "6", "test.png");
try
{
final Process proc = builder.start();
final Thread io = new Thread()
{
#Override
public void run()
{
final InputStream read = proc.getInputStream();
int c;
try
{
while((c = read.read()) != -1)
System.out.print((char)c);
}
catch (IOException e)
{
e.printStackTrace();
}
}
};
io.start();
proc.waitFor();
}
catch (IOException | InterruptedException e)
{
e.printStackTrace();
}
}
The C program probably detects that its stdout is not connected to an interactive console and buffers its output; you cannot change this from Java.
Assuming you use stdio, to make the C program produce output more fluidly you can add fflush(stdout) in appropriate places, or you can disable buffering with a call to setvbuf:
setvbuf(stdout, NULL, _IONBF, 0);
I'm trying to create a Thread that keeps netsh windows command-line tool open so I can execute netsh commands without open it every single time.
The thing is, once I've created the Thread, just the first command call works... the subsequent calls seems to have no effect.
Here is my code:
public class NetshThread implements Runnable{
private static Process netshProcess = null;
private static BufferedInputStream netshInStream = null;
private static BufferedOutputStream netshOutStream = null;
public BufferedReader inPipe = null;
public void run(){
startNetsh();
}
public void startNetsh(){
try {
netshProcess = Runtime.getRuntime().exec("netsh");
netshInStream = new BufferedInputStream(netshProcess.getInputStream());
netshOutStream = new BufferedOutputStream(netshProcess.getOutputStream());
inPipe = new BufferedReader(new InputStreamReader(netshInStream));
} catch (IOException e) {
e.printStackTrace();
}
}
public void executeCommand(String command){
System.out.println("Executing: " + command);
try {
String str = "";
netshOutStream.write(command.getBytes());
netshOutStream.close();
while ((str = inPipe.readLine()) != null) {
System.out.println(str);
}
} catch (IOException e) {
e.printStackTrace();
}
}
public void closeNetsh(){
executeCommand("exit");
}
public static void main(String[] args){
NetshThread nthread = new NetshThread();
nthread.run();
String command = "int ip set address " +
"\"Local Area Connection 6\" static .69.69.69 255.255.255.0";
nthread.executeCommand(command);
command = "int ip set address " +
"\"Local Area Connection 6\" static 69.69.69.69 255.255.255.0";
nthread.executeCommand(command);
System.out.println("*** DONE ***");
}
}
Thank you!!! =)
Update 1:
Ok... I'm now using a PrintWriter instead... so I think I don't need to flush anything anymore, since the constructor is:
new PrintWriter(netshOutStream, true); (just like Mr. Shiny told me)...
Suppose I decide to break the while loop when the first output line is available... I doesn't work either... the next command wont be executed.... My code now looks like:
import java.io.*;
public class NetshThread implements Runnable{
private static Process netshProcess = null;
private static BufferedInputStream netshInStream = null;
private static BufferedOutputStream netshOutStream = null;
public BufferedReader inPipe = null;
private PrintWriter netshWriter = null;
public void run(){
startNetsh();
}
public void startNetsh(){
try {
netshProcess = Runtime.getRuntime().exec("netsh");
netshInStream = new BufferedInputStream(netshProcess.getInputStream());
netshOutStream = new BufferedOutputStream(netshProcess.getOutputStream());
netshWriter = new PrintWriter(netshOutStream, true);
inPipe = new BufferedReader(new InputStreamReader(netshInStream));
} catch (IOException e) {
e.printStackTrace();
}
}
public void executeCommand(String command){
System.out.println("Executing: " + command);
try {
String str = "";
netshWriter.println(command);
while ((str = inPipe.readLine()) != null) {
System.out.println(str);
break;
}
} catch (IOException e) {
e.printStackTrace();
}
}
public void closeNetsh(){
executeCommand("exit");
}
public static void main(String[] args){
NetshThread nthread = new NetshThread();
Thread xs = new Thread(nthread);
xs.run();
String command = "int ip set address " +
"\"Local Area Connection 6\" static .69.69.69 255.255.255.0";
nthread.executeCommand(command);
command = "int ip set address " +
"\"Local Area Connection 6\" static 69.69.69.69 255.255.255.0";
nthread.executeCommand(command);
System.out.println("*** DONE ***");
}
}
and the output I get:
Executing: int ip set address "Local
Area Connection 6" static .69.69.69
255.255.255.0 netsh>.69.69.69 is not an acceptable value for addr.
Executing: int ip set address "Local
Area Connection 6" static 69.69.69.69
Why the second command is not executed???
255.255.255.0
* DONE *
Update 2:
Everything seemed to work just fine until a teacher tried my app in a spanish-windows enviroment....
my code looks like this:
Scanner fi = new Scanner(netshProcess.getInputStream());
public void executeCommand(String command) {
System.out.println("Executing: " + command);
String str = "";
netshWriter.println(command);
fi.skip("\\s*");
str = fi.nextLine();
System.out.println(str);
}
and what i need is to somehow set the netshWriter encoding to the windows default.
Can anyone know who to do this?
You are closing the output stream.
You need to move the stream processing into separate threads. What's happening is that inPipe.readLine() is blocking waiting for netsh to return data. Apache has a package that deals with process handling. I'd look at using that instead of rolling your own (http://commons.apache.org/exec/)
This seems wrong in many ways.
First, why a Runnable object? This isn't ever passed to a Thread anywhere. The only thread you're creating isn't a java thread, it is an OS process created by exec().
Second, you need a way to know when netsh is done. Your loop that reads the output of netsh will just run forever because readLine will only return null when netsh closes its standard out (which is never, in your case). You need to look for some standard thing that netsh prints when it is done processing your request.
And as others mentioned, close is bad. Use a flush. And hope netsh uses a flush back to you...
I'd try:
PrintWriter netshWriter = new PrintWriter(netshOutputStream, true); // auto-flush writer
netshWriter.println(command);
No close()ing the stream, flush the stream automatically, and uses a writer to send character data rather than relying on the platforms "native character set".
You do definitely need to remove the close, else you'll never be able to execute another command. When you say "it won't work" once the close() call removed, do you mean no commands are processed?
Chances are that after you send the bytes for the command, you need to send some kind of confirmation key for the process to start, well, processing it. If you'd normally enter this from the keyboard it might be as simple as a carriage return, otherwise it might need to be a Ctrl-D or similar.
I'd try replacing the close() line with
netshOutStream.write('\n');
and see if that works. Depending on the software you might need to change the character(s) you send to signify the end of the command, but this general approach should see you through.
EDIT:
It would also be prudent to call
netshOutStream.flush();
after the above lines; without the flush there's no guarantee that your data will be written and in fact, since you're using a BufferedInputStream I'm 99% sure that nothing will be written until the stream is flushed. Hence why the code afterwards blocks, as you're waiting for a response while the process has not seen any input yet either and is waiting for you to send it some.
I've used scanner instead of BufferedReader, just because I like it. So this code works:
Scanner fi = new Scanner(netshProcess.getInputStream());
public void executeCommand(String command) {
System.out.println("Executing: " + command);
String str = "";
netshWriter.println(command);
fi.skip("\\s*");
str = fi.nextLine();
System.out.println(str);
}
It executes both commands.