I have problems sending data from java to a (linux)-subprocess created by ProcessBuilder/Process.
A shell-only based basic example would look like as follows and works fine.
echo "hello world" | cat - >/tmp/receive.dat
Now, I want to substituted the echo "hello world" by a java program which should internally create a new process (the cat - >/tmp/receive.dat) and then send data to it.
I tried the following, but the file /tmp/receive.dat remains untouched:
String[] cmdArray = { "/bin/cat", "-", ">/tmp/receive.dat" };
ProcessBuilder builder = new ProcessBuilder (cmdArray);
builder.directory (new File ("/tmp"));
Process p = builder.start ();
OutputStream pos = p.getOutputStream ();
byte [] bytes = "hello world".getBytes ();
pos.write (bytes);
pos.close ();
p.waitFor ();
The same happens under Windows, of course with an adapted cmdArray:
cmd /c type con >c:\tmp\receive.dat
Printing directly to system.out from java is no alternative as many subprocesses should be called within the livecycle of the java program.
thx for any help!
Tombo
The issue here is that /bin/cat does not actually write to files, it merely writes to standard out. The output redirection >/tmp/receive.dat is actually performed by the shell, but you are bypassing the shell by invoking cat in this manner.
If what you are trying to achieve is merely an OutputStream that writes to a file, then doing that via standard java I/O (e.g., FileOutputStream and friends) is what you want. It is also cross-platform (i.e., Windows-friendly), unlike anything that depends on the shell.
Regarding the comment about not being able to merely write to standard out from java because of subprocesses - any subprocess you invoke can inherit standard out from their parent process (the java process - look at ProcessBuilder.Redirect.INHERIT). So even if you are invoking subprocesses from java, redirecting all the output to a file should still work in the same way as your initial example (where the java program replaces the echo command).
You probably want ProcessBuilder#redirectOutput(File), as the > functionality is not of cat, rather what is calling cat (in our sense, the process builder).
Related
I have java cde that jars class files together:
List<String> args = new ArrayList<String>();
String path = FileSystemUtils.JavaBin() + "\\jar.exe";
args.add(path);
args.add("-cfv");
args.add(jarName);
args.addAll(FileSystemUtils.getAllFiles(directory, ".class"));
ProcessBuilder pb = new ProcessBuilder(args);
File wd = new File(directory);
pb.directory(wd);
Process p = pb.start();
//Waiting for process to exit
p.waitFor();
int res = p.exitValue();
Tis code works great.
However, on some computers - not on all of them, when there are 7+ files, the p.waitFor(); never return, even though the jar was created.
Looking at the task manager, jar.exe really did not terminate.... what can be the cause?
running the same command manually from the command line exits immediately.
This seems very weird. Does someone have any hint?
Found the solution myself.
Apparently if you use ProcessBuilder.start in Java to start an external process you have to consume its stdout/stderr, otherwise the external process hangs.
This is beacuase OS creates a pipe.
All Unix like OSs and Windows behave the same in this regard: A pipe with a 4K is created between parent and child. When that pipe is full (because one side isn't reading), the writing process blocks.
It seems that when there are 7+ files the jar.exe consume 4K, nad then stuck.
Javadoc of process:
By default, the created subprocess does not have its own terminal or
console. All its standard I/O (i.e. stdin, stdout, stderr) operations
will be redirected to the parent process, where they can be accessed
via the streams obtained using the methods getOutputStream(),
getInputStream(), and getErrorStream(). The parent process uses these
streams to feed input to and get output from the subprocess. Because
some native platforms only provide limited buffer size for standard
input and output streams, failure to promptly write the input stream
or read the output stream of the subprocess may cause the subprocess
to block, or even deadlock.
I have a Java application that calls a tcsh script which in turn calls a perl script in the same directory. If I run this script from the command by typing "runPerlScript.sh", it works completely fine, and generates several output files as it should. However, if I call the script from Java, using the code below:
String[] runCmd = {"/bin/tcsh","-c","/filepath/runPerlScript.sh"};
Process run = Runtime.getRuntime().exec(runCmd);
BufferedReader reader = new BufferedReader(new InputStreamReader(run.getInputStream()));
String line = "";
line = reader.readLine();
System.out.println("\nStarting while.");
while((line)!=null){
System.out.println("Output from script: "+line);
line=reader.readLine();
}
reader.close();
System.out.println("Finished running perl script.");
it prints out the echo statements from my shell script to my console (I'm using NetBeans), but generates only 4 output files (when normally it generates near 50). It seems as if the process is quitting to early, because after these 4 files are generated, an echo statement in my shell script that says "Finished running runPerlScript.sh" prints out to my console. I've tried several different ways to run this script, including ProcessBuilder, but none seem to generate the output files. The code I have above was in fact the only way I was able to generate ANY output, because ProcessBuilder just resulted in hangups. Does anyone know how I can continuously make the script run?
From the Runtime.exec() javadoc:
"Executes the specified string command in a separate process."
Assuming you want to wait for the process to end, you will need to wait for the process to terminate in your main java thread. The best way to do this would be by monitoring the Process returned by ProcessBuilder.start() and wait with Process.waitFor().
I'm trying to launch an external program from my java swing app using this:
Process proc = Runtime.getRuntime().exec(cmd);
But the external program never actually gets launched until I close out of my java app...everytime.
It waits to launch only after I have closed out.
the external program I am trying to run is an exe that takes arguments so:
cmd = "externalProgram.exe -v --fullscreen --nowing";
What could possibly be wrong here.
Funny enough it works as expected if i try something simple like:
Process proc = Runtime.getRuntime().exec("notepad.exe");
You may need to read from the process's standard output, or close the standard input, before it will proceed. For reading the output, the problem is that the buffer can get full, blocking the program; for closing the input, the problem is that some programs will try to read data from there if it's available, waiting to do so. One or both of these tricks is very likely to straighten things out for you.
You may also read the error output stream to check it the program is actually being unsuccessfully executed
String cmd = "svn.exe";
Process proc = Runtime.getRuntime().exec(cmd);
BufferedReader reader = new BufferedReader(new InputStreamReader(proc.getErrorStream()));
String line = null;
while((line=reader.readLine())!=null){
System.out.println(line);
}
reader.close();
My console shows
Type 'svn help' for usage.
Which evidently shows the program was executed by Java.
I am using Java on Windows XP and want to be able to send commands to another program such as telnet.
I do not want to simply execute another program. I want to execute it, and then send it a sequence of commands once it's running.
Here's my code of what I want to do, but it does not work:
(If you uncomment and change the command to "cmd" it works as expected. Please help.)
This is a simplified example. In production there will be many more commands sent, so please don't suggest calling "telnet localhost".
try
{
Runtime rt = Runtime.getRuntime();
String command = "telnet";
//command = "cmd";
Process pr = rt.exec(command);
BufferedReader processOutput = new BufferedReader(new InputStreamReader(pr.getInputStream()));
BufferedWriter processInput = new BufferedWriter(new OutputStreamWriter(pr.getOutputStream()));
String commandToSend = "open localhost\n";
//commandToSend = "dir\n" + "exit\n";
processInput.write(commandToSend);
processInput.flush();
int lineCounter = 0;
while(true)
{
String line = processOutput.readLine();
if(line == null) break;
System.out.println(++lineCounter + ": " + line);
}
processInput.close();
processOutput.close();
pr.waitFor();
}
catch(Exception x)
{
x.printStackTrace();
}
That looks OK, as it won't be producing that much output, but you should really read and write in separate threads so it doesn't fill up the buffer and block waiting you to read before you reach the next step.
So if it's reaching the point where you flush the command you send to it, find out whether the Windows telnet client supports receiving commands from standard input rather than a console by piping the text you're sending to its standard input to it in a command prompt.
For example, echo dir c:\ | cmd causes cmd to run, list the c: drive contents and exit, much the same behaviour as if you typed dir c:\ into the console. But echo open localhost | telnet causes telnet to clear the screen then exit, rather than behaving the same way as if you typed it into the console. As telnet needs to mask user input for passwords, it's quite likely that it's using the console API rather than reading from standard input. It's help doesn't list any command arguments to tell it to read from standard input, so maybe you need to use a telnet implementation which is better suited to scripting.
It's not directly an answer to your question, but...
Instead of using Runtime.exec() you should use a ProcessBuilder and redirect stderr to stdout (ProcessBuilder.redirectErrorStream(true)). Otherwise your process could block if it writes something to stderr (Windows doesn't like it when the output of a process isn't read).
If you want to control a telnet session programatically from Java, you might be able to use this Java telnet library... you can do the same things (open connections, send username/password, send commands and receive results) but without actually spawning a separate process.
You may take a look at the Telnet Ant task you can call it directly in your code with out having to use a build.xml file.
You can also take a look at the source code and see how they do it.
I have a python application which I cant edit its a black box from my point of view. The python application knows how to process text and return processed text.
I have another application written in Java which knows how to collect non processed texts.
Current state, the python app works in batch mode every x minutes.
I want to make the python
processing part of the process: Java app collects text and request the python app to process and return processed text as part of a flow.
What do you think is the simplest solution for this?
Thanks,
Rod
I don't know nothing about Jython and the like. I guess it's the best solution if you can execute two programs without executing a new process each time the Java app needs to transform text. Anyway a simple proof of concept is to execute a separate process from the Java App to make it work. Next you can enhance the execution with all that tools.
Executing a separate process from Java
String[] envprops = new String[] {"PROP1=VAL1", "PROP2=VAL2" };
Process pythonProc = Runtime.getRuntime().exec(
"the command to execute the python app",
envprops,
new File("/workingdirectory"));
// get an outputstream to write into the standard input of python
OutputStream toPython = pythonProc.getOutputStream();
// get an inputstream to read from the standard output of python
InputStream fromPython = pythonProc.getInputStream();
// send something
toPython.write(.....);
// receive something
fromPython.read(....);
Important: chars are NOT bytes
A lot of people understimate this.
Be careful with char to byte conversions (remember Writers/Readers are for chars, Input/OutputStreams are for bytes, encoding is necesary for convertir one to another, you can use OuputStreamWriter to convert string to bytes and send, InputStreamReader to convert bytes to chars and read them).
Look into Jython - you can run Python programs directly from Java code, and interact seamlessly back and forth.
Use ProcessBuilder to execute your Python code as a filter:
import java.io.BufferedReader;
import java.io.InputStreamReader;
public class PBTest {
public static void main(String[] args) {
ProcessBuilder pb = new ProcessBuilder("python", "-c", "print 42");
pb.redirectErrorStream(true);
try {
Process p = pb.start();
String s;
BufferedReader stdout = new BufferedReader (
new InputStreamReader(p.getInputStream()));
while ((s = stdout.readLine()) != null) {
System.out.println(s);
}
System.out.println("Exit value: " + p.waitFor());
p.getInputStream().close();
p.getOutputStream().close();
p.getErrorStream().close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
Expose one of the two as a service of some kind, web service maybe. Another option is to port the python code to Jython
One possible solution is jpype. This allows you to launch a JVM from Python and pass data back and forth between them.
Another solution may be to write the Python program as a filter (reading data from stdin and writing result to stdout) then run it as a pipe. However I do not know how well Java supports this - according to the Sun docs their concept of pipes only supports communication between threads on the same JVM.
An option is making the python application work as a server, listens for request via sockets (TCP).