I've written a java-program that starts sub-processes(through the ProcessBuilder) and treats their I/O (Process.getInputStream(),Process.getErrorStream(),Process.getOutputStream()) streams in different threads.
My java-program works as a server and I don't want extra output to the console from sub-processes and
in the IDE's console it works fine, but if I try to run the program java -jar ... from the windows or linux console, the sub-process writes output to my program and also to the console.
P.S. I execute Python script files from java, for example my python script script prints a test string "Hello World" 1000 times.
My program treats it fine, but I see it's output also in the console.
Is there some kind of "console" that I need to detach from the process somehow?
UPDATE:
Here I get a ProcessBuilder for running the python(I pass the directory of the script as a parameter):
private ProcessBuilder getBuilder(File directory) {
return new ProcessBuilder("python", "-u").directory(directory);
}
Here I start a Process from the builder and return it to further treatment(
I pass the directory of the script and args to the script):
public Process execute(Path dir, String... args) {
if (!Files.exists(dir) || !Files.isDirectory(dir))
throw new IllegalArgumentException("Provided path " + dir + " is not a directory or doesn't exist");
ProcessBuilder builder = getBuilder(dir.toFile());
if (args.length > 0) {
builder.command().addAll(Arrays.asList(args));
}
try {
return builder.start();
} catch (IOException io) {
io.printStackTrace();
return null;
}
}
Here is a test dumb script:
i=0;
while True:
if i%1000000==0:
print ('Hello, world!')
i+=1
I don't know if there a sence to post my stream handlers because they look like standard gobblers that was in other topics but they read from process out/error to the queue and get input from another queue and java program treats queues and has no output to console except logger messages.
But it seems the sub-process itself writes to the console, but maybe I'm wrong.
I didn't find out how to do it programmaticaly.
The only way that helps me is to redirect the output to /dev/null when I start the jar file.
It's quite appropriate, because I also get the output by network to the client.
Related
I have a Linux (Ubuntu 14.04.5 LTS) system command (an executable JAR file) that runs successfully if launched manually from the command line but hangs indefinitely when launched from within a Java application (v1.8.0_191).
I didn't write the executable JAR (it's called Tabula, an open-source PDF parser, but I don't think what the system command does is actually relevant to my problem). The executable JAR file takes the path to a PDF file as one of it's arguments, and it outputs JSON data to standard out. The average completion time of the command is usually just a few seconds.
For the majority of the PDF files that I process, there are no problems, but for a small bunch of PDFs the application that I've written hangs. I've discovered that the problem is the executable JAR command is not returning, so my application is waiting indefinitely. If I run ps -elf | grep java I can see the command is still running. Occasionally, the process will consume a very small amount of memory and CPU, but I have no idea what it's doing.
However, if I run that exact same executable JAR command myself on the command line, then it completes and returns the expected output in a few seconds. On the face it you would think that because the issue is not random and is confined to specific PDF input files, then it must be the files that the executable JAR does not like. If that was true, then why does the command always complete successfully if I run it manually from the command line.
I've extracted the relevant code into something smaller and more testable:
public static void main(String[] args) throws Exception {
if (args.length != 1) {
throw new Exception("Expected: folio ID as argument");
}
String folioID = args[0];
String[] command = new String[] {
"java", "-jar", Config.tabulaJARFile,
"--stream",
"--format", "JSON",
"--pages", "all",
String.format("%s/%s.pdf", Config.TABULA_INPUT_DIR, folioID)
};
System.out.println("Executing ==> " + String.join(" ", command));
Process process = Runtime.getRuntime().exec(command);
System.out.println("Waiting...");
int exitCode = process.waitFor();
if (exitCode != 0) {
throw new Exception("Invalid exit code ["+exitCode+"]");
}
System.out.println("Retrieving input stream...");
InputStream output = process.getInputStream();
String jsonData = IOUtils.toString(output, "UTF-8");
System.out.println(jsonData);
}
Example output of a run would be:
Executing ==> java -jar /home/ubuntu/tabula/tabula-1.0.3-jar-with-dependencies.jar --stream --format JSON --pages all /home/ubuntu/tabula/input/CK12345.pdf
Waiting...
...and that will hang indefinitely, but if I run that exact same command myself on the command line it will complete in a few seconds. I've also tried creating the command with a ProcessBuilder but the result is the same. Why might this be happening?
It seems that if I redirect the output of the process using a ProcessBuilder e.g. processBuilder.redirectOutput(Redirect.INHERIT) then the command completes successfully, but it prints the output to the console instead of letting me capture it in the application. I've managed a workaround by redirecting the output of the command to a file instead, then reading the contents of the file.
I have a Java application that calls a tcsh script which in turn calls a perl script in the same directory. If I run this script from the command by typing "runPerlScript.sh", it works completely fine, and generates several output files as it should. However, if I call the script from Java, using the code below:
String[] runCmd = {"/bin/tcsh","-c","/filepath/runPerlScript.sh"};
Process run = Runtime.getRuntime().exec(runCmd);
BufferedReader reader = new BufferedReader(new InputStreamReader(run.getInputStream()));
String line = "";
line = reader.readLine();
System.out.println("\nStarting while.");
while((line)!=null){
System.out.println("Output from script: "+line);
line=reader.readLine();
}
reader.close();
System.out.println("Finished running perl script.");
it prints out the echo statements from my shell script to my console (I'm using NetBeans), but generates only 4 output files (when normally it generates near 50). It seems as if the process is quitting to early, because after these 4 files are generated, an echo statement in my shell script that says "Finished running runPerlScript.sh" prints out to my console. I've tried several different ways to run this script, including ProcessBuilder, but none seem to generate the output files. The code I have above was in fact the only way I was able to generate ANY output, because ProcessBuilder just resulted in hangups. Does anyone know how I can continuously make the script run?
From the Runtime.exec() javadoc:
"Executes the specified string command in a separate process."
Assuming you want to wait for the process to end, you will need to wait for the process to terminate in your main java thread. The best way to do this would be by monitoring the Process returned by ProcessBuilder.start() and wait with Process.waitFor().
I am trying to execute an executable file and a perl script from within a java program. I have found many topics similar to this but most of them refer to windows. I know java is platform independent and it should work anyways but it doesn't. The solution I have tried already is the one based on the java Runtime and it's exec method. It works just fine on windows but since I'm porting my program on linux I need to adapt it. As I said I need to execute an executable file that I have compiled and was written in c++ which it looks like it's working but it finishes executing with an exit value of 1. I have no idea what it means but on windows it exits with 0 and that's how it should be on linux as well (?!?!). The pearl script on the other hand does not start at all. I use the command "perl script.pl" and it exits with a value of 255. Needless to say, it doesn't do what it's supposed to.
Does anybody know another way to execute these files? Or maybe where I am wrong with my implementation?
here's the code if you want to take a look at it:
This is the one for the perl script
public static void main(String[] args){
System.out.println("Starting");
try{
String[] cmd = {"perl", "cloc-1.53.pl"};
Process pr = Runtime.getRuntime().exec(cmd);
BufferedReader input = new BufferedReader(new InputStreamReader(pr.getInputStream()));
String line=null;
while((line=input.readLine()) != null) {
System.out.println(line);
}
int exitVal = pr.waitFor();
System.out.println("Exit code: " + exitVal);
} catch (Throwable t){
t.printStackTrace();
}
}
For the compiled file I change this:
String[] cmd = {"perl", "cloc-1.53.pl"};
with:
String cmd = "./UCC";
There should be no differece in starting processes on windows and linux.
Good article http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html
Its for the old way but gives good insight.
Article converting to the new way:
From Runtime.exec() to ProcessBuilder
From my java project I want to run an external .bat file in another thread. For this purpose I use the following method:
private void posAppRunner(final String path[], final Class targetClass) {
new Thread(new Runnable() {
public void run() {
try {
String line;
Process p = Runtime.getRuntime().exec(path);
BufferedReader input = new BufferedReader(new InputStreamReader(p.getInputStream()));
while ((line = input.readLine()) != null) {
System.out.println(line);
}
input.close();
} catch (IOException e) {
LogFactory.getLog(targetClass).warn("Error when starting a PosApplication: " + e.getMessage());
}
}
}).start();
I run the following .bat file:
call chdir %~dp0
start java <_some_arguments>
So when I do it locally from IntelliJ IDEA it works correct - a cmd process appears, after that a java process appears and after that the cmd process disappears.
But when I run my java project with this method through ANT under TeamCity windows service, only cmd process appears and nothing happens after. Java process that must be started from the bat file doesn't appear. It looks like I don't read the process output but I do!
Could you expain me, how to overcome this situation?
I believe that the problem is in current working directory. I am not so familiar with bat files and do not remember by heart what does %~dp0 mean. Anyway as the first attempt try to modify your batch file to contain the hard coded path. I believe that this will work.
In this case decide what is better for you: discover the path in java code and then pass it to batch file, generate batch file on the fly, so that it contains all parameters hard coded or debug the script. for example you can remove the start java <_some_arguments> and put
echo %~dp0 > c:\temp\batlog.log
this will print the parameter to log file. Now run it as service and see what does the log file contain. Probably you will see the problem immediately.
I am using Java on Windows XP and want to be able to send commands to another program such as telnet.
I do not want to simply execute another program. I want to execute it, and then send it a sequence of commands once it's running.
Here's my code of what I want to do, but it does not work:
(If you uncomment and change the command to "cmd" it works as expected. Please help.)
This is a simplified example. In production there will be many more commands sent, so please don't suggest calling "telnet localhost".
try
{
Runtime rt = Runtime.getRuntime();
String command = "telnet";
//command = "cmd";
Process pr = rt.exec(command);
BufferedReader processOutput = new BufferedReader(new InputStreamReader(pr.getInputStream()));
BufferedWriter processInput = new BufferedWriter(new OutputStreamWriter(pr.getOutputStream()));
String commandToSend = "open localhost\n";
//commandToSend = "dir\n" + "exit\n";
processInput.write(commandToSend);
processInput.flush();
int lineCounter = 0;
while(true)
{
String line = processOutput.readLine();
if(line == null) break;
System.out.println(++lineCounter + ": " + line);
}
processInput.close();
processOutput.close();
pr.waitFor();
}
catch(Exception x)
{
x.printStackTrace();
}
That looks OK, as it won't be producing that much output, but you should really read and write in separate threads so it doesn't fill up the buffer and block waiting you to read before you reach the next step.
So if it's reaching the point where you flush the command you send to it, find out whether the Windows telnet client supports receiving commands from standard input rather than a console by piping the text you're sending to its standard input to it in a command prompt.
For example, echo dir c:\ | cmd causes cmd to run, list the c: drive contents and exit, much the same behaviour as if you typed dir c:\ into the console. But echo open localhost | telnet causes telnet to clear the screen then exit, rather than behaving the same way as if you typed it into the console. As telnet needs to mask user input for passwords, it's quite likely that it's using the console API rather than reading from standard input. It's help doesn't list any command arguments to tell it to read from standard input, so maybe you need to use a telnet implementation which is better suited to scripting.
It's not directly an answer to your question, but...
Instead of using Runtime.exec() you should use a ProcessBuilder and redirect stderr to stdout (ProcessBuilder.redirectErrorStream(true)). Otherwise your process could block if it writes something to stderr (Windows doesn't like it when the output of a process isn't read).
If you want to control a telnet session programatically from Java, you might be able to use this Java telnet library... you can do the same things (open connections, send username/password, send commands and receive results) but without actually spawning a separate process.
You may take a look at the Telnet Ant task you can call it directly in your code with out having to use a build.xml file.
You can also take a look at the source code and see how they do it.