I'm trying to run a system command to extract a tar.bz2 file to a specified directory. This is the code:
ProcessBuilder myProc = new ProcessBuilder("tar", "-xjf", "/path/to/MyTarFile.tar.bz2");
myProc.directory(new File("/directory/i/want/results/in/"));
myProc.start();
System.out.println(myProc.command());
It runs without error, however the file is deleted and not extracted anywhere.
Any help would be greatly appreciated.
I know Runtime.exec() has a really nasty feature where if you don't manually drain STDOUT/STDERR, it effectively appears to hang. I would hope that ProcessBuilder corrected that deficiency, but this page includes this tidbit:
A word of caution about the examples in this tip. It is possible that the examples will deadlock if the subprocess generates enough output to overflow the system. A more robust solution requires draining the process stdout and stderr in separate threads.
So, make sure you're handling Process.getInputStream() and Process.getErrorStream(), as I recommended in the comments; it could solve the problem outright!
Change the myProc.start(); line to
Process p = myProc.start();
p.waitFor();
That will make sure your program doesn't exit until the tar is finished.
Run this to see errors. Perhaps one of your paths is incorrect.
import java.io.File;
import java.io.InputStream;
public class Untar {
public static void main(String[] args) throws Exception {
ProcessBuilder myProc = new ProcessBuilder("tar", "-xjf", "foo.tar.bz2");
myProc.directory(new File("newdir"));
Process p = myProc.start();
InputStream is = p.getErrorStream();
int c;
while( (c = is.read()) != -1 ){
System.out.print((char)c);
}
p.waitFor();
System.out.println(myProc.command());
}
}
Just tried this code. It works.
Check your paths.
Related
I'm running a jar file from another jar like here somebody answers but waiting for the process.
Process proc = Runtime.getRuntime().exec("java -jar A.jar" + stringParams);
try {
proc.waitFor();
} catch (InterruptedException e) {
e.printStackTrace();
}
InputStream in = proc.getInputStream();
InputStream err = proc.getErrorStream();
My problem comes when i have no feedback on the status of the program that is called, but i don't want my program continues beyond those lines. I would need the standard and error outputs but the results are shown when the execution is over. Is there any way of executing and getting those streams while the jar is still running?
Buffered/unbuffered
It seems like an issue with buffered output.
Executed process (in this case java -jar <path>) buffers output and writes it only when it's done (in big chunks, we don't like that!)
So one way to go is execute process through unbuffering (very hacky tools):
unbuffered <command>
stdbuf -i0 -o0 -e0 <command>
use terminal emulation
Hacking
stdbuf is part of GNU tools.
https://www.gnu.org/software/coreutils/manual/html_node/stdbuf-invocation.html
unbuffered is part of expect package.
https://wiki.tcl.tk/3548
The key thing is making the program thinking that it's in interactive mode (like you are launching it from console).
The first two options are very hacky and do not work in all cases (idk if java command works with them?)
Emulation
The third option is most promising.
We launch a program (terminal emulator) that emulates interactive terminal making program think it's working in real active session!
Pty4j
You might use pty4j too:
From there: https://github.com/traff/pty4j
// The command to run in a PTY...
String[] cmd = { "java", "-jar", "path_to_jar" };
// The initial environment to pass to the PTY child process...
String[] env = { "TERM=xterm" };
PtyProcess pty = PtyProcess.exec(cmd, env);
OutputStream os = pty.getOutputStream();
InputStream is = pty.getInputStream();
// ... work with the streams ...
// wait until the PTY child process terminates...
int result = pty.waitFor();
// free up resources.
pty.close();
Zt-exec
Maybe it's worth trying zt-exec?
I have no idea how it executes commands.
But it may be it (I didn't test that).
Using https://github.com/zeroturnaround/zt-exec
new ProcessExecutor().command("java", "-jar path_to_jar")
.redirectOutput(new LogOutputStream() {
#Override
protected void processLine(String line) {
...
}
})
.execute();
That should work, but I didn't test that.
In general, there are no ways to nicely resolve your problem.
Depending on what platforms you want to target consider using unbuffered, stdbuff or the (slowest) terminal emulation...
Please let me know if that helps and good luck! :)
This has got to be one of the strangest things I have ever observed. Consider the following Java program:
import java.io.IOException;
public class StrangeError {
public static void main(String[] args) {
try {
Process process = new ProcessBuilder(
"cmd",
"/c",
"\"C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\vcvarsall.bat\" amd64 && set"
).start();
process.waitFor();
} catch (IOException|InterruptedException e) {
System.out.println(e.getMessage());
}
}
}
I compiled it with javac StrangeError.java, copied it to my server running Windows Server 2012 R2, and ran it with java StrangeError.
Here's where things start to get weird. The program hangs, waiting for the process it spawned to finish. This is not the expected behavior, since the vcvarsall.bat script should complete immediately as well as set.
So I started playing around and discovered the following:
Removing set causes vcvarsall.bat to terminate
Removing vcvarsall.bat causes set to terminate
Replacing && with || causes everything to terminate correctly
Copying vcvarsall.bat to a location on the desktop and changing the path causes everything to terminate correctly
A nearly equivalent program works fine in Go using the same commands
I get this output if I run everything in WinDbg and interrupt the process after it hangs
This does not appear to be reproducible with vcvarsall.bat from MSVC2013 but is also reproducible with MSVC2015 on Windows 10
What on earth is wrong with the original program? If I copy and paste the entire command (cmd /c "C:\...) into Start->Run, it immediately launches cmd and terminates, as expected.
Is this a bug with Java? Is this a bug with Windows?
Is this a bug with Java? Is this a bug with Windows?
It's a bug in your code. :-)
By default, a child process created using a ProcessBuilder object has output redirected to a pipe, the parent end of which can be obtained using Process.getInputStream() and which is not automatically drained if your code does not make use of it.
Since your code simply calls .waitFor without making any provision to drain the pipe, it will deadlock as soon as the pipe's buffer overflows. I believe the default buffer size is 4,096 bytes. On my machine, the output of the command you're running is 5,192 bytes, but this will vary depending on the original contents of the environment block. (From the sounds of it, the output length in your environment is borderline, only just above the limit, so that even small changes like changing the version of VS make a difference.)
One of the many possible solutions, depending on what you're actually trying to do, is to tell Java not to pipe the child's output:
import java.io.IOException;
public class StrangeError {
public static void main(String[] args) {
try {
ProcessBuilder processb = new ProcessBuilder(
"cmd",
"/c",
"\"C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\vcvarsall.bat\" amd64 && set"
);
processb.redirectOutput(ProcessBuilder.Redirect.INHERIT);
Process process = processb.start();
process.waitFor();
} catch (IOException|InterruptedException e) {
System.out.println(e.getMessage());
}
}
}
Not possible to read standard input and output error inside the same ProcessBuilder.
So you need to create two ProcessBuilder
Process process1 = new ProcessBuilder(
"cmd",
"/c",
"\"C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\vcvarsall.bat\",
"amd64");
Process process2 = new ProcessBuilder(
"cmd",
"/c",
"set");
process1.start();
if (process1.waitFor() == 0) {
process2.start();
if (process2.waitFor() == 0) {
// Successfull execution
}
}
And one thing : I don't think it is a good practice to do shell/batch launches with Java (or another language). Maybe you should use a script (shell, batch, python, perl...) to control standard input/output streams.
In essence I have this program:
from sympy.solvers import solve
from sympy import Symbol
x = Symbol('x')
print solve(x**2 - 1, x)
And I call this from Java using this code:
public static BufferedReader runFile(Class<?> c, String py, List<String> args) {
String cmd = c.getResource(py).getPath();
cmd=cmd.split(py)[0];
cmd=cmd.substring(0,cmd.length()-1);
args.add(0, py);
args.add(0, "python");
final ProcessBuilder pb = new ProcessBuilder(args);
new ProcessBuilder();
pb.directory(new File(cmd));
pb.redirectError();
try {
System.out.println(pb.directory().getPath());
for(String s:pb.command()){
System.out.println(s);
}
Process p=pb.start();
return new BufferedReader(new InputStreamReader(p.getErrorStream()));
}
catch (final IOException e) {
throw new RuntimeException(e);
}
}
When I run the Python program from a terminal everything works as intended, with nothing in the error stream, and it prints [-1,1]. But if I run it from the program, I get this in the error stream:
Traceback (most recent call last):
File "solve.py", line 1, in <module>
from sympy.solvers import solve
ImportError: No module named sympy.solvers
Since specifying the full path of Python fixes your problem, you most likely have multiple installations of Python on your system. Rather than PYTHONPATH being different, I suspect it is actually PATH that is different. As a result, your command line uses the Python interpreter you intend, while Java uses another one.
To determine where this alternate install is, which -a python may be useful, but if not, examine PATH from inside your Java code and see if you can find Python in one of those directories.
Regardless, if you really need to specify the full Python path in Java, you should make this a configuration option. It will probably be different on different machines. Storing it in a file seems most prudent.
Your PYTHONPATH (or less likely your working directory) is different when running from your Java context.
You can
import sys
print sys.path
which may help you to ensure the path is the same for both.
Telling us more about how your environment is set up will help to get more specific answers.
eg. Maybe the Java is running via a web server or something?
Here are couple of ways to fix the path problem:
Make sure the directory containing sympy is in your PYTHONPATH environment variable
If you're really desperate, append the correct directory to sys.path
import sys
sys.append("/some/dir/with/sympy")
from sympy.solvers import solve
...
I am having an issue with some process wrapping, and it's only occurring in Windows XP. This code works perfectly in Windows 7. I'm really stumped as to why the streams are empty in XP. I've also tried using the String[] version of Process.Exec() and it made no difference.
I am using the following class to read from the process' STDOUT and STDERR (an instance for each stream):
import java.util.*;
import java.io.*;
public class ThreadedStreamReader extends Thread{
InputStream in;
Queue messageQueue;
public ThreadedStreamReader(InputStream s, Queue q)
{
in = s;
messageQueue = q;
}
public void run()
{
try
{
BufferedReader r = new BufferedReader(new InputStreamReader(in));
String line = null;
while((line = r.readLine()) != null)
{
synchronized(messageQueue)
{
messageQueue.add(line);
}
}
}catch(Exception e)
{
System.err.println("Bad things happened while reading from a stream");
}
}
}
And I use it here:
Process p = Runtime.getRuntime().exec("test.exe");
Queue<String> q = new LinkedList<String>();
ThreadedStreamReader stdout = new ThreadedStreamReader(p.getInputStream(), q);
ThreadedStreamReader stderr = new ThreadedStreamReader(p.getErrorStream(), q);
stdout.start();
stderr.start();
while(true)
{
while(q.size() > 0)
{
System.out.println(q.remove());
}
}
Anyone have any ideas? Thanks!
Edit: Added synchronization
Edit: Just as an update, the parent stream readers are blocked on their read operation. If I kill the child processes, with the task manager, they read in the null from the closing of the stream.
You need to use a threadsafe data structure; I don't think LinkedList is threadsafe.
One mistake that strikes me is that LinkedList is not synchronized, but you're trying to write to it in 2 threads.
Another thing to keep in mind is Process.getInputStream() returns the stdout stream of the process, so you should rename the variable currently called stdin to stdout to prevent confusion.
There are known bugs in pre-Vista Windows operating systems where loading DLLs can cause a hang in IO.
e.g. see http://weblogs.java.net/blog/kohsuke/archive/2009/09/28/reading-stdin-may-cause-your-jvm-hang and https://connect.microsoft.com/VisualStudio/feedback/details/94701/loadlibrary-deadlocks-with-a-pipe-read
I'm not sure if this is what you are running in to, but it may be related.
Also, I vaguely recall some issues in getting a valid stdin and stdout from non-console windows applications. If your call to 'test.jar' is using 'javaw' rather than 'java', then this could be the cause of your problem, too.
Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.
I want to exceute a simple command which works from the shell but doesn't work from Java.
This is the command I want to execute, which works fine:
soffice -headless "-accept=socket,host=localhost,port=8100;urp;"
This is the code I am excecuting from Java trying to run this command:
String[] commands = new String[] {"soffice","-headless","\"-accept=socket,host=localhost,port=8100;urp;\""};
Process process = Runtime.getRuntime().exec(commands)
int code = process.waitFor();
if(code == 0)
System.out.println("Commands executed successfully");
When I run this program I get "Commands executed successfully".
However the process is not running when the program finishes.
Is it possible that the JVM kills the program after it has run?
Why doesn't this work?
I'm not sure if I'm not mistaken, but as far as I see you're generating the commands but never passing them to the "execute" method... you're executing "".
Try using Runtime.getRuntime().exec(commands) =)
I would like to say how I solved this.
I created a sh script that basically run the command of soffice for me.
Then from Java I just run the script, and it works fine, like this:
public void startSOfficeService() throws InterruptedException, IOException {
//First we need to check if the soffice process is running
String commands = "pgrep soffice";
Process process = Runtime.getRuntime().exec(commands);
//Need to wait for this command to execute
int code = process.waitFor();
//If we get anything back from readLine, then we know the process is running
BufferedReader in = new BufferedReader(new InputStreamReader(process.getInputStream()));
if (in.readLine() == null) {
//Nothing back, then we should execute the process
process = Runtime.getRuntime().exec("/etc/init.d/soffice.sh");
code = process.waitFor();
log.debug("soffice script started");
} else {
log.debug("soffice script is already running");
}
in.close();
}
I also kill the soffice process by calling this method:
public void killSOfficeProcess() throws IOException {
if (System.getProperty("os.name").matches(("(?i).*Linux.*"))) {
Runtime.getRuntime().exec("pkill soffice");
}
}
Note that this only works in Linux.
I believe you aren't handling quoting correctly. The original sh command line includes double quotes to prevent the shell interpreting the semicolons. The shell strips them off before the soffice process sees them.
In your Java code the shell will never see the arguments, so the extra double quotes (escaped with backslashes) are not needed - and they are probably confusing soffice.
Here's the code with the extra quotes stripped out (and a semicolon thrown in)
String[] commands = new String[] {"soffice","-headless","-accept=socket,host=localhost,port=8100;urp;"};
Process process = Runtime.getRuntime().exec(commands);
int code = process.waitFor();
if(code == 0)
System.out.println("Commands executed successfully");
(Disclaimer: I don't know Java, and I haven't tested this!)
"/Applications/OpenOffice.org\ 2.4.app/Contents/MacOS/soffice.bin -headless -nofirststartwizard -accept='socket,host=localhost,port=8100;urp;StartOffice.Service'"
or simply escaping the quotes will work as well. We feed a command like this to an ant script that ultimately ends up in an exec call like you have above. I would also recommend restarting the process every 500 or so conversions because OOO does not properly free memory (depending on what version you are running).