Develop Reference

Command executed from terminal, failed from Java using ProcessBuilder

I'm trying to execute a command in my terminal. The problem is, when I execute the command in terminal, it succeed, but when I run the command from java, the command is executed but, I got an error message showing me that some python module is missing.
try{
String[] list = { "python3", "script.py" };
ProcessBuilder pb = new ProcessBuilder(list);
pb.directory(
new File("/home/script"));
System.out.println("" + pb.directory());
Process process = pb.start();
InputStream str = process.getErrorStream();
InputStreamReader isr = new InputStreamReader(str);
BufferedReader br = new BufferedReader(isr);
String line;
System.out.printf("Output of running %s is:", Arrays.toString(args));
while ((line = br.readLine()) != null) {
System.out.println(line);}
BufferedReader in = new BufferedReader(new InputStreamReader(process.getInputStream()));
String ret = in.readLine();
System.out.println("value is : "+ret);
process.waitFor();
process.destroy();
}catch (Exception ex) {
ex.printStackTrace();
}
The error message:
/home/script
Output of running [] is:Traceback (most recent call last):
File "scraper.py", line 8, in <module>
from selenium import webdriver
ModuleNotFoundError: No module named 'selenium'
value is : null
PS: When I execute the command directly from terminal, everything works good, I don't get the missing module error.

Similar to Java, python allows to import other stuff. That message tells you that your python script wants to use the module selenium, but can't find it.
Most likely you have some special ENV var setup when running commands manually in a shell/console. So check your .bashrc or .initrc or whatever defines your ENV variables. On a unix system, typing the command env might show you all settings, too. Simply check if the env var PYTHONPATH is setup.
As that call works from the command line, then for sure, the module is installed on your system. Your only problem is that python can't find it when you invoke that script through the Java ProcessBuilder!
One solution might be that you "manually" adjust the PYTHONPATH from within your script. Thus: figure the correct setup for PYTHONPATH, then update your script to "do the right thing".
For further details, see the python documentation!

Java processbuilder and using environment variables

What I want to do is I want to run a process, however because this process itself relies on environment variables, directly calling it causes error within the process. For those who are wondering what this is, it's rake tool. For this reason I thought maybe it's better to use bash and using it through bash would eliminate the issue. However that doesn't seem to be the case.
Here is my code:
public static void runPB(String directory) throws IOException {
ProcessBuilder processBuilder = new ProcessBuilder(
"/bin/bash");
processBuilder.directory(new File(directory));
Process process = processBuilder.start();
OutputStreamWriter osw = new OutputStreamWriter(process.getOutputStream());
osw.write("rake routes");
osw.close();
printStream(process.getErrorStream());
printStream(process.getInputStream());
}
public static void printStream(InputStream is) throws IOException {
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
}
I know it is environment related issue because the error that I am getting is described here cannot load such file -- bundler/setup (LoadError)
Further I checked processBuilder.environment() returns less environment variables than entering env. I went ahead and changed the osw.write() line and tried echo $GEM_HOME there, which doesn't print anything and if I do this on my OSs bash then I get the path, I also tried other common things like echo $SHELL and it prints the shell location in both Java code and in bash.
So my questions are:
1) Why is my operating system's environment variables are different than the ProcessBuilder.environment() method?
2) Does Process class consider using environment variables that were given out by ProcessBuilder.environment()? If so then how can we add the missing ones from the operating system's level?

1) The varaibles you see in your java process are those inheritd from the process you started the java process from. I.e. If you launch it from a shell it should have the same variables as the shell had. You need to investigate which variables are actually set before launching your Java application and why the ones you expect are not set in that context.
To answer part 2, yes, the process will be launched with the environment in ProcessBuilder.environment(). You can simply add things to the map returned by ProcessBuilder.environment(), that will extend the runtime environment:
ProcessBuilder pb = new ProcessBuilder("foo");
pb.environment().put("MY_VAR", "foobar");

Imagemagick can't open file when executed from Java Servlet

I'm trying to convert files from png's to pdf using imagemagick and Java. I've got everything working to a place when I'm executing imagemagick command to actually merge multiple png's into one pdf. The command itself looks properly, and it works fine when executed in the terminal but my application gives me error showing that imgck can't open the file (even though it exists and I've set permissions to the folder to 777 :
line: convert: unable to open image `"/Users/mk/Documents/workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/sch-java/print-1357784001005.png"': No such file or directory # error/blob.c/OpenBlob/2642.
This is my command :
/opt/local/bin/convert "/Users/mk/Documents/workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/sch-java/print-1357784001005.png" "/Users/mk/Documents/workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/sch-java/print-1357784001219.png" "/Users/mk/Documents/workspace/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/sch-java/complete-exportedPanel2013-01-1003:13:17.212.pdf"
And my Java code :
String filesString = "";
for (String s : pdfs){
filesString += "\""+ s + "\" ";
}
Process imgkProcess = null;
BufferedReader br = null;
File f1 = new File(pdfs[0]);
//returns true
System.out.println("OE: "+f1.exists());
String cmd = imgkPath+"convert "+ filesString+ " \""+outputPath+outName+"\"";
try {
imgkProcess = Runtime.getRuntime().exec(cmd);
InputStream stderr = imgkProcess.getErrorStream();
InputStreamReader isr = new InputStreamReader(stderr);
br = new BufferedReader(isr);
} catch (IOException e1) {
msg = e1.getMessage();
}
imgkProcess.waitFor();
while( (line=br.readLine() ) != null){
System.out.println("line: "+line);
}
The whole code is executed from a java servlet controller after getting request from a form. Any ideas what can cause this ? I'm using latest imgck, jdk, and osx 10.7 .

A few things:
When spawning anything but really trivial processes, it's usually better to use ProcessBuilder than Runtime.exec() - it gives you much better control
Even with ProcessBuilder, it often works better to write a shell script that does what you need. Then spawn a process to run the script. You get a lot more control in shell script than you do in ProcessBuilder
Remember that a spawned process is not a shell. It can't, for instance, evaluate expressions, or expand shell variables. If you need that, then you must execute a shell (like sh or bash). Better yet, write a shell script as described above
If all you need to do is to execute some ImageMagick commands, it would probably be easier to jmagick, a Java interface to ImageMagick - see http://www.jmagick.org/
Actually, since the you're assembling images into a PDF, the iText library - http://itextpdf.com is probably the best tool for the job, as it is native Java code, does not require spawning a native process, and will therefore be much more portable.

Solved it by adding all arguments to an arrayList and then casting it to String array.
ArrayList<String> cmd = new ArrayList<String>();
cmd.add(imgkPath+"convert");
for (int i=0, l=pdfs.length; i<l; i++){
cmd.add(pdfs[i]);
}
cmd.add(outputPath+outName);
imgkProcess = Runtime.getRuntime().exec(cmd.toArray(new String[cmd.size()]));

How do I execute an executable file from within a java file on linux?

I am trying to execute an executable file and a perl script from within a java program. I have found many topics similar to this but most of them refer to windows. I know java is platform independent and it should work anyways but it doesn't. The solution I have tried already is the one based on the java Runtime and it's exec method. It works just fine on windows but since I'm porting my program on linux I need to adapt it. As I said I need to execute an executable file that I have compiled and was written in c++ which it looks like it's working but it finishes executing with an exit value of 1. I have no idea what it means but on windows it exits with 0 and that's how it should be on linux as well (?!?!). The pearl script on the other hand does not start at all. I use the command "perl script.pl" and it exits with a value of 255. Needless to say, it doesn't do what it's supposed to.
Does anybody know another way to execute these files? Or maybe where I am wrong with my implementation?
here's the code if you want to take a look at it:
This is the one for the perl script
public static void main(String[] args){
System.out.println("Starting");
try{
String[] cmd = {"perl", "cloc-1.53.pl"};
Process pr = Runtime.getRuntime().exec(cmd);
BufferedReader input = new BufferedReader(new InputStreamReader(pr.getInputStream()));
String line=null;
while((line=input.readLine()) != null) {
System.out.println(line);
}
int exitVal = pr.waitFor();
System.out.println("Exit code: " + exitVal);
} catch (Throwable t){
t.printStackTrace();
}
}
For the compiled file I change this:
String[] cmd = {"perl", "cloc-1.53.pl"};
with:
String cmd = "./UCC";

There should be no differece in starting processes on windows and linux.
Good article http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html
Its for the old way but gives good insight.
Article converting to the new way:
From Runtime.exec() to ProcessBuilder

Unix Script not working in Java Process Runtime.exec()

I am developing an application in Spring Web MVC where i need to execute some of the linux script..
I am using tomcat version 5.5 for running my project in linux..
My code is looking like this :
Process proc = runtime.exec("sudo cp /var/tmp/mailserverfiles/editinterface.txt /etc/sysconfig/network-scripts/editinterface.txt");
InputStream inputstream = proc.getInputStream();
InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
BufferedReader bufferedreader = new BufferedReader(inputstreamreader);
String line;
while ((line = bufferedreader.readLine()) != null) {
System.out.println("\nOUTPUT = " + line);
}
System.out.print("\nbefore execute6");
try {
if (proc.waitFor() != 0) {
System.err.println("\nexit value = " + proc.exitValue());
}
} catch (InterruptedException e) {
System.err.println("\nERROR = " + e);
}
Here i want to cp a particular file from one location to another using linux script..
But when i am executing this part, i am getting
exit value = 1
as a output.. I have also tried to put this script into .sh file and try to execute that shell script here from Java Code, but i am getting same result..
Can anybody tell me, what should be the reason for this ?
Thanks in advance..

I would guess that sudo is expecting an interactive terminal in order to ask for a password. Since there is no interactive terminal, it prints an error message to stderr and exits with an exit code of 1. You are not reading the error stream, so you won't see any message that it might print.
You will definitely want to read the error stream in any case. Doing so now will help you diagnose what is going wrong at this point.

I assume the user that Tomcat is running under has unrestricted access to sudo? And that it's not being prompted for a password?

It is possible that your search path is weird and that "cp" and "sudo" are not found when you try to execute the command.
Here are some things you could try to track down your problem(s):
Try running the "cp" command without "sudo".
Try giving the full pathname of the command(s). This will avoid search path problems.
By default "sudo" logs failed commands using syslog(3). See if you can find traces in the corresponding logfiles.

Assuming you can run your command from a command line, logged in as the tomcat user - try
ProcessBuilder pb = new ProcessBuilder("/usr/bin/sudo", "cp",
"/var/tmp/mailserverfiles/editinterface.txt",
"/etc/sysconfig/network-scripts/editinterface.txt");
pb.redirectErrorStream(true);
Process proc = pb.start();
... rest of code as before
if things still fail, start debugging. strace should be helpful. e.g. run this shell script
from your java application, and figure out where things fail in the /tmp/trace.txt file:
#!/bin/sh
strace -f sudo cp /var/tmp/mailserverfiles/editinterface.txt /etc/sysconfig/network-scripts/editinterface.txt >/tmp/trace.txt 2>&1

Whilst not directly answering your question, the following will help. You need to read stdout and stderr (to capture all process output), and do this concurrently to prevent blocking of the spawned process. See this answer for more info.