Run python script through Java with files arguments [duplicate] - java

This question already has answers here:
Java Runtime.getRuntime(): getting output from executing a command line program
(12 answers)
Closed 2 years ago.
I have a standard Maven project and I want to run the meTypeset script. This script takes 3 args where the second one is a file and the third one is a folder created as output.
This is how the script runs in a cmd:
meTypeset.py docx <input> <output_folder> [options]
This is how I try to run it in Java:
public static void main(String args[]) {
String[] cmd = {
"python",
"resources\\pyscripts\\meTypeset.py",
"docx",
"resources\\exampledocs\\example_journal.docx",
"resources\\output"
};
try {
Runtime.getRuntime().exec(cmd);
} catch (IOException e) {
e.printStackTrace();
}
}
Nothing happens, no errors but no result also

Unlike python Java may need some help. Do I guess correctly you are running on Windows?
You invoke the Runtime.exec() method. The method returns a Process instance, and in it's documentation you can read
By default, the created process does not have its own terminal or
console. All its standard I/O (i.e. stdin, stdout, stderr) operations
will be redirected to the parent process, where they can be accessed
via the streams obtained using the methods getOutputStream(),
getInputStream(), and getErrorStream(). The parent process uses these
streams to feed input to and get output from the process. Because some
native platforms only provide limited buffer size for standard input
and output streams, failure to promptly write the input stream or read
the output stream of the process may cause the process to block, or
even deadlock.
So it is likely your process is started by the OS but gets blocked due to I/O restrictions. Get around that by reading the STDOUT and STDERR streams until your process finishes. One good programming model is visible at https://www.baeldung.com/run-shell-command-in-java

#Hiran Chaudhuri explained the error correctly. I am just posting how I solved it, thanks to # Sonnenhut comment.
Runtime rt = Runtime.getRuntime();
String[] commands = {
"python",
"src\\main\\resources/pyscripts/meTypeset.py",
"docx",
"src\\main\\resources/exampledocs/example_journal.docx",
"src\\main\\resources/output"
};
Process proc = rt.exec(commands);
BufferedReader stdInput = new BufferedReader(new
InputStreamReader(proc.getInputStream()));
BufferedReader stdError = new BufferedReader(new
InputStreamReader(proc.getErrorStream()));
// Read the output from the command
System.out.println("Here is the standard output of the command:\n");
String s = null;
while ((s = stdInput.readLine()) != null) {
System.out.println(s);
}
// Read any errors from the attempted command
System.out.println("Here is the standard error of the command (if any):\n");
while ((s = stdError.readLine()) != null) {
System.out.println(s);
}
}

Related

Trying to use xcopy within Java to copy a folder

I'm trying to execute a command line to copy a folder, but nothing happens . I tried the same command from the command line and it worked fine.
code
Runtime rt = Runtime.getRuntime();
String line;
try {
Process pr = rt.exec("xcopy //E //I notts nots2");
InputStreamReader mInputStreamReader = new InputStreamReader( pr.getInputStream());
BufferedReader input = new BufferedReader( mInputStreamReader );
while ( (line = input.readLine()) != null)
System.out.println(line);
} catch (IOException e) {
ted=ted+1;
}
1) pr.getInputStream() is not enough because it will not read the error output encountered during the process execution.
You should also read the error stream : pr.getErrorStream().
2) You should specify the working directory of the process otherwise the process inherits the working directory of the current process.
For example :
Process pr = rt.exec("xcopy //E //I notts nots2", null, new File("yourWorkingDirToRunTheProcess"));
Look at the answer to this question. It explains how to consume the standard output and standard error streams of the process.
You should also consider copying files using java API methods instead of running an external process. One reason being that your command (xcopy) won't work on anything but Windows. Another reason is that running an external process is much more error prone than using standard API methods.

shell process java synchronization

I want to run a shell script from a java program. This shell script invokes a system library which needs a big file as resource.
My java program calls this script for every word in a document. If I call this script again and again using Runtime.exec() the time taken is very high since the resource loading takes lot of time.
To overcome this I thought of writing the shell script as follows (to make it run continuously in background ):
count=0
while count -lt 10 ; do
read WORD
//execute command on this line
done
I need retrieve the output of the command in my java program and process it further.
How should I code the I/O operations for achieving this task?
I have tried writing words in to the process's output stream and reading back output from process's input stream. But this does not work and throws a broken pipe exception.
try {
parseResult = Runtime.getRuntime().exec(parseCommand);
parsingResultsReader = new BufferedReader(new InputStreamReader (parseResult.getInputStream()));
errorReader = new BufferedReader(new InputStreamReader (parseResult.getErrorStream()));
parseResultsWriter = new BufferedWriter(new OutputStreamWriter((parseResult.getOutputStream())));
} catch (IOException e) {
e.printStackTrace();
}
parseResultsWriter.write(word);
parseResultsWriter.flush();
while ((line = parsingResultsReader.readLine()) != null) {
// capture output in list here
}
Kindly help with this issue
//execute command on this line
Is this command a separate program? Then it will be launched for every word, so you'll get rid of only shell process which is lightweight anyway.
You have to learn how to run the heavyweight command for many words at once.

Why the command line utility takes very huge time to give output when invoke using java program?

I am using libdmtx which comes with a command line utility which reads the image files for ECC200 Data Matrix barcodes, reads their contents, and writes the decoded messages to standard output. I have used command line utility in my java program on linux platform. I am using ubuntu linux. I have installed the libdmtx on my linux machine. and when I invoke the command
dmtxread -n /home/admin/ab.tif
on linux terminal it gives the decoded value of barcode in image immediately i.e within 15 seconds.
but when I am going to invoke this same command for same file using my java program the program takes huge time i.e average 16 minutes for the same command and same file above.
Following is my java code which invokes the above command
public class Test {
public static void main(final String[] args) throws IOException, InterruptedException {
//Build command
List<String> commands = new ArrayList<String>();
commands.add("dmtxread");
commands.add("-n");
commands.add("/home/admin/ab.tif");
System.out.println(commands);
//Run macro on target
ProcessBuilder pb = new ProcessBuilder(commands);
pb.redirectErrorStream(true);
Process process = pb.start();
//Read output
StringBuilder out = new StringBuilder();
BufferedReader br = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line = null, previous = null;
while ((line = br.readLine()) != null){
System.out.println(line);
}
//Check result
if (process.waitFor() == 0)
System.out.println("Success!");
System.exit(0);
//Abnormal termination: Log command parameters and output and throw ExecutionException
System.err.println(commands);
System.err.println(out.toString());
System.exit(1);
}
}
I want to ask the experts that
Please can anyone explain me why java program takes such big time to invoke a simple command which will be get invoked withing 15 seconds if that command is directly run on command prompt.
Please can anyone tell me the solution to reduce this time.
I guess the program is taking this so much time because of JVMs internal thread which is invoking the process. Is my guess is right? If yes then how could I overcome to this problem.
Please guide me to solve this problem. Thanks You!

Java running bash command while feeding it input and getting output

I've got to write down some java code that runs an external process.
This process gets an input line from stdin and gives an output line on stdout.
The code I wrote is as follows
try
{
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec("./" + args[0]);
Process proc2 = rt.exec(c);
int exitVal = proc2.exitValue();
System.out.println("Process exitValue: " + exitVal);
}
catch (Throwable t){
}
Where args[0] is the external process and c is the input (String) I need to feed to the process. If it was running correctly, I would get an exitVal=0, instead I get nothing. Also, what I really want to print is the output of the external process (which would be "6" in the given example).
You must use getInputStream() and getOutputStream() on the process to communicate it, see the javadocs.
If you do not read all data from the process, it might hang and even deadlock. For a short introduction on common problems, see this blog post. At the very least, you should be reading all data from the process output.
You cannot "get nothing". Method exitValue() returns primitive type, therefore it even cannot be null. But this method can stuck however. This happens because your process is still running.
Is is possible that it is running because you did not "feed" it? In this case the only thing you have to do is to call proc2.getOutputStream() and write what you need to this stream. If you want to read the process' output use proc2.getInputStream() and read from the stream.
BTW, take a look on ProcessBuilder. It provides better, more "object oriented" API thant simple Runtime.exec().
You can feed the program input with '<' redirection:
{
Process p = Runtime.getRuntime().exec("wc < pom.xml");
p.waitFor();
System.out.println(p.exitValue());
BufferedReader bri = new BufferedReader
(new InputStreamReader(p.getInputStream()));
String line;
while ((line = bri.readLine()) != null) {
System.out.println(line);
}
bri.close();
}

Launching wkhtmltopdf from Runtime.getRuntime().exec(): never terminates?

I'm launching wkhtmltopdf from within my Java app (part of a Tomcat server, running in debug mode within Eclipse Helios on Win7 64-bit): I'd like to wait for it to complete, then Do More Stuff.
String cmd[] = {"wkhtmltopdf", htmlPathIn, pdfPathOut};
Process proc = Runtime.getRuntime().exec( cmd, null );
proc.waitFor();
But waitFor() never returns. I can still see the process in the Windows Task Manager (with the command line I passed to exec(): looks fine). AND IT WORKS. wkhtmltopdf produces the PDF I'd expect, right where I'd expect it. I can open it, rename it, whatever, even while the process is still running (before I manually terminate it).
From the command line, everything is fine:
c:\wrk>wkhtmltopdf C:\Temp\foo.html c:\wrk\foo.pdf
Loading pages (1/6)
Counting pages (2/6)
Resolving links (4/6)
Loading headers and footers (5/6)
Printing pages (6/6)
Done
The process exits just fine, and life goes on.
So what is it about runtime.exec() that's causing wkhtmltopdf to never terminate?
I could grab proc.getInputStream() and look for "Done", but that's... vile. I want something that is more general.
I've calling exec() with and without a working directory. I've tried with and without an empty "env" array. No joy.
Why is my process hanging, and what can I do to fix it?
PS: I've tried this with a couple other command line apps, and they both exhibit the same behavior.
Further exec woes.
I'm trying to read standard out & error, without success. From the command line, I know there's supposed to be something remarkably like my command line experience, but when I read the input stream returned by proc.getInputStream(), I immediately get an EOL (-1, I'm using inputStream.read()).
I checked the JavaDoc for Process, and found this
The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the [b]subprocess to block, and even deadlock[/b].
Emphasis added. So I tried that. The first 'read()' on the Standard Out inputStream blocked until I killed the process...
WITH WKHTMLTOPDF
With the generic command line ap & no params so it should "dump usage and terminate", it sucks out the appropriate std::out, then terminates.
Interesting!
JVM version issue? I'm using 1.6.0_23. The latest is... v24. I just checked the change log and don't see anything promising, but I'll try updating anyway.
Okay. Don't let the Input Streams fill or they'll block. Check. .close() can also prevent this, but isn't terribly bright.
That works in general (including the generic command line apps I've tested).
In specific however, it falls down. It appears that wkhtmltopdf is using some terminal manipulation/cursor stuff to do an ASCII-graphic progress bar. I believe this is causing the inputStream to immediately return EOF rather than giving me the correct values.
Any ideas? Hardly a deal-breaker, but it would definitely be Nice To Have.
I had the same exact issue as you and I solved it. Here are my findings:
For some reason, the output from wkhtmltopdf goes to STDERR of the process and NOT STDOUT. I have verified this by calling wkhtmltopdf from Java as well as perl
So, for example in java, you would have to do:
//ProcessBuilder is the recommended way of creating processes since Java 1.5
//Runtime.getRuntime().exec() is deprecated. Do not use.
ProcessBuilder pb = new ProcessBuilder("wkhtmltopdf.exe", htmlFilePath, pdfFilePath);
Process process = pb.start();
BufferedReader errStreamReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
//not "process.getInputStream()"
String line = errStreamReader.readLine();
while(line != null)
{
System.out.println(line); //or whatever else
line = reader.readLine();
}
On a side note, if you spawn a process from java, you MUST read from the stdout and stderr streams (even if you do nothing with it) because otherwise the stream buffer will fill and the process will hang and never return.
To futureproof your code, just in case the devs of wkhtmltopdf decide to write to stdout, you can redirect stderr of the child process to stdout and read only one stream like this:
ProcessBuilder pb = new ProcessBuilder("wkhtmltopdf.exe", htmlFilePath, pdfFilePath);
pb.redirectErrorStream(true);
Process process = pb.start();
BufferedReader inStreamReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
Actually, I do this in all the cases where I have to spawn an external process from java. That way I don't have to read two streams.
You should also read the streams of the spawned process in different threads if you dont want your main thread to block, since reading from streams is blocking.
Hope this helps.
UPDATE: I raised this issue in the project page and was replied that this is by design because wkhtmltopdf supports giving the actual pdf output in STDOUT. Please see the link for more details and java code.
A process has 3 streams: input, output and error. you can read both output and error stream at the same time using separate processes. see this question and its accepted answer and also this one for example.
You should read from the streams in a different thread.
final Semaphore semaphore = new Semaphore(numOfThreads);
final String whktmlExe = tmpwhktmlExePath;
int doccount = 0;
try{
File fileObject = new File(inputDir);
for(final File f : fileObject.listFiles()) {
if(f.getAbsolutePath().endsWith(".html")) {
doccount ++;
if(doccount >500 ) {
LOG.info(" done with conversion of 1000 docs exiting ");
break;
}
System.out.println(" inside for before "+semaphore.availablePermits());
semaphore.acquire();
System.out.println(" inside for after "+semaphore.availablePermits() + " ---" +f.getName());
new java.lang.Thread() {
public void run() {
try {
String F_ = f.getName().replaceAll(".html", ".pdf") ;
ProcessBuilder pb = new ProcessBuilder(whktmlExe , f.getAbsolutePath(), outPutDir + F_ .replaceAll(" ", "_") );//"wkhtmltopdf.exe", htmlFilePath, pdfFilePath);
pb.redirectErrorStream(true);
Process process = pb.start();
BufferedReader errStreamReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line = errStreamReader.readLine();
while(line != null)
{
System.err.println(line); //or whatever else
line = errStreamReader.readLine();
}
System.out.println("after completion for ");
} catch (Exception e) {
e.printStackTrace();
}finally {
System.out.println(" in finally releasing ");
semaphore.release();
}
}
}.start();
}
}
}catch (Exception ex) {
LOG.error(" *** Error in pdf generation *** ", ex);
}
while (semaphore.availablePermits() < numOfThreads) {//till all threads finish
LOG.info( " Waiting for all threads to exit "+ semaphore.availablePermits() + " --- " +( numOfThreads - semaphore.availablePermits()));
java.lang.Thread.sleep(10000);
}

Categories

Resources