I have a problem with executing command in Runtime.getRuntime().exec(command) on my linux DEV machine.
I'm trying to convert html content to mobi format using calibre but it doesn't work.
Even if I added log.warn to display what this process returns it shows nothing.
It works on my local machine where I do have Windows 10 and I read on the Internet that I should point out that command should be launched on linux so I added String[] cmd = {"bash", "-c", command}; but it still doesn't work.
My command looks like this:
/usr/src/calibre/ebook-convert /tmp/filesDirectory_mobi3970575619159760977/d7ed6792-b3cb-4761-bb6a-b9facf9e7a6c9250643820137116550.html /tmp/filesDirectory_mobi3970575619159760977/d7ed6792-b3cb-4761-bb6a-b9facf9e7a6c11909891397415910433.mobi
And here's my code:
#Override
public Document convert(Document document, DocumentFormat documentFormat) {
Document htmlDocument = htmlDocumentConverter.convert(document, documentFormat);
try {
log.info("Converting document from {} to {}", getSourceFormat().toString(), getTargetFormat().toString());
CalibreConfigData calibreData = calibreConfig.getConfigurationData(CalibreConversion.HTML_TO_MOBI);
Files.write(calibreData.getSourceFilePath(), htmlDocument.getContent());
String command = calibreData.getCalibreCommand();
String[] cmd = {"bash", "-c", command};
var r = Runtime.getRuntime().exec(cmd);
r.waitFor();
log.warn("Process: " + new String(r.getInputStream().readAllBytes(), StandardCharsets.UTF_8));
byte[] convertedFileAsBytes = Files.readAllBytes(calibreData.getConvertedFilePath());
// Files.deleteIfExists(calibreData.getSourceFilePath());
// Files.deleteIfExists(calibreData.getConvertedFilePath());
// Files.deleteIfExists(calibreData.getFilesDirectoryPath());
return new Document(convertedFileAsBytes);
} catch (InterruptedException | IOException e) {
log.error("Conversion failed due to problem: " + e);
throw new ConversionException("Conversion failed due to problem: " + e);
}
}
I checked created tempFiles and found out that file .html has content inside but file .mobi is empty even after executing above's command.
A main method of a Calibre conversion invoker could be as simple as the below:
public static void main(String[] args) {
try {
String[] command = { "/usr/src/calibre/ebook-convert", args[0], args[1] };
ProcessBuilder pb = new ProcessBuilder(command);
pb.inheritIO();
pb.start();
} catch (Throwable t) {
t.printStackTrace();
}
}
I write a PDF generator that is implemented in Java and uses PowerShell scripts to compile TEX files with latexmk.
An example for a generated command is powershell.exe -NoProfile -NoLogo "&'C:\path\to\scripts\compileTex.ps1' 'C:\path\to\workingDir' 'filename.tex'".
If I this command manually in the cmd the command compiles the TEX file and returns properly.
When I call this script via Java it does not return.
I read many other questions that suggest adding < NUL at the end of the command. I do this by explicitly closing the input stream of the process. Resulting from that all PowerShell scripts return properly except for the one with the latexmk command.
I use the following PowerShell script for compilation:
param(
[Parameter(Mandatory = $true)][string] $workingDir,
[Parameter(Mandatory = $true)][string] $texFileName
)
if (Test-Path -LiteralPath "$workingDir") {
cd "$workingDir"
if (Test-Path -LiteralPath "$texFileName") {
$texFileBaseName = $texFileName.Trim(".tex")
latexmk -nobibtex -norc -pdf -jobname="$texFileBaseName" "$texFileName" # FIXME This call leads to a timeout
} else {
Write-Error "Could not find \"$texFileName\""
}
} else {
Write-Error "The working directory \"$workingDir\" does not exist"
}
I use the following Java snippet for calling this script:
StringJoiner innerCommand = new StringJoiner(" ", "\"&", "\"")
.add(String.format("'%s'", script.normalize().toString()));
for (String param : params) {
if (param.startsWith("-")) { // Its an option or switch
innerCommand.add(param);
} else {
innerCommand.add(String.format("'%s'", param));
}
}
StringJoiner command = new StringJoiner(" ")
.add("powershell.exe")
.add("-NoProfile")
.add("-NoLogo")
.add(innerCommand.toString());
try {
Process process = Runtime.getRuntime()
.exec(command);
process.getOutputStream().close(); // Make sure CMD does not wait for further commands
boolean exited = process.waitFor(COMMAND_TIMEOUT_VALUE, COMMAND_TIMEOUT_UNIT);
String stdOutResult = new String(process.getInputStream().readAllBytes());
T commandResult;
if (exited) {
if (process.exitValue() == 0) {
commandResult = onSuccess.apply(stdOutResult);
} else {
String stdErrResult = new String(process.getErrorStream().readAllBytes());
commandResult = onFailure.apply(stdOutResult, stdErrResult);
}
} else {
commandResult = onTimeout.get();
}
return commandResult;
} catch (IOException | InterruptedException ex) {
throw new SystemCommandFailedException(String.format("Processing %d errored", commandID), ex);
}
}
EDIT:
Appending a [Console]::Out.Flush() does not do the trick.
I am writing a java app that needs to perform mysql dump, and I am using the runtime.exec, based in the when runtime.exec won't article. The code is below:
public int exectuteCommand(){
Runtime rt = Runtime.getRuntime();
logger.debug("exexuting cmd: " + showCommand());
int exit = -1;
try {
Process proc = rt.exec(cmd);
ExtProcessStreamHandler errorHandler = new ExtProcessStreamHandler(proc.getErrorStream(), "ERROR");
ExtProcessStreamHandler outHandler = new ExtProcessStreamHandler(proc.getInputStream(), "OUTPUT");
// kick it off
errorHandler.start();
outHandler.start();
exit = proc.waitFor();
} catch (IOException e) {
logger.error("ERROR!! ~~ executing command " + showCommand(), e);
e.printStackTrace();
} catch (InterruptedException e) {
logger.error("ERROR!! ~~ unexpected return for " + showCommand() + " , returned " + exit, e);
e.printStackTrace();
}
return exit;
}
1) The command that the process returns works in the shell (I'm running this on a mac). The first error I had was an inability to find the mysqldump command. That results in this error:
java.io.IOException: Cannot run program "mysqldump": error=2, No such file or directory
I resolved that by adding the complete path of the file to the command. The $PATH var shows
/usr/local/mysql/bin/mysqldump
as the complete path. How can I make sure my java app has that info?
2) when adding the complete path to the command, I get this error msg:
INFO [Thread-1] (ExtProcessStreamHandler.java:28) - external process ERROR : mysqldump: Couldn't find table: ">"
Here is the code that builds the command array:
return new String[] {MYSQLDUMP_CMD, "-u", USER_DEFAULT, "-p"+ PW_DEFAULT, TEST_DB_NAME,
">", DUMP_LOC};
again, when I copy the command passed to the java app into the shell on my mac, it works. Not sure what I'm doing wrong.
thanks in advance!
It thinks ">" is an argument intended for mysqldump. You are invoking an executable, not evaluating a shell expression. If you want to pipe your output, do it with the outHandler and errorHandler in your code.
An alternative is to invoke a shell and pass the expression you want to evaluate as an argument:
expr = new StringBuilder()
.append(MYSQLDUMP_CMD).append(' ')
.append("-u").append(USER_DEFAULT).append(' ')
.append("-p").append(PW_DEFAULT).append(' ')
.append(TEST_DB_NAME).append(' ')
.append(">").append(' ')
.append(DUMP_LOC)
.toString();
return new String[] {"/bin/bash", "-c", expr};
If your code to build the command array doesn't wrap spaced arguments in single quotes (or if the JDK doesn't do this for you), then modify the StringBuilder statement to create the wrapped quotes for you.
Below Code is worked for me
public static void backup() {
String currentDate = LocalDate.now().format(DateTimeFormatter.ofPattern("yyyy_MM_dd"));
String backupPath = String.format("%s/%s.%s", Helper.BACKUP_PATH, currentDate, "sql");
File backupFile = new File(backupPath);
if (!backupFile.exists()) {
try {
backupFile.createNewFile();
String mysqlCom=String.format("mysqldump -u%s -p%s %s",USER_NAME,PASSWORD,DB);
String[] command = new String[] { "/bin/bash", "-c",mysqlCom};
ProcessBuilder processBuilder = new ProcessBuilder(Arrays.asList(command));
processBuilder.redirectError(Redirect.INHERIT);
processBuilder.redirectOutput(Redirect.to(backupFile));
Process process = processBuilder.start();
process.waitFor();
LOGGER.info("Backup done");
} catch (IOException e1) {
e1.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
} else {
LOGGER.info("Database already backuped today");
}
}
First some code:
Runtime runtime = Runtime.getRuntime();
String args[] = new String[2];
// args[0] = "/bin/bash";
// args[1] = "-c";
// args[2] = "/usr/bin/rpm2cpio "+archiveFile.getCanonicalPath()+" | /bin/cpio -idmv";
args[0] = "/usr/bin/rpm2cpio";
args[1] = archiveFile.getCanonicalPath();
Process rpm2cpioProcess = runtime.exec(args, null, dir);
// System.out.println("started rpm2cpio");
String args2[] = new String[3];
args2[0] = "/bin/cpio";
args2[1] = "-idmu";
args2[2] = "--quiet";
Process cpioProcess = runtime.exec(args2, null, dir);
// System.out.println("started cpio");
InputStream fromRpm2cpio = rpm2cpioProcess.getInputStream();
new ProcessInputStreamer(rpm2cpioProcess.getErrorStream());
OutputStream fromCpio = cpioProcess.getOutputStream();
new PipedStreamer(fromRpm2cpio, fromCpio);
new ProcessInputStreamer(cpioProcess.getErrorStream());
// System.out.println("pipe created");
while(cpioProcess!=null && fromRpm2cpio!=null) {
boolean doSleep = true;
// System.out.println("waking up");
if (cpioProcess!=null) {
try {
if (cpioProcess.exitValue()==0) {
cpioProcess = null;
doSleep = false;
}
} catch(IllegalThreadStateException e) {
}
}
if (rpm2cpioProcess!=null) {
try {
if (rpm2cpioProcess.exitValue()==0) {
rpm2cpioProcess = null;
doSleep = false;
}
} catch(IllegalThreadStateException e) {
}
}
if (doSleep) {
Thread.sleep(30);
}
// System.out.println("still running");
}
I'm trying to extract the content of an rpm archive. This code works fine after multiple modifications. My first attempt was to execute the next code through Java:
/bin/bash -c '/usr/bin/rpm2cpio <archive-file> | /bin/cpio -idmv'
Which worked fine the first time I ran it (you can see it in the code commented above). The second time I ran the code it got blocked since the extracted files already existed. So I thought maybe it has to do with the piping and thus split the call into two separate processes. This didn't help much either. So I then modified the arguments of the /bin/cpio from '-idmv' to '-idmu --quiet' and now it works. Unfortunately the -u option overwrites existing files 'unconditionally' which is not really needed. My question is why does it block with -idmv and why doesn't it block with -idmu ?
It could be waiting on standard input for some inputs. Redirect your standard input and/or output to /dev/null
I'd guess that your ProcessInputStreamer and/or PipedStreamer implement Runnable or extent Thread and you're not running them anywhere.
It is quite simple to run a Unix command from Java.
Runtime.getRuntime().exec(myCommand);
But is it possible to run a Unix shell script from Java code? If yes, would it be a good practice to run a shell script from within Java code?
You should really look at Process Builder. It is really built for this kind of thing.
ProcessBuilder pb = new ProcessBuilder("myshellScript.sh", "myArg1", "myArg2");
Map<String, String> env = pb.environment();
env.put("VAR1", "myValue");
env.remove("OTHERVAR");
env.put("VAR2", env.get("VAR1") + "suffix");
pb.directory(new File("myDir"));
Process p = pb.start();
You can use Apache Commons exec library also.
Example :
package testShellScript;
import java.io.IOException;
import org.apache.commons.exec.CommandLine;
import org.apache.commons.exec.DefaultExecutor;
import org.apache.commons.exec.ExecuteException;
public class TestScript {
int iExitValue;
String sCommandString;
public void runScript(String command){
sCommandString = command;
CommandLine oCmdLine = CommandLine.parse(sCommandString);
DefaultExecutor oDefaultExecutor = new DefaultExecutor();
oDefaultExecutor.setExitValue(0);
try {
iExitValue = oDefaultExecutor.execute(oCmdLine);
} catch (ExecuteException e) {
System.err.println("Execution failed.");
e.printStackTrace();
} catch (IOException e) {
System.err.println("permission denied.");
e.printStackTrace();
}
}
public static void main(String args[]){
TestScript testScript = new TestScript();
testScript.runScript("sh /root/Desktop/testScript.sh");
}
}
For further reference, An example is given on Apache Doc also.
I think you have answered your own question with
Runtime.getRuntime().exec(myShellScript);
As to whether it is good practice... what are you trying to do with a shell script that you cannot do with Java?
I would say that it is not in the spirit of Java to run a shell script from Java. Java is meant to be cross platform, and running a shell script would limit its use to just UNIX.
With that said, it's definitely possible to run a shell script from within Java. You'd use exactly the same syntax you listed (I haven't tried it myself, but try executing the shell script directly, and if that doesn't work, execute the shell itself, passing the script in as a command line parameter).
Yes it is possible to do so. This worked out for me.
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import org.omg.CORBA.portable.InputStream;
public static void readBashScript() {
try {
Process proc = Runtime.getRuntime().exec("/home/destino/workspace/JavaProject/listing.sh /"); //Whatever you want to execute
BufferedReader read = new BufferedReader(new InputStreamReader(
proc.getInputStream()));
try {
proc.waitFor();
} catch (InterruptedException e) {
System.out.println(e.getMessage());
}
while (read.ready()) {
System.out.println(read.readLine());
}
} catch (IOException e) {
System.out.println(e.getMessage());
}
}
Here is my example. Hope it make sense.
public static void excuteCommand(String filePath) throws IOException{
File file = new File(filePath);
if(!file.isFile()){
throw new IllegalArgumentException("The file " + filePath + " does not exist");
}
if(isLinux()){
Runtime.getRuntime().exec(new String[] {"/bin/sh", "-c", filePath}, null);
}else if(isWindows()){
Runtime.getRuntime().exec("cmd /c start " + filePath);
}
}
public static boolean isLinux(){
String os = System.getProperty("os.name");
return os.toLowerCase().indexOf("linux") >= 0;
}
public static boolean isWindows(){
String os = System.getProperty("os.name");
return os.toLowerCase().indexOf("windows") >= 0;
}
Yes, it is possible and you have answered it! About good practises, I think it is better to launch commands from files and not directly from your code. So you have to make Java execute the list of commands (or one command) in an existing .bat, .sh , .ksh ... files.
Here is an example of executing a list of commands in a file MyFile.sh:
String[] cmd = { "sh", "MyFile.sh", "\pathOfTheFile"};
Runtime.getRuntime().exec(cmd);
To avoid having to hardcode an absolute path, you can use the following method that will find and execute your script if it is in your root directory.
public static void runScript() throws IOException, InterruptedException {
ProcessBuilder processBuilder = new ProcessBuilder("./nameOfScript.sh");
//Sets the source and destination for subprocess standard I/O to be the same as those of the current Java process.
processBuilder.inheritIO();
Process process = processBuilder.start();
int exitValue = process.waitFor();
if (exitValue != 0) {
// check for errors
new BufferedInputStream(process.getErrorStream());
throw new RuntimeException("execution of script failed!");
}
}
As for me all things must be simple.
For running script just need to execute
new ProcessBuilder("pathToYourShellScript").start();
The ZT Process Executor library is an alternative to Apache Commons Exec. It has functionality to run commands, capturing their output, setting timeouts, etc.
I have not used it yet, but it looks reasonably well-documented.
An example from the documentation: Executing a command, pumping the stderr to a logger, returning the output as UTF8 string.
String output = new ProcessExecutor().command("java", "-version")
.redirectError(Slf4jStream.of(getClass()).asInfo())
.readOutput(true).execute()
.outputUTF8();
Its documentation lists the following advantages over Commons Exec:
Improved handling of streams
Reading/writing to streams
Redirecting stderr to stdout
Improved handling of timeouts
Improved checking of exit codes
Improved API
One liners for quite complex use cases
One liners to get process output into a String
Access to the Process object available
Support for async processes ( Future )
Improved logging with SLF4J API
Support for multiple processes
This is a late answer. However, I thought of putting the struggle I had to bear to get a shell script to be executed from a Spring-Boot application for future developers.
I was working in Spring-Boot and I was not able to find the file to be executed from my Java application and it was throwing FileNotFoundFoundException. I had to keep the file in the resources directory and had to set the file to be scanned in pom.xml while the application was being started like the following.
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<includes>
<include>**/*.xml</include>
<include>**/*.properties</include>
<include>**/*.sh</include>
</includes>
</resource>
</resources>
After that I was having trouble executing the file and it was returning error code = 13, Permission Denied. Then I had to make the file executable by running this command - chmod u+x myShellScript.sh
Finally, I could execute the file using the following code snippet.
public void runScript() {
ProcessBuilder pb = new ProcessBuilder("src/main/resources/myFile.sh");
try {
Process p;
p = pb.start();
} catch (IOException e) {
e.printStackTrace();
}
}
Hope that solves someone's problem.
Here is an example how to run an Unix bash or Windows bat/cmd script from Java. Arguments can be passed on the script and output received from the script. The method accepts arbitrary number of arguments.
public static void runScript(String path, String... args) {
try {
String[] cmd = new String[args.length + 1];
cmd[0] = path;
int count = 0;
for (String s : args) {
cmd[++count] = args[count - 1];
}
Process process = Runtime.getRuntime().exec(cmd);
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
try {
process.waitFor();
} catch (Exception ex) {
System.out.println(ex.getMessage());
}
while (bufferedReader.ready()) {
System.out.println("Received from script: " + bufferedReader.readLine());
}
} catch (Exception ex) {
System.out.println(ex.getMessage());
System.exit(1);
}
}
When running on Unix/Linux, the path must be Unix-like (with '/' as separator), when running on Windows - use '\'. Hier is an example of a bash script (test.sh) that receives arbitrary number of arguments and doubles every argument:
#!/bin/bash
counter=0
while [ $# -gt 0 ]
do
echo argument $((counter +=1)): $1
echo doubling argument $((counter)): $(($1+$1))
shift
done
When calling
runScript("path_to_script/test.sh", "1", "2")
on Unix/Linux, the output is:
Received from script: argument 1: 1
Received from script: doubling argument 1: 2
Received from script: argument 2: 2
Received from script: doubling argument 2: 4
Hier is a simple cmd Windows script test.cmd that counts number of input arguments:
#echo off
set a=0
for %%x in (%*) do Set /A a+=1
echo %a% arguments received
When calling the script on Windows
runScript("path_to_script\\test.cmd", "1", "2", "3")
The output is
Received from script: 3 arguments received
It is possible, just exec it as any other program. Just make sure your script has the proper #! (she-bang) line as the first line of the script, and make sure there are execute permissions on the file.
For example, if it is a bash script put #!/bin/bash at the top of the script, also chmod +x .
Also as for if it's good practice, no it's not, especially for Java, but if it saves you a lot of time porting a large script over, and you're not getting paid extra to do it ;) save your time, exec the script, and put the porting to Java on your long-term todo list.
I think with
System.getProperty("os.name");
Checking the operating system on can manage the shell/bash scrips if such are supported.
if there is need to make the code portable.
String scriptName = PATH+"/myScript.sh";
String commands[] = new String[]{scriptName,"myArg1", "myArg2"};
Runtime rt = Runtime.getRuntime();
Process process = null;
try{
process = rt.exec(commands);
process.waitFor();
}catch(Exception e){
e.printStackTrace();
}
Just the same thing that Solaris 5.10 it works like this ./batchstart.sh there is a trick I don´t know if your OS accept it use \\. batchstart.sh instead. This double slash may help.
for linux use
public static void runShell(String directory, String command, String[] args, Map<String, String> environment)
{
try
{
if(directory.trim().equals(""))
directory = "/";
String[] cmd = new String[args.length + 1];
cmd[0] = command;
int count = 1;
for(String s : args)
{
cmd[count] = s;
count++;
}
ProcessBuilder pb = new ProcessBuilder(cmd);
Map<String, String> env = pb.environment();
for(String s : environment.keySet())
env.put(s, environment.get(s));
pb.directory(new File(directory));
Process process = pb.start();
BufferedReader inputReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
BufferedWriter outputReader = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
BufferedReader errReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
int exitValue = process.waitFor();
if(exitValue != 0) // has errors
{
while(errReader.ready())
{
LogClass.log("ErrShell: " + errReader.readLine(), LogClass.LogMode.LogAll);
}
}
else
{
while(inputReader.ready())
{
LogClass.log("Shell Result : " + inputReader.readLine(), LogClass.LogMode.LogAll);
}
}
}
catch(Exception e)
{
LogClass.log("Err: RunShell, " + e.toString(), LogClass.LogMode.LogAll);
}
}
public static void runShell(String path, String command, String[] args)
{
try
{
String[] cmd = new String[args.length + 1];
if(!path.trim().isEmpty())
cmd[0] = path + "/" + command;
else
cmd[0] = command;
int count = 1;
for(String s : args)
{
cmd[count] = s;
count++;
}
Process process = Runtime.getRuntime().exec(cmd);
BufferedReader inputReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
BufferedWriter outputReader = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
BufferedReader errReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
int exitValue = process.waitFor();
if(exitValue != 0) // has errors
{
while(errReader.ready())
{
LogClass.log("ErrShell: " + errReader.readLine(), LogClass.LogMode.LogAll);
}
}
else
{
while(inputReader.ready())
{
LogClass.log("Shell Result: " + inputReader.readLine(), LogClass.LogMode.LogAll);
}
}
}
catch(Exception e)
{
LogClass.log("Err: RunShell, " + e.toString(), LogClass.LogMode.LogAll);
}
}
and for usage;
ShellAssistance.runShell("", "pg_dump", new String[]{"-U", "aliAdmin", "-f", "/home/Backup.sql", "StoresAssistanceDB"});
OR
ShellAssistance.runShell("", "pg_dump", new String[]{"-U", "aliAdmin", "-f", "/home/Backup.sql", "StoresAssistanceDB"}, new Hashmap<>());