I'm trying to use Java's ProcessBuilder class to execute a command that has a pipe in it. For example:
ls -l | grep foo
However, I get an error:
ls: |: no such file or directory
Followed by:
ls: grep: no such file or directory
Even though that command works perfectly from the command line, I can not get ProcessBuilder to execute a command that redirects its output to another.
Is there any way to accomplish this?
This should work:
ProcessBuilder b = new ProcessBuilder("/bin/sh", "-c", "ls -l| grep foo");
To execute a pipeline, you have to invoke a shell, and then run your commands inside that shell.
The simplest way is to invoke the shell with the command line as the parameter. After all, it's the shell which is interpreting "|" to mean "pipe the data between two processes".
Alternatively, you could launch each process separately, and read from the standard output of "ls -l", writing the data to the standard input of "grep" in your example.
Since Java 9, there’s genuine support for piplines in ProcessBuilder.
So you can use
List<String> result;
List<Process> processes = ProcessBuilder.startPipeline(List.of(
new ProcessBuilder("ls", "-l")
.inheritIO().redirectOutput(ProcessBuilder.Redirect.PIPE),
new ProcessBuilder("grep", "foo")
.redirectError(ProcessBuilder.Redirect.INHERIT)
));
try(Scanner s = new Scanner(processes.get(processes.size() - 1).getInputStream())) {
result = s.useDelimiter("\\R").tokens().toList();
}
to get the matching lines in a list.
Or, for Windows
List<String> result;
List<Process> processes = ProcessBuilder.startPipeline(List.of(
new ProcessBuilder("cmd", "/c", "dir")
.inheritIO().redirectOutput(ProcessBuilder.Redirect.PIPE),
new ProcessBuilder("find", "\"foo\"")
.redirectError(ProcessBuilder.Redirect.INHERIT)
));
try(Scanner s = new Scanner(processes.get(processes.size() - 1).getInputStream())) {
result = s.useDelimiter("\\R").tokens().toList();
}
These examples redirect stdin of the first process and all error streams to inherit, to use the same as the Java process.
You can also call .redirectOutput(ProcessBuilder.Redirect.INHERIT) on the ProcessBuilder of the last process, to print the results directly to the console (or wherever stdout has been redirected to).
Related
I am currently trying to write a small program in java which should take over the job of an old batch script I've been using.
This batch script executes a program called sum.exe (Supermicro Update Manager).
However, no matter which way I try, the program either does not respond, or straight up tells me it can't find the file in the directory where the file is.
boolean isWindows = System.getProperty("os.name").toLowerCase().startsWith("windows");
ProcessBuilder builder = new ProcessBuilder("C:\\Users\\[Username]\\SUM\\sum.exe");
if (isWindows) {
builder.command("sum.exe", "-i 192.168.4.10 -u ADMIN -p ADMIN -c CheckOOBSupport");
} else {
builder.command("sh", "-c", "ls");
}
builder.redirectErrorStream(true);
Process process = builder.start();
StreamGobbler streamGobbler = new StreamGobbler(process.getInputStream(), System.out::println);
StreamGobbler streamGobblerErrors = new StreamGobbler(process.getErrorStream(), System.out::println);
Executors.newSingleThreadExecutor().submit(streamGobbler);
Executors.newSingleThreadExecutor().submit(streamGobblerErrors);
int exitCode = process.waitFor();
assert exitCode == 0;
This is the code I currently have. The command I'm trying to call here will 100% give an error, so I made sure to redirect those as well.
As far as I understood, there are 3 different ways to set a Filepath for the Processbuilder.
Either you:
Set the path in the constructor
Set the path between your executable and arguments in the .command() method
Or you set the directory of the builder by creating a new file (and using System.Property)
I have a complete copy of the SUM-Folder under: C:\Users\[Username]\SUM, and I have tried all 3 options listed above with this, but always got the error message that the system could not find the file specified
Additionally, I'm still not sure if the command would even work this way. I have only ever used sum.exe via batch-Script or cmd.exe itself, so wouldn't the command need to be
builder.command("cmd.exe", "sum.exe -i 192.168.4.10 -u ADMIN -p ADMIN -c CheckOOBSupport)
instead?
Can anyone tell me what I'm doing wrong?
Thanks!
The ProcessBuilder command line is passed in the constructor or the command() method so in your example you've overridden the value used.
Choose the way you need:
ProcessBuilder builder = new ProcessBuilder("C:\\Users\\[Username]\\SUM\\sum.exe",
"-i", "192.168.4.10",
"-u", "ADMIN","-p", "ADMIN",
"-c", "CheckOOBSupport");
or
ProcessBuilder builder = new ProcessBuilder();
builder.command("sum.exe",
"-i", "192.168.4.10",
"-u", "ADMIN","-p", "ADMIN",
"-c", "CheckOOBSupport");
Note also that the arguments for the command need to supplied as separate string values rather than all concatenated together as one value, and you need absolute path to "sum.exe" if that is not found in the current directory or under a directory of environment variable "Path".
So guys I want to execute a command that you can execute on the cmd in my Java program. After doing some study, I thought i found a way to do this. However, my code doesn't work.
My code is
import java.io.*;
public class CmdTest {
public static void main(String[] args) throws Exception {
String[] command = {"ag","startTimes conf.js >> pro.txt"};
ProcessBuilder builder = new ProcessBuilder(command);
builder.directory(new File("./test-java/"));
Process p = builder.start();
}
}
The program executes but produces no output. I tried using other commands like "ls -a", but still no output.
Can someone please help me debug this or suggest a better way of doing this? Thank you
Edit 1: I am executing this on a Mac. If that is necessary for debugging
Edit 2: The usual ls and other commands are working with the solutions that you guys have provided. I however want to use the ag (the_silver_searcher) command in the Java program. When i try that, i get the following error -
Exception in thread "main" java.io.IOException: Cannot run program "ag startTimes conf.js >> pro.txt": error=2, No such file or directory
The existing answers give you the information on how to solve your problem in code, but they don't give a reason why your code is not working.
When you execute a program on a shell, there's significant processing done by the shell, before the program is ever executed. Your command line
String[] command = {"ag","startTimes conf.js >> pro.txt"};
ProcessBuilder builder = new ProcessBuilder(command);
assumes that the command ag is run with the single argument startTimes conf.js >> pro.txt - most likely not what you want to do. Let's go one step further: What if you wrote
String[] command = {"ag","startTimes", "conf.js", ">>", "pro.txt"};
ProcessBuilder builder = new ProcessBuilder(command);
?
This would assume that the ag command knows about the >> parameter to redirect its output - and here is where the shell comes into play: The >> operator is an instruction to the shell, telling it what to do with the output from stdout of the process. The process ag, when started by the shell, never has an idea of this redirection and has no clue about >> and the target file name at all.
With this information, just use the code samples from any of the other answers. I won't copy them into mine for proper attribution.
While there is ProcessBuilder, I've always used Runtime.getRuntime().exec("cmd");
Process Runtime.exec(String)
It returns a Process which you can get the input and output streams of
Even if you stay with the ProcessBuilder, you should still have access to the Process.get<Input/Output/Error>Stream()
You need to read the output of the process by opening an input stream from the process:
try (BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream())) {
System.out.println(reader.readLine()); // process the output stream somehow
}
Additionally you might the read the error stream ( p.getErrorStream()), which I often have done in a separate stream, in Java 8 you can use redirectErrorStream(true) on the ProcessBuilder to automatically add the error stream to the input stream. Of course you can't distinquish anymore from which stream the input comes, but it makes reading easier. If you don't read the input or error stream and the process's buffer becomes full the processes tend to pause until there is enough room in the buffer again.
You can also add
builder.redirectOutput(ProcessBuilder.Redirect.INHERIT);
before the start method which redirects the output to the console.
//"ls" command runs under the "sh" on linux(cmd.exe on windows), so first arg is "sh"
//second arg "-c" tells "sh" which exact command should be executed
//"ls" is actual command
//"startTimes" as I understand is a file or directory, it is arg for "ls" command
//"conf.js" is second arg for "ls" command
new ProcessBuilder("sh", "-c", "ls", "startTimes", "conf.js")
//set working dir for "sh" process"
.directory(new File("./test-java/"))
//output will be written to "pro.txt" in working dir of "sh" process
.redirectOutput(new File("./test-java/pro.txt"))
.start();
I am running the below query through Java on a Postgres DB using psql:
psql.exe -U <user> -w -h <host> -d <db_name> -a -f <file> 2> "<path_to_file>\psql.log"
Initially, for quite some time the java program did create the file. Then I ran into another problem, that it was not overwriting the log file. So i used file.delete() function after every time this log file got created via java.
Now, Java is not even creating the log file for some reason. If I run the above manually in command prompt, it runs absolutely fine, but not via java code. I can see this command getting run in the java log, but it does not create the log file even when i have removed the file.delete() function
I researched a lot on it but could not find any solution. Any help would be highly appreciated.
its a long code..so i will tell you the relevant part.
I am calling a function from a thread. Code is below for that function:
public static void SaveACopyfileToServer(int auditid,String filepath,String fname,String tb_name,String plpgsql_path) throws Exception
{
Map<String, String> env = System.getenv();
String plpgsql = "\""+plpgsql_path+"\" -U "+env.get("PG_USER")+" -w -h "+env.get("PG_HOST")+" -d "+env.get("PG_DB")+" -a -f "+"\""+filepath+"copy_"+tb_name+auditid+".sql\" 2> \"C:\\ER\\ETL\\logs\\psql.log\"";
System.out.println(plpgsql);
Process p = Runtime.getRuntime().exec(plpgsql);
p.getOutputStream().close();
p.waitFor();
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss:SSS");
Calendar cal10 = Calendar.getInstance();
System.out.println("Data loaded for "+tb_name+auditid+" at "+sdf.format(cal10.getTime()));
}
After this i am calling another function which is:
public static void extracterrorreason(String fname,int auditid,String sessionid,Connection con_pg) throws FileNotFoundException, IOException, InterruptedException{
File file = new File("C:\\ER\\ETL\\logs\\psql.log");
if(file.exists())
{
System.out.println("File present");
}
else
{
System.out.println(file+" not found");
}
if (file.length()!=0){
System.out.println("Log file being read is "+file);
BufferedReader br = new BufferedReader(new FileReader(file));
String line = br.readLine();
String out_err = line.substring(line.indexOf("ERROR"));
System.out.println(out_err);
System.out.println("Error while loading the file into Database for file "+fname);
String comment = "CopyToStage','"+out_err;
Utils.updateAuditDetailTable(auditid, sessionid, -1, comment, true, con_pg,"");
br.close();
//file.delete();
}
}
The first function used to create the psql.log file, but now it does not even create it. Not sure where is the problem. Every time i run the code and from the second function,i get the printline that log file not found. The part before the redirection of the output of the cmd command works fine.
I tried process builder also..
I even tried it with Process builder
String plpgsql = "\""+plpgsql_path+"\" -U "+env.get("PG_USER")+" -w -h "+env.get("PG_HOST")+" -d "+env.get("PG_DB")+" -a -f "+"\""+filepath+"copy_"+tb_name+auditid+".sql\" 2> \"C:\\ER\\ETL\\psql_" +auditid +".log\"";
ProcessBuilder pb = new ProcessBuilder("cmd.exe",plpgsql);
Process p =pb.start();
p.getOutputStream().close();
p.waitFor();
I expect that the problem is that Runtime.getRuntime().exec(plpgsql) is splitting the command line into arguments incorrectly. Basically, exec does not understand quoting. Instead, it splits wherever it sees one or more spaces ... even if those spaces are in quotes.
The solution is to use the exec(String[]) overload, and pass each individual argument as a separate string; e.g.
.exec(new String[]{plpgsql_path,
"-U",
env.get("PG_USER"),
"-w,
"-h",
// etcetera
});
UPDATE
I didn't notice that you were using > output redirection as well1.
That doesn't work with exec either. (And the same applies to all shell syntax.) To get redirection, you need to use ProcessBuilder and one of the redirect methods.
The other alternative is to run the command in a shell. Pass the command as a string, and let the shell take care of the quote handling, substitution of environment variables, globbing, redirection ... and so on.
For example (if you were running on UNIX, Linux or MacOSX):
.exec(new String[]{"/bin/sh", "-c", plpgsql});
For Windows
.exec(new String[]{"cmd.exe", "/C", plpgsql});
Note the "/C" option in the Windows case!
1 - It serves you right for not line-breaking that ~200 character line in your source code! Check out what Java coding standards say about source line lengths ...
I need to run a shell command in Windows:
c:\Python27\python.exe c:\probabilistic_cracker\process.py dic2.txt
which is running fine in a command shell.
In Java I do this:
ProcessBuilder pb = new ProcessBuilder(Arrays.asList("c:\\Python27\\python", " c:\\probabilistic_cracker\\process.py"," dic2.txt"));
Process p = pb.start();
or this
ProcessBuilder pb = new ProcessBuilder("c:\\Python27\\python", " c:\\probabilistic_cracker\\process.py"," dic2.txt");
in both cases the result is
c:\Python27\python: can't open file ' c:\probabilistic_cracker\process.py': [Errno 22] Invalid argument
Your command is built correctly but the way you pass it to ProcessBuilder isn't, as stated in its documentation you pass the args directly the way they are, there's no need to add spaces since the ProcessBuilder will take care of that for you.
ProcessBuilder pb = new ProcessBuilder("c:\\Python27\\python", "c:\\probabilistic_cracker\\process.py","dic2.txt");
So just removing those whitespaces you have in the beginning of each argument string will do the trick.
I am using Windows!
I want to call a small .exe application from my java command line which is called "saucy.exe". It needs an input file "input.saucy". Both are stored in the correct directory.
When I use the command
Process p = Runtime.getRuntime().exec("saucy input.saucy");
everything works fine and I get an output on the console.
However, when I try to write the output in a file
Process p = Runtime.getRuntime().exec("saucy input.saucy > output.saucy");
nothing happens.
I already found the advice in http://www.ensta-paristech.fr/~diam/java/online/io/javazine.html and tried to tokenize the command manually:
String[] cmd = {"saucy", "input.saucy > output.saucy"};
Process p = Runtime.getRuntime().exec(cmd);
It is still not working. Any advice? It is no option for me to write the output to a file with java code, because its too slow.
Again: I am using Windows (I stress that because I read several hints for Linux systems).
> is a shell command, but you are not using one. try
String[] cmd = { "cmd", "/C", "saucy input.saucy > output.saucy" };
If you are on Java 7 you can use the new ProcessBuilder.redirectOutput mechanism:
ProcessBuilder pb = new ProcessBuilder("saucy", "input.saucy");
// send standard output to a file
pb.redirectOutput(new File("output.saucy"));
// merge standard error with standard output
pb.redirectErrorStream(true);
Process p = pb.start();
Use the getInputStream(), getOutputStream() and getErrorStream() to retrieve the output (or send input).
http://docs.oracle.com/javase/1.4.2/docs/api/java/lang/Process.html