I am running the below query through Java on a Postgres DB using psql:
psql.exe -U <user> -w -h <host> -d <db_name> -a -f <file> 2> "<path_to_file>\psql.log"
Initially, for quite some time the java program did create the file. Then I ran into another problem, that it was not overwriting the log file. So i used file.delete() function after every time this log file got created via java.
Now, Java is not even creating the log file for some reason. If I run the above manually in command prompt, it runs absolutely fine, but not via java code. I can see this command getting run in the java log, but it does not create the log file even when i have removed the file.delete() function
I researched a lot on it but could not find any solution. Any help would be highly appreciated.
its a long code..so i will tell you the relevant part.
I am calling a function from a thread. Code is below for that function:
public static void SaveACopyfileToServer(int auditid,String filepath,String fname,String tb_name,String plpgsql_path) throws Exception
{
Map<String, String> env = System.getenv();
String plpgsql = "\""+plpgsql_path+"\" -U "+env.get("PG_USER")+" -w -h "+env.get("PG_HOST")+" -d "+env.get("PG_DB")+" -a -f "+"\""+filepath+"copy_"+tb_name+auditid+".sql\" 2> \"C:\\ER\\ETL\\logs\\psql.log\"";
System.out.println(plpgsql);
Process p = Runtime.getRuntime().exec(plpgsql);
p.getOutputStream().close();
p.waitFor();
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss:SSS");
Calendar cal10 = Calendar.getInstance();
System.out.println("Data loaded for "+tb_name+auditid+" at "+sdf.format(cal10.getTime()));
}
After this i am calling another function which is:
public static void extracterrorreason(String fname,int auditid,String sessionid,Connection con_pg) throws FileNotFoundException, IOException, InterruptedException{
File file = new File("C:\\ER\\ETL\\logs\\psql.log");
if(file.exists())
{
System.out.println("File present");
}
else
{
System.out.println(file+" not found");
}
if (file.length()!=0){
System.out.println("Log file being read is "+file);
BufferedReader br = new BufferedReader(new FileReader(file));
String line = br.readLine();
String out_err = line.substring(line.indexOf("ERROR"));
System.out.println(out_err);
System.out.println("Error while loading the file into Database for file "+fname);
String comment = "CopyToStage','"+out_err;
Utils.updateAuditDetailTable(auditid, sessionid, -1, comment, true, con_pg,"");
br.close();
//file.delete();
}
}
The first function used to create the psql.log file, but now it does not even create it. Not sure where is the problem. Every time i run the code and from the second function,i get the printline that log file not found. The part before the redirection of the output of the cmd command works fine.
I tried process builder also..
I even tried it with Process builder
String plpgsql = "\""+plpgsql_path+"\" -U "+env.get("PG_USER")+" -w -h "+env.get("PG_HOST")+" -d "+env.get("PG_DB")+" -a -f "+"\""+filepath+"copy_"+tb_name+auditid+".sql\" 2> \"C:\\ER\\ETL\\psql_" +auditid +".log\"";
ProcessBuilder pb = new ProcessBuilder("cmd.exe",plpgsql);
Process p =pb.start();
p.getOutputStream().close();
p.waitFor();
I expect that the problem is that Runtime.getRuntime().exec(plpgsql) is splitting the command line into arguments incorrectly. Basically, exec does not understand quoting. Instead, it splits wherever it sees one or more spaces ... even if those spaces are in quotes.
The solution is to use the exec(String[]) overload, and pass each individual argument as a separate string; e.g.
.exec(new String[]{plpgsql_path,
"-U",
env.get("PG_USER"),
"-w,
"-h",
// etcetera
});
UPDATE
I didn't notice that you were using > output redirection as well1.
That doesn't work with exec either. (And the same applies to all shell syntax.) To get redirection, you need to use ProcessBuilder and one of the redirect methods.
The other alternative is to run the command in a shell. Pass the command as a string, and let the shell take care of the quote handling, substitution of environment variables, globbing, redirection ... and so on.
For example (if you were running on UNIX, Linux or MacOSX):
.exec(new String[]{"/bin/sh", "-c", plpgsql});
For Windows
.exec(new String[]{"cmd.exe", "/C", plpgsql});
Note the "/C" option in the Windows case!
1 - It serves you right for not line-breaking that ~200 character line in your source code! Check out what Java coding standards say about source line lengths ...
Related
I'm trying to use Java's ProcessBuilder class to execute a command that has a pipe in it. For example:
ls -l | grep foo
However, I get an error:
ls: |: no such file or directory
Followed by:
ls: grep: no such file or directory
Even though that command works perfectly from the command line, I can not get ProcessBuilder to execute a command that redirects its output to another.
Is there any way to accomplish this?
This should work:
ProcessBuilder b = new ProcessBuilder("/bin/sh", "-c", "ls -l| grep foo");
To execute a pipeline, you have to invoke a shell, and then run your commands inside that shell.
The simplest way is to invoke the shell with the command line as the parameter. After all, it's the shell which is interpreting "|" to mean "pipe the data between two processes".
Alternatively, you could launch each process separately, and read from the standard output of "ls -l", writing the data to the standard input of "grep" in your example.
Since Java 9, there’s genuine support for piplines in ProcessBuilder.
So you can use
List<String> result;
List<Process> processes = ProcessBuilder.startPipeline(List.of(
new ProcessBuilder("ls", "-l")
.inheritIO().redirectOutput(ProcessBuilder.Redirect.PIPE),
new ProcessBuilder("grep", "foo")
.redirectError(ProcessBuilder.Redirect.INHERIT)
));
try(Scanner s = new Scanner(processes.get(processes.size() - 1).getInputStream())) {
result = s.useDelimiter("\\R").tokens().toList();
}
to get the matching lines in a list.
Or, for Windows
List<String> result;
List<Process> processes = ProcessBuilder.startPipeline(List.of(
new ProcessBuilder("cmd", "/c", "dir")
.inheritIO().redirectOutput(ProcessBuilder.Redirect.PIPE),
new ProcessBuilder("find", "\"foo\"")
.redirectError(ProcessBuilder.Redirect.INHERIT)
));
try(Scanner s = new Scanner(processes.get(processes.size() - 1).getInputStream())) {
result = s.useDelimiter("\\R").tokens().toList();
}
These examples redirect stdin of the first process and all error streams to inherit, to use the same as the Java process.
You can also call .redirectOutput(ProcessBuilder.Redirect.INHERIT) on the ProcessBuilder of the last process, to print the results directly to the console (or wherever stdout has been redirected to).
Question:
I have to call an exe file passing 2 string arguments. The call has to get executed as a different user
I referred few of the links that I got from hints. Couple of them were using powershell, few were using runas examples and so on.
But with powershell also, I am not sure if I will face the issue of "cannot be loaded because running scripts is disabled on this system".
Just because of this reason, I want to try something authentic.
Tried following approaches
Trying to do it through a call with RunAs command is making my life difficult. (called a batch file with RunAs command passing a separate pass.txt or savecred). savecred was ruled out by most people. Other one did not work as expected.
Powershell, has issues wherein, I always get a userid/password popup, even though I run through a Java file.
I also get an error in the command prompt as here below
+ CategoryInfo : InvalidOperation: (:) [New-Object], MethodExcept
ion
+ FullyQualifiedErrorId : ConstructorInvokedThrowException,Microsoft.Power
Shell.Commands.NewObjectCommand
(Sample code attached below)
public class SamplePS {
public static void main(String[] args) throws IOException {
Runtime runtime = Runtime.getRuntime();
String command = "powershell C:\\Users\\Ramu\\Project\\SampleFile.ps1";
//String cmds[] = {"C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\powershell.exe", "\\c", "SampleFile.ps"};
System.out.println("1");
Process proc = runtime.exec(command);
InputStream is = proc.getErrorStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader reader = new BufferedReader(isr);
String line;
while ((line = reader.readLine()) != null)
{
System.out.println(line);
}
System.out.println("2");
}
}
My ps1 file is as here below
$credential = New-Object System.Management.Automation.PSCredential <<domain>>\<<user id>>, <<password>>
start-process cmd.exe -arg "/k dir" -Credential $credential
What is the best way to handle this scenario?
Please suggest! I am not sure what approach to take.
Powershell or Perl or any other scripting.
Whatever it be, I have to make a call from Java file.
Please suggest!
Note: For time being, I dont mind the password being sent as a plain text.
The exe file is in Windows machine
Thanks!
Ram
I have written some code for executing .bat file. which contains some
commands like setting java classpath,etc..And finally there is one command
which runs a Java class file.The HelloWorld class converts some xml file and generating a new xml file in some folder. When I double click .bat file, it executes fine,
but when I try to run I am not getting any output as I was getting through
double click the .bat file. How to make a batch execute and probably it would be nice
if I could see the results through Java console.
Following is MyJava code to execute the .bat file
public void run2() {
try {
String []commands = {"cmd.exe","/C","C:/MyWork/Java/classes/run.bat"} ;
Process p = Runtime.getRuntime().exec(commands);
BufferedReader in = new BufferedReader(new InputStreamReader(
p.getInputStream()));
String line = null;
while ((line = in.readLine()) != null) {
System.out.println(line);
}
} catch (IOException e) {
e.printStackTrace();
}
}
And below the some commands which has been set to .bat file
set CLASSPATH=%CLASSPATH%;C:/MyWork/Java
set CLASSPATH=%CLASSPATH%;C:/MyWork/Java/classes
java -cp test.jar;test2.jar test.HelloWorld
Tried with "/C" commad as well. It does not execute. Actually it does not give effect of double click the .bat file. Is there any other way that I can try with?
I can see the contents inside the .bat file through Eclipse console. But it does not give the desired output. Desired output means when I double click .bat file, it executes well. But through java call, I can see the contents only .
When using cmd.exe use /C-Parameter to pass command:
String []commands = {"cmd.exe","/C","C:/MyWork/Java/classes/run.bat"} ;
according to this, the Windows CMD needs the /c argument, to execute commands like this. try this:
String []commands = {"cmd.exe","/c","C:/MyWork/Java/classes/run.bat"} ;
Windows uses \ backslash for Windows and MS-DOS path delimiter. Forward slash / is accepted by Java in the java.io package and translated to be a path delimiter, but will not be directly acceptable to Windows or accepted by the cmd.exe shell.
You may also need to specify either the working directory for the batch file to be executed in, or possibly a full path to the cmd.exe command interpreter.
See: Runtime.exec (String[] cmdarray, String[] envp, File dir)
String[] commands = {"C:\\Windows\\System32\\cmd.exe", "/c",
"C:\\MyWork\\Java\\classes\\run.bat"};
File workDir = new File( "C:/MyWork");
Process process = Runtime.getRuntime().exec( commands, null, workDir);
To verify if the batch file is run at all, add a pause command to the batch file. That will keep the window open so you can verify if the batch file is launched at all, and debug this stage-by-stage.
You do not read the error output of your batch file, therefore, you'll never see any error messages printed from there or from CMD.EXE itself. In addition, the sub-program may stall and just wait for you to read the error stream.
Please see related discussions here: How to make a java program to print both out.println() and err.println() statements?
i'm trying to execute a SOX command from java, but unfortunately its returning an error everytime. Every other SOX commands are working perfectly though!!
Here is the code :
class Simple {
public static void main(String args[]) throws IOException, Exception {
Process p;
BufferedReader br;
String co = "sox speech_16.wav -p pad 0 2.5 | sox - -m speech_16.wav speech_output.wav";
p = Runtime.getRuntime().exec(co);
br = new BufferedReader(new InputStreamReader(p.getInputStream()));
int returnCode = p.waitFor();
System.out.println("reurn code : "+returnCode);
}
}
When I'm executing the same sox command in terminal, its working fine. I really can't understand what the problem is!! Is it because of the '|' symbol??
The issue is that Runtime.exec() does not understand shell concepts such as "|". Instead try:
Runtime.getRuntime().exec("/bin/sh", "-c", co);
The problem is that exec runs a binary directly without invoking the shell. The "|" character is only recognized by the shell, not by sox. The "-c" tells the shell to run a single command, and passes the entire command as the single argument.
This is likely to be related to the environment in which the commands get executed, it could be any of the following:
The sox executable cannot be found (put the full path in the command)
The user does not have permission to run the sox command (check execute bit in file permissions)
Some environment variable needed by sox is not initialised when you run the command from Java (check sox documentation)
If speech_16.wav is an input file to sox then the file cannot be found (add full path of .wav file to command)
If sox needs to create an output file then it does not have permission to do so, either due to directory permissions, of because there is an existing file with that name which cannot be overwritten, or due to lack of space on the file-system.
I want to call shell script on windows environment using java code.
I am trying to execute code below to run my test script(not actual script):
Java Code:
public static void main (String args[]) {
Runtime r = Runtime.getRuntime();
try {
Process p = r.exec("C:\\cygwin\\bin\\bash -c '/cygdrive/d/scripts/test.sh'");
InputStream in = p.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(in));
System.out.println("OUT:");
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
in = p.getErrorStream();
br = new BufferedReader(new InputStreamReader(in));
System.out.println("ERR:");
while ((line = br.readLine()) != null) {
System.out.println(line);
}
p.waitFor();
} catch (Exception e) {
e.printStackTrace();
}
}
test.sh
#!/bin/bash
rm -rf ./test
But I am getting these error and script in not able to remove the directory.
ERR:
/cygdrive/d/ereader/scripts/test.sh: line 2: $'\r': command not found
/cygdrive/d/ereader/scripts/test.sh: line 3: rm: command not found
Another thing, when I run the script from the cygwin terminal it works fine. I checked the path variable they all are fine. But I try to execute same script through java code it gives error..
Now how to tell java program where to refer for rm commands?
The giveaway is the '\r' error.
Windows and Unix (which includes Mac and Linux) use different representations of a new line. Windows uses '\r\n' while Unix simply uses '\n'. Most programming editors account for this and only insert a '\n' so they work with Unix tools.
I would suggest retyping your shell script in another editor like Notepad++ (which only inserts '\n'), or make sure your current editor is set to use Unix newlines. You have to retype it or the bad '\r\n' sequences will get copied over. You might have some luck in doing a replace-all but that always acts flaky for me.
probably you will need to set the PATH variable first thing in the test.sh script. Make sure rm is in the PATH you set
Not really answer. You could try following to narrow down the problem
Use ProcessBuilder instead of Process. It is much more friendly to handle arguments.
Set absolute path to rm (/bin/rm ) in the script
remove using absolute path to directory or remove after verifying you are in the correct directory..
The bash prompt you have is result of cygwin.bat calling bash with --login. It will have path variables and other usesul stuff sources. bash -c does not do it.
Try launching bash.exe with bash -l -c <command> : This sources the bash profile.