I am trying to get the output of curl command in java.
I am able to execute the same curl command manually through terminal and obtain the output but when I try to execute through java code as below, the output obtained is null.
I could use Apache HttpClient for the same, but I want to try and use this curl cli.
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.util.Arrays;
public class MyCurlClient {
public static void main(String[] args) {
MyCurlClient obj = new MyCurlClient();
// in mac oxs
// String command = "ping -c 5 " + domainName;
String command = "curl "
+ "'http://localhost:8080/auth/login' -H 'Origin: http://localhost:9000' --data-binary '{\"username\":\"policy-engine\",\"password\":\"openstack\"}' --compressed";
String output = obj.executeCommand(command);
System.out.println(output);
}
private String executeCommand(String command) {
StringBuffer output = new StringBuffer();
Process p;
try {
p = Runtime.getRuntime().exec(command);
p.waitFor();
BufferedReader reader = new BufferedReader(new InputStreamReader(
p.getInputStream()));
System.out.println(reader.readLine()); // value is NULL
String line = "";
while ((line = reader.readLine()) != null) {
System.out.println(line);
output.append(line + "\n");
}
} catch (Exception e) {
e.printStackTrace();
}
return output.toString();
}
}
process.waitFor() waits until the process is terminated.
Therefore, you won't see any output from it. You need to capture the output before the process terminates.
Causes the current thread to wait, if necessary, until the process represented by this Process object has terminated. This method returns immediately if the subprocess has already terminated. If the subprocess has not yet terminated, the calling thread will be blocked until the subprocess exits.
You can fix your code by removing the process.waitFor() line.
You may want to try using an absolute path to the curl command, e.g.
command = "/usr/bin/curl " + ...
It's possible Java is not able to find the curl binary and that is why it doesn't work.
If this doesn't fix your problem, verify that the curl command is working at all by doing:
String[] command = new String[]{"curl", "http://localhost:8080/auth/login",
"-H", "Origin: http://localhost:9000", "--data-binary",
"{\"username\":\"policy-engine\",\"password\":\"openstack\"}", "--compressed"};
private String executeCommand(String... command) {
ProcessBuilder builder = new ProcessBuilder(command);
builder.redirectOutput(new File("curloutput.txt"));
p.start();
}
And see if the curl command actually outputs anything to the file. If it does, then this is an InputStream timing issue. If it does not, then something is wrong with the curl command itself.
Using the String[] along with removing the single quotes from the curl command solved the issue.
Uf only String[] is used(with single quotes) then you get the expected response but along with it, you also see "unauthorized" as output.
Result is -->
{"username":"policy-engine","token":"dc7e017f-d5a3-4b72-a1c2-066880e775c7"}unauthorized
but when both String[] is used (without single quotes) the response is clean and as expected.
Result is -->
{"username":"policy-engine","token":"dc7e017f-d5a3-4b72-a1c2-066880e775c7"}
Please remove the quotes in th curl command , For Eg -
String command = "curl http://localhost:8080/auth/login -H Origin: shttp://localhost:9000 --data-binary {\"username\":\"policy-engine\",\"password\":\"openstack\"} --compressed";
Related
I am pretty new to Java and trying to fetch some values from Hashicorp vault -
When I run the curl command manually it works and returns values but when I try to run the same via code, I am running into issues.
curl POST -H "X-Vault-Namespace: dev/rel-box-dev-seed" --data '{"param1":"3f3a-094-193a-cj2e-l6ekn516a","param2":"647a-9f3-934fd3-227e-lkrae24be37"}' https://vault.abc.com/v1/auth/role/login
My Java code is as follows -
import java.io.*;
import java.util.ArrayList;
import java.util.List;
public class JavaRunCommand {
public static void main(String[] args) {
String s = null;
try {
List<String> commands = new ArrayList<String>();
commands.add("curl");
commands.add("POST");
commands.add("-H");
commands.add("\\\"X-Vault-Namespace: dev/rel-box-dev-seed\\\"");
commands.add("--data");
commands.add("'{\"param1\":\"3f3a-094-193a-cj2e-l6ekn516a\",\"param2\":\"647a-9f3-934fd3-227e-lkrae24be37\"}'");
commands.add("https://vault.abc.com/v1/auth/role/login");
ProcessBuilder pb = new ProcessBuilder(commands);
Process p = pb.start();
BufferedReader stdInput = new BufferedReader(new
InputStreamReader(p.getInputStream()));
BufferedReader stdError = new BufferedReader(new
InputStreamReader(p.getErrorStream()));
// read the output from the command
System.out.println("Standard output of the command:\n");
while ((s = stdInput.readLine()) != null) {
System.out.println(s);
}
// read any errors from the attempted command
System.out.println("Standard error of the command (if any):\n");
while ((s = stdError.readLine()) != null) {
System.out.println(s);
}
System.exit(0);
}
catch (IOException e) {
System.out.println("Exception happened - here's what I know: ");
e.printStackTrace();
System.exit(-1);
}
}
}
When I run this code, I am getting the following error -
curl: (92) HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)
Does anyone know what I am doing wrong? Or if there is a better way to do curl operation?
My recommendation here is to watch this traffic across the wire with something like Fiddler, Wireshark, or tcpdump. Then, compare these two requests. There is some difference in how you are preparing them (maybe with quotes, as Luke Woodward suggested, or maybe some other difference.) Inspecting the two requests, and comparing them for differences, will help you understand your problem in this instance as well as help you improve request comparison in the future, in general.
Have you tried removing the quotes around the arguments? A ProcessBuilder will quote the arguments for you, so you don't need to quote them again:
List<String> commands = new ArrayList<String>();
commands.add("curl");
commands.add("POST");
commands.add("-H");
commands.add("X-Vault-Namespace: dev/rel-box-dev-seed");
commands.add("--data");
commands.add("{\"param1\":\"3f3a-094-193a-cj2e-l6ekn516a\",\"param2\":\"647a-9f3-934fd3-227e-lkrae24be37\"}");
commands.add("https://vault.app.ford.com/v1/auth/approle/login");
I'm looking a way to write running log of python which is executed by java app via script.
Let's say my script is:
import time
for x in range(120):
print("Running ", x)
time.sleep(1)
Here is my current solution:
Trigger script using java
String cmd = "python script.py";
var process = Runtime.getRuntime().exec(cmd, null, new File(sandboxPath));
Write log to new file:
String traceLogCmd = String.format("strace -p %s -s 9999 -e trace=write -o output.txt", process.pid());
Runtime.getRuntime().exec(traceLogCmd, null, new File(sandboxPath));
Now the problem is output.txt only has content whenever the python script is done executing so that I cannot tailf the output file.
Meanwhile if I execute python script.py and strace command dirrectly from terminal, the output is exactly what I expected.
Can someone correct me if I did something wrong or have a another way to get python log?
Thanks in advance.
Use ProcessBuilder instead of Runtime.exec(). More details: When Runtime.exec() won't
The following code will append to StringBuilder object output of the script sb.append(line);. It would not be difficult to write that content to a file.
Process p = new ProcessBuilder("sh", "-c", "python", "path-to-your-script").start();
String result = getCommandResult(p.getInputStream());
private static String getCommandResult(InputStream stream) throws IOException {
StringBuilder sb = new StringBuilder();
try (InputStreamReader isr = new InputStreamReader(stream);
BufferedReader in = new BufferedReader(isr)) {
String line;
while ((line = in.readLine()) != null) {
sb.append(line);
}
}
return sb.toString().trim();
}
I executed the ldapsearch command through java. See the below code.
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
public class TestMain
{
public static void main(String[] args) throws InterruptedException, IOException
{
String ldspCmd ="ldapsearch -ZZ -h ldap-url.com -x -D cn=username,ou=webapps,ou=ec,o=uoa -w $(echo PassWord | base64 -di) -b ou=ec_users,dc=ec,dc=auckland,dc=ac,dc=nz \"(groupMembership=cn=bpmusers,ou=ec_group,dc=ec,dc=auckland,dc=ac,dc=nz)\"";
String output = executeCommand(ldspCmd);
System.out.println(output);
}
private static String executeCommand(String command) {
StringBuffer output = new StringBuffer();
Process p;
try {
p = Runtime.getRuntime().exec(command);
p.waitFor();
BufferedReader reader =
new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = "";
while ((line = reader.readLine())!= null) {
output.append(line + "\n");
}
BufferedReader errorReader =
new BufferedReader(new InputStreamReader(p.getErrorStream()));
String erroLine = "";
while ((erroLine = errorReader.readLine())!= null) {
output.append(erroLine + "\n");
}
} catch (Exception e) {
e.printStackTrace();
}
return output.toString();
}
}
But I get the error "ldapsearch: unable to parse debug value "i)"".
But when I execute the same command through command line it executes correctly and returns the records.
What am I doing wrong here? Can anyone help me to sort this?
Constructs like $(echo PassWord | base64 -di) inside your argument list are interpreted and handled by your shell. And when you invoke a command from Java using Runtime.exec, you are not using a shell, you are passing the command directly to the operating system, so you don't get the benefit of the shell interpreting these constructs.
If you want those benefits, you need to explicitly invoke the shell.
Also, Java doesn't have the same complex logic to split arguments to a command that a shell does. Java just cuts the argument list at space characters.
So in your executeCommand method you have a line:
p = Runtime.getRuntime().exec(command);
You should change that to:
// Add shell invocation around the above command
String[] shellCommand = { "/bin/bash", "-c", command };
p = Runtime.getRuntime().exec(shellCommand);
I want to execute this command within my java program and check if it was successfully executed.
sudo ntpdate -u someserver.com
I create a bash with the command
#!/bin/sh
sudo ntpdate -u omeserver.com
and execute it with java
ProcessBuilder pb = new ProcessBuilder("/updateTime");
Process p = pb.start();
p.waitFor();
BufferedReader stdInput = new BufferedReader(new InputStreamReader(p.getInputStream()));
System.out.println(stdInput.readLine());
But I get no output, there are no lines in stdInput, how can I check if the command was correctly executed?
If I add for example Echo updated in the end of the bash file I get it in the stdInput, but it still don't mean that the time were updated
You'd probably get by easier when just calling sudo directly with ProcessBuilder instead of an external script. That's just redundant complexity for the task at hand.
You can feed ProcessBuilder with the whole command line, for example, like this:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
public class q39836547 {
private static String[] cmdl = { "/usr/bin/sudo",
"ntpdate",
"-u",
"some.ntp.server" };
public static void main(String[] as) throws IOException {
ProcessBuilder pb = new ProcessBuilder(cmdl);
Process p = pb.start();
BufferedReader stdin = new BufferedReader(new InputStreamReader(p.getInputStream()));
BufferedReader stderr = new BufferedReader(new InputStreamReader(p.getErrorStream()));
try { p.waitFor(); }
catch(InterruptedException e) { }
if(p.exitValue() != 0)
System.err.println("The process was not executed successfully.");
else
System.err.println("The process ran and exited cleanly.");
stdin.lines().forEach(s -> System.out.println("STDOUT: " + s));
stderr.lines().forEach(s -> System.out.println("STDERR: " + s));
}
}
You also have to waitFor() (as you properly did) the ntpdate to finish. Otherwise you might end up reading its standard input or standard error with getInputStream() or getErrorStream() before there is any output produced into either stream.
If you comment out the try-catch-block, you'll occasionally see how the process is still running while you're trying to read its input. That is likely to happen almost every time, actually.
I am using this method to send commands to the linux terminal
public static String execute(String command) {
StringBuilder sb = new StringBuilder();
String[] commands = new String[]{"/bin/sh", "-c", command};
try {
Process proc = new ProcessBuilder(commands).start();
BufferedReader stdInput = new BufferedReader(new InputStreamReader(proc.getInputStream()));
BufferedReader stdError = new BufferedReader(new InputStreamReader(proc.getErrorStream()));
String s = null;
while ((s = stdInput.readLine()) != null) {
sb.append(s);
sb.append("\n");
}
while ((s = stdError.readLine()) != null) {
sb.append(s);
sb.append("\n");
}
} catch (IOException e) {
e.printStackTrace();
}
return sb.toString();
}
It works fine with many commands like "ls", "date" etc..
but I wanna use a psql command to import a sql file to postgre like:
psql -h localhost -p 5432 -U postgres -d test -f test.sql
I typed this command by hand in terminal and it works fine but not with the method above (the method works well with date,ls...)
It looks like the method entries in a kind of infinite loop when it calls psql. The method does not end when the method calls the psql.
ps: I used export PGPASSWORD=pass to avoid pass the password.
I think your problem here might be that you are trying to consume the input and error streams sequentially, when you must actually do them simultaneously. Consequently your first readLine call is blocking, waiting for the readLine to finish on the other stream.
You can solve this by using multiple threads to read the streams, or by redirecting stderr, or by just ditching all the output. Have a look at these threads:
ProcessBuilder: Forwarding stdout and stderr of started processes without blocking the main thread
Java ProcessBuilder: Resultant Process Hangs
I solved the problem creating a file file.sh with
export PGPASSWORD=pass
psql -h ip -p port -U user -d databaseName -f sqlFilePath
unset PGPASSWORD
then I Use the method execute:
execute("chmod +x file.sh"); //to give permission to file execute
execute ("./file.sh") //to execute the file in terminal
in the end I delete the file.sh with
File file = new File("file.sh");
file.delete();