I have binary of engine already developed with Hadoop(HDFS,HBASE,MAPREDUCE) and java which is use to generate CSV file. There are some operation performed by this engine like table creation in HBASE and generating the CSV File from this HBASE. But this engine is performed all operation through only command line , as input is given from Linux terminal in form of command. Now my requirement is to connect this Linux terminal Through java program and run the commands But I am not able to successfully run any of the command
There are two option I tried but none of them successfully worked. Please provide any suggestion or solution to solve this problem as I am just beginner of Linux and hadoop to figure out the problem
1st-way
public class EngineTest {
public static void main(String[] args) {
try {
Process process = Runtime
.getRuntime()
.exec("/home/cloudera/PVGproto/Base/ anloss -i ${TOOL_INPUT}/census_10000_col5.csv -d ${TOOL_DEF}/attr_all_def.txt -q k=14,dage=2 -g ${TOOL_RES}/census_100_col8_gen.csv");
process.waitFor();
BufferedReader bufferedReader = new BufferedReader(
new InputStreamReader(process.getInputStream()));
String line = "";
String output = "";
while ((line = bufferedReader.readLine()) != null) {
output += line + "\n";
}
System.out.println(output);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
2nd-way
I made one ex.sh file, in this file I put the required command for execution and call this . sh file from the same java program but the same thing happened its not running the command through java program
Process process = Runtime.getRuntime().exec("/home/cloudera/PVGproto/Base/ex.sh");
but if I run the same ex.sh from Linux terminal its running all the command successfully.
ex.sh
# !/bin/bash
exec /home/cloudera/PVGproto/Base/ anloss -i ${TOOL_INPUT}/census_10000_col5.csv -d ${TOOL_DEF}/attr_all_def.txt -q k=14,age=2 -g ${TOOL_RES}/census_100_col8_gen.csv
echo command run successfully
Related
Here is my runCommand that takes Linux command input as a string
public static ArrayList<String> runCommand(String command) {
ArrayList<String> arrayList = new ArrayList<>();
try {
Process process = Runtime.getRuntime().exec(command);
BufferedReader reader =
new BufferedReader(new InputStreamReader(process.getInputStream()));
String line;
while ((line = reader.readLine()) != null) {
arrayList.add(line);
System.out.println(line);
}
int exitCode = process.waitFor();
System.out.println("\nExited with error code : "+exitCode);
System.out.println();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
return arrayList;
}
It is working great with normal command like ls or sudo lsof -t -i:132 or other. Even tailing live log from ping google.com. My socat command
sudo SOCAT_SOCKADDR=192.168.11.131 socat -d -d -T 10 UDP4-LISTEN:132,reuseaddr,fork UDP4:192.168.11.130:130,bind=192.168.11.131:133
(-d -d for verbose) in terminal creates socat process and tails the logs in terminals like incoming connection or connection status. but if I run this command via my runCommand(), this isn't printing anything in the terminal where the java jar application is running. And it worked fine for other cases of commands.
I tried placing the process.waitFor() before reading the loop but nothing. What is the problem here? my main goal is to parse the live-tailed log and do some other stuff depending on that.
I have the following problem which seems to be caused by the "docker pull" in my shell script, as the pull works concurrently
#!/bin/bash
#VARIABLES
NAME="my-app"
IMAGE="my-image:latest"
#DOCKER
docker stop $NAME
docker rm $NAME
docker pull -q $IMAGE
docker run --name $NAME -d -p 1234:8080 --log-opt fluentd-address=localhost:2233 $IMAGE
Running the script through the terminal works just fine everything works as expected. But when I run it with the Java's ProcessBuilder the script exits much quicker and it seems that it skips the "docker pull" step. As i am not a Java developer and I am not very well familiar with the Language I have the feeling that is something related to the multi-concurrent nature of the docker pull command and the way how the Java Process Builder executes the shell script
The Java class that runs the shell script is this
try {
Collection<Task> tasks = taskService.getProjectTasksByProjectKey(projectId);
Task findTask = findTaskByTaskId(tasks, taskId);
if (findTask.getTaskId() != null) {
ProcessBuilder pb = new ProcessBuilder(findTask.getCmdPath());
Process process = pb.start();
String output;
try (InputStream in = process.getInputStream();
InputStream err = process.getErrorStream();
OutputStream closeOnly = process.getOutputStream()) {
while (process.isAlive()) {
long skipped = in.skip(in.available())
+ err.skip(err.available());
if(skipped == 0L) {
process.waitFor(5L, TimeUnit.MILLISECONDS);
}
}
output = loadStream(in);
} finally {
process.destroy();
}
// String error = loadStream(process.getErrorStream());
// int rc = process.waitFor();
// log.debug("exit code ->>> " + rc);
// StringBuilder output = new StringBuilder();
// BufferedReader reader = new BufferedReader(
// new InputStreamReader(process.getInputStream()));
//
// String line;
//
// while ((line = reader.readLine()) != null) {
// output.append(line + "\n");
// }
//
// int exitVal = process.waitFor();
// if (exitVal == 0) {
// System.out.println(output);
//
// return output.toString();
// } else {
// //abnormal...
// }
return output;
}
else {
throw new InvalidTaskModelException(taskId);
}
} catch (InvalidModelException e) {
throw new InvalidModelException(projectId);
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
private static String loadStream(InputStream s) throws Exception
{
BufferedReader br = new BufferedReader(new InputStreamReader(s));
StringBuilder sb = new StringBuilder();
String line;
while((line=br.readLine()) != null)
sb.append(line).append("\n");
return sb.toString();
}
The commented lines are different ways I tried to do it.
If anyone encountered a similar problem any help would be much appreciated!
It is good that you already take care for the processes' STDOUT and STDIN. But rather than skipping copy them to System.out so you can see what is going on. I suspect something is not going as per your expectations.
Looking at the bash script you posted and the fact you are trying to run several processes: Is it possible your java code is running the bash script line by line? Be aware your java program it is not a BASH interpreter, so e.g. variable substitution should not work.
why you can not run each command in thread and joins them so that unless therad 1 is not completed . next thread can not start .
also please add command to verify image is downloaded successfully
docker pull -q $IMAGE
docker images | grep $IMAGE
docker run --name $NAME -d -p 1234:8080 --log-opt fluentd-address=localhost:2233 $IMAGE
i am guessing there are 2 possiblities are here
Check local directory persmission if possible give 755 permission to
it .
Java process itself is not able to execute docker command due
to permission issue, run process as sudo user.
I'm trying to use Java to execute a shell script that triggers a docker copy command. Everything I have tried has returned code 127 (command not found).
public static void main(String[] args)
{
System.out.println("Starting shell script");
String script = "SendToDocker.sh";
String scriptPath = "/path/";
try {
Process awk = new ProcessBuilder("/bin/bash", scriptPath + script).start();
awk.waitFor();
BufferedReader reader = new BufferedReader(new InputStreamReader(
awk.getInputStream()));
String s;
while ((s = reader.readLine()) != null) {
System.out.println("Script output: " + s);
System.out.println("output code: "+awk.exitValue());
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Finished");
}
SendToDocker.sh
#!/bin/bash
docker cp /path/asdf.csv 2d55a8445409:/
The script works perfectly fine if I run it in my terminal. Any help/ideas would be greatly appreciated.
*) Go to the directory where the Dockerfile is
*) docker build -t your_app_name .
*) docker run -it your_app_name
Try some docker api for java jar?
Here is a example:
https://github.com/docker-java/docker-java/tree/master/src/test/java/com/github/dockerjava/core/command
I was developing a project in Java to scan the File System and this involves executing dos commands in java with administrative privilege.
I already wrote the program to execute simple dos commands in Java.
public class doscmd {
public static void main(String args[]) {
try {
Process p = Runtime.getRuntime().exec("cmd /C dir");
p.waitFor();
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = reader.readLine();
while (line != null) {
System.out.println(line);
line = reader.readLine();
}
} catch (IOException e1) {
} catch (InterruptedException e2) {
}
System.out.println("Done");
}
}
But as you can see this does not allow to execute elevated commands.
I am developing the project in Netbeans IDE and i was hoping if any of you folks could tell me if there is any code in java to get admin privilege instead of converting the file to .exe and then clicking run as administrator.
Your JVM needs to be running with admin-privileges in order to start a process with admin-privileges.
Build your code and run it as an administrator - every process spawned by your class will have administrator privileges as well.
try this code, it works for me:
String command = "cmd /c start cmd.exe";
Process child = Runtime.getRuntime().exec(command);
OutputStream output = child.getOutputStream();
output .write("cd C:/ /r/n".getBytes());
output .flush();
output .write("DIR /r/n".getBytes());
output .close();
I've been struggling for a while now with this problem and i can't seem to fix it. i have tried ProcessBuilder for executing the custom command on linux terminal but its not working
Actually i have two .sh file setProto.sh and setTls.sh file which is used to set the environment.So for executing the command i need to run these two file first for each instance of linux terminal.Only then we can be able to run the custom command anloss on the same instance of linux terminal in which .sh file supposed to run.For some reason i'm not able to make it work what is the mistake in my code ? Here's the code.
import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
import java.io.InputStreamReader;
import java.lang.ProcessBuilder.Redirect;
public class EngineTest {
public static void main(String[] args) {
try {
ProcessBuilder builder = new ProcessBuilder(
"/. setProto.sh",
"/. setTls.sh",
"/anloss -i ${TOOL_INPUT}/census_10000_col5.csv -d ${TOOL_DEF}/attr_all_def.txt -q k=14,dage=2 -g ${TOOL_RES}/census_100_col8_gen.csv");
builder.directory(new File(System.getenv("HOME") + "/PVproto/Base"));
File log = new File("log");
builder.redirectErrorStream(true);
builder.redirectOutput(Redirect.appendTo(log));
Process process = builder.start();
BufferedReader bufferedReader = new BufferedReader(
new InputStreamReader(process.getInputStream()));
String line = "";
String output = "";
while ((line = bufferedReader.readLine()) != null) {
output += line + "\n";
}
System.out.println(output);
int exitValue = process.waitFor();
System.out.println("\n\nExit Value is " + exitValue);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
Process does not by default execute in the context of a shell; therefore your shell scripts cannot be executed the way you tried.
ProcessBuilder pb =
new ProcessBuilder( "/bin/bash",
"-c",
". setProto.sh && . setTls.sh && /anloss -i ${TOOL_INPUT}/census_10000_col5.csv -d ${TOOL_DEF}/attr_all_def.txt -q k=14,dage=2 -g ${TOOL_RES}/census_100_col8_gen.csv" );
I am not sure about /anloss - it's unusual for a command to be in root's home /. (Also, the /. you had there in front of the shell scripts - what should they achieve?)
Later
Make sure to replace /anloss with an absolute pathname or a relative pathname relative to $HOME/PVproto/Base, e.g., if it is in this directory, use ./anloss, if it is in $HOME/PVproto/Base/SomeSub, use SomeSub/anloss, etc.
Also, if setProto.sh and . setTls.sh are not in $HOME/PVproto/Base, use an appropriate absolute or relative pathname. If they are, use ./setProto.sh and ./setTls.sh to avoid dependency on the setting of environment variable PATH.
I think you need to use Runtime.exec() for executing the commands on linux. I suppose you are executing your java code on linux machine where linux scripts needs to be run.
Below code snippet will help you in resolving it.
Process process = Runtime.getRuntime().exec(scriptWithInputParameters);
int exitCode = process.waitFor();
if (exitCode == 0) {
System.out.println("Executed successfully");
}
else {
System.out.println("Failed ...");
}
Please note you need to handle error and output stream in different threads to avoid buffer overflow.
If the above works for you then this article will help you further