OK after few tries, I would like to rephrase my question :
"I have developed Web App with Angular 5 (frontend), spring-boot (backend) AND Java 8
The next step is to launch partner software, installed on remote server, from the interface.
It's an .exe program with some parameters, But I wish to test by just launching putty
My java class (by using #Ankesh answer)
#Service
public class DictService extends CoreServices {
public ApiResponse<?> launch(String idWS, String ipp, String nom, String prenom, String ddn) {
try {
boolean isWindows = System.getProperty("os.name")
.toLowerCase().startsWith("windows");
String path = "C:\\klinck\\PuTTY\\putty.exe";
ProcessBuilder builder = new ProcessBuilder();
if (isWindows) {
// builder.command("cmd.exe", "/c", "dir");
builder.command("cmd.exe", "/c", path);
} else {
// this is for bash on linux (can be omitted)
builder.command("sh", "-c", "ls");
}
System.out.println(builder);
// builder.directory(new File(System.getProperty("user.home")));
// Start the process here
// Redirect the errorstream
builder.redirectErrorStream(true);
System.out.println(""+builder.redirectErrorStream());
System.out.println("before start");
Process process = builder.start();
System.out.println("after start");
// Follow the process to get logging if required
StreamGobbler streamGobbler =
new StreamGobbler (process.getInputStream(), System.out::println);
//Submit log collection to and executor for proper scheduling and collection of loggs
System.out.println("before executors submit");
Executors.newSingleThreadExecutor().submit(streamGobbler);
// Collect exit code
System.out.println("before waitFor");
int exitCode = process.waitFor();
System.out.println("after waitFor");
// validate if the appliction exited without errors using exit code
assert exitCode == 0;
} catch (Exception ure) {
return new ApiResponse<>(false, "Une erreur interne est survenue. Merci de contacter le support", null, 0);
}
return new ApiResponse<>(true, null, null, 0);
}
private static class StreamGobbler implements Runnable {
private InputStream inputStream;
private Consumer<String> consumer;
public StreamGobbler(InputStream inputStream, Consumer<String> consumer) {
this.inputStream = inputStream;
this.consumer = consumer;
}
#Override
public void run() {
System.out.println("StreamGlobber run");
new BufferedReader(new InputStreamReader(inputStream)).lines()
.forEach(consumer);
}
}
}
A process in launched but the GUI doen't appear.
I noticed that this process locks tomcat log files (stderr and stdout)
I saw that it's possible :
Best Way to Launch External Process from Java Web-Service?
Executing external Java program from a webapp
But I don't succeed to adapt this code.
Here is log :
java.lang.ProcessBuilder#7b1ead05
true
before start
after start
before executors submit
before waitFor
StreamGlobber run
It seems like process is launched.
I don't understand....How can I solve that ?
You can use your SpringBoot backend to achieve this.
In java ProcessBuilder class can help you launch a command (which will run your executable). Here we are running cmd.exe which is available on system path.
ProcessBuilder builder = new ProcessBuilder();
if (isWindows) {
builder.command("cmd.exe", "/c", "dir");
} else {
// this is for bash on linux (can be omitted)
builder.command("sh", "-c", "ls");
}
builder.directory(new File(System.getProperty("user.home")));
// Start the process here
Process process = builder.start();
// Follow the process to get logging if required
StreamGobbler streamGobbler =
new StreamGobbler(process.getInputStream(), System.out::println);
//Submit log collection to and executor for proper scheduling and collection of loggs
Executors.newSingleThreadExecutor().submit(streamGobbler);
// Collect exit code
int exitCode = process.waitFor();
// validate if the appliction exited without errors using exit code
assert exitCode == 0;
Reference : https://www.baeldung.com/run-shell-command-in-java
Related
I want to run pmd inside a java process(created using ProcessBuilder) from within spring as a service.
public class PMDService {
private ProcessBuilder processBuilder;
private void createProcess() {
processBuilder = new ProcessBuilder();
final Map<String, String> envMap = processBuilder.environment();
String path = envMap.get("Path");
path += "../../../../../../../static-code-analyzers/pmd/bin;";
envMap.put("Path", path);
}
public String getCommand(PMDParameters params) {
final StringJoiner command = new StringJoiner(" ");
command.add("cmd")
.add("/c")
.add("pmd")
.add("-d")
.add(params.getSourceCodePath())
.add("-f")
.add(params.getOutputFormat())
.add("-R")
.add(params.getResultSet())
.add(">")
.add(params.getResultsPath());
return command.toString();
}
public void runAnalyzer(PMDParameters params) throws IOException, InterruptedException {
createProcess();
processBuilder.command(getCommand(params));
final Process process = processBuilder.start();
process.waitFor();
}
When I test the rest endpoint using postman i get following error :
""message": "Cannot run program \"cmd /c pmd -d
C:/bootcamp/Spring/springbootcode/springbootdemo -f xml -R
rulesets/java/quickstart.xml > .\": CreateProcess error=2, The system
cannot find the file specified","
Input json in postman
{
"sourceCodePath": "C:/bootcamp/Spring/springbootcode/springbootdemo",
"resultsPath": ".",
"outputFormat": "xml",
"resultSet": "rulesets/java/quickstart.xml"
}
Analysis
In the provided piece of code the program and its arguments are being passed as the single string value as the ProcessBuilder ProcessBuilder.command(String... command) method parameter.
But that should not be the case: instead, the program and its arguments should be passed separately.
Solution
Let's use the ProcessBuilder ProcessBuilder.command(List<String> command) method.
We will prepare the string list appropriately: it will contain the program and its arguments.
The draft solution:
public List<String> getProgramAndArguments(PMDParameters params) {
final List<String> programAndArguments = new ArrayList<String>();
programAndArguments.add("cmd");
programAndArguments.add("/c");
programAndArguments.add("pmd");
programAndArguments.add("-d");
programAndArguments.add(params.getSourceCodePath());
programAndArguments.add("-f");
programAndArguments.add(params.getOutputFormat());
programAndArguments.add("-R");
programAndArguments.add(params.getResultSet());
programAndArguments.add(">");
programAndArguments.add(params.getResultsPath());
return programAndArguments;
}
public void runAnalyzer(PMDParameters params) throws IOException, InterruptedException {
createProcess();
processBuilder.command(getProgramAndArguments(params));
final Process process = processBuilder.start();
process.waitFor();
}
Additionally, please, make sure that params.getResultsPath() has the correct value by providing the correct input: now it is ..
I'm on Windows, and i try to work on a Java application that was written to be use on a Linux OS, because the program will launch some shell script at some point.
I have WSL (Windows Subsystem for Linux, also know as Ubuntu bash), so executing shell script should not be a problem, but i have an error : 0x80070057
The code that launch the external process :
public Process startProcess(List<String> commands ) throws IOException {
ProcessBuilder etProcessBuilder= new ProcessBuilder(commands);
Process etProcess = etProcessBuilder.start();
ProcessOutputReader stdReader= new ProcessOutputReader(etProcess.getInputStream(), LOGGER::info);
ProcessOutputReader errReader= new ProcessOutputReader(etProcess.getErrorStream(), LOGGER::error);
new Thread(stdReader).start();
new Thread(errReader).start();
return etProcess;
}
The commands param are set with with something like this :
"/mnt/d/some/path/scripts/initEAF.sh"
"-argForTheScript"
"some value"
"-anotherArg"
"other value"
I also tried to add "bash.exe" as first command, but it doesn't seems to work.
The ProcessOutputReaderis a class to log the stream from the process
class ProcessOutputReader implements Runnable {
private final InputStream inputStream;
private Consumer<String> loggingFunction;
ProcessOutputReader(InputStream inputStream, Consumer<String> loggingFunction) {
this.inputStream = inputStream;
this.loggingFunction = loggingFunction;
}
private BufferedReader getBufferedReader(InputStream is) {
return new BufferedReader(new InputStreamReader(is));
}
#Override
public void run() {
BufferedReader br = getBufferedReader(inputStream);
String ligne;
try {
while ((ligne = br.readLine()) != null) {
loggingFunction.accept(ligne);
}
} catch (IOException e) {
LOGGER.error("Error occur while reading the output of process ", e);
}
}
}
Any idea is welcome.
*.sh is not an executable file.
You need run it by a shell, such as bash xxx.sh -args or sh xxx.sh -args if your java app run inside wsl.
If your java app run on Windows, it should be bash.exe -c xxx.sh
I have a java project, which complied into an executable jar file v-agent-exe.jar. This jar is a log server, log rows is sent to it for processing.
I can execute it by using this command:
`java -jar v-agent-exe.jar -a watch -f config.ini`.
After executed, this jar file will create a ServerSocket at port 1235 and listen for incoming data from clients. After data received, the program will process the data and send the result back to the client. When I execute the jar from CMD windows, the processing is working perfect.
Now I am trying to wrap the Jar file as a Windows service (I am using Windows 10). I created a "Windows service project"
in Visual studio like below:
- Caller class have call() method to execute the jar file using process.
- AgentService is the service, which execute Caller->call() in another thread.
- Program is the main entry to load AgentService.
Caller.cs
public class Caller
{
static Process proc;
public Process GetProcess(){
return proc;
}
public void call() {
try
{
String dir = AppDomain.CurrentDomain.BaseDirectory;
proc = new Process
{
StartInfo = new ProcessStartInfo
{
WorkingDirectory = dir,
FileName = "java.exe",
Arguments = #"-jar v-agent-exe.jar -a watch -f config.ini",
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
RedirectStandardInput = true,
CreateNoWindow = true
}
};
proc.Start();
while (!proc.StandardError.EndOfStream)
{
string line = proc.StandardError.ReadLine();
}
}
catch (Exception ex) {
VAgentService.writeLog("Error when call process: " + ex.Message);
}
}
}
AgentService
public partial class AgentService : ServiceBase
{
private string jarPath;
private string iniPath;
static Process proc;
Caller caller;
public AgentService()
{
InitializeComponent();
}
protected override void OnStart(string[] args)
{
writeLog("On start");
try
{
caller = new Caller();
writeLog("Prepare to launch thread");
Thread t = new Thread(new ThreadStart(caller.call));
t.Start();
}
catch (Exception ex)
{
EventLog.WriteEntry("Demo error: " + ex.Message);
}
}
protected override void OnStop()
{
proc = caller.GetProcess();
if (proc != null && !proc.HasExited)
{
proc.Kill();
}
else
{
...
}
}
}
Program.cs
static class Program
{
/// <summary>
/// The main entry point for the application.
/// </summary>
static void Main(String[] args)
{
ServiceBase[] ServicesToRun;
ServicesToRun = new ServiceBase[]
{
new AgentService()
};
ServiceBase.Run(ServicesToRun);
}
}
After build the the service project, I have AgentService.exe.
I install it to my system using:
sc create VAgentLogging binpath= %CD%\AgentService.exe depend= lmhosts start= auto
After start the service in service.msc, I can telnet to port "1235" which the java process is listening (I am sure about
only the jar running in this port). According to the
log of java program, it still can received some part of data but seem like it cannot send back to client or something,
which cause the followed process cannot be done.
I think my problem is: the jar file can executed as standalone but somehow it sucks when wrapped under my service project.
I haven't posted the jar's code yet because I think the error is related to the Windows service project. If you need the java code, please tell me and I will update it here.
Any help would be appreciated.
I am trying to submit a JAR with Spark job into the YARN cluster from Java code. I am using SparkLauncher to submit SparkPi example:
Process spark = new SparkLauncher()
.setAppResource("C:\\spark-1.4.1-bin-hadoop2.6\\lib\\spark-examples-1.4.1-hadoop2.6.0.jar")
.setMainClass("org.apache.spark.examples.SparkPi")
.setMaster("yarn-cluster")
.launch();
System.out.println("Waiting for finish...");
int exitCode = spark.waitFor();
System.out.println("Finished! Exit code:" + exitCode);
There are two problems:
While submitting in "yarn-cluster" mode, the application is sucessfully submitted to YARN and executes successfully (it is visible in the YARN UI, reported as SUCCESS and pi is printed in the output). However, the submitting application is never notified that processing is finished - it hangs infinitely after printing "Waiting to finish..." The log of the container can be found here
While submitting in "yarn-client" mode, the application does not appear in YARN UI and the submitting application hangs at "Waiting to finish..." When hanging code is killed, the application shows up in YARN UI and it is reported as SUCCESS, but the output is empty (pi is not printed out). The log of the container can be found here
I tried to execute the submitting application both with Oracle Java 7 and 8.
I got help in the Spark mailing list. The key is to read / clear getInputStream and getErrorStream() on the Process. The child process might fill up the buffer and cause a deadlock - see Oracle docs regarding Process. The streams should be read in separate threads:
Process spark = new SparkLauncher()
.setSparkHome("C:\\spark-1.4.1-bin-hadoop2.6")
.setAppResource("C:\\spark-1.4.1-bin-hadoop2.6\\lib\\spark-examples-1.4.1-hadoop2.6.0.jar")
.setMainClass("org.apache.spark.examples.SparkPi").setMaster("yarn-cluster").launch();
InputStreamReaderRunnable inputStreamReaderRunnable = new InputStreamReaderRunnable(spark.getInputStream(), "input");
Thread inputThread = new Thread(inputStreamReaderRunnable, "LogStreamReader input");
inputThread.start();
InputStreamReaderRunnable errorStreamReaderRunnable = new InputStreamReaderRunnable(spark.getErrorStream(), "error");
Thread errorThread = new Thread(errorStreamReaderRunnable, "LogStreamReader error");
errorThread.start();
System.out.println("Waiting for finish...");
int exitCode = spark.waitFor();
System.out.println("Finished! Exit code:" + exitCode);
where InputStreamReaderRunnable class is:
public class InputStreamReaderRunnable implements Runnable {
private BufferedReader reader;
private String name;
public InputStreamReaderRunnable(InputStream is, String name) {
this.reader = new BufferedReader(new InputStreamReader(is));
this.name = name;
}
public void run() {
System.out.println("InputStream " + name + ":");
try {
String line = reader.readLine();
while (line != null) {
System.out.println(line);
line = reader.readLine();
}
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Since this is an old post, i would like to add an update that might help whom ever read this post after. In spark 1.6.0 there are some added functions in SparkLauncher class. Which is:
def startApplication(listeners: <repeated...>[Listener]): SparkAppHandle
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.launcher.SparkLauncher
You can run the application with out the need for additional threads for the stdout and stderr handling plush there is a nice status reporting of the application running. Use this code:
val env = Map(
"HADOOP_CONF_DIR" -> hadoopConfDir,
"YARN_CONF_DIR" -> yarnConfDir
)
val handler = new SparkLauncher(env.asJava)
.setSparkHome(sparkHome)
.setAppResource("Jar/location/.jar")
.setMainClass("path.to.the.main.class")
.setMaster("yarn-client")
.setConf("spark.app.id", "AppID if you have one")
.setConf("spark.driver.memory", "8g")
.setConf("spark.akka.frameSize", "200")
.setConf("spark.executor.memory", "2g")
.setConf("spark.executor.instances", "32")
.setConf("spark.executor.cores", "32")
.setConf("spark.default.parallelism", "100")
.setConf("spark.driver.allowMultipleContexts","true")
.setVerbose(true)
.startApplication()
println(handle.getAppId)
println(handle.getState)
You can keep enquering the state if the spark application until it give success.
For information about how the Spark Launcher server works in 1.6.0. see this link:
https://github.com/apache/spark/blob/v1.6.0/launcher/src/main/java/org/apache/spark/launcher/LauncherServer.java
I implemented using CountDownLatch, and it works as expected.
This is for SparkLauncher version 2.0.1 and it works in Yarn-cluster mode too.
...
final CountDownLatch countDownLatch = new CountDownLatch(1);
SparkAppListener sparkAppListener = new SparkAppListener(countDownLatch);
SparkAppHandle appHandle = sparkLauncher.startApplication(sparkAppListener);
Thread sparkAppListenerThread = new Thread(sparkAppListener);
sparkAppListenerThread.start();
long timeout = 120;
countDownLatch.await(timeout, TimeUnit.SECONDS);
...
private static class SparkAppListener implements SparkAppHandle.Listener, Runnable {
private static final Log log = LogFactory.getLog(SparkAppListener.class);
private final CountDownLatch countDownLatch;
public SparkAppListener(CountDownLatch countDownLatch) {
this.countDownLatch = countDownLatch;
}
#Override
public void stateChanged(SparkAppHandle handle) {
String sparkAppId = handle.getAppId();
State appState = handle.getState();
if (sparkAppId != null) {
log.info("Spark job with app id: " + sparkAppId + ",\t State changed to: " + appState + " - "
+ SPARK_STATE_MSG.get(appState));
} else {
log.info("Spark job's state changed to: " + appState + " - " + SPARK_STATE_MSG.get(appState));
}
if (appState != null && appState.isFinal()) {
countDownLatch.countDown();
}
}
#Override
public void infoChanged(SparkAppHandle handle) {}
#Override
public void run() {}
}
Is it possible to launch a Java program from another Java program, just as if I were launching it using another Java command? When calling the main() method of a program from another program directly, the Java context is common to these both executions. I'm trying to have one Java context per thread.
Illustration:
src/com/project/ProjectLauncher.java
public class ProjectLauncher {
static {
PropertyConfigurator.configure("log4j.properties");
}
public static void main(String[] args) {
Logger.getLogger(ProjectLauncher.class).info("started!");
// Logs well as expected.
}
}
test/com/project/TestProject.java
public class TestProject extends TestCase {
public void testProject() {
ProjectLauncher.main(null);
Logger.getLogger(TestProject.class).info("tested!");
// The above line logs well, while log4j has been initialized in ProjectLauncher.
// I would like it to need its own initialization in this class.
}
}
I tried to launch the main method in another thread/runnable, but the logger is still initialized by ProjectLauncher.
Well when you start a Java process, its a new Instance of JVM. If you wish to start another JVM instance, then you need to start a separate process of it.
i.e.
List<String> command = new ArrayList<String>();
command.add("java");
command.add("ProjectLauncher");
ProcessBuilder builder = new ProcessBuilder(command);
builder.redirectErrorStream(true);
final Process process = builder.start();
try {
process.waitFor();
} catch (InterruptedException ex) {
ex.printStackTrace();
}
//if you wish to read the output of it then below code else you can omit it.
InputStream is = process.getErrorStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line;
while ((line = br.readLine()) != null) {
Logger.getLogger(MyClass.class.getName()).severe(line);
}
Above we are ultimately starting a new process which in reality is java ProjectLauncher. In case if the class is not already compiled, then you will have to compile it similar to above but using javac instead of java and ProjectLauncher.java instead of ProjectLauncher etc.