I'm fairly new to ProcessBuilder and working with threads. In it's current state I have a J-Button which starts a scheduled executor service. The scheduled executor service is used to delegate a process to one of two process builders. The application is meant to record a user conversation. During the conversation, after x minutes it creates a wav and delegates it to an available process for transcription. The problem begins when the transcription class is called. The process is started and the application runs as expected. However, the transcription process doesn't actually do anything until I exit the parent application. Only then it will begin. Checking the task manager it shows as a process but uses 0.0% of the CPU and around 238MB of memory until I exit then the two processes jump to 30%-40% and 500-1000 MB of memory. Also, I am using the .waitFor() but am using a thread to run the .waitFor() process as from what I gather it causes the application to hang. How would I go about fixing this. Sorry I am unable to provide more details but I'm new to this. Thanks in advance!
public class TranDelegator {
Future<?> futureTranOne = null;
Future<?> futureTranTwo = null;
ExecutorService transcriberOne = Executors.newFixedThreadPool(1);
ExecutorService transcriberTwo = Executors.newFixedThreadPool(1);
final Runnable transcribeChecker = new Runnable() {
public void run() {
String currentWav = null;
File inputFile = new File("C:\\convoLists/unTranscribed.txt");
BufferedReader reader = null;
try {
reader = new BufferedReader(new FileReader(inputFile));
} catch (FileNotFoundException e1) {
System.out.println("reader didn't initialize");
e1.printStackTrace();
}
try {
currentWav = reader.readLine();
} catch (IOException e) {
System.out.println("currentWav string issue");
e.printStackTrace();
}
try {
reader.close();
} catch (IOException e) {
System.out.println("reader couldn't close");
e.printStackTrace();
}
if(currentWav != null){
if (futureTranOne == null || futureTranOne.isDone()) {
futureTranOne = transcriberOne.submit((transcriptorOne));
}
else if (futureTranTwo == null || futureTranTwo.isDone()) {
futureTranTwo = transcriberTwo.submit((transcriptorTwo));
}
}
}
};
final Runnable transcriptorOne = new Runnable() {
public void run() {
System.out.println("ONE");
try {
String classpath = System.getProperty("java.class.path");
String path = "C:/Program Files/Java/jre7/bin/java.exe";
ProcessBuilder processBuilder = new ProcessBuilder(path, "-cp",
classpath, Transcriber.class.getName());
Process process = processBuilder.start();
try {
process.waitFor();
} catch (InterruptedException e) {
System.out.println("process.waitFor call failed");
e.printStackTrace();
}
} catch (IOException e) {
System.out.println("Unable to call transcribeConvo");
e.printStackTrace();
}
}
};
final Runnable transcriptorTwo = new Runnable() {
public void run() {
System.out.println("TWO");
try {
String classpath = System.getProperty("java.class.path");
String path = "C:/Program Files/Java/jre7/bin/java.exe";
ProcessBuilder processBuilder = new ProcessBuilder(path, "-cp",
classpath, Transcriber.class.getName());
Process process = processBuilder.start();
try {
process.waitFor();
} catch (InterruptedException e) {
System.out.println("process.waitFor call failed");
e.printStackTrace();
}
} catch (IOException e) {
System.out.println("Unable to call transcribeConvo");
e.printStackTrace();
}
}
};
}
public class Transcriber {
public static void main(String[] args) throws IOException,
UnsupportedAudioFileException {
retreiveEmpInfo();
TextoArray saveConvo = new TextoArray();
ArrayList<String> entireConvo = new ArrayList();
URL audioURL;
String currentWav = wavFinder();
ConfigReader configuration = new ConfigReader();
ArrayList<String> serverInfo = configuration
.readFromDoc("serverconfig");
while (currentWav != null) {
audioURL = new URL("file:///" + currentWav);
URL configURL = Transcriber.class.getResource("config.xml");
ConfigurationManager cm = new ConfigurationManager(configURL);
Recognizer recognizer = (Recognizer) cm.lookup("recognizer");
recognizer.allocate(); // allocate the resource necessary for the
// recognizer
System.out.println(configURL);
// configure the audio input for the recognizer
AudioFileDataSource dataSource = (AudioFileDataSource) cm
.lookup("audioFileDataSource");
dataSource.setAudioFile(audioURL, null);
// Loop until last utterance in the audio file has been decoded, in
// which case the recognizer will return null.
Result result;
while ((result = recognizer.recognize()) != null) {
String resultText = result.getBestResultNoFiller();
// System.out.println(result.toString());
Collections.addAll(entireConvo, resultText.split(" "));
}
new File(currentWav).delete();
saveConvo.Indexbuilder(serverInfo, entireConvo);
entireConvo.clear();
currentWav = wavFinder();
}
System.exit(0);
}
private static String wavFinder() throws IOException {
String currentWav = null;
int x = 1;
File inputFile = new File("C:\\convoLists/unTranscribed.txt");
File tempFile = new File("C:\\convoLists/unTranscribedtemp.txt");
BufferedReader reader = new BufferedReader(new FileReader(inputFile));
BufferedWriter writer = new BufferedWriter(new FileWriter(tempFile));
String currentLine = null;
String newLine = System.getProperty("line.separator");
while ((currentLine = reader.readLine()) != null) {
if (x == 1) {
currentWav = currentLine;
} else {
writer.write(currentLine);
writer.write(newLine);
}
x = 2;
}
reader.close();
writer.flush();
writer.close();
inputFile.delete();
// boolean successful =
tempFile.renameTo(inputFile);
// System.out.println("Success: " + successful);
// System.out.println("currentWav = " + currentWav);
return currentWav;
}
private static void retreiveEmpInfo() throws IOException {
File tempFile = new File("C:\\convoLists/tmp.txt");
BufferedReader reader = new BufferedReader(new FileReader(tempFile));
CurrentEmployeeInfo.setName(reader.readLine());
CurrentEmployeeInfo.setUserEmail(reader.readLine());
CurrentEmployeeInfo.setManagerEmail(reader.readLine());
reader.close();
}
}
This problem may be related to sub-process's input stream buffers.
You should clear the sub-process's input stream buffers.
These stream buffers got increased within the parent process's memory with time and at some moment your sub-process will stop responding.
There are few options to make sub-process work normally
Read continuously from sub-process's input streams
Redirect sub-process's input streams
Close sub-process's input streams
Closing sub-process's input streams
ProcessBuilder processBuilder = new ProcessBuilder(command);
Process process = processBuilder.start();
InputStream inStream = process.getInputStream();
InputStream errStream = process.getErrorStream();
try {
inStream.close();
errStream.close();
} catch (IOException e1) {
}
process.waitFor();
Reading sub-process's input streams
ProcessBuilder processBuilder = new ProcessBuilder(command);
Process process = processBuilder.start();
InputStreamReader tempReader = new InputStreamReader(new BufferedInputStream(p.getInputStream()));
final BufferedReader reader = new BufferedReader(tempReader);
InputStreamReader tempErrReader = new InputStreamReader(new BufferedInputStream(p.getErrorStream()));
final BufferedReader errReader = new BufferedReader(tempErrReader);
try {
while ((line = reader.readLine()) != null) {
}
} catch (IOException e) {
}
try {
while ((line = errReader.readLine()) != null) {
}
} catch (IOException e) {
}
process.waitFor();
Redirecting sub-process's input streams
ProcessBuilder processBuilder = new ProcessBuilder(command);
processBuilder.redirectInput();
processBuilder.redirectError();
Process process = processBuilder.start();
process.waitFor();
(from comments)
Looks like process hang is due to out/error streams becoming full. You need to consume these streams; possibly via a thread.
Java7 provides another way to redirect output.
Related : http://alvinalexander.com/java/java-exec-processbuilder-process-3
Related
I've been working on some web project and one of its requests execute command line using Java Process.
This is the method.
#ResponseBody
#RequestMapping(value = "/startTest", method = RequestMethod.POST)
public String startTest(int test_id) {
...
String cmd = "..."
ProcessUtil pu = new ProcessUtil();
try {
pu.execute(cmd);
File file = new File(System.getProperty("user.dir")
+ "\\datas\\cypress\\videos\\examples\\main.spec.js.mp4");
File fileToMove = new File(
".\\\\datas\\\\results\\" + uitest.getTest_filename() + ".mp4");
file.renameTo(fileToMove);
return "success";
} catch (Exception e) {
return "fail";
}
}
And Here is the ProcessUtil.java
public class ProcessUtil {
public static void execute(String cmd) {
Process process = null;
Runtime runtime = Runtime.getRuntime();
StringBuffer successOutput = new StringBuffer();
StringBuffer errorOutput = new StringBuffer();
BufferedReader successBufferReader = null;
BufferedReader errorBufferReader = null;
String msg = null;
List<String> cmdList = new ArrayList<String>();
if (System.getProperty("os.name").indexOf("Windows") > -1) {
cmdList.add("cmd");
cmdList.add("/c");
} else {
cmdList.add("/bin/sh");
cmdList.add("-c");
}
cmdList.add(cmd);
String[] array = cmdList.toArray(new String[cmdList.size()]);
try {
process = runtime.exec(array);
successBufferReader = new BufferedReader(new InputStreamReader(process.getInputStream(), "UTF-8"));
while ((msg = successBufferReader.readLine()) != null) {
successOutput.append(msg + System.getProperty("line.separator"));
}
errorBufferReader = new BufferedReader(new InputStreamReader(process.getErrorStream(), "UTF-8"));
while ((msg = errorBufferReader.readLine()) != null) {
errorOutput.append(msg + System.getProperty("line.separator"));
}
process.waitFor();
if (process.exitValue() == 0) {
System.out.println("success!");
System.out.println(successOutput.toString());
} else {
System.out.println("fail...");
System.out.println(successOutput.toString());
}
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
try {
process.destroy();
if (successBufferReader != null)
successBufferReader.close();
if (errorBufferReader != null)
errorBufferReader.close();
} catch (IOException e1) {
e1.printStackTrace();
}
}
}
}
The problem that I'm facing is when I open two Windows command windows and execute simultaneously it seemingly works fine. However, when I run that 'startTest' method by requesting to my server simultaneously, it pushes my CPU and RAM to nearly 100% and results seemed odd. I don't know much about multi-threading, but I guess I should execute the command via multiple windows(I think my commands were executed in the same environment and they crashed each other...). Please give me some advice to resolve this problem... Thank you in advance.
I was trying to get the logcat content into a JTextPane. I used following code hoping it will return the content as String but it freeze and also, doesn't produce an error.
Process exec = null;
try {
exec = Runtime.getRuntime().exec("adb logcat -d");
InputStream errorStream = exec.getErrorStream();
BufferedReader ebr = new BufferedReader(new InputStreamReader(errorStream));
String errorLine;
while ((errorLine = ebr.readLine()) != null) {
System.out.println("[ERROR] :- " + errorLine);
}
if (exec.waitFor() == 0) {
InputStream infoStream = exec.getInputStream();
InputStreamReader isr = new InputStreamReader(infoStream);
BufferedReader ibr = new BufferedReader(isr);
String infoLine;
while ((infoLine = ibr.readLine()) != null) {
System.out.println("[INFO] :- " + infoLine);
}
}
} catch (IOException | InterruptedException ex) {
ex.printStackTrace();
} finally {
if (exec != null) {
exec.destroy();
}
}
I referred to some tutorials but, they were not filling my problem. Is this wrong? Are there any other methods to get the logcat content as a String programmatically? Sorry if this is a dumb question.
The issue you're seeing is that you're trying to process command streams and wait for the executing process, all in the same thread. It's blocking because the process reading the streams is waiting on the process and you're losing the stream input.
What you'll want to do is implement the function that reads/processes the command output (input stream) in another thread and kick off that thread when you start the process.
Second, you'll probably want to use ProcessBuilder rather than Runtime.exec.
Something like this can be adapted to do what you want:
public class Test {
public static void main(String[] args) throws Exception {
String startDir = System.getProperty("user.dir"); // start in current dir (change if needed)
ProcessBuilder pb = new ProcessBuilder("adb","logcat","-d");
pb.directory(new File(startDir)); // start directory
pb.redirectErrorStream(true); // redirect the error stream to stdout
Process p = pb.start(); // start the process
// start a new thread to handle the stream input
new Thread(new ProcessTestRunnable(p)).start();
p.waitFor(); // wait if needed
}
// mimics stream gobbler, but allows user to process the result
static class ProcessTestRunnable implements Runnable {
Process p;
BufferedReader br;
ProcessTestRunnable(Process p) {
this.p = p;
}
public void run() {
try {
InputStreamReader isr = new InputStreamReader(p.getInputStream());
br = new BufferedReader(isr);
String line = null;
while ((line = br.readLine()) != null)
{
// do something with the output here...
}
}
catch (IOException ex) {
ex.printStackTrace();
}
}
}
}
How can i get the stderr of a python binary? It is showing the output of the python script but it is not showing the errors that might be in the script, what java method should i be using? or is there a python command line argument that i can use to display the errors?
private String exec(String command)
{
try
{
String s = getApplicationInfo().dataDir;
File file2 = new File(s+"/py");
file2.setExecutable(true);
File externalStorage = Environment.getExternalStorageDirectory();
String strUri = externalStorage.getAbsolutePath();
PrintWriter out = new PrintWriter(strUri+"/temp.py");
out.write(command);
out.close();
saveRawToFile();
Process process = Runtime.getRuntime().exec(s+"/py "+strUri+"/temp.py");
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
int read;
char[] buffer = new char[4096];
StringBuffer output = new StringBuffer();
while ((read = reader.read(buffer)) > 0)
{
output.append(buffer, 0, read);
}
reader.close(); process.waitFor();
return output.toString();
}
catch (IOException e)
{ throw new RuntimeException(e); }
catch (InterruptedException e)
{ throw new RuntimeException(e); }
}
private void output(final String str)
{
Runnable proc = new Runnable() {
public void run()
{
outputView.setText(str);
outputView.setTextIsSelectable(true);
}
};
handler.post(proc);
}
Use Process#getErrorStream(), just like you used getOutputStream().
I have a rather complex application that I'm trying to create for an android phone. I have a class that uses the Java Process Builder and some private classes to read from both the input and output streams.
At times when the IP I'm trying to ping does not respond the thread locks due to the process getting stuck, the executor service decides after 2 minutes to shutdown. This avoids the entire application locking but the two streams never close and the threads for the streams stay open.
Any idea how to kill the stream threads?
class StreamGobbler extends Thread {
InputStream is;
String type;
StreamGobbler(InputStream is, String type) {
this.is = is;
this.type = type;
}
public void run() {
InputStreamReader isr = new InputStreamReader(is);
try {
BufferedReader br = new BufferedReader(isr);
String line = null;
while ((line = br.readLine()) != null){
System.out.println(type + ">" + line);
}
} catch (IOException e) {
LogWriter.getInstance().writeToLogFile(e);
}finally{
try {
if(is != null){
is.close();
}
if(isr != null){
isr.close();
}
} catch (IOException e) {
LogWriter.getInstance().writeToLogFile(e);
}
}
}
public void kill() {
Thread.currentThread().interrupt();
}
}
public class PingRunner implements Callable<Double>{
private String pingVal;
private int exitVal;
private double laten;
private String ipAddress;
public PingRunner(String ipAddress) {
pingVal = "";
exitVal = -1;
laten = -1;
this.ipAddress = ipAddress;
}
#Override
public Double call() throws Exception {
List<String> commands = new ArrayList<String>();
commands.add("ping");
commands.add("-c");
commands.add("5");
commands.add(ipAddress);
try {
this.doCommand(commands);
} catch (IOException e) {
LogWriter.getInstance().writeToLogFile(e);
}
return laten;
}
private void doCommand(List<String> command) throws IOException{
ProcessBuilder pb = new ProcessBuilder(command);
Process process = pb.start();
// any error message?
StreamGobbler errorGobbler = new StreamGobbler(
process.getErrorStream(), "ERROR");
// any output?
OutputStreamGobbler outputGobbler = new OutputStreamGobbler(
process.getInputStream(), "OUTPUT");
// kick them off
errorGobbler.start();
outputGobbler.start();
// read the output from the command
try {
exitVal = process.waitFor();
//Sleep for 10 secs to try to clear the buffer
Thread.sleep(10000);
//pingVal = echo.toString();
if(exitVal == 0 && !pingVal.isEmpty()){
//System.out.println("PING STATS: "+pingVal);
try{
pingVal = pingVal.substring(pingVal.lastIndexOf("rtt min/avg/max/mdev"));
pingVal = pingVal.substring(23);
pingVal = pingVal.substring(pingVal.indexOf("/")+1);
laten = Double.parseDouble(pingVal.substring(0,pingVal.indexOf("/")));
}catch (IndexOutOfBoundsException e){
System.out.println("PING VAL: "+ pingVal);
LogWriter.getInstance().writeToLogFile(e);
}
}
} catch (InterruptedException e) {
LogWriter.getInstance().writeToLogFile(e);
errorGobbler.kill();
outputGobbler.kill();
}finally{
errorGobbler = null;
outputGobbler = null;
}
System.out.println("ExitValue: " + exitVal);
}
In my main class I have this method:
protected void ping() {
laten = -1;
serverIP = serverIPs.get(testIndex % 3);
PingRunner pRunner = new PingRunner(serverIP);
Set<Callable<Double>> runner = new HashSet<Callable<Double>>();
runner.add(pRunner);
ExecutorService executor = Executors.newSingleThreadExecutor();
try {
laten = executor.submit(pRunner).get(2, TimeUnit.MINUTES);
executor.shutdown();
} catch (InterruptedException e) {
LogWriter.getInstance().writeToLogFile(e);
} catch (ExecutionException e) {
LogWriter.getInstance().writeToLogFile(e);
} catch (CancellationException e) {
pRunner.kill();
executor.shutdown();
LogWriter.getInstance().writeToLogFile(e);
LogWriter.getInstance().writeToLogFile(
"ERROR: Unable to ping server: " + serverIP);
} catch (TimeoutException e) {
pRunner.kill();
executor.shutdown();
LogWriter.getInstance().writeToLogFile(e);
LogWriter.getInstance().writeToLogFile(
"ERROR: Unable to ping server: " + serverIP);
} finally {
executor = null;
System.gc();
}
Any idea how to kill the stream threads?
Not sure what is going on but one bug I see is:
public void kill() {
Thread.currentThread().interrupt();
}
That is interrupting the caller thread, not the gobbler thread. That should be:
public void kill() {
// kill the gobbler thread
interrupt();
}
This is true since the StreamGobbler extends Thread. As always, it is recommended that you implements Runnable and then have a private Thread field if necessary. Then you would do something like thread.interrupt();.
Also, you are not closing your streams correctly. Typically, when I wrap one stream in another, I set the wrapped stream to be null. Also, you are not closing the br BufferedReader. The code should be:
InputStreamReader isr = new InputStreamReader(is);
is = null;
BufferedReader br = null;
try {
br = new BufferedReader(isr);
isr = null;
String line = null;
...
} finally {
// IOException catches not listed
if(br != null){
br.close();
}
if(isr != null){
isr.close();
}
if(is != null){
is.close();
}
}
I'm a big fan of the org.apache.commons.io.IOUtils package that has the closeQuietly(...) method that turns the finally block into:
IOUtils.closeQuietly(br);
IOUtils.closeQuietly(isr);
IOUtils.closeQuietly(is);
I need to execute a command in my java program but after executing the command , it required another parameter ( a password in my case ). how can I manage the output process of Runtime.getRuntime().exec() to accept parameter for further execution ?
I tried new BufferedWriter(new OutputStreamWriter(signingProcess.getOutputStream())).write("123456"); but it did not work.
Does your program not feature a --password option ? Normally all command line based programs do, mainly for scripts.
Runtime.getRuntime().exec(new String[]{"your-program", "--password="+pwd, "some-more-options"});
Or the more complicated way and much more error-prone:
try {
final Process process = Runtime.getRuntime().exec(
new String[] { "your-program", "some-more-parameters" });
if (process != null) {
new Thread(new Runnable() {
#Override
public void run() {
try {
DataInputStream in = new DataInputStream(
process.getInputStream());
BufferedReader br = new BufferedReader(
new InputStreamReader(in));
String line;
while ((line = br.readLine()) != null) {
// handle input here ... ->
// if(line.equals("Enter Password:")) { ... }
}
in.close();
} catch (Exception e) {
// handle exception here ...
}
}
}).start();
}
process.waitFor();
if (process.exitValue() == 0) {
// process exited ...
} else {
// process failed ...
}
} catch (Exception ex) {
// handle exception
}
This sample opens a new thread (keep in mind concurrency and synchronisation) that's going to read the output of your process. Similar you can feed your process with input as long as it has not terminated:
if (process != null) {
new Thread(new Runnable() {
#Override
public void run() {
try {
DataOutputStream out = new DataOutputStream(
process.getOutputStream());
BufferedWriter bw = new BufferedWriter(
new OutputStreamWriter(out));
bw.write("feed your process with data ...");
bw.write("feed your process with data ...");
out.close();
} catch (Exception e) {
// handle exception here ...
}
}
}).start();
}
Hope this helps.
Runtime r=Runtime.getRuntime();
process p=r.exec("your string");
try this way
You should give in parameter your windows command if you work on windows
visit this link for more details : http://docs.oracle.com/javase/6/docs/api/java/lang/Runtime.html