How can i get the stderr of a python binary? It is showing the output of the python script but it is not showing the errors that might be in the script, what java method should i be using? or is there a python command line argument that i can use to display the errors?
private String exec(String command)
{
try
{
String s = getApplicationInfo().dataDir;
File file2 = new File(s+"/py");
file2.setExecutable(true);
File externalStorage = Environment.getExternalStorageDirectory();
String strUri = externalStorage.getAbsolutePath();
PrintWriter out = new PrintWriter(strUri+"/temp.py");
out.write(command);
out.close();
saveRawToFile();
Process process = Runtime.getRuntime().exec(s+"/py "+strUri+"/temp.py");
BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
int read;
char[] buffer = new char[4096];
StringBuffer output = new StringBuffer();
while ((read = reader.read(buffer)) > 0)
{
output.append(buffer, 0, read);
}
reader.close(); process.waitFor();
return output.toString();
}
catch (IOException e)
{ throw new RuntimeException(e); }
catch (InterruptedException e)
{ throw new RuntimeException(e); }
}
private void output(final String str)
{
Runnable proc = new Runnable() {
public void run()
{
outputView.setText(str);
outputView.setTextIsSelectable(true);
}
};
handler.post(proc);
}
Use Process#getErrorStream(), just like you used getOutputStream().
Related
I have an imagen in a variable of type jpeg or png and i am triying to convert it with command line jpegoptim or pngquant via java 1.6.
The problem is that i cannot make it work. I read a lot of answer but i did not find the answer.
this is the last function i tried:
public int execute(byte[] image) throws IOException {
ProcessBuilder processBuilder = new ProcessBuilder(commandLine);
final Process process = processBuilder.start();
final ByteArrayOutputStream outputBuffer = new ByteArrayOutputStream();
final ByteArrayOutputStream errorBuffer = new ByteArrayOutputStream();
try {
// Handle stdout...
new Thread() {
public void run() {
try {
IOUtils.copy(process.getInputStream(), outputBuffer);
} catch (Exception e) {
LOG.error("Error executing " + commandLine,e);
}
}
}.start();
// Handle sderr...
new Thread() {
public void run() {
try {
IOUtils.copy(process.getErrorStream(), errorBuffer);
} catch (Exception e) {
LOG.error("Error executing " + commandLine,e);
}
}
}.start();
process.getOutputStream().write(image);
process.getOutputStream().flush();
process.waitFor();
result = outputBuffer.toByteArray();
errorMsg = errorBuffer.toString("ASCII");
} catch (InterruptedException e) {
LOG.error("Error executing " + commandLine,e);
} finally {
if( process != null ) {
close(process.getErrorStream());
close(process.getOutputStream());
close(process.getInputStream());
process.destroy();
}
}
return process.exitValue();
}
and commandline is for jpg:
this.commandLine = new ArrayList<String>();
this.commandLine.add("jpegoptim");
this.commandLine.add("-m90");
this.commandLine.add("--stdout");
this.commandLine.add("--stdin");
and for png:
this.commandLine = new ArrayList<String>();
this.commandLine.add("pngquant");
this.commandLine.add("--quality=70-80");
this.commandLine.add(">stdout");
this.commandLine.add("<stdin");
The problem with this code is broken pipe. I tried different ways of code. I read different forum here but I dont know how to make this work.
Thanks in advance.
EDIT:
As LinuxDisciple suggested I changed the command to:
List<String> commandLine = new ArrayList<String>();
commandLine.add("tee");
commandLine.add("-a");
commandLine.add("logfile.log");
And the code to:
byte[] image = jpg.read(args[0]); // path to file...
ProcessBuilder processBuilder = new ProcessBuilder(commandLine);
final Process process = processBuilder.start();
OutputStream stdin = process.getOutputStream ();
final InputStream stderr = process.getErrorStream ();
final InputStream stdout = process.getInputStream ();
Thread errorT = new Thread() {
public void run() {
byte[] buf = new byte[1024];
int len;
try {
while ((len = stdout.read(buf)) != -1) {
// process byte buffer
System.out.write(buf, 0, len);
System.out.flush();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
};
errorT.start();
Thread outputT = new Thread() {
public void run() {
byte[] buf = new byte[1024];
int len;
try {
while ((len = stderr.read(buf)) != -1) {
System.err.write(buf, 0, len);
System.err.flush();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
};
outputT.start();
stdin.write(image);
stdin.flush();
//stdin.close();
try {
process.waitFor();
} catch (InterruptedException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
stdin.close();
stderr.close();
stdout.close();
The waitfor never returns... what is missing?
If I put "stdin.close();" after the flush (in the code above is commented) the process ends. but the stdout is not fully processed and an error is shown: java.io.IOException: Stream closed twice (one for each thread). The logfile.log is equal to the image, but the stdout is truncated.
I test this by command line:
java -cp testProc.jar testProc.Test image.jpg > image_new.jpg
logfile.log is equal to image.jpg
image_new.jpg is smaller and a truncated version of image.jpg.
some clue?
Start with your last two lines. You don't pass stdin using >stdin. Look for some ProcessBuilder examples to see how to use a BufferedWriter to write to the process's stdin (your Java process's stdout), then do the same for your subprocess' stdout with a BufferedReader.
If you still have issues after that, then for diagnosis, you could try replacing jpegoptim and pngquant with tee -a logfile, which will take stdin and just pipe it right back to stdout (and to a file called logfile). Then you've got a file that has all the piped-in data, or you'll get an empty file and the pipe error because your Java program isn't passing in the data and you can see if it looks right. If instead you get a file with data and you don't get a pipe error, it probably means jpegoptim isn't returning any data. At that point you could pipe the data from logfile into jpegoptim on the command line and play with the options until you get it working, and then use those working options in your Java code.
Finally I made it work!
I found the solution by pure fortune!...
I discommented the line:
stdin.close();
And added just after the line "process.waitFor();": this lines:
errorT.join();
outputT.join();
To process the images the commands are:
commandLine.add("jpegoptim");
commandLine.add("-m90");
commandLine.add("--stdout");
commandLine.add("--stdin");
and:
commandLine.add("pngquant");
commandLine.add("--quality=70-80");
commandLine.add("-");
The final code:
public int execute(List<String> commandLine, byte[] image) throws IOException {
ProcessBuilder processBuilder = new ProcessBuilder(commandLine);
final Process process = processBuilder.start();
OutputStream stdin = process.getOutputStream ();
final InputStream stderr = process.getErrorStream ();
final InputStream stdout = process.getInputStream ();
final ByteArrayOutputStream bos = new ByteArrayOutputStream();
Thread errorT = new Thread() {
public void run() {
byte[] buf = new byte[1024];
int len;
try {
while ((len = stdout.read(buf)) != -1) {
// process byte buffer
bos.write(buf, 0, len);
bos.flush();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
};
errorT.start();
Thread outputT = new Thread() {
public void run() {
byte[] buf = new byte[1024];
int len;
try {
while ((len = stderr.read(buf)) != -1) {
System.err.write(buf, 0, len);
System.err.flush();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
};
outputT.start();
stdin.write(image);
stdin.flush();
stdin.close();
try {
process.waitFor();
errorT.join();
outputT.join();
} catch (InterruptedException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
stdin.close();
stderr.close();
stdout.close();
System.out.write(bos.toByteArray());
return 0;
}
I'm trying to webget some bz2 files from Wikipedia, I don't care whether they are save as bz2 or unpacked, since I can unzip them locally.
When I call:
public static void getZip(String theUrl, String filename) throws IOException {
URL gotoUrl = new URL(theUrl);
try (InputStreamReader isr = new InputStreamReader(new BZip2CompressorInputStream(gotoUrl.openStream())); BufferedReader in = new BufferedReader(isr)) {
StringBuffer sb = new StringBuffer();
String inputLine;
// grab the contents at the URL
while ((inputLine = in.readLine()) != null) {
sb.append(inputLine + "\r\n");
}
// write it locally
Wget.createAFile(filename, sb.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
throw ioe;
}
}
I get a part of the unzipped file, never more than +- 883K.
When I don't use the BZip2CompressorInputStream, like:
public static void get(String theUrl, String filename) throws IOException {
try {
URL gotoUrl = new URL(theUrl);
InputStreamReader isr = new InputStreamReader(gotoUrl.openStream());
BufferedReader in = new BufferedReader(isr);
StringBuffer sb = new StringBuffer();
String inputLine;
// grab the contents at the URL
while ((inputLine = in.readLine()) != null) {
sb.append(inputLine);// + "\r\n");
}
// write it locally
Statics.writeOut(filename, false, sb.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
throw ioe;
}
}
I get a file of which the size is the same as it suppose to (compared to the KB not B). But also a message that that the zipped file is damaged, also when using byte [] instead of readLine(), like:
public static void getBytes(String theUrl, String filename) throws IOException {
try {
char [] cc = new char[1024];
URL gotoUrl = new URL(theUrl);
InputStreamReader isr = new InputStreamReader(gotoUrl.openStream());
BufferedReader in = new BufferedReader(isr);
StringBuffer sb = new StringBuffer();
// grab the contents at the URL
int n = 0;
while (-1 != (n = in.read(cc))) {
sb.append(cc);// + "\r\n");
}
// write it locally
Statics.writeOut(filename, false, sb.toString());
} catch (MalformedURLException mue) {
mue.printStackTrace();
} catch (IOException ioe) {
throw ioe;
}
}
Finally, when I bzip2 the inputstream and outputstream, I get a valid bzip2 file, but of the size like the first one, using:
public static void getWriteForBZ2File(String urlIn, final String filename) throws CompressorException, IOException {
URL gotoUrl = new URL(urlIn);
try (final FileOutputStream out = new FileOutputStream(filename);
final BZip2CompressorOutputStream dataOutputStream = new BZip2CompressorOutputStream(out);
final BufferedInputStream bis = new BufferedInputStream(gotoUrl.openStream());
final CompressorInputStream input = new CompressorStreamFactory().createCompressorInputStream(bis);
final BufferedReader br2 = new BufferedReader(new InputStreamReader(input))) {
String line = null;
while ((line = br2.readLine()) != null) {
dataOutputStream.write(line.getBytes());
}
}
}
So, how do I get the entire bz2 file, in either bz2 format or unzipped?
A bz2 file contains bytes, not characters. You can't read it as if it contained characters, with a Reader.
Since all you want to do is download the file and save it locally, all you need is
Files.copy(gotoUrl.openStream(), Paths.get(fileName));
How can I add a executable into assets and run it in Android and show the output?
I've a executable that will work. I assume there will need to be some chmod in the code.
Thank you.
here is my answer
put copyAssets() to your mainactivity.
someone's code:
private void copyAssets() {
AssetManager assetManager = getAssets();
String[] files = null;
try {
files = assetManager.list("");
} catch (IOException e) {
Log.e("tag", "Failed to get asset file list.", e);
}
for(String filename : files) {
InputStream in = null;
OutputStream out = null;
try {
in = assetManager.open(filename);
File outFile = new File(getFilesDir(), filename);
out = new FileOutputStream(outFile);
copyFile(in, out);
} catch(IOException e) {
Log.e("tag", "Failed to copy asset file: " + filename, e);
}
finally {
if (in != null) {
try {
in.close();
} catch (IOException e) {
// NOOP
}
}
if (out != null) {
try {
out.close();
} catch (IOException e) {
// NOOP
}
}
}
}
}
private void copyFile(InputStream in, OutputStream out) throws IOException {
byte[] buffer = new byte[1024];
int read;
while ((read = in.read(buffer)) != -1) {
out.write(buffer, 0, read);
}
}
also here is code to run command
public String runcmd(String cmd){
try {
Process p = Runtime.getRuntime().exec(cmd);
BufferedReader in = new BufferedReader(new InputStreamReader(p.getInputStream()));
int read;
char[] buffer = new char[4096];
StringBuffer out = new StringBuffer();
while ((read = in.read(buffer)) > 0) {
out.append(buffer, 0, read);
}
in.close();
p.waitFor();
return out.substring(0);
} catch (IOException e) {
throw new RuntimeException(e);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
you may need to change it to
String prog= "programname";
String[] env= { "parameter 1","p2"};
File dir= new File(getFilesDir().getAbsolutePath());
Process p = Runtime.getRuntime().exec(prog,env,dir);
to ensure proper parameter handling
also add this to your main code
to check proper copying of files
String s;
File file4 = new File(getFilesDir().getAbsolutePath()+"/executable");
file4.setExecutable(true);
s+=file4.getName();
s+=file4.exists();
s+=file4.canExecute();
s+=file4.length();
//output s however you want it
should write: filename, true, true, correct filelength.
Place your executable in raw folder, then run it by using ProcessBuilder or Runtime.exec like they do here http://gimite.net/en/index.php?Run%20native%20executable%20in%20Android%20App
I'm fairly new to ProcessBuilder and working with threads. In it's current state I have a J-Button which starts a scheduled executor service. The scheduled executor service is used to delegate a process to one of two process builders. The application is meant to record a user conversation. During the conversation, after x minutes it creates a wav and delegates it to an available process for transcription. The problem begins when the transcription class is called. The process is started and the application runs as expected. However, the transcription process doesn't actually do anything until I exit the parent application. Only then it will begin. Checking the task manager it shows as a process but uses 0.0% of the CPU and around 238MB of memory until I exit then the two processes jump to 30%-40% and 500-1000 MB of memory. Also, I am using the .waitFor() but am using a thread to run the .waitFor() process as from what I gather it causes the application to hang. How would I go about fixing this. Sorry I am unable to provide more details but I'm new to this. Thanks in advance!
public class TranDelegator {
Future<?> futureTranOne = null;
Future<?> futureTranTwo = null;
ExecutorService transcriberOne = Executors.newFixedThreadPool(1);
ExecutorService transcriberTwo = Executors.newFixedThreadPool(1);
final Runnable transcribeChecker = new Runnable() {
public void run() {
String currentWav = null;
File inputFile = new File("C:\\convoLists/unTranscribed.txt");
BufferedReader reader = null;
try {
reader = new BufferedReader(new FileReader(inputFile));
} catch (FileNotFoundException e1) {
System.out.println("reader didn't initialize");
e1.printStackTrace();
}
try {
currentWav = reader.readLine();
} catch (IOException e) {
System.out.println("currentWav string issue");
e.printStackTrace();
}
try {
reader.close();
} catch (IOException e) {
System.out.println("reader couldn't close");
e.printStackTrace();
}
if(currentWav != null){
if (futureTranOne == null || futureTranOne.isDone()) {
futureTranOne = transcriberOne.submit((transcriptorOne));
}
else if (futureTranTwo == null || futureTranTwo.isDone()) {
futureTranTwo = transcriberTwo.submit((transcriptorTwo));
}
}
}
};
final Runnable transcriptorOne = new Runnable() {
public void run() {
System.out.println("ONE");
try {
String classpath = System.getProperty("java.class.path");
String path = "C:/Program Files/Java/jre7/bin/java.exe";
ProcessBuilder processBuilder = new ProcessBuilder(path, "-cp",
classpath, Transcriber.class.getName());
Process process = processBuilder.start();
try {
process.waitFor();
} catch (InterruptedException e) {
System.out.println("process.waitFor call failed");
e.printStackTrace();
}
} catch (IOException e) {
System.out.println("Unable to call transcribeConvo");
e.printStackTrace();
}
}
};
final Runnable transcriptorTwo = new Runnable() {
public void run() {
System.out.println("TWO");
try {
String classpath = System.getProperty("java.class.path");
String path = "C:/Program Files/Java/jre7/bin/java.exe";
ProcessBuilder processBuilder = new ProcessBuilder(path, "-cp",
classpath, Transcriber.class.getName());
Process process = processBuilder.start();
try {
process.waitFor();
} catch (InterruptedException e) {
System.out.println("process.waitFor call failed");
e.printStackTrace();
}
} catch (IOException e) {
System.out.println("Unable to call transcribeConvo");
e.printStackTrace();
}
}
};
}
public class Transcriber {
public static void main(String[] args) throws IOException,
UnsupportedAudioFileException {
retreiveEmpInfo();
TextoArray saveConvo = new TextoArray();
ArrayList<String> entireConvo = new ArrayList();
URL audioURL;
String currentWav = wavFinder();
ConfigReader configuration = new ConfigReader();
ArrayList<String> serverInfo = configuration
.readFromDoc("serverconfig");
while (currentWav != null) {
audioURL = new URL("file:///" + currentWav);
URL configURL = Transcriber.class.getResource("config.xml");
ConfigurationManager cm = new ConfigurationManager(configURL);
Recognizer recognizer = (Recognizer) cm.lookup("recognizer");
recognizer.allocate(); // allocate the resource necessary for the
// recognizer
System.out.println(configURL);
// configure the audio input for the recognizer
AudioFileDataSource dataSource = (AudioFileDataSource) cm
.lookup("audioFileDataSource");
dataSource.setAudioFile(audioURL, null);
// Loop until last utterance in the audio file has been decoded, in
// which case the recognizer will return null.
Result result;
while ((result = recognizer.recognize()) != null) {
String resultText = result.getBestResultNoFiller();
// System.out.println(result.toString());
Collections.addAll(entireConvo, resultText.split(" "));
}
new File(currentWav).delete();
saveConvo.Indexbuilder(serverInfo, entireConvo);
entireConvo.clear();
currentWav = wavFinder();
}
System.exit(0);
}
private static String wavFinder() throws IOException {
String currentWav = null;
int x = 1;
File inputFile = new File("C:\\convoLists/unTranscribed.txt");
File tempFile = new File("C:\\convoLists/unTranscribedtemp.txt");
BufferedReader reader = new BufferedReader(new FileReader(inputFile));
BufferedWriter writer = new BufferedWriter(new FileWriter(tempFile));
String currentLine = null;
String newLine = System.getProperty("line.separator");
while ((currentLine = reader.readLine()) != null) {
if (x == 1) {
currentWav = currentLine;
} else {
writer.write(currentLine);
writer.write(newLine);
}
x = 2;
}
reader.close();
writer.flush();
writer.close();
inputFile.delete();
// boolean successful =
tempFile.renameTo(inputFile);
// System.out.println("Success: " + successful);
// System.out.println("currentWav = " + currentWav);
return currentWav;
}
private static void retreiveEmpInfo() throws IOException {
File tempFile = new File("C:\\convoLists/tmp.txt");
BufferedReader reader = new BufferedReader(new FileReader(tempFile));
CurrentEmployeeInfo.setName(reader.readLine());
CurrentEmployeeInfo.setUserEmail(reader.readLine());
CurrentEmployeeInfo.setManagerEmail(reader.readLine());
reader.close();
}
}
This problem may be related to sub-process's input stream buffers.
You should clear the sub-process's input stream buffers.
These stream buffers got increased within the parent process's memory with time and at some moment your sub-process will stop responding.
There are few options to make sub-process work normally
Read continuously from sub-process's input streams
Redirect sub-process's input streams
Close sub-process's input streams
Closing sub-process's input streams
ProcessBuilder processBuilder = new ProcessBuilder(command);
Process process = processBuilder.start();
InputStream inStream = process.getInputStream();
InputStream errStream = process.getErrorStream();
try {
inStream.close();
errStream.close();
} catch (IOException e1) {
}
process.waitFor();
Reading sub-process's input streams
ProcessBuilder processBuilder = new ProcessBuilder(command);
Process process = processBuilder.start();
InputStreamReader tempReader = new InputStreamReader(new BufferedInputStream(p.getInputStream()));
final BufferedReader reader = new BufferedReader(tempReader);
InputStreamReader tempErrReader = new InputStreamReader(new BufferedInputStream(p.getErrorStream()));
final BufferedReader errReader = new BufferedReader(tempErrReader);
try {
while ((line = reader.readLine()) != null) {
}
} catch (IOException e) {
}
try {
while ((line = errReader.readLine()) != null) {
}
} catch (IOException e) {
}
process.waitFor();
Redirecting sub-process's input streams
ProcessBuilder processBuilder = new ProcessBuilder(command);
processBuilder.redirectInput();
processBuilder.redirectError();
Process process = processBuilder.start();
process.waitFor();
(from comments)
Looks like process hang is due to out/error streams becoming full. You need to consume these streams; possibly via a thread.
Java7 provides another way to redirect output.
Related : http://alvinalexander.com/java/java-exec-processbuilder-process-3
I'd like to send a file contained in a ZIP archive unzipped to an external program for further decoding and to read the result back into Java.
ZipInputStream zis = new ZipInputStream(new FileInputStream(ZIPPATH));
Process decoder = new ProcessBuilder(DECODER).start();
???
BufferedReader br = new BufferedReader(new InputStreamReader(
decoder.getInputStream(),"us-ascii"));
for (String line = br.readLine(); line!=null; line = br.readLine()) {
...
}
What do I need to put into ??? to pipe the zis content to the decoder.getOutputStream()? I guess a dedicated thread is needed, as the decoder process might block when its output is not consumed.
Yes a thread is needed (or you wait/block until the copy is finished) for copying the InputStream to the OutputStream. Check the org.apache.commons.net.io.Util class for several helper methods to copy the data.
Ok, I got as far as following:
public class CopyStream extends Thread {
static final int BUFFERSIZE = 10 * 1024;
InputStream input; OutputStream output;
boolean closeInputOnExit, closeOutputOnExit, flushOutputOnWrite;
public IOException ex;
public CopyStream (InputStream input, boolean closeInputOnExit, OutputStream output, boolean closeOutputOnExit,
boolean flushOutputOnWrite) {
super("CopyStream");
this.input = input; this.closeInputOnExit = closeInputOnExit;
this.output = output; this.closeOutputOnExit = closeOutputOnExit;
this.flushOutputOnWrite = flushOutputOnWrite;
start();
}
public void run () {
try {
byte[] buffer = new byte[BUFFERSIZE];
for (int bytes = input.read(buffer); bytes>=0; bytes = input.read(buffer)) {
output.write(buffer,0,bytes);
if (flushOutputOnWrite) output.flush();
}
} catch (IOException ex) {
this.ex = ex;
} finally {
if (closeInputOnExit) {
try {
input.close();
} catch (IOException ex) {
if (this.ex==null) this.ex = ex;
}
}
if (closeOutputOnExit) {
try {
output.close();
} catch (IOException ex) {
if (this.ex==null) this.ex = ex;
}
}
}
}
}
Then the code would look as following:
ZipInputStream zis = new ZipInputStream(new FileInputStream(ZIPPATH));
for (ZipEntry ze = zis.getNextEntry(); ze!=null; ze = zis.getNextEntry()) {
Process decoder = new ProcessBuilder(EXTERNALPROCESSOR).start();
CopyStream cs1 = new CopyStream(is,false,decoder.getOutputStream(),true,true);
CopyStream cs2 = new CopyStream(decoder.getErrorStream(),true,System.err,false,true);
BufferedReader br = new BufferedReader(new InputStreamReader(decoder.getInputStream(),"us-ascii"));
ArrayList<String> lines = new ArrayList<String>();
for (String line = br.readLine(); line!=null; line = br.readLine()) {
lines.add(line);
}
if (decoder.exitValue()!=0) throw new IOException("Decoder exits with "+decoder.exitValue());
try {
cs1.join(100);
} catch (InterruptedException ex) {
throw new IOException(ex);
}
if (cs1.isAlive()) throw new IOException("cs1 not terminated");
if (cs1.ex!=null) throw cs1.ex;
try {
cs2.join(100);
} catch (InterruptedException ex) {
throw new IOException(ex);
}
if (cs2.isAlive()) throw new IOException("cs2 not terminated");
if (cs2.ex!=null) throw cs2.ex;
for (String line: lines) {
processline(line);
}
}
However, I find this a bit fragile. Isn't this a pattern for which some more robust implementation is around?