Develop Reference

How do I properly capture the logcat using a process and filtering on PID?

It seems straightforward enough.
I'm using Runtime.getRuntime().exec(commandString);
I've tried
String commandString = "logcat -v raw --pid=" + PID);
Runtime.getRuntime().exec(commandString);
I've tried it without the -v raw and have also tried (and need to) use multiple PIDs using |.
Nothing seems to work. I have a new BufferedReader(new InputStreamReader(p.getInputStream())); that gets nothing. If I don't filter on PID it works but prints out everything that hits the logcat which is not very useful.
What am I missing?
Here is the whole block of the important bit.
try {
Runtime.getRuntime().exec("logcat -c").waitFor();
StringBuilder pids = new StringBuilder("");
for (int i = 0; i < PIDs.size(); i++){
pids.append(Integer.toString(PIDs.get(i)));
if (i < PIDs.size() - 1){
pids.append("|");
}
}
String commmandString = "logcat -v raw --pid=" + pids);
p = Runtime.getRuntime().exec(commmandString);
mBufferedReader = new BufferedReader(new InputStreamReader(p.getInputStream()));
} catch (IOException e) {} catch (InterruptedException e) {}
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
p.destroy();
};
});
mThreadAlive = true;
mLoggingThread.start();

This should help you redirect the output to an InputStream (which you could wrap in a BufferedReader ):
String[] argv =
new String[] { "logcat", "-v", "raw", "--pid=" + pid };
Process process =
new ProcessBuilder(argv)
.inheritIO()
.redirectOutput(ProcessBuilder.Redirect.PIPE)
.start();
try (InputStream in = process.getInputStream()) {
// Consume InputStream
int status = process.waitFor();
if (status != 0) {
throw new IOException(argv + " returned exit status " + status);
}
}

execute julia scripts from Java

I'm coding Julia script with ZeroMQ.
My goal is to communicate with ZMQ between two scripts. Here is an example:
# script1
using ZMQ
ctx = ZMQ.Context()
sockDealer = ZMQ.Socket(ctx, DEALER)
ZMQ.set_identity(sockDealer, "idA")
ZMQ.connect(sockDealer, "tcp://localhost:5555")
ZMQ.send(sockDealer, "hello world!")
ZMQ.close(sockDealer)
ZMQ.close(ctx)
#script2
using ZMQ
function pollrecv(socket::ZMQ.Socket,zmsg::Message)
rc = -1
while true
rc = ccall((:zmq_msg_recv, ZMQ.zmq), Cint, (Ptr{Message}, Ptr{Void}, Cint),
&zmsg, socket.data, ZMQ.ZMQ_DONTWAIT)
if rc == -1
# Base.Libc.EAGAIN = 11
# Problem unsolved: Failure to find Base.Libc.EAGAIN
if !(ZMQ.zmq_errno() == 11)
throw(ZMQ.StateError(ZMQ.jl_zmq_error_str()))
end
return false
else
ZMQ.get_events(socket) != 0 && notify(socket)
break
end
end
return true
end
ctx = ZMQ.Context()
sockRouter = ZMQ.Socket(ctx, ROUTER)
ZMQ.bind(sockRouter, "tcp://*:5555")
fini = false
while !fini
println("listening...")
idSock = Message()
while pollrecv(sockRouter, idSock)
msg = ZMQ.recv(sockRouter)
println("msg recv: " * bytestring(msg))
fini = true
end
sleep(1)
end
ZMQ.close(sockRouter)
ZMQ.close(ctx)
I can execute them with Julia on the command prompt. Everything goes fine. Script 2 can receive the message of Script 1.
Now, I need to execute them from Java. Meaning that I need to create a java project which is just like a controller. Here is my Java project:
public class Container {
private Vector<String[]> commands;
public Container() {
this.commands = new Vector<String[]>();
}
public void addCommand(String[] strs) {
this.commands.addElement(strs);
}
public void execute() {
for(int i = 0; i < this.commands.size(); i++) {
try {
Process p = Runtime.getRuntime().exec(this.commands.get(i));
if(p.waitFor() != 0){
System.err.println("exit value = " + p.exitValue());
}
BufferedReader in = new BufferedReader(new InputStreamReader(p.getInputStream()));
StringBuffer stringBuffer = new StringBuffer();
String line = null;
while((line = in.readLine()) != null){
stringBuffer.append(line + "-");
}
System.out.println(stringBuffer.toString());
} catch (IOException ex) {
// TODO Auto-generated catch block
ex.printStackTrace();
} catch(InterruptedException e){
System.err.println(e);
}
}
}
}
//main
public class Main {
public static void main(String[] args) {
Container c = new Container();
String[] script1 = {"/usr/bin/julia", "/home/thomas/Julia/script1.jl"};
String[] script2 = {"/usr/bin/julia", "/home/thomas/Julia/script2.jl"};
c.addCommand(script1);
c.addCommand(script2);
c.execute();
}
}
However, when I run my java project, I can see that it keeps running but I can't see anything on the console: no result, no message, no error.
I think there is something wrong in my java project.

You'll want to run the two scripts concurrently: script2 is the server script, so it should be running when you run script1. As it is now, Process.waitFor() will wait for script1, the client script, to complete, before executing the server script script2 in the next for iteration.
You could start them as such:
String[] clientScript = { "/usr/bin/julia", "/home/thomas/Julia/script1.jl" };
String[] serverScript = { "/usr/bin/julia", "/home/thomas/Julia/script2.jl" };
Process server = Runtime.getRuntime().exec(serverScript);
Process client = Runtime.getRuntime().exec(clientScript);
and instantiate two threads to read their outputs:
(new ProcessReader(server)).start();
(new ProcessReader(client)).start();
using
public class ProcessReader extends Thread {
private Process p;
public ProcessReader(Process p) {
this.p = p;
}
#Override
public void run() {
BufferedReader in = new BufferedReader(new InputStreamReader(p.getInputStream()));
try {
String line;
while ((line = in.readLine()) != null) {
System.out.println("Read: " + line);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
Although, since the clientScript doesn't produce any output, you could just
start the scripts, and only read the output from the server script - no thread needed.
There's one more thing to consider: the serverScript must be listening... before the clientScript attempts to connect. So you may want to do this:
Process server = Runtime.getRuntime().exec(serverScript);
BufferedReader in = new BufferedReader(new InputStreamReader(server.getInputStream()));
if ( in.readLine().equals("listening...") ) {
Process client = Runtime.getRuntime().exec(clientScript);
String line;
while ( (line=in.readLine()) != null )
System.out.println("Read: " + line );
}

Since this is not a specific answer to your question, this may help to you or other users that attempt to similar jobs.
JuliaCaller is an early stage library for calling Julia from Java. It executes the Julia executable as a Java process and runs a script in the Julia side. This script opens a TCP server that listens on a given port number. Every command, statement or expression sent from Java is then executed and results are sent back to Java in JSON format.
This library also implements the standard javax.script interface, that means, Julia libraries, functions, and programs can run like a scripting language that is implemented in Java (mimics).
Here is the example:
Constants.setProperties(Constants.JULIA_PATH, "/usr/local/bin/julia");
Constants.setProperties(Constants.JULIA_PORT, "8001");
// Creating a scripting interface for Julia
manager = new ScriptEngineManager();
engine = manager.getEngineByName("Julia");
// Sending command 'a = 3' to Julia from Java
engine.eval("a = 3");
// Handling the result in Java
Object a = engine.get("a");
More examples are given in the GitHub page.
Source code with Apache License

Problems again with child processes in Java

I am on Ubuntu 14.04.
I am trying to run something like ps aux | grep whatevah through Java's class ProcessBuilder. I create two child processes and I make them communicate synchronously, but for some reason, I can not see anything in the terminal.
This is the code:
try {
// What comes out of process1 is our inputStream
Process process1 = new ProcessBuilder("ps", "aux").start();
InputStream is1 = process1.getInputStream();
BufferedReader br1 = new BufferedReader (new InputStreamReader(is1));
// What goes into process2 is our outputStream
Process process2 = new ProcessBuilder("grep", "gedit").start();
OutputStream os = process2.getOutputStream();
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(os));
// Send the output of process1 to the input of process2
String p1Output = null;
while ((p1Output = br1.readLine()) != null) {
bw.write(p1Output);
System.out.println(p1Output);
}
// Synchronization
int finish = process2.waitFor();
System.out.println(finish);
// What comes out of process2 is our inputStream
InputStream is2 = process2.getInputStream();
BufferedReader br2 = new BufferedReader(new InputStreamReader(is2));
String combOutput = null;
while ((combOutput = br2.readLine()) != null)
System.out.println(combOutput);
os.close();
is1.close();
is2.close();
} catch (IOException e) {
System.out.println("Command execution error: " + e.getMessage());
} catch (Exception e) {
System.out.println("General error: " + e.getMessage());
}
(The System.out.println(p1Output); is just for me to check, the print that has to work is the last one, printing the result of ps aux | grep whatevah.)
I've tried several things, the less silly include:
If I comment everything regarding process2, I get the result of ps aux printed on the terminal
If I run the program as is, it prints nothing to the terminal.
If I uncomment the waitFor call, only ps aux gets printed.
If change the commands to, for example, ls -al and ls -al, then both get printed.
I tried changing "aux" for "aux |" but still nothing is printed.
Closed the buffers, also nothing
etc.
Any help will be sorely appreciated.
Cheers!
EDIT
Minutes after accepting Ryan's amazing answer I made my last try to make this code work. And I succeeded! I changed:
while ((p1Output = br1.readLine()) != null) {
bw.write(p1Output);
System.out.println(p1Output);
}
for:
while ((p1Output = br1.readLine()) != null) {
bw.write(p1Output + "\n");
System.out.println(p1Output);
}
bw.close();
and it works! I remember closing the buffer before, so I don't know what went wrong. Turns out you should not stay awake until late trying to make a piece of code work XD.
Ryan's answer down here is still amazing, though.

Given the advice in the comments, the important thing to note is the necessity to use threads to process input/output for a process in order to achieve what you want.
I've used the link posted by jtahlborn and adapted this solution that you might be able to use.
I created a simple example that will list files in a directory and grep through the output.
This example simulates the command ls -1 | grep some from a directory called test with three files somefile.txt someotherfile.txt and this_other_file.csv
EDIT: The original solution didn't really fully use the "pipe" methodology, as it was waiting fully for p1 to finish before starting p2. Rather, it should start them both, and then the output of the first should be piped to the second. I've updated the solution with a class that accomplishes this.
import java.io.*;
import java.util.Scanner;
public class Main {
public static void main(String[] args) {
try {
// construct a process
ProcessBuilder pb1 = new ProcessBuilder("ls", "-1");
// set working directory
pb1.directory(new File("test"));
// start process
final Process process1 = pb1.start();
// get input/error streams
final InputStream p1InStream = process1.getInputStream();
final InputStream p1ErrStream = process1.getErrorStream();
// handle error stream
Thread t1Err = new InputReaderThread(p1ErrStream, "Process 1 Err");
t1Err.start();
// this will print out the data from process 1 (for illustration purposes)
// and redirect it to process 2
Process process2 = new ProcessBuilder("grep", "some").start();
// process 2 streams
final InputStream p2InStream = process2.getInputStream();
final InputStream p2ErrStream = process2.getErrorStream();
final OutputStream p2OutStream = process2.getOutputStream();
// do the same as process 1 for process 2...
Thread t2In = new InputReaderThread(p2InStream, "Process 2 Out");
t2In.start();
Thread t2Err = new InputReaderThread(p2ErrStream, "Process 2 Err");
t2Err.start();
// create a new thread with our pipe class
// pass in the input stream of p1, the output stream of p2, and the name of the input stream
new Thread(new PipeClass(p1InStream, p2OutStream, "Process 1 Out")).start();
// wait for p2 to finish
process2.waitFor();
} catch (IOException e) {
System.out.println("Command execution error: " + e.getMessage());
} catch (Exception e) {
System.out.println("General error: " + e.getMessage());
}
}
}
This is a class that will be used to simulate a process pipe. It uses some loops to copy bytes around, and could be more efficient, depending on your needs, but for the illustration, it should work.
// this class simulates a pipe between two processes
public class PipeClass implements Runnable {
// the input stream
InputStream is;
// the output stream
OutputStream os;
// the name associated with the input stream (for printing purposes only...)
String isName;
// constructor
public PipeClass(InputStream is, OutputStream os, String isName) {
this.is = is;
this.os = os;
this.isName = isName;
}
#Override
public void run() {
try {
// use a byte array output stream so we can clone the data and use it multiple times
ByteArrayOutputStream baos = new ByteArrayOutputStream();
// read the data into the output stream (it has to fit in memory for this to work...)
byte[] buffer = new byte[512]; // Adjust if you want
int bytesRead;
while ((bytesRead = is.read(buffer)) != -1) {
baos.write(buffer, 0, bytesRead);
}
// clone it so we can print it out
InputStream clonedIs1 = new ByteArrayInputStream(baos.toByteArray());
Scanner sc = new Scanner(clonedIs1);
// print the info
while (sc.hasNextLine()) {
System.out.println(this.isName + " >> " + sc.nextLine());
}
// clone again to redirect to the output of the other process
InputStream clonedIs2 = new ByteArrayInputStream(baos.toByteArray());
buffer = new byte[512]; // Adjust if you want
while ((bytesRead = clonedIs2.read(buffer)) != -1) {
// write it out to the output stream
os.write(buffer, 0, bytesRead);
}
}
catch (IOException ex) {
ex.printStackTrace();
}
finally {
try {
// close so the process will finish
is.close();
os.close();
}
catch (Exception ex) {
ex.printStackTrace();
}
}
}
}
This is a class that was created for handling process output, adapted from this reference
// Thread reader class adapted from
// http://www.javaworld.com/article/2071275/core-java/when-runtime-exec---won-t.html
public class InputReaderThread extends Thread {
// input stream
InputStream is;
// name
String name;
// is there data?
boolean hasData = false;
// data itself
StringBuilder data = new StringBuilder();
// constructor
public InputReaderThread(InputStream is, String name) {
this.is = is;
this.name = name;
}
// set if there's data to read
public synchronized void setHasData(boolean hasData) {
this.hasData = hasData;
}
// data available?
public boolean hasData() { return this.hasData; }
// get the data
public StringBuilder getData() {
setHasData(false); // clear flag
StringBuilder returnData = this.data;
this.data = new StringBuilder();
return returnData;
}
#Override
public void run() {
// input reader
InputStreamReader isr = new InputStreamReader(this.is);
Scanner sc = new Scanner(isr);
// while data remains
while ( sc.hasNextLine() ) {
// print out and append to data
String line = sc.nextLine();
System.out.println(this.name + " >> " + line);
this.data.append(line + "\n");
}
// flag there's data available
setHasData(true);
}
}
The produced output is:
Process 1 Out >> somefile.txt
Process 1 Out >> someotherfile.txt
Process 1 Out >> this_other_file.csv
Process 2 Out >> somefile.txt
Process 2 Out >> someotherfile.txt
To show that piping is really working, changing the command to ps -a | grep usr the output is:
Process 1 Out >> PID PPID PGID WINPID TTY UID STIME COMMAND
Process 1 Out >> I 15016 1 15016 15016 con 400 13:45:59 /usr/bin/grep
Process 1 Out >> 15156 1 15156 15156 con 400 14:21:54 /usr/bin/ps
Process 1 Out >> I 9784 1 9784 9784 con 400 14:21:54 /usr/bin/grep
Process 2 Out >> I 15016 1 15016 15016 con 400 13:45:59 /usr/bin/grep
Process 2 Out >> 15156 1 15156 15156 con 400 14:21:54 /usr/bin/ps
Process 2 Out >> I 9784 1 9784 9784 con 400 14:21:54 /usr/bin/grep
Seeing the grep command in process 2's output shows that the piping is working, with the old solution I posted, this would be missing.
Note the handling of the error stream, which is always good practice, even if you don't plan to use it.
This is a quick and dirty solution that could benefit from some additional thread management techniques, but it should get you what you want.

How to run a bat by java and can stop it when time out?

I need a function whose argument is bat's fileName and a float means timeOut. I've used Process in java to do it. But if I want to stop it, I find the p.destroy() cannot stop the exe file which be called by bat file and still runs.So ,how can I stop it like "Ctrl"+"C" in cmd?
public void exec(String path, float timeOutFloat) throws IOException, InterruptedException {
Runtime r = Runtime.getRuntime();
Process p = r.exec(new String[] { path });
ThreadReadExec2 thread = new ThreadReadExec2(p.getInputStream());
thread.start();
long timeOut = (long) ((float) timeOutFloat) * 1000;
long time = 0;
long onceTime = 100;
while (thread.isAlive()) {
Thread.sleep(onceTime);
time += onceTime;
if (time > timeOut) {
p.destroy();
Thread.sleep(onceTime);
}
}
int res = p.waitFor();
System.out.println("res:" + res);
}
class ThreadReadExec2 extends Thread {
InputStream input;
public ThreadReadExec2(InputStream input) {
this.input = input;
}
#Override
public void run() {
BufferedReader ir = new BufferedReader(new InputStreamReader(input));
try {
String line;
while ((line = ir.readLine()) != null) {
System.out.println(line);
}
} catch (IOException e) {
}
}
}

The reason that this is happening is likely because the destroy method is killing the command shell you invoked via the batch file, but not the process that that batch file in turn invoked.
You essentially need to kill the process tree of the batch file.
There are some methods in this question that suggest ways to accomplish this in Java, although it's somewhat roundabout (link to the one that seems the most promising):
https://stackoverflow.com/a/7627648/3583500

Java blocks after calling runtime.exec()

First some code:
Runtime runtime = Runtime.getRuntime();
String args[] = new String[2];
// args[0] = "/bin/bash";
// args[1] = "-c";
// args[2] = "/usr/bin/rpm2cpio "+archiveFile.getCanonicalPath()+" | /bin/cpio -idmv";
args[0] = "/usr/bin/rpm2cpio";
args[1] = archiveFile.getCanonicalPath();
Process rpm2cpioProcess = runtime.exec(args, null, dir);
// System.out.println("started rpm2cpio");
String args2[] = new String[3];
args2[0] = "/bin/cpio";
args2[1] = "-idmu";
args2[2] = "--quiet";
Process cpioProcess = runtime.exec(args2, null, dir);
// System.out.println("started cpio");
InputStream fromRpm2cpio = rpm2cpioProcess.getInputStream();
new ProcessInputStreamer(rpm2cpioProcess.getErrorStream());
OutputStream fromCpio = cpioProcess.getOutputStream();
new PipedStreamer(fromRpm2cpio, fromCpio);
new ProcessInputStreamer(cpioProcess.getErrorStream());
// System.out.println("pipe created");
while(cpioProcess!=null && fromRpm2cpio!=null) {
boolean doSleep = true;
// System.out.println("waking up");
if (cpioProcess!=null) {
try {
if (cpioProcess.exitValue()==0) {
cpioProcess = null;
doSleep = false;
}
} catch(IllegalThreadStateException e) {
}
}
if (rpm2cpioProcess!=null) {
try {
if (rpm2cpioProcess.exitValue()==0) {
rpm2cpioProcess = null;
doSleep = false;
}
} catch(IllegalThreadStateException e) {
}
}
if (doSleep) {
Thread.sleep(30);
}
// System.out.println("still running");
}
I'm trying to extract the content of an rpm archive. This code works fine after multiple modifications. My first attempt was to execute the next code through Java:
/bin/bash -c '/usr/bin/rpm2cpio <archive-file> | /bin/cpio -idmv'
Which worked fine the first time I ran it (you can see it in the code commented above). The second time I ran the code it got blocked since the extracted files already existed. So I thought maybe it has to do with the piping and thus split the call into two separate processes. This didn't help much either. So I then modified the arguments of the /bin/cpio from '-idmv' to '-idmu --quiet' and now it works. Unfortunately the -u option overwrites existing files 'unconditionally' which is not really needed. My question is why does it block with -idmv and why doesn't it block with -idmu ?

It could be waiting on standard input for some inputs. Redirect your standard input and/or output to /dev/null

I'd guess that your ProcessInputStreamer and/or PipedStreamer implement Runnable or extent Thread and you're not running them anywhere.