Develop Reference

Java program continues to run (in background) even after last statement in main

Background
I am having a simple java program which reads data from std input. I am executing it though from a bash script executer.sh
python2 readLines.py | tee logfile | java MsgReader
readLines.py reads data from a file line by line and throws it on the stdout
MsgReader.java
import kafka.javaapi.producer.Producer;
public void process() {
String msg = "";
BufferedReader stdinReader = new BufferedReader(new InputStreamReader(System.in)
Producer producer = new Producer();//it takes config file as parameter which is not related to the question here
try {
while ((msg = stdinReader.readLine()) != null) {
producer.send(message);
}
stdinReader.close();
} catch (IOException|FailedToSendMessageException e) {
System.out.println("Send message failed!");
e.printStackTrace();
}
}
public static void main(String args[]) throws JSONException,IOException {
try{
Date now = new Date();
System.out.println("Start : " + now);
process();// Continuously reads logs
now = new Date();
System.out.println("After : " + now);
} catch(Exception e){
e.printStackTrace();
return;
}
}
Execution
./executer.sh & : executing in the background
Problem
After reading all the lines readLines.py ends, but executer.sh and MsgReader.java are stll executing as verified by ps command. The statement System.out.println("After : " + now); os recorded in the log file, indicating the program has reached the last statement in the main function.
How to terminate the java program and bash script gracefully.
I do not want to add System.exit() here.
Platform
CentOS release 6.7 (Final)
java 1.7.0_95
python 2.7.6

You did not close the Producer. From the Javadoc for KafkaProducer (the implementation of Producer):
The producer consists of a pool of buffer space that holds records that haven't yet been transmitted to the server as well as a background I/O thread that is responsible for turning these records into requests and transmitting them to the cluster. Failure to close the producer after use will leak these resources.
The background thread is still running and will prevent the process from ending until you close the Producer, preferably in a finally block after the try/catch. Also, that stdinReader.close(); belongs in the finally as well.

Redirecting STDIN and STDOUT of C program in Android

I am trying to port a C program to android using JNI. I have been able to set up the program and get java and c working fine together. The problem is I need to be able to use STDIN since the C program reads input from STDIN and returns a response through STDOUT (C program is a server-client application).
I don't know if it is worth mentioning but the C program uses STDIN_FILENO file descriptor for reading input from STDIN.
How do I read from STDOUT and WRITE to STDIN using Java?
I did some research and found some vague explanation at the following link: https://groups.google.com/forum/#!topic/android-ndk/Brm6jPr4C0Y that I don't understand.
Here's is the C code https://github.com/unekwu/kemi/blob/master/jni/dnscat/dnscat.c#L1270
More Details
The C program is usually run from command line like
dnscat --dns <ip> <port>
After which it starts listening for messages from the user. Normally entered from stdin.
Now in my Android app,
I'm able to run it with JNI by calling main and different name and passing ann array of strings to it. I'v verify the program starts up correcting. The problem is how I'll send the messages to the program since there's no stdin on android.

I've created a project in github, which you can download from here.
It creates 2 named pipes (FIFO), one for input and other for output.
It opens up one end of the pipe in write only mode in native code and other end of the pipe in read only mode in Java code. The file descriptor in native code is mapped to STDOUT i.e. 1, thereafter any writes to STDOUT in native code, will be redirected to other end of pipe which can read in Java code.
It opens up one end of the pipe in read only mode in native code and other end of the pipe in write only mode in Java code. The file descriptor in native code is mapped to STDIN i.e. 0, thereafter any writes to other end of pipe in Java code, will be read by native code using STDIN.
To achieve STDOUT redirection:
Native Code:
/*
* Step 1: Make a named pipe
* Step 2: Open the pipe in Write only mode. Java code will open it in Read only mode.
* Step 3: Make STDOUT i.e. 1, a duplicate of opened pipe file descriptor.
* Step 4: Any writes from now on to STDOUT will be redirected to the the pipe and can be read by Java code.
*/
int out = mkfifo(outfile, 0664);
int fdo = open(outfile, O_WRONLY);
dup2(fdo, 1);
setbuf(stdout, NULL);
fprintf(stdout, "This string will be written to %s", outfile);
fprintf(stdout, "\n");
fflush(stdout);
close(fdo);
Java Code:
/*
* This thread is used for reading content which will be written by native code using STDOUT.
*/
new Thread(new Runnable() {
#Override
public void run() {
BufferedReader in = null;
try {
in = new BufferedReader(new FileReader(mOutfile));
while(in.ready()) {
final String str = in.readLine();
mHandler.post(new Runnable() {
#Override
public void run() {
Toast.makeText(RedirectionJni.this, str, Toast.LENGTH_LONG).show();
}
});
}
in.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
To achieve STDIN redirection:
Native Code:
/*
* Step 1: Make a named pipe
* Step 2: Open the pipe in Read only mode. Java code will open it in Write only mode.
* Step 3: Make STDIN i.e. 0, a duplicate of opened pipe file descriptor.
* Step 4: Any reads from STDIN, will be actually read from the pipe and JAVA code will perform write operations.
*/
int in = mkfifo(infile, 0664);
int fdi = open(infile, O_RDONLY);
dup2(fdi, 0);
char buf[256] = "";
fscanf(stdin, "%*s %99[^\n]", buf); // Use this format to read white spaces.
close(fdi);
Java Code:
/*
* This thread is used for writing content which will be read by native code using STDIN.
*/
new Thread(new Runnable() {
#Override
public void run() {
BufferedWriter out = null;
try {
out = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(mInfile)));
String content = "This content is written to " + mInfile;
out.write(content.toCharArray(), 0, content.toCharArray().length);
out.flush();
out.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
Let me know if you need any help.

In the Java Code, get a command line by
Process p;
p = Runtime.getRuntime().exec("su"); // or execute something else, su is just to get root
DataOutputStream dos = new DataOutputStream(p.getOutputStream());
dos.writeBytes(ForExampleABinaryPath+" &\n");
dos.flush();
// to read (might has to run parallel in different thread)
InputStream is = p.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
is.readLine() // will get you a line o null
using write && flush and read in parallel, you might be able to achieve your goals.
good luck

Tomcat fails to execute external java program

I am a total newbie in JSPs/Tomcat and to a large extent in Java as well. Here's what I have to do -- when a user clicks a button/visits a URL, I want to launch a Java program (which takes some command line arguments).
I can very easily do
Runtime.exec("C:\\Python27\\python.exe test.py")
OR
Runtime.exec("java -cp %CLASSPATH%;C:\\devprojects HelloWorld"
and this works fine. Where HelloWorld.class just prints "HelloWorld".
However, when I attempt a java program which takes command line arguments, the GET request just hangs doing nothing. I don't know what logs to look for or what could be wrong here. After having spent TWO days on trying various things, I am just about to give up now.
Runtime.exec("java -cp %CLASSPATH%;C:\\devprojects Run --username Blah --password Foo");
What user does Tomcat end up running this java program as? Can I make it to be Administrator? This is on Windows 2008, does UAC interfere with things?
I cannot modify the Run.class here, I HAVE to run it as is and with command line parameters.
Please advise.

One idea: you are relying on the default tokenization of your command line as one complete String, and it is not parsing the last one as you expect. Instead you should use the form of this method that takes a String[], after you have chopped up the command line yourself:
http://docs.oracle.com/javase/6/docs/api/java/lang/Runtime.html#exec(java.lang.String[])
Or, it is waiting for input from you, or waiting for you to read its output. This could explain the hang. Search the internet for the dangers of streams and Runtime.exec().
Consider ProcessBuilder instead.

Remember also that you have to be sure that the executed file dont run "forever", and
if you need to pass some arguments, you could use this:
static String startProcess(String command, String dir) throws IOException {
StringBuffer ret = new StringBuffer();
String[] comm = new String[3];
comm[0] = COMMAND_INTERPRETER[0];
comm[1] = COMMAND_INTERPRETER[1];
comm[2] = command;
long start = System.currentTimeMillis();
try {
//Start process
Process ls_proc = Runtime.getRuntime().exec(comm, null, new File(dir));
//Get input and error streams
BufferedInputStream ls_in = new BufferedInputStream(ls_proc.getInputStream());
BufferedInputStream ls_err = new BufferedInputStream(ls_proc.getErrorStream());
boolean end = false;
while (!end) {
int c = 0;
while ((ls_err.available() > 0) && (++c <= 1000)) {
ret.append(conv2Html(ls_err.read()));
}
c = 0;
while ((ls_in.available() > 0) && (++c <= 1000)) {
ret.append(conv2Html(ls_in.read()));
}
try {
ls_proc.exitValue();
//if the process has not finished, an exception is thrown
//else
while (ls_err.available() > 0)
ret.append(conv2Html(ls_err.read()));
while (ls_in.available() > 0)
ret.append(conv2Html(ls_in.read()));
end = true;
}
catch (IllegalThreadStateException ex) {
//Process is running
}
//The process is not allowed to run longer than given time.
if (System.currentTimeMillis() - start > MAX_PROCESS_RUNNING_TIME)
//this is very important
{
ls_proc.destroy();
end = true;
ret.append("!!!! Process has timed out, destroyed !!!!!");
}
try {
Thread.sleep(50);
}
catch (InterruptedException ie) {}
}
}
catch (IOException e) {
ret.append("Error: " + e);
}
return ret.toString();
}

java TCP socket message breaks

i have a java client-server app in java, both using the same connection class that contains both send/receive messages.
for some reason, some of the messages i send are received in a malformed order:
here's the code
//set up
_in = new BufferedReader(new InputStreamReader(this._socket.getInputStream()));
_out = new BufferedWriter(new OutputStreamWriter(this._socket.getOutputStream()));
this._socket.setSoTimeout(S_TIMEOUT);
public synchronized boolean send(String message){
try {
_out.write(message);
_out.write(Connection.DELIMITER);
_out.flush();
return true;
} catch (IOException e) {
}
return false;
}
public String receive(){
int c;
try {
String message = "";
System.out.println("Getting message:");
c = _in.read();
while(c != -1 && c != Connection.DELIMITER) {
message += (char) c;
c = _in.read();
}
if (c == -1) {
return null;
}
return message;
} catch (IOException e) { }
return null;
}
some messages, for example "new_order" will might return with "ew_ord".
some characters are lost, others are sent separately. this seems odd as its TCP
could this be an encoding related issue?
Delimiter is (char) 0
socket timeout is 20000 (ie 20 senconds). every 10 seconds i send an empty message to make sure socket does not close
EDIT:
although it was solved using the Scanner, i must say that the original code worked fine for many messages/various machines for a very long time (a few weeks), and then suddenly failed to work with one specific message on one specific machine (other messages went through just fine). i've done socket data transfer in java MANY times and i've written many read/write methods to handle the sockets. it's the first time i ran into this.
although in the original code i set the encoding (in the posted code i didn't), i believe that the problem was encoding related. at one point, the message that was received had every second character missing. afterwards i changed it a bit, and the first/second character of the message were received in a separate message. from my understanding, it's either an encoding issue or some firewall/other security program that was running on the message sender machine, that decided to filter outgoing packets.

Try replacing your receive with a Scanner and let it do the work for you.
// in your setup
Scanner sc = new Scanner(_in).useDelimiter(Connection.DELIMETER);
public String receive() {
try {
return sc.next();
} catch(IOException e) {
return "";
}
}

For starters, I would make sure you're printing exceptions in those catch blocks.
Then, you're using the platform default encoding for converting characters to bytes. If these two processes are running on different machines, it's possible they're using different encodings. I would make sure you're specifying an encoding when you set up the Reader and Writer.

You can use UTF encoding for getting Full String of Message.
U can try this code and I am Sure About this code because i used it in My Chat Application.
String data=" ";
socket = new Socket("localhost",999);
while(true)
{
dos = new DataOutputStream(socket.getOutputStream());
dis = new DataInputStream(socket.getInputStream());
data = dis.readUTF();
jta.append(data +"\n");
}
Where jta is JTextArea.
It's for Client Side
Now For Server Side:
try
{
server = new ServerSocket(999);
Socket soc = server.accept();
while(true)
{
String data="";
try
{
dis = new DataInputStream(soc.getInputStream());
dos = new DataOutputStream(soc.getOutputStream());
data = dis.readUTF();
}
catch(Exception e)
{ }
jta.append(data + "\n");
}
}
catch (IOException e)
{
JOptionPane.showMessageDialog(this, e);
System.exit(-1);
}

Java Process "The pipe has been ended" problem

I am using Java Process API to write a class that receives binary input from the network (say via TCP port A), processes it and writes binary output to the network (say via TCP port B). I am using Windows XP. The code looks like this. There are two functions called run() and receive(): run is called once at the start, while receive is called whenever there is a new input received via the network. Run and receive are called from different threads.
The run process starts an exe and receives the input and output stream of the exe. Run also starts a new thread to write output from the exe on to the port B.
public void run() {
try {
Process prc = // some exe is `start`ed using ProcessBuilder
OutputStream procStdIn = new BufferedOutputStream(prc.getOutputStream());
InputStream procStdOut = new BufferedInputStream(prc.getInputStream());
Thread t = new Thread(new ProcStdOutputToPort(procStdOut));
t.start();
prc.waitFor();
t.join();
procStdIn.close();
procStdOut.close();
} catch (Exception e) {
e.printStackTrace();
printError("Error : " + e.getMessage());
}
}
The receive forwards the received input from the port A to the exe.
public void receive(byte[] b) throws Exception {
procStdIn.write(b);
}
class ProcStdOutputToPort implements Runnable {
private BufferedInputStream bis;
public ProcStdOutputToPort(BufferedInputStream bis) {
this.bis = bis;
}
public void run() {
try {
int bytesRead;
int bufLen = 1024;
byte[] buffer = new byte[bufLen];
while ((bytesRead = bis.read(buffer)) != -1) {
// write output to the network
}
} catch (IOException ex) {
Logger.getLogger().log(Level.SEVERE, null, ex);
}
}
}
The problem is that I am getting the following stack inside receive() and the prc.waitfor() returns immediately afterwards. The line number shows that the stack is while writing to the exe.
The pipe has been ended
java.io.IOException: The pipe has been ended
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:260)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
at java.io.FilterOutputStream.write(FilterOutputStream.java:80)
at xxx.receive(xxx.java:86)
Any advice about this will be appreciated.

This means you are writing to the pipe after the other end has already closed it.
That indicates a major error in your application protocol.

I have had the same problem recently and I have found a solution.
First of all, "The pipe has been ended" error is not a Java error - it comes from Windows system. According to MSDN:
The using process has closed the pipe or, if you are trying to write
to the pipe, there are no available readers.
Not very informative. However, if process has closed the pipe itself, it may mean that some errors occurred in process.
To check this, redirect errors coming from process, for instance, to a file:
File f = new File("errors.txt");
pb.redirectError(f);
In my case (I've been trying to execute SrcML parser) file contained this:
.\libs\srcML-Win\src2srcml.exe: unrecognised option `--language Java'
Try 'src2srcml --help' for more information.
Fixing this solved the problem.