Develop Reference

Data loss during java serial communcation

I am trying to receive a telemetry string from arduino board to javafx apllication by jserialcomm.
My arduino output rate is about 100Hz now. in this situation i want to receive data in application about 1Hz. this is what i am doing:
This are only some important parts of code:
Runnable r1 = new Runnable() {
public void run() {
try {
while (true) {
refresher(rx);
Thread.sleep(1000L);
}
} catch (InterruptedException iex) {}
}
};
Thread thr1 = new Thread(r1);
thr1.start();
public void refresher(SerialPort rx){
readRX(rx);
parseString(lastTelemetry);
}
private void readRX(SerialPort rx){
Scanner ss = new Scanner(rx.getInputStream());
while(ss.hasNextLine()){
lastTelemetry = ss.nextLine();
if (lastTelemetry.isEmpty()) continue;
System.out.println(lastTelemetry);
break;
}
}
But the recieved string is not complete. some lines are complete and some or lost . this is what my output look like:
8,0,330,1306.42,86586.00,0,31.36,0,0,0,0,0,0,0,0,62.27,-6.81,4.53,0.00,00
0,0,0,0,0,0,0,0,66.24,-6.81,4.52,-0.30,00
1.36,0,0,0,0,0,0,0,0,70.22,-6.81,4.52,-0.10,00
7098,0,396,1306.33,86587.00,0,31.36,0,0,0,0,0,0,0,0,75.22,-6.81,4.51,-0.10,00

You probably shouldn't be creating a new input stream new Scanner(rx.getInputStream()) every time you read input. It is likely that if data is being buffered it is lost when you make a new stream. Create the input stream once when you open the serial port and pass that as the parameter to the readRX method instead of the SerialPort.
Also, I couldn't find anything in a cursory read of the Javadoc where you specify the buffer size, or what happens if the buffer overflows. That is another factor to consider.

I think there's a problem with the definition of hasNextLine().
See the following example:
byte[] data = "test".getBytes();
ByteArrayInputStream inputStream = new ByteArrayInputStream(data);
Scanner scanner = new Scanner(inputStream);
while (scanner.hasNextLine()) {
String text = scanner.nextLine();
System.out.println(text);
}
There is no \n at the end or something like this. But it still outputs "test". So I expect you assume a different behaviour of the next line part.
Also es mentioned by someone else, cachine might be an additional failure source.

ServerSocket java not reading inputStream?

I have written a server is java here is the code:
public mainClass()
{
try
{
ss = new ServerSocket(8080);
while (true)
{
socket = ss.accept();
System.out.println("It is accept!");
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
//out = new PrintWriter(socket.getOutputStream(),true);
line = in.readLine();
System.out.println("you input is :" + line);
//out.close();
in.close();
socket.close();
}
}
catch (IOException e)
{
}
and I am using an iPhone application as the client.
now what my problem is that the server is not reading the inputstream while the appication is running on the iphone.. But as soon as the application is terminated the java program prints out the String which has been sent to the server..Not sure what is happening here..sorry if this is not a good question..
- (void)viewDidLoad {
[super viewDidLoad];
socket = [[LXSocket alloc]init];
if ([socket connect:#"10.211.55.2" port:8080]) {
NSLog(#"socket has been created");
}
else {
NSLog(#"socket couldn't be created created");
}
#try {
[socket sendString:#"Hi This is a second test"];
}
#catch (NSException * e) {
NSLog(#"Unable to send data");
}
[super viewDidLoad];
}
thanks,
TC

From my own experience, readLine is not a good idea, especially when working with different languages and platforms, a better approach will be to use InputStreamReader and its read(char[] buff) method, and agree on both sides regarding the length to be sent each time.
Again, I have no reference to that, only my experience.
Also, looking at your code, you send a string without a new line character: [socket sendString:#"Hi This is a second test"]; maybe adding \n at the end will solve it for you.

My guess is that the client application doesn't send any line break at the end of the string it sends. So BufferedReader.readLine() waits for an EOL character, and only returns the string when the client application ends, because at this point the connection is closed and the reader knows there won't ever be an EOL, and the string is the last line it will ever receive.

BufferedReader can be dangerous; the buffering can cause short lines to get "stuck" if you're only reading a little data at a time, or if the data is coming across a network. If you're only using BufferedReader to get readLine(), then do this:
new BufferedReader(new InputStreamReader(socket.getInputStream()), 1);
That extra argument sets the buffer size to 1 character, effectively turning it off. That generally solves this kind of problem.

How to write "enter key pressed" to a stream?

Sorry for this odd-sounding title...
I have the following situation: I want my Java program to interact with an external console. In order to "send" the individual commands to that console, I need to simulate what would be an "enter key pressed" on a normal console. To clarify what I want, imagine mysql had no other API and I would need to interact via console. Although this is not my actual problem, it is close enough.
I have the following code:
String command = "/usr/local/mysql/bin/mysql";
Process child = Runtime.getRuntime().exec(command);
StreamGobbler gobbler = new StreamGobbler(child.getInputStream());
gobbler.start();
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(child.getOutputStream()));
out.write("help");
// here enter key needs to be pressed
out.flush();
// out.close();
If the call to out.close() is executed, everything is fine. But of course, this way I can only send a single command, which is not what I want. But if out.close() is omitted, the other program never executes the command. My guess is that it still waits for the command to "finish", which on a normal console would be done by pressing enter. out.write(System.getProperty("line.separator")); and out.newLine(); (which are the same) do not solve the problem, neither does out.write("\r\n"); and out.write((char) 26); (EOF).
Of course, it might be, that I am doing it completely wrong (i.e., wrong approach). Then I would appreciate a pointer into the right direction...
Any help on this highly appreciated.

The following code works fine on both Windows 7 using Java 1.6.0_23 and on Ubuntu 8.04 using Java 1.6.0_22:
public class Laj {
private static class ReadingThread extends Thread {
private final InputStream inputStream;
private final String name;
public ReadingThread(InputStream inputStream, String name) {
this.inputStream = inputStream;
this.name = name;
}
public void run() {
try {
BufferedReader in = new BufferedReader(
new InputStreamReader(inputStream));
for (String s = in.readLine(); s != null; s = in.readLine()) {
System.console().writer().println(name + ": " + s);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
public static void main(String[] args) throws Exception {
String command = "psql -U archadm arch";
final Process child = Runtime.getRuntime().exec(command);
new ReadingThread(child.getInputStream(), "out").start();
new ReadingThread(child.getErrorStream(), "err").start();
BufferedWriter out = new BufferedWriter(
new OutputStreamWriter(child.getOutputStream()));
out.write("\\h");
out.newLine();
out.flush();
out.write("\\q");
out.newLine();
out.flush();
}
}
newLine() is the same as writing the platform line separator. As one would expect, it prints help preceded with "out: ", then exits. If I don't send "\q", it doesn't exit (obviously) but still prints help. Using "\r\n" or "\r" instead of the platform line separator doesn't look like a good idea to me, because such command-line utilities will usually detect that they don't get input from the terminal and assume it is in the native text format (think "psql < script.sql"). Good software should properly detect and accept all reasonable line endings though.

What about out.write((char) 13)? See this Wikipedia article. I don't have enough code to test this for you.

You also might want to try looking at this API
http://download.oracle.com/javase/6/docs/api/java/io/Console.html
From my experience, I've never tried doing anything more than running one process from the Process API. It seems like you want to enter multiple commands I think this API might let you do that.
EDIT: Found a tutorial on it to help you further.
http://download.oracle.com/javase/tutorial/essential/io/cl.html
Hope this helps,

How to write a byte array to OutputStream of process builder (Java)

byte[] bytes = value.getBytes();
Process q = new ProcessBuilder("process","arg1", "arg2").start();
q.getOutputStream().write(bytes);
q.getOutputStream().flush();
System.out.println(q.getInputStream().available());
I'm trying to stream file contents to an executable and capture the output but the output(InputStream) is always empty. I can capture the output if i specify the the file location but not with streamed input.
How might I overcome this?

Try wrapping your streams with BufferedInputStream() and BufferedOutputStream():
http://download.oracle.com/javase/6/docs/api/java/lang/Process.html#getOutputStream%28%29
Implementation note: It is a good idea for the output stream to be buffered.
Implementation note: It is a good idea for the input stream to be buffered.
Even with buffered streams, it is still possible for the buffer to fill if you're dealing with large amounts of data, you can deal with this by starting a separate thread to read from q.getInputStream(), so you can still be reading from the process while writing to the process.

Perhaps the program you execute only starts its work when it detects the end of its input data. This is normally done by waiting for an EOF (end-of-file) symbol. You can send this by closing the output stream to the process:
q.getOutputStream().write(bytes);
q.getOutputStream().close();
Try this together with waiting for the process.

I dont know if something else may also be wrong here, but the other process ("process") does not even have time to respond, you are not waiting for it (the method available() does not block). To try this out you can first insert a sleep(2000) after the flush(), and if that works you should switch to query'ing q.getInputStream().available() multiple times with short pauses in between.

I think, you have to wait, until the process finished.
I implemented something like this this way:
public class ProcessReader {
private static final int PROCESS_LOOP_SLEEP_MILLIS = 100;
private String result;
public ProcessReader(Process process) {
BufferedReader resultReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
StringBuilder resultOutput = new StringBuilder();
try {
while (!checkProcessTerminated(process, resultReader, resultOutput)) {
}
} catch (Exception ex1) {
throw new RuntimeException(ex1);
}
result = resultOutput.toString();
}
public String getResult(){
return result;
}
private boolean checkProcessTerminated(Process process, BufferedReader resultReader, StringBuilder resultOutput) throws Exception {
try {
int exit = process.exitValue();
return true;
} catch (IllegalThreadStateException ex) {
Thread.sleep(PROCESS_LOOP_SLEEP_MILLIS);
} finally {
while (resultReader.ready()) {
String out = resultReader.readLine();
resultOutput.append(out).append("\n");
}
}
return false;
}
}
I just removed now some specific code, that you dont need, but it should work, try it.
Regards

Writing Standard Input and waiting for Standard Output

I'm trying to create a Thread that keeps netsh windows command-line tool open so I can execute netsh commands without open it every single time.
The thing is, once I've created the Thread, just the first command call works... the subsequent calls seems to have no effect.
Here is my code:
public class NetshThread implements Runnable{
private static Process netshProcess = null;
private static BufferedInputStream netshInStream = null;
private static BufferedOutputStream netshOutStream = null;
public BufferedReader inPipe = null;
public void run(){
startNetsh();
}
public void startNetsh(){
try {
netshProcess = Runtime.getRuntime().exec("netsh");
netshInStream = new BufferedInputStream(netshProcess.getInputStream());
netshOutStream = new BufferedOutputStream(netshProcess.getOutputStream());
inPipe = new BufferedReader(new InputStreamReader(netshInStream));
} catch (IOException e) {
e.printStackTrace();
}
}
public void executeCommand(String command){
System.out.println("Executing: " + command);
try {
String str = "";
netshOutStream.write(command.getBytes());
netshOutStream.close();
while ((str = inPipe.readLine()) != null) {
System.out.println(str);
}
} catch (IOException e) {
e.printStackTrace();
}
}
public void closeNetsh(){
executeCommand("exit");
}
public static void main(String[] args){
NetshThread nthread = new NetshThread();
nthread.run();
String command = "int ip set address " +
"\"Local Area Connection 6\" static .69.69.69 255.255.255.0";
nthread.executeCommand(command);
command = "int ip set address " +
"\"Local Area Connection 6\" static 69.69.69.69 255.255.255.0";
nthread.executeCommand(command);
System.out.println("*** DONE ***");
}
}
Thank you!!! =)
Update 1:
Ok... I'm now using a PrintWriter instead... so I think I don't need to flush anything anymore, since the constructor is:
new PrintWriter(netshOutStream, true); (just like Mr. Shiny told me)...
Suppose I decide to break the while loop when the first output line is available... I doesn't work either... the next command wont be executed.... My code now looks like:
import java.io.*;
public class NetshThread implements Runnable{
private static Process netshProcess = null;
private static BufferedInputStream netshInStream = null;
private static BufferedOutputStream netshOutStream = null;
public BufferedReader inPipe = null;
private PrintWriter netshWriter = null;
public void run(){
startNetsh();
}
public void startNetsh(){
try {
netshProcess = Runtime.getRuntime().exec("netsh");
netshInStream = new BufferedInputStream(netshProcess.getInputStream());
netshOutStream = new BufferedOutputStream(netshProcess.getOutputStream());
netshWriter = new PrintWriter(netshOutStream, true);
inPipe = new BufferedReader(new InputStreamReader(netshInStream));
} catch (IOException e) {
e.printStackTrace();
}
}
public void executeCommand(String command){
System.out.println("Executing: " + command);
try {
String str = "";
netshWriter.println(command);
while ((str = inPipe.readLine()) != null) {
System.out.println(str);
break;
}
} catch (IOException e) {
e.printStackTrace();
}
}
public void closeNetsh(){
executeCommand("exit");
}
public static void main(String[] args){
NetshThread nthread = new NetshThread();
Thread xs = new Thread(nthread);
xs.run();
String command = "int ip set address " +
"\"Local Area Connection 6\" static .69.69.69 255.255.255.0";
nthread.executeCommand(command);
command = "int ip set address " +
"\"Local Area Connection 6\" static 69.69.69.69 255.255.255.0";
nthread.executeCommand(command);
System.out.println("*** DONE ***");
}
}
and the output I get:
Executing: int ip set address "Local
Area Connection 6" static .69.69.69
255.255.255.0 netsh>.69.69.69 is not an acceptable value for addr.
Executing: int ip set address "Local
Area Connection 6" static 69.69.69.69
Why the second command is not executed???
255.255.255.0
* DONE *
Update 2:
Everything seemed to work just fine until a teacher tried my app in a spanish-windows enviroment....
my code looks like this:
Scanner fi = new Scanner(netshProcess.getInputStream());
public void executeCommand(String command) {
System.out.println("Executing: " + command);
String str = "";
netshWriter.println(command);
fi.skip("\\s*");
str = fi.nextLine();
System.out.println(str);
}
and what i need is to somehow set the netshWriter encoding to the windows default.
Can anyone know who to do this?

You are closing the output stream.

You need to move the stream processing into separate threads. What's happening is that inPipe.readLine() is blocking waiting for netsh to return data. Apache has a package that deals with process handling. I'd look at using that instead of rolling your own (http://commons.apache.org/exec/)

This seems wrong in many ways.
First, why a Runnable object? This isn't ever passed to a Thread anywhere. The only thread you're creating isn't a java thread, it is an OS process created by exec().
Second, you need a way to know when netsh is done. Your loop that reads the output of netsh will just run forever because readLine will only return null when netsh closes its standard out (which is never, in your case). You need to look for some standard thing that netsh prints when it is done processing your request.
And as others mentioned, close is bad. Use a flush. And hope netsh uses a flush back to you...

I'd try:
PrintWriter netshWriter = new PrintWriter(netshOutputStream, true); // auto-flush writer
netshWriter.println(command);
No close()ing the stream, flush the stream automatically, and uses a writer to send character data rather than relying on the platforms "native character set".

You do definitely need to remove the close, else you'll never be able to execute another command. When you say "it won't work" once the close() call removed, do you mean no commands are processed?
Chances are that after you send the bytes for the command, you need to send some kind of confirmation key for the process to start, well, processing it. If you'd normally enter this from the keyboard it might be as simple as a carriage return, otherwise it might need to be a Ctrl-D or similar.
I'd try replacing the close() line with
netshOutStream.write('\n');
and see if that works. Depending on the software you might need to change the character(s) you send to signify the end of the command, but this general approach should see you through.
EDIT:
It would also be prudent to call
netshOutStream.flush();
after the above lines; without the flush there's no guarantee that your data will be written and in fact, since you're using a BufferedInputStream I'm 99% sure that nothing will be written until the stream is flushed. Hence why the code afterwards blocks, as you're waiting for a response while the process has not seen any input yet either and is waiting for you to send it some.

I've used scanner instead of BufferedReader, just because I like it. So this code works:
Scanner fi = new Scanner(netshProcess.getInputStream());
public void executeCommand(String command) {
System.out.println("Executing: " + command);
String str = "";
netshWriter.println(command);
fi.skip("\\s*");
str = fi.nextLine();
System.out.println(str);
}
It executes both commands.