Develop Reference

Receiving truncated data while reading from Serial Port

I am trying to establish communication between Arduino and Android over Uart. So, while reading buffer on Android side I am not getting data in chunks.
if (uartDevice != null) {
// Loop until there is no more data in the RX buffer.
try {
byte[] buffer = new byte[CHUNK_SIZE];
int read;
while ((read = uartDevice.read(buffer, buffer.length)) > 0) {
data = new String(buffer, StandardCharsets.UTF_8).substring(0, read);
System.out.println(String.format("%020x", new BigInteger(1, data.getBytes(/*YOUR_CHARSET?*/))));
} catch (IOException e) {
Log.w(TAG, "Unable to transfer data over UART", e);
}
Expected output is:
2a3619010101001a0708403031301010011214084030313010100112140845
Instead I am receiving:
2a361a010101001a070840303130101001121408403031
8403031301010011214084030313010100112140845
3031301010011214084030313010100112140845

If you want to write code that only prints the bytes that you get I would try the following:
if (uartDevice != null) {
// Loop until there is no more data in the RX buffer.
try {
byte[] buffer = new byte[CHUNK_SIZE];
int read;
while ((read = uartDevice.read(buffer, buffer.length)) > 0) {
for (int i = 0; i < read; i++) {
System.out.printf("%02x", buffer[i]);
}
}
} catch (IOException e) {
Log.w(TAG, "Unable to transfer data over UART", e);
}
System.out.println(); // Adds a newline after all bytes
}
The following is a method that takes a UartDevice as a parameter, reads from it until the end and returns a single byte array with the whole content. No arbitrary buffer that is guaranteed to hold the whole content is needed. The returned array is exactly as big as it needs to be. Only a small read buffer is used to increase performance. Error handling is ignored.
This assumes that the data is not larger than it fits into memory.
byte[] readFromDevice(UartDevice uartDevice) {
byte[] buffer = new byte[CHUNK_SIZE];
int read;
ByteArrayOutputStream data = new ByteArrayOutputStream();
while ((read = uartDevice.read(buffer, buffer.length)) > 0) {
data.write(buffer, 0, read);
}
return data.toByteArray();
}
The method returns when all data has been read and you can process the returned array at your leasure.

Handling binary packets

I coded this packet handler but I can imagine scenarios in which it will get stuck or won't be able to read incomplete data. My questions are:
Should I use two buffers, one for the current incoming data and other to append incomplete data to?
I'm being stupidly over-complicated?
Code:
byte[] buffer;
int bufferLength;
int bytesRead;
buffer = new byte[1024];
while (bluetoothConnected) {
try {
// Wait for packet header
if (mmInStream.available() >= 8) {
bufferLength = mmInStream.read(buffer);
bytesRead = 0;
// Parse every packet
while (true) {
int commandType = ByteBuffer.wrap(buffer, 0, 2).order(ByteOrder.LITTLE_ENDIAN).getShort();
int payloadSize = ByteBuffer.wrap(buffer, 2, 2).order(ByteOrder.LITTLE_ENDIAN).getShort();
int packetSize = PACKET_HEADER_SIZE + payloadSize;
// Break if payload is incomplete
if (bufferLength < (bytesRead + packetSize)) {
// Append to other buffer
break;
}
byte[] packet = new byte[packetSize];
System.arraycopy(buffer, bytesRead, packet, 0, packetSize);
parsePacketSequence(socket, packet);
bytesRead += packetSize;
// Break if all bytes are read
if (bufferLength == bytesRead)
{
break;
}
// Break if more bytes are needed
// Packet header incomplete
if ((bufferLength - bytesRead) < PACKET_HEADER_SIZE)
{
// Append to other buffer
break;
}
}
}
}
catch (IOException e) {
bluetoothConnected = false;
Log.d(TAG, "Error " + e);
break;
}
}

Should I use two buffers, one for the current incoming data and other to append incomplete data to?
No.
I'm being stupidly over-complicated?
Yes.
Here's a simple version using DataInputStream:
DataInputStream din = new DataInputStream(mmInStream);
while (bluetoothConnected) {
try {
// Read packet header
int commandType = swap(din.readShort());
int payloadSize = swap(din.readShort());
int packetSize = PACKET_HEADER_SIZE + payloadSize;
byte[] packet = new byte[packetSize];
din.readFully(packet);
parsePacketSequence(socket, packet);
}
catch (IOException e) {
bluetoothConnected = false;
Log.d(TAG, "Error " + e);
break;
}
}
The swap() method which converts a short in litte-endian byte order to Java byte order is left as an exercise for the reader.
NB I don't see how parsePacketSequence() can work if it doesn't know commandType.
E&OE

Limit size byte[] Java android

I have to fill a byte[] in my Android application. Sometime, this one is bigger than 4KB.
I initialize my byte[] like this :
int size = ReadTools.getPacketSize(ptr.dataInputStream);
byte[] myByteArray = new byte[size];
Here, my size = 22625. But when I fill up my byte[] like this :
while (i != size) {
myByteArray[i] = ptr.dataInputStream.readByte();
i++;
}
But when I print the content of my byte[], I have a byte[] with size = 4060.
Does Java split my byte[] if this one is bigger than 4060 ? And if yes, how can I have a byte[] superior to 4060 ?
Here is my full code:
public class ReadSocket extends Thread{
DataInputStream inputStream;
BufferedReader reader;
GlobalContent ptr;
public ReadSocket(DataInputStream inputStream, GlobalContent ptr)
{
this.inputStream = inputStream;
this.ptr = ptr;
}
public void run() {
int i = 0;
int j = 0;
try {
ptr.StatusThreadReadSocket = 1;
while(ptr.dataInputStream.available() == 0)
{
if(ptr.StatusThreadReadSocket == 0)
{
ptr.dataInputStream.close();
break;
}
}
if(ptr.StatusThreadReadSocket == 1)
{
int end = ReadTools.getPacketSize(ptr.dataInputStream);
byte[] buffer = new byte[end];
while (i != end) {
buffer[j] = ptr.dataInputStream.readByte();
i++;
j++;
}
ptr.StatusThreadReadSocket = 0;
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
...
}

Java doesn't split anything. You should post the minimal code which reproduces your error, and tell where ReadTools comes from.
There are two options here:
ReadTools.getPacketSize() returns 4096
You inadevertedly reassign myByteArray to another array
You should really post your full code and tell what library you use. Likely, it will have a method like
read(byte[] buffer, int offset, int length);
Which will save you some typing and also give better performance if all you need is bulk reading the content of the input in memory

Bluetooth-connection; can't send strings properly

I have troubles with my program when i need to send Strings from my server bluetooth-socket to my client bluetooth-socket.
Everything works fine as long as I am only sending one String at a time (for example chatting) but if I need to write more Strings at a short period of time (to interchange informations), the Strings will not get seperated from the client code. For example if I'm sending "FirstUser" and right after that "SecondUser" the client does not read "FirstUser" and then "SecondUser". It will read "FirstUserSecondUser". How can I avoid this behaviour?
Edit: If I let the Thread sleep before it is able to send a new message, it reads the right strings but this solution is not working fine for my need.
Server-Code: sending to all clients(edited)
public synchronized void sendToAll(String message)
{
try {
Thread.sleep(100);
} catch (InterruptedException e1) {
e1.printStackTrace();
}
publishProgress(message);
for(OutputStream writer:outputList) {
try {
writer.write(message.getBytes());
writer.flush();
} catch (IOException e) {
System.out.println("Some-Error-Code");
}
}
}
Server-Code: reading from a client:
public void run() {
String nachricht;
int numRead;
byte[] buffer = new byte[1024];
while (runningFlag)
{
try {
if((numRead = inputStream.read(buffer)) >= 0) {
nachricht = new String(buffer, 0, numRead);
serverThread.handleMessage(nachricht);
}
}
catch (IOException e) {
this.cancel();
e.printStackTrace();
}
}
}
Client-Code: reading from server(edited)
#Override
protected Void doInBackground(Integer... ints) {
String nachricht = new String();
byte[] buffer = new byte[1024];
int numRead;
while (runningFlag)
{
try {
if(((numRead = inputStream.read(buffer)) >= 0)) {
nachricht = new String(buffer, 0, numRead);
publishProgress(nachricht);
}
}
catch (IOException e) {
clientGame.finish();
e.printStackTrace();
}
}
return null;
}
Client-Code: writing to server
public synchronized void write(String nachricht)
{
try {
Thread.sleep(100);
} catch (InterruptedException e1) {
e1.printStackTrace();
}
try {
outputStream.write(nachricht.getBytes());
outputStream.flush();
} catch (IOException e) {
this.cancel();
e.printStackTrace();
}
}
I appreciate every little help :) .

You need to encapsulate your data item to avoid concatenation.
It means that you have to write and read a whole data item before continuing.
You should have some utility methods to do that instead of directly using methods of OutputStream and InputStream :
public static void writeItem(OutputStream out, String s) throws IOException
{
// Get the array of bytes for the string item:
byte[] bs = s.getBytes(); // as bytes
// Encapsulate by sending first the total length on 4 bytes :
// - bits 7..0 of length
out.write(bs.length); // modulo 256 done by write method
// - bits 15..8 of length
out.write(bs.length>>>8); // modulo 256 done by write method
// - bits 23..16 of length
out.write(bs.length>>>16); // modulo 256 done by write method
// - bits 31..24 of length
out.write(bs.length>>>24); // modulo 256 done by write method
// Write the array content now:
out.write(bs); // Send the bytes
out.flush();
}
public static String readItem(InputStream in) throws IOException
{
// first, read the total length on 4 bytes
// - if first byte is missing, end of stream reached
int len = in.read(); // 1 byte
if (len<0) throw new IOException("end of stream");
// - the other 3 bytes of length are mandatory
for(int i=1;i<4;i++) // need 3 more bytes:
{
int n = in.read();
if (n<0) throw new IOException("partial data");
len |= n << (i<<3); // shift by 8,16,24
}
// Create the array to receive len bytes:
byte[] bs = new byte[len];
// Read the len bytes into the created array
int ofs = 0;
while (len>0) // while there is some byte to read
{
int n = in.read(bs, ofs, len); // number of bytes actually read
if (n<0) throw new IOException("partial data");
ofs += n; // update offset
len -= n; // update remaining number of bytes to read
}
// Transform bytes into String item:
return new String(bs);
}
Then you use these methods both for server & client to read and write your String items.

Downloading with BufferInputStream not working properly

The following code doesn't work to download a file (btw clen is file's length):
int pos = 0, total_pos = 0;
byte[] buffer = new byte[BUFFER_SIZE];
while (pos != -1) {
pos = in.read(buffer, 0, BUFFER_SIZE);
total_pos += pos;
out.write(buffer);
setProgress((int) (total_pos * 100 / clen));
}
...but this works fine:
int buf;
while ((buf = in.read()) != -1)
out.write(buf);
I'm wondering why, even though the second code segment works quickly. On that note, is there any particular reason to use a byte[] buffer (since it doesn't seem to be faster, and BufferedInputStream already uses a buffer of its own....?)

Here's how it should be done.
public static void copyStream(InputStream is, OutputStream os)
{
byte[] buff = new byte[4096];
int count;
try {
while((count = is.read(buff)) > 0)
os.write(buff, 0, count);
}catch (Exception e) {
e.printStackTrace();
}finally {
try {
if(is != null)
is.close();
} catch (IOException e) {
e.printStackTrace();
}
try {
if(os != null)
os.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}

I've tried to make the minimum changes necessary to your code to get it working. st0le did a good job of providing a neater version of stream copying.
public class Test {
private static final String FORMAT = "UTF-8";
private static final int BUFFER_SIZE = 10; // for demonstration purposes.
public static void main(String[] args) throws Exception {
String string = "This is a test of the public broadcast system";
int clen = string.length();
ByteArrayInputStream in = new ByteArrayInputStream(string.getBytes(FORMAT));
OutputStream out = System.out;
int pos = 0, total_pos = 0;
byte[] buffer = new byte[BUFFER_SIZE];
while (pos != -1) {
pos = in.read(buffer, 0, BUFFER_SIZE);
if (pos > 0) {
total_pos += pos;
out.write(buffer, 0, pos);
setProgress((int) (total_pos * 100 / clen));
}
}
}
private static void setProgress(int i) {
}
}
You were ignoring the value of pos when you were writing out the buffer to the output stream.
You also need to re-check the value of pos because it may have just read the end of the file. You don't increment the total_pos in that case (although you should probably report that you are 100% complete)
Be sure to handle your resources correctly with close()s in the appropriate places.
-edit-
The general reason for using an array as a buffer is so that the output stream can do as much work as it can with a larger set of data.
Writing to a console there might not be much of a delay, but it might be a network socket being written to or some other slow device. As the JavaDoc states
The write method of OutputStream calls the write method of one argument on each of the bytes to be written out. Subclasses are encouraged to override this method and provide a more efficient implementation.
The benefit of using it when using a Buffered Input/Output Stream are probably minimal.