Develop Reference

Getting Garbage value while reading from serial port in java

I am reading data from serialport using jSerialComm package. I am getting following data
A total of 22 bytes are being received and I am getting these three garbage bytes too. The readable data is correct but what these garbage characters are happening?
Following is my code.
public static void main(String[] args) {
SerialPort serialPort = SerialPort.getCommPort("/dev/ttyUSB0");
if(serialPort.openPort())
{
System.out.println("Port Opened Successfully...");
}
else
{
System.out.println("Unable to open port....");
return;
}
serialPort.setComPortParameters(1200, 8, 1, 0);
try
{
while(true)
{
while(serialPort.bytesAvailable() != 0)
{
Thread.sleep(1000);
byte[] readBuffer = new byte[serialPort.bytesAvailable()];
int numRead = serialPort.readBytes(readBuffer, readBuffer.length);
String data = new String(readBuffer);
System.out.println("Read "+numRead+ " bytes." + readBuffer);
System.out.println(data);
}
}
}
catch(Exception e)
{
e.printStackTrace();
}
serialPort.closePort();
System.out.println("done...");
}

SerialPort.readBytes seems to be not thread safe, thus yielding "garbage" when called during character reception.
I suggest to use the snippet from the author's example which worked fine for me:
// Get a new instance of SerialPort by opening a port.
SerialPort port = SerialPort.open("COM2");
// Configure the connection
port.setTimeout(100);
port.setConfig(BaudRate.B115200, Parity.NONE, StopBits.ONE, DataBits.B8);
// You have the choice, you can either use the Java NIO channels
// or classic Input/Ouput streams to read and write data.
//DEL SerialChannel channel = port.getChannel();
InputStream istream = port.getInputStream();
// Read some data using a stream
byte[] byteBuffer = new byte[4096];
// Will timeout after 100ms, returning 0 if no bytes were available.
int n = istream.read(byteBuffer);
// *** Use n bytes of byteBuffer ***
//DEL ...
port.close();
//DEL : removed from the original code for clarity

You did not explain the protocol but I suggest to look it up. Presumably these are control characters or like a comment suggests binary data. You create a String from a byte buffer without an encoding so this also depends on your environment/ JVM's default encoding.
Try treating the first and the last two bytes as specified in the protocol in use for your project. It might also be related to jSerialComm not removing serial signalling, e.g. handshake, EOT and such.
If you're reverse-engineering the protocol instead maybe also try another library like RxTx to see if the bytes stay the same.
To inspect the bytes safely use for example BigInteger to print out a Hex-String instead:
BigInteger bigInteger = new BigInteger(1, bytes);
System.out.printf("%0" + (bytes.length << 1) + "x", bigInteger);

Use this code block and it will run perfectly;
serialPort.addDataListener(new SerialPortDataListener() {
#Override
public int getListeningEvents() {
return SerialPort.LISTENING_EVENT_DATA_RECEIVED;
}
#Override
public void serialEvent(SerialPortEvent serialPortEvent) {
if (serialPortEvent.getEventType() != SerialPort.LISTENING_EVENT_DATA_RECEIVED) {
return;
}
byte[] newData = serialPortEvent.getReceivedData();
String data = new String(newData);
System.out.println(data);
}
});

Serializing/Deserializing a standalone integer using protocol buffers

Up to now, I've been using Protocol Buffers to serialize and deserialize objects using the code-generated classes.
Now I am attempting to serialize and deserialize a single 64-bit integer. The problem is, I'm getting different results in Java and C#.
Here's how I'm doing it in Java....
private static byte[] convertLongToByteArray(long value) throws IOException {
int size = CodedOutputStream.computeInt64SizeNoTag(value);
byte[] buffer = new byte[size];
CodedOutputStream codedOutputStream = CodedOutputStream.newInstance(buffer);
codedOutputStream.writeInt64NoTag(value);
codedOutputStream.flush();
codedOutputStream.checkNoSpaceLeft();
return buffer;
}
And here's how I'm doing this in C#, using protobuf.net:
public void SerializeLongValue()
{
long n = 9876;
byte[] memoryBuffer = null;
using (MemoryStream destination = new MemoryStream())
{
ProtoBuf.Serializer.Serialize(destination, n);
destination.Flush();
memoryBuffer = destination.ToArray();
}
using (MemoryStream source = new MemoryStream(memoryBuffer))
{
long result = ProtoBuf.Serializer.Deserialize<long>(source);
Assert.AreEqual(n, result);
}
}
The java code converted the number 9876 to [0x94, 0x4D]
The C# code converted the number 9876 to [0x08, 0x94, 0x4D]
How do I do this so that both com.google.protobuf and protobuf.net produce identical outputs?

The protobuf.net method ProtoBuf.Serializer.Serialize forces a field header (field number=1) into the stream. That's the only way you can perform a serialization; this method invokes a number of internal methods which are not publicly available.
The solution I'm using is to change the Java code to also include a field header.
Here is my new Java code.
private static byte[] convertLongToByteArray(long value) throws IOException {
int size = CodedOutputStream.computeTagSize(1) + CodedOutputStream.computeInt64SizeNoTag(value);
byte[] buffer = new byte[size];
CodedOutputStream codedOutputStream = CodedOutputStream.newInstance(buffer);
codedOutputStream.writeInt64(1, value);
codedOutputStream.flush();
codedOutputStream.checkNoSpaceLeft();
return buffer;
}
public static long convertByteArrayToLong(byte[] byteArray) throws IOException {
CodedInputStream codedInputStream = CodedInputStream.newInstance(byteArray);
codedInputStream.readTag();
return codedInputStream.readInt64();
}
The changes I've made are:
When computing the required buffer size, include a tag size
Instead of CodedOutputStream.WriteInt64NoTag, call CodedOutputStream.WriteInt64
When reading back, call CodedOutputStream.ReadTag before calling CodedOutputStream.ReadInt64

BinaryWriter to MemoryStream in Java

I have C# program that makes a tcp connection with another c# program. In the c# program to send an message I did this :
private TcpClient client;
private void SendPulse()
{
byte[] send_Buffer;
port = 11000;
while (true)
{
lock (locked)
{
try
{
BlockID = 1003;
using (MemoryStream ms = new MemoryStream())
{
using (BinaryWriter w = new BinaryWriter(ms))
{
NetworkStream stream = client.GetStream();
BlockID = 1003;
LengthMessage = 84;
// Header :
w.Write(BeginMessage);
w.Write(BlockID);
w.Write(LengthMessage);
w.Write(RadarID);
w.Write(Time);
w.Write(ModeSystem);
w.Write(Icd_primary_var);
w.Write(Icd_secondary_ver);
// Data :
w.Write(StatusSystem);
send_Buffer = ms.ToArray();
stream.Write(send_Buffer, 0, send_Buffer.Length);
Thread.Sleep(3000); // Send pulse every 3 seconds.
}
}
}
catch
{
}
}
}
}
The idea is to write in binarywriter and than convert the memory we wrote on to byte array and to send it.
Now I have Java programming. I want to do it too, I have connection with C# but I dont know how to send it. I did DataOutputStream but it send every parameter alone, I want all in 1 array of bytes exactly like in the c# code.
Thanks for helpers.

If you want to use DataOutputStream, you can wrap it around a BufferedOutputStream and flush() it when you are done.
Or you can use an NIO ByteBuffer and write it to the socket.
To make the message easier to decode I would add the length to the start, unless you know it every message will be that length.

Java crypt base64 and compression at the same time

I need some consulting here.
I'm writing a client/server software.
One requirement is to use an XML file to transfer information between the client and the server. This decision was made to allow clients to be created on any language. I know I could use json for simpler communication, but I needed a better Object to Text Mapping, so I ended up with XML. So, to diminish the XMl size problem, I decided to compress it before sending over socket. On top of it, I want to encrypt the data so it will be harder to be broken by some interceptor. I have read a lot about encrypting and encryption types, but for testing purposes I'll just use Base64 and generic compression just to join up all the technologies, than I'll specialize the specifics. I found a lot of code and good advice here on stack exchange, and now I'm having some trouble.
What I have:
In memory Objects;
Static methods to convert Objects to XML in memory;
No use for Serialization because the clients can be written on C, .net, perl, etc;
Avoid disk write to reduce disk I/O, keeping everything in-memory;
The need to work with punctuation chars (UTF-8);
The need to work with stronger encrypt on the future;
The process I idealized is:
Convert object to XML - OK
Establish Client/server communication - OK
Encode XML String to Base64 - OK (no compression)
3.1. Compress XML String and than encode to Base64 - OK
Transfer between client/server - OK
Decode Base64 String to XML String - OK
5.1. Uncompress transferred byte array and decode Base64 - NOT OK
So, I can't find where I'm getting wrong on the "Compress String -> Encode it to Base64 -> transmit -> decode Base64 to String -> uncompress it" process... Here is the snippet from my code "inspired" on stack overflow:
I'm using this answer (with .net compatibility) as a reference to compress/uncompress:
How can I Zip and Unzip a string using GZIPOutputStream that is compatible with .Net?
public class Compressor2 {
public static byte[] compress(String string) throws IOException {
byte[] blockcopy = ByteBuffer
.allocate(4)
.order(java.nio.ByteOrder.LITTLE_ENDIAN)
.putInt(string.length())
.array();
ByteArrayOutputStream os = new ByteArrayOutputStream(string.length());
GZIPOutputStream gos = new GZIPOutputStream(os);
gos.write(string.getBytes());
gos.close();
os.close();
byte[] compressed = new byte[4 + os.toByteArray().length];
System.arraycopy(blockcopy, 0, compressed, 0, 4);
System.arraycopy(os.toByteArray(), 0, compressed, 4, os.toByteArray().length);
compressed = MyBase64.encode(new String(compressed)).getBytes();
return compressed;
}
public static String decompress(byte[] compressed) throws IOException {
compressed = MyBase64.decode(new String(compressed)).getBytes();
System.out.println(compressed);
System.out.println(new String(compressed));
final int BUFFER_SIZE = 32;
//ByteArrayInputStream is = new ByteArrayInputStream(compressed, 4, compressed.length - 4);
ByteArrayInputStream is = new ByteArrayInputStream(compressed);
GZIPInputStream gis = new GZIPInputStream(is, BUFFER_SIZE);
StringBuilder string = new StringBuilder();
byte[] data = new byte[BUFFER_SIZE];
int bytesRead;
while ((bytesRead = gis.read(data)) != -1) {
string.append(new String(data, 0, bytesRead));
}
gis.close();
is.close();
return string.toString();
}
from MyBase 64:
public static String encode(String text) throws UnsupportedEncodingException {
byte[] encodedBytes = Base64.encodeBase64(text.getBytes());
return new String(encodedBytes, "UTF-8");
}
public static String decode(String text) throws UnsupportedEncodingException {
byte[] decodedBytes = Base64.decodeBase64(text);
return new String(decodedBytes, "UTF-8");
}
and the test case:
/**
* Test of compress method, of class Compressor2.
*/
#Test
public void testCompress() throws Exception {
System.out.println("compress");
String string = "Hello all, how can I transfer this with punctuation? like á é í ó ú";
byte[] expResult = Compressor2.compress(string);
byte[] result = Compressor2.compress(string);
System.out.write(result);
System.out.print("\n");
System.out.println(new String(result));
assertArrayEquals(expResult, result);
for (byte character : result) {
System.out.print(String.valueOf(character));
}
System.out.print("\n");
for (byte character : result) {
System.out.print((character));
}
System.out.print("\n");
}
/**
* Test of decompress method, of class Compressor2.
*/
#Test
public void testDecompress() throws Exception {
System.out.println("decompress");
String expResult = "Hello all, how can I transfer this with punctuation? like á é í ó ú";
//String expResult = "The lazy dog ate an apple";
byte[] compressed = Compressor2.compress(expResult);
System.out.println(expResult);
System.out.write(compressed);
System.out.print("\n");
System.out.println("going to decompress");
String result = Compressor2.decompress(compressed);
System.out.println(result);
assertEquals(expResult, result);
// TODO punctuation errors?
}
System Out for checking the values:
compress
QwAAAB/vv70IAAAAAAAAAAXvv73vv70N77+9MAwE77+9VX4Adu+/ve+/vTHvv73vv70oFu+/vRMl77+9Mg8lBRUd77+9F++/vW5j77+9AlJd77+977+9RCLvv70Ob2Tvv73vv70G77+977+9Me+/vTPvv73vv71c77+977+9UDkZcSFuxIN4Ee+/vQ/nn5ENSAAAAA==
QwAAAB/vv70IAAAAAAAAAAXvv73vv70N77+9MAwE77+9VX4Adu+/ve+/vTHvv73vv70oFu+/vRMl77+9Mg8lBRUd77+9F++/vW5j77+9AlJd77+977+9RCLvv70Ob2Tvv73vv70G77+977+9Me+/vTPvv73vv71c77+977+9UDkZcSFuxIN4Ee+/vQ/nn5ENSAAAAA==
811196565656647118118554873656565656565656565658811811855511181185548785555435777651196955554357868852651001174347118101434711884721181185551118118554811170117434711882771085555435777103561086682851005555435770434347118875310655554357651087410055554357555543578267761181185548799850841181185551118118554871555543575555435777101434711884801181185551118118554999555543575555435785681079099837011712073785269101434711881471101105369788365656565656161
811196565656647118118554873656565656565656565658811811855511181185548785555435777651196955554357868852651001174347118101434711884721181185551118118554811170117434711882771085555435777103561086682851005555435770434347118875310655554357651087410055554357555543578267761181185548799850841181185551118118554871555543575555435777101434711884801181185551118118554999555543575555435785681079099837011712073785269101434711881471101105369788365656565656161
decompress
Hello all, how can I transfer this with punctuation? like á é í ó ú
QwAAAB/vv70IAAAAAAAAAAXvv73vv70N77+9MAwE77+9VX4Adu+/ve+/vTHvv73vv70oFu+/vRMl77+9Mg8lBRUd77+9F++/vW5j77+9AlJd77+977+9RCLvv70Ob2Tvv73vv70G77+977+9Me+/vTPvv73vv71c77+977+9UDkZcSFuxIN4Ee+/vQ/nn5ENSAAAAA==
going to decompress
[B#19f9bdc4
C
Sorry for the long post, I'm trying to give you something to work on to check where I'm getting this wrong. I Appreciate any help...
Edit.: Forgot to say, if you take the "MyBase64.decode or encode" line from the Compressor2 Classe, it works like a charm...

Sending a C++ struct over UDP in Java

I'm a C++ programmer and have a need to set up some UDP communications between a java android app and the C++ server running on a PC.
I have structure that I need to receive on the PC that consists of the following:
int
int
float
Unfortunately I'm totally at a loss as to how I can do this with Java.
I need to create a DatagramPacket but the constructor only takes a byte array. Now with C++ this would be an easy cast from a struct to a char*. However casting like this is not possible with Java.
I've create a simple class that has the above fields in it. That seems to be fine. My remaining issue is how to turn that into a byte array. Can anyone help a Java noob on this front?
Cheers!
Edit: I've created a function in the class that does the following
public byte[] GetBytes() throws IOException
{
ByteArrayOutputStream byteOut = new ByteArrayOutputStream();
DataOutputStream dataOut = new DataOutputStream( byteOut );
dataOut.writeInt( Integer.reverseBytes( int1) );
dataOut.writeInt( Integer.reverseBytes( int2 ) );
dataOut.writeFloat( float1 );
return byteOut.toByteArray();
}
Is there a better way to do this?
I'd rather not use the google protocol buffer mentioned in Steve's answer because, while its interesting, it would require too many changes to other platform implementations that I'd really rather not do.

You can use Google protocol buffers as a language-independent way to serialize structures for transmission and receipt. Both Java and C++ are available out of the box, and Jon Skeet has written a production-ready C# implementation.
I see a couple of examples of Protobuf in use on Android, including this.

Another, maybe simpler approach comes from Javolution.struct: http://javolution.org/target/site/apidocs/javolution/io/Struct.html
public static class Student extends Struct {
public final Enum32<Gender> gender = new Enum32<Gender>(Gender.values());
public final UTF8String name = new UTF8String(64);
public final Date birth = inner(new Date());
public final Float32[] grades = array(new Float32[10]);
public final Reference32<Student> next = new Reference32<Student>();
}
class UDPMessage extends Struct {
Student student = inner(new Student());
...
}
...
public void run() {
byte[] bytes = new byte[1024];
DatagramPacket packet = new DatagramPacket(bytes, bytes.length);
UDPMessage message = new UDPMessage();
message.setByteBuffer(ByteBuffer.wrap(bytes), 0);
// packet and message are now two different views of the same data.
while (isListening) {
multicastSocket.receive(packet);
int xxx = message.xxx.get();
... // Process message fields directly.
}
}
Quite ugly piece of code, but still prettier than dealing directly with JNI buffers or already-mentioned Google protocol buffers.