Develop Reference

Converting ints and strings into byte array and the reverse

I would like to convert some ints and some strings into a single byte array and then back again. I've done a bit of research on how to do converting, but I'm not sure if its all correct.
Converting a string to a byte array is easy:
byte[] bytes = string.getBytes();
Converting it back again via Arrays.toString() because that just creates a string of the bytes.
Does this work: String s = new String(bytes);?
Converting ints to byte array is like this:
int[] data = { int1, int2, int3 };
ByteBuffer byteBuffer = ByteBuffer.allocate(data.length * 4);
IntBuffer intBuffer = byteBuffer.asIntBuffer();
intBuffer.put(data);
byte[] my_app_state = byteBuffer.array();
But I don't know how to convert it back again.
My aim is to have say 4 ints and 2 strings converted to a single byte array and then convert them back again.
For example. I have these objects and would like them to converted to the same byte array.
int int1 = 1;
int int2 = 2;
int int3 = 3;
int int4 = 4;
String s1 = "mystring1"
String s2 = "mystring2"
Update: Removed code where I thought there was a problem. There wasn't.

For each operation, you need to determine the reverse operation, not just any operation which returns the right type. for example, the reverse of n * 2 is m / 2 not m - 2 even though the type is right.
Arrays.toString("Hi".getBytes()) => "{ 72, 105 }"
So you can do
text.getBytes() => new String(bytes) // if the same character encoding is used.
a better option is
text.getBytes("UTF-8") => new String(bytes, "UTF-8");
The problem with an array is you have two pieces of information a length and some bytes If you just write the bytes, you no longer know the length and so you can't easily decode it (perhaps impossible)
In your case, the simplest option is to use a Data Stream
// buffer which grows as needed.
ByteArrayOutputStream boas = new ByteArrayOutputStream();
// supports basic data types
DataOutputStream dos = new DataOutputStream(baos);
dos.writeInt(data.length);
for(int i: data) dow.writeInt(i);
// write the length of the string + the UTF-8 encoding of the text.
dos.writeUTF(s1);
dos.writeUTF(s2);
byte[] bytes = bytes.toByteArray();
To do the reverse, you use the InputStream and the readXxxx instead of writeXxxx methods.

Java makes it very simple to achieve this, as this is a very common use case. What you need looks very much like Serialization.
Serialization works like this: A single object can be converted to a set of bytes and stored in a byte array (usually for writing to a file or sending over a network).
The good things is that any object can become serializable by just implementing a marker interface (just 1 line of code). Also, all Wrapper datatypes and String and Collections objects like ArrayList are serializable.
Coming to your question: Put all your data in a single object and serialize that object. 3 options come to my mind:
1. An Object[] or ArrayList (if you know the order for sure, so that you can access based on position)
2. A HashMap, (if you can assign a name to each of them instead of relying on position)
3. Create your own data type with fields like int1, int2 or even more meaningful names. (Your class should implement Serializable).
Now, all your data is added into a single object. Convert this one object to a byte array and your job is done.
Check this link for how to convert a single object to byte array:
Java Serializable Object to Byte Array
Object[] payload = new Object[]{int1, int2, int3, int4, string1, string2};
ByteArrayOutputStream out = new ByteArrayOutputStream();
ObjectOutputStream os = new ObjectOutputStream(out);
os.writeObject(payload);
byte[] result = out.toByteArray(); //Done

for store Strings as well you have to use some thing like
IntBuffer intBuffer = byteBuffer.asIntBuffer();
CharBuffer stringBuffer = byteBuffer.asCharBuffer();
then you have to traverse the loop on char[][] string = {s1.toCharArray(),s2.toCharArray()};
to put the each character in stringBuffer and more thing you have to do is you to make your byteBuffer to sufficient to hold these values my friend

Reading serial information in Java

I'm working on porting an Android app that has already been developed in Python. In the Python program, there is a line that I'm trying to fully understand:
self.comfd = Serial(...) # from the pySerial API
....
self.buffer = list(struct.unpack('192H', self.comfd.read(384)))
From what I understand, self.comfd.read(384) is reading 384 bytes, and the unpack('192H' is unpacking 192 unsigned shorts from that data. Is this correct?
Now in Java, I've been able to read the buffer using
SerialPort device = SerialPort(file, baud, flags);
InputStream in = device.getInputStream();
My question is, now that I have the input stream, how do I create the unsigned shorts like the Python program is doing?
What I've tried (not producing correct values):
byte[] buffer = new byte[384];
in.read(buffer);
ByteBuffer bb = ByteBuffer.allocate(2);
for (int i = 0; i < buffer.length / 2; i++) {
bb.put(buffer[i]);
bb.put(buffer[i + 1]);
short val = bb.getShort(0);
System.out.println(val);
bb.clear();
}
What am I doing wrong? Thanks for any help!
edit: I incorporated Jason C's answer and also I was looping incorrectly. By changing it to
for (int i = 0; i < buffer.length; i=i+2) that fixed my problem.

You could use a char (it's a 16-bit unsigned value in Java), e.g.:
byte[] buffer = ...;
ByteBuffer bb = ByteBuffer.wrap(buffer); // don't need to put()
int val = (int)bb.getChar(0);
Use bb.order() to set big- vs. little-endian.
You can also just pack the 2 bytes into an int (assuming little-endian) without using a ByteBuffer. Byte is signed in Java, so you will have to convert the byte to an unsigned value before shifting, which you can do by temporarily storing it in a short (or an int or anything large enough to hold 0-255):
short b0 = (buffer[0] & 255); // trick converts to unsigned
short b1 = (buffer[1] & 255);
int val = b0 | (b1 << 8);
// or just put it all inline:
int val = (buffer[0]&255) | ((buffer[1]&255) << 8);
For big-endian data just swap b0 and b1.
Hope that helps.

Java has no unsigned numbers (char is 16bit unsigned but it's not a number and math with char will always result in implicit casts to int)
If you read 2 bytes of unsigned data into a short and want to see values in range from 0-65535 (instead of -32768 - 32767) you'll have to use a type that can have values in that range.
In case of 16bit short next bigger one is 32bit int. The conversion that does the trick is
short signed = ...;
int unsigned = signed & 0xFFFF;
Assuming signed has a value of 0xFFFF this is what happens:
short signed = -1; // FFFF on byte level
The expression signed & 0xFFFF contains a short and an int. 0xFFFF is a literal integer type number which when found in Java source is considered int. You could make it long by changing it to 0xFFFFL (you would need that if you want to convert unsigned int to long).
Since the & operator needs both sides in a common type Java will silently convert the smaller one.
int stillSigned = (int) signed; // hidden step
It will still have the exact same value of -1 since that's what it was before when looking at it unsigned but it is changed on bytelevel to 0xFFFFFFFF.
Now the bit-manipulation is applied to remove all the added FFs
int unsigned = stillSigned & 0xFFFF;
and you end up with 0x0000FFFF on byte level and can finally see the value of 65535.
Since you happen to have 16bit values you can use char and simply cast it to int.
char value = ...;
int unsigned = value;
But above approach works for any unsigned conversion: byteValue & 0xFF, shortValue & 0xFFFF, intValue & 0xFFFFFFFFL
The next thing you should do is not to use a simple InputStream to do
SerialPort device = SerialPort(file, baud, flags);
InputStream in = device.getInputStream();
byte[] buffer = new byte[384];
in.read(buffer);
Reason is that InputStream#read(byte[]) is not guaranteed to read all the bytes you want in your buffer. It returns you the number of bytes it has read or -1 if the stream is done. Manually writing code that ensures you have a filled buffer is nasty but there is a simple solution: DataInputStream
SerialPort device = SerialPort(file, baud, flags);
DataInputStream in = new DataInputStream(device.getInputStream());
byte[] buffer = new byte[384];
in.readFully(buffer);
DataInputStream has very nice functionality that you could use:
SerialPort device = SerialPort(file, baud, flags);
DataInputStream in = new DataInputStream(device.getInputStream());
int unsignedShort = in.readUnsignedShort();
Another way to get different numbers out of byte[] data is to use ByteBuffer since that provides methods like .getShort()
SerialPort device = SerialPort(file, baud, flags);
DataInputStream in = new DataInputStream(device.getInputStream());
byte[] buffer = new byte[384];
in.readFully(buffer);
ByteBuffer byteBuffer = ByteBuffer.wrap(buffer);
while (byteBuffer.hasRemaining()) {
int unsigned = byteBuffer.getChar();
System.out.println(unsigned);
}

Convert array of doubles to bytearray in Java

In past I haven't done much of byte shifting so I'm a bit loss here. Basically I have double array of size 26 and I should send the array in one UDP packet in Java. I found some examples of how to convert one double to bytearray, but I'm not sure how to apply it to double-array.
So how this should be done? Loop through the double array and convert each double and somehow concatenating them to one bytearray?

Convert your doubles into a byte array using java.nio.ByteBuffer
ByteBuffer bb = ByteBuffer.allocate(doubles.length * 8);
for(double d : doubles) {
bb.putDouble(d);
}
get the byte array
byte[] bytearray = bb.array();
send it over the net and then convert it to double array on the receiving side
ByteBuffer bb = ByteBuffer.wrap(bytearray);
double[] doubles = new double(bytearray.length / 8);
for(int i = 0; i < doubles.length; i++) {
doubles[i] = bb.getDouble();
}

So how this should be done? Loop through the double array and convert each double and somehow concatenating them to one bytearray?
Exactly. You can make use of DoubleBuffer, perhaps. (Marko linked it in his comment)
What Marko referred to was having actually a ByteBuffer and fetching a "DoubleBuffer"-View to it. So you can put the Doubles into the DoubleBuffer View and fetch the byte[] from the original ByteBuffer.

apache httpcore provides a org.apache.http.util.ByteArrayBuffer class which my be helpful
ByteArrayBuffer buffer = new ByteArrayBuffer(26);
buffer.append(...)

Get single bytes from multi-byte variable in java

How can I split a variable into single bytes in java? I have for example following snippet in C++:
unsigned long someVar;
byte *p = (byte*)(void*) someVar; // byte being typedef unsigned char (from 0 to 255)
byte *bytes = new byte[sizeof(someVar)];
for(byte i = 0;i<sizeof(someVar);i++)
{
bytes[i] = *p++;
}
.... //do something with bytes
I want to accomplish the same under java, but I can't seem to find an obvious workaround.

There are two ways to do it with the ByteBuffer class. One is to create a new byte array dynamically.
long value = 123;
byte[] bytes = ByteBuffer.allocate(8).putLong(value).array();
Another is to write to an existing array.
long value = 123;
byte[] bytes = new byte[8];
ByteBuffer.wrap(bytes).putLong(value);
// bytes now contains the byte representation of 123.

If you use Guava, there is a convenience Longs.toByteArray. It is simply a wrapper for John's ByteBuffer answer above, but if you already use Guava, it's slightly "nicer" to read.

Left shift unsigned byte, a better way?

I have an array of bytes (because unsigned byte isn't an option) and need to take 4 of them into a 32 bit int. I'm using this:
byte rdbuf[] = new byte[fileLen+1];
int i = (rdbuf[i++]) | ((rdbuf[i++]<<8)&0xff00) | ((rdbuf[i++]<<16)&0xff0000) | ((rdbuf[i++]<<24)&0xff000000);
If i don't do all the logical ands, it sign extends the bytes which is clearly not what I want.
In c this would be a no brainer. Is there a better way in Java?

You do not have to do this, you can use a ByteBuffer:
int i = ByteBuffer.wrap(rdbuf).order(ByteOrder.LITTLE_ENDIAN).getInt();
If you have many ints to read, the code becomes:
ByteBuffer buf = ByteBuffer.wrap(rdbuf).order(ByteOrder.LITTLE_ENDIAN);
while (buf.remaining() >= 4) // at least four bytes
i = bb.getInt();
Javadoc here. Recommended for use in any situation where binary data has to be dealt with (whether you read or write such data). Can do little endian, big endian and even native ordering. (NOTE: big endian by default).
(edit: #PeterLawrey rightly mentions that this looks like little endian data, fixed code extract -- also, see his answer for how to wrap the contents of a file directly into a ByteBuffer)
NOTES:
ByteOrder has a static method called .nativeOrder(), which returns the byte order used by the underlying architecture;
a ByteBuffer has a builtin offset; the current offset can be queried using .position(), and modified using .position(int); .remaining() will return the number of bytes left to read from the current offset until the end;
there are relative methods which will read from/write at the buffer's current offset, and absolute methods, which will read from/write at an offset you specify.

Instead of reading into a byte[] which you have to wrap with a ByteBuffer which does the shift/mask for you, you can use a direct ByteBuffer which avoid all this overhead.
FileChannel fc = new FileInputStream(filename).getChannel();
ByteBuffer bb = ByteBuffer.allocateDirect(fc.size()).order(ByteBuffer.nativeOrder());
fc.read(bb);
bb.flip();
while(bb.remaining() > 0) {
int n = bb.getInt(); // grab 32-bit from direct memory without shift/mask etc.
short s = bb.getShort(); // grab 16-bit from direct memory without shift/mask etc.
// get a String with an unsigned 16 bit length followed by ISO-8859-1 encoding.
int len = bb.getShort() & 0xFFFF;
StringBuilder sb = new StringBuilder(len);
for(int i=0;i<len;i++) sb.append((char) (bb.get() & 0xFF));
String text = sb.toString();
}
fc.close();