Develop Reference

Java: custom to byte conversions

I am working with some low capacity module and I need to compress the data as much as possible. The data will look like this:
DeviceEvent:
1 byte:
2 bits for status (00 each time)
6 bits for rgb color (3 x 2 bits)
2 bytes: number of minutes from now to a certain datetime
I need to create a constructor (preferably 2 constructors) for conversion from/to:
Event:
byte[] color (rgb, colors will get simplified to only 64 available)
some datetime (but I will get the integer for difference in minutes and it will be small enough to fit in two bits)
So basically I need:
byte[3] color <-> 1 byte status and color
int minutes <-> byte[2]
minutes
I will be thankful for any help

I'm not very sure what is your problem, probably this will help:
final byte red = 1; // 01 binary
final byte green = 2; // 10 binary
final byte blue = 3; // 11 binary
final byte finalColor = (byte) ((red & 0x3) << 4) | ((green & 0x3) << 2) | (blue & 0x3);
System.out.println(finalColor);// finalColor is 011011 = 27 decimal
final int minutes = 0x1234; // first byte is 0x12, second byte is 0x34
final byte[] bytes = {(byte) (((minutes) >>> 8) & 0xff), (byte) (minutes & 0xff)};
System.out.println(bytes[0]); // 0x12 = 18 decimal
System.out.println(bytes[1]); // 0x34 = 52 decimal

I am not sure what the second problem is. So I made these two functions that might help you:
public static int convertToInt(int a, int b, int c, int d) {
a = Math.min(a, 255);
b = Math.min(b, 255);
c = Math.min(c, 255);
d = Math.min(d, 255);
return ((a & 0xFF) << 24) | ((b & 0xFF) << 16) | ((c & 0xFF) << 8) | (d & 0xFF);
}
public static int[] extractComponents(int data) {
int a = (data >> 24) & 0xFF;
int b = (data >> 16) & 0xFF;
int c = (data >> 8) & 0xFF;
int d = data & 0xFF;
return new int[] {a, b, c, d};
}
The convertToInt function takes four numbers(that are less than 255) and puts them all in one int.
The extractComponents function does the opposite.
This is an example:
int data = 0xC8E0601B;
int[] dataA = extractComponents(data);
for(int i = 0 ; i < dataA.length; i++) System.out.printf("%x\n", dataA[i]);
System.out.printf("%x\n", convertToInt(dataA[0], dataA[1], dataA[2], dataA[3]));

Converting Byte[4] to float - Integer[4] array works but byte[4] does not

This is probably a basic question for out more experienced programmers out there. I'm a bit of a noob and can't work this one out. I'm trying to unpack a binary file and the doco is not too clear on how floats are stored. I have found a routine that does this, but it will only work if I pass an integer array of the bytes. The correct answer is -1865.0. I need to be able to pass the byte array and get the correct answer. How do I need to change the code to make float4byte return -1865.0. Thanks in advance.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
public class HelloWorld {
public static void main(String[] args) {
byte[] bytes = {(byte) 0xC3,(byte) 0X74,(byte) 0X90,(byte) 0X00 };
int[] ints = {(int) 0xC3,(int) 0X74,(int) 0X90,(int) 0X00 };
// This give the wrong answer
float f = ByteBuffer.wrap(bytes).order(ByteOrder.BIG_ENDIAN).getFloat();
System.out.println("VAL ByteBuffer BI: " + f);
// This give the wrong answer
f = ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN).getFloat();
System.out.println("VAL ByteBuffer LI: " + f);
//This gives the RIGHT answer
f = float4int (ints[0], ints[1], ints[2], ints[3]);
System.out.println("VAL Integer : " + f);
// This gives the wrong answer
f = float4byte (bytes[0], bytes[1], bytes[2], bytes[3]);
System.out.println("VAL Bytes : " + f);
}
private static float float4int(int a, int b, int c, int d)
{
int sgn, mant, exp;
System.out.println ("IN Int: "+String.format("%02X ", a)+
String.format("%02X ", b)+String.format("%02X ", c)+String.format("%02X ", d));
mant = b << 16 | c << 8 | d;
if (mant == 0) return 0.0f;
sgn = -(((a & 128) >> 6) - 1);
exp = (a & 127) - 64;
return (float) (sgn * Math.pow(16.0, exp - 6) * mant);
}
private static float float4byte(byte a, byte b, byte c, byte d)
{
int sgn, mant, exp;
System.out.println ("IN Byte : "+String.format("%02X ", a)+
String.format("%02X ", b)+String.format("%02X ", c)+String.format("%02X ", d));
mant = b << 16 | c << 8 | d;
if (mant == 0) return 0.0f;
sgn = -(((a & 128) >> 6) - 1);
exp = (a & 127) - 64;
return (float) (sgn * Math.pow(16.0, exp - 6) * mant);
}
}

The reason why your solution with ByteBuffer doesn't work: the bytes do not match the (Java) internal representation of the float value.
The Java representation is
System.out.println(Integer.toHexString(Float.floatToIntBits(-1865.0f)));
which gives c4e92000

bytes are signed in Java. When calculating the mantissa mant, the bytes are implicitly converted from bytes to ints - with the sign "extended", i.e. (byte)0x90 (decimal -112) gets converted 0xFFFFFF90 (32 bits int). However what you want is just the original bytes' 8 bits (0x00000090).
In order to compensate for the effect of sign extension, it suffices to change one line:
mant = (b & 0xFF) << 16 | (c & 0xFF) << 8 | (d & 0xFF)
Here, in (c & 0xFF), the 1-bits caused by sign extension are stripped after (implicit) conversion to int.
Edit:
The repacking of floats could be done via the IEEE 754 representation which can be obtained by Float.floatToIntBits (which avoids using slow logarithms). Some complexity in the code is caused by the change of base from 2 to 16:
private static byte[] byte4float(float f) {
assert !Float.isNaN(f);
// see also JavaDoc of Float.intBitsToFloat(int)
int bits = Float.floatToIntBits(f);
int s = (bits >> 31) == 0 ? 1 : -1;
int e = (bits >> 23) & 0xFF;
int m = (e == 0) ? (bits & 0x7FFFFF) << 1 : (bits& 0x7FFFFF) | 0x800000;
int exp = (e - 150) / 4 + 6;
int mant;
int mantissaShift = (e - 150) % 4; // compensate for base 16
if (mantissaShift >= 0) mant = m << mantissaShift;
else { mant = m << (mantissaShift + 4); exp--; }
if (mant > 0xFFFFFFF) { mant >>= 4; exp++; } // loose of precision
byte a = (byte) ((1 - s) << 6 | (exp + 64));
return new byte[]{ a, (byte) (mant >> 16), (byte) (mant >> 8), (byte) mant };
}
The code does not take into account any rules that may exist for the packaging, e.g. for representing zero or normalization of the mantissa. But it might serve as a starting point.

Thanks to #halfbit and a bit of testing and minor changes, this routine appears convert IEEE 754 float into IBM float.
public static byte[] byte4float(float f) {
assert !Float.isNaN(f);
// see also JavaDoc of Float.intBitsToFloat(int)
int bits = Float.floatToIntBits(f);
int s = (bits >> 31) == 0 ? 1 : -1;
int e = (bits >> 23) & 0xFF;
int m = (e == 0) ? (bits & 0x7FFFFF) << 1 : (bits& 0x7FFFFF) | 0x800000;
int exp = (e - 150) / 4 + 6;
int mant;
int mantissaShift = (e - 150) % 4; // compensate for base 16
if (mantissaShift >= 0) mant = m >> mantissaShift;
else mant = m >> (Math.abs(mantissaShift));
if (mant > 0xFFFFFFF) { mant >>= 4; exp++; } // loose of precision */
byte a = (byte) ((1 - s) << 6 | (exp + 64));
return new byte[]{ a, (byte) (mant >> 16), (byte) (mant >> 8), (byte) mant };
}
I think this is right and appears to be working.

How to pack and unpack a long variable given a particular format?

I am packing my address variable which is long datatype and it is made up of datacenter, client_id, data_id and data_counter as shown below -
long address = client_data((byte) 2, (short) 100, (byte) 22, (int) 120);
private static long client_data(byte datacenter, short client_id, byte data_id, int data_counter) {
return ((long) (datacenter) << 56) | ((long) client_id << 40) | ((long) data_id << 32) | ((long) data_counter);
}
Now I want to extract datacenter, client_id, data_id and data_counter back from address variable? So I started with the below code -
To extract datacenter this is what I am using -
value = ((address >>> 56) & ((1 << 8) - 1));
To extract client_id this is what I am using -
value = ((address >>> 40) & ((1 << 16) - 1));
To extract data_id this is what I am using -
value = ((address >>> 32) & ((1 << 8) - 1));
But how do I extract data_counter? Below piece of code doesn't work for me.
value = ((address) & ((1 << 32) - 1));

(1<<32) won't do what you want because 1 is treated as a 32 bit int by default. So it tries to calculate (1<<32) as a 32 bit int. But a 32 bit int doesn't have enough bits to hold (1<<32).
Use (1L<<32). Then you will get a long, which is big enough to hold that number.
Edit:
For clarity, so you end up with:
value = (address & ((1L << 32) - 1));

Here are a couple of suggestions. Using the byte extraction method that you have proposed, it might be worth considering expressing shifts and masks in terms of Byte.SIZE, Short.SIZE and Integer.SIZE respectively. Something like this:
public static long BYTE_MASK = (1L << Byte.SIZE) - 1;
public static long SHORT_MASK = (1L << Short.SIZE) - 1;
public static long INT_MASK = (1L << Integer.SIZE) - 1;
public static long DATACENTER_SHIFT = Integer.SIZE + Byte.SIZE + Short.SIZE;
// ...and so on
Alternatively, you could use a ByteBuffer to decode the value. Something like this:
long value = 0x0200641600000078L; // example data from question
ByteBuffer bb = ByteBuffer.allocate(Long.BYTES);
bb.putLong(value);
bb.flip();
byte datacenter = bb.get();
short client_id = bb.getShort();
byte data_id = bb.get();
int data_counter = bb.getInt();
This might be worth considering if your incoming data is an array of bytes, perhaps from reading an InputStream, as you would be able to avoid assembling into a long and then disassembling into component pieces. In this case, you could use ByteBuffer.wrap(incomingByteArray) instead of allocate/putLong/flip.

Negative results from bytes

I have this line below where byte[] shaBytes with value from a SHA1. The first four is added to result to a number which is used in a while loop. I'm trying to port an objC code to java while the line below results to a negative value such as -2063597568 after several iterations.
long tVar = (shaBytes[0] << 24) + (shaBytes[1] << 16) + (shaBytes[2] << 8) + (shaBytes[3] << 3);
So basically, the while loop loops when
tVar > 0xFFFFFFFFL >> 11
In objC shaBytes is an unsigned char which is used as parameter in a CC_SHA1. In objC the code would loop up 700+ iterations while my port only 3 because tVar becomes negative.

Java doesn't have unsigned bytes. All Java integer types are signed. This means that the left shift of a negative byte will be negative:
byte b = -30;
long x = b << 24;
System.out.printf("x = %d\n", x);
// prints -503316480
On other hand if you convert byte to long everything'll turn out okay:
byte b = -30;
long x = (b & 0xffL) << 24;
System.out.printf("x = %d\n", x);
// prints 3791650816
To convert byte to "unsigned" (remember, there is no unsigned stuff in Java) long value use:
long tVar = ((shaBytes[0] & 0xffL) << 24) + ((shaBytes[1] & 0xffL) << 16) + etc
This'll work because Java long is 64-bit and can handle 24 left shift of 8-bit value without signed/unsigned problems.

How do I split an integer into 2 byte binary?

Given
private int width = 400;
private byte [] data = new byte [2];
I want to split the integer "width" into two bytes and load data[0] with the high byte and data[1] with the low byte.
That is binary value of 400 = 1 1001 0000
so data[0] should contain 0000 0001
and data[1] should contain 1001 0000

Using simple bitwise operations:
data[0] = (byte) (width & 0xFF);
data[1] = (byte) ((width >> 8) & 0xFF);
How it works:
& 0xFF masks all but the lowest eight bits.
>> 8 discards the lowest 8 bits by moving all bits 8 places to the right.
The cast to byte is necessary because these bitwise operations work on an int and return an int, which is a bigger data type than byte. The case is safe, since all non-zero bits will fit in the byte. For more information, see Conversions and Promotions.
Edit: Taylor L correctly remarks that though >> works in this case, it may yield incorrect results if you generalize this code to four bytes (since in Java an int is 32 bits). In that case, it's better to use >>> instead of >>. For more information, see the Java tutorial on Bitwise and Bit Shift Operators.

For converting two bytes the cleanest solution is
data[0] = (byte) width;
data[1] = (byte) (width >>> 8);
For converting an integer to four bytes the code would be
data[0] = (byte) width;
data[1] = (byte) (width >>> 8);
data[2] = (byte) (width >>> 16);
data[3] = (byte) (width >>> 24);
It doesn't matter whether >> or >>> is used for shifting, any one bits created by sign extension will not end up in the resulting bytes.
See also this answer.

This should do what you want for a 4 byte int. Note, it stores the low byte at offset 0. I'll leave it as an exercise to the reader to order them as needed.
public static byte[] intToBytes(int x) {
byte[] bytes = new byte[4];
for (int i = 0; x != 0; i++, x >>>= 8) {
bytes[i] = (byte) (x & 0xFF);
}
return bytes;
}

Integer is 32 bits (=4 bytes) in java, you know?
width & 0xff will give you the first byte,
width & 0xff00 >> 8 will give you the second, etc.

To get the high byte, shift right by 8 bits then mask off the top bytes. Similarly, to get the low byte just mask off the top bytes.
data[0] = (width >> 8) & 0xff;
data[1] = width & 0xff;

int width = 400;
byte [] data = new byte [2];
data[0] = (byte) ((width & 0xFF00) >> 8);
data[1] = (byte) (width & 0xFF);
for(int b = 0; b < 2; b++) {
System.out.println("printing byte " + b);
for(int i = 7; i >= 0; i--) {
System.out.println(data[b] & 1);
data[b] = (byte) (data[b] >> 1);
}
}

I suggest you have a look at the source for HeapByteBuffer. It has the conversion code for all primitive data types. (In fact you could just use a ByteBuffer ;)