The following code attempts to store 4 longs in a byte array. Random access is important which is why I'm not doing this with byte streams. Why is the below code not working? Can you think of a more efficent way to do this?
public static void storeLong(long value, byte[] buf, int offset) {
buf[offset] = (byte) (value & 0xFFL);
buf[offset+1] = (byte) ((value >>> 8) & 0xFFL);
buf[offset+2] = (byte) ((value >>> 16) & 0xFFL);
buf[offset+3] = (byte) ((value >>> 24) & 0xFFL);
buf[offset+4] = (byte) ((value >>> 32) & 0xFFL);
buf[offset+5] = (byte) ((value >>> 40) & 0xFFL);
buf[offset+6] = (byte) ((value >>> 48) & 0xFFL);
buf[offset+7] = (byte) ((value >>> 56) & 0xFFL);
}
public static long retrieveLong(byte[] buf, int offset) {
return ((long)buf[offset])
+ (((long)buf[offset+1])<<8)
+ (((long)buf[offset+2])<<16)
+ (((long)buf[offset+3])<<24)
+ (((long)buf[offset+4])<<32)
+ (((long)buf[offset+5])<<40)
+ (((long)buf[offset+6])<<48)
+ (((long)buf[offset+7])<<56);
}
public static void main(String[] args) {
byte[] buf = new byte[32];
storeLong(-1, buf, 0);
storeLong(1, buf, 8);
storeLong(Long.MAX_VALUE, buf, 16);
storeLong(Long.MIN_VALUE, buf, 24);
System.out.println(-1);
System.out.println(1);
System.out.println(Long.MAX_VALUE);
System.out.println(Long.MIN_VALUE);
System.out.println(retrieveLong(buf, 0));
System.out.println(retrieveLong(buf, 8));
System.out.println(retrieveLong(buf, 16));
System.out.println(retrieveLong(buf, 24));
}
The output I get from the above is the following. You can see that the first four numbers do not match the next 4:
-1
1
9223372036854775807
-9223372036854775808
-72340172838076673
1
9151031864016699135
-9223372036854775808
Don't use + and byte is signed:
public static long retrieveLong(byte[] buf, int offset) {
return ((long)buf[offset] & 255)
| (((long)buf[offset + 1] & 255) << 8)
| (((long)buf[offset + 2] & 255) << 16)
| (((long)buf[offset + 3] & 255) << 24)
| (((long)buf[offset + 4] & 255) << 32)
| (((long)buf[offset + 5] & 255) << 40)
| (((long)buf[offset + 6] & 255) << 48)
| (((long)buf[offset + 7] & 255) << 56);
}
You have to and each byte with 255 to make it 'unsigned'. Also you have to use binary or instead of add.
I did some tests and found that using a java.nio.LongBuffer is twice as fast as my code
ByteBuffer bb = ByteBuffer.allocate(4*8);
LongBuffer lb = bb.asLongBuffer();
lb.put(0, -1);
lb.put(1, 1);
lb.put(2, Long.MAX_VALUE);
lb.put(3, Long.MIN_VALUE);
System.out.println(lb.get(0));
System.out.println(lb.get(1));
System.out.println(lb.get(2));
System.out.println(lb.get(3));
I can then get the byte array using bb.array()
Thanks to Louis Wasserman and Rene Jeschke for their efforts
Related
I have an array of 5 bytes (fixed length). First 21 bits represent a counter and the rest 19 some id. I need to increment the counter by one. How do I do that?
This answer is based on the memory layout as I understand it from the OP's comments:
array[0]: bits 0..7 are counter bits 0..7 (lsb)
array[1]: bits 0..7 are counter bits 8..15
array[2]: bits 0..4 are counter bits 16..20 (msb)
bits 5..7 are part of the id
array[3]: bits 0..7 are part of the id
array[3]: bits 0..7 are part of the id
Then
int byte0 = (int) array[0] & 0xff;
int byte1 = (int) array[1] & 0xff;
int byte2 = (int) array[2] & 0x1f;
int count = byte0 | (byte1 << 8) | (byte2 << 16);
count = (count+1) & 0x1fffff;
array[0] = (byte) (count & 0x0000ff);
array[1] = (byte) ((count & 0x00ff00) >> 8);
array[2] = (array[2] & 0xe0) | (byte) ((count & 0x1f0000) >> 16);
should do the trick.
I'm not sure I solved your problem. Since the memory layout is not described in detail, I assumed a layout. You can obtain the counter, increase it by 1, and then set the counter. If the layout is as desired, it needs to be fully tested. I'm not dealing with data overflow situations, and you may need to add some restrictions.
public class Test {
public static void main(String[] args) {
byte[] bytes = new byte[5];
bytes[2] = 5;
for (int i = 0; i <= 0x1fffff; i++) {
setCount(bytes, i);
if (getCount(bytes) != i) {
System.out.println(i);
debug(bytes);
}
}
}
public static int getCount(byte[] bytes) {
return ((bytes[2] >> 3) & 0x1f) + ((bytes[1] << 5) & 0x1fff) + ((bytes[0] << 13) & 0x1fffff);
}
public static void setCount(byte[] bytes, int count) {
bytes[0] = (byte) (count >> 13 & 0xff);
bytes[1] = (byte) (count >> 5 & 0xff);
bytes[2] = (byte) ((bytes[2] & 0x7) + ((count & 0x1f) << 3));
}
public static void debug(byte[] bytes) {
for (byte b : bytes) {
for (int i = 7; i >= 0; i--) {
System.out.print(b >> i & 1);
}
System.out.print(" ");
}
System.out.println();
System.out.println("Count:" + getCount(bytes));
}
public static void printInt(int num) {
for (int i = 31; i >= 0; i--) {
System.out.print(num >> i & 1);
}
System.out.println();
}
}
I'm not sure I solved your problem.
public class Main
{
public static void main(String[] args)
{
//################
//# BIG ENDIAN #
//################
//initial counter == 1 (b0000_0000_0000_0000_0000_1)
//initial ids == 3 (b000_0000_0000_0000_0011)
byte[] array = { 0, 0, 8, 0, 3 };
int counter = ((array[0] & 0xFF) << 13)
| ((array[1] & 0xFF) << 5)
| ((array[2] & 0xF8) >> 3);
System.out.println(counter); //1
++counter;
System.out.println(counter); //2
//Repack the counter into the array.
array[0] = (byte)((counter & 0x1FE000) >> 13);
array[1] = (byte)((counter & 0x1FE0) >> 5);
array[2] = (byte)(((counter & 0x1F) << 3)| (array[2] & 0x7));
System.out.println(Arrays.toString(array)); //[0, 0, 16, 0, 3]
//Retry & Verify
counter = ((array[0] & 0xFF) << 13) //Same as above. Nothing is changed.
| ((array[1] & 0xFF) << 5)
| ((array[2] & 0xF8) >> 3);
System.out.println(counter); //2
++counter;
System.out.println(counter); //3
//###################
//# LITTLE ENDIAN #
//###################
//initial ids == 3 (b0000_0000_0000_0000_011)
//initial counter == 1 (b0_0000_0000_0000_0000_0001)
byte[] arr = { 0, 0, 96, 0, 1 };
counter = ((arr[2] & 0x1F) << 16)
| ((arr[3] & 0xFF) << 8)
| (arr[4] & 0xFF);
System.out.println(counter); //1
++counter;
System.out.println(counter); //2
//Repack the counter into the array.
arr[2] = (byte)((arr[2] & 0xE0) | ((counter & 0x1F0000) >> 16));
arr[3] = (byte)((counter & 0xFF00) >> 8);
arr[4] = (byte)(counter & 0xFF);
System.out.println(Arrays.toString(arr)); //[0, 0, 96, 0, 2]
//Retry & Verify
counter = ((arr[2] & 0x1F) << 16) //Same as above. Nothing is changed.
| ((arr[3] & 0xFF) << 8)
| (arr[4] & 0xFF);
System.out.println(counter); //2
++counter;
System.out.println(counter); //3
}
}
This question already has answers here:
How do I convert Long to byte[] and back in java
(18 answers)
Closed 6 years ago.
I have made long value from byte array using this code
byte[] by = {07, 53 -70, 74};
long value = 0;
for (int i = 0; i < by.length; i++) {
value = ((value << 8) + (by[i] & 0xff));
}
System.out.println(value);
out put is 520010
now I want reverse process on this and I tried it this way
long ts = 520010;
tm_stp[0] = (byte) ((byte) ts>>24);
tm_stp[1] = (byte) ((byte) ts>>16);
tm_stp[2] = (byte) ((byte) ts>>8);
tm_stp[3] = (byte) ts;
for (byte b : tm_stp) {
System.out.println(b);
}
and output is 0 0 0 74
what is wrong in my second part of code please help me, Thanks!
The problem is the fact that you cast to byte too early:
tm_stp[0] = (byte) ((byte) ts>>24);
tm_stp[1] = (byte) ((byte) ts>>16);
tm_stp[2] = (byte) ((byte) ts>>8);
//^^^^
This causes the ts value to be truncated, replacing the first 24 bytes with 0s. After that, shifting by anything greater than or equal to 8 will return 0. To avoid incorrect behaviour with negative values, you should also use a bitmask. The correct code should look like this:
tm_stp[0] = (byte) ((ts >> 24) & 0xFF);
tm_stp[1] = (byte) ((ts >> 16) & 0xFF);
tm_stp[2] = (byte) ((ts >> 8) & 0xFF);
tm_stp[3] = (byte) ((ts >> 0) & 0xFF); // >> 0 not actually required, only for symmetry
Don't convert ts to byte before shifting.
tm_stp[0] = (byte) (ts >> 24);
tm_stp[1] = (byte) (ts >> 16);
tm_stp[2] = (byte) (ts >> 8);
tm_stp[3] = (byte) ts;
This is probably a basic question for out more experienced programmers out there. I'm a bit of a noob and can't work this one out. I'm trying to unpack a binary file and the doco is not too clear on how floats are stored. I have found a routine that does this, but it will only work if I pass an integer array of the bytes. The correct answer is -1865.0. I need to be able to pass the byte array and get the correct answer. How do I need to change the code to make float4byte return -1865.0. Thanks in advance.
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
public class HelloWorld {
public static void main(String[] args) {
byte[] bytes = {(byte) 0xC3,(byte) 0X74,(byte) 0X90,(byte) 0X00 };
int[] ints = {(int) 0xC3,(int) 0X74,(int) 0X90,(int) 0X00 };
// This give the wrong answer
float f = ByteBuffer.wrap(bytes).order(ByteOrder.BIG_ENDIAN).getFloat();
System.out.println("VAL ByteBuffer BI: " + f);
// This give the wrong answer
f = ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN).getFloat();
System.out.println("VAL ByteBuffer LI: " + f);
//This gives the RIGHT answer
f = float4int (ints[0], ints[1], ints[2], ints[3]);
System.out.println("VAL Integer : " + f);
// This gives the wrong answer
f = float4byte (bytes[0], bytes[1], bytes[2], bytes[3]);
System.out.println("VAL Bytes : " + f);
}
private static float float4int(int a, int b, int c, int d)
{
int sgn, mant, exp;
System.out.println ("IN Int: "+String.format("%02X ", a)+
String.format("%02X ", b)+String.format("%02X ", c)+String.format("%02X ", d));
mant = b << 16 | c << 8 | d;
if (mant == 0) return 0.0f;
sgn = -(((a & 128) >> 6) - 1);
exp = (a & 127) - 64;
return (float) (sgn * Math.pow(16.0, exp - 6) * mant);
}
private static float float4byte(byte a, byte b, byte c, byte d)
{
int sgn, mant, exp;
System.out.println ("IN Byte : "+String.format("%02X ", a)+
String.format("%02X ", b)+String.format("%02X ", c)+String.format("%02X ", d));
mant = b << 16 | c << 8 | d;
if (mant == 0) return 0.0f;
sgn = -(((a & 128) >> 6) - 1);
exp = (a & 127) - 64;
return (float) (sgn * Math.pow(16.0, exp - 6) * mant);
}
}
The reason why your solution with ByteBuffer doesn't work: the bytes do not match the (Java) internal representation of the float value.
The Java representation is
System.out.println(Integer.toHexString(Float.floatToIntBits(-1865.0f)));
which gives c4e92000
bytes are signed in Java. When calculating the mantissa mant, the bytes are implicitly converted from bytes to ints - with the sign "extended", i.e. (byte)0x90 (decimal -112) gets converted 0xFFFFFF90 (32 bits int). However what you want is just the original bytes' 8 bits (0x00000090).
In order to compensate for the effect of sign extension, it suffices to change one line:
mant = (b & 0xFF) << 16 | (c & 0xFF) << 8 | (d & 0xFF)
Here, in (c & 0xFF), the 1-bits caused by sign extension are stripped after (implicit) conversion to int.
Edit:
The repacking of floats could be done via the IEEE 754 representation which can be obtained by Float.floatToIntBits (which avoids using slow logarithms). Some complexity in the code is caused by the change of base from 2 to 16:
private static byte[] byte4float(float f) {
assert !Float.isNaN(f);
// see also JavaDoc of Float.intBitsToFloat(int)
int bits = Float.floatToIntBits(f);
int s = (bits >> 31) == 0 ? 1 : -1;
int e = (bits >> 23) & 0xFF;
int m = (e == 0) ? (bits & 0x7FFFFF) << 1 : (bits& 0x7FFFFF) | 0x800000;
int exp = (e - 150) / 4 + 6;
int mant;
int mantissaShift = (e - 150) % 4; // compensate for base 16
if (mantissaShift >= 0) mant = m << mantissaShift;
else { mant = m << (mantissaShift + 4); exp--; }
if (mant > 0xFFFFFFF) { mant >>= 4; exp++; } // loose of precision
byte a = (byte) ((1 - s) << 6 | (exp + 64));
return new byte[]{ a, (byte) (mant >> 16), (byte) (mant >> 8), (byte) mant };
}
The code does not take into account any rules that may exist for the packaging, e.g. for representing zero or normalization of the mantissa. But it might serve as a starting point.
Thanks to #halfbit and a bit of testing and minor changes, this routine appears convert IEEE 754 float into IBM float.
public static byte[] byte4float(float f) {
assert !Float.isNaN(f);
// see also JavaDoc of Float.intBitsToFloat(int)
int bits = Float.floatToIntBits(f);
int s = (bits >> 31) == 0 ? 1 : -1;
int e = (bits >> 23) & 0xFF;
int m = (e == 0) ? (bits & 0x7FFFFF) << 1 : (bits& 0x7FFFFF) | 0x800000;
int exp = (e - 150) / 4 + 6;
int mant;
int mantissaShift = (e - 150) % 4; // compensate for base 16
if (mantissaShift >= 0) mant = m >> mantissaShift;
else mant = m >> (Math.abs(mantissaShift));
if (mant > 0xFFFFFFF) { mant >>= 4; exp++; } // loose of precision */
byte a = (byte) ((1 - s) << 6 | (exp + 64));
return new byte[]{ a, (byte) (mant >> 16), (byte) (mant >> 8), (byte) mant };
}
I think this is right and appears to be working.
I convert a short number to a 3 byte array using the following code:
static byte[] convertTo3ByteArray(short s) {
byte[] ret = new byte[3];
ret[0] = (byte) (s & 0xff);
ret[1] = (byte) ((s >> 8) & 0xff);
ret[2] = (byte) (0x00);
return ret;
}
This works very well.
I found a code on Stackoverflow to convert the array back to a number:
static int convertToInt(byte[] b) {
return ((b[0] << 0) | (b[1] << 8) | (b[2] << 16));
}
And when I convert 258 to byte array, and then use this code, it returns 258.
But for number 675, this code returns -93.
How do I have to change the convertToShort method to get 675 back?
I suppose it has something to do with bitshift and loss of data? Or signed bytes?
Try with this modified method:
static int convertToShort(byte[] b) {
return (((b[0] & 0xFF) << 0) | ((b[1] & 0xFF) << 8) | ((b[2] & 0xFF) << 16));
}
In the array some bytes are negative, you need to convert them back to "positive values" with byteVal & 0xFF before doing the bit shift
A short has 16 bits of information, so that would be two bytes. When you try to store a third byte with | (b[2] << 16), it would go off the end of the short's bits, which is a problem. I.e. you can't do what you want to.
Changing to using a char datatype will fix this issue as they are the only unsigned type in Java:
https://stackoverflow.com/a/21089624/1590490
static char[] convertTo3ByteArray(short s) {
char[] ret = new char[3];
ret[0] = (char) (s & 0xff);
ret[1] = (char) ((s >> 8) & 0xff);
ret[2] = (char) (0x00);
return ret;
}
static int convertToShort(char[] b) {
return ((b[0]) | (b[1] << 8) | (b[2] << 16)); // the original << 0 shift does nothing
}
I would like to convert a signed int into a signed byte[] array, and later convert it back into a signed int.
However, ByteBuffers (The usual int->buffer->byte[] array) are too slow for this case.
Can this be done using basic operations?
I've seem many attempts, but I haven't seen one that works in all cases. (Usually, they fail for negative numbers.)
I am working in Java, so it is not possible to use unsigned values, even in intermediate steps.
private void writeInt(int val, byte[] data, int offset) {
data[offset ] = (byte)(val >>> 24);
data[offset + 1] = (byte)(val >>> 16);
data[offset + 2] = (byte)(val >>> 8);
data[offset + 3] = (byte)val;
}
private int readInt(byte[] data, int offset) {
return (data[offset] << 24)
| ((data[offset + 1] & 0xFF) << 16)
| ((data[offset + 2] & 0xFF) << 8)
| (data[offset + 3] & 0xFF);
}