I have some weird data that's stored in 5 bytes and I need to be able to convert it to an int to be able to manipulate it easier. I have some Python code (that was provided to me) that does this already, but I need to be able to have a solution in Java. I think my problem is now Python and Java differ in the way they store bytes.
Python (98% sure this works correctly):
def bin5(b,k):
""" Returns binary integer from bytes k,k+1,...,k+4 in b."""
b0 = b[k ]
b1 = b[k+1]
b2 = b[k+2]
b3 = b[k+3]
b4 = b[k+4]
if b0<0: b0 += 256
if b1<0: b1 += 256
if b2<0: b2 += 256
if b3<0: b3 += 256
if b4<0: b4 += 256
return b0*65536.0+b1*256.0+b2+b3/256.0+b4/65536.0
Java attempt:
// Returns binary integer from bytes k,k+1,...,k+4 in b.
private static int bin5(byte[] b, int k) {
byte b0 = b[k];
byte b1 = b[k + 1];
byte b2 = b[k + 2];
byte b3 = b[k + 3];
byte b4 = b[k + 4];
return (int)(b0 * 65536.0 + b1 * 256.0 + b2 + b3 / 256.0 + b4 / 65536.0);
}
I'm certain the problem is in the last return statement of the Java code. Also, it will work for some byte arrays, but not for others. I can't find a reason for this behavior.
EDIT: Example:
If the Python code reads the bytes: 0 11 -72 0 0 for b0 thru b5 respectfully, it will change the -72 to 184 and then calculate the value 3000.0 based on the equation above. Based on the survey/data parameters, this value is correct.
My intuition says that the python code IS faulty for some values. One such value is when it reads a values 0 -127 -66 0 0 (b0 thru b5 respectfully) which turns into: 0 129 190 0 0 and then the value 33214 is output by the conversion. This is impossible based on the survey/data parameters. BUT there is a possibility that this could be a faulty data point.
EDIT 2:
0 13 9 0 0 should return 3337 (and does in the python code). However under Java, it returns 3593.
You could do
private static double bin5(byte[] b, int k) {
int b0 = b[k] & 0xFF; // treat as unsigned byte
int b1 = b[k + 1] & 0xFF;
int b2 = b[k + 2] & 0xFF;
int b3 = b[k + 3] & 0xFF;
int b4 = b[k + 4] & 0xFF;
return (b0 * 65536 + b1 * 256 + b2 + b3 / 256.0 + b4 / 65536.0);
}
As powers of 2 can be represented exactly with double you won't get any rounding error.
This code is strange; it does not return an integer at all but a float...
Anyway, the Java equivalent of the python code is something like this (note: NO bounds checking is done at all):
private static double bin5(final byte[] b, final int k)
{
final ByteBuffer buf = ByteBuffer.allocate(8);
buf.position(3);
buf.put(b, k, 5);
buf.rewind();
final long l = buf.getLong();
return (double) l / 65536.0;
}
EDIT: if the last two elements of the byte array at offset k are always 0 (which it looks like they are) then you can replace from buf.rewind() onwards with:
buf.position(2);
return (double) buf.getInt();
Related
So I am saving an audio file. And i have to convert float[] to byte[]. This works fine:
final byte[] byteBuffer = new byte[buffer.length * 2];
int bufferIndex = 0;
for (int i = 0; i < byteBuffer.length; i++) {
final int x = (int) (buffer[bufferIndex++] * 32767.0);
byteBuffer[i] = (byte) x;
i++;
byteBuffer[i] = (byte) (x >>> 8);
if (bufferIndex < 5) {
System.out.println(buffer[bufferIndex]);
System.out.println(byteBuffer[i - 1]);
System.out.println(byteBuffer[i]);
}
}
But when i want to read the bytes and convert it back to floats just the first 4 numbers match with the old ones:
for (int i =0; i < length; i++) {
i++;
float val = (((audioB[i]) & 0xff) << 8) | ((audioB[i-1]) & 0xff);
val = (float) (val /32767.0);
if (bufferindex < 5) {
System.out.println(val);
System.out.println(audioB[i-1]);
System.out.println(audioB[i]);
}
bufferindex++;
}
The output :
0.07973075
0
0
0.149165
52
10
0.19944257
23
19
0.22437502
-121
25
---------
0.0
0
0
0.07971435
52
10
0.14914395
23
19
0.19943845
-121
25
0.22437209
Why ?
Rather than implementing your own bit shifting magic, why not use the java.nio.ByteBuffer class?
byte[] bytes = ByteBuffer.allocate(8).putFloat(1.0F).putFloat(2.0F).array();
ByteBuffer bb = ByteBuffer.wrap(bytes);
float f1 = bb.getFloat();
float f2 = bb.getFloat();
You are stuffing 4 byte float data into just 2 bytes per sample. This will obviously loose some precisition. Hint: Do the backwards calculation (2 byte -> float) in your first loop and compare the result with the original value, and look at intermediate values like x.
I don't understand what is this doCalculatePi means or does, in the following example:
public static double doCalculatePi(final int sliceNr) {
final int from = sliceNr * 10;
final int to = from + 10;
final int c = (to << 1) + 1;
double acc = 0;
for (int a = 4 - ((from & 1) << 3), b = (from << 1) + 1; b < c; a = -a, b += 2) {
acc += ((double) a) / b;
}
return acc;
}
public static void main(String args[]){
System.out.println(doCalculatePi(1));
System.out.println(doCalculatePi(2));
System.out.println(doCalculatePi(3));
System.out.println(doCalculatePi(4));
System.out.println(doCalculatePi(10));
System.out.println(doCalculatePi(100));
}
I have printed the values to understand what the results are but I still have no clue what this code calculates. The conditions inside the loop are not clear.
<< means left shift operation, which shifts the left-hand operand left by the number of bits specified by the right-hand operand (See oracle docs).
Say, you have a decimal value, 5 which binary representation is 101
Now for simplicity, consider,
byte a = (byte)0x05;
Hence, the bit representation of a will be,
a = 00000101 // 1 byte is 8 bit
Now if you left shift a by 2, then a will be
a << 2
a = 00010100 //shifted place filled with zero/s
So, you may now understand that, left shift a by 3 means
a << 3
a = 00101000
For better understanding you need to study Bitwise operation.
Note, you are using int instead of byte, and by default, the int data type is a 32-bit signed integer (reference here), so you have to consider,
int a = 5;
in binary
a << 3
a = 00000000 00000000 00000000 00101000 // total 32 bit
My guess is that it approximates PI with
PI = doCalculatePi(0)+doCalculatePi(1)+doCalculatePi(2)+...
Just a guess.
Trying this
double d = 0;
for(int k = 0; k<1000; k++) {
System.out.println(d += doCalculatePi(k));
}
gives me
3.0418396189294032
3.09162380666784
3.1082685666989476
[...]
3.1414924531892394
3.14149255348994
3.1414926535900394
<< is the Bitshift operator.
Basically, every number is represented as a series of binary digits (0's and 1's), and you're shifting each of those digits to the left by however many places you indicate. So for example, 15 is 00001111 and 15 << 1 is 00011110 (or 30), while 15 << 2 is (00111100) which is 60.
There's some special handling that comes into play when you get to the sign bit, but you should get the point.
I am working with Local Binary Patterns (LBP) which produce numbers in the range 0-255.
That means that they can fit in a byte (256 different values may be included into a byte). So that explains why many (if not all) implementation in java I have found uses byte[] to store these values.
The problem is that since I am interested in the rank of these values when converted to byte (from int for example) they do not keep the previous rank they had (as int for example) since byte are signed (as all but chars in java I think) and so the greater 128 values (127 and after) of the range 0-255 becomes negative numbers. Furthermore I think they are inverted in order (the negative ones).
Some examples to be more specific:
(int) 0 = (byte) 0
(int) 20 = (byte) 20
(int) 40 = (byte) 40
(int) 60 = (byte) 60
(int) 80 = (byte) 80
(int) 100 = (byte) 100
(int) 120 = (byte) 120
(int) 140 = (byte) -116
(int) 160 = (byte) -96
(int) 180 = (byte) -76
(int) 200 = (byte) -56
(int) 220 = (byte) -36
(int) 240 = (byte) -16
My question is whether there is a specific way to maintain the order of int values when converted to byte (meaning 240 > 60 should hold true in byte also -16 < 60!) while keeping memory needs minimum (meaning use only 8bits if that many are required). I know I could consider comparing the byte in a more complex way (for example every negative > positive and if both bytes are negative inverse the order) but I think it's not that satisfactory.
Is there any other way to convert to byte besides (byte) i?
You could subtract 128 from the value:
byte x = (byte) (value - 128);
That would be order-preserving, and reversible later by simply adding 128 again. Be careful to make sure you do add 128 later on though... It's as simple as:
int value = x + 128;
So for example, if you wanted to convert between an int[] and byte[] in a reversible way:
public byte[] toByteArray(int[] values) {
byte[] ret = new byte[values.length];
for (int i = 0; i < values.length; i++) {
ret[i] = (byte) (values[i] - 128);
}
return ret;
}
public int[] toIntArray(int[] values) {
int[] ret = new byte[values.length];
for (int i = 0; i < values.length; i++) {
ret[i] = values[i] + 128;
}
return ret;
}
If you wanted to keep the original values though, the byte comparison wouldn't need to be particularly complex:
int unsigned1 = byte1 & 0xff;
int unsigned2 = byte2 & 0xff;
// Now just compare unsigned1 and unsigned2...
In my current project I need to send over the network a parsed structure that contains some n-bit fields. for instance:
protocol version: 1 byte
messageId: 1 byte
creationTime: 6 bytes
traceId: 3 bits
reliability: 7 bits
Etc...
Thus, I created a simple POJO class to represent this for parsing & unparsing, but I have some doubts on what type to use for those fields, since this decision can make parsing & unparsing easy or a bit of a nightmare. I must say that the Message to be sent over the network has a very specific size constraint: it cannot surpass the sum of all fields.
I first tought in using bytes for everything and then have a Message.getBytes() method that will convert the message and for those fields that are less than a byte, use bitwise operations to discard the unnecessary bits.
Am I going in the right direction or there is another much simpler way to do this? I just feel I'm re-inventing the wheel here, this feels kinda boilerplate code...
Thanks!
EDIT: If anybody else stumbles here, I'll post how I solved this (thanks to a mate at work that helped me with these), so just keep reading:
Luckily enough, my protocol socket size rounds up to a fixed number of bytes (49) and those fields that are less than byte size, sum up a byte at then end, resulting in that I can combine both fields in a single byte prior parsing/unparsing.
That said, imagine I have two fields, say field1 and field2, first in 7 bits and the other just a bit. To combine those, I just do this trick:
byte resultingByte = short2Byte((short) ((field1 % 128) * 2 + (field2 ? 1 : 0)));
Note that both field1 and field2 are short type. I found this the most convenient way to work at the bit level. Thus, I mod first field, making sure I'm only getting 7 bits, move the bits to the left divining by 2 since only one bit is needed to be moved. Lastly I add the field 2 short, which can be 1 or 0. Then I have a short with the required values in the 8 Less Signigicant Bits.
I created commodity methods to convert from short2Byte, Long, and some others:
private byte [] to2Bytes(int in) {
ByteBuffer ret = ByteBuffer.allocate(2);
int val = in % 65536;
short s1 = (short) (val / 256);
short s0 = (short) (val % 256);
ret.put(short2Byte(s1));
ret.put(short2Byte(s0));
return ret.array();
}
private byte [] to4Bytes(long in) {
ByteBuffer ret = ByteBuffer.allocate(4);
long div = 4294967296L;
long val = in % div;
int rem = 0;
short s3 = (short) (val / 16777216L);
rem = (int) (val % 16777216L);
short s2 = (short) (rem / 65536);
rem = rem % 65536;
short s1 = (short) (rem / 256);
short s0 = (short) (rem % 256);
ret.put(short2Byte(s3));
ret.put(short2Byte(s2));
ret.put(short2Byte(s1));
ret.put(short2Byte(s0));
return ret.array();
}
private byte [] time2Bytes(Long time) {
ByteBuffer ret = ByteBuffer.allocate(6);
String hex = Long.toHexString(time).toUpperCase();
while (hex.length() < 12) {
hex = "0" + hex;
}
while (hex.length() > 12) {
hex = hex.substring(1);
}
try {
for (int i = 0; i < 6; i++) {
String strByte = "" + hex.charAt(i*2) + hex.charAt(i*2 + 1);
short b = Short.parseShort(strByte, 16);
if (b > 127) {
b -= 256;
}
ret.put((byte) b);
}
}
catch (NumberFormatException e) {
// Exception captured for correctness
e.printStackTrace();
}
return ret.array();
}
private long bytes2time(byte b5, byte b4, byte b3, byte b2, byte b1, byte b0) {
long l5, l4, l3, l2, l1, l0;
l5 = byte2short(b5) * 1099511627776L;
l4 = byte2short(b4) * 4294967296L;
l3 = byte2short(b3) * 16777216L;
l2 = byte2short(b2) * 65536L;
l1 = byte2short(b1) * 256L;
l0 = byte2short(b0) * 1L;
return l5 + l4 + l3 + l2 + l1 + l0;
}
private long bytes2long(byte b3, byte b2, byte b1, byte b0) {
long l3, l2, l1, l0;
l3 = byte2short(b3) * 16777216L;
l2 = byte2short(b2) * 65536L;
l1 = byte2short(b1) * 256L;
l0 = byte2short(b0) * 1L;
return l3 + l2 + l1 + l0;
}
private int bytes2int(byte b1, byte b0) {
return (int)byte2short(b1) * 256 + (int)byte2short(b0);
}
private short byte2short(byte b) {
if (b < 0) {
return (short) (b+256);
}
return (short)b;
}
private byte short2Byte(short s) {
if (s < 128) {
return (byte) s;
}
else {
return (byte) (s-256);
}
}
At the end I'm sending a byte array with 49 bytes. Unparsing is very similar process, obviously. There must be a proper way to do this, but well, it works...Hope this helps someone!
You can to use ByteBuffer and BitSet class to write and read messages, but it can still become a nightmare even worse then pure bit manipulation (and it will affect performance)
This question already has answers here:
Java: parse int value from a char
(9 answers)
Closed 2 years ago.
Given the following code:
char x = '5';
int a0 = x - '0'; // 0
int a1 = Integer.parseInt(x + ""); // 1
int a2 = Integer.parseInt(Character.toString(x)); // 2
int a3 = Character.digit(x, 10); // 3
int a4 = Character.getNumericValue(x); // 4
System.out.printf("%d %d %d %d %d", a0, a1, a2, a3, a4);
(version 4 credited to: casablanca)
What do you consider to be the "best-way" to convert a char into an int ? ("best-way" ~= idiomatic way)
We are not converting the actual numerical value of the char, but the value of the representation.
Eg.:
convert('1') -> 1
convert('2') -> 2
....
How about Character.getNumericValue?
I'd strongly prefer Character.digit.
The first method. It's the most lightweight and direct, and maps to what you might do in other (lower-level) languages. Of course, its error handling leaves something to be desired.
If speed is critical (rather than validation you can combine the result)
e.g.
char d0 = '0';
char d1 = '4';
char d2 = '2';
int value = d0 * 100 + d1 * 10 + d2 - '0' * 111;
Convert to Ascii then subtract 48.
(int) '7' would be 55
((int) '7') - 48 = 7
((int) '9') - 48 = 9
The best way to convert a character of a valid digit to an int value is below. If c is larger than 9 then c was not a digit. Nothing that I know of is faster than this. Any digits in ASCII code 0-9(48-57) ^ to '0'(48) will always yield 0-9. From 0 to 65535 only 48 to 57 yield 0 to 9 in their respective order.
int charValue = (charC ^ '0');