I'm writing file save and load functions for a specific file format that I have no control over and the format specifies that at a specific byte position I must write 4 bytes of data to signify a 32bit unsigned value... in my test file this value is 16052, or 0x00003EB4... so I write to the data to the byte array that will be saved in this manner:
data[index] = 0xB4;
data[index+1] = 0x3E;
data[index+2] = 0x00;
data[index+3] = 0x00;
You can see the data is in little-endian format, which is correct... the problem is when I try to load this data with my file load function java sees the data in as such:
-76, 62, 0, 0
the 0xB4 value is being interpreted as -76 because bytes are signed in java... when I try to recompose these 4 bytes into a single 32bit value using the following code the value ends up as -76...
value = data[index+3];
value <<= 8;
value |= data[index+2];
value <<= 8;
value |= data[index+1];
value <<= 8;
value |= data[index];
What this should do is the following:
set value to 0x00 (high order byte), shift left 8 bits, or 0x00 onto the lower 8 bits, shift left 8 bits, or 0x3E onto the lower 8 bits, shift left 8 bits, or 0xB4 (low order byte) onto the lower 8 bits.
This should produce the value 0x00003EB4... which is what I started with... however for some reason I cannot figure out it is giving me the value -76 after that operation.
I am convinced this is due to java's interpretation of the 0xB4 byte as the value -76 which is screwing up the bitwise OR operation...
My question is what must I do to get around this?
Thank you.
When you are loading the bytes, they are signed. When they are coerced to integers, they are sign extended. To solve this problem, you can do a bitwise AND with 0xFF to take only the 8 LSbs of the signed integer.
In your case value |= (data[index+i]) should become value |= (data[index+i] & 0xFF) (where i is replaced with the index offsets that you have).
Related
This question already has answers here:
Packing bytes into a long with |= is giving unexpected results
(1 answer)
Issues when converting bytes to integers (Java specific?)
(2 answers)
Closed 24 days ago.
I am trying to convert a byte array of 4 to an integer.
The code i am using is this:
byte[] bytes = new byte[4];
bytes[0] = (byte)0x95;
bytes[1] = (byte)0x19;
bytes[2] = (byte)0x07;
bytes[3] = (byte)0x00;
int number = bytes[0] + (bytes[1] << 8) + (bytes[2] << 16) + (bytes[3] << 24);
When running this exact code on my machine, the expected result would be 0x00071995, but somehow i get the result 0x00071895. How? Why?
java 19.0.1 2022-10-18
Java(TM) SE Runtime Environment (build 19.0.1+10-21)
A java byte is signed, and 0x95 represents a negative number with value -107 (since the leading bit is set). When it's promoted to a wider integer as part of the arithmetic operation, it ends up being sign-extended to 0xffffff95 (i.e. still -107), or 256 less than your desired value, thus causing the discrepancy in the next byte up.
Unsigned and signed addition should not give any differences in binary
There is no difference as long as you stay within the same width of integer type. The quantity 0x95 equivalent to the quantity -0x6B, modulo 256. However, the two are not equivalent modulo 4294967296. 0x95 simply does not overflow the signed range of a 32-bit int.
Another way of looking at this - signed and unsigned addition themselves do not give a difference in the binary representation. However, signed and unsigned promotion do cause a difference - a signed value is sign-extended (i.e. padded with 1s at the left when the sign bit was already 1) but an unsigned value is zero extended. A signed 8-bit 0x95 sign-extends to 0xffffff95, and an unsigned 8-bit 0x95 zero-extends to 0x00000095.
If I cast every byte[] element to long and i do a logical and with 0xFF i get the expected result
This ensures that you zero-extend instead of sign-extending; the undesired leading sign bits are forced to zero by the logical AND.
I am trying to take hex values and move them into a byte array. When I do this they are not converted how I need them to be. Most of them are converted to negative numbers. For example, after the execution of the C# code "baCommandPDI[0] = 0xD1;", the value of baCommandPDI[0] is "209". This is what I need. No matter what I try in Java, the value is "-47". I need it to be "209" in the byte array, because this byte array is sent out over TCP/IP to some external hardware. The external hardware cannot interpret -47 as 209. It must be 209. I have tried the following code:
int intTemp = 0xD1; After this line is executed the value of intTemp is 209, great!
int intHexValue = 0xD1 & 0xFF; After this line is executed the value of intHexValue is still 209, I don't understand how, but it ia what it is.
baCommand[0] = (byte) intHexValue; After this line is executed the value of baCommand[0] is -47, bad. This is not what I want. Everytime it converts the value to a byte it makes it signed.
Is there any other way I can do this? Can I use 11010001 and some how assign that value to a byte without making it negative?
In Java, bytes are always signed. But like an unsigned byte in another language, 8 bits are still used. The byte representations for 209 (unsigned) and -47 are the same.
1101 0001
The only difference is how Java interprets the most significant bit -- -128 vs. 128 in an unsigned type. The difference is 256, which is the difference between 209 and -47. You can send the value as is, because the bit values are the same, only the interpretations of this bit pattern are different.
If you would like to convert the value -47 to 209 to convince yourself in Java, you can promote the type to a longer primitive type and mask out the last 8 bits. This line will do this is one line, so you can debug and see that the bits are the same and that the unsigned version is equivalent.
int test = baCommandPDI[0] & 0xFF;
Printing/debugging test will let you see the value 209.
Java does not have a concept of unsigned bytes.
To get 209 from 0xD1, try:
0xD1 & 0xFF;
I need to generate a 3-bytes code (like A502F1). I am given a criteria:
1st byte is (serialCodeNumber / (256*256) ) & 0xFF
2nd is (serialCodeNumber / 256) & 0xFF
3th is (serialCodeNumber) & 0xFF
serialCodeNumber is a sequence 1-0xFFF
What does that mean!?
I would generate it like this:
String codeNum = new BigInteger(256, random).toString(16).toUpperCase().substring(0, 6);
But what is the right way of doing it as the requirement says?
I'm not quite sure what is meant by the serialCodeNumber, since if it is later on divided by 65025 it has to be a considerably larger number than 0xFFF (which is 4095) for it to make any reasonable sense.
But let's take a look at the conditions, they would all make sense once you are accustomed to the bitwise AND operator. A good read is available here on how it works but the meat of the matter from that question in my opinion is this sentence by Markus Jarderot:
The result is the bits that are turned on in both numbers.
Since in your conditions you have & 0xFF and 0xFF is 255, or in binary it's 11111111 the first eight bits that are all turned on. This is a neat trick to just retrieve only the first 8 bits of any number. And as we all know 8 bits make up a byte. (Are you starting to see where this all is coming together now?)
As for the conditions before the & 0xFF, some might recognize them as bit shift operations hidden behind divisions.
(serialCodeNumber / (256*256)) is equivalent to (serialCodeNumber >> 16)
and
(serialCodeNumber / 256) is equivalent to (serialCodeNumber >> 8)
But that is not that important in this case.
So the first condition takes the serialCodeNumber divides it by 65025 (256*256) and then looks at the 8 right most bits and ignores any other, from those 8 bits it constructs a byte.
In Java you can pretty much just write the condition as it is:
byte myFirstByte = (byte) ((serialCodeNumber / (256*256)) & 0xFF);
The other conditions aren't much different:
byte mySecondByte = (byte) ((serialCodeNumber / (256)) & 0xFF);
and
byte myThirdByte = (byte) ((serialCodeNumber) & 0xFF);
Once you have all three of your bytes, I'm assuming you need to convert them to a hex String. So I'll add them into a byte array.
byte[] myArray = {myFirstByte,mySecondByte,myThirdByte};
And borrow some method on how to convert byte arrays to HEX strings from this question.
String codeNum = bytesToHex(myArray);
And the result will look something like this:
F03DD7
EDIT:
Since you have to generate a serial number that has to be up to 6 bytes in value, I'd recommend using a long number.
A 6 byte number will be anywhere from 1 to 281474976710655, so you probably need to generate one randomly.
First instantiate a Random object which you will be able to poll numbers from:
Random random = new Random();
Once you have that, poll a long from it for the range 1 to 281474976710655.
For this you can borrow KennyTM's answer from this question.
So you can then generate the number like so:
long serialCodeNumber = nextLong(random, 281474976710655L)+1L;
We add the +1L at the end since we want it to include the last number as well as start from 1 instead of 0.
If you ever need to show a HEX string of the serialCodeNumber you can then just call:
String serialHex = Long.toHexString(serialCodeNumber);
But make sure to add any additional "0"s at the left side based on the length of the string so that it is 6-bytes = 12 characters long.
I am trying to understand how the following line of code works:
for (int i = 0; i < numSamples; i++) {
short ampValue = 0;
for (int byteNo = 0; byteNo < 2; byteNo++) {
ampValue |= (short) ((data[pointer++] & 0xFF) << (byteNo * 8));
}
amplitudes[i] = ampValue;
}
As far as I understand, this is reading 2 bytes (as 2 bytes per sample) in a inclusive manner, i.e. the ampValue is composed of two byte reads. The data is the actual data sample (file) and the pointer is increasing to read it upto the last sample. But I don't understand this part:
"data[pointer++] & 0xFF) << (byteNo * 8)); "
Also, I am wondering whether it makes any difference if I want to read this as a double instead of short?
Looks like data[] is the array of bytes.
data[pointer++] gives you a byte value in the range [-128..127].
0xFF is an int contstant, so...
data[pointer++] & 0xFF promotes the byte value to an int value in the range [-128..127]. Then the & operator zeroes out all of the bits that are not set in 0xFF (i.e., it zeroes out the 24 upper bits, leaving only the low 8 bits.
The value of that expression now will be in the range [0..255].
The << operator shifts the result to the left by (byteNo * 8) bits. That's the same as saying, it multiplies the value by 2 raised to the power of (byteNo * 8). When byteNo==0, it will multiply by 2 to the power 0 (i.e., it will multiply by 1). When byteNo==1, it will multiply by 2 to the power 8 (i.e., it will multiply by 256).
This loop is creating an int in the range [0..65535] (16 bits) from each pair of bytes in the array, taking the first member of each pair as the low-order byte and the second member as the high-order byte.
It won't work to declare ampValue as double, because the |= operator will not work on a double, but you can declare the amplitudes[] array to be an array of double, and the assignment amplitudes[i] = ampValue will implicitly promote the value to a double value in the range [0.0..65535.0].
Additional info: Don't overlook #KevinKrumwiede's comment about a bug in the example.
In Java, all bytes are signed. The expression (data[pointer++] & 0xFF) converts the signed byte value to an int with the value of the byte if it were unsigned. Then the expression << (byteNo * 8) left-shifts the resulting value by zero or eight bits depending on the value of byteNo. The value of the whole expression is assigned with bitwise or to ampValue.
There appears to be a bug in this code. The value of ampValue is not reset to zero between iterations. And amplitude is not used. Are those identifiers supposed to be the same?
Let's break down the statement:
|= is the bitwise or and assignment operator. a |= b is equivalent to a = a | b.
(short) casts the int element from the data array to a short.
pointer++ is a post-increment operation. The value of pointer will be returned and used and then immediately incremented every single time it's accessed in this fashion - this is beneficial in this case because the outer-loop is cycling through 2-byte samples (via the inner loop) from the contiguous data buffer, so this keeps incrementing.
& is the bitwise AND operator and 0xFF is the hexadecimal value for the byte 0b11111111 (255 in decimal); the expression data[pointer++] & 0xFF is basically saying, for each bit in the byte retrieved from the data array, AND it with 1. In this context, it forces Java, which by default stores signed byte objects (i.e. values from -128 to 127 in decimal), to return the value as an unsigned byte (i.e. values from 0 to 255 decimal).
Since your samples are 2 bytes long, you need to shift the second lot of 8 bits left, as the most significant bits, using the left bit-shift operator <<. The byteNo * 8 ensures that you're only shifting bits when it's the second of the two bytes.
After the two bytes have been read, ampValue will now contain the value of the sample as a short.
I have a byte array in which a value is stored as a 16bit unsigned integer. This is spread across two positions in my byte array, DataArray[11] and DataArray[12]. The documentation I have for the packet which contains the byte array tells me that the value I need to extract is packed least significant bit first. I'm having trouble wrapping my head around bitmasks and bit shifting, and I'm actually unclear if I need to use one or the other, or both.
This is what I have so far, but the results don't seem right:
int result = (DataArray[11] << 8 | DataArray[12]) & 0xFF;
You're trying to get a 16-bit integer, right? But you're masking it using & 0xff - which limits you to 8 bits. I suggest you mask each byte rather than the result:
int result = (DataArray[11] & 0xff) |
((DataArray[12] & 0xff) << 8);
I've included more parentheses here than are probably required, just for the sake of sanity and not needing to worry about precedence.
I've also swapped the ordering so that you're shifting DataArray[12] rather than DataArray[11], as it's meant to be least-significant byte first.