I have been reading this blog about bit manipulation in networking packages. I have this code and I am seeing that its reading in an integer which is typically 4 bytes long (32 bits)
public int readInt() {
return (pData[caret++] & 0xff) << 24 | (pData[caret++] & 0xff) << 16
| (pData[caret++] & 0xff) << 8 | pData[caret++] & 0xff;
}
Why are we moving by 8, 16 and 24 when reading in an integer value? What is the goal and what does it achieve?
That is because pData is a byte array. What readInt() does is pack 4 bytes into an integer then return the result. Because an integer is 4 bytes, by ((value & 0xFF) << 24) will set for the 1'st slot, ((value & 0xFF) << 16) for the 2'nd slot, ((value & 0xFF) << 8) for the 3'rd slot and (value & 0xFF) for the last slot. And to merge these four values into one (that in this case is an integer), bitwise OR ( |) must be performed between each value. If pData is indeed an integer array, thats probably because it wants to use unsigned integers (0 - 255). So what readInt() does is pack 8 LSB (least significant bits) for each int (up to 4 integers) into an integer then return the result.
Related
I had a look at the Twofish source code in Java and discovered this line, which I don't quite understand.
public static byte[] blockEncrypt (byte[] in, int inOffset, Object sessionKey) {
int x0 = (in[inOffset++] & 0xFF) | (in[inOffset++] & 0xFF) << 8 | (in[inOffset++] & 0xFF) << 16 | (in[inOffset++] & 0xFF) << 24;
}
Here, a byte is offset with a logical AND with 0xFF (11111111). Why is this done? If I take an 8-bit value and calculate it & (logical AND) 0xFF, the value I had previously is returned in the end, because only the bits that were 1 in the original value are taken over. (11010011 & 11111111 should result in 11010011)
My goal is to convert 2 bytes to a Char.
The input will be from a byte[] then converted to char. My initial strategy was to use bitwise operations.
My expected result: 0xFF << 8 | 0xFF equals to 0xFFFF. It does. But, in practice it does not because before the bitwise operations, I need to store the data in byte[].
My actual result: Due to the way I access the data. (byte) 0xFF << 8 | (byte) 0xFF will equal to -1.
byte[] bytes = new byte[]{(byte) 0xFF, (byte) 0xFF};
int integer = bytes[0] << 8 | bytes[1]; // -1 or 0xFFFFFFFF
You need:
int integer = (bytes[0] & 0xff) << 8 | (bytes[1] & 0xff);
Which gives the expected value of 65535.
I'm a bit confused regarding a conversion from bytes to integers. Consider the following code:
byte[] data = new byte[] { 0, (byte) 0xF0 };
int masked = data[0] << 8 & 0xFF | data[1] & 0xFF; //240
int notMasked = data[0] << 8 | data[1]; //-16
Because bytes in java are signed, data[1] is not 240 decimal, but rather the 2's complement, -16. However, it should still be, in binary: 0x11110000 so, why do I need to do data[1] & 0xFF ?
Is Java converting everything to Integer before passing it to the | operator? Why does &0xFF make a difference then?
Java bytes are signed (unfortunately) - so when you promote the value to an int in order to perform the bitwise |, it ends up being sign-extended as 0xFFFFFFF0. That then messes up the | with data[0]. The masking with & 0xff converts it to an integer value of 240 (just 0x000000F0) instead.
However, you've stlil got a problem. This code:
int masked = data[0] << 8 & 0xFF | data[1] & 0xFF;
should be:
int masked = ((data[0] & 0xff) << 8) | (data[1] & 0xFF);
... otherwise you're masking after the shift, which won't work. I've added brackets because I'm never sure of the predence of &, << and |...
It is similar to a known "puzzle"
byte x = -1;
x = x >>>= 1;
System.out.println(x);
produces
-1
No shift? This is because before compiling arithemtic / shift / comparison expressions javac promotes byte (as well as short and char) to int or to long (if there is any long in the expression), so it works as follows
x -> int = 0xFFFFFFFF; 0xFFFFFFF >>> 1 = 0x7FFFFFF; (byte)0x7FFFFFF -> 0xFF
Ok, so I have searched and searched and nothing worked...
I have this array of int, each int occupies only the low order byte. For instance, I have
data[0] = Ox52
data[1] = Oxe4
data[2] = Ox18
data[3] = Oxcb
I want that the standard output contains exactly those bytes (or in other words, if I write this in a file and I examine the file with a Hex editor, I should see):
52e418cb
How can I do that?
Thank you for your help
The correct way of doing this is to shift the bytes according to their desired position and then stitch them together using the OR operator. But, you should also perform a bit mask on the lower 8 bits of the byte before shifting it. This is needed because a byte is first converted to an int (before the shifting is done). This is no big deal, but when the highest bit is 1 (i.e.: the byte is negative), your integer will become negative as well, which causes all the leading bits to be set on 1.
So:
(byte) 10000000 = (int) 11111111 11111111 11111111 10000000
Using this negative int value with the OR operator will cause a wrong result. So, the working line is this one:
((data[0] & 0xFF) << 24) | ((data[1] & 0xFF) << 16) | ((data[2] & 0xFF) << 8) | (data[3] & 0xFF)
The following seems to work fine. I'm using the OutputStream.write(int) method.
int[] ints = new int[] { 0x52, 0xe4, 0x18, 0xcb };
FileOutputStream os = new FileOutputStream(new File("/tmp/x"));
for (int i : ints) {
os.write(i);
}
os.close();
Results:
> hexdump /tmp/x
0000000 52 e4 18 cb
Just shift and OR them together before writing them to the file/output:
(data[0] << 24) | (data[1] << 16) | (data[2] << 8) | data[3]
I'm trying to convert a short into 2 bytes...and then from those 2 bytes try to get the same short value. For that, I've written this code:
short oldshort = 700;
byte 333= (byte) (oldshort);
byte byte2= (byte) ((oldshort >> 8) & 0xff);
short newshort = (short) ((byte2 << 8) + byte1);
System.out.println(oldshort);
System.out.println(newshort);
For the value of 700 (oldshort), newhosrt is 444. After some testing, it looksl ike \tThis code only works for some values. Like...if oldshort=50, then it will work fine..but if it is -200, or bigger values than 127 (i think) it doesn't work. I guess that there is a problem with the signed bytes, two's complement value, etc...but I can't figure out how to solve it.
Any idea?? Any native way to do this in java?? Thanks in advance!
When recombining, you need to mask the byte1 to stop it being sign extended.
E.g.
short oldshort = 700;
byte byte1= (byte) (oldshort);
byte byte2= (byte) ((oldshort >> 8) & 0xff);
short newshort = (short) ((byte2 << 8) + (byte1&0xFF);
System.out.println(oldshort);
System.out.println(newshort);
EDIT:
All operations on bytes and shorts in java are actually done as integers. So when you write
+byte1, what is really happening is that the byte is first cast to an integer (sign-extended). It will still have the same value, but now has more bits. We can then mask off the bottom 8 bits to get the original 8-bits from the short - without the sign.
E.g. short =511 = 0x01FE
// lots of 0x000's because the operations are done on 32-bit int's
byte1 = (0x000001FE & 0x000000FF) = (0x01FE & 0xFF) = 0xFE = (byte)-2
byte2 = 0x1
newShort = (byte2 << 8) + (byte1 & 0xFF)
= (0x1 << 8) + (0xFE & 0xFF)
// since the ops are performed as int's
= (0x00000001 << 8) + (0xFFFFFFFE & 0x000000FF)
// 0xFFFFFFFE = -2
= (0x00000100) + (0x000000FE)
= 0x000001FE
= 511
You could also use com.google.common.primitives.Shorts, which has methods:
public static byte[] toByteArray(short value)
public static short fromByteArray(byte[] bytes)