I was wondering the meaning of "Bitwise Shifting". Currently I am trying to create Image Filters in Java using BufferedImage. I found some good answers on how to access the RGB values individually in other posts but wasnt exactly sure on how it works. I understand the RGB values are stored in a 32-bit int each 8-bit part representing the value.
This is what I was looking at: a link
Basically looking for someone to explain this process, the Google searches Ive done were too technical.
When you have a single integer that contains multiple bytes of information, then masking and shifting are the processes used to access the individual pieces. Assuming the bytes are stored like such (they probably aren't, but...) then this is what you could do to retrieve them:
aRGB: 255, 65, 33, 17
binary: 11111111 01000001 00100001 00010001
To retrieve the red value (65) from a variable x:
x && 0x00FF0000
11111111 01000001 00100001 00010001
* 00000000 11111111 00000000 00000000
-------------------------------------
00000000 01000001 00000000 00000000
Then the shifting operation, to move the bits to where they make sense as a lone value:
00000000 01000001 00000000 00000000 >> 16 = 00000000 00000000 00000000 01000001
The binary mask captures only the second byte of the value, by setting all the other bits to 0; only the bits that match the second byte remain as they are (multiplied by 1, not 0). Then, you shift the bits to the right 16 places, to strip off those extra 0's. Obviously, the leading 0's no longer matter, so the result is just a plain binary 01000001, or decimal 65.
It is technical in nature. You store a value 0 - 255, or in binary:
from
00000000
to
11111111
then you take a larger container like this one:
0000000000000000
and insert it there so if your color is 117
0000000001110101
and shift it to the left one byte so you get
0111010100000000
then you add the second color, say it 65 or 01000001 and get
0111010101000001
then you again shift it one byte to the left and get
011101010100000100000000
at last you add the third color and its for example a 255 or 11111111 you get
011101010100000111111111
so you have stored three values 0 - 255 via bitshift.
There is more than one way to think about a pattern of bits in memory. Usually, we think of the bits stored in an integer as representing a single number. The higher-order bits contribute greater value than the lower-order bits and their values are summed to arrive at a single number, so the following pattern:
00001101 00110000 00000111
Would be interpreted as:
2^0 + 2^1 + 2^3 + 2^13 + 2^14 + 2^15 + 2^17 + 2^18 = 864,263
But we're free to think of this pattern as three individual groups of 8-bit numbers (each representing the numeric value of a color component). However, to convert a bit pattern in the higher-order bits into the number we intend it to represent, we need to interpret those bits as though they appeared in the rightmost 8-bit group. This is why we shift.
For example, to get the value of the leftmost 8-bit group, we need to mask all bits not in that group, and then shift those bits 16 places to the right, so that they occupy the rightmost position:
// first, assume n = 00001101 00110000 00000111
(n & 0xFF0000) >> 16 // red
Zeros are automatically shifted into the vacated bits, leaving:
00000000 00000000 00001101
Which we can now interpret as:
13
Similarly, we can calculate the value of the bits in the center and rightmost groups:
(n & 0x00FF00) >> 8 // green
n & 0x0000FF // blue (no shift necessary)
Setting the value of one of these components requires shifting in the other direction. For example, the following shifts 75 into the center 8-bit position:
n = (n & (0xFF00FF)) | (75 << 8)
We're first resetting the green value (with the 0xFF00FF mask) and then shifting the number 75 8 bits to the left. Finally, we combine these two numbers.
We can see that this works by shifting it back out again:
(n & 0x00FF00) >> 8 // => 75
Resetting or maximizing one of the components is even simpler. As we've seen in the earlier example, zero-ing a component can be done like this:
n = n & 0xFF00FF // clear out green
The following mask, on the other hand, maximizes the value of the green component:
n = n | 0x00FF00
Bitwise operations are operations on the bits of the binary representation of the data rather than the data represented as an integer or floating point number. For example, 8 may be represented as 1000 in binary. Shifting it to the right twice moves all the digits to the right by 3, padding with 0's, resulting in 0010, or 2.
For more, see the Wikipedia article.
Related
This question already has answers here:
How are integers cast to bytes in Java?
(8 answers)
Type casting into byte in Java
(6 answers)
Explicit conversion from int to byte in Java
(4 answers)
Closed 4 months ago.
For example of this is my input:
byte x=(byte) 200;
This will be the output:
-56
if this is my input:
short x=(short) 250000;
This will be the output:
-12144
I realize that the output is off because the number does not fit into the datatype, but how can I predict what this output will be in this case? In my computer science exam this my be one of the questions and I do not understand why exactly 200 changes to -56 and so one.
I realize that the output is off because the number does not fit into the datatype, but how can I predict what this output will be in this case? In my computer science exam this my be one of the questions and I do not understand why exactly 200 changes to -56 and so one.
The relevant aspects are what overflow looks like, and how the bits that represent the underlying data are treated.
Computers are all bits, grouped together in groups of 8; a group of 8 bits is called a byte.
byte b = 5; for example, is stored in memory as 0000 0101.
Bits can be 0. Or 1. That's it. That's where it ends. And everything is, in the end, bits. This means: That - is not a thing. Computers do not know what - is and cannot store it. We need to write code and agree on some sort of meaning to represent them.
2's complement
So what's -5 in bits? It's 1111 1011. Which seems bizarre. But it's how it works. If you write: byte b = -5;, then b will contain 1111 1011. It is because javac made that happen. Similarly, if you then call System.out.println(b), then the println method gets the bit sequence 1111 1011. Why does the println method decide to print a - symbol and then a 5 symbol? Because it's programmed that way: We all are in agreement that 1111 1011 is -5. So why is that?
Because of a really cool property - signed/unsigned irrelevancy.
The rule is 2's complement: To switch the sign (i.e. turn 5, which is 0000 0101 into -5 which is 1111 1011), you flip every bit, and then add 1 to the end result. Try it with 0000 0101 - and you'll see it's 1111 1011. This algorithm is reversible - apply the same algorithm (flip every bit, then add 1) and you can turn -5 into 5.
This 2's complement thing has 2 great advantages:
There is only one 0 value. If we just flipped all bits, we'd have both 1111 1111 and 0000 0000 both representing some form of 0. In basic math, there's no such thing as 'negative 0' - it's the same as positive 0. Similarly if we just decided the first bit is the sign and the remaining 7 bits are the number, then we'd have both 1000 0000 and 0000 0000 both being 0, which is annoying and inefficient, why waste 2 different bit sequences on the same number?
plus and minus are sign-mode independent. The computer doesn't have to KNOW whether we are doing the 2's complement thing or not. Take the bit sequence 1111 1011. If we treat that as unsigned bits, then that is 251 (it's 128 + 64 + 32 + 16 + 8 + 2 + 1). If we treat that as a signed number, then the first bit is 1, so the thing is negative: We apply 2's complement and figure out that it is -5. So, is it -5 or 251? It's both, at once! Depends on the human/code that interprets this bit sequence which one it is. So how could the computer possibly do a + b given this? The weird answer is: It doesn't matter - because the math works out the same way. 251 - 10 is 241. -5 - 10 is -15. -15 and 241 are the exact same bit sequence.
Overflow
A byte is 8 bits, and there are 256 different sequences of bits, and then you have listed each and every possible variant. (2^8 = 256. Hence, a 16-bit number can be used to convey 65536 different things, because 2^16 is 65536, and so on). So, given that bytes are 8 bits and we decreed they are signed, and 2's complement signed, that means that the smallest number you can send with it is -128, which in bits is 1000 0000 (use 2's complement to check my work), and +127, which in bits is 0111 1111. So what happens if you add 1 to 127? That'd seemingly be +128 except that's not storable in 8 bits if we decree that we interpret these bits as 2's complement signed (which java does). What happens? The bits 'roll over'. We just add 1 as normal, which turns 0111 1111 into 1000 0000 which is -128:
byte b = 127;
b = (byte)(b + 1);
System.out.println(b); // prints -128
Imagine the number line - stretching out into infinity on both ends, from -infinite to +infinite. That's the usual way math works. Computers (or rather, int, long, etc) do not work like that. Instead of a line, it is a circle. Take your infinite number line and take some scissors, and snip that number line at -128 (because a 2's comp signed byte cannot represent -129 or anything else below -128), and at +127 (because our byte cannot represent 128 or anything above it).
And now tape the 2 cut ends together.
That's the number line. What's 'to the right' of 125? 126 - that's what +1 means: Move one to the right on the number line.
What's 'to the right' of +127? Why, -128. Because we taped it together.
Similarly, -127 - 5 is +123. '-5' is 'move 5 places to the left on the number line (or rather, number circle)'. Going in 1 decrements:
-127 (we start here)
-128 (-127 -1)
+127 (-127 -2)
+126 (-127 -3)
+125 (-127 -4)
+124 (-127 -5)
Hence, 124.
Same math applies to short (-32768 to +32767), char (which is really a 16-bit unsigned number - so 0 to 65535), int (-2147483648 to +2147483647), and even long (-2^63 to +2^63-1 - those get a little large).
short x = 32765;
x += 5;
System.out.println(x); // prints -32766.
I have tried to convert 248 to binary and then convert it to two's compliment and finally convert it back to decimal.
Problem: result is 8 not -8
Convert the hex number digit by digit to a 4-bit binary, for example (A) to (1010) and put then together. Now you can calculate the 2‘s complement as explained below.
You'll see you need at least 9 bits for (F8)16, so if you're only working with a 8-bit variable for instance, you might encounter issues.
Calculating the 2‘s complement:
(248)10 = (0 1111 1000)2.
Complement all the bits:
(1 0000 0111)2.
Add 1:
(1 0000 1000)2.
This will give you (-248)10. Notice that if you're only looking at the last 8 bits (0000 1000)2, the bit string evaluates to 8. I hope this helps in some way.
Java specifically keeps a single byte in the min/max range of -128 to 127.
A value like 248 (hex: 0xF8) is already stored as a Short (2 bytes or 16 bits) in Java.
[0000 0000] [1111 1000] = 248 in Java. But you need only [1111 1000] for -8.
You'd think "If I remove starting byte [0000 0000] then it works..??" No because now you have 1 byte left and that remaining byte with [1111 1000] alone cannot make 248 since Java's maximum for a single byte is 127.
Solution:
In those available 16 bits, replace all bits leading up to your value with 1.
Visually your bits [0000 0000] [1111 1000] = 248 must become as [1111 1111] [1111 1000] = -8.
One quick way to achieve the result is through this logic result = (-256) | ( x );.
Code example:
int x = 248; //# your input value from reading
int result = (-256) | ( x );
System.out.println("value is " + result ); //# gives: value is -8
I'm trying to use an API where there's a socket is used to communicate. A request is made up of different parts and one of them is the header which is stated as so:
Fixed header: 2 bytes, fixed at 0xffff
Generally I'm not good with bytes and streams, since I've never used it. So how should i create said byte array? I've tried the following
byte[] header = new byte[]{(byte)0xff, (byte)0xff};
But they bytes each become -1, which I believe is because 0xFF translates to 255 which is outside of the signed byte range (-128 to +127), but then how do I create a header like that?
You just did it.
In the end, computers just know about bits. The rest is what the code, and the humans looking at it, make of it. A bit is a 0 or a 1. If you bought a computer with 4GB RAM, then your computer can remember 34359738368 of those.
That's a bit unwieldy, so AMD, or intel, or TSMC, or whomever baked your chip, baked into the chip's design that the chip groups them in sets of 8 (and for certain jobs, in sets of 64 or even higher). But that's where it ends. It's just bits, really. Negative number? What's that? 2? What is this 2 you speak of. I know only 0 and 1.
So that's unwieldy too, so we humans don't wanna say: This byte holds value 00000101. We'll just say 'that holds 5'.
bits = decimal
00000000 = 0
00000001 = 1
00000010 = 2
00000011 = 3
00000100 = 4
00000101 = 5
... and so on
That's great, but what about -1? We just have 0 and 1. There's no - so how do we do this?
That's where it gets interesting. It's a convention, not something in the computer. There's this thing called two's complement: We all agree to check the first bit. If it is a 1, then we shall call this -X, where X is found by applying the following algorithm: Flip every bit (all zeroes become one, all ones become zeroes), and add 1 to it.
11111011 = -5.
Why? Well, flip every bit: 00000100
then add 1 to it : 00000101
which is 5.
But that immediately eats half of what we can represent. After all, the biggest number we can now store in a byte is 127: 01111111, which is 127. If we add 1 to this number, then we get to 10000000, but hey that starts with a 1 bit, so assuming we are all in agreement that this means it is negative, that means 1000000 is -128 (bit of an exotic case).
And sometimes that's annoying or not worth it. So sometimes we all agree that the number cannot be negative at all, and 1000000 is just 128. and 11111111 is just 255.
The computer has no idea. 255 is 11111111 and so is -1. So what's 11111111? The computer doesn't know. It doesn't even know what 2 is. It just knows zeroes and ones, and as far as the computer is concerned, 11111111 is what it is. (the math works out that + and - 'just work' regardless of whether we decree these numbers are to be seen as two's complement signed or not, cool, huh? Try it! If 11111011 is both -5 as well as 251 depending on the opinion of the one reading off the number, what happens? -5 + 2 is -3. 251 + 2 is 253. -3 and 253 boil down to the same sequence of bits. Just an example. This is, incidentally, why we do the weirdo 'flip all bits and add 1' stuff. So that + and - just work and you don't need to pass along whether you consider the bits 'signed' or 'unsigned'.
In java, all numeric types except char (which is a numeric type. You'd think it represents a character, but it really doesn't) are signed. byte is 'signed 8-bit number' (so, can represent from -128 to +127, inclusive). char is the only exception, that is an 'unsigned 16-bit number', so can hold from 0 to 65535, inclusive. It's just if you e.g. call System.out.println((char) 65);, the println method will interpret that number as: "Look this up in the unicode table and print whatever you find there", so that prints 'A'. That's part of the source code of that particular println method, it's nothing inherent about the char type in java, which is just 'a number between 0 and 65535'.
So, when you print your byte array containing 0xFF, 0xFF in java, because java agreed that we consider it signed, it prints -1, -1. But that's just java-ese for 0xFF, 0xFF. Your byte array contains 0xFF, 0xFF because at the bit level -1 and 255 are the exact same number. For bytes anyway. Not so for all the other ones (char, short, int, long).
To recap:
byte x = (byte) 200;
byte x = (byte) 0xC8;
byte x = -56;
In all these cases, x ends up holding the bits 11001000. There is no way to tell the difference. You can't ask the system: So, uh, is this x equal to 200, or 0xC8, or -56? What was used to set it? Because the computer does not know - the compiler translates all of the above code to the exact same end result, which is 11001000.
255 is -1.
Well, to start you must know that in Java all integer types are signed. This means that the most significant bit is reserved to represent the sign. That is why in Java the constant Byte.MAX_VALUE says it can go up to 127.
Now, this means you can store 8 bits in a byte, but if you happen to turn on the sign bit, whatever you store would be represented by Java as negative number.
Since 0xff turns on all the byte bits (i.e. 11111111) instead of getting 255 as you were expecting, what you're getting is -1, because that number represents -1 in Java.
Perhaps to understand it I can show you how the bits work in Java. Imagine a type called nimble of only 4 bits, where the most significant bit is reserved for sign.
This is how it would look in Java if it existed:
Imaginary Signed Type: Nimble (4 bits)
Dec. Bin. Hex.
--------------------
+0 0000 0x0
+1 0001 0x1
+2 0010 0x2
+3 0011 0x3
+4 0100 0x4
+5 0101 0x5
+6 0110 0x6
+7 0111 0x7
-8 1000 0x8
-7 1001 0x9
-6 1010 0xA
-5 1011 0xB
-4 1100 0xC
-3 1101 0xD
-2 1110 0xE
-1 1111 0xF
Notice how those numbers where the most significant bit is on become negative numbers. If this nimble was a unsigned type, then it wouldn't have negative numbers and it could reach 15.
That's why Java bytes go from -128 to 127, instead of up to 255 as you were expecting.
Now, when it comes to creating byte arrays to send to a stream, perhaps instead of creating the byte array yourself, you could wrap your socket output stream to a type-aware stream like a DataOuputStream, which allows you to send data of specific type.
For example:
try(DataOutputStream out = new DataOutpuStream(socket.getOutputStream())) {
dOut.writeByte((byte)0xff);
dOut.writeByte((byte)0xff);
}
That way you may avoid all the difficulties of having to create a header array.
But bottom line, you are array if fine.
Referring to page number 79 of " Java The complete Reference" 7th edition by Herbert Schildt.
The author says : " If the integer’s value is larger than the range of a
byte, it will be reduced modulo (the remainder of an integer division by the) byte’s range".
The range of byte in java is -128 to 127. So the maximum value that fits in a byte is 128. If an integer value is assigned to a byte as shown below :
int i = 257;
byte b;
b = (byte) i;
Since 257 crosses the range 127, 257 % 127 = 3 should be stored in 'b'.
But am getting the output as 1 instead of 3.
Where have I gone wrong in understanding the concept?
Just consider the binary representation of the numbers :
257 is represented in binary as 00000000 00000000 00000001 00000001
When you cast this 32 bits int to an 8 bits byte, you keep only the lowest 8 bits :
00000001
which is 1
257 = 00000000 000000000 00000001 00000001 in bits and a byte is made up of 8 bits only...
As a result only the lower 8 bits are stored and 1 is the output.
Here is the code:
int i = 200;
byte b = (byte) 200;
System.out.println(b);
System.out.println((short) (b));
System.out.println((b & 0xff));
System.out.println((short) (b & 0xff));
Here is the output:
-56
-56
200
200
Bitwise AND with 0xff shouldn't have changed anything in b, but apparently it does have an effect, why?
It has an effect because 200 is beyond the maximum possible (signed) byte, 127. The value is already assigned -56 because of this overflow. The most significant byte, worth -128, is set.
11001000
The first 2 output statements show -56 because of this, and the fact that casting to a short will perform sign extension to preserve negative values.
When you perform & 0xff, 2 things happen. First, the value is promoted to an int, with sign extension.
11111111 11111111 11111111 11001000
Then, the bit-and is performed, keeping only the last 8 bits. Here, the 8th bit is no longer -128, but 128, so 200 is restored.
00000000 00000000 00000000 11001000
This occurs whether the value is casted to a short or not; a short has 16 bits and can easily represent 200.
00000000 11001000
Java byte is a signed type. That is why you see a negative number when you print it: 200, or 0xC8, is above the largest positive number representable by byte, so it gets interpreted as a negative byte.
However, 0xff constant is an int. When you perform arithmetic and bitwise logic operations on a byte and an int*, the result becomes an int. That is why you see 200 printed in the second set of examples: (b & 0xff) produces an integer 200, which remains 200 after shrinking conversion to short, because 200 fits into a short without becoming negative.
* or another byte, for that matter; Java standard specifies a list of conversions that get applied depending on operand types.
working with different integer types is a mine field.
for example, what's going on here?
byte b = (byte) 200;
it's actually equivalent to
int i = 200;
byte b = (byte)i;
and the narrowing cast (byte) simply takes the lowest 8 bits of the int value.