Split IP octet bytes - java

I had to split IP address bytes and preserve them in a tree. Here what I've implemented, but I've the feeling that there is a faster way to achieve the same goal. So if you have any ideas, please let me know :)
private static final int MAX_IP_BITS_SIZE = 16 * Byte.SIZE;
/**
* Splits octet bytes in new byte array based on the splitter size.
*
* #param bytes address representation of IPv4 or IPv6 to be split.
* #param splitSize is the amount of bit to split on. It could be from 1 to 7, otherwise returns unsplit bytes.
* #return byte[] split version of the address bytes.
*/
private byte[] splitAddressBytes(byte[] bytes, final int splitSize)
{
if (splitSize < 1 || splitSize > 7)
{
return bytes;
}
int byteIndex = 0;
int changeByteIndex = 0;
byte[] octetBytes = new byte[MAX_IP_BITS_SIZE];
for (int i = 0; i < Byte.SIZE * bytes.length; i++)
{
/*
* Get byte index based on the splitter size, e.g. if the splitter size is 3, which mean that 'i % Byte.SIZE / splitSize' will
* vary from 0 to 2, i.e. on each change between those value means we can move to the next index.
*/
int inputBitIndex = i % Byte.SIZE;
if (changeByteIndex != inputBitIndex / splitSize)
{
changeByteIndex = inputBitIndex / splitSize;
byteIndex++;
}
/*
* Traverse all bits one by one and check if it is set to 1, if so add it to the result byte. Get the bit at the position
* "i / Byte.SIZE", then shift each bit at the first position and check if it is 1 or 0, i.e. i % Byte.SIZE is the position of
* the bit that we want to check (0,1,2,3,4,5,6,7), e.g. 128 is 1000 0000 and i % Byte.SIZE when i=0 is 0, as and i % Byte.SIZE
* is 0, then do not shift, just AND 1000 0000 and 0x80 (which is 128 or 1000 0000). In that case the result is 128, which is
* not 0 so the bit at that position is 1. Second attempt i=1, so i / Byte.SIZE = 0 and i % Byte.SIZE = 1. In that case we want
* to check second bit which is 1000 0000 << 1 = 0000 0000, from where 0000 0000 & 1000 0000 = 0, so the bit at that position is
* 0.
*/
if ((bytes[i / Byte.SIZE] << inputBitIndex & 0x80) != 0)
{
/*
* Now when we know that bit at that position (i) is set to 1 we have to place it to the particular byte at the desired
* position based on the splitter size, i.e. 'octetBytes[byteIndex] & 0xFF' - get current value of the bit based on the
* index (which also take into account split size) and apply bitwise OR with the bit that is set to 1 based on the bit
* shift. So to determine the index position of the bit: '(Byte.SIZE - splitSize)' - get offset or the starting point from
* the byte based on the splitter size. 'inputBitIndex % splitSize' - get which bit has to be set to 1, i.e. in case of
* splitter set to 3 the block size is 3 and that will result in 0,1, or 2. '(Byte.SIZE - splitSize) + (inputBitIndex %
* splitSize))' - combining offset and bit position we have the final bit position where we have to place the bit with value
* 1. So with bit shift of 0x80 at that position, e.g. 1000 0000 >>> 5 = 0000 0100 we have the desired byte to add to
* already existing. e.g. current byte is 0000 0000 and we want to add 1 at the correct position, 0000 0000 | 0x80 (which is
* 128 or 1000 0000) >>> 5, where 5 is the bit position from 5 trough 7 in case of splitter size 3. e.g. 0000 0000 | 0000
* 0100 = 0000 0100.
*/
octetBytes[byteIndex] = (byte)(octetBytes[byteIndex] & 0xFF | (0x80 >>> (Byte.SIZE - splitSize) + (inputBitIndex % splitSize)));
}
}

Related

Switching the first 4 bits of a byte and the last half

I need to switch the first half and the second half of a byte: Make 0011 0101 to 0101 0011 for example
I thought it might work this way:
For example, i have 1001 1100
i bitshift to the left 4 times and get 1111 1001(because if the first bit is a 1 the others become a one too)
i bitshift to the right 4 times and get 1100 0000(the second half of the byte gets filled with 0s)
i don't want 1111 1001 but 0000 1001 so i do 0x00001111 & 1111 1001 (which filters the frist 4 bits) to make 1111 1001 to 0000 1001
then i add everything up:
0000 1001 + 1100 0000 = 1100 1001
I got this:
bytes[i] = (byte) (( 0x00001111 & (bytes[i] >> 4)) + (bytes[i] << 4)
);
here is one output: 11111111 to 00000001
I do not really understand why this is happening, I know the binary System and I think I know how bitshifting works but I can't explain this one.
Sorry for bad english :)
Be careful with the >>> operation, which shifts the sign bits without sign extending so zero bits will fill in on the left. The problem is that it is an integer operation. The >> works the same way except it sign extends thru the int.
int i = -1;
i >>>= 30;
System.out.println(i); // prints 3 as expected.
byte b = -1;
b >>>= 6;
System.out.printnln(b); // prints -1 ???
The byte is still -1 because byte b = -1 was shifted as though it was an int then reassigned to a byte. So the byte remained negative. To get 3, you would need to do something that seems strange, like the following.
byte b = -1;
b >>>=30;
System.out.println(b); // prints 3
So to do your swap you need to do the following;
byte b = 0b10100110;
b = (byte)(((b>>>4)&0xF)|(b<<4));
The 0xF mask, masks off those lingering high order bits left over from the conversion from integer back to byte.
I'm not sure about the syntax for bit manipulation in Java, although here's how you can do it.
bitmask = 0x1111;
firstHalf = ((bytes[i] >> 4) & bitmask);
secondHalf = ((bytes[i] & bitmask) << 4)
result = firstHalf | secondHalf;
I don't want 1111 1001 but 0000 1001
If so, you need to use shift right zero fill operator(>>>) instead of preserving sign of the number.
I don't think the formula found works properly.
public byte reverseBitsByte(byte x) {
int intSize = 8;
byte y=0;
for(int position=intSize-1; position>0; position--){
y+=((x&1)<<position);
x >>= 1;
}
return y;
}
static int swapBits(int a) {
// Написать решение сюда ↓
int right = (a & 0b00001111);
right= (right<<4);
int left = (a & 0b11110000);
left = (left>>4);
return (right | left);
}

How does Integer.numberOfTrailingZero(int i) work?

This question is related to but it is different in understanding how the code actually works. More precisely, I do not understand how numberOfTrailingZeros(int i) in java 8 here compute the final result. The code is as follows
public static int numberOfTrailingZeros(int i) {
// HD, Figure 5-14
int y;
if (i == 0) return 32;
int n = 31;
y = i <<16; if (y != 0) { n = n -16; i = y; }
y = i << 8; if (y != 0) { n = n - 8; i = y; }
y = i << 4; if (y != 0) { n = n - 4; i = y; }
y = i << 2; if (y != 0) { n = n - 2; i = y; }
return n - ((i << 1) >>> 31);
}
Now I understand the purpose of the shift operations from 16 to 2, but won't n have already the number of trailing zeros by the last shift operation:
y = i << 2; if (y != 0) { n = n - 2; i = y; }.
That is I do not understand the purpose of this particular line
n - ((i << 1) >>> 31);
why do we need that when n has already the right value?
Can anyone give a detailed example of what is going on?
Thanks!
I will try to explain the algorithm. It is a bit optimized, but I'll start with a (hopefully) more simplified approach.
It is used to determine the number of trailing zero bits of a 32 bit number, that is, how many zeros are on the right side (assuming most significant bit on left). The main idea is to divide the field in two half: if the right one is all zero we add the number of bits in the right half to the result and continue examining the left one (again dividing it); if the right one is not all zero, we can ignore the left one and continue examining the right one (adding nothing to the result).
Example: 0000 0000 0000 0010 0000 0000 0000 0000 (0x0002 0000)
first step (not java code, all numbers base 2, result is decimal):
i = 0000 0000 0000 0010 0000 0000 0000 0000
left = 0000 0000 0000 0010
right = 0000 0000 0000 0000
result = 0
since right is zero, we add 16 (actual number of bits in right part) to the result and continue examining the left part
second step:
i = 0000 0000 0000 0010 // left from previous
left = 0000 0000
right = 0000 0010
result = 16
now right is not zero, so we add nothing to result and continue with right part
third step:
i = 0000 0010 // right from previous
left = 0000
right = 0010
result = 16
right is not zero, nothing added to result, continue with right part
4th step:
i = 0010 // right from previous
left = 00
right = 10
result = 16
5th step:
i = 10 // right from previous
left = 1
right = 0
result = 16
now right is zero, so we add 1 (number of bits in right part) and there is nothing else to divide (that is the return line off original code)
result = 17
Optimizations: instead of having the left and the right part, the algorithm only examines the left left x-most bits of the number and just shift the right part into the left if the right one is not zero, example for first step:
y = i << 16; if (y != 0) { ... i = y;}
and, to avoid having an else part (I think), it starts the result with 31 (sum of all part lengths 1+2+4+8+16) and subtracts the bit count if the right side (after shifting the now left one) is not zero. Again for the first step:
y = i << 16; if (y != 0) { n = n - 16; ....}
Second optimization, last step, instead of
y = i << 1; if (y != 0) { n = n - 1; /* i = y not needed since we are done*/ }
return n;
it does just
return n - ((i << 1) >>> 31);
here ((i << 1) >>>31) is shifting the second bit (second leftmost, second highest bit) of i to leftmost position (eliminating the first bit) and then shifting it to rightmost position, that is, resulting in 0 if the second bit is zero, 1 otherwise. Which is then subtracted from the result (to undo the sum 31 from the beginning).
The first (leftmost) bit need not be directly examined. It only matters if all other bits are zero, that is, the number is 0, checked at the very beginning (if (i == 0) return 32;) or it is -1 in which case the initial value of result is returned: 31.

Maximum number of consecutive 1 in a given number

public int tobinary(int x)
{
int count = 0;
while(x!=0)
{
x=(x&(x<<1)); //how this stuff is working
count++;
}
return count;
}
the above code is working fine but actually i did copy and paste.so i just want to know how that line of code which i mentioned above is working.it would be a great help for me.
for example i am giving i/p as 7 the binary format for this is 0111 so our answer will be 3 but how ?
As we know, the << operator shifts all bits in its operand to the left, here by 1. Also, the & operator performs a bitwise-and on all bits of both its operands.
When will x not be 0? When the bitwise-and operation finds 2 bits that are both set in both operands. The only time that this will be true is when there are 2 or more consecutive 1 bits in x and x << 1.
x : 0000 0111
x << 1: 0000 1110
-----------------
&: 0000 0110
count = 1
If it's not 0, then there are at least 2 consecutive 1 bits in the number, and the (maximum) number of consecutive 1 bits in the number has been reduced by 1. Because we entered the loop in the first place, count it and try again. Eventually there won't be any more consecutive 1 bits, so we exit the loop with the correct count.
x : 0000 0110
x << 1: 0000 1100
-----------------
&: 0000 0100
count = 2
x : 0000 0100
x << 1: 0000 1000
-----------------
&: 0000 0000
count = 3, exit loop and return.
x = (x & (x << 1)) is performed enough times to eliminate the longest consecutive groups of 1 bits. Each loop iteration reduces each consecutive group of 1s by one because the number is logically ANDed with itself shifted left by one bit. This continues until no consecutive group of 1s remains.
To illustrate it for number 110111101:
110111101 // initial x, longest sequence 4
1101111010 // x << 1, count 1
100111000 // new x, longest sequence 3
1001110000 // x << 1, count 2
110000 // new x, longest sequence 2
1100000 // x << 1, count 3
100000 // new x, longest sequence 1
1000000 // x << 1, count 4
0 // new x, end of loop
Do note that since Java 7 it's handy to declare binary literals with int input = 0b110111101.
<< (left shift) : Binary Left Shift Operator. The left operands value is moved left by the number of bits specified by the right operand.
i.e. x<<1 shifts the value bit from right to left by 1 and in the unit position add 0. So, lets say for x=7, bit representation will be 111. Performing x << 1 will return 110 i.e. discarded the head element, shifted the bits to left and added 0 in the end.
This while loop can be broken down into below iteration for initial value of x=7
iteration 1) 111 & 110 =110
iteration 2) 110 & 100 =100
iteration 3) 100 & 000 =000
As value of x is 0, no more iteration
Hope this explains the behaviour of this code. You can put System.out.println(Integer.toBinaryString(x)); to see the change in bit value of x yourself
So what happens is, for each iteration in the while loop,
x = x (AND operator) (2 * x).
For example, when x = 7
count = 0
Iteration 1:
//x = 7 & 14 which results in 6 ie
0111
1110 (AND)
-------
0110 => 6
-------
count = 1
Iteration 2:
//x = 6 & 12 results in 4
0110
1100 AND
-------
0100 => 4
-------
count = 2
Iteration 3:
// x = 4 & 8 results in 0
0100
1000 AND
-----
0000
-----
count = 3
That's how you get 3 for 7

Memory allocation for byte value upon shifting

I wrote a small demo for myself for better understanding of shifting a byte values. So I've byte 'd' with value 127:
class Demo {
public static void main (String args[]) {
byte d = 127; // 8-bit is equal to 0 111 1111
and what I try to do is to shift it in a value of 2 to the left:
int e = d << 2; /* 32-bit shifted value is equal to 0 000 0000 0000 0000 0000 0001 1111 1100
0 * 2^0 + 0 * 2^1 + 1 * 2^2 + 1 * 2^3 + 1 * 2^4 + 1 * 2^5 + 1 * 2^6 + 1 * 2^7 + 1 * 2^8 =
0 * 1 + 0 * 2 + 1 * 4 + 1 * 8 + 1 * 16 + 1 * 32 + 1 * 64 + 1 * 128 + 1 * 256 =
0 + 0 + 4 + 8 + 16 + 32 + 64 + 128 + 256 = 508
*/
Please let me know if my logic in comments is correct, cause it's first time for me of doing such a things. So the real question is about memory allocation once I try to cast 'e' back to byte:
byte f = (byte)(e); /* [] indicates 8-bit at 32-bit shifted value: 0 000 0000 0000 0000 0000 0001 [1111 1100]
0 * 2^0 + 0 * 2^1 + 1 * 2^2 + 1 * 2^3 + 1 * 2^4 + 1 * 2^5 + 1 * 2^6 - 1 * 2^7 =
0 * 1 + 0 * 2 + 1 * 4 + 1 * 8 + 1 * 16 + 1 * 32 + 1 * 64 - 1 * 128 =
0 + 0 + 4 + 8 + 16 + 32 + 64 - 128 = -4
*/
int g = (byte)(e);
}
}
both byte 'f' and int 'g' works perfect but the book I'm reading now advice me to use int with a similar conditions without any explanations so the real thing I worry about is 'memory allocation'. Does it really matter which type to use or once byte value is shifted and the memory got expended to 32-bit it's impossible to reduce it back to 8-bit?
Additionally I've a strange question. How can I check the real 'bitness' of any value? Let's say when I execute something like this:
byte a = 10;
System.out.println(Integer.toBinaryString(a));
int b = 10;
System.out.println(Integer.toBinaryString(b));
The result is '1010' in both cases. But if I get it right byte 'a' use 8 bits of memory and int 'b' use 32. But for:
byte c = 5 << 1;
System.out.println(Integer.toBinaryString(c));
Result is '1010' too but now it use 32 bits of memory while byte and short values are promoted to int when an expression is evaluated. How can I 'see' this in real and ensure by myself?
I'm not entirely sure what you are trying to do, but as for the assignment to f and g, both give the same result, since you assign to it (byte) e which fits within a byte variable.
If you would assign without casting, you'll get a diferent result, since the value of e is too large to fit within a byte.
As for the results of toBinaryString, they obviously don't show leading zeroes, which is why you see the same result for a byte variable and an int variable holding the same value. If you want to see all the bits, you'll have to print the leading zeroes yourself, and the number of leading zeroes depend on the type of the variable (so, for a byte variable, you'll have to add leading zeroes to reach a total of 8 bits, for int you'll have to reach 32 bits, etc...).
If you want to see the leading zeroes, you can make you own version of toBinaryString. Looking at an implementation of Integer, I found that toBinaryString calls toUnsignedString :
public static String toBinaryString(int i) {
return toUnsignedString(i, 1);
}
private static String toUnsignedString(int i, int shift) {
char[] buf = new char[32];
int charPos = 32;
int radix = 1 << shift;
int mask = radix - 1;
do {
buf[--charPos] = digits[i & mask];
i >>>= shift;
} while (i != 0);
return new String(buf, charPos, (32 - charPos));
}
As you can see, this method returns only part of the 32 chars that represent the binary value of the int. If you return new String(buf) instead, you'll get the full binary String with leading zeroes.

Java code To convert byte to Hexadecimal

I have an array of bytes.
I want each byte String of that array to be converted to its corresponding hexadecimal values.
Is there any function in Java to convert a byte array to Hexadecimal ?
byte[] bytes = {-1, 0, 1, 2, 3 };
StringBuilder sb = new StringBuilder();
for (byte b : bytes) {
sb.append(String.format("%02X ", b));
}
System.out.println(sb.toString());
// prints "FF 00 01 02 03 "
See also
java.util.Formatter syntax
%[flags][width]conversion
Flag '0' - The result will be zero-padded
Width 2
Conversion 'X' - The result is formatted as a hexadecimal integer, uppercase
Looking at the text of the question, it's also possible that this is what is requested:
String[] arr = {"-1", "0", "10", "20" };
for (int i = 0; i < arr.length; i++) {
arr[i] = String.format("%02x", Byte.parseByte(arr[i]));
}
System.out.println(java.util.Arrays.toString(arr));
// prints "[ff, 00, 0a, 14]"
Several answers here uses Integer.toHexString(int); this is doable, but with some caveats. Since the parameter is an int, a widening primitive conversion is performed to the byte argument, which involves sign extension.
byte b = -1;
System.out.println(Integer.toHexString(b));
// prints "ffffffff"
The 8-bit byte, which is signed in Java, is sign-extended to a 32-bit int. To effectively undo this sign extension, one can mask the byte with 0xFF.
byte b = -1;
System.out.println(Integer.toHexString(b & 0xFF));
// prints "ff"
Another issue with using toHexString is that it doesn't pad with zeroes:
byte b = 10;
System.out.println(Integer.toHexString(b & 0xFF));
// prints "a"
Both factors combined should make the String.format solution more preferrable.
References
JLS 4.2.1 Integral Types and Values
For byte, from -128 to 127, inclusive
JLS 5.1.2 Widening Primitive Conversion
I am posting because none of the existing answers explain why their approaches work, which I think is really important for this problem. In some cases, this causes the proposed solution to appear unnecessarily complicated and subtle. To illustrate I will provide a fairly straightforward approach, but I'll provide a bit more detail to help illustrate why it works.
First off, what are we trying to do? We want to convert a byte value (or an array of bytes) to a string which represents a hexadecimal value in ASCII. So step one is to find out exactly what a byte in Java is:
The byte data type is an 8-bit signed two's complement integer. It has a minimum value of -128 and a maximum value of 127 (inclusive). The byte data type can be useful for saving memory in large arrays, where the memory savings actually matters. They can also be used in place of int where their limits help to clarify your code; the fact that a variable's range is limited can serve as a form of documentation.
What does this mean? A few things: First and most importantly, it means we are working with 8-bits. So for example we can write the number 2 as 0000 0010. However, since it is two's complement, we write a negative 2 like this: 1111 1110. What is also means is that converting to hex is very straightforward. That is, you simply convert each 4 bit segment directly to hex. Note that to make sense of negative numbers in this scheme you will first need to understand two's complement. If you don't already understand two's complement, you can read an excellent explanation, here: http://www.cs.cornell.edu/~tomf/notes/cps104/twoscomp.html
Converting Two's Complement to Hex In General
Once a number is in two's complement it is dead simple to convert it to hex. In general, converting from binary to hex is very straightforward, and as you will see in the next two examples, you can go directly from two's complement to hex.
Examples
Example 1: Convert 2 to Hex.
1) First convert 2 to binary in two's complement:
2 (base 10) = 0000 0010 (base 2)
2) Now convert binary to hex:
0000 = 0x0 in hex
0010 = 0x2 in hex
therefore 2 = 0000 0010 = 0x02.
Example 2: Convert -2 (in two's complement) to Hex.
1) First convert -2 to binary in two's complement:
-2 (base 10) = 0000 0010 (direct conversion to binary)
1111 1101 (invert bits)
1111 1110 (add 1)
therefore: -2 = 1111 1110 (in two's complement)
2) Now Convert to Hex:
1111 = 0xF in hex
1110 = 0xE in hex
therefore: -2 = 1111 1110 = 0xFE.
Doing this In Java
Now that we've covered the concept, you'll find we can achieve what we want with some simple masking and shifting. The key thing to understand is that the byte you are trying to convert is already in two's complement. You don't do this conversion yourself. I think this is a major point of confusion on this issue. Take for example the follow byte array:
byte[] bytes = new byte[]{-2,2};
We just manually converted them to hex, above, but how can we do it in Java? Here's how:
Step 1: Create a StringBuffer to hold our computation.
StringBuffer buffer = new StringBuffer();
Step 2: Isolate the higher order bits, convert them to hex, and append them to the buffer
Given the binary number 1111 1110, we can isolate the higher order bits by first shifting them over by 4, and then zeroing out the rest of the number. Logically this is simple, however, the implementation details in Java (and many languages) introduce a wrinkle because of sign extension. Essentially, when you shift a byte value, Java first converts your value to an integer, and then performs sign extension. So while you would expect 1111 1110 >> 4 to be 0000 1111, in reality, in Java it is represented as the two's complement 0xFFFFFFFF!
So returning to our example:
1111 1110 >> 4 (shift right 4) = 1111 1111 1111 1111 1111 1111 1111 1111 (32 bit sign-extended number in two's complement)
We can then isolate the bits with a mask:
1111 1111 1111 1111 1111 1111 1111 1111 & 0xF = 0000 0000 0000 0000 0000 0000 0000 1111
therefore: 1111 = 0xF in hex.
In Java we can do this all in one shot:
Character.forDigit((bytes[0] >> 4) & 0xF, 16);
The forDigit function just maps the number you pass it onto the set of hexadecimal numbers 0-F.
Step 3: Next we need to isolate the lower order bits. Since the bits we want are already in the correct position, we can just mask them out:
1111 1110 & 0xF = 0000 0000 0000 0000 0000 0000 0000 1110 (recall sign extension from before)
therefore: 1110 = 0xE in hex.
Like before, in Java we can do this all in one shot:
Character.forDigit((bytes[0] & 0xF), 16);
Putting this all together we can do it as a for loop and convert the entire array:
for(int i=0; i < bytes.length; i++){
buffer.append(Character.forDigit((bytes[i] >> 4) & 0xF, 16));
buffer.append(Character.forDigit((bytes[i] & 0xF), 16));
}
Hopefully this explanation makes things clearer for those of you wondering exactly what is going on in the many examples you will find on the internet. Hopefully I didn't make any egregious errors, but suggestions and corrections are highly welcome!
The fastest way i've yet found to do this is the following:
private static final String HEXES = "0123456789ABCDEF";
static String getHex(byte[] raw) {
final StringBuilder hex = new StringBuilder(2 * raw.length);
for (final byte b : raw) {
hex.append(HEXES.charAt((b & 0xF0) >> 4)).append(HEXES.charAt((b & 0x0F)));
}
return hex.toString();
}
It's ~ 50x faster than String.format. if you want to test it:
public class MyTest{
private static final String HEXES = "0123456789ABCDEF";
#Test
public void test_get_hex() {
byte[] raw = {
(byte) 0xd0, (byte) 0x0b, (byte) 0x01, (byte) 0x2a, (byte) 0x63,
(byte) 0x78, (byte) 0x01, (byte) 0x2e, (byte) 0xe3, (byte) 0x6c,
(byte) 0xd2, (byte) 0xb0, (byte) 0x78, (byte) 0x51, (byte) 0x73,
(byte) 0x34, (byte) 0xaf, (byte) 0xbb, (byte) 0xa0, (byte) 0x9f,
(byte) 0xc3, (byte) 0xa9, (byte) 0x00, (byte) 0x1e, (byte) 0xd5,
(byte) 0x4b, (byte) 0x89, (byte) 0xa3, (byte) 0x45, (byte) 0x35,
(byte) 0xd6, (byte) 0x10,
};
int N = 77777;
long t;
{
t = System.currentTimeMillis();
for (int i = 0; i < N; i++) {
final StringBuilder hex = new StringBuilder(2 * raw.length);
for (final byte b : raw) {
hex.append(HEXES.charAt((b & 0xF0) >> 4)).append(HEXES.charAt((b & 0x0F)));
}
hex.toString();
}
System.out.println(System.currentTimeMillis() - t); // 50
}
{
t = System.currentTimeMillis();
for (int i = 0; i < N; i++) {
StringBuilder hex = new StringBuilder(2 * raw.length);
for (byte b : raw) {
hex.append(String.format("%02X", b));
}
hex.toString();
}
System.out.println(System.currentTimeMillis() - t); // 2535
}
}
}
Edit: Just found something just a lil faster and that holds on one line but is not compatible with JRE 9. Use at your own risks
import javax.xml.bind.DatatypeConverter;
DatatypeConverter.printHexBinary(raw);
Try this way:
byte bv = 10;
String hexString = Integer.toHexString(bv);
Dealing with array (if I understood you correctly):
byte[] bytes = {9, 10, 11, 15, 16};
StringBuffer result = new StringBuffer();
for (byte b : bytes) {
result.append(String.format("%02X ", b));
result.append(" "); // delimiter
}
return result.toString();
As polygenelubricants mentioned, String.format() is the right answer compare to Integer.toHexString() (since it deals with negative numbers in a right way).
A short and simple way to convert byte[] to hex string by using BigInteger:
import java.math.BigInteger;
byte[] bytes = new byte[] {(byte)255, 10, 20, 30};
String hex = new BigInteger(1, bytes).toString(16);
System.out.println(hex); // ff0a141e
How It Works?
The built-in system class java.math.BigInteger class (java.math.BigInteger) is compatible with binary and hex data:
Its has a constructor BigInteger(signum=1, byte[]) to create a big integer by byte[] (set its first parameter signum = 1 to handle correctly the negative bytes)
Use BigInteger.toString(16) to convert the big integer to hex string
To parse a hex number use new BigInteger("ffa74b", 16) - does not handle correctly the leading zero
If you may want to have the leading zero in the hex result, check its size and add the missing zero if necessary:
if (hex.length() % 2 == 1)
hex = "0" + hex;
Notes
Use new BigInteger(1, bytes), instead of new BigInteger(bytes), because Java is "broken by design" and the byte data type does not hold bytes but signed tiny integers [-128...127]. If the first byte is negative, the BigInteger assumes you pass a negative big integer. Just pass 1 as first parameter (signum=1).
The conversion back from hex to byte[] is tricky: sometimes a leading zero enters in the produced output and it should be cleaned like this:
byte[] bytes = new BigInteger("ffa74b", 16).toByteArray();
if (bytes[0] == 0) {
byte[] newBytes = new byte[bytes.length - 1];
System.arraycopy(bytes, 1, newBytes, 0, newBytes.length);
bytes = newBytes;
}
The last note is the if the byte[] has several leading zeroes, they will be lost.
The Best solution is this badass one-liner:
String hex=DatatypeConverter.printHexBinary(byte[] b);
as mentioned here
If you want a constant-width hex representation, i.e. 0A instead of A, so that you can recover the bytes unambiguously, try format():
StringBuilder result = new StringBuilder();
for (byte bb : byteArray) {
result.append(String.format("%02X", bb));
}
return result.toString();
If you are happy to use an external library, the org.apache.commons.codec.binary.Hex class has an encodeHex method which takes a byte[] and returns a char[]. This methods is MUCH faster than the format option, and encapsulates the details of the conversion. Also comes with a decodeHex method for the opposite conversion.
You can use the method from Bouncy Castle Provider library:
org.bouncycastle.util.encoders.Hex.toHexString(byteArray);
The Bouncy Castle Crypto package is a Java implementation of
cryptographic algorithms. This jar contains JCE provider and
lightweight API for the Bouncy Castle Cryptography APIs for JDK 1.5 to
JDK 1.8.
Maven dependency:
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
<version>1.60</version>
</dependency>
or from Apache Commons Codec:
org.apache.commons.codec.binary.Hex.encodeHexString(byteArray);
The Apache Commons Codec package contains simple encoder and decoders
for various formats such as Base64 and Hexadecimal. In addition to
these widely used encoders and decoders, the codec package also
maintains a collection of phonetic encoding utilities.
Maven dependency:
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.11</version>
</dependency>
This is the code that I've found to run the fastest so far. I ran it on 109015 byte arrays of length 32, in 23ms. I was running it on a VM so it'll probably run faster on bare metal.
public static final char[] HEX_DIGITS = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F'};
public static char[] encodeHex( final byte[] data ){
final int l = data.length;
final char[] out = new char[l<<1];
for( int i=0,j=0; i<l; i++ ){
out[j++] = HEX_DIGITS[(0xF0 & data[i]) >>> 4];
out[j++] = HEX_DIGITS[0x0F & data[i]];
}
return out;
}
Then you can just do
String s = new String( encodeHex(myByteArray) );
Java 17: Introducing java.util.HexFormat
Java 17 comes with a utility to convert byte arrays and numbers to their hexadecimal counterparts. Let's say we have an MD5 digest of "Hello World" as a byte-array:
var md5 = MessageDigest.getInstance("md5");
md5.update("Hello world".getBytes(UTF_8));
var digest = md5.digest();
Now we can use the HexFormat.of().formatHex(byte[]) method to convert the given byte[] to its hexadecimal form:
jshell> HexFormat.of().formatHex(digest)
$7 ==> "3e25960a79dbc69b674cd4ec67a72c62"
The withUpperCase() method returns the uppercase version of the previous output:
jshell> HexFormat.of().withUpperCase().formatHex(digest)
$8 ==> "3E25960A79DBC69B674CD4EC67A72C62"
BigInteger n = new BigInteger(byteArray);
String hexa = n.toString(16);
Here is a simple function to convert byte to Hexadecimal
private static String convertToHex(byte[] data) {
StringBuffer buf = new StringBuffer();
for (int i = 0; i < data.length; i++) {
int halfbyte = (data[i] >>> 4) & 0x0F;
int two_halfs = 0;
do {
if ((0 <= halfbyte) && (halfbyte <= 9))
buf.append((char) ('0' + halfbyte));
else
buf.append((char) ('a' + (halfbyte - 10)));
halfbyte = data[i] & 0x0F;
} while(two_halfs++ < 1);
}
return buf.toString();
}
Others have covered the general case. But if you have a byte array of a known form, for example a MAC address, then you can:
byte[] mac = { (byte)0x00, (byte)0x00, (byte)0x00, (byte)0x00, (byte)0x00 };
String str = String.format("%02X:%02X:%02X:%02X:%02X:%02X",
mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
There's your fast method:
private static final String[] hexes = new String[]{
"00","01","02","03","04","05","06","07","08","09","0A","0B","0C","0D","0E","0F",
"10","11","12","13","14","15","16","17","18","19","1A","1B","1C","1D","1E","1F",
"20","21","22","23","24","25","26","27","28","29","2A","2B","2C","2D","2E","2F",
"30","31","32","33","34","35","36","37","38","39","3A","3B","3C","3D","3E","3F",
"40","41","42","43","44","45","46","47","48","49","4A","4B","4C","4D","4E","4F",
"50","51","52","53","54","55","56","57","58","59","5A","5B","5C","5D","5E","5F",
"60","61","62","63","64","65","66","67","68","69","6A","6B","6C","6D","6E","6F",
"70","71","72","73","74","75","76","77","78","79","7A","7B","7C","7D","7E","7F",
"80","81","82","83","84","85","86","87","88","89","8A","8B","8C","8D","8E","8F",
"90","91","92","93","94","95","96","97","98","99","9A","9B","9C","9D","9E","9F",
"A0","A1","A2","A3","A4","A5","A6","A7","A8","A9","AA","AB","AC","AD","AE","AF",
"B0","B1","B2","B3","B4","B5","B6","B7","B8","B9","BA","BB","BC","BD","BE","BF",
"C0","C1","C2","C3","C4","C5","C6","C7","C8","C9","CA","CB","CC","CD","CE","CF",
"D0","D1","D2","D3","D4","D5","D6","D7","D8","D9","DA","DB","DC","DD","DE","DF",
"E0","E1","E2","E3","E4","E5","E6","E7","E8","E9","EA","EB","EC","ED","EE","EF",
"F0","F1","F2","F3","F4","F5","F6","F7","F8","F9","FA","FB","FC","FD","FE","FF"
};
public static String byteToHex(byte b) {
return hexes[b&0xFF];
}
Just like some other answers, I recommend to use String.format() and BigInteger. But to interpret the byte array as big-endian binary representation instead of two's-complement binary representation (with signum and incomplete use of possible hex values range) use BigInteger(int signum, byte[] magnitude), not BigInteger(byte[] val).
For example, for a byte array of length 8 use:
String.format("%016X", new BigInteger(1,bytes))
Advantages:
leading zeros
no signum
only built-in functions
only one line of code
Disadvantage:
there might be more efficient ways to do that
Example:
byte[] bytes = new byte[8];
Random r = new Random();
System.out.println("big-endian | two's-complement");
System.out.println("-----------------|-----------------");
for (int i = 0; i < 10; i++) {
r.nextBytes(bytes);
System.out.print(String.format("%016X", new BigInteger(1,bytes)));
System.out.print(" | ");
System.out.print(String.format("%016X", new BigInteger(bytes)));
System.out.println();
}
Example output:
big-endian | two's-complement
-----------------|-----------------
3971B56BC7C80590 | 3971B56BC7C80590
64D3C133C86CCBDC | 64D3C133C86CCBDC
B232EFD5BC40FA61 | -4DCD102A43BF059F
CD350CC7DF7C9731 | -32CAF338208368CF
82CDC9ECC1BC8EED | -7D3236133E437113
F438C8C34911A7F5 | -BC7373CB6EE580B
5E99738BE6ACE798 | 5E99738BE6ACE798
A565FE5CE43AA8DD | -5A9A01A31BC55723
032EBA783D2E9A9F | 032EBA783D2E9A9F
8FDAA07263217ABA | -70255F8D9CDE8546
Creating (and destroying) a bunch of String instances is not a good way if performance is an issue.
Please ignore those verbose (duplicate) arguments checking statements (ifs). That's for (another) educational purposes.
Full maven project: http://jinahya.googlecode.com/svn/trunk/com.googlecode.jinahya/hex-codec/
Encoding...
/**
* Encodes a single nibble.
*
* #param decoded the nibble to encode.
*
* #return the encoded half octet.
*/
protected static int encodeHalf(final int decoded) {
switch (decoded) {
case 0x00:
case 0x01:
case 0x02:
case 0x03:
case 0x04:
case 0x05:
case 0x06:
case 0x07:
case 0x08:
case 0x09:
return decoded + 0x30; // 0x30('0') - 0x39('9')
case 0x0A:
case 0x0B:
case 0x0C:
case 0x0D:
case 0x0E:
case 0x0F:
return decoded + 0x57; // 0x41('a') - 0x46('f')
default:
throw new IllegalArgumentException("illegal half: " + decoded);
}
}
/**
* Encodes a single octet into two nibbles.
*
* #param decoded the octet to encode.
* #param encoded the array to which each encoded nibbles are written.
* #param offset the offset in the array.
*/
protected static void encodeSingle(final int decoded, final byte[] encoded,
final int offset) {
if (encoded == null) {
throw new IllegalArgumentException("null encoded");
}
if (encoded.length < 2) {
// not required
throw new IllegalArgumentException(
"encoded.length(" + encoded.length + ") < 2");
}
if (offset < 0) {
throw new IllegalArgumentException("offset(" + offset + ") < 0");
}
if (offset >= encoded.length - 1) {
throw new IllegalArgumentException(
"offset(" + offset + ") >= encoded.length(" + encoded.length
+ ") - 1");
}
encoded[offset] = (byte) encodeHalf((decoded >> 4) & 0x0F);
encoded[offset + 1] = (byte) encodeHalf(decoded & 0x0F);
}
/**
* Decodes given sequence of octets into a sequence of nibbles.
*
* #param decoded the octets to encode
*
* #return the encoded nibbles.
*/
protected static byte[] encodeMultiple(final byte[] decoded) {
if (decoded == null) {
throw new IllegalArgumentException("null decoded");
}
final byte[] encoded = new byte[decoded.length << 1];
int offset = 0;
for (int i = 0; i < decoded.length; i++) {
encodeSingle(decoded[i], encoded, offset);
offset += 2;
}
return encoded;
}
/**
* Encodes given sequence of octets into a sequence of nibbles.
*
* #param decoded the octets to encode.
*
* #return the encoded nibbles.
*/
public byte[] encode(final byte[] decoded) {
return encodeMultiple(decoded);
}
Decoding...
/**
* Decodes a single nibble.
*
* #param encoded the nibble to decode.
*
* #return the decoded half octet.
*/
protected static int decodeHalf(final int encoded) {
switch (encoded) {
case 0x30: // '0'
case 0x31: // '1'
case 0x32: // '2'
case 0x33: // '3'
case 0x34: // '4'
case 0x35: // '5'
case 0x36: // '6'
case 0x37: // '7'
case 0x38: // '8'
case 0x39: // '9'
return encoded - 0x30;
case 0x41: // 'A'
case 0x42: // 'B'
case 0x43: // 'C'
case 0x44: // 'D'
case 0x45: // 'E'
case 0x46: // 'F'
return encoded - 0x37;
case 0x61: // 'a'
case 0x62: // 'b'
case 0x63: // 'c'
case 0x64: // 'd'
case 0x65: // 'e'
case 0x66: // 'f'
return encoded - 0x57;
default:
throw new IllegalArgumentException("illegal half: " + encoded);
}
}
/**
* Decodes two nibbles into a single octet.
*
* #param encoded the nibble array.
* #param offset the offset in the array.
*
* #return decoded octet.
*/
protected static int decodeSingle(final byte[] encoded, final int offset) {
if (encoded == null) {
throw new IllegalArgumentException("null encoded");
}
if (encoded.length < 2) {
// not required
throw new IllegalArgumentException(
"encoded.length(" + encoded.length + ") < 2");
}
if (offset < 0) {
throw new IllegalArgumentException("offset(" + offset + ") < 0");
}
if (offset >= encoded.length - 1) {
throw new IllegalArgumentException(
"offset(" + offset + ") >= encoded.length(" + encoded.length
+ ") - 1");
}
return (decodeHalf(encoded[offset]) << 4)
| decodeHalf(encoded[offset + 1]);
}
/**
* Encodes given sequence of nibbles into a sequence of octets.
*
* #param encoded the nibbles to decode.
*
* #return the encoded octets.
*/
protected static byte[] decodeMultiple(final byte[] encoded) {
if (encoded == null) {
throw new IllegalArgumentException("null encoded");
}
if ((encoded.length & 0x01) == 0x01) {
throw new IllegalArgumentException(
"encoded.length(" + encoded.length + ") is not even");
}
final byte[] decoded = new byte[encoded.length >> 1];
int offset = 0;
for (int i = 0; i < decoded.length; i++) {
decoded[i] = (byte) decodeSingle(encoded, offset);
offset += 2;
}
return decoded;
}
/**
* Decodes given sequence of nibbles into a sequence of octets.
*
* #param encoded the nibbles to decode.
*
* #return the decoded octets.
*/
public byte[] decode(final byte[] encoded) {
return decodeMultiple(encoded);
}
This is a very fast way. No external libaries needed.
final protected static char[] HEXARRAY = "0123456789abcdef".toCharArray();
public static String encodeHexString( byte[] bytes ) {
char[] hexChars = new char[bytes.length * 2];
for (int j = 0; j < bytes.length; j++) {
int v = bytes[j] & 0xFF;
hexChars[j * 2] = HEXARRAY[v >>> 4];
hexChars[j * 2 + 1] = HEXARRAY[v & 0x0F];
}
return new String(hexChars);
}
I couldn't figure out what exactly you meant by byte String, but here are some conversions from byte to String and vice versa, of course there is a lot more on the official documentations
Integer intValue = 149;
The corresponding byte value is:
Byte byteValue = intValue.byteValue(); // this will convert the rightmost byte of the intValue to byte, because Byte is an 8 bit object and Integer is at least 16 bit, and it will give you a signed number in this case -107
get the integer value back from a Byte variable:
Integer anInt = byteValue.intValue(); // This will convert the byteValue variable to a signed Integer
From Byte and Integer to hex String:
This is the way I do it:
Integer anInt = 149
Byte aByte = anInt.byteValue();
String hexFromInt = "".format("0x%x", anInt); // This will output 0x95
String hexFromByte = "".format("0x%x", aByte); // This will output 0x95
Converting an array of bytes to a hex string:
As far as I know there is no simple function to convert all the elements inside an array of some Object to elements of another Object, So you have to do it yourself. You can use the following functions:
From byte[] to String:
public static String byteArrayToHexString(byte[] byteArray){
String hexString = "";
for(int i = 0; i < byteArray.length; i++){
String thisByte = "".format("%x", byteArray[i]);
hexString += thisByte;
}
return hexString;
}
And from hex string to byte[]:
public static byte[] hexStringToByteArray(String hexString){
byte[] bytes = new byte[hexString.length() / 2];
for(int i = 0; i < hexString.length(); i += 2){
String sub = hexString.substring(i, i + 2);
Integer intVal = Integer.parseInt(sub, 16);
bytes[i / 2] = intVal.byteValue();
String hex = "".format("0x%x", bytes[i / 2]);
}
return bytes;
}
It is too late but I hope this could help some others ;)
If you like streams, here is a single-expression version of the format-and-concatenate approach:
String hex = IntStream.range(0, bytes.length)
.map(i -> bytes[i] & 0xff)
.mapToObj(b -> String.format("%02x", b))
.collect(Collectors.joining());
It's a shame there isn't a method like Arrays::streamUnsignedBytes for this.
If you use Tink, then there is:
package com.google.crypto.tink.subtle;
public final class Hex {
public static String encode(final byte[] bytes) { ... }
public static byte[] decode(String hex) { ... }
}
so something like this should work:
import com.google.crypto.tink.subtle.Hex;
byte[] bytes = {-1, 0, 1, 2, 3 };
String enc = Hex.encode(bytes);
byte[] dec = Hex.decode(enc)
Just adding my two cents as I see many answers using a char array and/or using a StringBuilder instance and claiming to be fast or faster.
Since I had a different idea using the ASCII table organization having 0-9 at code points 48-57 and A-F at 65-70 and a-f at 97-102, I wanted to test which idea is the fastest.
Since I have not done something like that for quite some years now, I am giving this an extensive shoot out. I use 1 billion bytes in different sized arrays (1M, 1K, 10) so we have 1000 times 1M bytes per array, 1M times 1000 bytes per array and 100M times for 10 bytes per array.
Turns out the char array from 1-F wins. Using a char array as output instead of a StringBuilder also wins hands down (less objects, no test of capacity and no new array nor copying when growing). Furthermore you seem to get a small penalty when using a foreach (for(var b : bytes)) loop.
The version using my idea was approx. 15% slower for 1M bytes per array, 21% slower for 1000 bytes per array and 18% slower for 10 bytes per array. The StringBuilder version was 210%, 380% and 310% slower.
Bad but not that unexpected as a lookup of small arrays in first level cache beats one if and an add... . (one cache access + offset calculation vs. one if, one jump, one add -> not sure about the jump thou).
My version:
public static String bytesToHex(byte [] bytes) {
char [] result = new char [bytes.length * 2];
for(int index = 0; index < bytes.length; index++) {
int v = bytes[index];
int upper = (v >>> 4) & 0xF;
result[index * 2] = (char)(upper + (upper < 10 ? 48 : 65 - 10));
int lower = v & 0xF;
result[index * 2 + 1] = (char)(lower + (lower < 10 ? 48 : 65 - 10));
}
return new String(result);
}
PS: Yes I did multiple runs and took the best run each, did warm up and also did runs with 10 billion characters to be sure same picture... .
Use
Integer.toHexString((int)b);

Categories

Resources