How do I convert a byte array to a long in Java? - java

I am reading 8 bytes of data in from a hardware device. I need to convert them into a numeric value. I think I want to convert them to a long as that should fit 8 bytes. I am not very familiar with Java and low level data type operations. I seem to have two problems (apart from the fact there is almost no documentation for the hardware in question), The bytes are expecting to be unsigned, so I can't do a straight integer conversion. I am not sure what endianness they are.
Any advice would be appreciated.
Ended up with this (taken from some source code I probably should have read a week ago):
public static final long toLong (byte[] byteArray, int offset, int len)
{
long val = 0;
len = Math.min(len, 8);
for (int i = (len - 1); i >= 0; i--)
{
val <<= 8;
val |= (byteArray [offset + i] & 0x00FF);
}
return val;
}

Shifting bytes according to the endianness of the data is fairly straightforward. There is a small trick with the long datatype, however, because a binary operation on integral types of int or smaller promotes the operands to int. For left shifts larger than 31 bits, this will result in zero, since all of the bits have been shifted out of the int range.
So, force promotion to long by including a long operand in the calculation. Below, I do this by masking each byte with the value 0xFFL, which is a long, forcing the result to be a long.
byte[] buf = new byte[8];
/* Fill buf somehow... */
long l = ((buf[0] & 0xFFL) << 56) |
((buf[1] & 0xFFL) << 48) |
((buf[2] & 0xFFL) << 40) |
((buf[3] & 0xFFL) << 32) |
((buf[4] & 0xFFL) << 24) |
((buf[5] & 0xFFL) << 16) |
((buf[6] & 0xFFL) << 8) |
((buf[7] & 0xFFL) << 0) ;

Byte#longValue() should do it
And if not (thanks for the source example) you can use java.nio.ByteBuffer such as in
public static long toLong(byte[] b) {
ByteBuffer bb = ByteBuffer.allocate(b.length);
bb.put(b);
return bb.getLong();
}
The initial order is BIG_ENDIAN you can reed more here

I believe that you could benefit from using java.nio. This is how you can store 8 bytes in a long:
// Byte Array TO Long
public static long batol(byte[] buff) {
return batol(buff, false);
}
public static long batol(byte[] buff, boolean littleEndian) {
assert(buff.length == 8);
ByteBuffer bb = ByteBuffer.wrap(buff);
if (littleEndian) bb.order(ByteOrder.LITTLE_ENDIAN);
return bb.getLong();
}
Of course, the resulting longs will have signed representation, but they will have identical binary values to the source data. For a 64 bit+ unsigned representation and arithmatic, you'll need to use BigInteger. Here's how to convert from "unsigned" data stored in a long to a correct BigInteger:
// "Unsigned" Long TO Big Integer
public static BigInteger ultobi(long ul) {
byte[] buff = new byte[8];
ByteBuffer.wrap(buff).asLongBuffer().put(ul);
return new BigInteger(+1, buff);
}

ByteBuffer buffer = ByteBuffer.wrap(bytes);
buffer.getLong();

For the endianness, test with some numbers you know, and then you will be using a byte shifting to move them into the long.
You may find this to be a starting point.
http://www.janeg.ca/scjp/oper/shift.html
The difficulty is that depending on the endianess will change how you do it, but you will shift by 24, 16, 8 then add the last one, basically, if doing 32 bits, but you are going longer, so just do extra shifting.

Take a look at BigInteger(byte[]). It is almost what you want except that it is a signed one. So you may add one more byte to it before you pass it on to BigInteger.
Another thing is that you should be aware of what endian your bytes are.
Hope this helps.

If you're reading from an InputStream, you may also want to look at DataInputStream.readLong(). Java 1.5 introduced Long.reverseBytes(long) which may help you with endianness.

You can use:
byte bVal = 127;
Long longVar = Long.valueOf(bVal);

public static long convertToLong(byte[] array)
{
ByteBuffer buffer = ByteBuffer.wrap(array);
return buffer.getLong();
}

The next code parses bytes as a signed number of arbitrary length ≤ 8.
static long bytesToSignedNumber(boolean reverseOrder, byte... bytes) {
if (bytes.length > 8) bytes = Arrays.copyOfRange(bytes, 0, 8); //delete this line to ignore the higher excess bytes instead of the lower ones
int l = bytes.length;
long number = 0;
for (int i = 0; i < l; i++) {
long current;
if (l==3 || l==5 || l==6 || l==7 //delete this line if you want to operate with 3,5,6,7-byte signed numbers (unlikely)
|| !reverseOrder && (i > 0)
|| reverseOrder && (i < l-1)) {
current = ((long) bytes[i]) & 0x0000_0000_0000_00ff; //unsigned
} else {
current = (long) bytes[i]; //signed
}
if (reverseOrder) number += current << (8 * i);
else number = (number << 8) + current;
}
return number;
}
It parses an byte array as a number, of minimum existing type, converted to long. A few examples:
bytesToSignedNumber(false, 0x01, 0x04) returns 260 (2 bytes as short)
bytesToSignedNumber(false, 0xF1, 0x04) returns -3836 (2 bytes as short)
bytesToSignedNumber(false, 0x01, 0x01, 0x04) returns 65796 (3 bytes as int)
bytesToSignedNumber(false, 0xF1, 0x01, 0x04) returns 15794436 (3 bytes as int)
bytesToSignedNumber(false, 0xF1, 0x01, 0x01, 0x04) returns -251592444 (4 bytes as int)
bytesToSignedNumber(false, 0x0F, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x04) returns 1081146489369067777 (8 of 9 bytes as long)

Another Alternative
From Google
com.google.common.primitives
Longs.fromByteArray(bytes);

Related

turn my 32 character int into a 32 byte array as is in Java

I am wondering how can I turn my 32 character int into a 32-byte array as it is represented.
Example:
I have this int:
int test = 123456789;
And I want to turn it into this:
byte[] Write_Page_Four = new byte[] {
(byte) 0x00, (byte) 0x00,
(byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00,
(byte) 0x00, (byte) 0x00, (byte) 0x00, (byte) 0x00,
(byte) 0x00, (byte) 0x01, (byte) 0x23, (byte) 0x45,
(byte) 0x67, (byte) 0x89};
Currently, I'm thinking of splitting my int by 2 and just manually assigning them to the byte array but I am having some troubles in doing so, and I believe that this is not the best practice for my problem.
This is what I have ATM, which is returning error and still work on progress though and I could use some advice on it:
String test2 = "01";
String test1 = "0x"+test2;
byte test = Byte.valueOf(test1);
System.out.println("teeeeest-----"+test);
byte[] Write_Page_Four = new byte[] {(byte) test};
And this one is returning an error:
java.lang.NumberFormatException: For input string: "0x01"
What is causing problems
Byte.valueOf doesn't parse data like the Java compiler does: it expects as input as a decimal number.
What you can use however, is Byte.valueOf(String,int) with an arbitrary radix. In that case you can solve it using:
byte test = Byte.valueOf(test2,16); //using test2, not test1
Mind that should not add "0x" in the front. Nevertheless this is an inefficient way to do this.
Ints are 32-bits, not 32-bytes
A second problem is that you state that you can store a number like 12345678901234567890123456789011 into an int. You cannot. An int has 32 bits. This means its representation is limited to more or less 2.1B. So I think you mean you store 12345678901234567890123456789011 in a String?
Number systems
Mind that the number 12345678901234567890123456789011 is not represented internally as (byte) 0x12, (byte) 0x34,... unless you are working with binary coded decimals. This is because a computer uses the binary number system (and thus groups bytes with the hexadecimal representation), whereas humans use the decimal representation. 123456789 for instance will be represented as 0x07,0x5B,0xCD 0x15.
Serializing an int (or other datastructure) using an array of bytes
You can convert an int (and other datatypes) into an array of bytes using this code:
ByteBuffer b = ByteBuffer.allocate(4);
b.putInt(test);
byte[] result = b.array(); //result will be 4 bytes,
//since you can represent any int with four bytes.
Or, in case you want to represent the int like the way you do this, you could use the following method:
int t = test;
byte[] dat = new byte[5];//at most 5 bytes needed
for(int j = 4; test != 0; j--) {
int rm = t%100;
dat[j] = (byte) (rm%10+((rm/10)<<8));
t /= 100;
}
//result is dat
Instead of processing the textual representation of the number I'd recommend to simply calculate the single numbers:
Get two digits of the number each time:
int inp = 1234...;
for(int lowerBound = 1 ; lowerBound < Integer.MAX_VALUE ; lowerBound *= 100)
//twoDigit contains two digits of the input-number
int twoDigit = (inp /lowerBound) % 100;
Transform these two digits into a byte:
byte transform(int i){
if(i == 0)
return 0x00;
int lsd = (i % 10); //least significant digit of the input (decimal)
int msd = (i / 10); //most significant digit
//merge lsd and msd into a single number, where the lower 4 bits are reserved for
//lsd and the higher 4 bits for msd
return lsd | (msd << 4);
}
The complete code would look like this:
import java.util.Arrays;
public class test
{
private static final int BYTES = 4;
public static void main(String[] args){
int v = 12345678;
int at_arr = BYTES - 1;
byte[] result = new byte[BYTES];//int is 32-bit/4 byte long
for(int lowerBound = 1 ; lowerBound < Integer.MAX_VALUE && at_arr > -1; lowerBound *= 100, at_arr--)
result[at_arr] = transformDigits((v / lowerBound) % 100);
for(byte b : result)
System.out.print(" 0x" + Integer.toString(b , 16) + ",");
System.out.println();
}
static byte transformDigits(int i){
if(i == 0)
return 0x00;
int lsd = (i % 10); //least significant digit of the input (decimal)
int msd = (i / 10); //most significant digit
//merge lsd and msd into a single number, where the lower 4 bits are reserved for
//lsd and the higher 4 bits for msd
return (byte) (lsd | (msd << 4));
}
}
This code can be used basically for any integral type, if the types and value of BYTES are updated appropriately.
Here's how to convert an int to a byte[]:
int test = 123456789;
byte[] bytes = new byte[4];
bytes[0] = (byte)(test >> 24);
bytes[1] = (byte)(test >> 16);
bytes[2] = (byte)(test >> 8);
bytes[3] = (byte)test;
System.out.printf("%02x %02x %02x %02x%n", bytes[0], bytes[1], bytes[2], bytes[3]);
Output
07 5b cd 15
You can also inline it, if you want:
int test = 123456789;
byte[] bytes = new byte[] { (byte)(test >> 24),
(byte)(test >> 16),
(byte)(test >> 8),
(byte)test };
System.out.printf("%02x %02x %02x %02x%n", bytes[0], bytes[1], bytes[2], bytes[3]);

Subtraction/difference between bytes in Java

I'm trying to subtract one byte from another while making sure no overflow happens, but get unexpected results.
My case is that I have an byte black and white image, of which I want to subtract the background. Hence I need to work with bytes and prevent overflows from happening. I have some difficulty presumably with the signedness of the bytes when subtracting the background image from the other image. Data2 is the background array, and data1 is the other image.
In the following code, I expect data2 array to be subtracted from data1 array. However I get low values when I am sure there should be high ones.
for (int i = 0; i < data1.length; i++) {
if (data1[i] > data2[i]) {
dest[i] = (byte) (data1[i] - data2[i]);
} else {
dest[i] = 0;
}
}
I figured I should make sure data2 byte isn't negative and being added to data1.
So I came to:
for (int i = 0; i < data1.length; i++) {
if (data1[i] > data2[i]) {
dest[i] = (byte) ((int)data1[i] & 0xff - (int)data2[i] & 0xff - 128);
} else {
dest[i] = 0;
}
}
However this also doesn't give the right results.
My thoughts on this currently are:
(byte) ((int)data1[i] & 0xff - (int)data2[i] & 0xff - 128);
(int) cast: make sure bytes are cast to integer.
&0xff: make value unsigned.
- subtraction smaller value from bigger value.
- 128: subtract 128 to make signed again.
(byte): cast back to byte.
I hope I'm doing something stupidly wrong here, or my problem resides somewhere else of which I can't figure out where.
Edit
I seem to have figured out a part of the issue:
data1[i] > data2[i] is handled wrong (in my case) when the bytes are signed. Instead:
if ((data1[i] & 0xff) > (data2[i] & 0xff)) {
seems to produce the right results, instead of the previous comparison.
The point here is that your bytes come from an API that uses 8 bits to encode the light of the pixel, so they range 0; 0xFF. However Java bytes are -128; 127, so any bit pattern after 0x7F will be interpreted as a negative number. For example the bits in 0xF0 are 128 if the byte is unsigned and -16 if interpreted as a signed byte:
System.out.println(0xFF > 0); // true
System.out.println((byte) 0xFF > 0); // false
So when comparing your unsigned bytes you want to promote pixels to int with Byte.toUnsignedInt(byteVal) (or ((int) byteVal) & 0xFF on Java 7).
Always remember Java bytes are signed. If data1[i] is 127, and data2[i] is -128, then data1[i] > data2[i], but data1[i] - data2[i] does not fit into a signed byte; that result is 255.
If you treat the bytes as unsigned, that's fine. That more-or-less means printing them out after using & 0xFF, and such. That will work just fine; it will give the right results if you treat them as unsigned correctly.
To ensure that you only substract non negative bytes from from bytes that are greater you should use :
for (int i = 0; i < data1.length; i++) {
if (data1[i] > data2[i] && data2[i]>=0 ) {
dest[i] = (byte) (data1[i] - data2[i]);
} else {
dest[i] = 0;
}
}
Your second code does not work because the & operator promotes the values to type int.
Consider the case where data1=127 (0x7F) and data2=-128 (0x80).
data1 & 0xff is type int and has value 127 (0x0000007F)
data2 & 0xff is type int and has value 128 (0x00000080)
data1[i] & 0xff - data2[i] & 0xff is type int and has value -1 (0xFFFFFFFF)
(byte)(data1[i] & 0xff - data2[i] & 0xff) is type byte and has value -1 (0xFF)
So you still have gotten an overflow
For some reason, comparing bytes is handled weirdly. If I convert the bytes to unsigned ints in the comparison, the comparison works correctly and my results are as I expected them.
I can then subtract the bytes as if they were unsigned as Louis Wasserman pointed out, which was new to me.
for (int i = 0; i < data1.length; i++) {
if ((data1[i] & 0xff) > (data2[i] & 0xff)) {
dest[i] = (byte)(data1[i] - data2[i]);
} else {
dest[i] = 0;
}
}

Java code To convert byte to Hexadecimal

I have an array of bytes.
I want each byte String of that array to be converted to its corresponding hexadecimal values.
Is there any function in Java to convert a byte array to Hexadecimal ?
byte[] bytes = {-1, 0, 1, 2, 3 };
StringBuilder sb = new StringBuilder();
for (byte b : bytes) {
sb.append(String.format("%02X ", b));
}
System.out.println(sb.toString());
// prints "FF 00 01 02 03 "
See also
java.util.Formatter syntax
%[flags][width]conversion
Flag '0' - The result will be zero-padded
Width 2
Conversion 'X' - The result is formatted as a hexadecimal integer, uppercase
Looking at the text of the question, it's also possible that this is what is requested:
String[] arr = {"-1", "0", "10", "20" };
for (int i = 0; i < arr.length; i++) {
arr[i] = String.format("%02x", Byte.parseByte(arr[i]));
}
System.out.println(java.util.Arrays.toString(arr));
// prints "[ff, 00, 0a, 14]"
Several answers here uses Integer.toHexString(int); this is doable, but with some caveats. Since the parameter is an int, a widening primitive conversion is performed to the byte argument, which involves sign extension.
byte b = -1;
System.out.println(Integer.toHexString(b));
// prints "ffffffff"
The 8-bit byte, which is signed in Java, is sign-extended to a 32-bit int. To effectively undo this sign extension, one can mask the byte with 0xFF.
byte b = -1;
System.out.println(Integer.toHexString(b & 0xFF));
// prints "ff"
Another issue with using toHexString is that it doesn't pad with zeroes:
byte b = 10;
System.out.println(Integer.toHexString(b & 0xFF));
// prints "a"
Both factors combined should make the String.format solution more preferrable.
References
JLS 4.2.1 Integral Types and Values
For byte, from -128 to 127, inclusive
JLS 5.1.2 Widening Primitive Conversion
I am posting because none of the existing answers explain why their approaches work, which I think is really important for this problem. In some cases, this causes the proposed solution to appear unnecessarily complicated and subtle. To illustrate I will provide a fairly straightforward approach, but I'll provide a bit more detail to help illustrate why it works.
First off, what are we trying to do? We want to convert a byte value (or an array of bytes) to a string which represents a hexadecimal value in ASCII. So step one is to find out exactly what a byte in Java is:
The byte data type is an 8-bit signed two's complement integer. It has a minimum value of -128 and a maximum value of 127 (inclusive). The byte data type can be useful for saving memory in large arrays, where the memory savings actually matters. They can also be used in place of int where their limits help to clarify your code; the fact that a variable's range is limited can serve as a form of documentation.
What does this mean? A few things: First and most importantly, it means we are working with 8-bits. So for example we can write the number 2 as 0000 0010. However, since it is two's complement, we write a negative 2 like this: 1111 1110. What is also means is that converting to hex is very straightforward. That is, you simply convert each 4 bit segment directly to hex. Note that to make sense of negative numbers in this scheme you will first need to understand two's complement. If you don't already understand two's complement, you can read an excellent explanation, here: http://www.cs.cornell.edu/~tomf/notes/cps104/twoscomp.html
Converting Two's Complement to Hex In General
Once a number is in two's complement it is dead simple to convert it to hex. In general, converting from binary to hex is very straightforward, and as you will see in the next two examples, you can go directly from two's complement to hex.
Examples
Example 1: Convert 2 to Hex.
1) First convert 2 to binary in two's complement:
2 (base 10) = 0000 0010 (base 2)
2) Now convert binary to hex:
0000 = 0x0 in hex
0010 = 0x2 in hex
therefore 2 = 0000 0010 = 0x02.
Example 2: Convert -2 (in two's complement) to Hex.
1) First convert -2 to binary in two's complement:
-2 (base 10) = 0000 0010 (direct conversion to binary)
1111 1101 (invert bits)
1111 1110 (add 1)
therefore: -2 = 1111 1110 (in two's complement)
2) Now Convert to Hex:
1111 = 0xF in hex
1110 = 0xE in hex
therefore: -2 = 1111 1110 = 0xFE.
Doing this In Java
Now that we've covered the concept, you'll find we can achieve what we want with some simple masking and shifting. The key thing to understand is that the byte you are trying to convert is already in two's complement. You don't do this conversion yourself. I think this is a major point of confusion on this issue. Take for example the follow byte array:
byte[] bytes = new byte[]{-2,2};
We just manually converted them to hex, above, but how can we do it in Java? Here's how:
Step 1: Create a StringBuffer to hold our computation.
StringBuffer buffer = new StringBuffer();
Step 2: Isolate the higher order bits, convert them to hex, and append them to the buffer
Given the binary number 1111 1110, we can isolate the higher order bits by first shifting them over by 4, and then zeroing out the rest of the number. Logically this is simple, however, the implementation details in Java (and many languages) introduce a wrinkle because of sign extension. Essentially, when you shift a byte value, Java first converts your value to an integer, and then performs sign extension. So while you would expect 1111 1110 >> 4 to be 0000 1111, in reality, in Java it is represented as the two's complement 0xFFFFFFFF!
So returning to our example:
1111 1110 >> 4 (shift right 4) = 1111 1111 1111 1111 1111 1111 1111 1111 (32 bit sign-extended number in two's complement)
We can then isolate the bits with a mask:
1111 1111 1111 1111 1111 1111 1111 1111 & 0xF = 0000 0000 0000 0000 0000 0000 0000 1111
therefore: 1111 = 0xF in hex.
In Java we can do this all in one shot:
Character.forDigit((bytes[0] >> 4) & 0xF, 16);
The forDigit function just maps the number you pass it onto the set of hexadecimal numbers 0-F.
Step 3: Next we need to isolate the lower order bits. Since the bits we want are already in the correct position, we can just mask them out:
1111 1110 & 0xF = 0000 0000 0000 0000 0000 0000 0000 1110 (recall sign extension from before)
therefore: 1110 = 0xE in hex.
Like before, in Java we can do this all in one shot:
Character.forDigit((bytes[0] & 0xF), 16);
Putting this all together we can do it as a for loop and convert the entire array:
for(int i=0; i < bytes.length; i++){
buffer.append(Character.forDigit((bytes[i] >> 4) & 0xF, 16));
buffer.append(Character.forDigit((bytes[i] & 0xF), 16));
}
Hopefully this explanation makes things clearer for those of you wondering exactly what is going on in the many examples you will find on the internet. Hopefully I didn't make any egregious errors, but suggestions and corrections are highly welcome!
The fastest way i've yet found to do this is the following:
private static final String HEXES = "0123456789ABCDEF";
static String getHex(byte[] raw) {
final StringBuilder hex = new StringBuilder(2 * raw.length);
for (final byte b : raw) {
hex.append(HEXES.charAt((b & 0xF0) >> 4)).append(HEXES.charAt((b & 0x0F)));
}
return hex.toString();
}
It's ~ 50x faster than String.format. if you want to test it:
public class MyTest{
private static final String HEXES = "0123456789ABCDEF";
#Test
public void test_get_hex() {
byte[] raw = {
(byte) 0xd0, (byte) 0x0b, (byte) 0x01, (byte) 0x2a, (byte) 0x63,
(byte) 0x78, (byte) 0x01, (byte) 0x2e, (byte) 0xe3, (byte) 0x6c,
(byte) 0xd2, (byte) 0xb0, (byte) 0x78, (byte) 0x51, (byte) 0x73,
(byte) 0x34, (byte) 0xaf, (byte) 0xbb, (byte) 0xa0, (byte) 0x9f,
(byte) 0xc3, (byte) 0xa9, (byte) 0x00, (byte) 0x1e, (byte) 0xd5,
(byte) 0x4b, (byte) 0x89, (byte) 0xa3, (byte) 0x45, (byte) 0x35,
(byte) 0xd6, (byte) 0x10,
};
int N = 77777;
long t;
{
t = System.currentTimeMillis();
for (int i = 0; i < N; i++) {
final StringBuilder hex = new StringBuilder(2 * raw.length);
for (final byte b : raw) {
hex.append(HEXES.charAt((b & 0xF0) >> 4)).append(HEXES.charAt((b & 0x0F)));
}
hex.toString();
}
System.out.println(System.currentTimeMillis() - t); // 50
}
{
t = System.currentTimeMillis();
for (int i = 0; i < N; i++) {
StringBuilder hex = new StringBuilder(2 * raw.length);
for (byte b : raw) {
hex.append(String.format("%02X", b));
}
hex.toString();
}
System.out.println(System.currentTimeMillis() - t); // 2535
}
}
}
Edit: Just found something just a lil faster and that holds on one line but is not compatible with JRE 9. Use at your own risks
import javax.xml.bind.DatatypeConverter;
DatatypeConverter.printHexBinary(raw);
Try this way:
byte bv = 10;
String hexString = Integer.toHexString(bv);
Dealing with array (if I understood you correctly):
byte[] bytes = {9, 10, 11, 15, 16};
StringBuffer result = new StringBuffer();
for (byte b : bytes) {
result.append(String.format("%02X ", b));
result.append(" "); // delimiter
}
return result.toString();
As polygenelubricants mentioned, String.format() is the right answer compare to Integer.toHexString() (since it deals with negative numbers in a right way).
A short and simple way to convert byte[] to hex string by using BigInteger:
import java.math.BigInteger;
byte[] bytes = new byte[] {(byte)255, 10, 20, 30};
String hex = new BigInteger(1, bytes).toString(16);
System.out.println(hex); // ff0a141e
How It Works?
The built-in system class java.math.BigInteger class (java.math.BigInteger) is compatible with binary and hex data:
Its has a constructor BigInteger(signum=1, byte[]) to create a big integer by byte[] (set its first parameter signum = 1 to handle correctly the negative bytes)
Use BigInteger.toString(16) to convert the big integer to hex string
To parse a hex number use new BigInteger("ffa74b", 16) - does not handle correctly the leading zero
If you may want to have the leading zero in the hex result, check its size and add the missing zero if necessary:
if (hex.length() % 2 == 1)
hex = "0" + hex;
Notes
Use new BigInteger(1, bytes), instead of new BigInteger(bytes), because Java is "broken by design" and the byte data type does not hold bytes but signed tiny integers [-128...127]. If the first byte is negative, the BigInteger assumes you pass a negative big integer. Just pass 1 as first parameter (signum=1).
The conversion back from hex to byte[] is tricky: sometimes a leading zero enters in the produced output and it should be cleaned like this:
byte[] bytes = new BigInteger("ffa74b", 16).toByteArray();
if (bytes[0] == 0) {
byte[] newBytes = new byte[bytes.length - 1];
System.arraycopy(bytes, 1, newBytes, 0, newBytes.length);
bytes = newBytes;
}
The last note is the if the byte[] has several leading zeroes, they will be lost.
The Best solution is this badass one-liner:
String hex=DatatypeConverter.printHexBinary(byte[] b);
as mentioned here
If you want a constant-width hex representation, i.e. 0A instead of A, so that you can recover the bytes unambiguously, try format():
StringBuilder result = new StringBuilder();
for (byte bb : byteArray) {
result.append(String.format("%02X", bb));
}
return result.toString();
If you are happy to use an external library, the org.apache.commons.codec.binary.Hex class has an encodeHex method which takes a byte[] and returns a char[]. This methods is MUCH faster than the format option, and encapsulates the details of the conversion. Also comes with a decodeHex method for the opposite conversion.
You can use the method from Bouncy Castle Provider library:
org.bouncycastle.util.encoders.Hex.toHexString(byteArray);
The Bouncy Castle Crypto package is a Java implementation of
cryptographic algorithms. This jar contains JCE provider and
lightweight API for the Bouncy Castle Cryptography APIs for JDK 1.5 to
JDK 1.8.
Maven dependency:
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
<version>1.60</version>
</dependency>
or from Apache Commons Codec:
org.apache.commons.codec.binary.Hex.encodeHexString(byteArray);
The Apache Commons Codec package contains simple encoder and decoders
for various formats such as Base64 and Hexadecimal. In addition to
these widely used encoders and decoders, the codec package also
maintains a collection of phonetic encoding utilities.
Maven dependency:
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.11</version>
</dependency>
This is the code that I've found to run the fastest so far. I ran it on 109015 byte arrays of length 32, in 23ms. I was running it on a VM so it'll probably run faster on bare metal.
public static final char[] HEX_DIGITS = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F'};
public static char[] encodeHex( final byte[] data ){
final int l = data.length;
final char[] out = new char[l<<1];
for( int i=0,j=0; i<l; i++ ){
out[j++] = HEX_DIGITS[(0xF0 & data[i]) >>> 4];
out[j++] = HEX_DIGITS[0x0F & data[i]];
}
return out;
}
Then you can just do
String s = new String( encodeHex(myByteArray) );
Java 17: Introducing java.util.HexFormat
Java 17 comes with a utility to convert byte arrays and numbers to their hexadecimal counterparts. Let's say we have an MD5 digest of "Hello World" as a byte-array:
var md5 = MessageDigest.getInstance("md5");
md5.update("Hello world".getBytes(UTF_8));
var digest = md5.digest();
Now we can use the HexFormat.of().formatHex(byte[]) method to convert the given byte[] to its hexadecimal form:
jshell> HexFormat.of().formatHex(digest)
$7 ==> "3e25960a79dbc69b674cd4ec67a72c62"
The withUpperCase() method returns the uppercase version of the previous output:
jshell> HexFormat.of().withUpperCase().formatHex(digest)
$8 ==> "3E25960A79DBC69B674CD4EC67A72C62"
BigInteger n = new BigInteger(byteArray);
String hexa = n.toString(16);
Here is a simple function to convert byte to Hexadecimal
private static String convertToHex(byte[] data) {
StringBuffer buf = new StringBuffer();
for (int i = 0; i < data.length; i++) {
int halfbyte = (data[i] >>> 4) & 0x0F;
int two_halfs = 0;
do {
if ((0 <= halfbyte) && (halfbyte <= 9))
buf.append((char) ('0' + halfbyte));
else
buf.append((char) ('a' + (halfbyte - 10)));
halfbyte = data[i] & 0x0F;
} while(two_halfs++ < 1);
}
return buf.toString();
}
Others have covered the general case. But if you have a byte array of a known form, for example a MAC address, then you can:
byte[] mac = { (byte)0x00, (byte)0x00, (byte)0x00, (byte)0x00, (byte)0x00 };
String str = String.format("%02X:%02X:%02X:%02X:%02X:%02X",
mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
There's your fast method:
private static final String[] hexes = new String[]{
"00","01","02","03","04","05","06","07","08","09","0A","0B","0C","0D","0E","0F",
"10","11","12","13","14","15","16","17","18","19","1A","1B","1C","1D","1E","1F",
"20","21","22","23","24","25","26","27","28","29","2A","2B","2C","2D","2E","2F",
"30","31","32","33","34","35","36","37","38","39","3A","3B","3C","3D","3E","3F",
"40","41","42","43","44","45","46","47","48","49","4A","4B","4C","4D","4E","4F",
"50","51","52","53","54","55","56","57","58","59","5A","5B","5C","5D","5E","5F",
"60","61","62","63","64","65","66","67","68","69","6A","6B","6C","6D","6E","6F",
"70","71","72","73","74","75","76","77","78","79","7A","7B","7C","7D","7E","7F",
"80","81","82","83","84","85","86","87","88","89","8A","8B","8C","8D","8E","8F",
"90","91","92","93","94","95","96","97","98","99","9A","9B","9C","9D","9E","9F",
"A0","A1","A2","A3","A4","A5","A6","A7","A8","A9","AA","AB","AC","AD","AE","AF",
"B0","B1","B2","B3","B4","B5","B6","B7","B8","B9","BA","BB","BC","BD","BE","BF",
"C0","C1","C2","C3","C4","C5","C6","C7","C8","C9","CA","CB","CC","CD","CE","CF",
"D0","D1","D2","D3","D4","D5","D6","D7","D8","D9","DA","DB","DC","DD","DE","DF",
"E0","E1","E2","E3","E4","E5","E6","E7","E8","E9","EA","EB","EC","ED","EE","EF",
"F0","F1","F2","F3","F4","F5","F6","F7","F8","F9","FA","FB","FC","FD","FE","FF"
};
public static String byteToHex(byte b) {
return hexes[b&0xFF];
}
Just like some other answers, I recommend to use String.format() and BigInteger. But to interpret the byte array as big-endian binary representation instead of two's-complement binary representation (with signum and incomplete use of possible hex values range) use BigInteger(int signum, byte[] magnitude), not BigInteger(byte[] val).
For example, for a byte array of length 8 use:
String.format("%016X", new BigInteger(1,bytes))
Advantages:
leading zeros
no signum
only built-in functions
only one line of code
Disadvantage:
there might be more efficient ways to do that
Example:
byte[] bytes = new byte[8];
Random r = new Random();
System.out.println("big-endian | two's-complement");
System.out.println("-----------------|-----------------");
for (int i = 0; i < 10; i++) {
r.nextBytes(bytes);
System.out.print(String.format("%016X", new BigInteger(1,bytes)));
System.out.print(" | ");
System.out.print(String.format("%016X", new BigInteger(bytes)));
System.out.println();
}
Example output:
big-endian | two's-complement
-----------------|-----------------
3971B56BC7C80590 | 3971B56BC7C80590
64D3C133C86CCBDC | 64D3C133C86CCBDC
B232EFD5BC40FA61 | -4DCD102A43BF059F
CD350CC7DF7C9731 | -32CAF338208368CF
82CDC9ECC1BC8EED | -7D3236133E437113
F438C8C34911A7F5 | -BC7373CB6EE580B
5E99738BE6ACE798 | 5E99738BE6ACE798
A565FE5CE43AA8DD | -5A9A01A31BC55723
032EBA783D2E9A9F | 032EBA783D2E9A9F
8FDAA07263217ABA | -70255F8D9CDE8546
Creating (and destroying) a bunch of String instances is not a good way if performance is an issue.
Please ignore those verbose (duplicate) arguments checking statements (ifs). That's for (another) educational purposes.
Full maven project: http://jinahya.googlecode.com/svn/trunk/com.googlecode.jinahya/hex-codec/
Encoding...
/**
* Encodes a single nibble.
*
* #param decoded the nibble to encode.
*
* #return the encoded half octet.
*/
protected static int encodeHalf(final int decoded) {
switch (decoded) {
case 0x00:
case 0x01:
case 0x02:
case 0x03:
case 0x04:
case 0x05:
case 0x06:
case 0x07:
case 0x08:
case 0x09:
return decoded + 0x30; // 0x30('0') - 0x39('9')
case 0x0A:
case 0x0B:
case 0x0C:
case 0x0D:
case 0x0E:
case 0x0F:
return decoded + 0x57; // 0x41('a') - 0x46('f')
default:
throw new IllegalArgumentException("illegal half: " + decoded);
}
}
/**
* Encodes a single octet into two nibbles.
*
* #param decoded the octet to encode.
* #param encoded the array to which each encoded nibbles are written.
* #param offset the offset in the array.
*/
protected static void encodeSingle(final int decoded, final byte[] encoded,
final int offset) {
if (encoded == null) {
throw new IllegalArgumentException("null encoded");
}
if (encoded.length < 2) {
// not required
throw new IllegalArgumentException(
"encoded.length(" + encoded.length + ") < 2");
}
if (offset < 0) {
throw new IllegalArgumentException("offset(" + offset + ") < 0");
}
if (offset >= encoded.length - 1) {
throw new IllegalArgumentException(
"offset(" + offset + ") >= encoded.length(" + encoded.length
+ ") - 1");
}
encoded[offset] = (byte) encodeHalf((decoded >> 4) & 0x0F);
encoded[offset + 1] = (byte) encodeHalf(decoded & 0x0F);
}
/**
* Decodes given sequence of octets into a sequence of nibbles.
*
* #param decoded the octets to encode
*
* #return the encoded nibbles.
*/
protected static byte[] encodeMultiple(final byte[] decoded) {
if (decoded == null) {
throw new IllegalArgumentException("null decoded");
}
final byte[] encoded = new byte[decoded.length << 1];
int offset = 0;
for (int i = 0; i < decoded.length; i++) {
encodeSingle(decoded[i], encoded, offset);
offset += 2;
}
return encoded;
}
/**
* Encodes given sequence of octets into a sequence of nibbles.
*
* #param decoded the octets to encode.
*
* #return the encoded nibbles.
*/
public byte[] encode(final byte[] decoded) {
return encodeMultiple(decoded);
}
Decoding...
/**
* Decodes a single nibble.
*
* #param encoded the nibble to decode.
*
* #return the decoded half octet.
*/
protected static int decodeHalf(final int encoded) {
switch (encoded) {
case 0x30: // '0'
case 0x31: // '1'
case 0x32: // '2'
case 0x33: // '3'
case 0x34: // '4'
case 0x35: // '5'
case 0x36: // '6'
case 0x37: // '7'
case 0x38: // '8'
case 0x39: // '9'
return encoded - 0x30;
case 0x41: // 'A'
case 0x42: // 'B'
case 0x43: // 'C'
case 0x44: // 'D'
case 0x45: // 'E'
case 0x46: // 'F'
return encoded - 0x37;
case 0x61: // 'a'
case 0x62: // 'b'
case 0x63: // 'c'
case 0x64: // 'd'
case 0x65: // 'e'
case 0x66: // 'f'
return encoded - 0x57;
default:
throw new IllegalArgumentException("illegal half: " + encoded);
}
}
/**
* Decodes two nibbles into a single octet.
*
* #param encoded the nibble array.
* #param offset the offset in the array.
*
* #return decoded octet.
*/
protected static int decodeSingle(final byte[] encoded, final int offset) {
if (encoded == null) {
throw new IllegalArgumentException("null encoded");
}
if (encoded.length < 2) {
// not required
throw new IllegalArgumentException(
"encoded.length(" + encoded.length + ") < 2");
}
if (offset < 0) {
throw new IllegalArgumentException("offset(" + offset + ") < 0");
}
if (offset >= encoded.length - 1) {
throw new IllegalArgumentException(
"offset(" + offset + ") >= encoded.length(" + encoded.length
+ ") - 1");
}
return (decodeHalf(encoded[offset]) << 4)
| decodeHalf(encoded[offset + 1]);
}
/**
* Encodes given sequence of nibbles into a sequence of octets.
*
* #param encoded the nibbles to decode.
*
* #return the encoded octets.
*/
protected static byte[] decodeMultiple(final byte[] encoded) {
if (encoded == null) {
throw new IllegalArgumentException("null encoded");
}
if ((encoded.length & 0x01) == 0x01) {
throw new IllegalArgumentException(
"encoded.length(" + encoded.length + ") is not even");
}
final byte[] decoded = new byte[encoded.length >> 1];
int offset = 0;
for (int i = 0; i < decoded.length; i++) {
decoded[i] = (byte) decodeSingle(encoded, offset);
offset += 2;
}
return decoded;
}
/**
* Decodes given sequence of nibbles into a sequence of octets.
*
* #param encoded the nibbles to decode.
*
* #return the decoded octets.
*/
public byte[] decode(final byte[] encoded) {
return decodeMultiple(encoded);
}
This is a very fast way. No external libaries needed.
final protected static char[] HEXARRAY = "0123456789abcdef".toCharArray();
public static String encodeHexString( byte[] bytes ) {
char[] hexChars = new char[bytes.length * 2];
for (int j = 0; j < bytes.length; j++) {
int v = bytes[j] & 0xFF;
hexChars[j * 2] = HEXARRAY[v >>> 4];
hexChars[j * 2 + 1] = HEXARRAY[v & 0x0F];
}
return new String(hexChars);
}
I couldn't figure out what exactly you meant by byte String, but here are some conversions from byte to String and vice versa, of course there is a lot more on the official documentations
Integer intValue = 149;
The corresponding byte value is:
Byte byteValue = intValue.byteValue(); // this will convert the rightmost byte of the intValue to byte, because Byte is an 8 bit object and Integer is at least 16 bit, and it will give you a signed number in this case -107
get the integer value back from a Byte variable:
Integer anInt = byteValue.intValue(); // This will convert the byteValue variable to a signed Integer
From Byte and Integer to hex String:
This is the way I do it:
Integer anInt = 149
Byte aByte = anInt.byteValue();
String hexFromInt = "".format("0x%x", anInt); // This will output 0x95
String hexFromByte = "".format("0x%x", aByte); // This will output 0x95
Converting an array of bytes to a hex string:
As far as I know there is no simple function to convert all the elements inside an array of some Object to elements of another Object, So you have to do it yourself. You can use the following functions:
From byte[] to String:
public static String byteArrayToHexString(byte[] byteArray){
String hexString = "";
for(int i = 0; i < byteArray.length; i++){
String thisByte = "".format("%x", byteArray[i]);
hexString += thisByte;
}
return hexString;
}
And from hex string to byte[]:
public static byte[] hexStringToByteArray(String hexString){
byte[] bytes = new byte[hexString.length() / 2];
for(int i = 0; i < hexString.length(); i += 2){
String sub = hexString.substring(i, i + 2);
Integer intVal = Integer.parseInt(sub, 16);
bytes[i / 2] = intVal.byteValue();
String hex = "".format("0x%x", bytes[i / 2]);
}
return bytes;
}
It is too late but I hope this could help some others ;)
If you like streams, here is a single-expression version of the format-and-concatenate approach:
String hex = IntStream.range(0, bytes.length)
.map(i -> bytes[i] & 0xff)
.mapToObj(b -> String.format("%02x", b))
.collect(Collectors.joining());
It's a shame there isn't a method like Arrays::streamUnsignedBytes for this.
If you use Tink, then there is:
package com.google.crypto.tink.subtle;
public final class Hex {
public static String encode(final byte[] bytes) { ... }
public static byte[] decode(String hex) { ... }
}
so something like this should work:
import com.google.crypto.tink.subtle.Hex;
byte[] bytes = {-1, 0, 1, 2, 3 };
String enc = Hex.encode(bytes);
byte[] dec = Hex.decode(enc)
Just adding my two cents as I see many answers using a char array and/or using a StringBuilder instance and claiming to be fast or faster.
Since I had a different idea using the ASCII table organization having 0-9 at code points 48-57 and A-F at 65-70 and a-f at 97-102, I wanted to test which idea is the fastest.
Since I have not done something like that for quite some years now, I am giving this an extensive shoot out. I use 1 billion bytes in different sized arrays (1M, 1K, 10) so we have 1000 times 1M bytes per array, 1M times 1000 bytes per array and 100M times for 10 bytes per array.
Turns out the char array from 1-F wins. Using a char array as output instead of a StringBuilder also wins hands down (less objects, no test of capacity and no new array nor copying when growing). Furthermore you seem to get a small penalty when using a foreach (for(var b : bytes)) loop.
The version using my idea was approx. 15% slower for 1M bytes per array, 21% slower for 1000 bytes per array and 18% slower for 10 bytes per array. The StringBuilder version was 210%, 380% and 310% slower.
Bad but not that unexpected as a lookup of small arrays in first level cache beats one if and an add... . (one cache access + offset calculation vs. one if, one jump, one add -> not sure about the jump thou).
My version:
public static String bytesToHex(byte [] bytes) {
char [] result = new char [bytes.length * 2];
for(int index = 0; index < bytes.length; index++) {
int v = bytes[index];
int upper = (v >>> 4) & 0xF;
result[index * 2] = (char)(upper + (upper < 10 ? 48 : 65 - 10));
int lower = v & 0xF;
result[index * 2 + 1] = (char)(lower + (lower < 10 ? 48 : 65 - 10));
}
return new String(result);
}
PS: Yes I did multiple runs and took the best run each, did warm up and also did runs with 10 billion characters to be sure same picture... .
Use
Integer.toHexString((int)b);

Convert short to byte[] in Java

How can I convert a short (2 bytes) to a byte array in Java, e.g.
short x = 233;
byte[] ret = new byte[2];
...
it should be something like this. But not sure.
((0xFF << 8) & x) >> 0;
EDIT:
Also you can use:
java.nio.ByteOrder.nativeOrder();
To discover to get whether the native bit order is big or small. In addition the following code is taken from java.io.Bits which does:
byte (array/offset) to boolean
byte array to char
byte array to short
byte array to int
byte array to float
byte array to long
byte array to double
And visa versa.
ret[0] = (byte)(x & 0xff);
ret[1] = (byte)((x >> 8) & 0xff);
A cleaner, albeit far less efficient solution is:
ByteBuffer buffer = ByteBuffer.allocate(2);
buffer.putShort(value);
return buffer.array();
Keep this in mind when you have to do more complex byte transformations in the future. ByteBuffers are very powerful.
An alternative that is more efficient:
// Little Endian
ret[0] = (byte) x;
ret[1] = (byte) (x >> 8);
// Big Endian
ret[0] = (byte) (x >> 8);
ret[1] = (byte) x;
Figured it out, its:
public static byte[] toBytes(short s) {
return new byte[]{(byte)(s & 0x00FF),(byte)((s & 0xFF00)>>8)};
}
Short to bytes convert method In Kotlin works for me:
fun toBytes(s: Short): ByteArray {
return byteArrayOf((s.toInt() and 0x00FF).toByte(), ((s.toInt() and 0xFF00) shr (8)).toByte())
}
Several methods have been mentioned here. But which one is the best? Here follows some proof that the following 3 approaches result in the same output for all values of a short
// loops through all the values of a Short
short i = Short.MIN_VALUE;
do
{
// method 1: A SIMPLE SHIFT
byte a1 = (byte) (i >> 8);
byte a2 = (byte) i;
// method 2: AN UNSIGNED SHIFT
byte b1 = (byte) (i >>> 8);
byte b2 = (byte) i;
// method 3: SHIFT AND MASK
byte c1 = (byte) (i >> 8 & 0xFF);
byte c2 = (byte) (i & 0xFF);
if (a1 != b1 || a1 != c1 ||
a2 != b2 || a2 != c2)
{
// this point is never reached !!
}
} while (i++ != Short.MAX_VALUE);
Conclusion: less is more ?
byte b1 = (byte) (s >> 8);
byte b2 = (byte) s;
(As other answers have mentioned, watch out for LE/BE).
It depends how you want to represent it:
big endian or little endian? That will determine which order you put the bytes in.
Do you want to use 2's complement or some other way of representing a negative number? You should use a scheme that has the same range as the short in java to have a 1-to-1 mapping.
For big endian, the transformation should be along the lines of:
ret[0] = x/256; ret[1] = x%256;
public short bytesToShort(byte[] bytes) {
return ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN).getShort();
}
public byte[] shortToBytes(short value) {
byte[] returnByteArray = new byte[2];
returnByteArray[0] = (byte) (value & 0xff);
returnByteArray[1] = (byte) ((value >>> 8) & 0xff);
return returnByteArray;
}
short to byte
short x=17000;
byte res[]=new byte[2];
res[i]= (byte)(((short)(x>>7)) & ((short)0x7f) | 0x80 );
res[i+1]= (byte)((x & ((short)0x7f)));
byte to short
short x=(short)(128*((byte)(res[i] &(byte)0x7f))+res[i+1]);

How do I split an integer into 2 byte binary?

Given
private int width = 400;
private byte [] data = new byte [2];
I want to split the integer "width" into two bytes and load data[0] with the high byte and data[1] with the low byte.
That is binary value of 400 = 1 1001 0000
so data[0] should contain 0000 0001
and data[1] should contain 1001 0000
Using simple bitwise operations:
data[0] = (byte) (width & 0xFF);
data[1] = (byte) ((width >> 8) & 0xFF);
How it works:
& 0xFF masks all but the lowest eight bits.
>> 8 discards the lowest 8 bits by moving all bits 8 places to the right.
The cast to byte is necessary because these bitwise operations work on an int and return an int, which is a bigger data type than byte. The case is safe, since all non-zero bits will fit in the byte. For more information, see Conversions and Promotions.
Edit: Taylor L correctly remarks that though >> works in this case, it may yield incorrect results if you generalize this code to four bytes (since in Java an int is 32 bits). In that case, it's better to use >>> instead of >>. For more information, see the Java tutorial on Bitwise and Bit Shift Operators.
For converting two bytes the cleanest solution is
data[0] = (byte) width;
data[1] = (byte) (width >>> 8);
For converting an integer to four bytes the code would be
data[0] = (byte) width;
data[1] = (byte) (width >>> 8);
data[2] = (byte) (width >>> 16);
data[3] = (byte) (width >>> 24);
It doesn't matter whether >> or >>> is used for shifting, any one bits created by sign extension will not end up in the resulting bytes.
See also this answer.
This should do what you want for a 4 byte int. Note, it stores the low byte at offset 0. I'll leave it as an exercise to the reader to order them as needed.
public static byte[] intToBytes(int x) {
byte[] bytes = new byte[4];
for (int i = 0; x != 0; i++, x >>>= 8) {
bytes[i] = (byte) (x & 0xFF);
}
return bytes;
}
Integer is 32 bits (=4 bytes) in java, you know?
width & 0xff will give you the first byte,
width & 0xff00 >> 8 will give you the second, etc.
To get the high byte, shift right by 8 bits then mask off the top bytes. Similarly, to get the low byte just mask off the top bytes.
data[0] = (width >> 8) & 0xff;
data[1] = width & 0xff;
int width = 400;
byte [] data = new byte [2];
data[0] = (byte) ((width & 0xFF00) >> 8);
data[1] = (byte) (width & 0xFF);
for(int b = 0; b < 2; b++) {
System.out.println("printing byte " + b);
for(int i = 7; i >= 0; i--) {
System.out.println(data[b] & 1);
data[b] = (byte) (data[b] >> 1);
}
}
I suggest you have a look at the source for HeapByteBuffer. It has the conversion code for all primitive data types. (In fact you could just use a ByteBuffer ;)

Categories

Resources