Develop Reference

Java: convert byte[] to base36 String

I'm a bit lost. For a project, I need to convert the output of a hash-function (SHA256) - which is a byte array - to a String using base 36.
So In the end, I want to convert the (Hex-String representation of the) Hash, which is
43A718774C572BD8A25ADBEB1BFCD5C0256AE11CECF9F9C3F925D0E52BEAF89
to base36, so the example String from above would be:
3SKVHQTXPXTEINB0AT1P0G45M4KI8U0HR8PGB96DVXSTDJKI1
For the actual conversion to base36, I found some piece of code here on StackOverflow:
public static String toBase36(byte[] bytes) {
//can provide a (byte[], offset, length) method too
StringBuffer sb = new StringBuffer();
int bitsUsed = 0; //will point how many bits from the int are to be encoded
int temp = 0;
int tempBits = 0;
long swap;
int position = 0;
while((position < bytes.length) || (bitsUsed != 0)) {
swap = 0;
if(tempBits > 0) {
//there are bits left over from previous iteration
swap = temp;
bitsUsed = tempBits;
tempBits = 0;
}
//fill some bytes
while((position < bytes.length) && (bitsUsed < 36)) {
swap <<= 8;
swap |= bytes[position++];
bitsUsed += 8;
}
if(bitsUsed > 36) {
tempBits = bitsUsed - 36; //this is always 4
temp = (int)(swap & ((1 << tempBits) - 1)); //get low bits
swap >>= tempBits; //remove low bits
bitsUsed = 36;
}
sb.append(Long.toString(swap, 36));
bitsUsed = 0;
}
return sb.toString();
}
Now I'm doing this:
// this creates my hash, being a 256-bit byte array
byte[] hash = PBKDF2.deriveKey(key.getBytes(), salt.getBytes(), 2, 256);
System.out.println(hash.length); // outputs "256"
System.out.println(toBase36(hash)); // outputs total crap
the "total crap" is something like
-7-14-8-1q-5se81u0e-3-2v-24obre-73664-7-5-5cor1o9s-6h-4k6hr-5-4-rt2z0-30-8-2u-8-onz-4a2j-6-8-18-8trzza3-3-2x-6-4153to-4e3l01me-6-azz-2-k-4ckq-nav-gu-irqpxx-el-1j-6-rmf8hs-1bb5ax-3z25u-2-2r-t5-22-6-6w1v-1p
so it's not even close to what I want. I tried to find a solution now, but it seems I'm a bit lost here. How do I get the base36-encoded String representation of the Hash that I need?

Try using BigInteger:
String hash = "43A718774C572BD8A25ADBEB1BFCD5C0256AE11CECF9F9C3F925D0E52BEAF89";
//use a radix of 16, default would be 10
String base36 = new BigInteger( hash, 16 ).toString( 36 ).toUpperCase();

This might work:
BigInteger big = new BigInteger(your_byte_array_to_hex_string, 16);
big.toString(36);

How do I convert a large binary String to byte array java?

I have a large binary string "101101110...", and I am trying to store it into a byte array. what is the best way of doing it?
Lets say I have largeString = "0100111010111011011000000001000110101"
Result that I'm looking for:
[78, 187, 96, 17, 21]
01001110 10111011 01100000 00010001 10101
What i've tried:
byte[] b= new BigInteger(largeString,2).toByteArray();
however it did not give me the result I'm looking for...

You can easily build an ArrayList on which you can call toArray if you want an actual array;
List<Integer> list = new ArrayList<>();
for(String str : largeString.split("(?<=\\G.{8})"))
list.add(Integer.parseInt(str, 2));
System.out.println(list); // Outputs [78, 187, 96, 17, 21]

Assuming that your binary string module 8 equals 0 binString.lenght()%8==0
/**
* Get an byte array by binary string
* #param binaryString the string representing a byte
* #return an byte array
*/
public static byte[] getByteByString(String binaryString) {
int splitSize = 8;
if(binaryString.length() % splitSize == 0){
int index = 0;
int position = 0;
byte[] resultByteArray = new byte[binaryString.length()/splitSize];
StringBuilder text = new StringBuilder(binaryString);
while (index < text.length()) {
String binaryStringChunk = text.substring(index, Math.min(index + splitSize, text.length()));
Integer byteAsInt = Integer.parseInt(binaryStringChunk, 2);
resultByteArray[position] = byteAsInt.byteValue();
index += splitSize;
position ++;
}
return resultByteArray;
}
else{
System.out.println("Cannot convert binary string to byte[], because of the input length. '" +binaryString+"' % 8 != 0");
return null;
}
}

Do it in a loop. Split the string at 8-character chunks and convert them separately. In "pseudocode" it's something like:
byte[] result = new byte[subs.size()];
int i = 0;
int j = 0;
while(i+8 <= s.length){
result[j] = new Byte.valueOf(largeString.substring(i, i+8), 2);
i+=8;
j++;
}
result[j] = new Byte.valueOf(largeString.substring(i, largeString.length));

DES Send and Receive Modes for DESFire Authentication

I'm trying to authenticate DESFire card with my android application. I use the example in this link to decypher the bytes I got from the card. For that, I ruled out padding in decryption (commented out below), because DESFire documentation points it out. Also, if I don't do so, decryption returns 7 bytes for input of 8 bytes. Below are DES and TripleDES decryption functions I use:
public static byte[] TripleDES_Decrypt(byte[] data,byte[][] keys)
{
int i;
byte[] tmp = new byte[data.length];
byte[] bloc = new byte[8];
K = generateSubKeys(keys[0]);
K1 = generateSubKeys(keys[1]);
K2 = generateSubKeys(keys[2]);
for (i = 0; i < data.length; i++) {
if (i > 0 && i % 8 == 0) {
bloc = encrypt64Bloc(bloc,K2, true);
bloc = encrypt64Bloc(bloc,K1, false);
bloc = encrypt64Bloc(bloc,K, true);
System.arraycopy(bloc, 0, tmp, i - 8, bloc.length);
}
if (i < data.length)
bloc[i % 8] = data[i];
}
bloc = encrypt64Bloc(bloc,K2, true);
bloc = encrypt64Bloc(bloc,K1, false);
bloc = encrypt64Bloc(bloc,K, true);
System.arraycopy(bloc, 0, tmp, i - 8, bloc.length);
//tmp = deletePadding(tmp);
return tmp;
}
public static byte[] decrypt(byte[] data, byte[] key) {
int i;
byte[] tmp = new byte[data.length];
byte[] bloc = new byte[8];
K = generateSubKeys(key);
for (i = 0; i < data.length; i++) {
if (i > 0 && i % 8 == 0) {
bloc = encrypt64Bloc(bloc,K, true);
System.arraycopy(bloc, 0, tmp, i - 8, bloc.length);
}
if (i < data.length)
bloc[i % 8] = data[i];
}
bloc = encrypt64Bloc(bloc,K, true);
System.arraycopy(bloc, 0, tmp, i - 8, bloc.length);
//tmp = deletePadding(tmp);
return tmp;
}
According to DesFire document, I need two modes of decryption, send and receive. This blog post has some explanation about it.
However, the DESFire crypto is a bit different from the normal DES/CBC scheme: The PCD uses DES “send mode” when sending data (xor before DES), and the card uses DES “recieve mode” when recieving data (xor after DES). But when the PCD recieves data, it uses normal DES/CBC mode (xor after DES), and the card uses normal DES send mode when sending data (xor before DES).
And in Android side I follow the examples and recommendations:
// connected to tag and application
// result = encoded(randB) + af
byte[] result = idTag.transceive(Utils.wrapMessage((byte)0x0a, new byte[]{(byte)0x0}));
byte[] b0 = new byte[8];
for(int i = 0; i < 8; i++) {
b0[i] = result[i];
}
// key
byte[] key = new byte[] {(byte)0x0,(byte)0x0,(byte)0x0,(byte)0x0,
(byte)0x0,(byte)0x0,(byte)0x0,(byte)0x0,
(byte)0x0,(byte)0x0,(byte)0x0,(byte)0x0,
(byte)0x0,(byte)0x0,(byte)0x0,(byte)0x0 };
byte[][] keys = new byte[3][];
keys[0]=key; keys[1]=key; keys[2]=key;
// decrypt encoded(randB)
byte[] r0 = DES.TripleDES_Decrypt(b0, keys);
// generate randA (integer 0-7 for trying)
byte[] nr = new byte[8];
for(int i = 0; i < 8; i++) {
nr[i] = Byte.parseByte(Integer.toString(i), 16);
}
// decrypt randA
byte[] b1 = DES.TripleDES_Decrypt(nr, keys);
// shift randB and get randB'
byte[] r1 =new byte[8];
for(int i = 0; i < 7; i++) {
r1[i] = r0[i + 1];
}
r1[7]=r0[0];
// concat (randA + randB')
byte[] b2 = new byte[16];
for(int i = 0; i < 16; i++)
{
if(i <= 7) {
b2[i] = b1[i];
} else {
b2[i] = r1[i - 8];
}
}
// XOR (randA + randB') with IV
// IV is told to be consisting of 0's,
// but XOR something with 0 results the same?
for(int i=0;i<16;i++) {
b2[i] = (byte) (b2[i] ^ (byte)0x0);
}
// send AF and decrypt(A+B)
// wrap message adds needed wrapping to message (90 to left, offset bytes etc.)
result = isodepTag.transceive(Utils.wrapMessage((byte)0xaf, DES.TripleDES_Decrypt(b2, keys)));
I get the first result, the encrypted randB. However, the second "result" is always "91ae", means authentication error. I'm doing something wrong here, send wrong data to card.
Can anyone tell me what must I change in the code to work in these modes? What should I XOR with data before/after TripleDES?
Not the real question, but I read that default "Key" in DesFire card is 16 zero bytes. Also the document points that I need to use TripleDES for 16 bytes of key, DES for 8 bytes of key. So I'm using and need to use TripleDES as I haven't changed the default key, am I right?
For those who need the know about CipherBlockChaining.
EDIT: I found out that I need to do XORing before and after TripleDES and I mustn't touch TripleDES's internal operations at all. I will be trying that in a while.
Deleted the inner TripleDES lines, just saying for the ones seeing the question for the first time.

OK I got the solution. My mistake was that I was sending
3DES(randA + randB')
But I should send
3DES(randA) + 3DES(randB' XOR 3DES(randA))
Here's the authentication code for Android/Java (it's so sad that this is the only one that can be found on the net currently!):
The actual authentication code:
// send initial authentication request
byte[] result = idTag.transceive(Utils.wrapMessage((byte)0x0a, new byte[]{(byte)0x0}));
// get encrypted(randB) from the response
byte[] b0 = new byte[8];
for(int i = 0; i < 8; i++) {
b0[i] = result[i];
}
// 16 bytes default key
byte[] key = new byte[] {(byte)0x0,(byte)0x0,(byte)0x0,(byte)0x0,
(byte)0x0,(byte)0x0,(byte)0x0,(byte)0x0,
(byte)0x0,(byte)0x0,(byte)0x0,(byte)0x0,
(byte)0x0,(byte)0x0,(byte)0x0,(byte)0x0 };
// keys for TripleDes
byte[][] keys = new byte[3][];
keys[0] = key; keys[1] = key; keys[2] = key;
// decrypt encoded(randB)
byte[] r0 = DES.TripleDES_Decrypt(b0, keys);
// generate randA (integer 0-7 for trying, should randomize for real-life use)
byte[] nr = new byte[8];
for(int i = 0; i < 8; i++) {
nr[i] = Byte.parseByte(Integer.toString(i), 16);
}
// decrypt randA, should XOR with IV, but IV is all 0's, not necessary
byte[] b1 = DES.TripleDES_Decrypt(nr, keys);
// shift randB one byte left and get randB'
byte[] r1 =new byte[8];
for(int i = 0; i < 7; i++) {
r1[i] = r0[i + 1];
}
r1[7]=r0[0];
// xor randB' with randA and decrypt
byte[] b2 = new byte[8];
for(int i = 0; i < 8; i++) {
b2[i] = (byte) (b1[i] ^ r1[i]);
}
b2 = DES.TripleDES_Decrypt(b2, keys);
// concat (randA + randB')
byte[] b1b2 = new byte[16];
for (int i = 0; i < b1b2.length; i++) {
if(i <= 7) {
b1b2[i] = b1[i];
} else {
b1b2[i]=b2[i-8];
}
}
result = idTag.transceive(Utils.wrapMessage((byte)0xaf, b1b2));
TripleDes is the one in the question. wrapMessage function:
public static byte[] wrapMessage (byte command, byte[] parameters) throws Exception {
ByteArrayOutputStream stream = new ByteArrayOutputStream();
stream.write((byte) 0x90);
stream.write(command);
stream.write((byte) 0x00);
stream.write((byte) 0x00);
if (parameters != null) {
stream.write((byte) parameters.length);
stream.write(parameters);
}
stream.write((byte) 0x00);
return stream.toByteArray();
}
EDIT: Thanks to VGe0rge, we found out the reason why this authentication doesn't work from time to time. Instead of calling the 3DES function in the question, just call:
Cipher.getInstance("DESede/CBC/NoPadding");

Ismat, did you try to write data to the DESFIRE files?
As you explained, when sending data to the card: The PCD uses DES “send mode” when sending data (xor before DES), and the card uses DES “recieve mode” when recieving data (xor after DES)
So I can't get the proper code in order to implement the TDES with the XOR before..I need to do all the crypt-decrypt-crypt and the process is to slow for my application:
res = criptoTransformDec.TransformBlock(datosEscribir, 0, 8, datosEscribir, 0);
res = criptoTransformEnc.TransformBlock(datosEscribir, 0, 8, datosEscribir, 0);
res = criptoTransformDec1.TransformBlock(datosEscribir, 0, 8, datosEscribir, 0);
int l_iAux = 0;
while (l_iAux < (datosEscribir.Length - 8))
{
criptoTransformDec2 = desDec.CreateDecryptor(claveSes1, tdesInitialVector);
//desEnc2 = new DESCryptoServiceProvider();
criptoTransformEnc2 = desEnc.CreateEncryptor(claveSes2, tdesInitialVector);
//desDec3 = new DESCryptoServiceProvider();
criptoTransformDec3 = desDec.CreateDecryptor(claveSes1, tdesInitialVector);
Array.Copy(datosEscribir, 8 + l_iAux, aux1, 0, 8);
Array.Copy(datosEscribir, l_iAux, aux2, 0, 8);
DesfireBarik.XorStr(ref aux1, ref aux2, 8);
res = criptoTransformDec2.TransformBlock(aux1, 0, 8, datosEscribir, 8 + l_iAux);
res = criptoTransformEnc2.TransformBlock(datosEscribir, 8 + l_iAux, 8, datosEscribir, 8 + l_iAux);
res = criptoTransformDec3.TransformBlock(datosEscribir, 8 + l_iAux, 8, datosEscribir, 8 + l_iAux);
l_iAux += 8;
}
private static void XorStr (ref byte[] str1, ref byte[] str2, int qty )
{
int i = 0;
for (i = 0; i < qty; i++ )
str1[i] = (byte)(str1[i] ^ str2[i]);
}

extract ID from byte[]

I read byte[] from socket as Param_Code which includes ID.
byte[] cbuf = new byte[4];
socketReader.read(cbuf, 0, 4);
int Param_Code = byteArrayToIntBI(cbuf, 0);
public static int byteArrayToIntBI(byte[] b, int offset) {
int value = 0;
for (int i = 3; i > -1; i--) {
int shift = (i) * 8;
value += (b[i + offset] & 0x000000FF) << shift;
}
return value;
}
• Option 1: Sometimes the Param_code corresponds only to the ID
• Option 2: Sometimes the Param_code corresponds to 0x40000000 + the ID
• Option 3: Sometimes the Param_code corresponds to 0x80000000 + the ID
My question is, How can i get ID from Param_code considering above options.

How about using ByteBuffer?
byte[] cbuf = new byte[4];
socketReader.read(cbuf, 0, 4);
ByteBuffer buffer = ByteBuffer.wrap(cbuf);
int Param_Code = buffer.getInt();

Store binary sequence in byte array?

I need to store a couple binary sequences that are 16 bits in length into a byte array (of length 2). The one or two binary numbers don't change, so a function that does conversion might be overkill. Say for example the 16 bit binary sequence is 1111000011110001. How do I store that in a byte array of length two?

String val = "1111000011110001";
byte[] bval = new BigInteger(val, 2).toByteArray();
There are other options, but I found it best to use BigInteger class, that has conversion to byte array, for this kind of problems. I prefer if, because I can instantiate class from String, that can represent various bases like 8, 16, etc. and also output it as such.
Edit: Mondays ... :P
public static byte[] getRoger(String val) throws NumberFormatException,
NullPointerException {
byte[] result = new byte[2];
byte[] holder = new BigInteger(val, 2).toByteArray();
if (holder.length == 1) result[0] = holder[0];
else if (holder.length > 1) {
result[1] = holder[holder.length - 2];
result[0] = holder[holder.length - 1];
}
return result;
}
Example:
int bitarray = 12321;
String val = Integer.toString(bitarray, 2);
System.out.println(new StringBuilder().append(bitarray).append(':').append(val)
.append(':').append(Arrays.toString(getRoger(val))).append('\n'));

I have been disappointed with all of the solutions I have found to converting strings of bits to byte arrays and vice versa -- all have been buggy (even the BigInteger solution above), and very few are as efficient as they should be.
I realize the OP was only concerned with a bit string to an array of two bytes, which the BitInteger approach seems to work fine for. However, since this post is currently the first search result when searching "bit string to byte array java" in Google, I am going to post my general solution here for people dealing with huge strings and/or huge byte arrays.
Note that my solution below is the only solution I have ran that passes all of my test cases -- many online solutions to this relatively simple problem simply do not work.
Code
/**
* Zips (compresses) bit strings to byte arrays and unzips (decompresses)
* byte arrays to bit strings.
*
* #author ryan
*
*/
public class BitZip {
private static final byte[] BIT_MASKS = new byte[] {1, 2, 4, 8, 16, 32, 64, -128};
private static final int BITS_PER_BYTE = 8;
private static final int MAX_BIT_INDEX_IN_BYTE = BITS_PER_BYTE - 1;
/**
* Decompress the specified byte array to a string.
* <p>
* This function does not pad with zeros for any bit-string result
* with a length indivisible by 8.
*
* #param bytes The bytes to convert into a string of bits, with byte[0]
* consisting of the least significant bits in the byte array.
* #return The string of bits representing the byte array.
*/
public static final String unzip(final byte[] bytes) {
int byteCount = bytes.length;
int bitCount = byteCount * BITS_PER_BYTE;
char[] bits = new char[bitCount];
{
int bytesIndex = 0;
int iLeft = Math.max(bitCount - BITS_PER_BYTE, 0);
while (bytesIndex < byteCount) {
byte value = bytes[bytesIndex];
for (int b = MAX_BIT_INDEX_IN_BYTE; b >= 0; --b) {
bits[iLeft + b] = ((value % 2) == 0 ? '0' : '1');
value >>= 1;
}
iLeft = Math.max(iLeft - BITS_PER_BYTE, 0);
++bytesIndex;
}
}
return new String(bits).replaceFirst("^0+(?!$)", "");
}
/**
* Compresses the specified bit string to a byte array, ignoring trailing
* zeros past the most significant set bit.
*
* #param bits The string of bits (composed strictly of '0' and '1' characters)
* to convert into an array of bytes.
* #return The bits, as a byte array with byte[0] containing the least
* significant bits.
*/
public static final byte[] zip(final String bits) {
if ((bits == null) || bits.isEmpty()) {
// No observations -- return nothing.
return new byte[0];
}
char[] bitChars = bits.toCharArray();
int bitCount = bitChars.length;
int left;
for (left = 0; left < bitCount; ++left) {
// Ignore leading zeros.
if (bitChars[left] == '1') {
break;
}
}
if (bitCount == left) {
// Only '0's in the string.
return new byte[] {0};
}
int cBits = bitCount - left;
byte[] bytes = new byte[((cBits) / BITS_PER_BYTE) + (((cBits % BITS_PER_BYTE) > 0) ? 1 : 0)];
{
int iRight = bitCount - 1;
int iLeft = Math.max(bitCount - BITS_PER_BYTE, left);
int bytesIndex = 0;
byte _byte = 0;
while (bytesIndex < bytes.length) {
while (iLeft <= iRight) {
if (bitChars[iLeft] == '1') {
_byte |= BIT_MASKS[iRight - iLeft];
}
++iLeft;
}
bytes[bytesIndex++] = _byte;
iRight = Math.max(iRight - BITS_PER_BYTE, left);
iLeft = Math.max((1 + iRight) - BITS_PER_BYTE, left);
_byte = 0;
}
}
return bytes;
}
}
Performance
I was bored at work so I did some performance testing comparing against the accepted answer here for when N is large. (Pretending to ignore the fact that the BigInteger approach posted above doesn't even work properly as a general approach.)
This is running with a random bit string of size 5M and a random byte array of size 1M:
String -> byte[] -- BigInteger result: 39098ms
String -> byte[] -- BitZip result: 29ms
byte[] -> String -- Integer result: 138ms
byte[] -> String -- BitZip result: 71ms
And the code:
public static void main(String[] argv) {
int testByteLength = 1000000;
int testStringLength = 5000000;
// Independently random.
final byte[] randomBytes = new byte[testByteLength];
final String randomBitString;
{
StringBuilder sb = new StringBuilder();
Random rand = new Random();
for (int i = 0; i < testStringLength; ++i) {
int value = rand.nextInt(1 + i);
sb.append((value % 2) == 0 ? '0' : '1');
randomBytes[i % testByteLength] = (byte) value;
}
randomBitString = sb.toString();
}
byte[] resultCompress;
String resultDecompress;
{
Stopwatch s = new Stopwatch();
TimeUnit ms = TimeUnit.MILLISECONDS;
{
s.start();
{
resultCompress = compressFromBigIntegerToByteArray(randomBitString);
}
s.stop();
{
System.out.println("String -> byte[] -- BigInteger result: " + s.elapsed(ms) + "ms");
}
s.reset();
}
{
s.start();
{
resultCompress = zip(randomBitString);
}
s.stop();
{
System.out.println("String -> byte[] -- BitZip result: " + s.elapsed(ms) + "ms");
}
s.reset();
}
{
s.start();
{
resultDecompress = decompressFromIntegerParseInt(randomBytes);
}
s.stop();
{
System.out.println("byte[] -> String -- Integer result: " + s.elapsed(ms) + "ms");
}
s.reset();
}
{
s.start();
{
resultDecompress = unzip(randomBytes);
}
s.stop();
{
System.out.println("byte[] -> String -- BitZip result: " + s.elapsed(ms) + "ms");
}
s.reset();
}
}
}