Develop Reference

Convert hex to INT 32 and STRING Little Endian

First question:
Hex: F1620000
After convert hex to INT 32 i expect 253229, but i get -245235712.
I've tried these methods:
Integer.parseUnsignedInt(value, 16));
(int)Long.parseLong(value, 16));
new BigInteger(value, 16).intValue());
How i can get correct value?
Second question:
Hex: 9785908D9B9885828020912E208D2E
After covert hex to STRING i can get this value:
\u0097\u0085\u0090\u008d\u009b\u0098\u0085\u0082\u0080 \u0091. \u008d.
How can I display this value correctly in json? (usning JSONObject).
StringBuilder result = new StringBuilder();
for (int i = 0; i < value.length(); i += 2) {
String str = value.substring(i, i + 2);
result.append((char)Integer.parseInt(str, 16));
}

All your attempts are sufficient for parsing a hexadecimal string in an unsigned interpretation, but did not account for the “little endian” representation. You have to reverse the bytecode manually:
String value = "F1620000";
int i = Integer.reverseBytes(Integer.parseUnsignedInt(value, 16));
System.out.println(i);
25329
For your second task, the missing information was how to interpret the bytes to get to the character content. After searching a bit, the Codepage 866 seems to be the most plausible encoding:
String value = "9785908D9B9885828020912E208D2E";
byte[] bytes = new BigInteger(value, 16).toByteArray();
String result = new String(bytes, bytes[0]==0? 1: 0, value.length()/2, "CP866");
ЧЕРНЫШЕВА С. Н.

Encoding byte array to Hex string

I have come across a legacy piece of code encoding byte array to hex string which is in production and have never caused an issue.
This piece of code is used as:
We encrypt a user password. The encryptor returns a byte[].
We convert the byte[] to Hex String using this encoder code and then use that String representation in our properties file and so on.
However, yesterday we have hit a password, whose encrypted byte[] version is getting encoded incorrectly.
import java.math.BigInteger;
import java.util.HashMap;
import org.apache.commons.codec.DecoderException;
import org.apache.commons.codec.binary.Hex;
public class ByteArrayToHexEncoder {
public static void main(String[] args) throws DecoderException {
String hexString = "a0d21588c0a2c2fc68dc859197fc78cd"; // correct hex representation
// equivalent byte array: this is the byte array returned by the encryptor
byte[] byteArray = Hex.decodeHex(hexString.toCharArray());
// legacy encoder
System.out.println("Legacy code encodes as: " + encodeHexBytesWithPadding(byteArray));
// commons-codec encoder
System.out.println("Commons codec encode as: " + new String(Hex.encodeHex(byteArray)));
}
private static final String PADDING_ZEROS =
"0000000000000000000000000000000000000000000000000000000000000";
private static final HashMap<Integer, Character> MAP_OF_HEX = new HashMap<>();
static {
MAP_OF_HEX.put(0, '0');
MAP_OF_HEX.put(1, '1');
MAP_OF_HEX.put(2, '2');
MAP_OF_HEX.put(3, '3');
MAP_OF_HEX.put(4, '4');
MAP_OF_HEX.put(5, '5');
MAP_OF_HEX.put(6, '6');
MAP_OF_HEX.put(7, '7');
MAP_OF_HEX.put(8, '8');
MAP_OF_HEX.put(9, '9');
MAP_OF_HEX.put(10, 'a');
MAP_OF_HEX.put(11, 'b');
MAP_OF_HEX.put(12, 'c');
MAP_OF_HEX.put(13, 'd');
MAP_OF_HEX.put(14, 'e');
MAP_OF_HEX.put(15, 'f');
}
public static String encodeHexBytesWithPadding(byte[] inputByteArray) {
String encodedValue = encodeHexBytes(inputByteArray);
int expectedSize = inputByteArray.length * 2;
if (encodedValue.length() < expectedSize) {
int zerosToPad = expectedSize - encodedValue.length();
encodedValue = PADDING_ZEROS.substring(0, zerosToPad) + encodedValue;
}
return encodedValue;
}
public static String encodeHexBytes(byte[] inputByteArray) {
String encodedValue;
if (inputByteArray[0] < 0) {
// Something is wrong here! Don't know what!
byte oldValue = inputByteArray[0];
inputByteArray[0] = (byte) (oldValue & 0x0F);
int nibble = (oldValue >> 4) & 0x0F;
encodedValue = new BigInteger(inputByteArray).toString(16);
inputByteArray[0] = oldValue;
encodedValue = MAP_OF_HEX.get(nibble) + encodedValue;
} else {
encodedValue = new BigInteger(inputByteArray).toString(16);
}
return encodedValue;
}
}
The legacy code outputs the encoded value as: 0ad21588c0a2c2fc68dc859197fc78cd while the correct expected value should be: a0d21588c0a2c2fc68dc859197fc78cd.
I am trying to understand what's wrong with the encoder and need some help understanding.

BigInteger(byte[]) constructor is there to handle two's complement representation of a number where the most significant bit also denotes the sign. The Hex common-codec simply translates each byte into a hex representation, there is no special meaning to the most significant bit.
Your legacy code in the if (inputByteArray[0] < 0) branch attempts to modify the first byte in the byte[] input probably to work around the representation of negative numbers in the two-complement's form e.g. -1 being represented as ff. Unfortunately this is implemented incorrectly in your legacy code:
String input = "a000000001";
byte[] bytes = Hex.decodeHex(input.toCharArray());
System.out.println(encodeHexBytesWithPadding(bytes));
System.out.println(Hex.encodeHexString(bytes));
will print
00000000a1
a000000001
showing that the legacy code values are completely wrong.
There is not much to salvage here IMO, instead use Hex.encodeHexString() instead or check other options discussed in this question.

Android's string hash doesn't match serverisde's

I'm developing an Android app and I need to send some data from server to Android device.
To prevent app from downloading too much data,I wrote a php service, which takes hash (md5 hash of last downloaded data), provided by Android and compares it to latest data's hash on server. If hashes match each other, it prints 'no_new_data', otherwise it prints latest data. Php uses md5($string) method to calculate hash - this part seems to work fine.
The problem is that hash calculated on device never matches server's one - it is wrong, even though string seems to be same. I tried even with changing encoding but it didn't help.
My md5 java code:
public static String md5(String base){
try {
MessageDigest md = MessageDigest.getInstance("MD5");
md.update(base.getBytes());
byte byteData[] = md.digest();
//convert the byte to hex format method 1
StringBuffer sb = new StringBuffer();
for (int i = 0; i < byteData.length; i++) {
sb.append(Integer.toString((byteData[i] & 0xff) + 0x100, 16).substring(1));
}
//System.out.println("Digest(in hex format):: " + sb.toString());
//convert the byte to hex format method 2
StringBuffer hexString = new StringBuffer();
for (int i=0;i<byteData.length;i++) {
String hex=Integer.toHexString(0xff & byteData[i]);
if(hex.length()==1) hexString.append('0');
hexString.append(hex);
}
return hexString.toString();
}catch (Exception e){
return "a";
}
}
Thnks :)

Sometimes md5 hash is different from serverside hash. Try this method.
public static String getMD5Hash(String s) throws NoSuchAlgorithmException {
String result = s;
if (s != null) {
MessageDigest md = MessageDigest.getInstance("MD5"); // or "SHA-1"
md.update(s.getBytes());
BigInteger hash = new BigInteger(1, md.digest());
result = hash.toString(16);
while (result.length() < 32) { // 40 for SHA-1
result = "0" + result;
}
}
return result;
}

Never, ever use String.getBytes(), which depends on the platform-default charset, which is almost never what you want. It seems likely that the platform default charset differs between Android and your server side.
Pass it a Charset instead, e.g.
myString.getBytes(StandardCharsets.UTF_8)
if you have Java 7, or
myString.getBytes("UTF-8")
if you cannot.

How to translate to JavaScript this custom MD5 Java function

I have this custom function to calculate MD5 hash, written in Java. I can't change it. I need to translate it to JavaScript to use it on client side. I tried on my own but I can't manage with JavaScript data types (expecially Java char[])... Any help is appreciated, thanks!
// codes array
char[] codes = new char[64];
// initialise
private void initCodes(){
codes = new char[64];
codes[0] = '$';
int count = 0;
for (char i='0';i<='9';i++){ count++; codes[count] = i; }
for (char i='A';i<='Z';i++){ count++; codes[count] = i; }
for (char i='a';i<='z';i++){ count++; codes[count] = i; }
codes[63] = '£';
}
// custom MD5 algorithm
public String customMD5(String source) {
initCodes();
byte[] buf = new byte[source.length()];
buf = source.getBytes();
MessageDigest algorithm = null;
try {
algorithm = MessageDigest.getInstance("MD5");
} catch(NoSuchAlgorithmException e){}
algorithm.reset();
algorithm.update(buf);
byte[] digest = algorithm.digest();
int len = digest.length;
char[] encrypted = new char[len];
for (int i=0;i<len;i++)
encrypted[i] = codes[(int)(Math.floor((double)((digest[i]+128)/4)))];
return new String(encrypted);
}

See this part here:
MessageDigest algorithm = null;
try{
algorithm = MessageDigest.getInstance("MD5");
}catch(NoSuchAlgorithmException e){}
? That's where that stuff is accessing the MD5 code that's built into the Java runtime. You'll have to come up with your own implementation of MD5 there, which (to put it mildly) will be the tricky part.
All that the posted Java code really does (on top of calling the runtime to do the actual hashing) is to map the resulting hash (part of it, anyway) through a character lookup table.
edit — the lookup table built by that Java code is an array with "$", the digits, the upper-case letters, the lower-case letters, and then (surprisingly) "£". (The last character is surprising because it's not an old-school 7-bit ASCII character code, but whatever.) In JavaScript, that's:
var codes = "$0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz£";
The Java code then takes each 8-bit byte produced by the hash algorithm and looks up a code character by adding 128 to the byte and then dividing by 4. Java bytes are treated as signed values, so that has the effect of mapping every byte into the range 0 ... 63. That value is then used as a lookup into the code array.
Thus if you have a JavaScript MD5 facility that can give you back an array of numbers in the range -128 ... 127 (that is, signed 8-bit values), you could translate the result through the code array like this:
var digest = MagicJavaScriptMD5(source);
var result = [];
for (var i = 0; i < digest.length; ++i)
result.push(codes.charAt(~~((digest[i] + 128) / 4)));
var resultString = result.join('');
EDIT by the OP:
I take the liberty of posting here the right solution, that is highly derived from #Pointy's one. It requires md5.js from http://pajhome.org.uk/crypt/md5/.
/* MD5 in byte[] format */
function byteArray_md5(s) {
var output = [];
var input = rstr_md5(str2rstr_utf8(s)); //here it uses md5.js
for(var i = 0; i < input.length; i++)
output[i] = input.charCodeAt(i);
return output;
}
/* MD5 with custom mapping.
* It's a normal MD5 with a final char mapping of the hash.
*/
function md5WithCustomMapping(source) {
var codes = "$0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz£";
var digest = byteArray_md5(source);
var result = [];
for (var i = 0; i < digest.length; ++i)
result.push(
codes.charAt(
~~( ( digest[i] + 128 * (digest[i]<128 ? 1 : -1) )/4 )
)
);
return result.join('');
}

String transformations and locales on Android

I have an Android app that is the "server" in a client/server design. In the app, I need to compute an MD5 hash against a set of strings and return the result to the client in order to let the conversation between them to continue. My code to do this has been pieced together from numerous examples out there. The algorithm of computing the hash (not designed by me) goes like this:
Convert the string into an array of bytes
Use the MessageDigest class to generate a hash
Convert resulting hash back to a string
The hash seems to be correct for 99% of my customers. One of the customers seeing the wrong hash is running with a German locale, and it started to make me wonder if language could be factoring into the result I get. This is the code to make the byte array out of the string:
public static byte[] hexStringToByteArray(String s)
{
byte[] data = null;
if(s.length() % 2 != 0)
{
s = "0" + s;
}
int len = s.length();
data = new byte[len / 2];
for (int i = 0; i < len; i += 2)
{
data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
+ Character.digit(s.charAt(i+1), 16));
}
return data;
}
And here's the current version of the hashing function:
public static String hashDataAsString(String dataToHash)
{
MessageDigest messageDigest;
try
{
messageDigest = MessageDigest.getInstance("MD5");
messageDigest.reset();
byte[] data = hexStringToByteArray(dataToHash);
messageDigest.update(data);
final byte[] resultByte = messageDigest.digest();
return new String(Hex.encodeHex(resultByte));
}
catch(NoSuchAlgorithmException e)
{
throw new RuntimeException("Failed to hash data values", e);
}
}
I'm using the Hex.encodeHex function from Apache Commons.
I've tried switching my phone to a German locale, but my unit tests still produce the correct hash result. This customer is using stock Froyo, so that eliminates the risk that a custom ROM is at fault here. I've also found this alternative for converting from bytes to a string:
public static String MD5_Hash(String s) {
MessageDigest m = null;
try {
m = MessageDigest.getInstance("MD5");
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
}
//m.update(s.getBytes(),0,s.length());
byte [] data = hexStringToByteArray(s);
m.update(data, 0, data.length);
String hash = new BigInteger(1, m.digest()).toString(16);
return hash;
}
In my unit tests, it results in the same answer. Could BigInteger be a safer alternative to use here?

In your hashDataAsString method, do you need to do hexStringToByteArray? Is the incoming data a hex string or just an arbitrary string? Could you not use String.getBytes()?
If you are doing string/byte conversions, do you know the encoding of the incoming data and the encoding assumptions of your data consumers? Do you need to use a consistent encoding at both ends (e.g. ASCII or UTF-8)?
Do you include non-ASCII data in your unit tests?