Develop Reference

SHA512 in C # and Java are different

There is such code on C # and java, sha512 in them differs, whether it is possible to make somehow that the result sha512 was identical? I understand the problem in BaseConverter, analog Base64 in Java? Tried
Base64.getEncoder().encodeToString(str);
But I get an error because of getEncoder(). Do I need a library for this?
Code in C#:
public string Hash(string str)
{
string resultStr = String.Empty;
byte[] data = new UTF8Encoding().GetBytes(str);
byte[] result;
SHA512 shaM = new SHA512Managed();
result = shaM.ComputeHash(data);
resultStr = ReverseString(BitConverter.ToString(result).ToLower().Replace("-", String.Empty));
return resultStr.Substring(5, 25);
}
public static string ReverseString(string s)
{
char[] charArray = s.ToCharArray();
Array.Reverse(charArray);
return new string(charArray);
}
Code in Java:
public String Hash(String str) {
try {
MessageDigest digest = MessageDigest.getInstance("SHA-512");
digest.update(str.getBytes("UTF-16LE"));
byte messageDigest[] = digest.digest();
StringBuffer hexString = new StringBuffer();
for (int i = 0; i < messageDigest.length; i++) {
String h = Integer.toHexString(0xFF & messageDigest[i]);
while (h.length() < 2)
h = "0" + h;
hexString.append(h);
}
result = hexString.toString().toLowerCase();
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
return ReverseString(result).substring(5, 25);
}
public static String ReverseString(String s)
{
return new StringBuilder(s).reverse().toString();
}

You're hashing different data - in Java you're converting the string to UTF-16:
digest.update(str.getBytes("UTF-16LE"));
In C# you're using UTF-8:
byte[] data = new UTF8Encoding().GetBytes(str);
(I'm not sure why you're creating a new UTF8Encoding rather than using Encoding.UTF8, admittedly.)
With different input, you will get different hashes.
In general, the way to diagnose problems like this is to compare the data at every step of the transformation, whether that's through logging or debugging. In this case you have four transformations:
Message string to message bytes
Message bytes to hash bytes
Hash bytes to hash string (hex)
Reversed hash string (hex)
Next time, check the output of each step, and you'll work out where the problem is.
(It's not obvious why you'd want to reverse the hex output anyway, but that's a different matter.)

The problem was in the input line for hashing (the string was the salt without the rest of the data, the rest of the data was empty because there was an error in the definition of EditText (EditText returned an empty string) and also fixed the encoding in Java for UTF-8.

SHA-1 hashing on Java and C#

I'm trying to validate the content of an XML node with SHA-1 , basically, we generate an SHA-1 hash with the content of that node and both sides (client C# and server Java) should have exactly the same hash.
The problem is , I have checked with a diff tool the content of both texts and there is not any difference. But I'm getting a different hash than the client.
C# hash : 60-53-58-69-29-EB-53-BD-85-31-79-28-A0-F9-42-B6-DE-1B-A6-0A
Java hash: E79D7E6F2A6F5D776447714D896D4C3A0CBC793
The way the client (C#) is generating the hash is this:
try
{
Byte[] stream = null;
using (System.Security.Cryptography.SHA1CryptoServiceProvider shaProvider = new System.Security.Cryptography.SHA1CryptoServiceProvider())
{
stream = shaProvider.ComputeHash(System.Text.Encoding.UTF8.GetBytes(text));
if (stream == null)
{
hash = "Error";
}
else
{
hash = System.BitConverter.ToString(stream);
}
}
}
catch (Exception error)
{
hash = string.Format("Error SHA-1: {0}", error);
}
return hash;
and this is how the server (Java) is generating the hash:
byte[] key = content.getBytes();
MessageDigest md = MessageDigest.getInstance("SHA1");
byte[] hash = md.digest(key);
String result = "";
for (byte b : hash) {
result += Integer.toHexString(b & 255);
}
return result.toUpperCase();
can someone help me ? .. thanks :)
UPDATE:
In order to check what's going on I have checked other ways to get a SHA1 hash in C# and I found this:
/// <summary>
/// Compute hash for string encoded as UTF8
/// </summary>
/// <param name="s">String to be hashed</param>
/// <returns>40-character hex string</returns>
public static string SHA1HashStringForUTF8String(string s)
{
byte[] bytes = Encoding.UTF8.GetBytes(s);
using (var sha1 = SHA1.Create())
{
byte[] hashBytes = sha1.ComputeHash(bytes);
return System.BitConverter.ToString(hashBytes).Replace("-",string.Empty);
}
}
This code gives this output:
E79D07E6F2A6F5D776447714D896D4C3A0CBC793
AND !! I just noticed that Python is giving the same output (sorry, I should double checked this)
So this is the deal
Using this provider: System.Security.Cryptography.SHA1CryptoServiceProvider shaProvider = new System.Security.Cryptography.SHA1CryptoServiceProvider()
Is giving a completly different output on three different machines ..
Using the above method in C# gives the same result as python does, also, for some reason Java is giving a sightly different output:
E79D7E6F2A6F5D776447714D896D4C3A0CBC793
Ideas?, is java the problem? the byte to hex method on java is the problem? there is another alternative?

Try using this as your hashing in C#:
static string Hash(string input)
{
using (SHA1Managed sha1 = new SHA1Managed())
{
var hash = sha1.ComputeHash(Encoding.UTF8.GetBytes(input));
var sb = new StringBuilder(hash.Length * 2);
foreach (byte b in hash)
{
// can be "x2" if you want lowercase
sb.Append(b.ToString("x2"));
}
return sb.ToString();
}
}
Hash("test"); //a94a8fe5ccb19ba61c4c0873d391e987982fbbd3
And then use this as your Java hashing:
private static String convertToHex(byte[] data) {
StringBuilder buf = new StringBuilder();
for (byte b : data) {
int halfbyte = (b >>> 4) & 0x0F;
int two_halfs = 0;
do {
buf.append((0 <= halfbyte) && (halfbyte <= 9) ? (char) ('0' + halfbyte) : (char) ('a' + (halfbyte - 10)));
halfbyte = b & 0x0F;
} while (two_halfs++ < 1);
}
return buf.toString();
}
public static String SHA1(String text) throws NoSuchAlgorithmException, UnsupportedEncodingException {
MessageDigest md = MessageDigest.getInstance("SHA-1");
byte[] textBytes = text.getBytes("iso-8859-1");
md.update(textBytes, 0, textBytes.length);
byte[] sha1hash = md.digest();
return convertToHex(sha1hash);
}
SHA1("test"); //a94a8fe5ccb19ba61c4c0873d391e987982fbbd3
Note you need the following imports:
import java.io.UnsupportedEncodingException; import
java.security.MessageDigest; import
java.security.NoSuchAlgorithmException;
Throws declarations are option, adjust to best fit your code!

Your problem is that you're not hashing the same bytes in both API.
If you choose to modify java's version, it should look like this:
byte[] key = content.getBytes("UTF8");
[...]
If you choose to modify c#' version, it should look like this:
stream = shaProvider.ComputeHash(System.Text.Encoding.UTF16.GetBytes(text));
[...]
Either way, both api should get the key's bytes through the same encoding.

Porting Java encryption routine to C#

I'm attempting with little success to port over Google's code to generate a secure token for their captcha (https://github.com/google/recaptcha-java/blob/master/appengine/src/main/java/com/google/recaptcha/STokenUtils.java):
The original utility has the following:
private static final String CIPHER_INSTANCE_NAME = "AES/ECB/PKCS5Padding";
private static String encryptAes(String input, String siteSecret) {
try {
SecretKeySpec secretKey = getKey(siteSecret);
Cipher cipher = Cipher.getInstance(CIPHER_INSTANCE_NAME);
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
return BaseEncoding.base64Url().omitPadding().encode(cipher.doFinal(input.getBytes("UTF-8")));
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
private static SecretKeySpec getKey(String siteSecret){
try {
byte[] key = siteSecret.getBytes("UTF-8");
key = Arrays.copyOf(MessageDigest.getInstance("SHA").digest(key), 16);
return new SecretKeySpec(key, "AES");
} catch (NoSuchAlgorithmException | UnsupportedEncodingException e) {
e.printStackTrace();
}
return null;
}
public static void main(String [] args) throws Exception {
//Hard coded the following to get a repeatable result
String siteSecret = "12345678";
String jsonToken = "{'session_id':'abf52ca5-9d87-4061-b109-334abb7e637a','ts_ms':1445705791480}";
System.out.println(" json token: " + jsonToken);
System.out.println(" siteSecret: " + siteSecret);
System.out.println(" Encrypted stoken: " + encryptAes(jsonToken, siteSecret));
Given the values I hardcoded, I get Irez-rWkCEqnsiRLWfol0IXQu1JPs3qL_G_9HfUViMG9u4XhffHqAyju6SRvMhFS86czHX9s1tbzd6B15r1vmY6s5S8odXT-ZE9A-y1lHns" back as my encrypted token.
My Java and crypto skills are more than a little rusty, and there aren't always direct analogs in C#. I attempted to merge encrypeAes() and getKey() with the following, which isn't correct:
public static string EncryptText(string PlainText, string siteSecret)
{
using (RijndaelManaged aes = new RijndaelManaged())
{
aes.Mode = CipherMode.ECB;
aes.Padding = PaddingMode.PKCS7;
var bytes = Encoding.UTF8.GetBytes(siteSecret);
SHA1 sha1 = SHA1.Create();
var shaKey = sha1.ComputeHash(bytes);
byte[] targetArray = new byte[16];
Array.Copy(shaKey, targetArray, 16);
aes.Key = targetArray;
ICryptoTransform encrypto = aes.CreateEncryptor();
byte[] plainTextByte = ASCIIEncoding.UTF8.GetBytes(PlainText);
byte[] CipherText = encrypto.TransformFinalBlock(plainTextByte, 0, plainTextByte.Length);
return HttpServerUtility.UrlTokenEncode(CipherText); //Equivalent to java's BaseEncoding.base64Url()?
}
}
The C# version produces the incorrect value of: Ye+fySvneVUZJXth67+Si/e8fBUV4Sxs7wEXVDEOJjBMHl1encvt65gGIj8CiFzBGp5uUgKYJZCuQ4rc964vZigjlrJ/430LgYcathLLd9U=

Your code almost works as expected. It's just that you somehow mixed up the outputs of the Java version (and possibly the C# version).
If I execute your Java code (JDK 7 & 8 with Guava 18.0), I get
Ye-fySvneVUZJXth67-Si_e8fBUV4Sxs7wEXVDEOJjBMHl1encvt65gGIj8CiFzBGp5uUgKYJZCuQ4rc964vZigjlrJ_430LgYcathLLd9U
and if I execute your C# code (DEMO), I get
Ye-fySvneVUZJXth67-Si_e8fBUV4Sxs7wEXVDEOJjBMHl1encvt65gGIj8CiFzBGp5uUgKYJZCuQ4rc964vZigjlrJ_430LgYcathLLd9U1
So, the C# version has an additional "1" at the end. It should be a padding character, but isn't. This means that HttpServerUtility.UrlTokenEncode() doesn't provide a standards conform URL-safe Base64 encoding and you shouldn't use it. See also this Q&A.
The URL-safe Base64 encoding can be easily derived from the normal Base64 encoding (compare tables 1 and 2 in RFC4648) as seen in this answer by Marc Gravell:
string returnValue = System.Convert.ToBase64String(toEncodeAsBytes)
.TrimEnd(padding).Replace('+', '-').Replace('/', '_');
with:
static readonly char[] padding = { '=' };
That's not all. If we take your Java output of
Ye+fySvneVUZJXth67+Si/e8fBUV4Sxs7wEXVDEOJjBMHl1encvt65gGIj8CiFzBGp5uUgKYJZCuQ4rc964vZigjlrJ/430LgYcathLLd9U=
and decrypt it, then we get the following token:
{"session_id":"4182e173-3a24-4c10-b76c-b85a36be1173","ts_ms":1445786965574}
which is different from the token that you have in your code:
{'session_id':'abf52ca5-9d87-4061-b109-334abb7e637a','ts_ms':1445705791480}
The main remaining problem is that you're using invalid JSON. Strings and keys in JSON need to be wrapped in " and not '.
Which means that the encrypted token actually should have been (using a valid version of the token from your code):
D9rOP07fYgBfza5vbGsvdPe8fBUV4Sxs7wEXVDEOJjBMHl1encvt65gGIj8CiFzBsAWBDgtdSozv4jS_auBU-CgjlrJ_430LgYcathLLd9U

Here's a C# implementation that reproduces the same result as your Java code:
class Program
{
public static byte[] GetKey(string siteSecret)
{
byte[] key = Encoding.UTF8.GetBytes(siteSecret);
return SHA1.Create().ComputeHash(key).Take(16).ToArray();
}
public static string EncryptAes(string input, string siteSecret)
{
var key = GetKey(siteSecret);
using (var aes = AesManaged.Create())
{
if (aes == null) return null;
aes.Mode = CipherMode.ECB;
aes.Padding = PaddingMode.PKCS7;
aes.Key = key;
byte[] inputBytes = Encoding.UTF8.GetBytes(input);
var enc = aes.CreateEncryptor(key, new byte[16]);
return UrlSafeBase64(enc.TransformFinalBlock(inputBytes,0,input.Length));
}
}
// http://stackoverflow.com/a/26354677/162671
public static string UrlSafeBase64(byte[] bytes)
{
return Convert.ToBase64String(bytes).TrimEnd('=')
.Replace('+', '-')
.Replace('/', '_');
}
static void Main(string[] args)
{
string siteSecret = "12345678";
string jsonToken = "{'session_id':'abf52ca5-9d87-4061-b109-334abb7e637a','ts_ms':1445705791480}";
Console.WriteLine(" json token: " + jsonToken);
Console.WriteLine(" siteSecret: " + siteSecret);
Console.WriteLine(EncryptAes(jsonToken, siteSecret));
Console.ReadLine();
}
}
I don't know why you said you're getting Irez-rWkCEqnsiRLWfol0IXQu1JPs3qL_G_9HfUViMG9u4XhffHqAyju6SRvMhFS86czHX9s1tbzd6B15r1vmY6s5S8odXT-ZE9A-y1lHns from the Java program because I'm not getting that output. The output I'm getting from both the C# version and the Java version is this:
Ye-fySvneVUZJXth67-Si_e8fBUV4Sxs7wEXVDEOJjBMHl1encvt65gGIj8CiFzBGp5uUgKYJZCuQ4rc964vZigjlrJ_430LgYcathLLd9U
As you can see here:
The code for both versions is available here
Live demo of the C# version.
The Java version was copy/pasted from your code and is using guava-18.0 and compiled with JDK8 x64 (I'm not a java expert so I'm just adding these in case it makes any difference).

Android's string hash doesn't match serverisde's

I'm developing an Android app and I need to send some data from server to Android device.
To prevent app from downloading too much data,I wrote a php service, which takes hash (md5 hash of last downloaded data), provided by Android and compares it to latest data's hash on server. If hashes match each other, it prints 'no_new_data', otherwise it prints latest data. Php uses md5($string) method to calculate hash - this part seems to work fine.
The problem is that hash calculated on device never matches server's one - it is wrong, even though string seems to be same. I tried even with changing encoding but it didn't help.
My md5 java code:
public static String md5(String base){
try {
MessageDigest md = MessageDigest.getInstance("MD5");
md.update(base.getBytes());
byte byteData[] = md.digest();
//convert the byte to hex format method 1
StringBuffer sb = new StringBuffer();
for (int i = 0; i < byteData.length; i++) {
sb.append(Integer.toString((byteData[i] & 0xff) + 0x100, 16).substring(1));
}
//System.out.println("Digest(in hex format):: " + sb.toString());
//convert the byte to hex format method 2
StringBuffer hexString = new StringBuffer();
for (int i=0;i<byteData.length;i++) {
String hex=Integer.toHexString(0xff & byteData[i]);
if(hex.length()==1) hexString.append('0');
hexString.append(hex);
}
return hexString.toString();
}catch (Exception e){
return "a";
}
}
Thnks :)

Sometimes md5 hash is different from serverside hash. Try this method.
public static String getMD5Hash(String s) throws NoSuchAlgorithmException {
String result = s;
if (s != null) {
MessageDigest md = MessageDigest.getInstance("MD5"); // or "SHA-1"
md.update(s.getBytes());
BigInteger hash = new BigInteger(1, md.digest());
result = hash.toString(16);
while (result.length() < 32) { // 40 for SHA-1
result = "0" + result;
}
}
return result;
}

Never, ever use String.getBytes(), which depends on the platform-default charset, which is almost never what you want. It seems likely that the platform default charset differs between Android and your server side.
Pass it a Charset instead, e.g.
myString.getBytes(StandardCharsets.UTF_8)
if you have Java 7, or
myString.getBytes("UTF-8")
if you cannot.

String transformations and locales on Android

I have an Android app that is the "server" in a client/server design. In the app, I need to compute an MD5 hash against a set of strings and return the result to the client in order to let the conversation between them to continue. My code to do this has been pieced together from numerous examples out there. The algorithm of computing the hash (not designed by me) goes like this:
Convert the string into an array of bytes
Use the MessageDigest class to generate a hash
Convert resulting hash back to a string
The hash seems to be correct for 99% of my customers. One of the customers seeing the wrong hash is running with a German locale, and it started to make me wonder if language could be factoring into the result I get. This is the code to make the byte array out of the string:
public static byte[] hexStringToByteArray(String s)
{
byte[] data = null;
if(s.length() % 2 != 0)
{
s = "0" + s;
}
int len = s.length();
data = new byte[len / 2];
for (int i = 0; i < len; i += 2)
{
data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
+ Character.digit(s.charAt(i+1), 16));
}
return data;
}
And here's the current version of the hashing function:
public static String hashDataAsString(String dataToHash)
{
MessageDigest messageDigest;
try
{
messageDigest = MessageDigest.getInstance("MD5");
messageDigest.reset();
byte[] data = hexStringToByteArray(dataToHash);
messageDigest.update(data);
final byte[] resultByte = messageDigest.digest();
return new String(Hex.encodeHex(resultByte));
}
catch(NoSuchAlgorithmException e)
{
throw new RuntimeException("Failed to hash data values", e);
}
}
I'm using the Hex.encodeHex function from Apache Commons.
I've tried switching my phone to a German locale, but my unit tests still produce the correct hash result. This customer is using stock Froyo, so that eliminates the risk that a custom ROM is at fault here. I've also found this alternative for converting from bytes to a string:
public static String MD5_Hash(String s) {
MessageDigest m = null;
try {
m = MessageDigest.getInstance("MD5");
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
}
//m.update(s.getBytes(),0,s.length());
byte [] data = hexStringToByteArray(s);
m.update(data, 0, data.length);
String hash = new BigInteger(1, m.digest()).toString(16);
return hash;
}
In my unit tests, it results in the same answer. Could BigInteger be a safer alternative to use here?

In your hashDataAsString method, do you need to do hexStringToByteArray? Is the incoming data a hex string or just an arbitrary string? Could you not use String.getBytes()?
If you are doing string/byte conversions, do you know the encoding of the incoming data and the encoding assumptions of your data consumers? Do you need to use a consistent encoding at both ends (e.g. ASCII or UTF-8)?
Do you include non-ASCII data in your unit tests?