Develop Reference

Java Mac HMAC vs C++ OpenSSL hmac

This is going to be a long question but I have a really weird bug. I use OpenSSL in C++ to compute a HMAC and compare them to a simular implementation using javax.crypto.Mac. For some keys the HMAC calculation is correct and for others there is a difference in HMAC. I believe the problem occurs when the keys get to big. Here are the details.
Here is the most important code for C++:
void computeHMAC(std::string message, std::string key){
unsigned int digestLength = 20;
HMAC_CTX hmac_ctx_;
BIGNUM* key_ = BN_new();;
BN_hex2bn(&key_, key);
unsigned char convertedKey[BN_num_bytes(key_)];
BIGNUM* bn = BN_new();
HMAC_CTX_init(&hmac_ctx_);
BN_bn2bin(bn, convertedKey);
int length = BN_bn2bin(key_, convertedKey);
HMAC_Init_ex(&hmac_ctx_, convertedKey, length, EVP_sha1(), NULL);
/*Calc HMAC */
std::transform( message.begin(), message.end(), message.begin(), ::tolower);
unsigned char digest[digestLength];
HMAC_Update(&hmac_ctx_, reinterpret_cast<const unsigned char*>(message.c_str()),
message.length());
HMAC_Final(&hmac_ctx_, digest, &digestLength);
char mdString[40];
for(unsigned int i = 0; i < 20; ++i){
sprintf(&mdString[i*2], "%02x", (unsigned int)digest[i]);
}
std::cout << "\n\nMSG:\n" << message << "\nKEY:\n" + std::string(BN_bn2hex(key_)) + "\nHMAC\n" + std::string(mdString) + "\n\n";
}
The java test looks like this:
public String calculateKey(String msg, String key) throws Exception{
HMAC = Mac.getInstance("HmacSHA1");
BigInteger k = new BigInteger(key, 16);
HMAC.init(new SecretKeySpec(k.toByteArray(), "HmacSHA1"));
msg = msg.toLowerCase();
HMAC.update(msg.getBytes());
byte[] digest = HMAC.doFinal();
System.out.println("Key:\n" + k.toString(16) + "\n");
System.out.println("HMAC:\n" + DatatypeConverter.printHexBinary(digest).toLowerCase() + "\n");
return DatatypeConverter.printHexBinary(digest).toLowerCase();
}
Some test runs with different keys (all strings are interpreted as hex):
Key1:
736A66B29072C49AB6DC93BB2BA41A53E169D14621872B0345F01EBBF117FCE48EEEA2409CFC1BD92B0428BA0A34092E3117BEB4A8A14F03391C661994863DAC1A75ED437C1394DA0741B16740D018CA243A800DA25311FDFB9CA4361743E8511E220B79C2A3483FCC29C7A54F1EB804481B2DC87E54A3A7D8A94253A60AC77FA4584A525EDC42BF82AE2A1FD6E3746F626E0AFB211F6984367B34C954B0E08E3F612590EFB8396ECD9AE77F15D5222A6DB106E8325C3ABEA54BB59E060F9EA0
Msg:
test
Hmac OpenSSL:
b37f79df52afdbbc4282d3146f9fe7a254dd23b3
Hmac Java Mac:
b37f79df52afdbbc4282d3146f9fe7a254dd23b3
Key 2: 636A66B29072C49AB6DC93BB2BA41A53E169D14621872B0345F01EBBF117FCE48EEEA2409CFC1BD92B0428BA0A34092E3117BEB4A8A14F03391C661994863DAC1A75ED437C1394DA0741B16740D018CA243A800DA25311FDFB9CA4361743E8511E220B79C2A3483FCC29C7A54F1EB804481B2DC87E54A3A7D8A94253A60AC77FA4584A525EDC42BF82AE2A1FD6E3746F626E0AFB211F6984367B34C954B0E08E3F612590EFB8396ECD9AE77F15D5222A6DB106E8325C3ABEA54BB59E060F9EA0
Msg:
test
Hmac OpenSSL:
bac64a905fa6ae3f7bf5131be06ca037b3b498d7
Hmac Java Mac:
bac64a905fa6ae3f7bf5131be06ca037b3b498d7
Key 3: 836A66B29072C49AB6DC93BB2BA41A53E169D14621872B0345F01EBBF117FCE48EEEA2409CFC1BD92B0428BA0A34092E3117BEB4A8A14F03391C661994863DAC1A75ED437C1394DA0741B16740D018CA243A800DA25311FDFB9CA4361743E8511E220B79C2A3483FCC29C7A54F1EB804481B2DC87E54A3A7D8A94253A60AC77FA4584A525EDC42BF82AE2A1FD6E3746F626E0AFB211F6984367B34C954B0E08E3F612590EFB8396ECD9AE77F15D5222A6DB106E8325C3ABEA54BB59E060F9EA0
Msg:
test
Hmac OpenSSL:
c189c637317b67cee04361e78c3ef576c3530aa7
Hmac Java Mac:
472d734762c264bea19b043094ad0416d1b2cd9c
As the data shows, when the key gets to big, an error occurs. If have no idea which implementation is faulty. I have also tried with bigger keys and smaller keys. I haven't determined the exact threshold. Can anyone spot the problem? Is there anyone capable of telling me which HMAC is incorrect in the last case by doing a simulation using different software or can anyone tell me which 3rd implementation I could use to check mine?
Kind regards,
Roel Storms

When you convert a hexadecimal string to a BigInt in Java, it assumes the number is positive (unless the string includes a - sign).
But the internal representation of it is twos-complement. Meaning that one bit is used for the sign.
If you are converting a value that starts with a hex between 00 and 7F inclusive, then that's not a problem. It can convert the byte directly, because the leftmost bit is zero, which means that the number is considered positive.
But if you are converting a value that starts with 80 through FF, then the leftmost bit is 1, which will be considered negative. To avoid this, and keep the BigInteger value exactly as it is supplied, it adds another zero byte at the beginning.
So, internally, the conversion of a number such as 7ABCDE is the byte array
0x7a 0xbc 0xde
But the conversion of a number such as FABCDE (only the first byte is different!), is:
0x00 0xfa 0xbc 0xde
This means that for keys that begin with a byte in the range 80-FF, the BigInteger.toByteArray() is not producing the same array that your C++ program produced, but an array one byte longer.
There are several ways to work around this - like using your own hex-to-byte-array parser or finding an existing one in some library. If you want to use the one produced by BigInteger, you could do something like this:
BigInteger k = new BigInteger(key, 16);
byte[] kByteArr = k.toByteArray();
if ( kByteArr.length > (key.length() + 1) / 2 ) {
kByteArr = Arrays.copyOfRange(kByteArr,1,kByteArr.length);
}
Now you can use the kByteArr to perform the operation properly.
Another issue you should watch out for is keys whose length is odd. In general, you shouldn't have a hex octet string that has an odd length. A string like F8ACB is actually 0F8ACB (which is not going to cause an extra byte in BigInteger) and should be interpreted as such. This is why I wrote (key.length() + 1) in my formula - if key is odd-length, it should be interpreted as a one octet longer. This is also important to watch out for if you write your own hex-to-byte-array converter - if the length is odd, you should add a zero at the beginning before you start converting.

Get hash of a String as String

I'm here:
String password = "123";
byte passwordByte[] = password.getBytes();
MessageDigest md = MessageDigest.getInstance("SHA-512");
byte passwortHashByte[] = md.digest(passwordByte);
The passwortHashByte-Array contains only a lot of numbers. I want to convernt this numbers to one String which contains the hash-code as plaintext.
How i do this?

I want to convernt this numbers to one String which contains the hash-code as plaintext.
The hash isn't plain-text. It's binary data - arbitrary bytes. That isn't plaintext any more than an MP3 file is.
You need to work out what textual representation you want to use for the binary data. That in turn depends on what you want to use the data for. For the sake of easy diagnostics I'd suggest a pure-ASCII representation - probably either base64 or hex. If you need to easily look at individual bytes, hex is simpler to read, but base64 is a bit more compact.
It's also important to note that MD5 isn't a particularly good way of hashing passwords... and it looks like you're not even salting them. It may be good enough for a demo app which will never be released into the outside world, but you should really look into more secure approaches. See Jeff Atwood's blog post on the topic for an introduction, and ideally get hold of a book about writing secure code.

Here is how I did it for my website.
private static byte[] fromHex(String hex) {
byte[] bytes = new byte[hex.length() / 2];
for (int i = 0; i < hex.length() / 2; i++) {
bytes[i] = (byte)(Character.digit(hex.charAt(i * 2), 16) * 16 + Character.digit(hex.charAt(i * 2 + 1), 16) - 128);
}
return bytes;
}
private static String toHex(byte[] bytes) {
String hex = new String();
for (int i = 0; i < bytes.length; i++) {
String c = Integer.toHexString(bytes[i] + 128);
if (c.length() == 1) c = "0" + c;
hex = hex + c;
}
return hex;
}
That'll allow you to convert your byte array to and from a hex string.

Well, byte passwortHashByte[] = md.digest(passwordByte); can contain some controll characters, which will broke your String. Consider encoding passwortHashByte[] to Base64 form, if you really need String from it. You can use Apache Commons Codecs for making base64 form.

RSA Authentication Issue

I'm making a system where I want to verify the server's identity via RSA, but I can't seem to get the server to properly decrypt the client's message.
The public and private keys are in slot 0 of the array, and mod is in slot 1, so they are setup correctly.
Client side code
int keyLength = 3072 / 8;//RSA key size
byte[] data = new byte[keyLength];
//Generate some random data. Note that
//Only the fist half of this will be used.
new SecureRandom().nextBytes(data);
int serverKeySize = in.readInt();
if (serverKeySize != keyLength) {//Definitely not the right heard
return false;
}
//Take the server's half of the random data and pass ours
in.readFully(data, keyLength / 2 , keyLength / 2);
//Encrypt the data
BigInteger[] keys = getKeys();
BigInteger original = new BigInteger(data);
BigInteger encrypted = original.modPow(keys[0], keys[1]);
data = encrypted.toByteArray();
out.write(data);
//If the server's hash doesn't match, the server has the wrong key!
in.readFully(data, 0, data.length);
BigInteger decrypted = new BigInteger(data);
return original.equals(decrypted);
Server side code
int keyLength = 3072 / 8;//Key length
byte[] data = new byte[keyLength];
//Send the second half of the key
out.write(data, keyLength / 2, keyLength / 2);
in.readFully(data);
BigInteger[] keys = getKeys();
BigInteger encrypted = new BigInteger(data);
BigInteger original = encrypted.modPow(keys[0], keys[1]);
data = original.toByteArray();
out.write(data);
AFAIK that implementation is correct however it doesn't seem to produce the correct output. Also no, I do not wish to use a Cipher for various reasons.

There are some critical details that are not being accounted for. The data you want to apply RSA to must be encoded as BigInteger x, with 0 <= x < n, where n is your modulus. You aren't doing that. In fact, because you are filling your entire data array with random data you cannot guarantee that. The PKCS#1 padding algorithm is designed to do this correctly, but since you are rolling your own you'll have to fix this in your code. Also, examine carefully how the BigInteger(byte[]) constructor and BigInteger.toByteArray() decode/encode integers. Naively many expect simply the base 256 encoding, and forget that BigInteger must accommodate negative integer also. It does so by using the ASN.1 DER integer rules. If the positive integer's high-order byte would be >= 128 then a leading zero byte is added.

How to iteratively sha256 in Python using native lib (ie hashlib), using byte[] as input and not hex string

Background: I have an iterative hash algorithm I need to compute from a Python script and a Java web application.
Psuedo code:
hash = sha256(raw)
for x=1 to 64000 hash = sha256(hash)
where hash is a byte array of length 32, and not a hex string of length 64.
The reason I want to keep it in bytes is because, though Python can convert to hex string in between each iteration and keep the processing time under a second, Java takes 3 seconds for the String overhead.
So, the Java code looks like this:
// hash one time...
byte[] result = sha256(raw.getBytes("UTF-8"));
// then hash 64k-1 more times
for (int x = 0; x < 64000-1; x++) {
result = sha256(result);
}
// hex encode and print result
StringBuilder sb = new StringBuilder();
Formatter formatter = new Formatter(sb);
for (int i=0; i<buf.length; i++) {
formatter.format("%02x", buf[i]);
}
System.out.println(sb.toString());
And the Python code looks like this:
import hashlib
# hash 1 time...
hasher = hashlib.sha256()
hasher.update(raw)
digest = hasher.digest()
# then hash 64k-1 times
for x in range (0, 64000-1):
# expect digest is bytes and not hex string
hasher.update(digest)
digest = hasher.digest()
print digest.encode("hex")
The Python result calculated the hash on the hex representation of the first digest (String), rather than the raw digest bytes. So, I get varying outputs.

Method .update of hasher appends argument to previous text (Python docs). Instead you should create new hasher each time you want to compute digest.
import hashlib
# hash 1 time...
digest = hashlib.sha256(raw).digest()
# then hash 64k-1 times
for x in range(0, 64000-1):
digest = hashlib.sha256(digest).digest()
print digest.encode("hex")

simple symmetric encryption of long to String (and back) in java

I'm looking for a simple way to translate long to String and back in a way that will "hide" the long value.
I'd prefer to avoid adding another .jar to the project for this feature.
It does not have to be a hard-to-crack encryption, just to look random to the inexperienced eye.
Addition:
My purpose here is to attach a counter value (of type long) to URLs as a tracking parameter without the users aware of the counter's value (sort of like tinyURL's hash), so that the servlet will know the value of the counter when the URL is clicked.
Thanks

If
X * Y = 1 (mod 2^32)
then
A * (X * Y) = A (mod 2^32)
(A * X) * Y = A (mod 2^32)
So, you can "encrypt" some 32-bit number by multiplying it by X, and then later "decrypt" by multiplying by Y. All you need is to find some non-trivial X and Y that satisfy the condition.
For instance, (X, Y) = (3766475841, 1614427073), or (699185821, 3766459317). I just found these with a simple brute force program.
Once you have A*X you can encode it using Base-64 or hexadecimal or some similar scheme in the URL. I'd suggest Base64 because it takes up less space and looks fairly "random".

If you're just obfuscating, something trivial like
long -> string -> char array
for each character, XOR with the previous output value (with some arbitrary value for before the first character)
char array -> string
To reveal for each character, XOR with the previous obfuscated input value (with some arbitrary value for before the first character)

Uhm… without further details it’s hard to come up with a solution on this. You can do lots of different things to accomplish your goal… I guess.
You could XOR it with a random number and store both numbers. This obviously won’t work when your algorithm is out in the open (e.g. if you want to put it in open source).
Or you could use some symmetric encryption algorithm, e.g. Twofish or Serpent.
All of this is rather futile if the number is stored on the client system and evaluated by a client application. As soon as you hand out your application clients can access everything it stores. If the information is worth it, decryption programs will be available soon after your application is released. If it’s not worth it, then why bother?

I like to use Long.toString(value, 36). This prints the number in base 36 which is compact, though cryptic (provided the number is >= 10) You can parse it with Long.parseLong(text, 36)

XOR with an one-time pad (OTP) provides perfect secrecy. I wrote a cipher using OTP to encrypt integers. I just adapted it to long to meet your requirement. It encrypts a long into a 24 char hex string with salt prepended,
-3675525778535888036 => 4fe555ca33021738a3797ab2
-6689673470125604264 => 76092fda5cd67e93b18b4f2f
8956473951386520443 => 0fb25e533be315bdb6356a2a
4899819575233977659 => 7cf17d74d6a2968370fbe149
Here is the code,
public class IntegerCipher {
// This is for salt so secure random is not needed
private static Random PRNG = new Random();
private String secret;
public IntegerCipher(String secret) {
if (secret.length() < 4)
throw new IllegalArgumentException("Secret is too short");
this.secret = secret;
}
public String encrypt(long value) {
int salt = PRNG.nextInt();
long otp = generatePad(salt);
long cipher = value ^ otp;
return String.format("%08x%016x", salt, cipher);
}
public long decrypt(String ciphertext) {
if (ciphertext.length() != 24)
throw new IllegalArgumentException("Invalid cipher text");
try {
int salt = (int) Long.parseLong(ciphertext.substring(0, 8), 16);
long cipher = Long.parseLong(ciphertext.substring(8, 16), 16) << 32
| Long.parseLong(ciphertext.substring(16), 16);
long otp = generatePad(salt);
return cipher ^ otp;
} catch (NumberFormatException e) {
throw new IllegalArgumentException("Invalid hex value: "
+ e.getMessage());
}
}
private long generatePad(int salt) {
String saltString = Integer.toString(salt);
String lpad = saltString + secret;
String rpad = secret + saltString;
return ((long) lpad.hashCode()) << 32 | (long) rpad.hashCode();
}
public static void main(String[] args) {
IntegerCipher cipher = new IntegerCipher("Top Secret");
Random rand = new Random();
for (int i = 0; i < 100; i++) {
Long n = rand.nextLong();
String ciphertext = cipher.encrypt(n);
Long m = cipher.decrypt(ciphertext);
System.out.printf("%24d => %s\n", n, ciphertext);
assert(m == n);
}
}
}

For encrypting short plaintexts in a deterministic way, you might want to have a look at
FFSEM.