I am using consuming a C# web services and one of the parameters is sending a md5 hash. Java creates MD5 hash with signed (contains negative number in the byte array) and C# generates unsigned (contains no negative number in the byte array).
I have gone through multiple similar question in Stack Overflow but did not find any to my satisfaction.
All I need is unsigned byte array similar to the one c# generates. I have tried using BigInteger but I need it in an unsigned byte array since I need do further processing after that. BigInteger gives me one single integer and using tobytearray() still has negative numbers.
If I have to do 2 complement, then how can I do that. Then I can loop through the byte array and convert negative number to positive number.
I am using the following Java code for generating MD5 hash:
String text = "abc";
MessageDigest md = MessageDigest.getInstance("MD5");
byte[] md5hash = new byte[32];
try {
md.update(text.getBytes("utf-8"), 0, text.length());
} catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
md5hash = md.digest();
Java bytes are signed numbers, but that only means that when considering a byte (which is a sequence of 8 bits) as a number, Java treats one of the bits as a sign bit, whereas other language read the same sequence of bits as an unsigned number, containing no sign bit.
The MD5 algorithm is a binary algorithm that transforms a sequence of bits (or bytes) into another sequence of bits (or bytes). The way Java does that is the same as the way any other language does it. It's only when displaying the bytes as numbers that you'll get different outputs depending on the way the language transforms bytes into numbers.
So the short answer is, send an MD5 hash generated using Java to a C# program, and it will work fine.
If you want to display the byte array in Java as unsigned numbers, just use the following code:
for (byte b : bytes) {
System.out.println(b & 0xFF);
}
Related
The question is about the correct way of creating a hash in Java:
Lets assume I have a positive BigInteger value that I would like to create a hash from. Lets assume that below instance of the messageDigest is a valid instance of (SHA-256)
public static final BigInteger B = new BigInteger("BD0C61512C692C0CB6D041FA01BB152D4916A1E77AF46AE105393011BAF38964DC46A0670DD125B95A981652236F99D9B681CBF87837EC996C6DA04453728610D0C6DDB58B318885D7D82C7F8DEB75CE7BD4FBAA37089E6F9C6059F388838E7A00030B331EB76840910440B1B27AAEAEEB4012B7D7665238A8E3FB004B117B58", 16);
byte[] byteArrayBBigInt = B.toByteArray();
this.printArray(byteArrayBBigInt);
messageDigest.reset();
messageDigest.update(byteArrayBBigInt);
byte[] outputBBigInt = messageDigest.digest();
Now I only assume that the code below is correct, as according to the test the hashes I produce match with the one produced by:
http://www.fileformat.info/tool/hash.htm?hex=BD0C61512C692C0CB6D041FA01BB152D4916A1E77AF46AE105393011BAF38964DC46A0670DD125B95A981652236F99D9B681CBF87837EC996C6DA04453728610D0C6DDB58B318885D7D82C7F8DEB75CE7BD4FBAA37089E6F9C6059F388838E7A00030B331EB76840910440B1B27AAEAEEB4012B7D7665238A8E3FB004B117B58
However I am not sure why we are doing the step below i.e.
because the returned byte array after the digest() call is signed and in this case it is a negative, I suspect that we do need to convert it to a positive number i.e. we can use a function like that.
public static String byteArrayToHexString(byte[] b) {
String result = "";
for (int i=0; i < b.length; i++) {
result += Integer.toString((b[i] & 0xff) + 0x100, 16).substring(1);
}
return result;
}
thus:
String hex = byteArrayToHexString(outputBBigInt)
BigInteger unsignedBigInteger = new BigInteger(hex, 16);
When I construct a BigInteger from the new hex string and convert it back to byte array then I see that the sign bit, that is most significant bit i.e. the leftmost bit, is set to 0 which means that the number is positive, moreover the whole byte is constructed from zeros ( 00000000 ).
My question is: Is there any RFC that describes why do we need to convert the hash always to a "positive" unsigned byte array. I mean even if the number produced after the digest call is negative it is still a valid hash, right? thus why do we need that additional procedure. Basically, I am looking for a paper: standard or rfc describing that we need to do so.
A hash consists of an octet string (called a byte array in Java). How you convert it to or from a large number (a BigInteger in Java) is completely out of the scope for cryptographic hash algorithms. So no, there is no RFC to describe it as there is (usually) no reason to treat a hash as a number. In that sense a cryptographic hash is rather different from Object.hashCode().
That you can only treat hexadecimals as unsigned is a bit of an issue, but if you really want to then you can first convert it back to a byte array, and then perform new BigInteger(result). That constructor does threat the encoding within result as signed. Note that in protocols it is often not needed to convert back and forth to hexadecimals; hexadecimals are mainly for human consumption, a computer is fine with bytes.
I wrote a RSA encryption in Java. I am trying to turn the numbers that it outputs into text or characters. For example if I feed it Hello I get:
23805663430659911910
However, online RSA encryptions return something to the effect of this:
GVom5zCerZ+dmOCE7YAp0F+N3L26L
I would just like to know how to convert my numbers into something similar. The number returned by my system is a BigInteger. This is what I've tried so far:
RSA rsa = new RSA("Hello");
BigInteger cypher_number = rsa.encrypt(); // 23805663430659911910
byte[] cypher_bytes = cypher_number.toByteArray(); // [B#368102c8
String cypher_text = new String(cypher_bytes); // J^��*���
// Now even though cypher_text is J^��*��� I wouldn't care as long as I can turn it back.
byte[] plain_bytes = cypher_text.getBytes(); // [B#6996db8 | Not the same as cypher_bytes but lets keep going.
BigInteger plain_number = new BigInteger(plain_bytes); // 28779359581043512470254837759607478877667261
// plain_number has more than doubled in size compared to cypher_number and won't decrypt properly.
Using bytes it the only way I can think of. Can someone please help me understand what I'm supposed to be doing or if it's even possible?
This is generally a 2-step process:
convert to binary encoding of the number;
convert the binary encoding to a text base encoding.
For both steps there are multiple schemes possible.
For binary encoding: the PKCS#1 specifications have always included one that converts the number to a statically sized integer. To be precise, it describes the number into a statically sized, unsigned, big endian octet string. An octet string is nothing but a byte array.
Now, BigInteger.toByteArray returns a dynamically sized, signed, big endian octet string. So you need to implement the possible resizing and removal of initial 00 byte in a separate method, which I have at my other post here. Fortunately going back to a number is much easier as the Java implementation provides a BigInteger(int sign, byte[] value) constructor that reads in an unsigned number and skips leading zero bytes.
Having a standardized and statically sized octet string can be terribly useful, so I would not go for any other scheme.
This leaves the conversion to and from text. For that you can (indeed) use the java.util.Base64 class, which doesn't need much explaining. The only note that I must make is that it converts to an ASCII byte[] for some of the methods, so you need to use the encodeToString(byte[] src) instead.
Another method would be hexadecimals, but since Java doesn't contain a hex encoder for byte arrays in the base classes, I'd go for base 64 instead.
I have found the answer. In case you've found this looking for the answer, you just need to encode the numbers into Base64.
The following code converts the number into a dynamically sized, signed, big endian encoded integer, and then converts it back into a number using the reverse process.
// Encode
BigInteger numbers = new BigInteger("5109763");
byte[] bytes = Base64.getEncoder().encode(numbers.toByteArray());
String encoded = new String(bytes); // Encoded value
// Decode
byte[] decoded_bytes = Base64.getDecoder().decode(encoded.getBytes());
BigInteger numbers_again = new BigInteger(decoded_bytes); // Original numbers
I am using AES encryption algorithm in java to encrypt my database values..My encryption function returns encrypted value as String but the columns of type "Int" fails to store such string values which is quite logical..Is there a way to encrypt the integers as integers (numerical values)? Thankyou.
Plain AES returns an array of bytes. You can store this as an array of bytes, a Base64 text string or as a BigInteger:
BigInteger myBigInt = new BigInteger(AESByteArray);
It is very unlikely that the 128 bit, or larger, AES result will fit into a 32 bit Java int.
If you want 32 bit input and 32 bit output, so everything fits into a Java int, then either write your own 32 bit Feistel cipher, or use Hasty Pudding Cipher, which can be set for any bit size you require.
Encrypting integer into integer is FPE (format preserving encryption). FPE does not change data type or data length.
Here is a reason why databases implementing FPE only for character data, never for int.
AES 128 will encrypt 128-bit block. Which is 16 bytes.
If you want to encrypt 64 or 32 bit integer(4 or 8 byte values), you still have to encrypt 16 byte block. This problem can be solved by adding 8 (or 12) bytes to int32 or int64 values. This creates issue - if added bytes are always 0, you create huge weakness in encryption, as your data set is severely limited. It can be used for brute force attack on AES etc. In turn, this can be solved by filing with cryptographically strong random number added 8 or 12 bytes (that also creates a weakness, as most likely your random genertor is not strong enough). When decrypting, you can purge extra added bytes and extract only 4 or 8 bytes our of 16 bytes.
Still, life is not perfect. AES encryption does not change size of block, it always produces 16 bytes. You can encrypt your int into 16 bytes, but database can store only 8 bytes for int.
Unless you will store data in binary(16) column. But that is not an integer, and you are asking for integer.
In theory, numeric(38) is taking 16 bytes. In some databases it is possible to set 16 bytes to arbitrary value and then extract. I have not seen it is implemented.
You can always encode your string in an integer, however it could be a large integer.
If you can't afford large integer, you can encode it in multiple small integers.
If you can afford neither large integer nor multiple integers, maybe you can't do it well anyway, using a block cipher in ECB mode is almost always a bad idea.
Try converting the output of the encryption from string to binary, and then from binary to a decimal integer.
So, I'm using a proprietary library that has its own implementation for the creation of RSA key pairs. The public key struct looks like this:
typedef struct
{
unsigned int bits; //Length of modulus in bits
unsigned char modulus[MAX_RSA_MOD_LEN]; //Modulus
unsigned char exponent[MAX_RSA_MOD_LEN]; //Exponent
} RSA_PUB_KEY
I need to figure out a way to extract both the exponent and the module so I can send them to a server as part of a validation scheme. I guess that this is a pretty standard procedure (or so I hope). I've already read these two similar questions:
How to convert an Unsigned Character array into a hexadecimal string in C
Printing the hexadecimal representation of a char array[]
But so far I've had no luck. I'm also not sure of how to use if at all necessary the "bits" field to extract the modulus. In short what I have to do is be able to recreate this public key in Java:
BigInteger m = new BigInteger(MODULUS);
BigInteger e = new BigInteger(EXPONENT);
RSAPublicKeySpec keySpec = new RSAPublicKeySpec(m, e);
KeyFactory fact = KeyFactory.getInstance("RSA");
PublicKey pubKey = fact.generatePublic(keySpec);
return pubKey;
Edit:
This is what I'm doing right now: (RSAPublic is a RSA_PUB_KEY struct as described above).
//RSAPublic.bits = length of modulus in bits
log("Modulus length: "+std::to_string(RSAPublic.bits));
log("Key length: "+std::to_string(keyLengthInBits));
//Calculating buffer size for converted hexadec. representations
int modulusLengthInBytes = (RSAPublic.bits+7)/8 ;
int exponentLengthInBytes = (keyLengthInBits+7)/8;
char convertedMod[modulusLengthInBytes*2+1];
char convertedExp[exponentLengthInBytes*2+1];
//Conversion
int i;
for(i=0; i<modulusLengthInBytes ; i++){
sprintf(&convertedMod[i*2], "%02X", RSAPublic.modulus[i]);
}
for(i=0; i<exponentLengthInBytes ; i++){
sprintf(&convertedExp[i*2], "%02X", RSAPublic.exponent[i]);
}
//Print results
printf("Modulus: %s\n", convertedMod);
printf("Exponent: %s\n", convertedExp);
And this is the output:
Modulus length: 16
Key length: 512
Modulus: 0000
Exponent: 0A000200FFFFFFFFFFFF0000600007004DDA0100B01D0000AEC642017A4513000000000000000000000000000000000000000000000000000000000000000000
I'm assuming that you can't just send binary data since you mention the hexadecimal conversion. The most compact way you can send the data as text would be with base 64 but this is more complex than hexadecimal.
Client side
Convert the unsigned char array to a hexadecimal string using a method from the links you have. The bits field will determine how many bytes from the array to use given by (bits+7)/8.
Depending on implementation you might have to explicitly select the overflow bits or the rest might be zeroed, this also depends on the endianness so since you are unsure on implementation details you might have to fiddle around with it a bit.
Once you have the encoded strings, send them to the server.
Server side
Read the encoded strings from the connection and then pass them to the BigInteger(String val, int radix) constructor using the radix of hexadecimal (16).
You will then have A BigInteger with the value you require.
If the first bytes of the public exponent are all zero's then you are dealing with a big endian array. This is most common. In principle the public exponent can be as large as the modulus, but this is commonly not the case. Most common values are 65537, 17 and 3, maybe even 2 but the 3 and 2 are not such good values. Other 2-4 byte primes are also common.
Now if you know the endianness, you can have a look at the modulus. If the highest byte value is 00 then you are dealing with a signed representation of the modulus. Otherwise it is likely unsigned. The highest order byte of the modulus that contains bits should always be 80 or higher. The reason is that otherwise the key size would be smaller than the given key size. This is assuming that the key size is a multiple of 8 of course.
Java only works with big endian for BigInteger (and any other number representation). So if you have little endian encoding in C then you need to reverse the values in Java. It is probably the best to reverse the hexadecimal values in the string to accomplish that. Make sure you handle 2 hexadecimal characters at a time.
Then, as DrYap suggested, use the hexadecimal constructor of BigInteger. Note that if you end up using a byte array then you may want to use new BigInteger(1, MODULUS) as this makes sure you get a positive number regardless of the highest order bit value in the encoding.
Does anyone know how I would go about converting this java code to python?
/**
* signs the data for the account account
*/
private byte[] sign(String pkStr,byte[] data, String keyType) throws Exception {
BASE64Decoder decoder = new BASE64Decoder();
KeyFactory keyFac = null;
//instantiate the key factory based on the key alg type
if(keyType.equals("DSA")){
keyFac = KeyFactory.getInstance("DSA");
}else if(keyType.equals("RSA")){
keyFac = KeyFactory.getInstance("RSA");
}
//generate the public key
PKCS8EncodedKeySpec dprks = new PKCS8EncodedKeySpec(decoder.decodeBuffer(pkStr));
PrivateKey pk = keyFac.generatePrivate(dprks);
return(signBytes(data,pk,keyType));
}
/**
* sign the data with the key
*/
private byte[] signBytes(byte [] data,
PrivateKey signingPrivateKey, String keyType)throws Exception {
Signature dsa = null;
//instantiate the signature alg based on the key type
if(keyType.equals("DSA")){
dsa = Signature.getInstance("SHA1withDSA");
}else if(keyType.equals("RSA")){
dsa = Signature.getInstance("SHA1withRSA");
}
/* Initializing the object with a private key */
dsa.initSign(signingPrivateKey);
/* Update and sign the data */
dsa.update(data);
byte[] sig = dsa.sign();
return sig;
}
The "keyType" seems to always get passed as "DSA", so I've looked at M2Crypto.DSA and that looks promising. The DSA.sign function however returns a tuple of 2 byte strings that I'm not at all sure what to do with.
A DSA signature is defined as a pair of integers (called r and s, respectively). The DSA standard does not mandate a specific encoding of such a signature into a sequence of bytes. So every protocol which uses DSA signatures has to define its own encoding.
There are two commonly used DSA signature encodings; one is the direct concatenation of the big-endian unsigned encodings of the r and s values, both being normalized to the length (in bytes) of the q parameter in the public key (the "subgroup size", usually a 160-bit prime integer, thus yielding a 40-byte signature). The documentation of M2Crypto.DSA is quite terse, but my guess is that it returns r and s separately but already in that format.
Java uses the other encoding, which is ASN.1 based. This is the encoding used throughout X.509 and anything based on it (including signatures in SSL/TLS). ASN.1 is a generic standard for representation and serialization of structured data. In this case, the signature should be the serialization of an ASN.1 SEQUENCE containing two INTEGER values (r and s, in that order). As per the ASN.1 and DER encoding rules, the signature shall then have this format:
0x30 A 0x02 B R 0x02 C S
where:
R is the big-endian signed encoding of r, of minimal length: this means that the first byte shall have a value between 0 and 127, and it shall have value 0 only if the second byte has a value between 128 and 255. In other words, encode r as a sequence of bytes with big-endian convention (most significant byte comes first), making sure that you have as few leading zero bits as possible, provided that you keep at least one (this is what the "signed" encoding means: since r is positive, its most significant bit must be 0). Since r is an integer between 0 and q-1, the length of R will be at most one more byte than the length of q, but it can be smaller.
S is the big-endian signed encoding of s (same treatment than for r; note: R and S may have distinct lengths).
B is a single byte containing the length of R (in bytes).
C is a single byte containing the length of S (in bytes).
A is a single byte containing B+C+2 (i.e. the length, in bytes, of what follows the byte A).
Writing specialized encoding and decoding functions for ASN.1-based DSA signatures is a bit tedious but not hard; just take care of producing R and S sequences of the right size. Alternatively, you could use an existing ASN.1 encoding/decoding library, which is overkill but might be easier, depending on your situation.
According to http://download.oracle.com/javase/1.5.0/docs/guide/security/CryptoSpec.html#AppB (for some weird reason that has two Appendix Bs, and you need to scroll down to the second) Java uses the ASN.1 encoding SEQUENCE ::= { r INTEGER, s INTEGER }
You should be able to generate (and parse) this in Python using pyasn1 - http://pyasn1.sourceforge.net/
ASN.1 is a standard for encoding binary data. So the information above specifies how the Java code is combining the two values that the Python code returns. You can therefore do the same, and so maintain the same byte format for the signature.