In the application I'm writing I need to do HKDF to derive two different keys from one password. Searching for examples on how to it in Java I found these two:
https://github.com/WhisperSystems/libsignal-protocol-java/blob/master/java/src/main/java/org/whispersystems/libsignal/kdf/HKDF.java
https://www.javatips.net/api/keywhiz-master/hkdf/src/main/java/keywhiz/hkdf/Hkdf.java
In both cases HKDF is implemented on top of the HMAC provided by JCA. I haven't read those implementations in detail yet but I was wondering, is this not implemented anywhere in JCA or itself? Do I have to implement my own HKDF?
The part that worries me the most, is making a mistake in applying the info argument. It looks non-trivial and critical.
HKDF Implementations in Java
No, Hashed Message Authentication Code (HMAC)-based key derivation function (HKDF) has, like most KDFs, no standard implementation in JCA (as of 2020).
There are some implementations embedded in other projects (like you already said):
mozilla-services/sync-crypto
WhisperSystems/libsignal-protocol-java
square/keywhiz
Also there is, of course, Bouncy Castle which use their own Hmac/Mac implementations and APIs. BC is however a massive dependency, and may be unpractical for e.g. embedded or mobile use case. For this I implemented a standalone, lightweight java lib (passing all of the RFC 5869 test vectors), which works with any javax.crypto.Mac instance:
https://github.com/patrickfav/hkdf
If you prefer, you could, of course implement it on your own, it's a fairly straight forward spec, when using the built-in JCA Hmac implementation.
Info Parameter in HKDF
From the RFC 5869:
While the 'info' value is optional in the definition of HKDF, it is
often of great importance in applications. Its main objective is to
bind the derived key material to application- and context-specific
information. (...) In particular, it may prevent the derivation
of the same keying material for different contexts.
So for example if you want to derive an Secret Key and IV from the same source material you would use the info parameter (using this lib):
//example input
String userInput = "this is a user input with bad entropy";
HKDF hkdf = HKDF.fromHmacSha256();
//extract the "raw" data to create output with concentrated entropy
byte[] pseudoRandomKey = hkdf.extract(staticSalt32Byte, userInput.getBytes(StandardCharsets.UTF_8));
//create expanded bytes for e.g. AES secret key and IV
byte[] expandedAesKey = hkdf.expand(pseudoRandomKey, "aes-key".getBytes(StandardCharsets.UTF_8), 16);
byte[] expandedIv = hkdf.expand(pseudoRandomKey, "aes-iv".getBytes(StandardCharsets.UTF_8), 16);
//Example boilerplate encrypting a simple string with created key/iv
SecretKey key = new SecretKeySpec(expandedAesKey, "AES"); //AES-128 key
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
cipher.init(Cipher.ENCRYPT_MODE, key, new IvParameterSpec(expandedIv));
byte[] encrypted = cipher.doFinal("my secret message".getBytes(StandardCharsets.UTF_8));
Related
Preface: I don't know whether it's more appropriate to ask this question here or on the Crypto site. Feel free to move or delete or whatever the appropriate SE action is.
I've been asked to help update some encryption software. Broadly speaking, the software already does the following steps, none of which are particularly unusual. I've left out the error handling and Provider arguments for simplicity of posting:
1) Generates a random symmetric secret key for use with AES-128 (or AES-256, mutatis mutandis):
KeyGenerator keygen = KeyGenerator.getInstance("AES");
keygen.init (128, a_SecureRandom_instance);
SecretKey sessionKey = keygen.generateKey();
2) Wraps the symmetric secret key, depending on whether the user is using...
2a) ...an RSA public key from a keypair:
// OAEP wasn't used in this software for hysterical raisins
Cipher wrapper = Cipher.getInstance("RSA/ECB/PKCS1Padding");
wrapper.init (Cipher.WRAP_MODE, user_RSA_PublicKey);
2b) ...a passphrase:
SecretKey stretched = ...passphrase stretched through a PBKDF like bcrypt...;
// I don't remember whether it's specified as "AES" or "AESWrap" here
Cipher wrapper = Cipher.getInstance("AES or AESWrap/ECB/NoPadding");
wrapper.init (Cipher.WRAP_MODE, stretched);
2c) Either route, the session key is wrapped:
byte[] wrapped = wrapper.wrap(sessionKey);
3) The session key is used to create a Cipher using Cipher.ENCRYPT_MODE along with a random IV, and then shedloads of data are run through it. That part is pretty standard but I can post it if you really want to see CipherInputStream usage. The wrapped session key is stored along with the encrypted data and a bunch of HMACs of everything under the sun.
Later at decryption time, the user provides either the RSA private key, or the passphrase for stretching; the software unwraps the symmetric key and decrypts the data.
All of that has been working for some time. But of course RSA keypairs are getting big and slow, so they'd like to support an additional possibility for step #2 above, in which the public keys are generated using an elliptic curve algorithm (P384 ECDH is the usual case). And this is where we're getting confused.
There doesn't seem to be a JCE replacement algorithm/transformation for elliptic curves in the Cipher wrapper = Cipher.getInstance("RSA/ECB/PKCS1Padding") invocation. The only one listed in the Java documentation is "ECIES", which seems(?) to be aimed more towards multiple party key agreement?
All of the APIs that I can find for the Java builtin JCE, or even looking over Bouncy Castle, only mention ECDH keys in the context of key agreement and transport, where they're used to generate a symmetric secret key instead of wrapping an existing one.
I feel we're missing something here, possibly because of poor assumptions. Is Cipher.wrap() genuinely not an option with ECDH keys? Or it is, but we need to do something funky in order to create the ECIES Cipher instance?
RSA is an algorithm (or depending on how you look at it, two very similar but distinct algorithms) for encryption and signature. RSA encryption can encrypt dat which is a (lower-level) key, in which case it is called wrapping.
DH is a key agreement algorithm, in both its classical (aka integer, Zp, modp, or finite-field/FF) form and its elliptic-curve form (ECDH). Note that at least for classic DH the raw agreement value g^a^b mod n = g^b^a mod n has enough mathematical structure people aren't comfortable using it directly as a key, so we run it through a key derivation function, abbreviated KDF. It's not clear to me that ECDH values really need KDF, at least for the commonly used X9/NIST curves including P384 (you might look or ask on crypto.SX if you care), but using a KDF is cheap and well-established so we do so.
I've never heard anyone competent call (EC)DH key transport, because it is specifically NOT. It is included in key exchange, a term created to cover the (useful) union of transport and agreement.
You can create an encryption scheme using (EC)DH by using the agreed (and derived) value as the key for symmetric encryption -- and that is (EC)IES. I don't know what made you think IES is something else. Modern practice is that the symmetric encryption should be authenticated, and the standardized forms of IES as a separate scheme use CBC encryption with HMAC authentication, although you could validly design a scheme that uses e.g. GCM instead.
As you saw, the BouncyCastle provider provides implementations of DH (classic) or EC IES "with{AES,DESEDE}-CBC" (in versions before 1.56 the -CBC was sometimes omitted) which use KDF2 from P1363a with SHA1, the indicated CBC cipher, and HMAC-SHA1. The 'standard' (Sun/Oracle/Open) providers do not, but you can combine the raw agreement operation, the KDF, plus symmetric encryption and MAC to produce the same result.
This is similar (though different in some details) to the operation of CMS-formerly-PKCS7 with classic DH and ECDH and PGP with ECDH. (PGP does not support classic DH; it used ElGamal encryption instead as the alternative to RSA.) Not to mention TLS (sometimes through 1.2, always in 1.3) and SSH which use either classic or EC DH 'exchange' (here interactive, and usually ephemeral-ephemeral instead of having at least the receiver static) to produce a secret which is derived and used as key material for symmetric encryption with authentication of the data.
My qustion is about how should I specify the custom value of AlgorithmParameterSpec in the below program? So that I could use the exact same output value for two different programs, one of which is Server and other is Client. Like I have used the fixed value for Key Generation, I want the same for this AlgorithmParameterSpec.
Client Code Snippet
....
String desKey = "0123456789abcdef"; // value from user
byte[] keyBytes = DatatypeConverter.parseHexBinary(desKey);
SecretKeyFactory factory = SecretKeyFactory.getInstance("DES");
SecretKey key = factory.generateSecret(new DESKeySpec(keyBytes));
AlgorithmParameterSpec paramSpec = new IvParameterSpec(iv);
....
Both client and server are on different pcs and are connected on same LAN. However I dont want to send any file from one to another, and neither do I want to write the value of that parameter from client into any file and use it in server. That has already caused too much problems. Is there a way to do this? Or I have to send this generated value from client to server using readUTF and writeUTF?
Original DES aka single-DES has been broken since last century, and should not be used.
In Java, instantiating plain DES like that defaults to DES/ECB/PKCS5Padding (in general Java treats PKCS5 padding as including PKCS7). ECB does not use an IV, and DES has no other parameters, so a java Cipher object for DES/ECB does not need or use any type of parameters object. However, in most applications, ECB mode even with a good cipher (not DES) is insecure and should not be used.
If you change to a more secure mode that does use an IV, like CTR CBC OFB CFB, or better in most applications but only supported in j8+ GCM, then IvParameterSpec is indeed the correct type to use.
However, with modes that use an IV, using a fixed or otherwise duplicated IV is almost always insecure, so you must have a process that provides a unique IV for each encryption and the corresponding decryption(s).
This is often done by choosing a random IV when encrypting and transmitting and/or storing it along with the ciphertext (sometimes simply as the first part of the ciphertext) to be used when decrypting, but other designs that accomplish the same thing are possible. For CTR-based modes including GCM, especially when applied to a stream of traffic between the same parties, using a (scaled) counter provides uniqueness. For CBC the IVs must not only be unique but unpredictable; random is usually easiest for this but it is possible to use an encrypted counter.
Both key and IV (when used) should actually be bits not characters. However it is often convenient to represent them in characters using hex (as you did) or base64. Such character representations can be transmitted and received by a huge variety of methods, of which DataInput/OutputStream or ObjectInput/OutputStream is one among many. There are also many ways to transmit bits (binary) on all networks used in this century (although back in the 1970s and 1980s this was often problematical). Choosing among these may depend on what else your programs are doing and in particular whether they already use things like HTTP, XML, JSON, etc.
I see many examples where the secret key is generated this way:
KeyGenerator generator = KeyGenerator.getInstance("AES");
generator.init(KEY_LEN);
SecretKey secretKey = generator.generateKey();
I'm in doubt if there's a difference (even conceptual) with the following:
byte[] material = new byte[KEY_LEN / Byte.SIZE];
SecureRandom.getInstanceStrong().nextBytes(material);
SecretKey secretKey = new SecretKeySpec(material, "AES");
Stated that both methods are 3 lines, is there some practical impact in preferring the first over the second?
Thanks
You could look at the actual source code for generateKey() to see the difference but ultimately they are both going to do the same steps to generate an AES key. I would argue the latter
byte[] material = new byte[KEY_LEN / Byte.SIZE];
SecureRandom.getInstanceStrong().nextBytes(material);
SecretKey secretKey = new SecretKeySpec(material, "AES");
is a little more brittle for the average coder, requiring them to understand the SecureRandom class. If you eliminate the second line altogether the code runs just fine with an all zero key, an obvious vulnerability that's also easy for an attacker to check. Also, using generateKey() can produce a properly formatted key if the algorithm has some particular requirements. For example, the now obsolete DES and Triple DES algorithms had a weird parity bit in each byte that some DES implementations expected to see.
There are many reasons why you would want to use the KeyGenerator method, which was designed for the purpose:
readability: generateKey tells you exactly what the algorithm is doing;
portability: e.g. when choosing a different algorithm);
correctness: SecretKeySpec may not validate the key entirely;
security: you would leave the key material exposed in the material variable, which may not be cleared or even garbage collected after the key is not required anymore;
hardware support: importing key material is often not supported for hardware devices; keys should be generated on the device itself (using a specialized KeyFactory implementation).
There is no reason at all to use the second method. If you want to use a very specific random number generator for generating the key then you can use one of the specialized init methods but beware that this may not be compatible with hardware devices.
The two code examples seem to be doing the same and as already answered usually they are.
However using security devices (e. g. HSM, smartcards or other crypto devices) by default they won't allow exposing the key bytes so you would be able to generate a key from arbitritrary byte array only in very limited cases..
I'm using BouncyCastle to encrypt/decrypt some files using AES and PKCS5 padding in CBC mode :
Cipher c = Cipher.getInstance("AES/CBC/PKCS5Padding", "BC");
Now two questions:
How can I check that the provided key for decrypting data is correct or not ?
How Can I check encrypted input is untouched (e.g. not changed by user using an HEX editor)?
Thanks
You can use an AEAD mode, like CCM or GCM, in place of CBC. These modes authenticate an encrypted message, so if the wrong key is used, or the cipher text has been altered, you can detect it. You wouldn't be able to distinguish these cases though.
There is support in Java 7's cryptography API for GCM, but the SunJCE provider that ships with Oracle's Java implementation doesn't support it yet. You can get support through third-party providers like BouncyCastle.
You can achieve the same things if you use additional cryptographic services, like a digital signature or message authentication code.
Encryption is not just about the algorithm and the encryption key, it's also a lot about
the system organization.
In general, you can't determine that the key is correct. Any key can be used to decrypt the
data that's supposed to be decrypted, but it's up to some other mechanism to tell you if that
is the "correct" result.
In general, you can't determine if the data to be decrypted is untouched, except through some
external check. It's a property of most encryption systems that changing any of the encrypted
data would change the decrypted output drastically, probably into something you'd interpret
as garbage.
You should add a MAC which first verifies the integrity of the message, and only then you should decrypt it. A common choice of MAC is HMAC with whatever hash function you prefer, such as SHA-2.
Instead of doing this yourself, it's often a good idea to use an authenticated cipher. AES-GCM is a common choice. But you need to be really careful to never reuse an IV in that case.
The JCE ciphers are usually very basic. If you need a full featured protection including integrity and key testing, you need to combine them. And as usual it is better to not device that yourself. So better opt for a more high level format like PKCS7/12 or PGP.
Depending on the Padding used some ciphers will give you a PaddingException when you try to decrypt it with the wrong key. For stronger integrity check I would use a padding consiting of HMAC bytes.
A pretty complete method is included in the JCE, it is the AESWrap algorithm. It requires padded data but will ensure integrity. It is best combined with a length byte as described in RFC 3537. Note, that this is only intended for smaller amounts of secrets (like symmetric keys). The RFC3537 padding is restricted to 255 bytes.
To use this with a password derived key, you can use this:
char[] pass = ... // your password
byte[] codeBytes = ... // up to 255 bytes you want to protect
// generate wrapping key from password
SecretKeyFactory f = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
SecureRandom rand = SecureRandom.getInstance("SHA1PRNG");
byte[] salt = new byte[16]; rand.nextBytes(salt);
SecretKey kek = f.generateSecret(new PBEKeySpec(pass, salt, 1000, 128));
kek = new SecretKeySpec(password.getEncoded(), "AES"); // convert into AES
// RFC3537 padding (lengthbyte)
byte[] wrappedCodeBytes = new byte[codeBytes + 1 % 8];
System.arraycopy(codeBytes,0,wrappedCodeBytes,1,wrappedCodeBytes.length);
paddedCodeBytes[0]=(byte)codeBytes.length;
byte[] pad = new byte[paddedCodeBytes.length - codeBytes.length -1]; rand.nextBytes(pad);
System.arraycopy(pad,0,paddedCodeBytes,codeBytes.length+1,pad.length);
// AESWrap is WRAP_MODE:needs a SecretKey
SecretKey paddedCodeKey = new SecretKeySpec(paddedCodeBytes, "RAW");
// now wrap the password with AESWrap kek is 128 bit
Cipher c = Cipher.getInstance("AESWrap"); // default IV
c.init(Cipher.WRAP_MODE, kek);
byte[] result = c.warp(paddedCodeKey);
The unwrapping is left for the reader as an exercise :) The example code uses 128bit keysize, since more entropy cant be expected from the PBKDF2 anyway.
Note that this will detect wrong passwords with high probability, and some critics will see this as a weakness of AESWrap.
Take a look at this tutorial on BC encryption, specifically the InitCiphers methods, and in detail at the second code block which specifies the actual type of cipher.
How can I check that the provided key for decrypting data is correct or not?
According to JCE Javadocs, specifically the constructor of Class SecretKeySpec:
This constructor does not check if the given bytes indeed specify a secret key of the specified algorithm. For example, if the algorithm is DES, this constructor does not check if key is 8 bytes long, and also does not check for weak or semi-weak keys. In order for those checks to be performed, an algorithm-specific key specification class (in this case: DESKeySpec) should be used.
Note that Interface KeySpec lists all implementing classes, basically a list of validation options.
How Can I check encrypted input is untouched (e.g. not changed by user using an HEX editor)?
Indeed. That's a good one. 'Input' is pretty generic. Do you mean the actual content to decrypt? Well, if it's munged I believe it will not decrypt properly. Does that make sense?
IFF you are talking about the case of a key with parity bits being altered, as described in item (6) at the Bouncy Castle FAQ, you will have to do an actual parity check on the key. Only the first 56 bytes of the key are used for the encryption ops, and the last 8 bytes are reserved for parity checking. So, essentially, the last part of the 'key' can be changed and the first part is still useful. To detect whether either the parity or the key have been altered, you would run a parity check. I found this little ditty on doing a parity check. And, for more info on how parity is set in these keys, see comments in the JDK7 Crypto Provider source for Class DESKeyGenerator by Jan Luehe (near bottom) which discuss parity setting.
I recently had some interaction with BC, and I hope this info helps.
What's happening in this line of code ?
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
I specially don't understand getInstance("PBKDF2WithHmacSHA1") part
This funky looking string defines the secret-key algorithm to be used. It is:
PBKDF2WithHmacSHA1
PBKDF2 With Hmac SHA1
the PBKDF2 function (from PKCS#5 version 2.0)
which will be using SHA-1 HMAC for its pseudo-random number generator
References:
We find similar algorithm names in Java Crypto Extension Guide Appending A, somehow PKCS5 version 2 may not have been available/documented then (or indeed as suggested by brianegge, may be a matter of export restriction, a common issue with cryptographic items).
The algorithm name does show up in RFC3962 (AES for Kerberos) which may not be the very application you have in mind, but defined, all the same)
Different distributions of Java contain different crypto. This is due to export restrictions and patents. The line of code is requesting a factory which can create that type of key.
Specifically, PBKDF2WithHmacSHA1 constructs secret keys using the Password-Based Key Derivation Function function found in PKCS5 v2.0.
"PBKDF2" is a function defined in PKCS #5 used to derive key material from a password.
PBKDF2 requires a pseudo-random function, and in this case, a message authentication code based on the SHA-1 hash is used—"HmacSHA1".
So, this line is creating a factory. The factory might produce SecretKey objects that can be used to key a Cipher instance for a symmetric encryption algorithm or a Mac algorithm. Or, it can be used to make a "transparent" specification of an existing SecretKey.
One important thing to note about PBKDF2 is that it doesn't produce secret keys for any particular algorithm. It's a deterministic way to generate key "material" from a seed (a password), in such a way that the seed cannot be recovered from the generated key. Once the required number of bytes are generated, they are usually wrapped in a SecretKeySpec with the correct algorithm name.
You can see other standard names for secret key factories in the Java Crypto Architecture Standard Names documentation.