Develop Reference

Bouncy Castle is throwing "mac check in OCB failed" when trying to decrypt an AES message

I need to decrypt an AES message. I was able to make it work on python(using the pycryptodome library) but I wasn't successful on kotlin/java. Since the decryption mode is OCB and Java does not support it by default, I had to use the Bouncy Castle library.
Here's the working python code:
def decrypt_aes_message(shared_key, encrypted_message):
encrypted_msg = b64decode(encrypted_message["encryptedMessage"].encode())
tag = b64decode(encrypted_message["tag"].encode())
nonce = b64decode(encrypted_message["nonce"].encode())
cipher = AES.new(shared_key.encode(), AES.MODE_OCB, nonce=nonce)
return cipher.decrypt_and_verify(encrypted_msg, tag).decode()
Here's the java code:
fun decryptAesMessage2(sharedKey: String, encryptedMessageData: Map<String, String>): ByteArray {
var encryptedMessage = encryptedMessageData["encryptedMessage"]!!.utf8Base64Decode()
var tag = encryptedMessageData["tag"]!!.utf8Base64Decode()
var nonce = encryptedMessageData["nonce"]!!.utf8Base64Decode()
var key = KeyParameter(sharedKey.toByteArray(Charsets.UTF_8))
var params = AEADParameters(key, tag.size*8, nonce)
var cipher = OCBBlockCipher(AESEngine(), AESEngine())
cipher.init(false, params)
val out = ByteArray(cipher.getOutputSize(encryptedMessage.size))
var offset = cipher.processBytes(encryptedMessage, 0, encryptedMessage.size, out, 0)
offset += cipher.doFinal(out, offset) // Throwing exception here
return out
}
The java code is throwing the exception org.bouncycastle.crypto.InvalidCipherTextException: mac check in OCB failed on cipher.doFinal
The file debug.zip has the complete problem reproducer.
Inside the zip file you'll find:
py_working_code.py - the working python script(needs pycryptodome in order to work. You can install pycryptodome using pip install pycryptodome)
bc-debug - gradle project reproducing the problem

There are two issues, one is a bug in your Kotlin code, the other is a library bug:
Bug in Kotlin code
While PyCryptodome processes ciphertext and tag separately, BC/Kotlin expects the concatenation of both in the order: ciphertext|tag.
Therefore the line encryptedMessage += tag must be added in the Kotlin code:
fun decryptAesMessage2(sharedKey: String, encryptedMessageData: Map<String, String>): ByteArray {
var encryptedMessage = encryptedMessageData["encryptedMessage"]!!.utf8Base64Decode()
var tag = encryptedMessageData["tag"]!!.utf8Base64Decode()
encryptedMessage += tag // Fix
var nonce = encryptedMessageData["nonce"]!!.utf8Base64Decode()
var key = KeyParameter(sharedKey.toByteArray(Charsets.UTF_8))
var params = AEADParameters(key, tag.size*8, nonce)
var cipher = OCBBlockCipher(AESEngine(), AESEngine())
cipher.init(false, params)
val out = ByteArray(cipher.getOutputSize(encryptedMessage.size))
var offset = cipher.processBytes(encryptedMessage, 0, encryptedMessage.size, out, 0)
offset += cipher.doFinal(out, offset) // Throwing exception here
return out
}
Test: Below, identical test data is successfully decrypted using the Python code and the fixed Kotlin code:
Python:
encrypted_message = {
'encryptedMessage': 'LzoelJ9Nv4cruj0JUlxFrNR+mqyO2rvwqDHYwnj0OkvJ+BBvug+ORYVkxA==',
'tag': 'hl56drXePWiLkVavVwF3/w==',
'nonce': b64encode(b'012345678901').decode()
}
dt = decrypt_aes_message('01234567890123456789012345678901', encrypted_message)
print(dt) # The quick brown fox jumps over the lazy dog
Kotlin:
val encrypted_message = mutableMapOf<String, String>()
encrypted_message["encryptedMessage"] = "LzoelJ9Nv4cruj0JUlxFrNR+mqyO2rvwqDHYwnj0OkvJ+BBvug+ORYVkxA=="
encrypted_message["tag"] = "hl56drXePWiLkVavVwF3/w=="
encrypted_message["nonce"] = Base64.getEncoder().encodeToString("012345678901".toByteArray(Charsets.UTF_8))
val dt = decryptAesMessage2("01234567890123456789012345678901", encrypted_message)
println(String(dt, Charsets.UTF_8)) // The quick brown fox jumps over the lazy dog
Library bug
Another problem is that both implementations produce different results for a nonce length of 120 bits (15 bytes), the maximum allowed nonce length for OCB (see RFC 7253, 4.2. Encryption: OCB-ENCRYPT):
The following test uses a fixed key and plaintext and compares the results between the Python and BC/Kotlin code for nonce lengths 13, 14, and 15 bytes:
Key: b'01234567890123456789012345678901'
Plaintext: b'testmessage'
Nonce Länge 13 bytes 14 bytes 15 bytes
Nonce b'0123456789012' b'01234567890123' b'012345678901234'
Python (ct|tag) 0xb35a69a245ab18fe3b6bae38b179c2a43b341f67c0451256b76bd7 0xff9be97fcb6e1ac57e6997bc3e84598a83ab70947ccac500fcf75e 0xff9be97fcb6e1ac57e6997bc3e84598a83ab70947ccac500fcf75e
BC/Kotlin 0xb35a69a245ab18fe3b6bae38b179c2a43b341f67c0451256b76bd7 0xff9be97fcb6e1ac57e6997bc3e84598a83ab70947ccac500fcf75e 0xa4355068324065f2ad194b058bdb86caa67c225b99021dbd588034
The Python implementation returns the same ciphertext/tag for nonce lengths 14 and 15, while the results for BC/Kotlin differ. This indicates a bug in the Python implementation.
Unfortunately, RFC 7253, Appendix A. Sample Results only provides test vectors whose nonces are all 12 bytes in size, so the bug cannot be assigned more clearly.
I.e. if you use a 15 bytes nonce, both implementations are not compatible, where the problem is most likely caused by the Python implementation.
Edit
Analysis of linked example:
You are affected by both bugs. According to your comment, you have already fixed the bug in the Kotlin code (concatenation of ciphertext and tag). Since the nonce you used in the example is 15 bytes (knQgYf1MsOs8smx9GtWM corresponds Base64 decoded to 0x92742061fd4cb0eb3cb26c7d1ad58c) the bug described in the second part of my answer is the cause of the problem. This bug is not in your code, but in one of the libraries, most likely in the Python library. Therefore you cannot fix it (at least not without greater effort).
Workarounds
As shown in the above test, the Python code for a nonce length of 15 bytes seems to simply ignore the 15th byte, i.e. if you use the 14 bytes nonce knQgYf1MsOs8smx9GtU= (0x92742061fd4cb0eb3cb26c7d1ad5) in your Python code, the Python code returns the same ciphertext and the same tag as with the 15 bytes nonce knQgYf1MsOs8smx9GtWM, which is why decryption with the 14 bytes nonce is also possible:
key = "f009Cip5hM4Obbb6E2MT5npJBHlc82vD"
message_data = {"encryptedMessage":"XMQx/xbVVTbMdpMiTXVp5XPICm11Vw2pgALpVI0NgbdqLLmikhPuu9M+qQzyOVZlZZBRlscijpyAZDsLGcTSPP54O35oKNp//PuOrWsN/ZZMkCByKCSBysJLRiZV1OjZDg01gi5/nYNbUgGGd8uRGKfBaKjjXngZ1J89GOvDeWPQcjbfbdzd9w+jbZGZ5jnAIChOL1Uqohf+6KHtjR/H06fFTHwB1abzAQrGbCNBNXBmN9+zEu7Auy3NPWKrZ+SL5Nk=","tag":"ZcqXSBqYU5TjgdMC+bMeUQ==","nonce":"knQgYf1MsOs8smx9GtU="}
decrypted_message = decrypt_aes_message (key, message_data)
print (decrypted_message) # https://app.passiv.com/snapTrade/redeemToken?token=v9uJsXYsi%2B6s9kyohisc6DFntJ/yD6m/2zhmO5xp6Vmezcyi8nwx63YtkqnnaogZvFmqs7L99EtZ0mxN9mAQTNoThHj3GaypXXUdiQIzig%3D%3D&clientId=SCANZ&broker=ALPACA
If you use this 14 bytes nonce knQgYf1MsOs8smx9GtU= in the Kotlin code, the decryption is also successful. This is the workaround for this specific example!
The general workaround, as long as the library bug is not fixed, is not to use a 15 bytes nonce, but only a maximum 14 bytes nonce!
On this website there are also testvectors with 15 bytes nonces listed. As expected, these are not met by the PyCryptodome implementation. I have filed an issue on PyCryptodome based on one of the 15 bytes nonce test vectors: Issue #664.

java.security.InvalidKeyException thrown only for older java versions during AES decryption

My goal is to encrypt cache assets (in this case 3D models using a custom format) that are being decrypted by a client distributed to users. For this I used the following implementation:
public class AESTestStackoverflow {
private final static byte[] secretKey = new byte[]{-74, 80, 22, 62, -70, -117, 22, 110, 57, -51, 2, 70, 68, -29, 14, -100, -24, 121, -122, 81, 5, 23, -90, 78, -99, -116, 29, -38, 118, 121, 126, 51};
SecretKeySpec secretKeySpec;
#Before
public void setUp() {
secretKeySpec = new SecretKeySpec(secretKey, "AES");
}
#Test
public void testDecryption() {
final byte[] raw = new byte[]{1,2,3,4,5,6,7,8,9,10};
final EncryptionResult result = AES.encrypt(raw, secretKeySpec);
Assert.assertNotNull(result);
final byte[] encrypted = result.getData();
final byte[] iv = result.getIv();
final byte[] decrypted = AES.decrypt(encrypted, iv, secretKeySpec);
Assert.assertArrayEquals(raw, decrypted);
}
}
This implementation works perfectly fine for the latest java versions but fails for java versions equal to or below 1.8.131 (not sure on the exact version that it breaks, but the latest java versions in the 1.8.2xx range appear to work fine). Users running these java versions get the following exception:
java.security.InvalidKeyException: No installed provider supports this key: javax.crypto.spec.SecretKeySpec
at javax.crypto.Cipher.chooseProvider(Cipher.java:893)
at javax.crypto.Cipher.init(Cipher.java:1396)
at javax.crypto.Cipher.init(Cipher.java:1327)
I did do some research on this error and I found several posts stating this might have to do with some part of the JCE library not being shipped with the JRE, and the provided solution was to manually install these into your JRE.
My manager does not want to force users to update their java version, and obviously we cannot ask users to modify their JRE manually, so I am at a bit of a loss on how to continue. I have tried doing some more research but I am not well versed in the security libraries or any related JVM settings. I would appreciate it if someone can shed some light on why this happens, and if it can be made compatible for older java version (from 1.8.xxx onwards).

I didn't test you your code but my simple program will test for something that is called "unlimited crypto policies". Due to US export restrictions
the US companies were not allowed to ship their programs with an unlimited key format. You are using AES with a 32 bytes long key so it is called
AES-256 (32 bytes * 8 bit = 256) but the government allowed only 16 bytes long keys ("AES 128").
My program runs the test depending on the Java version that is in use. For an easier test I setup a JFiddle-link where you can test the program directly
and for convinience change the Java version. The JFiddle is available under this link: https://www.jdoodle.com/a/2nZe
When running the test the Java version is JDK 11.0.4 but you can change (please scroll down) to JDK 1.8.0_66.
output for Java 11:
Check for unlimited crypto policies
restricted cryptography: false Notice: 'false' means unlimited policies
Security properties: unlimited
Max AES key length = 2147483647
output for Java 1.8.0_66:
restricted cryptography: true Notice: 'false' means unlimited policies
Security properties: null
Max AES key length = 128
To enable the unlimited policy for older Java's like 1.8.0_66 your clients have to load two small files from Oracle's pages and place them in their Java VM.
As you stated your manager does not allow this there is only one way left - you have to use (only) 16 byte long AES keys.
Security warning: due to upcoming Quantum computers the actual recommendation for AES keys is 32 byte long keys.
code:
import javax.crypto.Cipher;
import java.security.NoSuchAlgorithmException;
import java.security.Security;
public class UnlimitedCryptoPoliciesCheck {
public static void main(String[] args) throws NoSuchAlgorithmException {
System.out.println("Check for unlimited crypto policies");
//Security.setProperty("crypto.policy", "limited"); // muss ganz am anfang gesetzt werden !
System.out.println("restricted cryptography: " + restrictedCryptography() + " Notice: 'false' means unlimited policies"); // false mean unlimited crypto
System.out.println("Security properties: " + Security.getProperty("crypto.policy"));
int maxKeyLen = Cipher.getMaxAllowedKeyLength("AES");
System.out.println("Max AES key length = " + maxKeyLen);
}
/**
* Determines if cryptography restrictions apply.
* Restrictions apply if the value of {#link Cipher#getMaxAllowedKeyLength(String)} returns a value smaller than {#link Integer#MAX_VALUE} if there are any restrictions according to the JavaDoc of the method.
* This method is used with the transform <code>"AES/CBC/PKCS5Padding"</code> as this is an often used algorithm that is an implementation requirement for Java SE.
*
* #return <code>true</code> if restrictions apply, <code>false</code> otherwise
* https://stackoverflow.com/posts/33849265/edit, author Maarten Bodewes
*/
public static boolean restrictedCryptography() {
try {
return Cipher.getMaxAllowedKeyLength("AES/CBC/PKCS5Padding") < Integer.MAX_VALUE;
} catch (final NoSuchAlgorithmException e) {
throw new IllegalStateException("The transform \"AES/CBC/PKCS5Padding\" is not available (the availability of this algorithm is mandatory for Java SE implementations)", e);
}
}
}

Is it possible to lower the encryption level to avoid the error: Unable to initialize due to invalid secret key

import org.junit.Test;
import org.springframework.security.crypto.encrypt.Encryptors;
import org.springframework.security.crypto.encrypt.TextEncryptor;
public class EncryptorTest {
#Test
public void testEncryption() {
TextEncryptor te = Encryptors.text("password", "abcdef");
te.encrypt("Hello World!");
}
}
I get this error:
java.lang.IllegalArgumentException: Unable to initialize due to invalid secret key
It seems that I have to download some policy file JCE extension to get this to work: http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html
I want to avoid this added dependency during deployment, and honestly I don't need 256 bit encryption for this.
Is it possible to lower the level to something that works w/o having to download this JCE extension?

No the value 256 is hard coded into the source:
PBEKeySpec keySpec = new PBEKeySpec(password.toCharArray(), Hex.decode(salt), 1024, 256);
This can be found in AesBytesEncryptor on line 54 at least up to release 3.2.5.
Note that Spring uses the Apache license, so there is nothing stopping you from creating your own implementation that uses 128 as constant.

Public Key exchange between iOS and Java

I would like to share a public key generated on an iPhone/iPad with an Java based Server or an Android device. On the Java side (Server or Android) I would like to use java.security and libcommonCrypto (the SecKeyRef stuff) on iOS.
I've got everything working except the public key exchange between these two platforms. By using SecItemCopyMatching I can only export the public key into some format not supported by anything else then Apple. With Java I can load public keys as X509 certificate (through java.security.X509EncodedKeySpec) or as module and exponent (through java.security.RSAPublicKeySpec).
Now I need to know how to export a SecKeyRef as X509 certificate or (what I guess is the easier solution) get the module and exponent from it. And I also need the way backwards.
Here is some sample public key loaded from a SecKeyRef and encoded to BASE64:
MIGJAoGBAMYgXP6rvD/Y8F0VQE0HvxpVnnOxXYl5TDlOfW/leyrCLWGWg9Jp+Tl9dYvK/zWgNpoEfFzMVRpUk9UHcIaDWHW3g0BpS2MVC3Vs/0e2eu6S2WMGHpzqcJB51jJRbnqXQ23nVKC2YE520Po3EvFyTr8MlFJqTCJrovgc7fze4RI5AgMBAAE=

The protocol Apples libcommonCrypto is using is described in RFC3347 and is the modulus and public exponent as a ASN.1 sequence.
An RSA public key should be represented with the ASN.1 type RSAPublicKey:
RSAPublicKey ::= SEQUENCE {
modulus INTEGER, -- n
publicExponent INTEGER -- e
}
Quote from RFC3347, Ver. 2.1, A.1.1

Java: Patching client side security policy from applet for AES256

I require AES256 encryption/decryption in a commercial web application.
Currently everything is good with a key size of 128. This is not satisfactory cryptographically so my problem is how best to get round this issue without requiring the user to install anything manually.
I have the unlimited jurisdiction jar files from Oracle but I have no idea if replacing these in the user's JRE/lib/security directory will be compatible with older versions. Obviously I don't want to corrupt the user's JRE. Also I have write permission to my JRE security directory but I assume some user's will not have these privileges.
Is there a simple way around this issue, or am I stuck with either weak encryption or a potentially problematic step for users?
Update for "unrestricting" javax.crypto.JceSecurity
#ntoskml You are correct. getMaxAllowedKeyLength still returns the limited key size but the encryption succeeds with key size == 256 :). I will update my test method and set the key size if strong encryption is available. Thanks
>>> from javax.crypto import Cipher
>>> Cipher.getMaxAllowedKeyLength("AES")
128
>>> from java.lang import Class
>>> c = Class.forName("javax.crypto.JceSecurity")
>>> isRestricted = c.getDeclaredField("isRestricted")
>>> isRestricted.setAccessible(True)
>>> isRestricted.set(None, False)
>>> isRestricted.get(None)
False
>>> Cipher.getMaxAllowedKeyLength("AES")
128
>>> from javax.crypto import KeyGenerator
>>> kge = KeyGenerator.getInstance("AES")
>>> kge.init(256)
>>> aesKey = kgen.generateKey()
>>> c2 = Cipher.getInstance("AES")
>>> c2.init(Cipher.ENCRYPT_MODE, aesKey)
>>> c2.doFinal("test")
array('b', [-81, 99, -61, -51, 93, -42, -68, -28, 107, 59, -109, -98, -25, 127, 37, 23])
And the test case after restarting Jython console
>>> # Reflection as above
>>> isRestricted.get(None)
True
>>> kge.init(256)
>>> aesKey = kge.generateKey()
>>> c2.init(Cipher.ENCRYPT_MODE, aesKey)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
at javax.crypto.Cipher.checkCryptoPerm(Cipher.java:1011)
at javax.crypto.Cipher.implInit(Cipher.java:786)
at javax.crypto.Cipher.chooseProvider(Cipher.java:849)
at javax.crypto.Cipher.init(Cipher.java:1213)
at javax.crypto.Cipher.init(Cipher.java:1153)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
java.security.InvalidKeyException: java.security.InvalidKeyException: Illegal key size or default parameters
Bingo :) Thanks for sharing #ntoskml

EDIT: Here's an updated answer to this question: How to avoid installing "Unlimited Strength" JCE policy files when deploying an application?
It is possible to disable the key size restrictions simply by using a few lines of reflection. We use this method in our program which needs access to 256-bit cryptography for interoperability purposes.
private static void removeCryptographyRestrictions() {
if (!isRestrictedCryptography()) {
return;
}
try {
java.lang.reflect.Field isRestricted;
try {
final Class<?> c = Class.forName("javax.crypto.JceSecurity");
isRestricted = c.getDeclaredField("isRestricted");
} catch (final ClassNotFoundException e) {
try {
// Java 6 has obfuscated JCE classes
final Class<?> c = Class.forName("javax.crypto.SunJCE_b");
isRestricted = c.getDeclaredField("g");
} catch (final ClassNotFoundException e2) {
throw e;
}
}
isRestricted.setAccessible(true);
isRestricted.set(null, false);
} catch (final Throwable e) {
logger.log(Level.WARNING,
"Failed to remove cryptography restrictions", e);
}
}
private static boolean isRestrictedCryptography() {
return "Java(TM) SE Runtime Environment"
.equals(System.getProperty("java.runtime.name"));
}
However, our program is not an applet, and I am not sure whether applets have access to the reflection API.
The question about legality also remains. There is a reason for that limit. Consult a lawyer if you are concerned.
If possible, try to keep it to 128-bit keys. Even when taking Moore's law into consideration, breaking 128-bit AES would take billions upon billions of years. Longer keys offer no benefit in the real world – particularly when the keys are derived from passwords, which don't have anywhere near 256 bits of entropy anyway.

You are either stuck with the weak encryption or a potentially problematic step for users if you stick to the SunJCE.
There is obviously no problem importing an AES library, there is just a problem using it using an instance of Cipher. If you have a specific piece of software that does not depend on JCA, you can for instance rewrite it to use the lightweight crypto API of Bouncy Castle.
Note that many other parts of the Bouncy API themselves depend on the JCE. The lightweight API is also trickier to use and less documented/tested than the SunJCE.
The Bouncy Castle lightweight API is pretty large as well. It contains a lot of functionality that you won't need. So it is probably too large for your applet. If it is I would advice you to create a new library that only contains the specific classes that you need from Bouncy Castle. The Bouncy Castle is fortunately very liberally licensed. As long as you keep the copyright statements etc. in place, you can easily split it off.