Develop Reference

Reading encrypted private key in PKCS#8 format through bouncycastle, Java failing in docker container

I am trying to read a PKCS#8 private key which looks like following:
key.k8 --> (Sample key. Passphrase - 123456):
-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIFLTBXBgkqhkiG9w0BBQ0wSjApBgkqhkiG9w0BBQwwHAQILbKY9hPxYSoCAggA
MAwGCCqGSIb3DQIJBQAwHQYJYIZIAWUDBAEqBBCvaGt2Hmm2NpHpxbLvHKyOBIIE
0IQ7dVrAGXLZl0exYIvyxLAu6zO00jL6b3sb/agTcCFOz8JU6fBanxY0d5aYO4Dn
mynQG7BoljU470s0zIwW/wk0MmdUFl4nXWBX/4qnG0sZqZ9KZ7I8R/WrBkmpX8C/
4pjdVhu8Ht8dfOYbkbjMBTohDJz8vJ0QwDIXi9yFjjef+QjwrFOl6kAeDJFVMGqc
s7K/wOnhsL1XxfW9uTulPiZh5YTZKcatMkeGDR7c+cg5I+Mutim92diWuCekhNoa
uvhUy1M3cbs7Azp1Mhz+V0CDKklI95EvN4u23WhiJPCjAofC/e45/heOP3Dwm7WZ
zHEY1C/X8PsTl6MEEIF3ZJP+4Vr0corAs1L2FqE6oOng8dFFYmF5eRyBx6bxFd05
iYbfOH24/b3qtFKPC689kGEd0gWp1dwES35SNNK+cJqVRTjgI0oKhOai3rhbGnmp
tx4+JqploQgTorj4w9asbtZ/qZA2mYSSR/Q64SHv7LfoUCI9bgx73MqRQBgvI5yS
b4BoFBnuEgOduZLaGKGjKVW3m5/q8oiDAaspcSLCJMIrdOTYWJB+7mfxX4Xy0vEe
5m2jXpSLQmrfjgpSTpHDKi/3b6OzKOcHjSFBf8IoiHuLc5DVvLECzDUxxaMrTZ71
0YXvEPwl2R9BzEANwwR9ghJvFg1Be/d5W/WA1Efe6cNQNBlmErxD6l+4KDUgGjTr
Aaksp9SZAv8uQAsg7C57NFHpTA5Hznr5JctL+WlO+Gk0cAV6i4Py3kA6EcfatsnS
PqP2KbxT+rb2ATMUZqgWc20QvDt6j0CTA1BuVD1PNhnAUFvb2ocyEEXOra22DPPS
UPu6jirSIyFcjqFjJ9A1FD9L4/UuX2UkDSLqblFlYB1+G55KZp+EKz8SZoN5qXy1
LyMtnacEP5OtRDrOjopzVNiuV1Uv63M9QVi1hZlVLJEomgjWuvuyEuIwDaY2uryW
vx+jJEZyySFkb1JwAbrm+p6sCTFnbQ/URKC2cit/FJyKqNim6VQvGL8Sez34qV3z
D13QJgTZfsy+BaZoaQ6cJTXtJ8cN0IcQciOiDNBKMW66zO6ujS8G+KNviNQypDm6
h4sOgjMqLaZ4ezPEdNj/gaxV7Y15nVRu0re8dVkaa5t9ft/sh6A+yeTD5tS5hHkf
NI7uJPTaTXVoz7xq2PAJUTWujMLMZKtmNOzNqYvxWRy3tCOFobBQkMxqEBEwHd+x
SA+gFcJKJ+aNfCGZJ5fFr8rNlhtOF6uMwOAlfiUlP/pCUDUCKPjZVj4K95yNc8Io
jSZSPb5tGPe0HqXgc6IAfQarlUZt90oVtzL0OfOfTxe1bEzS2ccNadbx/6vjLBc4
q5UuUBppl3rXpbuZ7J1Rp3n2byF4APxFdT2LHKq+MYMfWUToau/TCMT4lFIM9tM8
7TuuyUT2PKzf/xlsl4iScw96z9xxGPQrXn7IA2W5iL+0eCLztJdjNRX1FisdfIBL
PraOVlmF8jHKbFdRZ8Yi8pApbQjvHi24g7dX7u/cq1FH/VE+nJ0O8YVCYVDw13CW
h0p7yD7BuB0R+0WnR0yvkp30vK4/rtCB+Ob8bH/+HvAZrAU5X8jq/wsQbLkrLHZV
6A6GGfX8+hy5AoaXsH1BHnMyXkaF6Mv29z8JcslDJxX/
-----END ENCRYPTED PRIVATE KEY-----
Following code is being used to parse the private key:
InputStream privateKeyInputStream = getPrivateKeyInputStream(); // reads the key file from classpath and share as DataStream
logger.info("InputStreamExists --> {} ", privateKeyInputStream.available());
PEMParser pemParser = new PEMParser(new InputStreamReader(privateKeyInputStream));
Object pemObject = pemParser.readObject();
if (pemObject instanceof PKCS8EncryptedPrivateKeyInfo) {
// Handle the case where the private key is encrypted.
PKCS8EncryptedPrivateKeyInfo encryptedPrivateKeyInfo = (PKCS8EncryptedPrivateKeyInfo) pemObject;
InputDecryptorProvider pkcs8Prov =
new JceOpenSSLPKCS8DecryptorProviderBuilder().build(passphrase.toCharArray());
privateKeyInfo = encryptedPrivateKeyInfo.decryptPrivateKeyInfo(pkcs8Prov); // fails here
}
InputStream resourceAsStream = null;
if ("local".equals(privateKeyMode)) {
resourceAsStream = this.getClass().getResourceAsStream(privateKeyPath);
} else {
File keyFile = new File(privateKeyPath);
logger.info(
"Key file found in {} mode. FileName : {}, Exists : {}",
privateKeyMode,
keyFile.getName(),
keyFile.exists());
try {
resourceAsStream = new DataInputStream(new FileInputStream(keyFile));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
When I am running this code through intelliJ on windows, the code works fine but when I run it through docker container I am getting following exception:
org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: failed to construct sequence from byte[]: Extra data detected in stream
snowflake-report-sync | at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[bcpkix-jdk15on-1.64.jar!/:1.64.00.0]
snowflake-report-sync | at com.optum.snowflakereportsync.configuration.SnowFlakeConfig.getPrivateKey(SnowFlakeConfig.java:103) ~[classes!/:na]
snowflake-report-sync | at com.optum.snowflakereportsync.configuration.SnowFlakeConfig.getConnectionProperties(SnowFlakeConfig.java:67) ~[classes!/:na]
Following is Dockerfile used:
FROM adoptopenjdk/openjdk11-openj9:latest
COPY build/libs/snowflake-report-sync-*.jar snowflake-report-sync.jar
RUN mkdir /encryption-keys
COPY encryption-keys/ /encryption-keys/ #keys are picked from docker filesystem when running in container
EXPOSE 8080
CMD java -Dcom.sun.management.jmxremote -noverify ${JAVA_OPTS} -jar snowflake-report-sync.jar
Options tried:
Ensured that key file is being read while running in container. Logger "InputStreamExists --> {}" gives number of bytes
Ran dos2unix on key.k8 just to make sure there are no Window's "^M" characters which be could be causing issue as container is linux one : FROM adoptopenjdk/openjdk11-openj9:latest
Not sure what I am doing wrong but any help or pointers would be appreciated.

Like #Bragolgirith suspected, BouncyCastle seems to have problems with OpenJ9. I guess it is not a Docker issue, because I can reproduce it on GitHub Actions, too. It is also not limited to BouncyCastle 1.64 or 1.70, it happens in both versions. It also happens on OpenJ9 JDK 11, 14, 17 on Windows, MacOS and Linux, but for the same matrix of Java and OS versions it works on Adopt-Hotspot and Zulu.
Here is an example Maven project and a failed matrix build. So if you select another JVM type, you should be fine. I know that #Bragolgirith already suggested that, but I wanted to make the problem reproducible for everyone and also provide an MCVE, in case someone wants to open a BC or OpenJ9 issue.
P.S.: It is also not a character set issue with the InputStreamReader. This build fails exactly the same as before after I changed the constructor call.
Update: I have created BC-Java issue #1099. Let's see what the maintainers can say about this.
Update 2: The solution to your problem is to explicitly set the security provider to BC for your input decryptor provider. Thanks to David Hook for his helpful comment in #1099.
BouncyCastleProvider securityProvider = new BouncyCastleProvider();
Security.addProvider(securityProvider);
// (...)
InputDecryptorProvider pkcs8Prov = new JceOpenSSLPKCS8DecryptorProviderBuilder()
// Explicitly setting security provider helps to avoid ambiguities
// which otherwise can cause problems, e.g. on OpenJ9 JVMs
.setProvider(securityProvider)
.build(passphrase.toCharArray());
See this commit and the corresponding build, now passing on all platforms, Java versions and JVM types (including OpenJ9).
Because #Bragolgirith mentioned it in his answer: If you want to avoid the explicit new JceOpenSSLPKCS8DecryptorProviderBuilder().setProvider(securityProvider), the call Security.insertProviderAt(securityProvider, 1) instead of simply Security.addProvider(securityProvider) would in this case also solve the problem. But this holds true only as long as no other part of your code or any third-party library sets another provider to position 1 afterwards, as explained in the Javadoc. So maybe it is not a good idea to rely on that.

Edit:
On second thought, when creating the JceOpenSSLPKCS8DecryptorProviderBuilder, you're not explicitly specifying the provider:
new JceOpenSSLPKCS8DecryptorProviderBuilder()
.setProvider(BouncyCastleProvider.PROVIDER_NAME) // add this line
.build(passphrase.toCharArray());
It seems OpenJ9 uses a different provider/algo selection mechanism and selects the SunJCE's AESCipher class as CipherSpi by default, while Hotspot selects BouncyCastleProvider's AES class.
Explicitly specifying the provider should work in all cases.
Alternatively, when adding the BouncyCastleProvider you could insert it at the first preferred position (i.e. Security.insertProviderAt(new BouncyCastleProvider(), 1) instead of Security.addProvider(new BouncyCastleProvider())) so that it gets selected.
(It's still unclear to me why the provider selection mechanism differs between the different JVMs.)
Original post:
I've managed to reproduce the issue and at this point I'd say it's an incompatibility issue with the OpenJ9 JVM.
Starting from a Hotspot base image instead, e.g.
FROM adoptopenjdk:11-jre-hotspot
makes the code work.
(Not yet entirely sure whether the fault lies with the Docker image itself, the OpenJ9 JVM or BouncyCastle)

SecretKeyFactory Not Available (Jasypt)

I am trying to encrypt a string using Jasypt 1.9.3 and my JDK version is 1.8.0_281.
This is the code I am have written:
Security.setProperty("crypto.policy", "unlimited");
if (pooledPBEStringEncryptor == null) {
pooledPBEStringEncryptor = new PooledPBEStringEncryptor();
pooledPBEStringEncryptor.setPassword(encryptionKey);
pooledPBEStringEncryptor.setAlgorithm("PBEWITHHMACSHA512ANDAES256");
pooledPBEStringEncryptor.setPoolSize(4);
pooledPBEStringEncryptor.setSaltGenerator(new RandomSaltGenerator());
}
encrypted = pooledPBEStringEncryptor.encrypt(cValue);
But when I run it, I get the error
Exception in thread "main" java.lang.RuntimeException: Security Error in doEncrypt: org.jasypt.exceptions.EncryptionInitializationException: java.security.NoSuchAlgorithmException: PBEWITHHMACSHA512ANDAES256 SecretKeyFactory not available
I ran the AlgorithmRegistry.getAllPBEAlgorithms() and my output is:
PBEWITHHMACSHA1ANDAES_128, PBEWITHHMACSHA1ANDAES_256, PBEWITHHMACSHA224ANDAES_128, PBEWITHHMACSHA224ANDAES_256, PBEWITHHMACSHA256ANDAES_128, PBEWITHHMACSHA256ANDAES_256, PBEWITHHMACSHA384ANDAES_128, PBEWITHHMACSHA384ANDAES_256, PBEWITHHMACSHA512ANDAES_128, PBEWITHHMACSHA512ANDAES_256, PBEWITHMD5ANDDES, PBEWITHMD5ANDTRIPLEDES, PBEWITHSHA1ANDDESEDE, PBEWITHSHA1ANDRC2_128, PBEWITHSHA1ANDRC2_40, PBEWITHSHA1ANDRC4_128, PBEWITHSHA1ANDRC4_40
When I use the algorithm PBEWITHHMACSHA256ANDAES_256 I get a different error.
Exception in thread "main" java.lang.RuntimeException: Security Error in doEncrypt: org.jasypt.exceptions.EncryptionOperationNotPossibleException
I am a little lost as to what to do.
I have downloaded the unlimited policy jars from Oracle and saved them in JAVA_HOME\jre\lib\security\ folder. And I am on Windows.

The code lacks the specification of the IV generator with setIvGenerator(), e.g.:
pooledPBEStringEncryptor.setIvGenerator(new RandomIvGenerator());
By default, NoIvGenerator is used, which causes the exception because the algorithm applies the CBC mode, which requires an IV.
The default salt generator, by the way, is RandomSaltGenerator, so this would not necessarily need to be specified with setSaltGenerator().
The PooledPBEStringEncryptor#encrypt() method returns the Base64 encoded concatenation of salt (16 bytes), IV (16 bytes) and ciphertext.
The exception org.jasypt.exceptions.EncryptionOperationNotPossibleException is a general exception that is generated in many error situations and is therefore not very meaningful, see here. This includes e.g. the missing of the JCE Unlimited Strength Jurisdiction Policy (which however seems to be installed on your system).
For completeness: The algorithm is called PBEWITHHMACSHA512ANDAES_256 (which you have already figured out yourself).
PBEWITHHMACSHA512ANDAES_256 derives a 32 bytes key for AES-256 from password and salt using PBKDF2. HMAC/SHA512 is applied. Since not explicitly specified, the default iteration count of 1000 is used. The algorithm applies the CBC mode for encryption (which is why the IV is needed).

Is it possible to lower the encryption level to avoid the error: Unable to initialize due to invalid secret key

import org.junit.Test;
import org.springframework.security.crypto.encrypt.Encryptors;
import org.springframework.security.crypto.encrypt.TextEncryptor;
public class EncryptorTest {
#Test
public void testEncryption() {
TextEncryptor te = Encryptors.text("password", "abcdef");
te.encrypt("Hello World!");
}
}
I get this error:
java.lang.IllegalArgumentException: Unable to initialize due to invalid secret key
It seems that I have to download some policy file JCE extension to get this to work: http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html
I want to avoid this added dependency during deployment, and honestly I don't need 256 bit encryption for this.
Is it possible to lower the level to something that works w/o having to download this JCE extension?

No the value 256 is hard coded into the source:
PBEKeySpec keySpec = new PBEKeySpec(password.toCharArray(), Hex.decode(salt), 1024, 256);
This can be found in AesBytesEncryptor on line 54 at least up to release 3.2.5.
Note that Spring uses the Apache license, so there is nothing stopping you from creating your own implementation that uses 128 as constant.

Java: Patching client side security policy from applet for AES256

I require AES256 encryption/decryption in a commercial web application.
Currently everything is good with a key size of 128. This is not satisfactory cryptographically so my problem is how best to get round this issue without requiring the user to install anything manually.
I have the unlimited jurisdiction jar files from Oracle but I have no idea if replacing these in the user's JRE/lib/security directory will be compatible with older versions. Obviously I don't want to corrupt the user's JRE. Also I have write permission to my JRE security directory but I assume some user's will not have these privileges.
Is there a simple way around this issue, or am I stuck with either weak encryption or a potentially problematic step for users?
Update for "unrestricting" javax.crypto.JceSecurity
#ntoskml You are correct. getMaxAllowedKeyLength still returns the limited key size but the encryption succeeds with key size == 256 :). I will update my test method and set the key size if strong encryption is available. Thanks
>>> from javax.crypto import Cipher
>>> Cipher.getMaxAllowedKeyLength("AES")
128
>>> from java.lang import Class
>>> c = Class.forName("javax.crypto.JceSecurity")
>>> isRestricted = c.getDeclaredField("isRestricted")
>>> isRestricted.setAccessible(True)
>>> isRestricted.set(None, False)
>>> isRestricted.get(None)
False
>>> Cipher.getMaxAllowedKeyLength("AES")
128
>>> from javax.crypto import KeyGenerator
>>> kge = KeyGenerator.getInstance("AES")
>>> kge.init(256)
>>> aesKey = kgen.generateKey()
>>> c2 = Cipher.getInstance("AES")
>>> c2.init(Cipher.ENCRYPT_MODE, aesKey)
>>> c2.doFinal("test")
array('b', [-81, 99, -61, -51, 93, -42, -68, -28, 107, 59, -109, -98, -25, 127, 37, 23])
And the test case after restarting Jython console
>>> # Reflection as above
>>> isRestricted.get(None)
True
>>> kge.init(256)
>>> aesKey = kge.generateKey()
>>> c2.init(Cipher.ENCRYPT_MODE, aesKey)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
at javax.crypto.Cipher.checkCryptoPerm(Cipher.java:1011)
at javax.crypto.Cipher.implInit(Cipher.java:786)
at javax.crypto.Cipher.chooseProvider(Cipher.java:849)
at javax.crypto.Cipher.init(Cipher.java:1213)
at javax.crypto.Cipher.init(Cipher.java:1153)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
java.security.InvalidKeyException: java.security.InvalidKeyException: Illegal key size or default parameters
Bingo :) Thanks for sharing #ntoskml

EDIT: Here's an updated answer to this question: How to avoid installing "Unlimited Strength" JCE policy files when deploying an application?
It is possible to disable the key size restrictions simply by using a few lines of reflection. We use this method in our program which needs access to 256-bit cryptography for interoperability purposes.
private static void removeCryptographyRestrictions() {
if (!isRestrictedCryptography()) {
return;
}
try {
java.lang.reflect.Field isRestricted;
try {
final Class<?> c = Class.forName("javax.crypto.JceSecurity");
isRestricted = c.getDeclaredField("isRestricted");
} catch (final ClassNotFoundException e) {
try {
// Java 6 has obfuscated JCE classes
final Class<?> c = Class.forName("javax.crypto.SunJCE_b");
isRestricted = c.getDeclaredField("g");
} catch (final ClassNotFoundException e2) {
throw e;
}
}
isRestricted.setAccessible(true);
isRestricted.set(null, false);
} catch (final Throwable e) {
logger.log(Level.WARNING,
"Failed to remove cryptography restrictions", e);
}
}
private static boolean isRestrictedCryptography() {
return "Java(TM) SE Runtime Environment"
.equals(System.getProperty("java.runtime.name"));
}
However, our program is not an applet, and I am not sure whether applets have access to the reflection API.
The question about legality also remains. There is a reason for that limit. Consult a lawyer if you are concerned.
If possible, try to keep it to 128-bit keys. Even when taking Moore's law into consideration, breaking 128-bit AES would take billions upon billions of years. Longer keys offer no benefit in the real world – particularly when the keys are derived from passwords, which don't have anywhere near 256 bits of entropy anyway.

You are either stuck with the weak encryption or a potentially problematic step for users if you stick to the SunJCE.
There is obviously no problem importing an AES library, there is just a problem using it using an instance of Cipher. If you have a specific piece of software that does not depend on JCA, you can for instance rewrite it to use the lightweight crypto API of Bouncy Castle.
Note that many other parts of the Bouncy API themselves depend on the JCE. The lightweight API is also trickier to use and less documented/tested than the SunJCE.
The Bouncy Castle lightweight API is pretty large as well. It contains a lot of functionality that you won't need. So it is probably too large for your applet. If it is I would advice you to create a new library that only contains the specific classes that you need from Bouncy Castle. The Bouncy Castle is fortunately very liberally licensed. As long as you keep the copyright statements etc. in place, you can easily split it off.

What specific hash algorithm does MessageDigest.getInstance("SHA") return?

MessageDigest.getInstance("SHA") seems to work and gives me a MessageDigest, but I can't tell what algorithm it's giving me.
Is it SHA-1 or SHA-0 or ..?
I'm not interested in what happens on my machine. I want to know whether it will return sha0 or sha1 for all valid implementations of Java (or it's undefined).

The JCE Specification lists standard names that an implementation is expected to support. "SHA-1" is specified, as are SHA-256, SHA-384, and SHA-512. "SHA", "SHA-0" and SHA-2" are not standard names and therefore may not be supported at all. You cannot guarantee what "SHA" will return, if anything at all, because it is not in the standard.

SHA-0 is obsolete. For use with the Java JCE MessageDigest, SHA == SHA-1 for some JCE providers. By the way, SHA-1 is not considered to be secure with today's computers and technology. SHA-512 is still secure for pretty much anything. SHA-256 is ok for most things, still.
You can list the protocols available in the Java version you are using with this code. (I got it here ):
import java.security.Provider;
import java.security.Security;
public class JceLook {
public static void main(String[] args) {
System.out.println("Algorithms Supported in this JCE.");
System.out.println("====================");
// heading
System.out.println("Provider: type.algorithm -> className" + "\n aliases:" + "\n attributes:\n");
// discover providers
Provider[] providers = Security.getProviders();
for (Provider provider : providers) {
System.out.println("<><><>" + provider + "<><><>\n");
// discover services of each provider
for (Provider.Service service : provider.getServices()) {
System.out.println(service);
}
System.out.println();
}
}
}
It will show information like this for all the various algorithms available. (Note that this is actual output from the program above for some update level of Oracle/Sun Java 6 and it shows that SHA is equivalent to SHA-1 and SHA1. You can pass any of the three strings to MessageDigest and get the same result. But this depends on the Cryptography Provider (the JCE) and might not be the same.)
SUN: MessageDigest.SHA -> sun.security.provider.SHA
aliases: [SHA-1, SHA1]
attributes: {ImplementedIn=Software}
If you load additional providers (e.g. BouncyCastle) it will show those too.