Develop Reference

Construct Java netty SslContext and apache SSLContexts to trust certificates from dynamic .pem files without a preloaded truststore

Question on the possibility to construct Java io.netty.handler.ssl.SslContext and org.apache.http.ssl.SSLContexts to trust certificates from multiple dynamic generated .pem files without a preloaded truststore.
In my case, I have 3 dynamically generated certificates, a root CA, an intermediate CA, another root CA, all in .pem format. (All starting with -----BEGIN CERTIFICATE----- )
I would like to create both the io.netty.handler.ssl.SslContext and org.apache.http.ssl.SSLContexts with the above generated certificates.
If possible, I would like to achieve this without creating a preloaded truststore.p12.
Probably something like keytool -import -trustcacerts -file root_ca.pem -alias root-ca -keystore truststore.p12 And have a final truststore.p12 with everything, and have the app read the final truststore.p12 might work, but this is something I would like to avoid.
Is there a way to achieve the construction of both io.netty.handler.ssl.SslContext and org.apache.http.ssl.SSLContexts without a preloaded trust store, but directly from the .pem files?
Thank you

Apache (for sure) uses the JSSE-provided TrustManager with minor tweaks. This TrustManager is actually initalized from a KeyStore object in memory that contains the certs; this KeyStore is often read in from a truststore file (or at least pseudo-file like a classloader resource), but need not be.
// prepare in standard JCA
KeyStore ks = KeyStore.getInstance("PKCS12"); // type doesn't matter as long as it's filebased
ks.load(null);
CertificateFactory cf = CertificateFactory.getInstance("X.509");
int counter = 0;
for( String f : new String[]{"pem1","pem2","pem3"} ){ // or other source(s) of data
try( InputStream is = new FileInputStream(f) ){
ks.setCertificateEntry("alias"+(++counter), cf.generateCertificate(is) );
// or other aliases as long as they're unique, maybe even filenames
}
}
// now use in Apache
SSLContext ctx = SSLContexts.custom().loadTrustMaterial(ks).build();
HttpClient client = HttpClientBuilder.create().setSSLContext(ctx)....build();
I expect more or less the same is true with netty, but I'm not familiar with it.
JCA CertificateFactory actually reads either PEM or DER, if you needed that.

java.lang.NullPointerException Error during the SecurityHelper.prepareSignatureParams() OpenSAML call [duplicate]

This question already has answers here:
What is a NullPointerException, and how do I fix it?
(12 answers)
Closed 6 years ago.
I have hit a dead end using the OpenSAML support for preparing a SAML Payload to accomplish a SSO transaction with another service that I am working with. I receive a NullPointerException that is thrown after I use the SecurityHelper.prepareSignatureParams() routine. I have a Stacktrace, but it wold be pretty ugly to append.
Let me first say what I was able to do...
For the purposes of learning the technology and to make sure it would work, I was able to successfully build a SAML payload, sign it using a Certificate and a Private Key that was stored in a Java Key Store file that I created locally on my workstation using the Keytool program with the -genkeypair option.
As I understand things, my JKS file contains a Self Signed Certificate and a Private Key. I was able to open the JKS file, gather the Certificate and the Private Key to build a Signing Certificate. The Signing Certificate was used to sign the SAML Payload that I created You'll see how I did this if you look at the code samples that I'll append.
What isn't working...
I want to use the same SAML support to sign my SAML Payload using a Trusted Certificate that I have for my website that I received from GoDaddy. To do this, I installed the Trusted Certificate into my webserver's keystore at: '\Program Files\Java\jre1.8.0_102\lib\security\cacerts'. I understand that the cacerts file is the KeyStore for our webserver. I installed the Trusted Certificate using the Keytool -importcert command. One big difference is that the Trusted Certificate DOESN'T have a Private Key. So when preparing the Signing Certificate using the Open SAML support, I am not able to add a Private Key to the Credential object (because I don't have one).
When attempting the above for the Trusted Certificate, I am able to get to the part where I am preparing the Signature Parms (SecurityHelper.prepareSignatureParams()). That's where I get the Null Pointer.
If you could take a look at the code that I am using. I am including the code (that signs my payload successfully) that reads from the local JKS file and also the code (that gets the Null Pointer Exception) when I try to using the Trusted Certificate on the server (both cases). There's not much different between the two cases:
// Signing process using OpenSAML
// Get instance of an OpenSAML 'KeyStore' object...
KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
// Read KeyStore as File Input Stream. This is either the local JKS
// file or the server's cacerts file.
File ksFile = new File(keyStoreFileName);
// Open an Input Stream with the Key Store File
FileInputStream ksfInputStream = new FileInputStream(ksFile);
// Load KeyStore. The keyStorePassord is the password assigned to the keystore, Usually 'changeit'
// before being changed.
keyStore.load(ksfInputStream, keyStorePassword);
// Close InputFileStream. No longer needed.
ksfInputStream.close();
// Used to get Entry objects from the Key Store
KeyStore.PrivateKeyEntry pkEntry = null;
KeyStore.TrustedCertificateEntry tcEntry = null;
PrivateKey pk = null;
X509Certificate x509Certificate = null;
BasicX509Credential credential = null;
// The Java Key Store specific code...
// Get Key Entry From the Key Store. CertificateAliasName identifies the
// Entry in the KeyStore. KeyPassword is assigned to the Private Key.
pkEntry = (KeyStore.PrivateKeyEntry) keyStore.getEntry(certificateAliasName, new KeyStore.PasswordProtection(keyPassword));
// Get the Private Key from the Entry
pk = pkEntry.getPrivateKey();
// Get the Certificate from the Entry
x509Certificate = (X509Certificate) pkEntry.getCertificate();
// Create the Credential. Assign the X509Certificate and the Privatekey
credential = new BasicX509Credential();
credential.setEntityCertificate(x509Certificate);
credential.setPrivateKey(pk);
// The Trusted Certificate specific code...
// Accessing a Certificate that was issued from a trusted source - like GoDaddy.com
//
// Get Certificate Entry From the Key Store. CertificateAliasName identifies the Entry in the KeyStore.
// There is NO password as there is no Private Key associate with this Certificate
tcEntry = (TrustedCertificateEntry) keyStore.getEntry(certificateAliasName, null);
// Get the Certificate from the Entry
x509Certificate = (X509Certificate) tcEntry.getTrustedCertificate();
// Create the Credential. There is no Provate Ley to assign into the Credential
credential = new BasicX509Credential();
credential.setEntityCertificate(x509Certificate);
// Back to code that is not specific to either method...
//
// Assign the X509Credential object into a Credential Object. The BasicX509Credential object
// that has a Certificate and a Private Key OR just a Certificate added to it is now saved as a
// Cendential object.
Credential signingCredential = credential;
// Use the OpenSAML builder to create a signature object.
Signature signingSignature = (Signature) Configuration.getBuilderFactory().getBuilder(Signature.DEFAULT_ELEMENT_NAME).build Object(Signature.DEFAULT_ELEMENT_NAME);
// Set the previously created signing credential
signingSignature.setSigningCredential(signingCredential);
// Get a Global Security Configuration object.
SecurityConfiguration secConfig = Configuration.getGlobalSecurityConfiguration();
// KeyInfoGenerator. Not sure what this is, but the example I am working from shows
// this being passed as null.
String keyInfoGeneratorProfile = "XMLSignature";
// Prepare the Signature Parameters.
//
// This works fine for the JKS version of the KeyStore, but gets a Null Pointer exception when I run to the cacerts file.
SecurityHelper.prepareSignatureParams(signingSignature, signingCredential, secConfig, keyInfoGeneratorProfile <or null>);
// I need to set into the SigningSignature object the signing algorithm. This is required when using the TrustedCertificate
signingSignature.setSignatureAlgorithm(SignatureConstants.ALGO_ID_SIGNATURE_RSA_SHA256);
// This is my code that builds a SAML Response. The SAML Payload contains data
// about the SSO session that I will be creating...
Response samlResponse = createSamlResponse.buildSamlResponseMessage();
// Sign the Response using the Certificate that was created earlier
samlResponse.setSignature(signingSignature);
// Get the marshaller factory to marshall the SamlResponse
MarshallerFactory marshallerFactory = Configuration.getMarshallerFactory();
Marshaller responseMarshaller = marshallerFactory.getMarshaller(samlResponse);
// Marshall the Response
Element responseElement = responseElement = responseMarshaller.marshall(samlResponse);
// Sign the Object...
Signer.signObject(signingSignature);
NOTE: My attempt to sign a SAML Payload was modeled after an OPENSAML example that I found here: https://narendrakadali.wordpress.com/2011/06/05/sign-assertion-using-opensaml/
Hoping that someone can show me the error of my ways or what I am missing.
Thanks for any suggestions.
EDIT (01/26/2016)
I was able to get past the NULL pointer I was receiving while preparing the Signature Params (SecurityHelper.prepareSignatureParams()). Code changes included updating my xmlsec.jar file to version 2.0.8 (xmlsec-2.0.8.jar) and I explicitly setting the signature algorithm to SHA256 when using the Trusted Certificate (from GoDaddy). See my code example for the use of:
signingSignature.setSignatureAlgorithm(SignatureConstants.ALGO_ID_SIGNATURE_RSA_SHA256);
The above changes allows the SAML payload to be built and sent to the connection endpoint.
However, I am still not establishing the SSO connection to my endpoint.
Here's what I see happening:
During processing while the SAML payload is being constructed and specifically, the SAML payload's Signature is being signed:
Signer.signObject(signature);
I get an an error message from SAML:
ERROR: org.opensaml.xml.signature.Signer - An error occured computing the digital signature
The stack trace (just the ending portion):
org.apache.xml.security.signature.XMLSignatureException: Sorry, you supplied the wrong key type for this operation! You supplied a null but a java.security.PrivateKey is needed.
at org.apache.xml.security.algorithms.implementations.SignatureBaseRSA.engineInitSign(SignatureBaseRSA.java:149)
at org.apache.xml.security.algorithms.implementations.SignatureBaseRSA.engineInitSign(SignatureBaseRSA.java:165)
at org.apache.xml.security.algorithms.SignatureAlgorithm.initSign(SignatureAlgorithm.java:238)
at org.apache.xml.security.signature.XMLSignature.sign(XMLSignature.java:631)
at org.opensaml.xml.signature.Signer.signObject(Signer.java:77)
I searched the error messages, but I am not coming up with much.
I don't understand the root of the error message - That the wrong key type was supplied (null) and that OpenSAML seems to be expecting a java.Security.PrivateKey.
When using the Trusted Certificate, I don't have a Private Key, Correct? How would I be able to provide a Private Key? In the case of the Trusted Certificate I read a Trusted Certificate (TrustedCertificateEntry) from the KeyStore. The TrustedCertificateEntry object allows me to access the Certificate, but there's no method for obtaining a Private Key (as well there shouldn't be).
However, when I use my Self Signed Certificate to perform the signing operation, I understand that I do have both the Certificate (the Public Key) and the Private Key contained in the JKS file (the KeyStore). I think that's why when I read from the JKS file, I am able to read a Private Key Entry (KeyStore.PrivateKeyEntry) that has methods for accessing both the Public Key (the Certificate) and the Private Key.
What am I missing about the Trusted Certificate case? The OpenSAML support seems to be expecting a Private key to be able to compute the Signature.
In the case of the Trusted Certificate, is there a way to package the original Private Key into my Key Store (along with the Trusted Certificate)? I am not sure if this is what is normally done or even possible.
Hopefully some guidance as to what I am doing here, Please!
EDIT (01/26/2017) - 2 to provide additional detail.
I'll share a portion of the SAML payload that gets sent...
In the case of the Self Signed Certificate, I see a SignatureValue tag and a X509Certificate tag. Both have binary data included within the begin and end of the tag.
In the case of the Trusted Certificate, I've got an empty Signature Value tag that looks like:
<ds:SignatureValue/>
The Certificate tag is still present and contains the certificate bytes.
So, looking at the error I see from OpenSAML, it is more obvious that it can't compute a Signature using the data that is available in the Trusted Certificate.

Ok, this quite a long question. As I understand the root of the problem is the message "Sorry, you supplied the wrong key type for this operation! You supplied a null but a java.security.PrivateKey is needed." You are trying to sign a message using a public key. This is not possible. Just looking logically on it, signing using a public key would not provide any proof that the signer is intended as it is available to everyone.
What you need to do is sign using a private key. in your case you have generated a public and private key on you computer, then sent CSR to the CA and recieved a certificate signed by the CA.
You should use the privat key from you local computer to sign the message and send the CA signed certificate to the recipient so they can use it to confirm your signature.
In this blog post of mine I explain how to obtain credentials, including the private key from a Java keystore.

Java - verifying certificate with system TrustStore

Premise: I have a certificate and I want to verify that the system 'trusts' this certificate (signed by a trusted root CA by Java / Operating System)
I have found some varying solutions on how to accomplish this.
Option 1:
Use SSL classes to derive trust.
TrustManagerFactory tmfactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmfactory.init((KeyStore) null);
for (TrustManager trustManager : tmfactory.getTrustManagers()) {
if (trustManager instanceof X509TrustManager) {
try {
((X509TrustManager) trustManager).checkClientTrusted(new X509Certificate[] {new JcaX509CertificateConverter().getCertificate(holder)}, "RSA");
System.out.println("This certificate is trusted by a Root CA");
} catch (CertificateException e) {
e.printStackTrace();
}
}
}
Since this approach relies heavily on SSL classes (which are not needed by the current project) we are looking for alternatives.
Option 2:
Load Java's cacertsfile into a keystore and check each 'most-trusted' certificate against my certificate for equality.
String filename = System.getProperty("java.home") + "/lib/security/cacerts".replace('/', File.separatorChar);
FileInputStream is = new FileInputStream(filename);
KeyStore keystore = KeyStore.getInstance(KeyStore.getDefaultType());
String password = "changeit";
keystore.load(is, password.toCharArray());
// This class retrieves the most-trusted CAs from the keystore
PKIXParameters params = new PKIXParameters(keystore);
// Get the set of trust anchors, which contain the most-trusted CA certificates
Set<X509Certificate> rootCertificates = params.getTrustAnchors().parallelStream().map(TrustAnchor::getTrustedCert).collect(Collectors.toSet());
return rootCertificates.contains(holderX509);
The problem with this approach is that it requires a password to verify integrity of the JKS encoded file. While the SSL one seemingly does not (or rather uses System.getProperty("javax.net.ssl.trustStorePassword") which again is heavily tied to SSL.
Question: Does there exist a solution that is in between manually loading certificates from a file and pure SSL? I feel as if there should be some class that I can call to simply verify the system trust of a certificate without having to jump through a couple hoops.

After reading Beginning Cryptography With Java by David Hook I have produced the following example to verify a certificate chain (which accomplishes the original goal of using the system truststore to verify Root CA's)
CertificateFactory certificateFactory = CertificateFactory.getInstance("X.509", new BouncyCastleProvider());
InputStream is = new ByteArrayInputStream(some bytes in an array);
CertPath certPath = certificateFactory.generateCertPath(is, "PKCS7"); // Throws Certificate Exception when a cert path cannot be generated
CertPathValidator certPathValidator = CertPathValidator.getInstance("PKIX", new BouncyCastleProvider());
PKIXParameters parameters = new PKIXParameters(KeyTool.getCacertsKeyStore());
PKIXCertPathValidatorResult validatorResult = (PKIXCertPathValidatorResult) certPathValidator.validate(certPath, parameters); // This will throw a CertPathValidatorException if validation fails
This also accomplishes the goal of not having to use SSL classes - instead Java security classes / algorithms are used.

Short of downloading a third-party library, there probably isn't another alternative.
Why are you trying to avoid the "SSL" library? It's part of the standard library and so puts no burden on your program.
In any case, certificate verification is a big part of SSL. I doubt anyone's gone to the trouble of creating a library that does so without also implementing some substantial subset of the SSL protocol. There's just no real reason to do so.

My code can verify self signed certificate but is failing when VeriSign issued certificate is used

My application is using java security APIs to sign a file and verify it.
While signing , I am using PFX file and password as inputs and after signing I am generating a signature file using the bytes.
While verification process I am using signature file ,certificate file and the signed file as inputs.
Please find the code I am using in verification below:
// KeyFilePath= path of certificate file
// fileToVerify = path of signed file
// signatureFilePath = path of signature file
InputStream inputStream = new FileInputStream(KeyFilePath);
CertificateFactory certificateFactory = CertificateFactory.getInstance("X.509");
X509Certificate x509Certificate = (X509Certificate) certificateFactory.generateCertificate(inputStream);
// input the signature bytes
String sigFile = signatureFilePath;
FileInputStream sigFileInputStream = new FileInputStream(sigFile);
byte[] sigToVerify = new byte[sigFileInputStream.available()];
sigFileInputStream.read(sigToVerify);
sigFileInputStream.close();
PublicKey pubKey = x509Certificate.getPublicKey();
Signature signature = Signature.getInstance(signAlgorithm);
signature.initVerify(pubKey);
// Update and verify the data
try {
FileInputStream dataFileInputStream = new FileInputStream(fileToVerify);
BufferedInputStream bufferedInputStream = new BufferedInputStream(dataFileInputStream);
byte[] buffer = new byte[IVerifyDigitalSignature.BYTE_SIZE];
int bufferedInputStreamLength;
while (bufferedInputStream.available() != IVerifyDigitalSignature.ZERO_LENGTH) {
bufferedInputStreamLength = bufferedInputStream.read(buffer);
signature.update(buffer, IVerifyDigitalSignature.ZERO_LENGTH, bufferedInputStreamLength);
}
bufferedInputStream.close();
// Verify the Signature
x509Certificate.verify(pubKey);
verifyDigitalSignature = signature.verify(sigToVerify);
Please help me in resolving the same as it is yet not closed.

If you want to do this yourself, yes you must iterate over the certs in the chain from a trust anchor to your desired cert,
however long that it is (it may vary for different CAs, classes, and at different times).
Verifying the signature on each "child" (lower level) cert using the publickey from the "parent" (next higher) cert is only a
fairly small part of this; there are many other steps needed.
Often just finding the right certs can be an issue; if you already have a correct chain you have a head start.
But are you sure you have "the" right chain? Frequently there are several possible chains for a given cert,
and sometimes some of them are valid but others have expired or become unverifiable.
Verisign in particular issued I believe all recent certs under their G5 root but provided an alternate path
back to (effectively) G1 for reliers that aren't up to date, and sometimes can't be updated.
The algorithm for most situations is defined in "PKIX" RFC5280,
except that OCSP RFC6960 instead of CRL for revocation is becoming more common.
You might get away with omitting cross-hierarchy and NameConstraints, which AFAIK aren't actually used
by public CAs like Verisign, and the Policy stuff which CAs do use but users/reliers don't care about.
https://security.stackexchange.com/questions/37409/certificate-chain-checking has a good but not complete introduction.
But you're probably better off using Java's (really JCE's) CertPathValidator for "PKIX" -- and if needed CertPathBuilder --
to which I already referred you. This has already been written and tested by experts. Just calling it
is still a little bit complicated, but nowhere near as complicated as rewriting all the things it does.

Why do I get the error "Cannot store non-PrivateKeys" when creating an SSL Socket in Java?

I am working on an older IBM iSeries (IBM-i, i5OS, AS/400, etc), with a Java 5 JVM (Classic, not ITJ J9) on O/S version V5R3M0.
Here is the scenario in a nutshell:
I created a key-store of type JKS using Portecle 1.7 (Note: I did try converting my key-store to JCEKS but that was rejected as an unsupported format, so it appears that JKS is the only option with the iSeries machine (at least the version I am on).
I then created a key-pair and CSR and sent the CSR to Thawte to be signed.
I imported the signed certificate from Thawte successfully using the PKCS#7 format to import the entire certificate chain, which included my certificate, the Thawte intermediary and the Thawte server root.
This all worked as expected.
However, when I ran up the JVM, configured properly to point to the store and supply it's password (which I have done in the past with self-signed certificates created in Portecle for testing), and try to start my web server on 443, I get the following security exception:
java.security.KeyStoreException: Cannot store non-PrivateKeys
Can anyone tell me where I went wrong, or what I should check next?

The "Cannot store non-PrivateKeys" error message usually indicates you are trying to use secret symmetric keys with a JKS keystore type. The JKS keystore type only supports asymmetric (public/private) keys. You would have to create a new keystore of type JCEKS to support secret keys.

As it turns out, this was a subtle problem, and it's worth giving the answer here in case someone else has something similar.
The TLDR answer is that I did not check that my key and certificate were not null and as a result attempted to add a null key and certificate to a key-store. The longer answer follows.
The way we have our web server set up to use SSL, specifically to support our user's typical configuration where the IP address is used to configure the web site listen address rather than a DNS name, is that it locates the certificate in the master key-store using the alias, and creates an ephemeral key-store containing just the certificate for that web site, using that key-store to configure an SSL context and an SSL socket factory, like so:
// CREATE EPHEMERAL KEYSTORE FOR THIS SOCKET USING THE DESIRED CERTIFICATE
try {
final char[] BLANK_PWD=new char[0];
SSLContext ctx=SSLContext.getInstance("TLS");
KeyManagerFactory kmf=KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
Key ctfkey=mstkst.getKey(svrctfals,BLANK_PWD);
Certificate[] ctfchn=mstkst.getCertificateChain(svrctfals);
KeyStore sktkst;
sktkst=KeyStore.getInstance("jks");
sktkst.load(null,BLANK_PWD);
sktkst.setKeyEntry(svrctfals,ctfkey,BLANK_PWD,ctfchn);
kmf.init(sktkst,BLANK_PWD);
ctx.init(kmf.getKeyManagers(),null,null);
ssf=ctx.getServerSocketFactory();
}
catch(java.security.GeneralSecurityException thr) {
throw new IOException("Cannot create server socket factory using ephemeral keystore ("+thr+")",thr);
}
Notice that it uses a blank password for extracting the private key and certificates from the master key-store. That was my problem - I had, out of habit from using keytool, created the private key-pair with a password (the same password as the key-store).
Because I had a password on the certificate, the key and certificate were not extracted, and null was passed to sktkst.setKeyEntry(svrctfals,ctfkey,BLANK_PWD,ctfchn); However, setKeyEntry checks the passed Key using instanceof and concludes (correctly) that null is not an instanceof PrivateKey, resulting in the misleading error I was seeing.
The corrected code checks that a key and certificate are found and sends appropriate errors:
// CREATE EPHEMERAL KEYSTORE FOR THIS SOCKET USING THE DESIRED CERTIFICATE
try {
final char[] BLANK_PWD=new char[0];
SSLContext ctx=SSLContext.getInstance("TLS");
KeyManagerFactory kmf=KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
Key ctfkey=mstkst.getKey(svrctfals,BLANK_PWD);
Certificate[] ctfchn=mstkst.getCertificateChain(svrctfals);
KeyStore sktkst;
if(ctfkey==null) {
throw new IOException("Cannot create server socket factory: No key found for alias '"+svrctfals+"'");
}
if(ctfchn==null || ctfchn.length==0) {
throw new IOException("Cannot create server socket factory: No certificate found for alias '"+svrctfals+"'");
}
sktkst=KeyStore.getInstance("jks");
sktkst.load(null,BLANK_PWD);
sktkst.setKeyEntry(svrctfals,ctfkey,BLANK_PWD,ctfchn);
kmf.init(sktkst,BLANK_PWD);
ctx.init(kmf.getKeyManagers(),null,null);
ssf=ctx.getServerSocketFactory();
}
catch(java.security.GeneralSecurityException thr) {
throw new IOException("Cannot create server socket factory using ephemeral keystore ("+thr+")",thr);
}

Instead of using an ephemeral keystore, you could handle everything within a single SSLContext.
You would need to initialise your SSLContext using an custom X509KeyManager instead of using the one given by the default KeyManagerFactory. In this X509KeyManager,chooseServerAlias(String keyType, Principal[] issuers, Socket socket) should return a different alias depending on the local address obtained from the socket.
This way, you wouldn't have to worry about copying the private key from one keystore to another, and this would even work for keystore types from which you can't extract (and thus copy) but only use the private key, e.g. PKCS#11.