I am working on integrating my application with Apache Kafka. While everything works as expected while connecting to test brokers.
I have hit a blocker with 2-way SSL in preprod environment. My application is deployed in Websphere Application server and the certs/keys are maintained in a Websphere keyring. Issue is Kafka producer configuration is unable to interact with the keyring to find the trusted certs or the key and hence the connectivity is failing.
I can't use JKS files since that will defeat the purpose of the keyring and is against the application design. The whole problem appears to be the interaction of Kafka client code with keyring, during application startup. Any suggestion on this is appreciated.
org.apache.kafka.common.network.Selector) - [Producer clientId= xxxxxxx] Connection with disconnected due to authentication exception
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at com.ibm.jsse2.bb.B(bb.java:525)
at com.ibm.jsse2.oc.b(oc.java:394)
at com.ibm.jsse2.oc.c(oc.java:146)
at com.ibm.jsse2.oc.wrap(oc.java:316)
at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:39)
at org.apache.kafka.common.network.SslTransportLayer.handshakeWrap(SslTransportLayer.java:434)
at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:299)
at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:253)
at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:79)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:486)
at org.apache.kafka.common.network.Selector.poll(Selector.java:424)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:460)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
at java.lang.Thread.run(Thread.java:798)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at com.ibm.jsse2.k.a(k.java:5)
at com.ibm.jsse2.oc.a(oc.java:170)
at com.ibm.jsse2.bb.a(bb.java:560)
at com.ibm.jsse2.bb.a(bb.java:432)
at com.ibm.jsse2.cb.a(cb.java:30)
at com.ibm.jsse2.cb.a(cb.java:394)
at com.ibm.jsse2.bb.t(bb.java:170)
at com.ibm.jsse2.bb$1.a(bb$1.java:4)
at com.ibm.jsse2.bb$1.run(bb$1.java:2)
at java.security.AccessController.doPrivileged(AccessController.java:492)
at com.ibm.jsse2.bb$c_.run(bb$c_.java:11)
at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:388)
at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:468)
at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:326)
... 8 more
Caused by: com.ibm.jsse2.util.h: PKIX path building failed: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is:
java.security.cert.CertPathValidatorException: The certificate issued by xxxxxxxxxx is not trusted; internal cause is:
java.security.cert.CertPathValidatorException: Certificate chaining error
at com.ibm.jsse2.util.f.a(f.java:70)
at com.ibm.jsse2.util.f.b(f.java:95)
at com.ibm.jsse2.util.e.a(e.java:20)
at com.ibm.jsse2.zc.a(zc.java:35)
at com.ibm.jsse2.zc.a(zc.java:156)
at com.ibm.jsse2.zc.checkServerTrusted(zc.java:125)
at com.ibm.jsse2.cb.a(cb.java:302)
... 17 more
Caused by: java.security.cert.CertPathBuilderException: PKIXCertPathBuilderImpl could not build a valid CertPath.; internal cause is:
java.security.cert.CertPathValidatorException: The certificate issued by xxxxxxxxxxx is not trusted; internal cause is:
java.security.cert.CertPathValidatorException: Certificate chaining error
at com.ibm.security.cert.PKIXCertPathBuilderImpl.engineBuild(PKIXCertPathBuilderImpl.java:410)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:256)
at com.ibm.jsse2.util.f.a(f.java:144)
... 23 more
Caused by: java.security.cert.CertPathValidatorException: The certificate issued by xxxxxxxxxxxxxxx is not trusted; internal cause is:
java.security.cert.CertPathValidatorException: Certificate chaining error
at com.ibm.security.cert.BasicChecker.<init>(BasicChecker.java:111)
at com.ibm.security.cert.PKIXCertPathValidatorImpl.engineValidate(PKIXCertPathValidatorImpl.java:176)
at com.ibm.security.cert.PKIXCertPathBuilderImpl.myValidator(PKIXCertPathBuilderImpl.java:737)
at com.ibm.security.cert.PKIXCertPathBuilderImpl.buildCertPath(PKIXCertPathBuilderImpl.java:649)
at com.ibm.security.cert.PKIXCertPathBuilderImpl.buildCertPath(PKIXCertPathBuilderImpl.java:595)
at com.ibm.security.cert.PKIXCertPathBuilderImpl.buildCertPath(PKIXCertPathBuilderImpl.java:595)
at com.ibm.security.cert.PKIXCertPathBuilderImpl.engineBuild(PKIXCertPathBuilderImpl.java:356)
... 25 more
Caused by: java.security.cert.CertPathValidatorException: Certificate chaining error
at com.ibm.security.cert.CertPathUtil.findIssuer(CertPathUtil.java:316)
at com.ibm.security.cert.BasicChecker.<init>(BasicChecker
I'm not sure if this is going to work but you can try this:
extract the certificate and key from Keyring before you start up the producer;
save them in *.jks files somewhere in your machine;
pass the path of your newly created Keystore and Truststore to your
Kafka producer
Unfortunately, Java Kafka Clients can only interact with *.jks files so a proper conversion needs to be done before start-up. Another option would be to do the same thing but at a pre-deployment phase ( before starting up your application, you prepare the Keystore and Truststore ).
I know I'm a bit late to the party but I was looking for a solution to this issue as well and found a way to provide a Kafka Producer with WAS SSL configuration data. The idea came from the following IBM documentation that explains how to use the com.ibm.websphere.ssl.JSSEHelper to obtain information specified in an WAS managed SSL Configuration: https://www.ibm.com/docs/en/was/8.5.5?topic=ascdoprse-programmatically-specifying-outbound-ssl-configuration-using-jssehelper-api
The following would work to provide SSL configuration data to your Kafka Producer/Consumer properties:
com.ibm.websphere.ssl.JSSEHelper jsseHelper = JSSEHelper.getInstance();
Properties sslProperties = jsseHelper.getProperties("<your_ssl_conf_alias>")
consumerProperties.put(org.apache.kafka.common.config.SslConfigs.SSL_TRUSTSTORE_LOCATION_CONFIG,sslProperties.getProperty("com.ibm.ssl.trustStore"));
You can use an existing alias or create a new one in the WAS console under:
SSL certificate and key management > SSL configurations
Related
I have a local environment with WebLogic 10.3.4 and and .ear app deployed on it. This app must communicate with external services via REST APIs. These external services are exposed in https and use wildcard certificates.
I receive the following exceptions when I try to connect to to one of these services.
org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://dds-service.domain.com" General SSLEngine problem; nested exception is javax.net.ssl.SSLHandshakeException: General SSLEngine problem [...]
[...] Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target [...]
[...] Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
So what I tried in first instance was to open the same url the app tries to connect to in my browser, download its wildcard certificate (.cer Base64 encoded) and with key tool import it into the jvm's trust store that WebLogic loads and looks up when trying to validate a cert. I am sure is the correct one because i imported other certificates that caused the same error and also because of this log
<Loading trusted certificates from the jks keystore file C:\WEBLOG~1\wls\JROCKI~1.1-3\jre\lib\security\cacerts.>
At this point, I suppose the problem is related to the way I import the wildcard certificate in WebLogic. I tried to look for different ways to do it but, like this one, require a .pfx file that is not currently available to me at the moment.
Do I need a .pfx to solve this or is there another way?
If someone will ever have the same problem, here is the solution i found: it appears that WLS 10.3.x has issues in trusting certificates wth keys longer than 128 bits, this is what caused the problem in first instance. That said, 2 actions solved my problem
-DUseSunHttpHandler=true added as a VM argument in the setDomainEnv script
Enable JSSE SSL via WLS adminn console (Environment > Servers > server name > Configuration > SSL > Advanced > check the JSSE SSL box)
Click Save, and restart WLS server.
I have an Apache web server fronting a Tomcat 8 web server that is running my website, and I'm switching the top-level domain from my.website.ie to my.website.com. I have some code that runs in response to a particular request that generates a PDF. That code fetches an image (using a URL) which is served from the same web server e.g.
Image.getInstance(new URL("https://my.website.com/img/myimage.png"))
In addition to the domain change, I'm also changing my SSL certificate provider to LetsEncrypt (free SSL certs). My development website at the new .com domain is running and the certificate is valid and does not expire for a number of months.
I have another development server running on a separate machine that is still using the .ie domain. The Tomcat codebase running on both of these servers are identical right now. They are both trying to fetch the image at the URL shown above in that particular piece of code.
On the .ie server, the request that generates the PDF works correctly, without any problems fetching the image. On the .com server, the request fails with this error:
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
...
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
...
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
My understanding of this error is that the certificate at the target URL is not trusted (e.g. self-signed), but that is not true in this case. Also, both of the servers are hitting the same URL for the image, so why is one server trusting the certificate and the other not?
I haven't made any additional configuration changes on the .ie server that I haven't made on the .com server (with respect to setting up the new certificate), so is there some other (mis)configuration that I haven't considered?
Not trusted means the used CA is not trusted by the software.
Self-signed certificates are never trusted.
Java has it's own trust store (only on Linux the system trust store is used AFAIR?). If the CA certificate is newer than the used Java version it may happen that Java does not trust the CA. Conclusion: Update your Java.
According to this Stackoverflow answer you need at least Java 8u101 for Let's Encrypt support:
Does Java support Let's Encrypt certificates?
The error says that the chain doesn't lead to a root certificate that is trusted. Trusted CA root certificates are stored in Java's root keystore where the root certificate issued by Let's Encrypt is obviously missing.
You can add the root certificate manually to the store or check if newer versions of Java already contain the certificate.
I see SSL exception while invoking a rest api using Jersey API client library.
This used to work fine till few days ago. Based on the exception, I understand it is certificate expiry related. However I am not sure if the client machine needs to have the certificate (if any) upgraded or the API implementation server needs to upgrade. Code I use to call the API is mentioned below. I verified the ssl certs on my machine using How to determine SSL cert expiration date from a PEM encoded certificate? and I could not find any expired certificates.
com.sun.jersey.api.client.ClientHandlerException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
Client client = Client.create(JerseyJacksonConfigFactory.getConfig());
this.resource = client.resource(baseUrl);
resource.path("foo").entity(barEntity).type(MediaType.APPLICATION_JSON_TYPE).accept(MediaType.APPLICATION_JSON).put();
I have a web application using java 7 and tomcat 6 on ubuntu 12.04. It uses apache cxf to make REST calls to a web service on a different server over https. It has been working fine until about 11am today. Then for each REST call whether GET or POST, I receive
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target errors in catalina.out.
The certificate is in the
jdk/jre/lib/security/cacerts
and does not expire until 2018. When I restart tomcat the program runs fine.
Any idea what would trigger this?
Thanks in advance
Randy
If you are entirely certain that this is not a validity period problem with the service's certificate, here are some more situations to check for:
The certificate on the service side was changed so the one in cacerts is now not doing anything
The certificate you installed was the a system's certificate and you are now connecting to a different node due to load balancing (if so, you would need an intermediate or CA cert)
The service requires Mutual TLS (aka 2-way SSL) and the certificate on your side is now missing or invalid
A host name has changed such that the certificate(s) are no longer valid
Some software or configuration file has changed and now the client and service do not have SSL/TLS versions in common.
I receive the following Exception while trying to send an email (using Seam)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find vali
d certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:285)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:191)
at sun.security.validator.Validator.validate(Validator.java:218)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:126)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:209)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:249)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1014)
... 68 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:174)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:238)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:280)
... 74 more
I tested the server by using a plain javamail app with no extra settings and it worked fine.
But using Seams mail-tags the Exception occurs.
- Is there a way to disable SSL?
I realy don't need SSL.
I found these properties in a forum
mail.smtp.ssl.trust="*"
mail.smtp.starttls.enable="true"
How could I pass them the properties above through seam framework down to javamail ?
The error you're getting means that one of the certificates (presumably the server's certificate) isn't trusted by your JavaMail client. Since you seem to be using STARTTLS, you're effectively using SSL/TLS.
You could perhaps try something like mail.smtp.starttls.enable="false" if you don't want to use SSL/TLS at all, although some SMTP servers will force you to use it (either SSL/TLS on connection or via STARTTLS) to proceed any further.
Alternatively, if you change your mind and want/need to use SSL, make sure your trust store on the client side contains a trust anchor (CA certificate) that can be used to verify your server certificate. (Note that the mail.smtp.ssl.checkserveridentity default to false is insecure, so you'd want to change that to true, and not use mail.smtp.ssl.trust="*".)
According to Seam reference manual and Seam forum you should be able to disable TLS and SSL directly in your components.xml configuration:
<mail:mail-session debug="true" tls="false" ssl="false" ... />