Develop Reference

Getting SSLHandshake Exception between two Websphere Servers

There are 2 Websphere servers. First is Client and Second is Server (8.5.5.13).
An End-User makes a SOAP request to Client. Client receives the request, do some processing and then makes a REST request to the Server.
When the Server receives the request from Client it throws the below exception.
ERROR 22304 --- [ebContainer : 2]
c.i.w.s.c.impl.SSLHandshakeErrorTracker : SSLC0008E: Unable to
initialize SSL connection. Unauthorized access was denied or security
settings have expired. Exception is
javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1
not enabled or not supported
We have tested after enabling TLSv1.2 and then TLSv1 at the our Server level. We have exported the certificate from the Server and added in the Client trust store as well. But still we are not able to resolve the error.
Questions:
1. Do we need to keep the same TLS version at our Server which is present in the Client WebSphere?
2. We have deployed our application is java7. Is it due to Java version we are facing the issue?
3. Is it due to Certificate only, or something else?
When we disable the Global security at our Server then the request is processing correctly.

This error was due to the TLS version incompitability at Server. The client was sending the request using TLSv1 and server has TLSv1.2 enabled in it. For SSL handshake, the first thing is TLS compitability. The client and server should have same TLS versions.
After this if you are using one way SSL handshake, then the certificate from the server machine needs to extract and the same should be installed in the Client machine. This way SSL Handshake will happen.

JAVA TLS 1.2 downgrade to TLS 1.0 sometimes error

I have TLS client writen in JAVA. When i set
sslContext = TLS1.2
and enabled protocols to SSLv3 TLSv1 TLSv1.1 TLS1.2
it works fine with almost all servers. Client start handshake with TLSv1.2. If server can only TLSv1, connection downgrades to TLSv1 during handshake.
But! We use about 15 such TLSv1 services and 4 of them fails to downgrade. Servers are all different:
Microsoft-IIS/7.5, nginx, IBM_HTTP_SERVER, SQLAnywhere.
Errors like this:
SEND TLSv1 ALERT: fatal, description = unexpected_message X2
handling exception: javax.net.ssl.SSLException: Invalid Padding length: 139
javaException : Received fatal alert: unexpected_message
If i set enabled protocols to TLSv1, that servers works fine. But i want set sslContext and enabled protocols to be compatible with all services and preferably work with TLSv1.2
Any suggestions will be appreciated!

But i want set sslContext and enabled protocols to be compatible with all services and preferably work with TLSv1.2
First, there are enough servers out there which don't do TLS 1.2 but the best they can do is TLS 1.0 (and some even only SSL 3.0). A proper implementation of the TLS stack would in this case simply send a TLS 1.0 reply and thus enforcing the connection to be TLS 1.0 which is supported in your client. Forcing these servers to TLS 1.2 will not work because they don't implement it. All you could do is replace the servers with newer implementations.
And then there are several implementations out there which incorrectly implement the protocol. Some of these simply close the connection if you send a TLS 1.2 ClientHello or use some TLS extensions they never heard of. If you can not replace these servers you have to communicate with them in a way they will understand, like speaking only TLS 1.0 etc. Browsers already deal with this rotten servers by slowly downgrading the connection, i.e. if TLS 1.2 causes a handshake failure they will try again with TLS 1.0 etc. Often browsers even have hard coded a list of servers which are too rotten to even play nice after trying simple downgrads and which need to have special ciphers or something like this.

SSL handshake unrecognized_name exception

We have a process use HttpClient to make https connection with different URLs. Our JDK version is 1.7. We found some URLs, if we use default JSSE SIN extension flag, which is turn on, we got the following exception:
javax.net.ssl.SSLProtocolException: handshake alert: unrecognized_name
If we set -Djsse.enableSNIExtension=false, some other URLs throw this exception:
javax.net.ssl.SSLException: Received fatal alert: unrecognized_name
Does anybody have idea about this?

The SNI extension will add the server_name(hostname) in the client hello message at the SSL handshake stage.
It may be needed by a server that serves requests for different hostnames or virtual hosts. It is used to resolve the Certificate for the hostname the client intend to communicate with. The server may be configured with a default certificate if the host name is not added or not found. The default certificate may be a multisigned certificate for all the hosts it serves.
The first warning you get may be a server responding with a default certificate but that also warns that the hostname is not recognized. The server probably does not recognize the hostname simply because it is not included in the configuration. Instead of ignoring the warning and use the default certificate, Java seems to throw an exception.
In the other case where you disabled SNI, you do not include the hostname anymore and the request is sent to a server that does not have a default certificate. It will respond with a fatal error because it cannot resolve any certificate for you.

Java SSL: "fatal error: 80: ... unwrapping net record" after adding the HTTPS EndpointIdentificationAlgorithm

Java 7u9
Error msg in title is: "fatal error: 80: problem unwrapping net record". SO wouldn't let me put "problem" in the title.
I am building a Java HTTPS client against Netty. The SSL handshake was working until I added added the "HTTPS" endpoint identification algorithm to enable server hostname verification:
SSLEngine engine = tcpHelper.getSSLContext().createSSLEngine();
SSLParameters sslParameters = engine.getSSLParameters();
sslParameters.setEndpointIdentificationAlgorithm("HTTPS");
engine.setSSLParameters(sslParameters);
engine.setUseClientMode(true);
After adding the algorithm, the SSL handshake hangs and the connection eventually times out. With SSL debugging enabled (javax.net.debug=all), I can see that the handshake now fails after ServerHello, after the server sends it's cert chain, after
*** ServerHelloDone
1761586552#qtp-1653588482-2, WRITE: TLSv1.2 Handshake, length = 3294
on the server. The client receives and displays the cert chain, and then fails with:
New I/O worker #3, fatal error: 80: problem unwrapping net record
java.lang.RuntimeException: Delegated task threw Exception/Error
%% Invalidated: [Session-1, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256]
New I/O worker #3, SEND TLSv1.2 ALERT: fatal, description = internal_error
New I/O worker #3, WRITE: TLSv1.2 Alert, length = 2

Java 7u9
First, thanks to Bruno for his help on this related question which lead me to final answer.
Answering my own question. Hard won knowledge.
The solution is to add the host and port of the request target to the constructor when you create the SSL Engine:
SSLEngine engine = tcpHelper.getSSLContext().createSSLEngine(targetHost, targetPort);
Without this, Java will ultimately throw a NullPointerException way down deep in the SSL libs (IPAddressUtil.textToNumericFormatV4), which results in the not-very-helpful error message in the SSL debug output.

Receiving SSLHandshakeException: handshake_failure despite my client ignoring all certs

I have a Java program that connects to a webserver using SSL/TLS, and sends various HTTP requests over that connection. The server is localhost and is using a self-signed cert, but my code is using custom TrustManagers, and ignores invalid certificates. It has worked perfectly until now.
The only difference on the server is that it used to run jboss 6 and is now running jboss 7. I'm not sure if this is a configuration issue, or whether there is a problem with my code, but I get the same errors if I try to connect using other Java-based programs like WebScarab or ZAP.
In any case, is there anything I can do to my code to get around this problem? Here is the error in full:
Received fatal alert: handshake_failure
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown Source)
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.recvAlert(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(Unknown Source)
Here are the debug messages before the failure:
main, WRITE: TLSv1 Handshake, length = 75
main, WRITE: SSLv2 client hello message, length = 101
main, READ: TLSv1 Alert, length = 2
main, RECV TLSv1 ALERT: fatal, handshake_failure

So I found the problem. There might be a bug in Java, but the client seems to initiate a TLSv1 Handshake, but then sends an SSLv2 client hello message, at which point the server rejects the connection.
This happens even if you create your SSLContext with an instance of TLS:
SSLContext sslContext = SSLContext.getInstance("TLS");
The solution is to set a system property before any connection attempts are made:
System.setProperty("https.protocols", "TLSv1");
There are probably other solutions to it, but this one worked for me.

The info you provide is very little as well as your stack trace.
I'll take a guess here.
What I suspect is that in the new server the protocol is TLSv1 while your clients try to connect with SSLv3 (or less) and as a result the handshake fails.
Change you clients to use higher version of TLS
or
Make your webserver support SSLv3 as well. I know how to do this in Tomcat but not in JBoss.
If this doesn't work update the post with more info (and a full stack trace).
You should enable ssl debug info -Djavax.net.debug=ssl

Was this ever resolved?
I had the exact same problem, essentially I was receiving a handshake exception immediately following the clientHello. So The chain of events was
I would present my certificate to the server
Server would imediately respond with a handshake failure. (I would not even get a Server Hello back).
Eventually I found that the server was requiring a stronger encryption/decryption algorithm than what I Was supplying in the initial handshake phase (Ie. Client and Server could not agree on a mutual encryption algorithm to use for the ssl communication).
I need to install the Unlimited Java JCE (Java Cryptography Extension Policy). There are export rules on using this, so if you ship your code overseas that may have implications..however this is what solved my problem.
This link explains how to install the updated policies
http://suhothayan.blogspot.com/2012/05/how-to-install-java-cryptography.html
This was also a great link that helped me understand exactly what was going on
https://support.f5.com/kb/en-us/solutions/public/15000/200/sol15292.html#id
This may or may not be the issue, but when the handshake fails immediately after the client Hello, it looks like the client and the server can not agree on something (in many cases its the encryption algorithms that they will mutually need to communicate).

You are seeing this error most probably because the keystore that your JBoss 6 had access to is not accessible to your JBoss 7 instance.
What I would recommend is the following.
Your self-signed server certificate must be imported into a truststore
keytool -import -alias gridserver -file server.crt -storepass $YOUR_PASSWORD_HERE -keystore server.keystore
Add the following properties to your run.conf
-Djavax.net.ssl.keyStoreType=pkcs12
-Djavax.net.ssl.trustStoreType=jks
-Djavax.net.ssl.keyStore=clientcertificate.p12
-Djavax.net.ssl.trustStore=server.keystore
-Djavax.net.debug=ssl # very verbose debug. Turn this off after everything looks good.
-Djavax.net.ssl.keyStorePassword=$YOUR_PASSWORD_HERE
-Djavax.net.ssl.trustStorePassword=$YOUR_PASSWORD_HERE

The stack trace is from you client code and your client 'Received [a] fatal alert'. In other words, the SSL error happened in Jboss, not your client.
Your client side custom TrustManagers have therefore nothing to do with it. My wild guess is that your new Jboss 7 is configured to require client certificate and your client did not present any.
To debug your SSL connection, use openssl and try this:
openssl s_client -connect jboss.server.com:443
or is it is an SSLV3 server
openssl s_client -connect jboss.server.com:443 -ssl3
This should print a lot of interesting information.

I think this is related to a Java 7 bug. It is hard to be sure without more details.

For me solution was : System.setProperty("https.protocols", "TLSv1.1,TLSv1.2");