This is a follow-up question to "SSL Handshaking Using Self-Signed Certs and SSLEngine (JSSE)".
I have implemented a NIO Webserver that can process SSL and non-SSL messages on the same port. In order to distinguish between SSL and non-SSL messages, I check the first byte of the inbound request to see if it is a SSL/TLS message. Example:
byte a = read(buf);
if (totalBytesRead==1 && (a>19 && a<25)){
parseTLS(buf);
}
In the parseTLS() method I instantiate an SSLEngine, initiate the handshake, wrap/unwrap messages, etc. Everything seems to work fine for most modern web browsers (Firefox 10, IE 9, Safari 5, etc).
Problem is that older web browsers like IE 6 and libraries like Java's URLConnection class seem to initiate the SSL/TLS handshake differently. For example, the first few bytes from IE 6 look something like this (hex values):
80 4F 01 03 00 ...
If I pass the message to the SSLEngine, it doesn't seem to recognize the message and throws an Exception.
javax.net.ssl.SSLException: Unsupported record version Unknown-0.0
So what exactly is IE 6 and Java's URLConnection class sending over? Is this a valid SSL/TLS message that the JSSE SSLEngine can support? Do I have to do some pre-processing or negotiate with the client to send a different message?
Thanks in advance!
UPDATE
Thanks to Bruno and EJP and some further debugging I have a much better understanding of what's going on. As Bruno correctly pointed out, the IE6 and Java 6 clients are sending over a SSLv2 ClientHello. Contrary to one of my earlier comments, the SSLEngine in Java 1.6 can in fact unwrap the SSLv2 message and generate a valid response to send back to the client. The SSLException I reported earlier was an error on my side and has nothing to do with the SSLEngine (I incorrectly assumed that the client was done sending data over and I ended up with an empty ByteBuffer when the SSLEngine was expecting more data to unwrap).
This looks like an SSLv2 Client Hello (see TLS specification):
TLS 1.1 clients that support SSL Version 2.0 servers MUST send SSL
Version 2.0 client hello messages [SSL2]. TLS servers SHOULD accept
either client hello format if they wish to support SSL 2.0 clients on
the same connection port. The only deviations from the Version 2.0
specification are the ability to specify a version with a value of
three and the support for more ciphering types in the CipherSpec.
80 4F is the length and the high bit must be set to 1 (see msg_length description).
01 is the message type (Client Hello)
03 00 is the highest supported version (SSLv3 here)
Since Java 7, this is now disabled by default.
EDIT:
Just to clarify, this isn't really an SSLv2 Client Hello, this is a Client Hello for SSLv3 in the SSLv2 format. In this case, the server will reply with a (proper) SSLv3 Server Hello (corresponding to the 03 00 requested version number). The same also works for TLS 1.0, 1.1 and 1.2, although the usage of this format is progressively deprecated.
A JSSE 7 SSLServerSocket will still understand such a Client Hello and reply appropriately with the SSLv3/TLS1.x Server Hello.
Related
I have TLS client writen in JAVA. When i set
sslContext = TLS1.2
and enabled protocols to SSLv3 TLSv1 TLSv1.1 TLS1.2
it works fine with almost all servers. Client start handshake with TLSv1.2. If server can only TLSv1, connection downgrades to TLSv1 during handshake.
But! We use about 15 such TLSv1 services and 4 of them fails to downgrade. Servers are all different:
Microsoft-IIS/7.5, nginx, IBM_HTTP_SERVER, SQLAnywhere.
Errors like this:
SEND TLSv1 ALERT: fatal, description = unexpected_message X2
handling exception: javax.net.ssl.SSLException: Invalid Padding length: 139
javaException : Received fatal alert: unexpected_message
If i set enabled protocols to TLSv1, that servers works fine. But i want set sslContext and enabled protocols to be compatible with all services and preferably work with TLSv1.2
Any suggestions will be appreciated!
But i want set sslContext and enabled protocols to be compatible with all services and preferably work with TLSv1.2
First, there are enough servers out there which don't do TLS 1.2 but the best they can do is TLS 1.0 (and some even only SSL 3.0). A proper implementation of the TLS stack would in this case simply send a TLS 1.0 reply and thus enforcing the connection to be TLS 1.0 which is supported in your client. Forcing these servers to TLS 1.2 will not work because they don't implement it. All you could do is replace the servers with newer implementations.
And then there are several implementations out there which incorrectly implement the protocol. Some of these simply close the connection if you send a TLS 1.2 ClientHello or use some TLS extensions they never heard of. If you can not replace these servers you have to communicate with them in a way they will understand, like speaking only TLS 1.0 etc. Browsers already deal with this rotten servers by slowly downgrading the connection, i.e. if TLS 1.2 causes a handshake failure they will try again with TLS 1.0 etc. Often browsers even have hard coded a list of servers which are too rotten to even play nice after trying simple downgrads and which need to have special ciphers or something like this.
I am trying to record a https site through jmeter (version 2.13, java version - 1.8u31) and I am getting SSLHandshakeException while connecting to a https site. The error message is
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2011)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1113)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1363)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1391)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1375)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:436)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.jmeter.protocol.http.sampler.MeasuringConnectionManager$MeasuredConnection.open(MeasuringConnectionManager.java:107)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:643)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.executeRequest(HTTPHC4Impl.java:517)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:331)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:74)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1146)
at org.apache.jmeter.protocol.http.proxy.Proxy.run(Proxy.java:240)
I have turned on debug logging for SSL but I wasn't able to understand the root cause. It seems that the java client sends the ClientHello but does not receive the ServerHello message (where the server chooses the highest version of SSL and the best cipher suite that both the client and server support and sends this information to the client). I see differences between the protocol versions being sent, read and received by the client (TLSv1.1 vs TLSv1.2)
Is this the root cause ? If so, how can I fix it?
The logs are pasted here - Java SSLHandshakeException Logs - Pastebin.com
Update
As #Anand Bhatt suggested, I analyzed the site with ssllabs and understood the following
The server does not support TLSv1.2 which is supported by java 8
The server supports only one cipher suite - TLS_RSA_WITH_AES_256_CBC_SHA
Java 8u31 doesn't support the cipher suite that the server supports and that's most probably the issue.
Does that sound right? If so, how do we make the java 8 client support the cipher suite that the server supports?
SSLlabs is apparently testing "out of the box" support. Java crypto has a crock dating back to the 1990s when the US government severely restricted export of crypto software,
and as a result the JRE (or JDK) as distributed by then-Sun now-Oracle does not permit use of 256-bit symmetric encryption, which your server is demanding. You must download and install
the "JCE Unlimited Strength Jurisdiction Policy Files" for your Java (major) version; 8 is at http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html .
The README in the file gives longwinded details, but basically you replace two tiny jar files in JRE/lib/security.
TLSv1.2 is not a real issue now. TLS protocol automatically negotiates the highest version supported (and enabled) by both ends.
Java 8 implements SSLv3, TLSv1.0, TLSv1.1 and TLSv1.2, but recent updates (8u31 or 7u75 and up) disable SSLv3 by default because of POODLE;
you can re-enable it if you choose, but you should be reluctant to. (Java 7 implements the same protocol versions, but client by default disables 1.1 and 1.2 because of compatibility concerns at its release several years ago.)
However, because of POODLE and BEAST some security authorities no longer accept SSLv3 and TLSv1.0
as adequately secure; an important example is credit and debit cards, as detailed in https://security.stackexchange.com/a/87077/39571 .
TLSv1.2 includes some technical improvements over 1.1, making it preferred today, and there might be future discoveries that make those
improvements crucial; if your server can't support 1.2 (and maybe higher) at that point you would be in trouble. Similarly the fact that the server's only
supported suite uses plain-RSA key-exchange, i.e. NOT forward secrecy, is considered suboptimal now, and over time may become unacceptable.
keytool (at least with the normally used keystore and truststore files) has nothing to do with symmetric cryptography.
It could likely be relevant if the server uses a CA root (or more exactly and slightly more general, trust anchor)
that your JRE and/or application does not trust, and/or if the server wants client authentication at SSL/TLS level,
which is fairly rare. (Most websites authenticate at the web-application level, or at least HTTP level, if at all.)
SSLLabs checking of the server cert chain (and several other things also) is generally stricter than Java's, and they
didn't complain in that area, so it's unlikely you have a problem there.
I have a unix script running in server "CCC" from which am calling a servlet running in another server "GGG".(The server GGG is a secure server, having extra firewalls compared to normal server)
Also, the CCC is a ETL server, which is having a JRE. The script is calling the servlet using http, since I rule is applied, so redirection is happening to https, and the port no is also changing.
And am getting a bad certificate error, when i try to invoke the servlet.
The servlet URL when given in normal IE or chrome is giving me a valid response, am able to hit the server GGG.
The error am getting is :
javax.net.ssl.SSLHandshakeException: bad certificate
at com.ibm.jsse.bg.a(Unknown Source)
at com.ibm.jsse.b.a(Unknown Source)
at com.ibm.jsse.b.write(Unknown Source)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:139)
at org.apache.commons.httpclient.HttpConnection.flushRequestOutputStream(HttpConnection.java:827)
at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:1975)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:993)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:397)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
at com.tgt.task.client.TaskClient.doGet(TaskClient.java:153)
at com.tgt.task.client.TaskClient.runClient(TaskClient.java:91)
at com.tgt.task.client.TaskClient.completeTask(TaskClient.java:68)
at com.tgt.task.client.TaskClient.main(TaskClient.java:53)
So, initially i checked the certs in cacerts for CCC server, it is having all the root certs, except the cert issued to the URL.
We found the issue started after the BIG IP cert renewal for the URL, i have followed up with my network team to have a look into the issue, they have confirmed that traffic from server CCC is going through fine to GGG passing the firewalls.
So captured the traffic when we tried to invoke the URL, we got the below:
client server SSLV2 282 Client Hello
server client SSLV3 1631 Server Hello
server client SSLv3 433 Certificate
client server SSLV3 190 Alert(Level: Fatal, Description: Bad Certificate)
Can you let me know is the issue because of the SSL versions used by the client and the server ?
or some other thing that i missed out in my analysis.
I even tried to import the cert for the URL in to cacerts in server CCC, but i got the errors:
keytool error: java.lang.Exception: Input not an X.509 certificate
So , i took the PKCS7 cert for the URl and tried to convert it into .cer and tried to import, but got the same error.
If it works with the browser, but not with a script, it has often to do with Server Name Indication (SNI). With SNI you can have multiple hosts with different certificates behind the same IP address. While all current browser support SNI, other implementations might not or might not enable SNI by default. If the client fails to use SNI on a site with multiple certificates behind the same IP it just gets the default certificate, which is often the wrong one.
I don't which Java version you are running, but I suggest you determine your version and then google of how to use SNI with your version. From my understanding SNI was enabled only in Java version 7.
I have checked the captured network packet and found that the URL is providing the correct certificate to the server CCC as per the SSL configuartions at BIG IP end, the client CCC is not able to validate it. And returning a bad certificate message.
here i tried openssl and tried the command:
s_client -connect host:443
verify error:num=20:unable to get local issuer certificate
verify error:num=27:certificate not trusted
verify error:num=21:unable to verify the first certificate
`No client certificate CA names sent`
SSL handshake has read 1745 bytes and written 304 bytes
New, TLSv1/SSLv3, Cipher is RC4-SHA
Server public key is 1024 bit
Compression: NONE
Expansion: NONE
SSL-Session:
Start Time: 1402019174
Timeout : 300 (sec)
Verify return code: 21 (unable to verify the first certificate)
I guess my server is not sending the complete trust chains list, as a result, client is not able to validate it.
Am not sure whether my analysis is correct or not, also i referred the below post:
OpenSSL: unable to verify the first certificate for Experian URL
If i need to send my server to send the complete trusted chain list, where i need to change the settings ? at BIG Ip end, or at server end ?
(Disclaimer: I am by no stretch of the imagination a security expert nor a windows expert for that matter)
Setup:
server on our end: java 1.6 (already added bouncycastle to the security file) on windows 2003 server
third party client: windows 2008 server with biztalk
all renegotiation system properties introduced due to the renegotiation attack are "enabled" on the server side (not safe I know)
Ideally we want to fix this at our end but it is possible to propose a fix to the client if necessary.
The client server has to connect to our server over a HTTPS connection but it always fails, wireshark shows the following conversation:
> TLSv1: Client Hello
< TLSv1: Alert (21): Unexpected Message
As per the RFC (http://www.ietf.org/rfc/rfc2246.txt) the alert(21) refers to a failed decryption and from what I can see in wireshark, none of the ciphers proposed by the client are actually supported by JRE 1.6 (as per http://docs.oracle.com/javase/6/docs/technotes/guides/security/SunProviders.html#SupportedCipherSuites)
In an effort to reproduce the error to be able to examine it closer, I tested with some other software:
wfetch on windows xp with "https" selected will perform the initial client handshake in SSLv2, the server will switch to TLSv1 to answer, this works
wfetch on windows xp with configured to use "TLSv1" for the initial handshake will fail in the same way as the biztalk server
wfetch on windows 2008 with configured "https" will use "TLSv1" for the initial handshake and fail in the same way as the biztalk server
IE (on windows xp) will initially try a TLSv1 handshake with the same failed result but immediately tries again using SSLv3 which works
(at this point I figure all microsoft software uses a central configuration available at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\Schannel)
firefox uses SSLv3 for the entire conversation, so no problem there
OpenSSL performs an initial handshake in SSLv2, and the server switches to TLSv1 when it answers, no problem there
OpenSSL can be forced to do the initial handshake in TLSv1 as well, it offers a list of 27 ciphers (as opposed to the 11 ciphers proposed by windows-based software) and can connect without a problem
To my untrained eye this reinforces the idea that an incompatible cipher proposition is the root cause where windows only supports cipher suites that are not supported by JVM (for TLSv1).
I have installed bouncy castle as an additional provider in the java.security file to no avail.
I have searched high and low and only found a reference that maybe websphere supports the windows ciphers for TLSv1 but no way of downloading a standalone provider to test it.
JRE 1.7 is not supported by the software we run on our JVM, so upgrading is not an option (perhaps the security provider can be downgraded safely? I haven't found a download for it yet though)
I have found no way to add a cipher to windows short of writing c++ code (I've played around with the above mentioned registry settings without effect).
So in conclusion I wonder if one of the following things would fix it and how they should be accomplished:
add a provider to the jvm that can work with the ciphers for TLSv1 that are proposed by windows
somehow force the client to do the initial handshake in SSLv3 (preferably not SSLv2) or at least retry if the TLSv1 handshake fails
somehow add a JVM-supported cipher for TLSv1 to the client windows
Any other solutions are of course also appreciated.
EDIT
The Java version is Java version (64 bit): 1.6.0_19-b04.
The list of proposed ciphers is:
TLS_RSA_WITH_RC4_128_MD5
TLS_RSA_WITH_RC4_128_SHA
TLS_RSA_WITH_3DES_EDE_CBC_SHA
TLS_RSA_WITH_DES_CBC_SHA
TLS_RSA_EXPORT1024_WITH_RC4_56_SHA
TLS_RSA_EXPORT1024_WITH_DES_CBC_SHA
TLS_RSA_EXPORT_WITH_RC4_40_MD5
TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5
TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
TLS_DHE_DSS_WITH_DES_CBC_SHA
TLS_DHE_DSS_EXPORT1024_WITH_DES_CBC_SHA
The unlimited strength cryptography policy files are installed. I have tried to set javax.net.debug=all and started the server from the console, no additional output appeared. I have set sun.security.ssl.allowUnsafeRenegotiation=true to no avail.
EDIT 2
It turns out the software we are using uses a custom stack for HTTPs instead of the default. A fix was issued which seems to solve the problem though I don't know exactly which part of the TLS request triggered the error (seeing as most TLSv1 handshakes did succeed).
Thanks for the feedback, it has been an interesting if futile search. Live and learn.
It turns out the software we are using uses a custom stack for HTTPs instead of the default. A fix was issued which seems to solve the problem though I don't know exactly which part of the TLS request triggered the error (seeing as most TLSv1 handshakes did succeed).
Thanks for the feedback, it has been an interesting if futile search. Live and learn.
You could read my article on detecting cipher strength (just to make sure you installed the jce ciphers correctly) . In your question you say you installed unlimited ciphers but then you reference 128 and 40-bit keys. So, I am confused by what you have. Also, could you check the cipher strength on the SSL cert you are trying to connect to and let us know what it is and what the algorithm is? Also, make sure your policy file for JDK has the proper rights to allow unlimited strength.
Finally, can you connect to a "known good" SSL site to verify your client handshakes correctly? (Gmail web for example)
I am using https to connect to an https server.
Specifically I am using apache httpclient and I configure the ssl context to use my keystore and truststore.
The https server I am using is IIS7 and is configured to require client authentication.
I think I have set it up properly.
Anyway, if I configure the httpClent's ssl context with a keystore (i.e. with client certificates) valid for IIS then there is no problem connecting.
Now my problem is the following:
If I do not configure the ssl context with any client certificate to send to IIS, there is no connection with the server. What makes me think though, is the fact that I was expecting to see some java exception in the code as a result of a hanshake failure alert.
Monitoring what is happening with wireshark, I could not see a certificate request from IIS to my application, but I noticed that after ServerHelloDone everything was encrypted.
I did not expect that. I think the handshake is usually in cleartext.
I used private key to decrypt traces and i saw a certificate request from IIS but after many starting and opening of new connections.
My app send back as a response a certificate of length 0 and IIS replies with a TLSv1 Finished.
After that the packets stop (i.e. seems that the communication ends).
I was expecting a handshake alert.
My question is, is this how it is supposed to work or at least how IIS works?
Or if I do not see the alert something is wrong with my use case?
Thanks
It sounds like IIS is only requiring client certificates for certain URLs (ie, for example.com/foo, but not example.com/bar).
In the initial handshake, it does not know which url you are requesting, so it does not require a certificate. When it sees that you are requesting a restricted resource (/foo), it then rehandshakes, requiring a certificate.
However, I would still expect a handshake_failure to occur.
As I was saying in an answer to this question, as far as I remember, IIS uses re-negotiation to get the client certificate. You should be able to change this behaviour using netsh and clientcertnegotiate=enable (depending on the version of IIS you're using).
You might also be interest in this similar question.
Failing to supply a certificate in response to a CertificateRequest isn't an SSL protocol error, so there is no handshake_error. 'Requiring' instead of just 'needing' client certificates is added-in by SSL libraries, and all they can do if you don't send one is just close the connection.