I have an Apache web server that runs several TLS virtualhosts with different certs and SNI.
I can access the various virtual hosts just fine using curl (presumably SNI makes it work). I can also access them fine with a little command-line Java program that basically just openConnection()s on a URL.
In my Tomcat application, the basic same client-side code accesses the same Apache server as a client, but always ends up with the default cert (defaulthost.defaultdomain) instead of the cert of the virtual host that was specified in the URL that it attempts to access. (This produces a SunCertPathBuilderException -- basically it can't verify the certificate path to the cert, which of course is true as it is a non-official cert. But then the default cert should not be used anyway.)
It's just as if SNI had been deactivated client-side in my application / Tomcat. I am at a loss why it should behave differently between my app and the command-line; same JDK, same host etc.
I found property jsse.enableSNIExtension, but I verified that it is set to true for both cases. Questions:
Any ideas, even wild ones, why these two programs behave differently?
Any ideas how I would debug this?
This is Arch Linux on 86_64, JDK 8u77, Tomcat 8.0.32.
This answer comes late, but we just have hit the problem (I can't believe it, it seems a very big bug).
All what it said seems true, but it's not default HostnameVerifier the culprit but the troubleshooter. When HttpsClient do afterConnect first try to establish setHost (only when socket is SSLSocketImpl):
SSLSocketFactory factory = sslSocketFactory;
try {
if (!(serverSocket instanceof SSLSocket)) {
s = (SSLSocket)factory.createSocket(serverSocket,
host, port, true);
} else {
s = (SSLSocket)serverSocket;
if (s instanceof SSLSocketImpl) {
((SSLSocketImpl)s).setHost(host);
}
}
} catch (IOException ex) {
// If we fail to connect through the tunnel, try it
// locally, as a last resort. If this doesn't work,
// throw the original exception.
try {
s = (SSLSocket)factory.createSocket(host, port);
} catch (IOException ignored) {
throw ex;
}
}
If you use a custom SSLSocketFactory without override createSocket() (the method without parameters), the createSocket well parametrized is used and all works as expected (with client sni extension). But when second way it's used (try to setHost en SSLSocketImpl) the code executed is:
// ONLY used by HttpsClient to setup the URI specified hostname
//
// Please NOTE that this method MUST be called before calling to
// SSLSocket.setSSLParameters(). Otherwise, the {#code host} parameter
// may override SNIHostName in the customized server name indication.
synchronized public void setHost(String host) {
this.host = host;
this.serverNames =
Utilities.addToSNIServerNameList(this.serverNames, this.host);
}
The comments say all. You need to call setSSLParameters before client handshake. If you use default HostnameVerifier, HttpsClient will call setSSLParameters. But there is no setSSLParameters execution in the opposite way. The fix should be very easy for Oracle:
SSLParameters paramaters = s.getSSLParameters();
if (isDefaultHostnameVerifier) {
// If the HNV is the default from HttpsURLConnection, we
// will do the spoof checks in SSLSocket.
paramaters.setEndpointIdentificationAlgorithm("HTTPS");
needToCheckSpoofing = false;
}
s.setSSLParameters(paramaters);
Java 9 is working as expected in SNI. But they (Oracle) seem not to want fix this:
https://bugs.openjdk.java.net/browse/JDK-8072464
https://bugs.openjdk.java.net/browse/JDK-8144566
and many more
After some hours of debugging the JDK, here is the unfortunate result. This works:
URLConnection c = new URL("https://example.com/").openConnection();
InputStream i = c.getInputStream();
...
This fails:
URLConnection c = new URL("https://example.com/").openConnection();
((HttpsURLConnection)c).setHostnameVerifier( new HostnameVerifier() {
public boolean verify( String s, SSLSession sess ) {
return false; // or true, won't matter for this
}
});
InputStream i = c.getInputStream(); // Exception thrown here
...
Adding the setHostnameVerifier call has the consequence of disabling SNI, although the custom HostnameVerifier is never invoked.
The culprit seems to be this code in sun.net.www.protocol.https.HttpsClient:
if (hv != null) {
String canonicalName = hv.getClass().getCanonicalName();
if (canonicalName != null &&
canonicalName.equalsIgnoreCase(defaultHVCanonicalName)) {
isDefaultHostnameVerifier = true;
}
} else {
// Unlikely to happen! As the behavior is the same as the
// default hostname verifier, so we prefer to let the
// SSLSocket do the spoof checks.
isDefaultHostnameVerifier = true;
}
if (isDefaultHostnameVerifier) {
// If the HNV is the default from HttpsURLConnection, we
// will do the spoof checks in SSLSocket.
SSLParameters paramaters = s.getSSLParameters();
paramaters.setEndpointIdentificationAlgorithm("HTTPS");
s.setSSLParameters(paramaters);
needToCheckSpoofing = false;
}
where some bright mind checks whether the configured HostnameVerifier's class is the default JDK class (which, when invoked, just returns false, like my code above) and based on that, changes the parameters for the SSL connection -- which, as a side effect, turns off SNI.
How checking the name of a class and making some logic depend on it is ever a good idea escapes me. ("Mom! We don't need virtual methods, we can just check the class name and dispatch on that!") But worse, what in the world does SNI have to do with the HostnameVerifier in the first place?
Perhaps the workaround is to use a custom HostnameVerifier with the same name, but different capitalization, because that same bright mind also decided to do case-insensitive name comparison.
'nuff said.
This is a Java 8 bug (JDK-8144566) fixed by 8u141. See Extended server_name (SNI Extension) not sent with jdk1.8.0 but send with jdk1.7.0 for more.
Related
It is a known problem to use the Java FTPSClient of Apache commons-net with session resumption. Session resumption is a security feature which a FTPS server can require for data connections. The Apache FTPSClient does not support session resumption, and the JDK APIs make it hard to build a custom implementation. There are a couple of workarounds using reflection, see e.g. this answer and this commons-net bug entry.
I use such a workaround (see snipped below) in JDK 11 and tested it against a local FileZilla Server. It works with FileZilla Server 0.9.6, but it doesn't with FileZilla Server 1.2.0, which is the latest version at the time of writing. With that version, when trying to establish a data connection, the server responds with:
425 Unable to build data connection: TLS session of data connection not resumed.
As I said, FileZilla Server 0.9.6 is fine with how I do session resumption, and I made sure that the setting for requiring session resumption is activated.
In FileZilla Server 1.2.0, such settings are now set implicitly and cannot be changed via the GUI, maybe not at all. Are there some server settings that I can tweak for this to work? Or is it an issue with how I implemented the workaround? Does anyone experience similar issues?
This is the workaround I am using:
public class FTPSClientWithSessionResumption extends FTPSClient {
static {
System.setProperty("jdk.tls.useExtendedMasterSecret", "false");
System.setProperty("jdk.tls.client.enableSessionTicketExtension", "false");
}
#Override
protected void _connectAction_() throws IOException {
super._connectAction_();
execPBSZ(0);
execPROT("P");
}
#Override
protected void _prepareDataSocket_(Socket socket) throws IOException {
if (useSessionResumption && socket instanceof SSLSocket) {
// Control socket is SSL
final SSLSession session = ((SSLSocket)_socket_).getSession();
if (session.isValid()) {
final SSLSessionContext context = session.getSessionContext();
try {
final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
sessionHostPortCache.setAccessible(true);
final Object cache = sessionHostPortCache.get(context);
final Method putMethod = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
putMethod.setAccessible(true);
Method getHostMethod;
try {
getHostMethod = socket.getClass().getMethod("getPeerHost");
}
catch (NoSuchMethodException e) {
// Running in IKVM
getHostMethod = socket.getClass().getDeclaredMethod("getHost");
}
getHostMethod.setAccessible(true);
Object peerHost = getHostMethod.invoke(socket);
InetAddress iAddr = socket.getInetAddress();
int port = socket.getPort();
putMethod.invoke(cache, String.format("%s:%s", peerHost, port).toLowerCase(Locale.ROOT), session);
putMethod.invoke(cache, String.format("%s:%s", iAddr.getHostName(), port).toLowerCase(Locale.ROOT), session);
putMethod.invoke(cache, String.format("%s:%s", iAddr.getHostAddress(), port).toLowerCase(Locale.ROOT), session);
}
catch (Exception e) {
throw new IOException(e);
}
}
else {
throw new IOException("Invalid SSL Session");
}
}
}
}
The address under which the socket is cached is determined using getPeerHost, getInetAddress().getHostName(), and getInetAddress().getHostAddress(). I tried several combinations of doing or not doing these three, but I always get the same result.
Edit:
Here is a screenshot of the server logs of the full session:
As stated in this StackOverflow post it is possible to tell the JVM that only TLS 1.2 should be used.
Here is the link to the original answer which worked for me: command for java to use TLS1.2 only
You have to add a command line parameter at the start of the JVM in this case this is: java -Djdk.tls.client.protocols=TLSv1.2 -jar ... <rest of command line here>
This simple parameter worked for me, now I can connect and transfer data from a FTP-Server wich runs FileZilla FTP-Server 1.3.0
I challenged myself to create a minecraft server with the proviso that most of the plugins I will use will be just mine. I have already made many different plugins from shops to various minigames. Now I have decided to code my own login registration plugin. Basically everything is already functional and ready, but I would like to add one feature there. This feature is that when a player connects to the server (warez), my plugin checks the player's session to see if it is original (I mean that player have bought game and he is logged throught original mojang launcher with valid mojang session. Players should connecting to server in warez mode (not logged in, using not official launcher)). I don't just mean the name as mentioned here, but his mojang session. In short, whether it is logged in via the original launcher with its e-mail and password.
I absolutely don't know how verifying the originality of players works, I know that mojang has an API but I don't know if it offers such a possibility. I would also like to know if the client is sending a hash from the UUID or sessionID to the server, which can be compared with the mojang API, or I don't know.
UPDATE:
I found these two articles:
1) https://wiki.vg/Protocol#Login
2) https://wiki.vg/Protocol_Encryption#Authentication
From this articles I roughly understood that during the connection of the client I will send the server ID and ciphers together with the server and then the subsequent hash on which they will agree will send the client via POST to the mojang servers and my server should then ask from the mojang if it is logged in on my server the client.
UPDATE2
I thought of the following:
If I programmed my own proxy in the phenomenon to which players would connect and this proxy would reproduce everything to the server. Thus, I would be able to let in who I would like to go and I can also do cross-checks via the mojang page with the server ID and hash. But it would be 3-rd party software, it would not be a plugin.
Modify the spigot itself, by that I mean do the above somewhere at the SSLServerSocket level, where the spigot server receives all the socket and the data from them. There, if I code the bridge over which the data would pass, I am also able to agree with the client SERVER ID, calculate a hash and verify it from the mojang server. But it would still not be within the plugin but in the servers.
Override some of the deep parts of the server mentioned above from the plugin. The plugin would replace some parts of the server after loading.
Now my questions is, how to replace some mentioned parts of the server from plugin? Is good idea to try use reflection (i am noob, with reflection) and replace some functions with my functions, that would calling back to spigot low level code?
Thank you very much for any advice.
Have a nice day.
PS: Sorry for my bad english.
public boolean isCracked(ProxiedPlayer player) {
String name = player.getName();
UUID actualUUID = player.getUniqueId();
String actualUUIDStr = uuid.toString();
String offlineUUIDStr = getMd5("OfflinePlayer:"+name);
if(offlineUUIDStr.equals(actualUUIDStr)) {
return true;
}
return false;
}
// Taken from https://www.geeksforgeeks.org/md5-hash-in-java/
public static String getMd5(String input)
{
try {
// Static getInstance method is called with hashing MD5
MessageDigest md = MessageDigest.getInstance("MD5");
// digest() method is called to calculate message digest
// of an input digest() return array of byte
byte[] messageDigest = md.digest(input.getBytes());
// Convert byte array into signum representation
BigInteger no = new BigInteger(1, messageDigest);
// Convert message digest into hex value
String hashtext = no.toString(16);
while (hashtext.length() < 32) {
hashtext = "0" + hashtext;
}
return hashtext;
}
// For specifying wrong message digest algorithms
catch (NoSuchAlgorithmException e) {
throw new RuntimeException(e);
}
}
}
I am working on a application that connects to an SFTP server and downloads files using Apache Commons VFS, it works just fine, with the exception that the system needs to allow the user to specify a proxy, as needed.
Now, I know Apache Commons VFS is built on top of Jsch and I know Jsch contains the classes: com.jcraft.jsch.ProxyHTTP, com.jcraft.jsch.ProxySOCKS4 and com.jcraft.jsch.ProxySOCKS5.
The code below is an extract of VFS class org.apache.commons.vfs2.provider.sftp.SftpClientFactory:
public static Session createConnection(
...
final SftpFileSystemConfigBuilder.ProxyType proxyType = builder.getProxyType(fileSystemOptions);
...
final String proxyUser = builder.getProxyUser(fileSystemOptions);
final String proxyPassword = builder.getProxyPassword(fileSystemOptions);
Proxy proxy = null;
if (SftpFileSystemConfigBuilder.PROXY_HTTP.equals(proxyType)) {
proxy = createProxyHTTP(proxyHost, proxyPort);
((ProxyHTTP)proxy).setUserPasswd(proxyUser, proxyPassword);
} else if (SftpFileSystemConfigBuilder.PROXY_SOCKS5.equals(proxyType)) {
proxy = createProxySOCKS5(proxyHost, proxyPort);
((ProxySOCKS5)proxy).setUserPasswd(proxyUser, proxyPassword);
} else if (SftpFileSystemConfigBuilder.PROXY_STREAM.equals(proxyType)) {
proxy = createStreamProxy(proxyHost, proxyPort, fileSystemOptions, builder);
}
...
As you can you see, there's no "if" statement to instantiate ProxySOCKS4!
I have duplicated the SftpClientFactory class, set my version to load before the original class on the classpath and changed the code as follow:
public static Session createConnection(
...
final SftpFileSystemConfigBuilder.ProxyType proxyType = builder.getProxyType(fileSystemOptions);
...
final String proxyUser = builder.getProxyUser(fileSystemOptions);
final String proxyPassword = builder.getProxyPassword(fileSystemOptions);
Proxy proxy = null;
if (SftpFileSystemConfigBuilder.PROXY_HTTP.equals(proxyType)) {
proxy = createProxyHTTP(proxyHost, proxyPort);
((ProxyHTTP)proxy).setUserPasswd(proxyUser, proxyPassword);
/// change start (I also created the PROXY_SOCKS4 constant)
} else if (SftpFileSystemConfigBuilder.PROXY_SOCKS4.equals(proxyType)) {
proxy = createProxySOCKS4(proxyHost, proxyPort);
((ProxySOCKS4)proxy).setUserPasswd(proxyUser, proxyPassword);
/// change end
} else if (SftpFileSystemConfigBuilder.PROXY_SOCKS5.equals(proxyType)) {
proxy = createProxySOCKS5(proxyHost, proxyPort);
((ProxySOCKS5)proxy).setUserPasswd(proxyUser, proxyPassword);
} else if (SftpFileSystemConfigBuilder.PROXY_STREAM.equals(proxyType)) {
proxy = createStreamProxy(proxyHost, proxyPort, fileSystemOptions, builder);
}
...
.. and guess what, when I set my application to use a Socks 4 Proxy it works alright with the change above. It is important to say that setting the application to work with Socks 5 does not work if the proxy server is a Socks 4 type, and that's true not only for my application with VFS, but also any other client I tested, like Fillezila or WinSCP.
So, the main question is:
Why does VFS predicts the usage of ProxyHTTP, ProxySOCKS5 but completely ignores the JSch ProxySOCKS4 class? Am I missing some SFTP or Proxy concept here or should I consider VFS bugged? That's the first time I work with VFS.
Please consider the question in bold as the main question not to make it too broad.
I wasn't able to get or find a better answer in time, so what I did to solve my problem was exactly what I described in the question.
I duplicated the classes SftpClientFactory e SftpFileSystemConfigBuilder, made the necessary adjustments and used them instead of the original classes, it's ugly and now I am stuck with a specific VFS version, I know, but the problem was solved.
Lesson for next time: use Jsch instead of VFS.
I'll leave the question open though, in case someone else have a proper solution or answer.
I'm trying to test my SSL implementation in unit tests and have one scenario I can't quite understand.
When I connect to a host once and fail, every following connection will fail also, even if it has the correct certificates. I'm assuming somewhere along the way I'd have to flush a cache.
Here is my code, server and client are running locally. I use one jks-File for both trustStore and keyStore. The error occurrs no matter what the initial error was, I'll always get the first error the next time.
If I don't perform the first request the second one works.
If you're wondering what the use case is here, we have some local servers that use https certificates from an internal PKI, when someone misconfigures a server or a certificate, we'd like to be able to change them obviously, without shutting down the whole VM.
//attempt a connection without certificates, will fail
try (final InputStream stream = new URL("https://localhost:" + port).openStream()){
System.out.println(IOUtils.toString(stream, Charset.defaultCharset()));
} catch (IOException e){
System.out.println("Failed to load: " + StackTraceUtil.getStackTrace(e));
}
//copies the jks file to a temporary location
final File jksFile = copyJKSFile();
//ignore host names, running locally, won't use this in production
HttpsURLConnection.setDefaultHostnameVerifier((hostname, sslSession) -> hostname.equalsIgnoreCase("localhost"));
//set the system properties
System.setProperty("javax.net.ssl.keyStore", jksFile.getAbsolutePath());
System.setProperty("javax.net.ssl.keyStorePassword", password);
System.setProperty("javax.net.ssl.trustStore", jksFile.getAbsolutePath());
System.setProperty("javax.net.ssl.trustStorePassword", password);
//this should work now
try (final InputStream stream = new URL("https://localhost:" + port).openStream()){
System.out.println(IOUtils.toString(stream, Charset.defaultCharset()));
} catch (IOException e){
System.out.println("Failed to load: " + StackTraceUtil.getStackTrace(e));
}
Thanks for any help!
So I found a solution and I thought I'd share it in case someone else would have this problem at some point.
The class javax.net.ssl.HttpsURLConnection uses a javax.net.ssl.SSLSocketFactory to load the Key- and TrustStore, which uses a javax.net.ssl.SSLContext internally. When you don't overwrite anything, it uses the default implementation, which loads the files and can't be reset once loaded.
So what I did was not to use the default implementation, but to set my own SSLContext, when I knew the files would change.
final SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
sslContext.init(null, null, null);
HttpsURLConnection.setDefaultSSLSocketFactory(sslContext.getSocketFactory());
If you want to use older versions of TLS, the full list should be here
We are using a JAX-WS client over HTTPS to send messages (backed by CXF, which I think uses SSLSocket).
We wish to log the remote certificate details, together with the message details, if the remote certificate is not trusted/invalid.
Initially I hoped we would get a usefull exception, but the interesting exceptions in the stack trace are internal (like sun.security.validator.ValidatorException and sun.security.provider.certpath.SunCertPathBuilderException), so shouldn't really be used, and regardless don't seem to hold the remote certificate.
So my question is, what would be the most tidy way to get the certificate, at the level where I also have the message details (outside the JAX-WS call)?
So far my best guess is to add my own javax.net.ssl.X509TrustManager, which wraps the currently used one, and puts the Certificate on a ThreadLocal, where the caller can lately pick it up. It doesn't seem very tidy, but it's the best that seems possible so far :)
Many thanks for any suggestions!
The main point is that JSSE is doing and hiding all of the things you are looking for, in your question. But luckily it seems that CXF allows some customization.
The idea is to customize the SSLSocketFactory ( http://cxf.apache.org/docs/tls-configuration.html#TLSConfiguration-ClientTLSParameters ) with your own implementation, and this one must create Sockets that come with your own HandshakeCompletedListener. This is this last object which will dump the information that you are looking for, I give you an implementation example :
class CustomHandshakeCompletedListener implements HandshakeCompletedListener {
private HandshakeCompletedEvent hce;
private String cipher;
private Certificate[] peerCertificates = null;
private Principal peerPrincipal = null;
public void handshakeCompleted(HandshakeCompletedEvent hce) {
this.hce = hce;
// only cipersuites different from DH_anon* will return a server certificate
if(!cipher.toLowerCase().contains("dh_anon")) {
try {
cipher = hce.getCipherSuite();
peerCertificates = hce.getPeerCertificates();
peerPrincipal = hce.getPeerPrincipal();
// do anything you want with these certificates and ciphersuite
}
catch(SSLPeerUnverifiedException spue) {
System.err.println("unexpected exception :");
spue.printStackTrace();
}
}
}
There is still some work to achieve your goal, let us know if it's working well this clue.