Develop Reference

Apache HttpClient for Java Specific GET instead of CONNECT

Is there a way to specify the connection method when using a proxy?
I'm using the below code which sends an HTTP CONNECT. That is not supported by my load balancer. A GET request would terminate the TLS connection between the proxy and the website. The CONNECT method creates a TLS connection end to end between the end user and website. Essentially I need to inspect the traffic at the proxy.
HttpHost proxy = new HttpHost(proxyHost, proxyPort);
HttpClientBuilder clientBuilder = HttpClientBuilder.create();
clientBuilder
.setConnectionManager(connectionManager)
.setProxy(proxy)
.setDefaultRequestConfig(config);
Below is what the connection looks like:
Hypertext Transfer Protocol
CONNECT xyz.com:443 HTTP/1.1\r\n
Host: xyz.com\r\n
User-Agent: Apache-HttpClient/4.5.13 (Java/19.0.1)\r\n
\r\n

For HTTPS communications through a proxy, the client will use a CONNECT request to create a connection through the proxy to the remote trusted site.
With HTTPS, the client will only communicate with a server that it trusts. The trust usually requires that the server presents an X.509 certificate that is signed by a trusted authority (not self-signed), not expired, and has a subject that matches the site the client is trying to reach.
When communicating with a proxy, the proxy is typically unable to match that criteria. However, there are proxies that can match the above criteria by generating a new cert on the fly that has a matching subject to that expected by the client.
In that case, once the client has established a TLS connection with the proxy, the proxy can then decrypt the payload of the HTTPS traffic.
For the proxy to create a certificate trusted by the client, it must be signed by a CA (certificate authority) that is trusted by the client. Now, by default, the client will not trust the cert since it will not recognize the CA. However, you may modify the client by placing a cert for the CA on the client. Once the client trusts the CA, it will trust all of the certs generated by the proxy.
That being said, here's configuration for an HTTPS proxy (written by Cisco) that provides instructions for just that:
https://www.cisco.com/c/en/us/support/docs/unified-communications/jabber/212044-Configure-Charles-Proxy-to-Capture-HTTPS.html
The configuration action you'll need for Apache (if Apache supports this) will be similar - you'll need to get a cert from the Apache proxy to install on your client.
I'm currently searching the web for similar proxies. I'm essentially looking for a key word that describes this technique of enabling your HTTPS proxy to do man-in-the-middle attacks with a dynamically generated cert matching the request. If I find the terms, I'll update the post.
EDIT: Update for squid
I found an article about configuring squid for inspecting HTTPS traffic: https://scubarda.com/2020/03/23/configure-squid-proxy-for-ssl-tls-inspection-https-interception/
In the article, the author indicates that you would have to install the X.509 Cyber Saiyan certificate in your browser (client's) certificate/authorities section (but this is just for this specific configuration).
Some good keywords to search for now are HTTPS inspection...

If this is a temporary debug solution, you can use one of the Man In The Middle (MITM) proxies that are designed just for this:
Burp
Charles
Zap
Once installed somewhere, you'll just need to import the proxy CA cert into the server making the connections (each of the above proxies has a how-to guide for this), and set the proxy address to the MITM.

ActiveMQ TLS host hostname verifications

In ActiveMQ 5.15.6 they enabled the TLS host name verification, and I see my application failing after upgrade.
In their documentation they mentioned they have fixed the server side TLS validation by making default to False. Is their any way I can make TLS hostname verification FALSE at client side as well?

Yes, you can disable TLS hostname verification at the client side as well. Use something like this:
ssl://hostname:61616?socket.verifyHostName=false
or
ssl://hostname:61616?verifyHostName=false

Behavior of HttpsURLConnectioin with default implementation of HostnameVerifier and SSLSocketFactory

I am trying to understand a piece of code that does not override HostnameVerifier and SSLSocketFactory for HttpsURLConnection. The current code is able to make the SSL request fine, which is a little confusing to me.
How does the default implementation of HostnameVerifier and SSLSocketFactory classes work? Does it verify the certificate CA and hostname or it bypasses all these checks all together?
Also, is there any special behavior if the request has localhost in it?
The URL looks like this
https://localhost:/
Thanks for your help.

It will check the CA certificate against the trusted ca-certificates in the trusted store of the JDK jre/lib/security/cacerts or whatever you have configured.
It will also check, that all certificates in the chain are trusted, not only the immediate signer. It will, by default, not check online for revoked certificates, but you can enable that in the java.security policy file.
It will also check the hostname against the certificate.
It supports additional CNs and wildcard CNs.
Extended question regarding localhost:
localhost is just a hostname like any other. It must be listed in the certificate. But you will not find an official CA to give you that certificate, I guess :-)
If you want to play with your real cert on localhost, you address it with a different hostname. If you want to run through a cert for foo.bar.com, add to your hosts file (Win: %WINDOWS%\system32\drivers\etc\hosts, Unix /etc/hosts) the line:
127.0.0.1 foo.bar.com
Now fire up your browser and connect to https://foo.bar.com. It will connect to your local server and post a SSL encrypted HTTP request. That request contains the hostname from the URL, not the real hostname.
I was oversimplyfying: The browser receives the SSL certificate from the server and compares the CN against the hostname it just called. If this matches, it will check if the certificate can be trusted (don't nail me down on the order, maybe it will check the trustlevel first).
Here is where you fail: The certificate provided by the server does not contain the CN localhost, and thus, the client will abort the connection. In your case, the client is the HttpSSLrequest.
To make the client believe all is well, you can do two things:
- fake the hostname, and add the signing CA in the cacerts truststore.
- implement a hostname and certificate validator, that lets you through.
Regarding the tutorial, a self-signed certificate is used, you will have to convince the browser to accept it by overriding the warning on first access.

The SSL Cert is bound to the actual host name. If you have an SSL cert for "qa.example.com", it won't work on your machine named "dev.example.com"

I want to understand the below line
"The SSL Cert is bound to the actual host name. If you have an SSL cert for "qa.example.com", it won't work on your machine named "dev.example.com".
Questions:
How is possible to create a cert bound to a hostname?
I have a client and server model. The server is not validating the client during communication. So far I had ONE set of key and certs which is installed on multiple servers and the communication works fine.
Now we have new set of certs which is bound to a hostname. Each server has a set of certs.
When swapped the communication does not work. "Handshake failure" obtained.
What's happening here? Who validates the hostname?

The client validates the certificate returned by the server. Among the various verifications performed the client checks that the hostname embedded in the certificate matches the hostname of the server. If the hostname doesn't match the handshake fails.
This means that if your certificate was issued for qa.example.com, it won't work on dev.example.com. For the same certificate to work for both hostnames you'll need a wildcard certificate (issued for *.example.com).

The hostname is in the certificates, you need to generate new certificates with different hostnames. The certificate names is validates by the client when it attempts to establish a secure connection.

Question on ssl handshake and behavior in java

I am using https to connect to an https server.
Specifically I am using apache httpclient and I configure the ssl context to use my keystore and truststore.
The https server I am using is IIS7 and is configured to require client authentication.
I think I have set it up properly.
Anyway, if I configure the httpClent's ssl context with a keystore (i.e. with client certificates) valid for IIS then there is no problem connecting.
Now my problem is the following:
If I do not configure the ssl context with any client certificate to send to IIS, there is no connection with the server. What makes me think though, is the fact that I was expecting to see some java exception in the code as a result of a hanshake failure alert.
Monitoring what is happening with wireshark, I could not see a certificate request from IIS to my application, but I noticed that after ServerHelloDone everything was encrypted.
I did not expect that. I think the handshake is usually in cleartext.
I used private key to decrypt traces and i saw a certificate request from IIS but after many starting and opening of new connections.
My app send back as a response a certificate of length 0 and IIS replies with a TLSv1 Finished.
After that the packets stop (i.e. seems that the communication ends).
I was expecting a handshake alert.
My question is, is this how it is supposed to work or at least how IIS works?
Or if I do not see the alert something is wrong with my use case?
Thanks

It sounds like IIS is only requiring client certificates for certain URLs (ie, for example.com/foo, but not example.com/bar).
In the initial handshake, it does not know which url you are requesting, so it does not require a certificate. When it sees that you are requesting a restricted resource (/foo), it then rehandshakes, requiring a certificate.
However, I would still expect a handshake_failure to occur.

As I was saying in an answer to this question, as far as I remember, IIS uses re-negotiation to get the client certificate. You should be able to change this behaviour using netsh and clientcertnegotiate=enable (depending on the version of IIS you're using).
You might also be interest in this similar question.

Failing to supply a certificate in response to a CertificateRequest isn't an SSL protocol error, so there is no handshake_error. 'Requiring' instead of just 'needing' client certificates is added-in by SSL libraries, and all they can do if you don't send one is just close the connection.