Develop Reference

Apache HttpClient for Java Specific GET instead of CONNECT

Is there a way to specify the connection method when using a proxy?
I'm using the below code which sends an HTTP CONNECT. That is not supported by my load balancer. A GET request would terminate the TLS connection between the proxy and the website. The CONNECT method creates a TLS connection end to end between the end user and website. Essentially I need to inspect the traffic at the proxy.
HttpHost proxy = new HttpHost(proxyHost, proxyPort);
HttpClientBuilder clientBuilder = HttpClientBuilder.create();
clientBuilder
.setConnectionManager(connectionManager)
.setProxy(proxy)
.setDefaultRequestConfig(config);
Below is what the connection looks like:
Hypertext Transfer Protocol
CONNECT xyz.com:443 HTTP/1.1\r\n
Host: xyz.com\r\n
User-Agent: Apache-HttpClient/4.5.13 (Java/19.0.1)\r\n
\r\n

For HTTPS communications through a proxy, the client will use a CONNECT request to create a connection through the proxy to the remote trusted site.
With HTTPS, the client will only communicate with a server that it trusts. The trust usually requires that the server presents an X.509 certificate that is signed by a trusted authority (not self-signed), not expired, and has a subject that matches the site the client is trying to reach.
When communicating with a proxy, the proxy is typically unable to match that criteria. However, there are proxies that can match the above criteria by generating a new cert on the fly that has a matching subject to that expected by the client.
In that case, once the client has established a TLS connection with the proxy, the proxy can then decrypt the payload of the HTTPS traffic.
For the proxy to create a certificate trusted by the client, it must be signed by a CA (certificate authority) that is trusted by the client. Now, by default, the client will not trust the cert since it will not recognize the CA. However, you may modify the client by placing a cert for the CA on the client. Once the client trusts the CA, it will trust all of the certs generated by the proxy.
That being said, here's configuration for an HTTPS proxy (written by Cisco) that provides instructions for just that:
https://www.cisco.com/c/en/us/support/docs/unified-communications/jabber/212044-Configure-Charles-Proxy-to-Capture-HTTPS.html
The configuration action you'll need for Apache (if Apache supports this) will be similar - you'll need to get a cert from the Apache proxy to install on your client.
I'm currently searching the web for similar proxies. I'm essentially looking for a key word that describes this technique of enabling your HTTPS proxy to do man-in-the-middle attacks with a dynamically generated cert matching the request. If I find the terms, I'll update the post.
EDIT: Update for squid
I found an article about configuring squid for inspecting HTTPS traffic: https://scubarda.com/2020/03/23/configure-squid-proxy-for-ssl-tls-inspection-https-interception/
In the article, the author indicates that you would have to install the X.509 Cyber Saiyan certificate in your browser (client's) certificate/authorities section (but this is just for this specific configuration).
Some good keywords to search for now are HTTPS inspection...

If this is a temporary debug solution, you can use one of the Man In The Middle (MITM) proxies that are designed just for this:
Burp
Charles
Zap
Once installed somewhere, you'll just need to import the proxy CA cert into the server making the connections (each of the above proxies has a how-to guide for this), and set the proxy address to the MITM.

how to add 'host_name' extension for SNI in java (using jetty)

i am trying in every way to add the SNI to my client (jetty 11.0.9). The server requires me to send host_name, but there is no way with jetty. can anyone do it?

Jetty's HttpClient uses the Java SSLEngine to initiate TLS connections and will use SNI by default.
Just use one of the HttpClient.newRequest() methods to create a Request with either an absolute URI (that includes the authority, hostname, port) or the newRequest() methods that take a hostname / port pair.
Note that the Java SSLEngine will follow the TLS/SNI spec and will not send SNI if the destination host is localhost or an IP Address (you must use a hostname, always, with SNI)

Using Proxy server for Java application

So i have a java app that uses Google Analytics API to gather some info. I am putting this application to run in my oracle cloud managed server which has a firewall and blocks any web calls to work. So, they setup a proxy for me to use....I've never set up a proxy to work with a java application before, I've been reading at tutorials like this: http://docs.oracle.com/javase/1.5.0/docs/guide/net/proxies.html
And i have no idea how to set this up...anyone want to point me in the right direction?

You must tell your application that there's a proxy somewhere.
As the documentation says, you must set some properties in your virtual machine. You can do it programatically:
//Set the http proxy to webcache.mydomain.com:8080
System.setProperty("http.proxyHost", "webcache.mydomain.com");
System.setPropery("http.proxyPort", "8080");
// Next connection will be through proxy.
URL url = new URL("http://java.sun.com/");
InputStream in = url.openStream();
// Now, let's 'unset' the proxy.
System.clearProperty("http.proxyHost");
// From now on http connections will be done directly.
Or use https.proxy... if the connection is HTTPS.
Besides, if you have access to the application server start script, you could add those properties as VM properties with -Dhttp.proxyHost....

The solution in my case was to configure the JVM with a HTTPS proxy:
System.setProperty("https.proxyHost", "proxy");
System.setProperty("https.proxyPort", "3128");

How can I configure HTTPClient to authenticate against a SOCKS proxy?

I need to set up proxy authentication against a SOCKS proxy.
I found out this post giving instructions that appear to work with common HTTP proxies.
httpclient.getHostConfiguration().setProxy("proxyserver.example.com", 8080);
HttpState state = new HttpState();
state.setProxyCredentials(new AuthScope("proxyserver.example.com", 8080),
new UsernamePasswordCredentials("username", "password"));
httpclient.setState(state);
Would that work with SOCKS proxies as well or do I have to do something different?

Java supports Socks proxy configuration via preferences:
socksProxyHost for the host name of the SOCKS proxy server
socksProxyPort for the port number, the default value being 1080
e.g.
java -DsocksProxyHost=socks.mydomain.com
(edit) For your example, if the socks proxy was configured in the way outlined before:
httpclient.getHostConfiguration().setProxy("proxyserver.example.com", 8080);
Credentials cred = new UsernamePasswordCredentials("username","password");
httpclient.getState().setProxyCredentials(AuthScope.ANY, cred);
You can also use this variant (without httpclient):
SocketAddress addr = new
InetSocketAddress("webcache.mydomain.com", 8080);
Proxy proxy = new Proxy(Proxy.Type.SOCKS, addr); // Type.HTTP for HTTP
So completing the previous example, we can now add:
URL url = new URL("http://java.sun.com/");
URConnection conn = url.openConnection(proxy);
HTH

The Features page of Apache HTTPClient says:
Transparent connections through SOCKS proxies (version 4 & 5) using native Java socket support.
With "transparent", I guess they mean that it works without you needing to do anything special. Do you have a SOCKS proxy available somewhere? Can't you just try it out to see if it works?

SOCKS is not supported by HttpClient 3 natively. You can try the SOCKS support in JDK as suggested by others. The side effect is that your whole JVM will go through the same SOCKS proxy.
Java 5 supports Username/Password authentication in SOCKS (type 2). All you have to do is to setup the authenticator like this,
Authenticator.setDefault(new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(username, password.toCharArray());
}
});
Again, this may not work for you because it affects all authentication in your JVM (HTTP auth, Proxy Auth).

You can provide a custom socket factory which implements the SOCKS protocol, and register it as your default HTTP protocol handler. This solution has a limitation similar to tuergeist's answer above has - it applies globally, to any HTTP connection you'll establish through HttpClient.
If you find this a problem, take a look at this correspondence, where Oleg suggests using HttpClient 4.0, but also refers to a possible patch in HostConfiguration class for HttpClient 3.x.
Another possible solution, which is my personal favorite, is to write a wrapper HTTP proxy to the socks proxy.

I tried
System.setProperty("socksProxyHost", "socks.xyz.com");
System.setProperty("socksProxyPort", "1000");
and it's working fine.

Launch options to force Java socket connections to localhost?

I'm trying to find a way to force any connection attempts a Jar makes to an external IP through my proxy server, which is running on localhost(Also a Java application).
Once the proxy server receives the connection it will open a connection with the external IP and begin routing the IO to and from the client/server.
I've been Googling this for 2 days, and I haven't had any luck, I believe I'm using the wrong terms in my search attempts.
If you've got any ideas, please let me know, I'll try anything.
Thanks in advance. - Sean.

If is that a "real" Proxy the you could specify the proxy to use using java system properties.
You have two alternatives:
Specify the proxy in the command line
Hardcode it into your app
Well you actually have three
Specify a .properties file, and read from there, and set it as System property ( which is pretty much option 2 but more dynamic )
From command line you'll use:
java -Dhttp.proxyHost=localhost -Dhttp.proxyPort=8080 -jar YourJar.jar
With that all the http connections you perform will go through localhost at port 8080
The second is add this at the main method of your program:
public static void main( String [] args ) {
System.setProperty("http.proxyHost", "localhost");
System.setProperty("http.proxyPort", "8080");
.....
}
Which does the same.
Finally loading from myapp.properties
public static void main( String [] args ) {
try { // there are cleaner ways of course
ResorceBundle bundle = ResourceBundle.getBundle("myapp");
System.setProperty("http.proxyHost", bundle.getString("proxy.server"));
System.setProperty("http.proxyPort", bundle.getString("proxy.port"));
} catch( MissingResourceException missingResourceException ){}
....
}
You just have to make sure myapp.properties is available from the classpath
More information about this functionality here

If you are asking about general (NOT HTTP / FTP specific!) proxying of Socket connections, then the simple answer is that it is not supported by Java.
When you configure a proxy for HTTP and FTP traffic, the proxying happens at the application protocol level. The Java-side proxy properties tell the URLConnection layer to connect to your designated proxy rather than the IP address from the URL your application is trying to connect to. The Java Socket level is unaware that this is happening. It just sees a requests to connect to the proxy.
This work because the HTTP and FTP protocols specifically support proxying. For instance, the first 'line' of an HTTP GET request message gives the full URL of the page that the client is requesting. If the GET request goes to a proxy, the proxy can figure out where is has to send it.
Looking at the problem of proxying at the Socket level, the first observation is that the standard Java class libraries don't support this. The second observation is that it is actually unimplementable ... unless you implement this as an alternative transport layer. The reason is that IP and TCP/IP simply do not support the notion of explicitly proxying or relaying messages / streams. And even if you did implement such a transport, it doesn't fit into the standard Socket model.
So, if you are really asking about proxying all of the network traffic for a Java application, this can only be implemented outside of the JVM; i.e. at the network transport level of the JVM's (physical or virtual) host operating system.

If it's HTTP traffic or FTP traffic, you could try the following system properties:
http.proxyHost (default: )
http.proxyPort (default: 80 if http.proxyHost specified)
http.nonProxyHosts (default:
See this link for details:
http://java.sun.com/docs/books/tutorial/networking/urls/_setProxy.html