I use Apache Commons HttpClient 3.1 as a kind of reverse-proxy. This proxy server runs in a servlet container in port 8081, and proxies some of the requests to port 8080 on the same server. As the legacy server on port 8080 builds some absolute urls using the HTTP Host header, I want to explicitly set that header.
It is not possible to set the Host-header as you set other headers, as HttpClient automatically overrides the value you set. The only way I've found to change the Host-header is to set the virtual host:
HttpClient = ...
HttpMethod = ...
HostParams hostParams = new HostParams();
hostParams.setVirtualHost("localhost:8081");
hostConfiguration.setParams(hostParams);
hostConfiguration.setHost("localhost", 8080);
client.executeMethod(hostConfiguration, method);
But this doesn't work as it should because HttpClient seems to add the port it connects to, to the Host:
11:07:05.011 [qtp1813719644-21] DEBUG httpclient.wire.header - >> "Host: localhost:8081:8080[\r][\n]"
Is it any way I can fix this behaviour? If not, does Apache Httpclient 4.x behave differently?
As your problem is traversing a proxy (which in your case is a Servlet+HTTPClient), configure your client to use localhost:8080 as proxy and url as normal: http://localhost:8081/... :
hostConfiguration.setProxy("localhost", 8080);
hostConfiguration.setHost("localhost", 8081);
Related
I configured java.net.http.HttpClient as shown below:
HttpClient client = HttpClient.newBuilder().version(HttpClient.Version.HTTP_1_1).build();
Also, I have a simple Spring Boot (Tomcat) HTTP server, which is running on the 8080 port. For each request, I check incoming headers in a controller and the number of TCP connections using the next command: lsof -i -P | grep "TCP" | grep "8080".
When I make a GET request from client then exactly one TCP connection is created for each request. Incoming headers don't have any information about keep-alive
When I try to set keep-alive header directly I got the exception.
HttpRequest req = HttpRequest.newBuilder()
.setHeader("Connection", "Keep-Alive")
.uri(uri)
.build();
When I make a GET request from a browser (safari) then the browser adds keep-alive headers to each request and only one TCP connection is created for multiply requests (as expected).
When I set version HTTP/2 and make the request from the client then only one TCP connection creates for all requests (as expected):
HttpClient client = HttpClient.newBuilder().version(HttpClient.Version.HTTP_2).build();
As described here - both HTTP/1.1 and HTTP/2 have keep-alive property which is enabled by default, but as you can see from the examples above it doesn't work for HTTP/1.1 in my case.
Does anyone know how to configure HttpClient properly? Or maybe, I'm doing something wrong?
I am using OkHttp3 in my Android app to make HTTP/1.x requests to my backend servers via a forward proxy, like so:
List<Protocol> protos = new ArrayList<>();
protos.add(Protocol.HTTP_2);
protos.add(Protocol.HTTP_1_1);
InetSocketAddress proxyAddr = new InetSocketAddress("proxy.example.com", 80);
Proxy proxy = new Proxy(Proxy.Type.HTTP, proxyAddr);
OkHttpClient cli = new OkHttpClient.Builder()
.proxy(proxy)
.protocols(protos)
.build();
String url = "http://www.example.com/";
Request req = new Request.Builder().url(url).build();
Response res = cli.newCall(req).execute();
I would like to upgrade to HTTP2. However, it seems to me that OkHttp3 can make HTTP2 requests only if we are not going via a HTTP proxy. So, the above code wouldn't work.
In other words, OkHttp3 supports the first 3 cases below but not the fourth. HTTP/2 below means h2 (HTTP/2 over TLS) not h2c (HTTP/2 over clear text).
a) client <-- HTTP/1.x --> upstream server
b) client <-- HTTP/1.x --> forward proxy <-- HTTP/x --> upstream server
c) client <-- HTTP/2 --> upstream server
d) client <-- HTTP/2 --> forward proxy <-- HTTP/x --> upstream server
Does anyone confirm or deny my understanding? Thanks.
OkHttp will do HTTP/2 over an HTTP proxy. You’ll need HTTPS on the server since OkHttp doesn’t implement plaintext HTTP/2.
Jesse, I tried retrieving https://www.google.com/ with Proxy.Type.HTTP via nghttp2's forward proxy nghttpx, which supports HTTP2 over TLS. Unfortunately, TLS handshaking did not happen and the forward proxy reported the following error.
... tls: handshake libssl error: error:1407609B:SSL routines:SSL23_GET_CLIENT_HELLO:https proxy request
From what I gather, this error means that okhttp3's proxy code is not doing TLS handshaking with the forward proxy.
This makes me think that HTTP2 over TLS via a forward proxy is kind of pointless, because the forward proxy won't be able to add any value to encrypted requests - the forward proxy is just a pass-through pipe. In fact, I think TLS via any forward proxy is pointless. End-to-end HTTP2 over TLS does make sense, but via a forward proxy doesn't.
I'm experimenting with HTTP/2. I found some examples in this repo. Here's the constructor (boilerplate code skipped):
public WebServer(String path, HttpServlet servlet, int port) {
Server server = new Server(new QueuedThreadPool());
ServletContextHandler context = new ServletContextHandler(server, "/", ServletContextHandler.SESSIONS);
context.addServlet(new ServletHolder(servlet), path);
ServerConnector connector = new ServerConnector(server, 1,1, new HTTP2ServerConnectionFactory(new HttpConfiguration()));
connector.setPort(port);
server.addConnector(connector);
}
Then I start() the server, run java http2 client from the mentioned above repo and it connects with no error. But then I try to do the same with curl:
# curl --http2 http://localhost:8080
invalid_preface
And the browser (Chrome) says:
GET http://localhost:8080/ net::ERR_INVALID_HTTP_RESPONSE
What's whong with the server?
There is nothing wrong with the server.
You have configured Jetty to listen to HTTP/2 only (no HTTP/1.1), and then you have asked curl and Chrome to make a HTTP/1.1 request, which Jetty replied with invalid_preface because it cannot understand it.
Bear in mind that browsers will only speak HTTP/2 over TLS (i.e. SSL), while curl is able to also speak clear-text HTTP/2.
This example shows you how to setup a server that supports both clear-text HTTP/1.1 and HTTP/2.
This example shows you how to setup a server that supports both clear-text and TLS HTTP/1.1 and HTTP/2.
See here for how to use curl with HTTP/2.
I have an application deployed on tomcat in linux rhel 5 , now this application makes an external call to internet and my server is behind the proxy server , now how do I configure the tomcat server for it to understand the proxy.
Is there a configuration I can do to redirect all requests send by tomcat to external servers
Also to mention that I did make the entries into catalina.properties
http.proxyHost=
http.proxyPort=8080
Alternatively, configure them as VM parameters in catalina.bat.
-Dhttp.proxyHost=<> -DproxyPort=8080
Another approach would be to configure them at the application level. If you are using java's own api to invoke external url, the proxy could be set as follows. In case you are using another library like apache httpclient, it provides methods to configure the proxy.
SocketAddress proxyAddress = new InetSocketAddress(proxyHost, proxyPort);
Proxy proxy = new Proxy(Type.HTTP, proxyAddress);
URL url = new URL(externalURL);
HttpURLConnection conn = (HttpURLConnection) url.openConnection(proxy);
I need to set up proxy authentication against a SOCKS proxy.
I found out this post giving instructions that appear to work with common HTTP proxies.
httpclient.getHostConfiguration().setProxy("proxyserver.example.com", 8080);
HttpState state = new HttpState();
state.setProxyCredentials(new AuthScope("proxyserver.example.com", 8080),
new UsernamePasswordCredentials("username", "password"));
httpclient.setState(state);
Would that work with SOCKS proxies as well or do I have to do something different?
Java supports Socks proxy configuration via preferences:
socksProxyHost for the host name of the SOCKS proxy server
socksProxyPort for the port number, the default value being 1080
e.g.
java -DsocksProxyHost=socks.mydomain.com
(edit) For your example, if the socks proxy was configured in the way outlined before:
httpclient.getHostConfiguration().setProxy("proxyserver.example.com", 8080);
Credentials cred = new UsernamePasswordCredentials("username","password");
httpclient.getState().setProxyCredentials(AuthScope.ANY, cred);
You can also use this variant (without httpclient):
SocketAddress addr = new
InetSocketAddress("webcache.mydomain.com", 8080);
Proxy proxy = new Proxy(Proxy.Type.SOCKS, addr); // Type.HTTP for HTTP
So completing the previous example, we can now add:
URL url = new URL("http://java.sun.com/");
URConnection conn = url.openConnection(proxy);
HTH
The Features page of Apache HTTPClient says:
Transparent connections through SOCKS proxies (version 4 & 5) using native Java socket support.
With "transparent", I guess they mean that it works without you needing to do anything special. Do you have a SOCKS proxy available somewhere? Can't you just try it out to see if it works?
SOCKS is not supported by HttpClient 3 natively. You can try the SOCKS support in JDK as suggested by others. The side effect is that your whole JVM will go through the same SOCKS proxy.
Java 5 supports Username/Password authentication in SOCKS (type 2). All you have to do is to setup the authenticator like this,
Authenticator.setDefault(new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(username, password.toCharArray());
}
});
Again, this may not work for you because it affects all authentication in your JVM (HTTP auth, Proxy Auth).
You can provide a custom socket factory which implements the SOCKS protocol, and register it as your default HTTP protocol handler. This solution has a limitation similar to tuergeist's answer above has - it applies globally, to any HTTP connection you'll establish through HttpClient.
If you find this a problem, take a look at this correspondence, where Oleg suggests using HttpClient 4.0, but also refers to a possible patch in HostConfiguration class for HttpClient 3.x.
Another possible solution, which is my personal favorite, is to write a wrapper HTTP proxy to the socks proxy.
I tried
System.setProperty("socksProxyHost", "socks.xyz.com");
System.setProperty("socksProxyPort", "1000");
and it's working fine.