Develop Reference

How to check number of http connections used at a specific point in time?

I have project with vertx where I am using a HtttpClient where I configurure a HttpConfigOption object in order to specify host, maxPoolSize, keepAlive, etc.
HttpClientOptions httpClientOptions = new HttpClientOptions()
.setKeepAlive(KeepAlive)
.setDefaultHost(baseUrl)
.setTcpNoDelay(true)
.setVerifyHost(verifyHost)
.setTrustAll(trustAll)
.setMaxPoolSize(50)
.setPoolCleanerPeriod(POOL_CLEANER_PERIOD)
.setDefaultPort(port).setSsl(isSSl);
I start having some timeouts and I want to know if the problem is from my connection pool or the timeouts are coming from the services that I consume.
Is there a way to print in vertx the number of connections that are at the moment in use? So for example let's say that I have a poll with max 50 connections and on timeout it will be nice if I can print the number of connections that are active, in use.
Thanks

You can use the vertx micrometer module to grab vertx_http_client_active_connections
ref: https://vertx.io/docs/vertx-micrometer-metrics/java/

Apache HttpClient Keep-Alive Strategy for active connections

In an Apache HttpClient with a PoolingHttpClientConnectionManager, does the Keep-Alive strategy change the amount of time that an active connection will stay alive until it will be removed from the connection pool? Or will it only close out idle connections?
For example, if I set my Keep-Alive strategy to return 5 seconds for every request, and I use the same connection to hit a single URL/route once every 2 seconds, will my keep-alive strategy cause this connection to leave the pool? Or will it stay in the pool, because the connection is not idle?

I just tested this and confirmed that the Keep-Alive strategy will only idle connections from the HttpClient's connection pool after the Keep-Alive duration has passed. The Keep-Alive duration determines whether or not the connection is idle, in fact - if the Keep-alive strategy says to keep connections alive for 10 seconds, and we receive responses from the server every 2 seconds, the connection will be kept alive for 10 seconds after the last successful response.
The test that I ran was as follows:
I set up an Apache HttpClient (using a PoolingHttpClientConnectionManager) with the following ConnectionKeepAliveStrategy:
return (httpResponse, httpContext) -> {
// Honor 'keep-alive' header
HeaderElementIterator it = new BasicHeaderElementIterator(
httpResponse.headerIterator(HTTP.CONN_KEEP_ALIVE));
while (it.hasNext()) {
HeaderElement he = it.nextElement();
String param = he.getName();
String value = he.getValue();
if (value != null && param.equalsIgnoreCase("timeout")) {
try {
return Long.parseLong(value) * 1000;
} catch(NumberFormatException ignore) {
}
}
}
if (keepAliveDuration <= 0) {
return -1; // the connection will stay alive indefinitely.
}
return keepAliveDuration * 1000;
};
}
I created an endpoint on my application which used the HttpClient to make a GET request to a URL behind a DNS.
I wrote a program to hit that endpoint every 1 second.
I changed my local DNS for the address that the HttpClient was sending GET requests to to point to a dummy URL that would not respond to requests. (This was done by changing my /etc/hosts file).
When I had set the keepAliveDuration to -1 seconds, even after changing the DNS to point to the dummy URL, the HttpClient would continuously send requests to the old IP address, despite the DNS change. I kept this test running for 1 hour and it continued to send requests to the old IP address associated with the stale DNS. This would happen indefinitely, as my ConnectionKeepAliveStrategy had been configured to keep the connection to the old URL alive indefinitely.
When I had set the keepAliveDuration to 10, after I had changed my DNS, I sent successful requests continuously, for about an hour. It wasn't until I turned off my load test and waited 10 seconds until we received a new connection. This means that the ConnectionKeepAliveStrategy removed the connection from the HttpClient's connection pool 10 seconds after the last successful response from the server.
Conclusion
By default, if an HttpClient does not receive a Keep-Alive header from a response it gets from a server, it assumes its connection to that server can be kept alive indefinitely, and will keep that connection in it's PoolingHttpClientConnectionManager indefinitely.
If you set a ConnectionKeepAliveStrategy like I did, then it will add a Keep-Alive header to the response from the server. Having a Keep-Alive header on the HttpClient response will cause the connection to leave the connection pool after the Keep-Alive duration has passed, after the last successful response from the server. This means that only idle connections are affected by the Keep-Alive duration, and "idle connections" are connections that haven't been used since the Keep-Alive duration has passed.

How to to limit connections for the same host (setMaxPerRoute) using PoolingHttpClientConnectionManager?

I use PoolingHttpClientConnectionManager for sending multiple GET/POST requests in parallel to following services:
(1) http://localhost:8080/submit
(2) http://localhost:8080/query
Both services are heavily used but the first service (1) has a higher priority.
I need to set setMaxPerRoute for service (1) so that it will consume 80% of available connections.
The rest 20% limit will be allocated for the rest requests with longer timeouts (including service (2)). Here is my code:
...
PoolingHttpClientConnectionManager httpClientManager =
new PoolingHttpClientConnectionManager();
httpClientManager.setMaxTotal(10);
httpClientManager.setDefaultMaxPerRoute(2);
HttpHost httpHost = new HttpHost("http://localhost/submit",8080);
HttpRoute submitRoute = new HttpRoute(httpHost);
httpClientManager.setMaxPerRoute(submitRoute, 8);
...
The problem is that HttpHost apparently cannot be the same to differentiate among routes. In fact, two URL-s have the same host (http://localhost:8080), but different request pages. In the result, both services are used the same resources.
Is there any way to implement such a limitation for the same host?
Thanks for help.

After suggestions from my co-workers I've found the solution.
We need to control the maximum number of connections so that when requesting URL (1) we have the pool with max 20 connections, whereas the rest type of requests including request (2) we have the pool with max 2 connections.
It can be solved by creating two different HttpClient objects each one has its own PoolingHttpClientConnectionManager. The first manager is set with setMaxTotal=20, while the second with setMaxTotal=2.
Now each pool is has different limitations for the same domain.

Apache HttpClient: setConnectTimeout() vs. setConnectionTimeToLive() vs. setSocketTimeout()

Can someone please explain what is the difference between these two:
client = HttpClientBuilder.create()
.setConnectionTimeToLive(1, TimeUnit.MINUTES)
.build();
and
RequestConfig requestConfig = RequestConfig.custom().setConnectTimeout(30 * 1000).build();
client = HttpClientBuilder
.create()
.setDefaultRequestConfig(requestConfig)
.build();
Is it better to use setSocketTimeout()?

A ConnectTimeout determines the maximum time to wait for the other side to answer "yes, I'm here, let's talk" when creating a new connection (ConnectTimeout eventually calls socket.connect(address, timeout). The wait-time is usually less than a second, unless the other side is really busy with just accepting new incoming connections or you have to go through the great firewall of China. In the latter cases it can be a minute (or more) before the new connection is created. If the connection is not established within the ConnectTimeout, you get an error (1).
setSocketTimeout eventually calls socket.setSoTimeout which is explained in this answer.
The ConnectionTimeToLive determines the maximum age of a connection (after which it will be closed), regardless of when the connection was last used. Normally, there is an "idle timeout" to cleanup connections, i.e. you or the other side will close a connection that is not used for a while. Typically, you will close an idle connection before the other side does to prevent errors. But there are two other cases I can think of where a maximum age for a connection is useful:
Bad network components: count yourself lucky if you have not met them. Some bad routers, firewalls, proxies, etc. will just drop (actively being used) connections after something like 30 minutes. Since you and the other side may not even be aware that a connection was dropped, you can get "connection reset" errors for no obvious reason at weird times.
Cached meta-data: most systems keep some meta-data about a connection in some sort of cache. Some systems manage this cache badly - cache size just grows with the age of the connection.
A note about the ConnectionTimeToLive implementation in Apache HttpClient 4.5.4: I think you must use the PoolingHttpClientConnectionManager for the option to work (it eventually all comes down to a call to this isExpired method). If you do not use this connection manager, test the option to make sure it really works.
(1) Interesting comment from EJP user207421 on this related answer

Connection Timeout:
It is the timeout until a connection with the server is established.
Socket Timeout:
this is the time of inactivity to wait for packets[data] to receive.
setConnectionRequestTimeout:
However it is specific for configuring the connection manager. It is the time to fetch a connection from the connection pool.
It returns the timeout in milliseconds used when requesting a connection from the connection manager. 0(zero) is used for an infinite timeout.
setConnectionTimeToLive
public final HttpClientBuilder setConnectionTimeToLive(long connTimeToLive, TimeUnit connTimeToLiveTimeUnit)
Sets maximum time to live for persistent connections
Please note this value can be overridden by the setConnectionManager(org.apache.http.conn.HttpClientConnectionManager) method.
Since:
4.4
Example: HttpClientStarter.java
#Override
public boolean start() {
RegistryBuilder<ConnectionSocketFactory> r = RegistryBuilder.<ConnectionSocketFactory> create();
// Register http and his plain socket factory
final SocketFactory ss = getLevel().find(SocketFactory.class);
ConnectionSocketFactory plainsf = new PlainConnectionSocketFactory() {
#Override
public Socket createSocket(HttpContext context) throws IOException {
return ss.createSocket();
}
};
r.register("http", plainsf);
// Register https
ConnectionSocketFactory sslfactory = getSSLSocketFactory();
if (sslfactory != null) {
r.register("https", getSSLSocketFactory());
} else {
log(Level.WARN, "ssl factory not found, won't manage https");
}
HttpClientBuilder builder = HttpClientBuilder.create();
builder.setUserAgent(USERAGENT);
builder.setConnectionTimeToLive(timeout, TimeUnit.SECONDS);
builder.evictIdleConnections((long) timeout, TimeUnit.SECONDS);
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(r.build());
cm.setMaxTotal(maxConnect * 2);
cm.setDefaultMaxPerRoute(2);
cm.setValidateAfterInactivity(timeout * 1000);
builder.setConnectionManager(cm);
RequestConfig rc = RequestConfig.custom()
.setConnectionRequestTimeout(timeout * 1000)
.setConnectTimeout(timeout * 1000)
.setSocketTimeout(timeout * 1000)
.build();
builder.setDefaultRequestConfig(rc);
client = builder.build();
return true;
}
Resource Link:
HttpClientStarter.java
HttpClient 4.x
Timeout
The HTTP specification does not determine how long a persistent connection may or should remain active. Some HTTP servers use a non-standard header, Keep-Alive, to tell clients the number of seconds they want to stay connected on the server side. HttClient will take advantage of this if this information is available. If the header information Keep-Alive does not exist in the response, HttpClient assumes the connection remains active indefinitely. However, many real-world HTTP servers are configured to discard persistent connections after certain periods of inactivity to conserve system resources, often without notification to the client.
Here you can rewrite one, here is set to 5 seconds
ConnectionKeepAliveStrategy keepAliveStrategy = new DefaultConnectionKeepAliveStrategy() {
#Override
public long getKeepAliveDuration(final HttpResponse response, final HttpContext context) {
long keepAlive = super.getKeepAliveDuration(response, context);
if (keepAlive == -1) {
keepAlive = 5000;
}
return keepAlive;
}
};
Connection eviction policy
The main disadvantage of a classic blocking I/O model is that network
sockets respond to I/O events only when I/O operations are blocked.
When a connection is released back to the manager, it can be kept
alive without monitoring the status of the socket and responding to
any I/O events. If the connection is closed on the server side, then
the client connection can not detect changes in the connection status
and shut down the local socket to respond properly.
HttpClient tries to alleviate this problem by testing if the connection is outdated, which is no longer valid as it is already closed on the server side before using the connection that made the HTTP request. Outdated connection check is not 100% stable, but instead requires 10 to 30 milliseconds for each request execution. The only workable socket model thread solution that does not involve every free connection is to use a dedicated monitoring thread to reclaim the connection that is considered expired because of prolonged inactivity. Monitoring thread can periodically call ClientConnectionManager#closeExpiredConnections() method to close all expired connections, withdraw from the connection pool closed connection. It can also optionally call the ClientConnectionManager#closeIdleConnections() method to close all connections that have been idle for more than a given period of time.
Resource Link:
http://dev.dafan.info/detail/513285

How does the socket connect timeout work?

To my understanding, the socket connection timeout is controlled by the TCP transport, which uses Retransmission Timeouts (RTOs). if the the ack does not come back before timer expires, the connect request (Sync) will be retransmitted, and the RTO will be doubled.
So what is the functionality of connection timeout in Java socket when we call Socket.connect(endpoint, connectTimeout)

So what is the functionality of connection timeout in Java socket when we call Socket.connect(endpoint, connectTimeout)
It sets an overall timeout for the connection to have been established; i.e. it says how long the application is prepared to wait for all of the packet-level timeouts, retransmissions, etc to succeed (or not) before giving up.