I am new in Apache HttpClient, I used the following code to get the HTTP connection timeout (disconnected) after certain time interval:
PostMethod method = new PostMethod(authURL);
HttpClient client = new HttpClient();
HttpClientParams params= new HttpClientParams();
params.setParameter(params.CONNECTION_MANAGER_TIMEOUT, 10); //10 Nano second
client.executeMethod(method);
but it wait for more than one minute without any hope to timeout/disconnect? Where can the problem be?
There are 2 timeouts involved in HTTPClient, try to set both,
client.getHttpConnectionManager().
getParams().setConnectionTimeout(5000);
client.getHttpConnectionManager().
getParams().setSoTimeout(5000);
However, the values will be ignored if the connection is stuck in a native socket call. So you might have to run the request in a different thread so you can time it out. See my answer to this question on how to do that,
java native Process timeout
The connection manager timeout triggers when the act of trying to get a connection from your connection manager takes too long. This is not the same as the timeout for the http connection itself. Use HttpClientParams.setSoTimeout() instead.
http://hc.apache.org/httpclient-3.x/apidocs/org/apache/commons/httpclient/params/HttpMethodParams.html#setSoTimeout%28int%29
Have you looked at setting SO_TIMEOUT ?
Sets the socket timeout (SO_TIMEOUT)
in milliseconds to be used when
executing the method. A timeout value
of zero is interpreted as an infinite
timeout.
Related
I'm making an HTTP connection using the Apache Common's HTTPClient.
The connection type is POST, and the client will be reading an output from the connection.
However I need to be able to abort the connection at any time regardless of the status of connection. What is the best way of doing this?
Is there a built in timeout?
Regards.
EDIT:
Just to clarify the my question:
I would like the user to be able to choose when the connection is to be cut.
In other words similar functionality to the parameter cURL:
CURLOPT_TIMEOUT
https://curl.haxx.se/libcurl/c/CURLOPT_TIMEOUT.html
When you're instantiating the instance of httpClient using HttpClientBuilder, you can pass an instance of RequestConfig to the builder, which inturn accepts 2 parameters.
SocketTimeout - Defines the socket timeout (SO_TIMEOUT) in milliseconds, which is the timeout for waiting for data or, put differently, a maximum period inactivity between two consecutive data packets).
ConnectTimeout - Determines the timeout in milliseconds until a connection is established. A timeout value of zero is interpreted as an infinite timeout.
ConnectionRequestTimeout - Returns the timeout in milliseconds used when requesting a connection from the connection manager. A timeout value of zero is interpreted as an infinite timeout.
What you're looking for is SocketTimeout.
RequestConfig requestConfig = RequestConfig.custom().setConnectTimeout(CONNECTION_TIMEOUT)
.setConnectionRequestTimeout(CONNECTION_TIMEOUT).setSocketTimeout(SOCKET_TIMEOUT).build();
httpClient = HttpClientBuilder.create().disableAutomaticRetries().setDefaultRequestConfig(requestConfig)
.setConnectionManager(poolingHttpClientConnectionManager).disableRedirectHandling().build();
You can take a look at ConnectionKeepAliveStrategy. Refer to "2.6. Connection keep alive strategy" section of page
If you're using DefaultHttpClient, it accepts HttpParams where you can provide connection related settings.
SO_TIMEOUT is the SocketTimeout - Defines the socket timeout (SO_TIMEOUT) in milliseconds, which is the timeout for waiting for data or, put differently, a maximum period inactivity between two consecutive data packets).
HttpParams httpParams = new BasicHttpParams();
httpParams.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, CONNECTION_TIMEOUT);
httpParams.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, SOCKET_TIMEOUT);
DefaultHttpClient backend = new DefaultHttpClient(httpParams);
A better way of doing it is by using HttpClientBuilder as #Bandi Kishore suggested. or by making use of PoolingClientConnectionManager which accepts these settings directly using setters.
Can someone please explain what is the difference between these two:
client = HttpClientBuilder.create()
.setConnectionTimeToLive(1, TimeUnit.MINUTES)
.build();
and
RequestConfig requestConfig = RequestConfig.custom().setConnectTimeout(30 * 1000).build();
client = HttpClientBuilder
.create()
.setDefaultRequestConfig(requestConfig)
.build();
Is it better to use setSocketTimeout()?
A ConnectTimeout determines the maximum time to wait for the other side to answer "yes, I'm here, let's talk" when creating a new connection (ConnectTimeout eventually calls socket.connect(address, timeout). The wait-time is usually less than a second, unless the other side is really busy with just accepting new incoming connections or you have to go through the great firewall of China. In the latter cases it can be a minute (or more) before the new connection is created. If the connection is not established within the ConnectTimeout, you get an error (1).
setSocketTimeout eventually calls socket.setSoTimeout which is explained in this answer.
The ConnectionTimeToLive determines the maximum age of a connection (after which it will be closed), regardless of when the connection was last used. Normally, there is an "idle timeout" to cleanup connections, i.e. you or the other side will close a connection that is not used for a while. Typically, you will close an idle connection before the other side does to prevent errors. But there are two other cases I can think of where a maximum age for a connection is useful:
Bad network components: count yourself lucky if you have not met them. Some bad routers, firewalls, proxies, etc. will just drop (actively being used) connections after something like 30 minutes. Since you and the other side may not even be aware that a connection was dropped, you can get "connection reset" errors for no obvious reason at weird times.
Cached meta-data: most systems keep some meta-data about a connection in some sort of cache. Some systems manage this cache badly - cache size just grows with the age of the connection.
A note about the ConnectionTimeToLive implementation in Apache HttpClient 4.5.4: I think you must use the PoolingHttpClientConnectionManager for the option to work (it eventually all comes down to a call to this isExpired method). If you do not use this connection manager, test the option to make sure it really works.
(1) Interesting comment from EJP user207421 on this related answer
Connection Timeout:
It is the timeout until a connection with the server is established.
Socket Timeout:
this is the time of inactivity to wait for packets[data] to receive.
setConnectionRequestTimeout:
However it is specific for configuring the connection manager. It is the time to fetch a connection from the connection pool.
It returns the timeout in milliseconds used when requesting a connection from the connection manager. 0(zero) is used for an infinite timeout.
setConnectionTimeToLive
public final HttpClientBuilder setConnectionTimeToLive(long connTimeToLive, TimeUnit connTimeToLiveTimeUnit)
Sets maximum time to live for persistent connections
Please note this value can be overridden by the setConnectionManager(org.apache.http.conn.HttpClientConnectionManager) method.
Since:
4.4
Example: HttpClientStarter.java
#Override
public boolean start() {
RegistryBuilder<ConnectionSocketFactory> r = RegistryBuilder.<ConnectionSocketFactory> create();
// Register http and his plain socket factory
final SocketFactory ss = getLevel().find(SocketFactory.class);
ConnectionSocketFactory plainsf = new PlainConnectionSocketFactory() {
#Override
public Socket createSocket(HttpContext context) throws IOException {
return ss.createSocket();
}
};
r.register("http", plainsf);
// Register https
ConnectionSocketFactory sslfactory = getSSLSocketFactory();
if (sslfactory != null) {
r.register("https", getSSLSocketFactory());
} else {
log(Level.WARN, "ssl factory not found, won't manage https");
}
HttpClientBuilder builder = HttpClientBuilder.create();
builder.setUserAgent(USERAGENT);
builder.setConnectionTimeToLive(timeout, TimeUnit.SECONDS);
builder.evictIdleConnections((long) timeout, TimeUnit.SECONDS);
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(r.build());
cm.setMaxTotal(maxConnect * 2);
cm.setDefaultMaxPerRoute(2);
cm.setValidateAfterInactivity(timeout * 1000);
builder.setConnectionManager(cm);
RequestConfig rc = RequestConfig.custom()
.setConnectionRequestTimeout(timeout * 1000)
.setConnectTimeout(timeout * 1000)
.setSocketTimeout(timeout * 1000)
.build();
builder.setDefaultRequestConfig(rc);
client = builder.build();
return true;
}
Resource Link:
HttpClientStarter.java
HttpClient 4.x
Timeout
The HTTP specification does not determine how long a persistent connection may or should remain active. Some HTTP servers use a non-standard header, Keep-Alive, to tell clients the number of seconds they want to stay connected on the server side. HttClient will take advantage of this if this information is available. If the header information Keep-Alive does not exist in the response, HttpClient assumes the connection remains active indefinitely. However, many real-world HTTP servers are configured to discard persistent connections after certain periods of inactivity to conserve system resources, often without notification to the client.
Here you can rewrite one, here is set to 5 seconds
ConnectionKeepAliveStrategy keepAliveStrategy = new DefaultConnectionKeepAliveStrategy() {
#Override
public long getKeepAliveDuration(final HttpResponse response, final HttpContext context) {
long keepAlive = super.getKeepAliveDuration(response, context);
if (keepAlive == -1) {
keepAlive = 5000;
}
return keepAlive;
}
};
Connection eviction policy
The main disadvantage of a classic blocking I/O model is that network
sockets respond to I/O events only when I/O operations are blocked.
When a connection is released back to the manager, it can be kept
alive without monitoring the status of the socket and responding to
any I/O events. If the connection is closed on the server side, then
the client connection can not detect changes in the connection status
and shut down the local socket to respond properly.
HttpClient tries to alleviate this problem by testing if the connection is outdated, which is no longer valid as it is already closed on the server side before using the connection that made the HTTP request. Outdated connection check is not 100% stable, but instead requires 10 to 30 milliseconds for each request execution. The only workable socket model thread solution that does not involve every free connection is to use a dedicated monitoring thread to reclaim the connection that is considered expired because of prolonged inactivity. Monitoring thread can periodically call ClientConnectionManager#closeExpiredConnections() method to close all expired connections, withdraw from the connection pool closed connection. It can also optionally call the ClientConnectionManager#closeIdleConnections() method to close all connections that have been idle for more than a given period of time.
Resource Link:
http://dev.dafan.info/detail/513285
I have a client on Host A using Apache HTTPClient (4.2.5) to POST a message to a server on Host B. On Host A, I am setting the connect and read/socket timeouts in (what I believe to be) the recommended fashion:
client = new DefaultHttpClient();
params = client.getParams();
HttpConnectionParams.setConnectionTimeout(params, 5000);
HttpConnectionParams.setSoTimeout(params, 20000);
I am attempting to test the timeout behavior in the event that the client transmission is abruptly terminated (e.g. by a crash of the target server on Host B) after some but not all of the data stream has been transmitted. My testing methodology has been thus:
Initiate HTTP POST from client application on Host A.
Shortly after the logs indicate that the client has invoked the underlying Apache HTTPClient framework, enable a firewall rule on Host B to block traffic from Host A.
I can confirm (via packet captures) that the test is, in fact, cutting off the request data mid-stream.
I would have expected the client application to subsequently time out after the read timeout interval (20 seconds), but what I'm seeing instead is that the client hangs for a much longer interval (in the 500-600 seconds range) before finally encountering a connect timeout.
Can anyone explain (or failing that, venture a guess) as to why the client is ignoring what I have specified as the timeout? Is there any way that I can enforce a timeout in this case?
Not sure why that isn't working but you could try an alternate approach where you create the Params object prior to instantiating the HttpClient.
Something to this effect:
HttpParams params = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(params, 5000);
HttpConnectionParams.setSoTimeout(params, 20000);
HttpClient httpClient = new DefaultHttpClient(params);
There is also a set of parameters unique to each http message which would override the setting of the client connection object. You might want to check to see if they specify the timeout themselves and thus leave the one you set unused (shouldn't happen unless explicitly set, but worth checking)
HttpMessage#getParam
Further, you could try to explicity set those parameters on your HttpGet or HttPost objects yourself and see if that alters the behavior.
So what happens is, the timeout property is applicable on each address that is obtained by DNS resolution. E.g abc.come resolved to 4 different ip address and you have set a timeout of 10 sec. So your request will wait for 4 x 10 = 40 sec in total before timing out.
I am trying to set the timeout of a connection on the client socket in Java. I have set the connection timeout to 2000 milliseconds, i.e:
this.socket.connect(this.socketAdd, timeOut);
This I am trying on a web application. When a user makes a request, I am passing values to socket server, but if I don't receive any response in 5 secs the socket should disconnect.
But in my case the whole request is getting submitted once again. Can any one please tell me where am I going wrong?
I want to cut the socket connection, if I don't get any response in 5 secs. How can I set it? Any sample code would help.
You can try the follwoing:
Socket client = new Socket();
client.connect(new InetSocketAddress(hostip, port_num), connection_time_out);
To put the whole thing together:
Socket socket = new Socket();
// This limits the time allowed to establish a connection in the case
// that the connection is refused or server doesn't exist.
socket.connect(new InetSocketAddress(host, port), timeout);
// This stops the request from dragging on after connection succeeds.
socket.setSoTimeout(timeout);
What you show is a timeout for the connection, this will timeout if it cannot connect within a certain time.
Your question implies you want a timeout for when you are already connected and send a request, you want to timeout if there is no response within a certain amount of time.
Presuming you mean the latter, then you need to timeout the socket.read() which can be done by setting SO_TIMEOUT with the Socket.setSoTimeout(int timeout) method. This will throw an exception if the read takes longer than the number of milliseconds specified. For example:
this.socket.setSoTimeout(timeOut);
An alternative method is to do the read in a thread, and then wait on the thread with a timeout and close the socket if it timesout.
I'm running into an issue when i try to use the HttpClient connecting
to a url. The http connection is taking a longer time to timeout, even after i set
a connection timeoout.
int timeoutConnection = 5000;
HttpConnectionParams.setConnectionTimeout(httpParameters, timeoutConnection);
int timeoutSocket = 5000;
HttpConnectionParams.setSoTimeout(httpParameters, timeoutSocket);
It works perfect most of the time. However, every once in while, the http connection runs for ever and ignore the setconnectiontimeout, especailly when the phone is connected to wifi, and the phone was idling.
So after the phone is idling, the first time i try to connect, the http connection ignores the setconnectiontimeout and runs forever, after i cancel it and try again, it works like charm everytime. But that one time that doesn't work it creates a threadtimeout error, i tried using a different thread, it works, but i know that the thread is running for long time.
I understand that the wifi goes to sleep on idle, but i dont understand why its ignoring the setconnectiontimeout.
Anyone can help, id really appreciated.
Not sure if this helps you, however I think it's worth sharing here. While playing with the timeout stuff I found there is a third timeout type you can assign:
// the timeout until a connection is established
private static final int CONNECTION_TIMEOUT = 5000; /* 5 seconds */
// the timeout for waiting for data
private static final int SOCKET_TIMEOUT = 5000; /* 5 seconds */
// ----------- this is the one I am talking about:
// the timeout until a ManagedClientConnection is got
// from ClientConnectionRequest
private static final long MCC_TIMEOUT = 5000; /* 5 seconds */
...
HttpGet httpGet = new HttpGet(url);
setTimeouts(httpGet.getParams());
...
private static void setTimeouts(HttpParams params) {
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT,
CONNECTION_TIMEOUT);
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, SOCKET_TIMEOUT);
params.setLongParameter(ConnManagerPNames.TIMEOUT, MCC_TIMEOUT);
}
I've met the same problem, I guess maybe the Android doesn't support this parameter.
In my case i tested all three parameters for the ThreadSafeClientConnManager
params.setParameter( ConnManagerPNames.MAX_CONNECTIONS_PER_ROUTE, new ConnPerRouteBean(20) );
params.setIntParameter( ConnManagerPNames.MAX_TOTAL_CONNECTIONS, 200 );
params.setLongParameter( ConnManagerPNames.TIMEOUT, 10 );
ThreadSafeClientConnManager connmgr = new ThreadSafeClientConnManager( params );
The first and second worked fine, but the third didn't work as documented. No exception was thrown and the executing thread was blocked indefinitely when the DefaultHttpClient#execute() was executing.
see http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html#d4e650
"...One can ensure the connection manager does not block indefinitely in the connection request operation by setting 'http.conn-manager.timeout' to a positive value. If the connection request cannot be serviced within the given time period ConnectionPoolTimeoutException will be thrown."
Thread t=new Thread()
{
public void run()
{
try
{
Thread.sleep(absolutetimeout);
httpclient.getConnectionManager().closeExpiredConnections();
httpclient.getConnectionManager().closeIdleConnections(absolutetimeout,TimeUnit.MILLISECONDS);
httpclient.getConnectionManager().shutdown();
log.debug("We shutdown the connection manager!");
}
catch(InterruptedException e)
{}
}
};
t.start();
HttpResponse res= httpclient.execute(httpget);
t.interrupt();
Is that along the lines of what you all are suggesting?
I'm not exactly sure how to cancel the execute once it has started, but this seemed to work for me. I'm not sure which of the three lines in the thread did the magic, or if it was some combination of all of them.
You could manage the timeouts yourself, this way you can be confident that no matter what state the connection gets in, unless you receive an acceptable response, that your timeout will fire and the http request will be aborted.
I've had similar issues with timeouts on android. To resolve it what I did was used the commands to not let the phone idle while I was attempting to establish a connection and during any reads or writes to the connection. Its probably worth a shot in this case as well.
Although I haven't seen this on the Android platform, I've seen similar things on other platforms and the solution in these cases is to manage the timeout yourself. Kick off another thread (the timeout thread) when your make your request. The timeout thread counts down the requisite time. If the timeout expires before you receive any data, the timeout thread cancels the original request and you retry with a new request. Harder to code, but at least you know it will work.
From you snippet it's not ultimately clear if you set the timeouts before calling HttpClient.executeMethod(..). So this is my guess.
Well, if you idle/multitask to another application, then your thread that is running might be stopped and destroyed. Maybe you should put the connection code inside a Service instead?:
http://developer.android.com/reference/android/os/AsyncTask.html
http://developer.android.com/reference/android/app/IntentService.html
How are you making the HTTP Connection? This looks like a threading issue. If you are using a background thread, then the thread may be killed along with any timeout registered. The fact that it works the next time tells me that your code will work, if you make the call in a android component and manage the WAKE_LOCK on it yourself. Anyways please post more information about the calling mechanism?
The problem might be in the Apache HTTP Client. See HTTPCLIENT-1098.
Fixed in 4.1.2.
The timeout exception tries to reverse DNS the IP, for logging purposes. This takes an additional time until the exception is actually fired.