Develop Reference

Spring-remoting SimpleHttpInvokerRequestExecutor retries request when it shouldn't

I am investigating a quite strange problem. The project I'm working on uses Spring-remoting to invoke methods over http. From what I have gathered so far the following happens:
My client code executes a request to the server
The server starts handling the request, but is slow
25-30 seconds later, a new request comes in to the server
The second request finishes, the client continues its processing
A while later, the first request get completed, but the client no longer cares
Since my client code executes only one request to the Spring remoting client, and the client continuous on after the second invocation it receives is completed, I can only conclude that this occurs somewhere in the Spring remoting client.
The client uses AbstractHttpInvokerRequestExecutor to make the actual http-invocation, and this in turn uses SimpleHttpInvokerRequestExecutor to make the request. But, from what I can read, this has no mechanism to retry the requests. So now I'm quite stuck.
Can anyone think of what might cause this behaviour? (I have tried to keep the question clean, but I have more details if needed.)

Just an idea to give you some direction, not necessarily a solution. Use a third party Http client (not one from Spring) to see if it changes a behavior. That might help you to see if it is SimpleHttpInvokerRequestExecutor that is "guilty" of re-try or something else. Here is a very simple 3d party HttpClient: Provided in MgntUtils Open source library (written by me). Very simple in use. Take a look at Javadoc. Library itself provided as Maven artifacts and on Git (including source code and Javadoc). All in all your code may look like this:
private static void testHttpClient() {
HttpClient client = new HttpClient();
client.setContentType("application/json");
String content = null;
try {
content = client.sendHttpRequest("http://www.google.com/", HttpMethod.GET);
//content holds the response. Do your logic here
} catch (IOException e) {
//Error Handling is here
content = TextUtils.getStacktrace(e, false);
}
}

What is the best way for pooling httpclient?

My main goal is for pooling a httpclient in silent or somehow, and i'd like to call a method to give me a httpclient...because i think its too resource usage for every rest call add a new httpclient instance and set the things and so on...is there best practice for it?

If you refer to Apache HTTP client then you may follow the steps below:
Initiate Apache HTTP client only once, since it is thread safe you can safely reuse it. If you use Spring then it should be safe to store it in the Spring Context as a Bean. See following link for the thread-safety.
Despite the fact that HTTP client instance itself is not pooled (since you gonna use single instance of it) what you can do to increase the performance is configuring pooled connection manager on the HTTP client. See following link for the details.
(search for the 'Pooling connection manager' on that page) . The actual code should be something similar to the snippet bellow :
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
// Increase max total connection to 200
cm.setMaxTotal(200);
// Increase default max connection per route to 20
cm.setDefaultMaxPerRoute(20);
// Increase max connections for localhost:80 to 50
HttpHost localhost = new HttpHost("localhost", 80);
cm.setMaxPerRoute(new HttpRoute(localhost), 50);
CloseableHttpClient httpClient = HttpClients.custom()
.setConnectionManager(cm)
.build();
Hope this helps .

Should I still set ConnectionRequestTimeout on Apache HttpClient if I don't use a custom connection manager?

I am using Apache RequestConfig to configure some timeouts on my HttpClient.
RequestConfig config = RequestConfig.custom()
.setConnectTimeout(timeout)
.setSocketTimeout(timeout)
.setConnectionRequestTimeout(timeout) // Can I leave this out..
.build();
CloseableHttpClient httpClient = HttpClients.custom()
//.setConnectionManager(connectionManager) // ..if I don't use this
.setDefaultRequestConfig(config)
.build();
Does it make any sense to call setConnectionRequestTimeout(timeout) even I don't have a custom Connection Manager / Pool set up?
As far as I understand, setConnectionRequestTimeout(timeout) is used to set the time to wait for a connection from the connection manager/pool.
Note that I am not setting a Connection Manager on the httpClient (see commented line).

connectionRequestTimeout happens when you have a pool of connections and they are all busy, not allowing the connection manager to give you a connection to make the request.
So, The answer to your question of:
Does it make any sense to call setConnectionRequestTimeout(timeout)
even I don't have a custom Connection Manager / Pool set up?
is YES.
This is because the default implementation has an internal connection pool.
So, yes it makes sense to specify a connection request timeout. Actually it is a good, safe practice.

Isuru's answer is mostly correct. The default connection manager is a PoolingHttpClientConnectionManager.
However, by default it will only have one connection in it's pool.
If you are using your HttpClient synchronously from the same thread you should never encounter a situation where the ConnectionRequestTimeout will take effect.
If you are using the HttpClient from multiple threads then you might want to set it, but you would probably also want to increase the pool size, among other things.
For single-threaded httpclient use it is safe to leave it out.

Multi-threaded connection manager with Jersey Apache Client 4

I have a Jersey client up and running, using the Apache Client 4 library, like this:
private Client createClient() {
ApacheHttpClient4Config cc = new DefaultApacheHttpClient4Config();
// boring stuff here
return ApacheHttpClient4.create(cc);
}
But this by default uses a BasicClientConnManager, which doesn't allow multi-threaded connections.
The ApacheHttpClient4Config Javadoc says that I need to set the PROPERTY_CONNECTION_MANAGER to a ThreadSafeClientConnManager instance if I want multi-threaded operation. I can do this, and it works OK:
private Client createClient() {
ApacheHttpClient4Config cc = new DefaultApacheHttpClient4Config();
cc.getProperties().put(ApacheHttpClient4Config.PROPERTY_CONNECTION_MANAGER,
new ThreadSafeClientConnManager());
// boring stuff here
return ApacheHttpClient4.create(cc);
}
But ThreadSafeClientConnManager is deprecated. This is annoying.
The more modern version is PoolingHttpClientConnectionManager. Unfortunately, though, the ApacheHttpClient4.create() method requires the connection manager to be an implementation of ClientConnectionManager (itself deprecated), and PoolingHttpClientConnectionManager doesn't implement that interface. So if I try to use it, my connection manager gets ignored and we're back to a BasicClientConnManager.
How can I end up with a thread-safe client without using anything that's deprecated?

You can create the client as follows (see https://github.com/phillbarber/connection-leak-test/blob/master/src/test/java/com/github/phillbarber/connectionleak/IntegrationTestThatExaminesConnectionPoolBeforeAndAfterRun.java#L30-L33):
client = new ApacheHttpClient4(new ApacheHttpClient4Handler(HttpClients.custom()
.setConnectionManager(new PoolingHttpClientConnectionManager())
.build(), null, false));

Interrupt a long running Jersey Client operation

I am using the Oracle Jersey Client, and am trying to cancel a long running get or put operation.
The Client is constructed as:
JacksonJsonProvider provider = new JacksonJsonProvider(new ObjectMapper());
ClientConfig clientConfig = new DefaultClientConfig();
clientConfig.getSingletons().add(provider);
Client client = Client.create(clientConfig);
The following code is executed on a worker thread:
File bigZipFile = new File("/home/me/everything.zip");
WebResource resource = client.resource("https://putfileshere.com");
Builder builder = resource.getRequestBuilder();
builder.type("application/zip").put(bigZipFile); //This will take a while!
I want to cancel this long-running put. When I try to interrupt the worker thread, the put operation continues to run. From what I can see, the Jersey Client makes no attempt to check for Thread.interrupted().
I see the same behavior when using an AsyncWebResource instead of WebResource and using Future.cancel(true) on the Builder.put(..) call.
So far, the only solution I have come up with to interrupt this is throwing a RuntimeException in a ContainerListener:
client.addFilter(new ConnectionListenerFilter(
new OnStartConnectionListener(){
public ContainerListener onStart(ClientRequest cr) {
return new ContainerListener(){
public void onSent(long delta, long bytes) {
//If the thread has been interrupted, stop the operation
if (Thread.interrupted()) {
throw new RuntimeException("Upload or Download canceled");
}
//Report progress otherwise
}
}...
I am wondering if there is a better solution (perhaps when creating the Client) that correctly handles interruptible I/O without using a RuntimeException.

I am wondering if there is a better solution (perhaps when creating the Client) that correctly handles interruptible I/O without using a RuntimeException.
Yeah, interrupting the thread will only work if the code is watching for the interrupts or calling other methods (such as Thread.sleep(...)) that watch for it.
Throwing an exception out of listener doesn't sound like a bad idea. I would certainly create your own RuntimeException class such as TimeoutRuntimeException or something so you can specifically catch and handle it.
Another thing to do would be to close the underlying IO stream that is being written to which would cause an IOException but I'm not familiar with Jersey so I'm not sure if you can get access to the connection.
Ah, here's an idea. Instead of putting the File, how about putting some sort of extension on a BufferedInputStream that is reading from the File but also has a timeout. So Jersey would be reading from the buffer and at some point it would throw an IOException if the timeout expires.

As of Jersey 2.35, the above API has changed. A timeout has been introduces in the client builder which can set read timeout. If the server takes too long to respond, the underlying socket will timeout. However, if the server starts sending the response, it shall not timeout. This can be utilized, if the server does not start sending partial response, which depends on the server implementation.
client=(JerseyClient)JerseyClientBuilder
.newBuilder()
.connectTimeout(1*1000, TimeUnit.MILLISECONDS)
.readTimeout(5*1000, TimeUnit.MILLISECONDS).build()
The current filters and interceptors are for data only and the solution posted in the original question will not work with filters and interceptors (though I admit I may have missed something there).
Another way is to get hold of the underlying HttpUrlConnection (for standard Jersey client configuration) and it seems to be possible with org.glassfish.jersey.client.HttpUrlConnectorProvider
HttpUrlConnectorProvider httpConProvider=new HttpUrlConnectorProvider();
httpConProvider.connectionFactory(new CustomHttpUrlConnectionfactory());
public static class CustomHttpUrlConnectionfactory implements
HttpUrlConnectorProvider.ConnectionFactory{
#Override
public HttpURLConnection getConnection(URL url) throws IOException {
System.out.println("CustomHttpUrlConnectionfactory ..... called");
return (HttpURLConnection)url.openConnection();
}//getConnection closing
}//inner-class closing
I did try the connection provider approach, however, I could not get that working. The idea would be to keep reference to the connection by some means (thread id etc.) and close it if the communication is taking too long. The primary problem was I could not find a way to register the provider with the client. The standard
.register(httpConProvider)
mechanism does not seem to work (or perhaps it is not supposed to work like that) and the documentation is a bit sketchy in that direction.