I'm experimenting with HTTP/2 client from jdk 9-ea+171. The code is taken from this example:
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
.uri(new URI("https://www.google.com/"))
.build();
HttpResponse<String> response
= client.send(request, HttpResponse.BodyHandler.asString());
But the client hangs on the last line forever. Please advice how to fix it?
Debugging shows it infinitely waits in method waitUntilPrefaceSent().
This is a bug in the latest build's implementation of a HTTP2 connection. It does not occure with previous builds.
First of all, you need to specify the GET method to avoid getting a null pointer exception.
What happens is that the main thread is waiting for the connection preface to be sent. It locks a count down latch to await the receival of this preface. In order to wake itself up, any HttpClient creates a helper thread that reads incoming traffic. This thread is supposed to wake up the main thread but sometimes, this never happens. If you run your example, often enough, you will see that this sometimes work. I guess there is a race for reading the preface.
Unfortunately, the reading of the preface does not respect any timeout either, so there is no way of waking up the main thread, other than interrupting the main thread.
Here is an official ticket: https://bugs.openjdk.java.net/browse/JDK-8181430
Related
According to this the following snippet should be Async.
Therefore, the output should read: TP1, TP2, TP3, http://openjdk.java.net/.
However, when I run it I get: TP1, TP2, http://openjdk.java.net/, TP3.
It seems "sendAsync" is blocking the main thread. This is not what I expected from an Async method.
Am I doing something wrong?
public static void main(String[] args) {
HttpClient client = HttpClient.newHttpClient();
System.out.println("TP1");
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("http://openjdk.java.net/"))
.build();
System.out.println("TP2");
client.sendAsync(request, HttpResponse.BodyHandlers.ofString())
.thenApply(HttpResponse::uri)
.thenAccept(System.out::println)
.join();
System.out.println("TP3");
}
Explanation
You call join() and that will explicitly wait and block until the future is completed.
From CompletableFuture#join:
Returns the result value when complete, or throws an (unchecked) exception if completed exceptionally. [...]
Although not explicitly mentioned but obvious from the name (refer to Thread#join which "Waits for this thread to die."), it can only return a result by waiting for the call to complete.
The method is very similar to CompletableFuture#get, they differ in their behavior regarding exceptional completion:
Waits if necessary for this future to complete, and then returns its result.
Solution
Put the future into a variable and join later, when you actually want to wait for it.
For example:
System.out.println("TP2");
var task = client.sendAsync(request, HttpResponse.BodyHandlers.ofString())
.thenApply(HttpResponse::uri)
.thenAccept(System.out::println);
System.out.println("TP3");
task.join(); // wait later
Or never wait on it. Then your main-thread might die earlier but the JVM only shuts down once all non-daemon threads are dead and the thread used by HttpClient for the async task is not a daemon thread.
Note
Also, never rely on the order of multithreaded execution.
Even if you wouldnt have made a mistake, the order you observe would be a valid order of a multithreaded execution.
Remember that the OS scheduler is free to decide in which order it executes what - it can be any order.
We have a callable class A which actually makes HttpCalls through HttpClient.executeMethod(GetMethod) with a lot of other pre-computations. HttpClient is initialized in the constructor with MultiThreadedHttpConnectionManager.
Another class B creates list of threads for class A through ExecutorService and submits task to the pool and expects future objects to be returned. We have following logic in class B:
for( Future f : futures ){
try{
String str = f.get(timeOut, TimeUnit.SECONDS);
}catch(TimeoutException te){
f.cancel(true);
}
}
This way, our thread gets terminated after a specified time and execution of the task will be terminated and this thread will be available for next task.
I want to confirm the following:
If an external connection is made though HttpClient, how does that get handled on future.cancel of the thread?
In above case or in general, does the http connection pool gets the connection back by properly releasing the previous one? We do release the connection in finally but I don't think interrupting the thread will hit that block.
Could it cause any kind of leak on client or extra resource consumption on the server?
Thanks!
It depends.
If the Http Client uses java.net.Socket, its I/O isn't interrruptible, so the cancel will have no effect.
If it uses NIO, the interrupt will close the channel and cause an exception. At the server this will cause a premature end of stream or an exception on write, either of which the server should cope with corectly.
I am using the Oracle Jersey Client, and am trying to cancel a long running get or put operation.
The Client is constructed as:
JacksonJsonProvider provider = new JacksonJsonProvider(new ObjectMapper());
ClientConfig clientConfig = new DefaultClientConfig();
clientConfig.getSingletons().add(provider);
Client client = Client.create(clientConfig);
The following code is executed on a worker thread:
File bigZipFile = new File("/home/me/everything.zip");
WebResource resource = client.resource("https://putfileshere.com");
Builder builder = resource.getRequestBuilder();
builder.type("application/zip").put(bigZipFile); //This will take a while!
I want to cancel this long-running put. When I try to interrupt the worker thread, the put operation continues to run. From what I can see, the Jersey Client makes no attempt to check for Thread.interrupted().
I see the same behavior when using an AsyncWebResource instead of WebResource and using Future.cancel(true) on the Builder.put(..) call.
So far, the only solution I have come up with to interrupt this is throwing a RuntimeException in a ContainerListener:
client.addFilter(new ConnectionListenerFilter(
new OnStartConnectionListener(){
public ContainerListener onStart(ClientRequest cr) {
return new ContainerListener(){
public void onSent(long delta, long bytes) {
//If the thread has been interrupted, stop the operation
if (Thread.interrupted()) {
throw new RuntimeException("Upload or Download canceled");
}
//Report progress otherwise
}
}...
I am wondering if there is a better solution (perhaps when creating the Client) that correctly handles interruptible I/O without using a RuntimeException.
I am wondering if there is a better solution (perhaps when creating the Client) that correctly handles interruptible I/O without using a RuntimeException.
Yeah, interrupting the thread will only work if the code is watching for the interrupts or calling other methods (such as Thread.sleep(...)) that watch for it.
Throwing an exception out of listener doesn't sound like a bad idea. I would certainly create your own RuntimeException class such as TimeoutRuntimeException or something so you can specifically catch and handle it.
Another thing to do would be to close the underlying IO stream that is being written to which would cause an IOException but I'm not familiar with Jersey so I'm not sure if you can get access to the connection.
Ah, here's an idea. Instead of putting the File, how about putting some sort of extension on a BufferedInputStream that is reading from the File but also has a timeout. So Jersey would be reading from the buffer and at some point it would throw an IOException if the timeout expires.
As of Jersey 2.35, the above API has changed. A timeout has been introduces in the client builder which can set read timeout. If the server takes too long to respond, the underlying socket will timeout. However, if the server starts sending the response, it shall not timeout. This can be utilized, if the server does not start sending partial response, which depends on the server implementation.
client=(JerseyClient)JerseyClientBuilder
.newBuilder()
.connectTimeout(1*1000, TimeUnit.MILLISECONDS)
.readTimeout(5*1000, TimeUnit.MILLISECONDS).build()
The current filters and interceptors are for data only and the solution posted in the original question will not work with filters and interceptors (though I admit I may have missed something there).
Another way is to get hold of the underlying HttpUrlConnection (for standard Jersey client configuration) and it seems to be possible with org.glassfish.jersey.client.HttpUrlConnectorProvider
HttpUrlConnectorProvider httpConProvider=new HttpUrlConnectorProvider();
httpConProvider.connectionFactory(new CustomHttpUrlConnectionfactory());
public static class CustomHttpUrlConnectionfactory implements
HttpUrlConnectorProvider.ConnectionFactory{
#Override
public HttpURLConnection getConnection(URL url) throws IOException {
System.out.println("CustomHttpUrlConnectionfactory ..... called");
return (HttpURLConnection)url.openConnection();
}//getConnection closing
}//inner-class closing
I did try the connection provider approach, however, I could not get that working. The idea would be to keep reference to the connection by some means (thread id etc.) and close it if the communication is taking too long. The primary problem was I could not find a way to register the provider with the client. The standard
.register(httpConProvider)
mechanism does not seem to work (or perhaps it is not supposed to work like that) and the documentation is a bit sketchy in that direction.
I am using org.apache.http and I've this code:
DefaultHttpClient client = new DefaultHttpClient();
HttpGet get = new HttpGet(url);
HttpResponse resp = client.execute(get);
HttpEntity entity = resp.getEntity();
InputStream input = entity.getContent();
...
//Read the bytes from input stream
This is the code I am using to download files over Http, I want to cancel the connection(may be user chooses to) What is the graceful way to close the connection. I found 2 ways, Both cancels the download.
Closing inputsteram, input.close(); which causes IOException.
Aborting HttpGet object, get.abort() causes SocketException.
I have try catch, so no erros, but without throwing exception,
is there a way to cancel or abort the connection?
What is the right way to go about it ?
The proper way doing this is sending FIN value to the server side.
How ever in android you do not have the option to be involved in this level, so you can implement by your self using C, or use one of the methods you mention in your question.
Using HttpUriRequest#about is the right way in my opinion. This will cause immediate termination of the underlying connection and its eviction from the connection pool. Newer versions of HttpClient (4.2 and newer) intercept SocketExceptions caused by premature request termination by the user. The problem is that Google ships a fork of HttpClient based on an extremely outdated version (pre-beta1). If you are not able or willing to use a newer version of HttpClient your only option is to catch and discard SocketException in your code.
Use this
client.getConnectionManager().closeExpiredConnections();
client.getConnectionManager().shutdown();
Now you can decide where would you like to write these 2 lines in code.. It will close the connection using the DefaultHttpClient object that you created.
Let me know if this helps you.
Try to cancel the task when you want to interrupt the connection:
task.cancel(true);
This will cancel the task and the threads running in it.
Check this for reference:
public final boolean cancel (boolean mayInterruptIfRunning)
Since: API Level 3
Attempts to cancel execution of this task. This attempt will fail if the task has already completed, already been cancelled, or could not be cancelled for some other reason. If successful, and this task has not started when cancel is called, this task should never run. If the task has already started, then the mayInterruptIfRunning parameter determines whether the thread executing this task should be interrupted in an attempt to stop the task.
Calling this method will result in onCancelled(Object) being invoked on the UI thread after doInBackground(Object[]) returns. Calling this method guarantees that onPostExecute(Object) is never invoked. After invoking this method, you should check the value returned by isCancelled() periodically from doInBackground(Object[]) to finish the task as early as possible.
Parameters
mayInterruptIfRunning true if the thread executing this task should be interrupted; otherwise, in-progress tasks are allowed to complete.
Returns
false if the task could not be cancelled, typically because it has already completed normally; true otherwise
I'm running into an issue when i try to use the HttpClient connecting
to a url. The http connection is taking a longer time to timeout, even after i set
a connection timeoout.
int timeoutConnection = 5000;
HttpConnectionParams.setConnectionTimeout(httpParameters, timeoutConnection);
int timeoutSocket = 5000;
HttpConnectionParams.setSoTimeout(httpParameters, timeoutSocket);
It works perfect most of the time. However, every once in while, the http connection runs for ever and ignore the setconnectiontimeout, especailly when the phone is connected to wifi, and the phone was idling.
So after the phone is idling, the first time i try to connect, the http connection ignores the setconnectiontimeout and runs forever, after i cancel it and try again, it works like charm everytime. But that one time that doesn't work it creates a threadtimeout error, i tried using a different thread, it works, but i know that the thread is running for long time.
I understand that the wifi goes to sleep on idle, but i dont understand why its ignoring the setconnectiontimeout.
Anyone can help, id really appreciated.
Not sure if this helps you, however I think it's worth sharing here. While playing with the timeout stuff I found there is a third timeout type you can assign:
// the timeout until a connection is established
private static final int CONNECTION_TIMEOUT = 5000; /* 5 seconds */
// the timeout for waiting for data
private static final int SOCKET_TIMEOUT = 5000; /* 5 seconds */
// ----------- this is the one I am talking about:
// the timeout until a ManagedClientConnection is got
// from ClientConnectionRequest
private static final long MCC_TIMEOUT = 5000; /* 5 seconds */
...
HttpGet httpGet = new HttpGet(url);
setTimeouts(httpGet.getParams());
...
private static void setTimeouts(HttpParams params) {
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT,
CONNECTION_TIMEOUT);
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, SOCKET_TIMEOUT);
params.setLongParameter(ConnManagerPNames.TIMEOUT, MCC_TIMEOUT);
}
I've met the same problem, I guess maybe the Android doesn't support this parameter.
In my case i tested all three parameters for the ThreadSafeClientConnManager
params.setParameter( ConnManagerPNames.MAX_CONNECTIONS_PER_ROUTE, new ConnPerRouteBean(20) );
params.setIntParameter( ConnManagerPNames.MAX_TOTAL_CONNECTIONS, 200 );
params.setLongParameter( ConnManagerPNames.TIMEOUT, 10 );
ThreadSafeClientConnManager connmgr = new ThreadSafeClientConnManager( params );
The first and second worked fine, but the third didn't work as documented. No exception was thrown and the executing thread was blocked indefinitely when the DefaultHttpClient#execute() was executing.
see http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html#d4e650
"...One can ensure the connection manager does not block indefinitely in the connection request operation by setting 'http.conn-manager.timeout' to a positive value. If the connection request cannot be serviced within the given time period ConnectionPoolTimeoutException will be thrown."
Thread t=new Thread()
{
public void run()
{
try
{
Thread.sleep(absolutetimeout);
httpclient.getConnectionManager().closeExpiredConnections();
httpclient.getConnectionManager().closeIdleConnections(absolutetimeout,TimeUnit.MILLISECONDS);
httpclient.getConnectionManager().shutdown();
log.debug("We shutdown the connection manager!");
}
catch(InterruptedException e)
{}
}
};
t.start();
HttpResponse res= httpclient.execute(httpget);
t.interrupt();
Is that along the lines of what you all are suggesting?
I'm not exactly sure how to cancel the execute once it has started, but this seemed to work for me. I'm not sure which of the three lines in the thread did the magic, or if it was some combination of all of them.
You could manage the timeouts yourself, this way you can be confident that no matter what state the connection gets in, unless you receive an acceptable response, that your timeout will fire and the http request will be aborted.
I've had similar issues with timeouts on android. To resolve it what I did was used the commands to not let the phone idle while I was attempting to establish a connection and during any reads or writes to the connection. Its probably worth a shot in this case as well.
Although I haven't seen this on the Android platform, I've seen similar things on other platforms and the solution in these cases is to manage the timeout yourself. Kick off another thread (the timeout thread) when your make your request. The timeout thread counts down the requisite time. If the timeout expires before you receive any data, the timeout thread cancels the original request and you retry with a new request. Harder to code, but at least you know it will work.
From you snippet it's not ultimately clear if you set the timeouts before calling HttpClient.executeMethod(..). So this is my guess.
Well, if you idle/multitask to another application, then your thread that is running might be stopped and destroyed. Maybe you should put the connection code inside a Service instead?:
http://developer.android.com/reference/android/os/AsyncTask.html
http://developer.android.com/reference/android/app/IntentService.html
How are you making the HTTP Connection? This looks like a threading issue. If you are using a background thread, then the thread may be killed along with any timeout registered. The fact that it works the next time tells me that your code will work, if you make the call in a android component and manage the WAKE_LOCK on it yourself. Anyways please post more information about the calling mechanism?
The problem might be in the Apache HTTP Client. See HTTPCLIENT-1098.
Fixed in 4.1.2.
The timeout exception tries to reverse DNS the IP, for logging purposes. This takes an additional time until the exception is actually fired.