Develop Reference

How do I create an async caching http client?

Using the org.apache.httpcomponents:httpasyncclient-cache:4.1.3 library, I'm trying to work out how I can create an asynchronous caching http client?
I can create each individually, using their respective builders, but I can't find a way to have both.
e.g.
CloseableHttpClient client = CachingHttpClientBuilder.create()
.setCacheConfig(cacheConfig())
.build();
CloseableHttpAsyncClient build = HttpAsyncClientBuilder.create()
.build();
Por que no los dos?
N.B. I'm not tied to this version of the library - happy for solutions using the latest version.
Related:
How do I implement client side http caching like a browser?

You can create a new CachingHttpAsyncClient directly after you have built a HttpAsyncClient. For example:
CloseableHttpAsyncClient asyncClient= HttpAsyncClientBuilder.create().build();
CachingHttpAsyncClient client = new CachingHttpAsyncClient(asyncClient, cacheConfig());
You can know more constructors from here.

How to force the http client to not handle the authentication challenges automatically in httpClient 4.3.x?

I am migrating my httpclient 3.x to httpclient 4.3.x.
With httpClient 3.x I used setDoAuthentication(false) which ensured that the method would not try to handle the authentication challenges automatically. How can I set the same functionality with httpClient 4.3.x?
I already went through the javadocs of httpmethods and httpclient and there is no way to do it. The closest I have been is the use of disableAuthCaching() in httpClientBuilder, but not sure whether it would serve the purpose.
Can anyone please provide some pointers?
I have read somewhere that there is a parameter through which we can achieve this behavior but I am not able to find it anywhere. Sorry if it looks like a pretty basic question.

I think I got it. I guess it can be done by setting the setDoAuthentication() method to false in the requestConfig for httpClient.

#spock8190 thanks for your post this gave me the correct hint.
But the new method name is setAuthenticationEnabled (at least for 4.5)
Determines whether authentication should be handled automatically.
Example:
HttpGet httpGet = new HttpGet("test.de");
RequestConfig requestConfig = RequestConfig.custom().setAuthenticationEnabled(true).build();
httpGet.setConfig(requestConfig);

HTTPClient sends out two requests when using Basic Auth?

I have been using HTTPClient version 4.1.2 to try to access a REST over HTTP API that requires Basic Authentication. Here is client code:
DefaultHttpClient httpClient = new DefaultHttpClient(new ThreadSafeClientConnManager());
// Enable HTTP Basic Auth
httpClient.getCredentialsProvider().setCredentials(
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),
new UsernamePasswordCredentials(this.username, this.password));
HttpHost proxy = new HttpHost(this.proxyURI.getHost(), this.proxyURI.getPort());
httpClient.getParams().setParameter(ConnRouteParams.DEFAULT_PROXY, proxy);
When I construct a POST request, like this:
HttpPost request = new HttpPost("http://my/url");
request.addHeader(new BasicHeader("Content-type", "application/atom+xml; type=entry")); // required by vendor
request.setEntity(new StringEntity("My content"));
HttpResponse response = client.execute(request);
I see in Charles Proxy that there are two requests being sent. One without the Authorization: Basic ... header and one with it. The first one fails with a 401, as you would expect, but the second goes through just fine with a 201.
Does anyone know why this happens? Thanks!
EDIT:
I should make clear that I have already looked at this question, but as you can see I set the AuthScope the same way and it didn't solve my problem. Also, I am creating a new HttpClient every time I made a request (though I use the same ConnectionManager), but even if I use the same HttpClient for multiple requests, the problem still persists.
EDIT 2:
So it looks like what #LastCoder was suggesting is the way to do. See this answer to another question. The problem stems from my lack of knowledge around the HTTP spec. What I'm looking to do is called "preemptive authentication" and the HttpClient docs mention it here. Thankfully, the answer linked to above is a much shorter and cleaner way to do it.

Rather than using .setCredentials() why don't you just encode USERNAME:PASSWORD and add the authentication header with .addHeader()

This means that your server/target endpoint is creating a new session for every client request. This forces every request of yours to go through a hand-shake, which means the clients first makes the call and realizes that it needs authorization, then it follows with the authorization. What you need to do is send the authorization preemptively as follows:
httpClient.getParams().setAuthenticationPreemptive(true);
Just to understand the process you may log your client request headers, to give you an idea of what your client is sending and receiving:
See if this works.

Base64 Encoding Basic Authentication Header Apache HTTP Client

Two related questions, I'm using Apache HTTP Client 4.x API. myHttpPost is an instance of HttpPost and myHttpClient is an instance of HttpClient. I'm trying to send a request using basic authentication. So I have a HttpClient and create a HttpPost.
The 'brute force' way of setting a basic authentication header seems to be to set it in the HttpPost header.
String encoding = Base64Encoder.encode("username" + ":" + "password");
myHttpPost.setHeader("Authorization", "Basic " + encoding);
The example above came from another stack overflow question (can't find link to it now). In relation to the Base64Encoder class - which package would I find it in or where would I download it from?
Main question - I was hoping to do basic authentication in a more aesthetic manner using the code below:
myHttpClient.getCredentialsProvider().setCredentials(
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, AuthScope.ANY_REALM, AuthPolicy.BASIC),
new UsernamePasswordCredentials("username", "password")
);
But this doesn't seem to work. So is the first example above the right way to do basic authentication with Apache HTTP Client 4.0? Or is there a cleaner/simpler way.

In relation to the Base64Encoder class - which package would I find it
in or where would I download it from?
Base64Encoder can come from various places, I couldn't find something that matches with your static encode method.
As for Credentials, you need to set scheme to Basic on your AuthScope, like so:
myHttpClient.getCredentialsProvider().setCredentials(
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, AuthScope.ANY_REALM, "basic"),
new UsernamePasswordCredentials("username", "password")
);
or
myHttpClient.getCredentialsProvider().setCredentials(
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, AuthScope.ANY_REALM, AuthPolicy.BASIC),
new UsernamePasswordCredentials("username", "password")
);

I know this is a really old post. But, just wanted to answer so that others will benefit in future:
If you are using a username which is to be represented using Variable width encoding (http://en.wikipedia.org/wiki/Variable-width_encoding), then make sure that you change the encoding used to form the bytes for (username:password) as follows:
HttpParams params = new BasicHttpParams();
params.setParameter(AuthPNames.CREDENTIAL_CHARSET, HTTP.UTF_8);
By default, the encoding used is HttpProtocolParams.HTTP_ELEMENT_CHARSET.

HttpClient does not attempt to authenticate with the origin or proxy server unless explicitly challenged. I suspect you would like HttpClient to authenticate preemptively. While the preemptive authentication is disabled per default (and I personally would discourage its application outside secure or internal networks) one can force the preemptive authentication using example below
http://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientPreemptiveBasicAuthentication.java

I tried out the solution suggested in http://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientPreemptiveBasicAuthentication.java and it works for Base64 Encoding Basic Authentication.
I guess the
// Create AuthCache instance
AuthCache authCache = new BasicAuthCache();
// Generate BASIC scheme object and add it to the local
// auth cache
BasicScheme basicAuth = new BasicScheme();
authCache.put(target, basicAuth);
// Add AuthCache to the execution context
HttpClientContext localContext = HttpClientContext.create();
localContext.setAuthCache(authCache);
makes the miracle :)
as without this i always receive "401 Unauthorized" HTTP response

Does Apache HttpClient add the Cookies set by the java.net.CookieHandler to Request?

My simple Apache HttpClient (4.0.1) client application makes an HttpGet request to a server URL in the main() method and prints the response. On startup, the application registers an implementation of java.net.CookieHandler in a static block.
On checking the cookies received on the server side, I found that the cookies are not being received by the Server when the HttpClient makes the GET request.
On the other hand, when I replaced the Apache HttpClient with a plain java.net.URL(HTTP_URL).openStream(), the cookies were set by the CookieHandler on the Request and were received by the Server.
Is it that CookieHandler does not work with Apache HttpClient?
Code:
Client.java
static {
CookieHandler.setDefault(new CookieHandler() {
public Map get(URI u, List r) {
return Collections.singletonMap("Cookie",
Collections.singletonList(COOKIE_STRING));
}
});
}
Using HttpClient (does not put cookies on request)
HttpClient client = new DefaultHttpClient();
HttpGet get = new HttpGet(HTTP_URL);
client.execute(get);
Using java.net.URL (sets the cookies on request)
URL url = new URL(HTTP_URL);
InputStream is = url.openStream();

Is it that CookieHandler does not work with Apache HttpClient?
That is correct.
The Apache HttpClient codebase uses its own cookie and cookie store representations / mechanisms. Here is a link to the relevant section of the HttpClient tutorial. (It is pretty sketchy, but if you look at the javadocs for the relevant classes, you should be able to figure out how to use it.)
(If you are using an older version of Apache HttpClient, beware that the APIs have changed significantly.)