As HttpClient document suggests - Generally it is recommended to have a single instance of HttpClient per communication component or even per application.
I got different behaviours between HttpClient is singleton or not.
1) With singleton, I first created a global static HttpClient instance, and send every request with this instance with below segment,
PostMethod post = new PostMethod(url);
int status = httpClient.executeMethod(post);
2) Without singleton, I send every request by creating a new HttpClient instance
PostMethod post = new PostMethod(url);
HttpClient httpClient = new HttpClient();
int status = httpClient.executeMethod(post);
The differences are, without singleton, everything is OK, I can get the correct result separately in consecutive requests. But with singleton, it seems there is some request context, the second request doesn't return the response string as expected because of the first request parameter (weird!!).
I don't have the service codes and server configuration. Can you help me figure out the possible reason?
Thanks in advance.
Related
Using Apache HttpClient 4.5.x in my client webapp which connects to (and log in to) another (say main) server webapp.
The relationship between these 2 webapps is many to many - meaning for some user's request in client webapp, it has to login as another user + make rest call, in the server webapp. So some separation of cookiestores is needed and there's no way (is there?) to get/set the cookie store after creating a httpclient instance, so each request thread received in client webapp does something like this (and need to optimize):
HttpClient client = HttpClientBuilder.create().setDefaultCookieStore(new BasicCookieStore()).build();
//Now POST to login end point and get back JSESSIONID cookie and then make one REST call, and then the client object goes out of scope when the request ends.
I was hoping to ask on the best practice of caching the httpclient instance object as its heavy and is supposed to be reused for at least multiple requests, if not for the whole client webapp as a static singleton.
Specifically, I was hoping for advice on which of these (if any) approaches would constitute a best practice:
Use a static ConcurrentHashMap to cache the httpclient and its associated basiccookiestore for each "user" in client webapp, and to login only when the contained cached cookie is near to its expiry time. Not sure about memory usage, and un/rarely-used httpclient stay in memory without eviction.
Cache only the Cookie (somehow), but recreate a new httpclient object whenever the need arises to use that cookie for a rest call. This saves the prior call to login until the cookie expires, but no reuse of htptclient.
PooledConnectionManager - but can't find examples easily, though might require devising an eviction strategy, max number of threads etc. (so can be complex).
Is there any better way of doing this ? Thanks.
References:
http://hc.apache.org/httpclient-3.x/performance.html
Generally it is recommended to have a single instance of HttpClient
per communication component or even per application
Similar at java httpclient 4.x performance guide to resolve issue
Using concurrent hash map would be the simplest way to achieve what you want to do.
Additionally, if you are using Spring, you might want to create a bean for holding the HTTP client.
Why would you want to do all this? One can assign a different CookieStore on a per request basis by using a local HttpContext.
If needed one can maintain a map of CookieStore instances per unique user.
CloseableHttpClient httpclient = HttpClients.createDefault();
CookieStore cookieStore = new BasicCookieStore();
// Create local HTTP context
HttpClientContext localContext = HttpClientContext.create();
// Bind custom cookie store to the local context
localContext.setCookieStore(cookieStore);
HttpGet httpget = new HttpGet("http://httpbin.org/cookies");
System.out.println("Executing request " + httpget.getRequestLine());
// Pass local context as a parameter
CloseableHttpResponse response = httpclient.execute(httpget, localContext);
try {
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
List<Cookie> cookies = cookieStore.getCookies();
for (int i = 0; i < cookies.size(); i++) {
System.out.println("Local cookie: " + cookies.get(i));
}
EntityUtils.consume(response.getEntity());
} finally {
response.close();
}
I've got an HttpClient instance that fetches a remote resource. I configure it to handle redirects.
HttpParams params = new BasicHttpParams();
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT,
SOCKET_TIMEOUT);
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT,
CONNECTION_TIMEOUT);
params.setLongParameter(ClientPNames.CONN_MANAGER_TIMEOUT,
CONN_MANAGER_TIMEOUT_VALUE);
params.setParameter(ClientPNames.COOKIE_POLICY,
CookiePolicy.BROWSER_COMPATIBILITY);
params.setBooleanParameter(ClientPNames.HANDLE_REDIRECTS, true);
params.setBooleanParameter(ClientPNames.REJECT_RELATIVE_REDIRECT,
false);
params.setIntParameter(ClientPNames.MAX_REDIRECTS, 4);
httpclient = new DefaultHttpClient(cm, params);
When I'm calling it from inside a webapp (Tomcat6) I get the 301 response. When I call it from JSE environment I get the 200 final response (redirects get handled). My first suspect was classloading issues, but printing out the source of HttpClient class shows that both times it's loaded from httpclient-4.2.5.jar
Any ideas how else I can debug this?
Run HttpClient with the context / wire logging turned on as described here and compare HTTP message exchanged in both environments.
The HttpClient instance was shared throughout the webapp, including SolrJ (Solr client), which set the "follow redirect" param to false. I figured this out by creating a copy of the RequestDirector with extra logging lines. I could have simply looked for all calls of HttpClient.getParams(). The more you know.
HttpClient client = new DefaultHttpClient();
String URL = ("http://www.alfa");
HttpGet request = new HttpGet(URL);
i want to load the card with "LOAD" method and use "CONFIRM" method
can someone suggest me how to write those methods or should i use only httpget and post methods
I'm not sure a LOAD method exists, according to the RFC: http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
Does the API you're requesting access to specifically state "LOAD" instead of "GET" must be used?
I have a protected resource which requires me to login. Im using the commons client with the following code block.
HttpClient httpClient = new HttpClient();
httpClient.getParams().setParameter("http.protocol.cookie-policy", CookiePolicy.BROWSER_COMPATIBILITY);
httpClient.getParams().setParameter("http.protocol.single-cookie-header", Boolean.TRUE);
PostMethod postMethod = new PostMethod("/admin/adminlogon.do");
postMethod.setRequestEntity(new StringRequestEntity("action=logon&adminUser=admin&adminPassword=password",
"application/x-www-form-urlencoded",
"UTF-8"));
postMethod.addParameter("action","logon");
postMethod.addParameter("adminUser","admin");
postMethod.addParameter("adminPassword","password");
httpClient.executeMethod(postMethod);
String response2 = postMethod.getResponseBodyAsString();
Above is where I basically login. This works fine im getting a nice little JSESSIONID cookie back.
GetMethod get = new GetMethod("/admin/api.do?action=getSomeJson");
httpClient.executeMethod(get);
When I check the logic on the sever the for the 2nd request I notice that we are using a different JSESSIONID. Therefore the get seems to fail to log in. I was under the impression the httpClient managed the cookies and sent the same cookie back. When I log into my app normally through the UI I see the same cookie in each request just not in the this test code.
String s = get.getResponseBodyAsString();
get.releaseConnection();
Do I need to do something with the httpClient to ensure it uses the same cookies from the first post request when it does its get request??
Thanks in advance.
Your assumption regarding HTTP client cookie behavior is correct.
In your case your not use the same httpClient instance. To fix it you need to allocate the httpClient only once (in PostConstructor):
httpClient = new DefaultHttpClient(); // or new HttpClient();
Then, you perform your calls using the same instance of the client. The client will take a cookie from a response, will store it in the cookieStore and will send it with the next request.
[Added after the comment]
The following code works for me:
httpClient = new DefaultHttpClient();
// Create a local instance of cookie store
cookieStore = new BasicCookieStore();
// Set the store
httpClient.setCookieStore(cookieStore);
I have been using HTTPClient version 4.1.2 to try to access a REST over HTTP API that requires Basic Authentication. Here is client code:
DefaultHttpClient httpClient = new DefaultHttpClient(new ThreadSafeClientConnManager());
// Enable HTTP Basic Auth
httpClient.getCredentialsProvider().setCredentials(
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),
new UsernamePasswordCredentials(this.username, this.password));
HttpHost proxy = new HttpHost(this.proxyURI.getHost(), this.proxyURI.getPort());
httpClient.getParams().setParameter(ConnRouteParams.DEFAULT_PROXY, proxy);
When I construct a POST request, like this:
HttpPost request = new HttpPost("http://my/url");
request.addHeader(new BasicHeader("Content-type", "application/atom+xml; type=entry")); // required by vendor
request.setEntity(new StringEntity("My content"));
HttpResponse response = client.execute(request);
I see in Charles Proxy that there are two requests being sent. One without the Authorization: Basic ... header and one with it. The first one fails with a 401, as you would expect, but the second goes through just fine with a 201.
Does anyone know why this happens? Thanks!
EDIT:
I should make clear that I have already looked at this question, but as you can see I set the AuthScope the same way and it didn't solve my problem. Also, I am creating a new HttpClient every time I made a request (though I use the same ConnectionManager), but even if I use the same HttpClient for multiple requests, the problem still persists.
EDIT 2:
So it looks like what #LastCoder was suggesting is the way to do. See this answer to another question. The problem stems from my lack of knowledge around the HTTP spec. What I'm looking to do is called "preemptive authentication" and the HttpClient docs mention it here. Thankfully, the answer linked to above is a much shorter and cleaner way to do it.
Rather than using .setCredentials() why don't you just encode USERNAME:PASSWORD and add the authentication header with .addHeader()
This means that your server/target endpoint is creating a new session for every client request. This forces every request of yours to go through a hand-shake, which means the clients first makes the call and realizes that it needs authorization, then it follows with the authorization. What you need to do is send the authorization preemptively as follows:
httpClient.getParams().setAuthenticationPreemptive(true);
Just to understand the process you may log your client request headers, to give you an idea of what your client is sending and receiving:
See if this works.