How to use HttpClient with multithreaded operation? - java

I've to do an application that performs a Login POST request in a certain host, then navigates some pages, finds and retrieves some data.
Becase the website resouce is protected by session, so I have to login the website first before I can do some operation such as get or post some data.
My question is because HttpClient is not thread-safe, how can I create only one HttpClient instance but threads can perform on it safely?
Remember that the underlying connection must login first before it can be used to operate.

Here is an answer: http://pro-programmers.blogspot.com/2009/06/apache-httpclient-multi-threads.html

You can make HttpClient thread safe by specifying a thread safe client manager.
API : http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/tsccm/ThreadSafeClientConnManager.html
http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/DefaultHttpClient.html#DefaultHttpClient%28org.apache.http.conn.ClientConnectionManager%29
Example : http://thinkandroid.wordpress.com/2009/12/31/creating-an-http-client-example/

Related

How do OkHttp caches work?

I recently made a change from my existing implementation of creating a new client for every request to one standard singleton client.
Now as i began reading into the docs, i found out that there's also a thing called cache which is being used. There was some content which said that having multiple caches trying to access the same cache directory may end up causing them stomping over each other and causing crashes.
The docs I'm looking at are in their github repo:
okhttp recipes for regular issues faced
My questions:
I'm not specifically setting any cache or cache control for my client. Just a few timeout values and a connection pool. Will there be any caching done by default if i use a new request and response object for each call i make?
client = new Okhttpclient.Builder()
. connectTimeout (10000,TimeUnit.MILLISECONDS)
.readTimeout(15000,TimeUnit.MILLISECONDS)
. connectionPool (new ConnectionPool ())
.build();
The above client is set as singleton and returned to all calling servlets. The request is created as
Request req = new Request.Builder()
.url(someurl)
.get()
.build();
Response res = client.newCall(req).execute();
If so, will there be issues as the mentioned stomping part. I don't need caching at all as mostly I'm just writing stuff to another server and when I'm reading i do need it to be the current values and not caches one... So do i need to explicitly set the cache-control set to force-network or will my default no specified setting be the same?
EDIT: this is the excerpt from the Response caching part of the docs
To cache responses, you'll need a cache directory that you can read
and write to, and a limit on the cache's size. The cache directory
should be private, and untrusted applications should not be able to
read its contents!
It is an error to have multiple caches accessing the same cache
directory simultaneously. Most applications should call new
OkHttpClient() exactly once, configure it with their cache, and use
that same instance everywhere. Otherwise the two cache instances will
stomp on each other, corrupt the response cache, and possibly crash
your program.
OkHttp cache directory is set for each client instance. What the doc is telling you is that you shouldn't configure multiple clients to use the same cache directory. Caching has to be enabled explicitly, and isn't enabled in the code snippet in your question.
Having configured caching on the client instance, you can control response caching for each request. See Cache.

ssl handshake on every request in multiThreaded client

Architecture is midTier Liberty server that receives http requests and brokers to various back ends, some REST, some just JSON. When I configure for SSL (only thru envVars which is quite cool) ... it appears I get a full handShake w/every request. Additionally, the server side uses a different thread with each request (may be related). This is Liberty so it is multiThreaded. Servlet has static ref to POJO that does all apache httpClient work. Not using HttpClientContext (in this case). Basic flow is at end (struggling w/formatting for post legality)
EnvVars are:
-Djavax.net.ssl.keyStore=/root/lWasServers/certs/zosConnKey.jks
-Djavax.net.ssl.trustStore=/root/lWasServers/certs/zosConnTrust.jks
-Djavax.net.ssl.keyStorePassword=fredpwd
-Dhttp.maxConnections=40
Looked at many similar problems, but again, right now this flow does not use client context. Hoping I'm missing something simple. Code being appended on first response as I continue to struggle here w/FF in RHEL.
private static PoolingHttpClientConnectionManager cm = null ;
private static CloseableHttpClient httpClient = null ;
// ....
cm = new PoolingHttpClientConnectionManager();
cm.setMaxTotal(512);
cm.setDefaultMaxPerRoute(256) ;
httpClient = HttpClients.custom().setConnectionManager(cm).build() ;
// ...
responseBody = httpClient.execute(httpGet, responseHandler);
If a persistent HTTP connection is stateful and is associated with a particular security context or identity, such as SSL key or NTLM user name, HttpClient tries to make sure this connection cannot be accidentally re-used within a different security context or by a different user. Usually the most straight-forward way of letting HttpClient know that requests are logically related and belong to the same session is by executing those requests with the same HttpContext instance. See HttpClient tutorial for details. One can also disable connection state tracking if HttpClient can only be accessed by a single user or within the same security context. Use with caution.
OK, while I'm not exactly an expert at reading the ssl trace, I do believe I have resolved it. I am on a thread but that is controlled by the server. I now pass the HttpSession in and keep a reference to the HttpClientConnection that I now create for each session. I pool these HttpClientConnection objects (rudimentary pooling, basically just get/release). So all calls w/in an http session use the same HttpClientContext. Now it appears that I am NOT handShaking all the time. There may have been a better way to do it, but this does indeed work, I have a few gremlins to look into (socket timeouts in < 1 millisecond?) ... but I'm confident that I'm non longer handShaking with each request (only each time I end up creating a new context) ... so this is all good. Thanks,

Handle request in Google App Engine Task Queue

We are building analytics tool which collect events from web site and mobile apps. We want to process the request with Task queue in Google App engine. When i refer the doc in Google Developers site
Queue queue = QueueFactory.getDefaultQueue();
queue.add(withUrl("/analytics").param("id", String.valueOf(id)));
It has option to give only param. But how do i pass entire HTTP request to task queue and process them?
I assume the main problem here is how to pass all the request parameters and the request body to the task.
Unfortunately there is no simple "relay" or "redirect" method to move your request to the task queue (but it would be nice). You have to use the Queue.add(TaskOptions taskOptions) method to add your task.
The recommended way to instantiate a TaskOptions object is to statically import TaskOptions.Builder.* and invoke a static creation method followed by an instance mutator (if needed).
And use one of the following (or any other payload() methods):
TaskOptions.payload(byte[] payload);
TaskOptions.payload(byte[] payload, String contentType);
TaskOptions.payload(String payload);
to set the content of the request. You can get the payload by reading it from the request.getInputStream().
The request parameters (if they are part of the URL and not the result of a form POST for example) you have to manually copy each with e.g. TaskOptions.param(String name, String value).

Apache HttpComponent Client save cookies to file and use them for future requests (using CookieStore)

Is it possible to save the cookies to a text file and use them in later requests?
Or can we define a text file as CookieStore?
Is there some good explained example?
How long are the cookies saved in programs normally and where? In the memory and for how long? Just for the time the program runs in the VM?
Do we have to fetch each cookie from the local CookieStore iterating through the list and add them manually to the text file and add these cookies later to the CookieStore again?
BasicCookieStore class shipped with HttpClient is Serializable, so its instances could be written to and read from an object stream. If you want a more elegant persistence mechanism, you will have to implement it by fetching individual cookies from the store and writing them to a persistent store.

multiple http requests when processing streaming response using httpcomponents

I'm a newbie to http and Apache's HttpComponents API.
I need to process a streaming response of an http request using Apache's HttpComponents, while there may be additional http requests made on that connection. For example, a POST request is initially made to http://mystreams.net, which later follows with additional requests, while throughout I have to listen and process the streaming response. I need to hold the same initial connection I made.
How can I do that? I was able to create a simple HttpClient and do a simple HttpPost request and then process the response entity that is non-streaming, but how do I hold on to it when it continues to stream data and at the same time make new requests to the same address using the same context (ie cookies)?
Is your streaming data coming back as a single HTTP response? If so, you won't be able to receive other responses on that connection until it is done. But you could take the cookies from that response (while it is still streaming the entity to you) and use them to make other requests on another connection.
HttpEntity entity = httpclient.execute(httpget).getEntity();
InputStream is = entity.getContent()
when calling the stream, use a new Thread, and make subsequent requests in the main thread (or, better, in separate thread for reach)
Also check here

Categories

Resources