i want to write a java servlet that will be called by different users to do httpclient post content to another side via "POST" . i wanted to hear opinion from guru in this case, do my servlet need to use threadpool or something since i want to serve different users at the same time and each user is executing different httpclient post
You should read the HttpClient threading guide because you are in a multi-threaded environment within a servlet container.
Are your out-going POST requests going to be synchronous or asynchronous? That is: will the user-request for which the POST is being performed wait for the POST to complete?
Servlet engines already use separate threads for each request being concurrently handled, so if your outgoing POSTs are meant to be synchronous then you don't need to make your own thread pool. If they're asynchronous, however, you may want to have a producer consumer queue where requests "produce" a command to perform a POST, and a set of worker threads consume (and then perform) these commands.
Related
I've been told during stress testing that I need to reduce my app's Thread usage.
All of our other apps follow a 1 request per page model while I've create a single page app that fires 8 AJAX requests on load.
I'm running on Websphere 8.5.5.1 (Servlet 3.0).
I am making 8 AJAX requests from the Browser one straight after the other to the same Servlet, so they are in Parallel.
The Servlet basically checks the request parameters and calls the corresponding ESB service over http and then returns the response.
Please correct me if anything I say here is rubbish as I'm not 100% on any of it...
My understanding is that when the request hits Websphere it gets a thread from the thread pool and uses that thread to execute the Servlet and holds on to the same thread until the response is returned. I'm also thinking that the thread will block while waiting for a response from the ESB. Is that right?
I'm looking at the following 3 options:
1) The best we could do with Async servlets in Servlet 3.0 is put the request on a queue and return the thread back to the pool. Then one or more threads would go through the queue servicing the requests. So we are just swapping one thread pool for another, but we can limit/control the size of the second one.
If we had Servlet 3.1 (JSR 340) we could use a ReadListener and I'm guessing we could avoid the thread blocking while waiting for the ESB response?
2) Queue the AJAX requets on the Frontend so instead of firing off 8 AJAX requests, I fire 3 and when they finish fire the next one and so on, so you never have more that 3 requests in parallel.
3) Make 1 AJAX Request to the Servlet and let it call all 8 ESB Services and combine the results.
Can you advice me if any of the above solutions are valid and if I even understand them fully and suggest other possible solutions?
Thanks.
Trying to answer point by point:
"Websphere it gets a thread from the thread pool and uses that thread to execute the Servlet and holds on to the same thread until the response is returned." Your understanding is incorrect. This would have been correct if WS was using old IO, but WS version 6.1 onwards use Native IO (NIO) and Asynchronous IO (AIO), which allows a handful of threads to cater thousands of concurrent connections. See this reference. Thus, you should not worry about making concurrent connections from AJAX client, and parallelism is fine.
Having said this, the application must try and avoid doing any blocking operations, that block any worker threads which otherwise are capable of handling multiple concurrent connections. Just think on how you can execute each task async, which would mean that you should be able to do things without blocking yourself. In your case, you are waiting (and blocked) on the responses from ESB. There will be ~8 blocking connections to ESB service. Please review how you are making calls to ESB, and if you can use NIO for these calls. Understand that WS is a client for calls to ESB, and it has to use non-blocking IO for the operations it does on ESB.
Alternatively: Is there a way, that ESB can also behave async, and release the WS request thread? ESB can callback a WS service URL, when its done with the request, and you can then create the AJAX response. That way you would not hold up WS threads.
In your options above, clearly #1, and #3 will not help you. #2 would of course limit the parallel operations to ESB to three. But, I would recommend to see if the calls to ESB can be done in a async manner, and non-blocking manner. That would be ideal.
Update
To make an async and non-blocking calls to an external service from your app server, consider using a library like async-http-client. This uses Netty by default, but other NIO frameworks can be plugged in as well. The README.md of this project has good examples on how to make an HTTP call, and write response on the completion of HTTP request. You can use the onCompletion to write response back to your client.
The concluding statement:
Be a non-blocking server when listening to your clients. So far as possible, make sure that the request threads of the container/server do not block.
Be a non-blocking client when making calls to an external service. This will allow you to make a large number of requests with a handful of threads.
I've been reading about asynchronous servlets in an attempt to see if it fits my use case.
The server will receive long-running requests that take time to read from the db, and process each message.
Here is the basic flow:
Client makes a request to validate some data.
Server receives a request for the long-running process (~1-2 minutes)
Request needs to talk to a db (redis, in my case) to grab 100,000 docs in batches.
Processing occurs for each doc, and if it is valid, it is written to the client. Similarly, upto 25 sample docs are written to the client.
When the full data set is processed (but no more than 25 docs are written), a result is calculated and sent.
Connection is successfully closed.
So there is a db wait, which all similar requests will have, and then the processing for each doc, which happens in the same thread.
Is there any benefit to switching to async servlets versus a regular synchronous servlet in this case?
Additionally, if I do use an async servlet, would there be any benefit in also trying to parallelize the processing of the docs with thread pools? It seems like I won't be getting any additional benefit, but want to clarify this as well.
Thanks for your help!
Is there any benefit to switching to async servlets versus a regular synchronous servlet in this case?
Yes. In fact it is the use case that all manuals tells you "async servlets are useful for this"
Additionally, if I do use an async servlet, would there be any benefit in also trying to parallelize the processing of the docs with thread pools? It seems like I won't be getting any additional benefit, but want to clarify this as well.
Unrelated. Async servlets just releases resources so your server keeps answering petitions even if lots of servlets are waiting for results. It is not related to the speed of answering the asynchronized request.
Any benefit that you get from optimizing your logic will be good for the server and the user, but it would be good either if you are using async servlets or sync servlets.
I plan to create a webservice (in Tomcat) that works with a certain file on the system and make some system calls. The challenge am having is to make sure only 1 request can be processed at a given time and the previous request needs to be pre-empted. I currently have a singleton class that works on the request. And requesting threads somehow wait until the running thread is done. But what is the general design pattern for such problems?
Any ideas how this can be achieved?
Thanks,
V
Since there may be too many requests calling to this service, the synchronous approach may not be achieved. There also may be a chance that some clients waiting so that it is time-out. Then I prefer the asynchronous as
The Service-A receives the request and put it to queue-A or DB table-A, together with generating the ref-id.
The Service-A returns the ref-id to the client for further monitoring.
There is a back-end process monitoring the queue-A or DB table-A and perform the request one-by-one.
After finish, put the result to another queue-B or DB table-B.
The client keep monitoring periodically via another Service-B by using the ref-id, if the request is done.
I hope this may help to achieve your requirement.
Regard,
Charlee Ch.
I would place the queues mentioned by Charlee in the ServletContext. It is initialized when your WebApplication is started.
You can initialize and destroy these queues an the backend process in an implementation of ServletContextListener. Store them with setAttribute on the ServletContext.
Now you're able to access these queues via ServletContext in your Servlet.
I am in the process of writing a web-app that uses multiple web APIs.
For a single request of a single user, my app may need to perform up to 30 HTTP requests to other sites. The site housing the web-app can have hundreds of concurrent users.
I've been looking around trying to figure out which library should I use. I'm looking for a mature project that has detailed documentation and tested code, one that will still be around in years to come. Not sure if something like that exists (!)
Couple of questions :
In a case such as described above, should I be using an asynchronous HTTP client (without threading), or a regular (possibly pooled) HTTP client (with threading)? Asynchronicity relieves my app from using threads, but makes the code more scattered - will the above mentioned number of requests burden my server too much? (it says here that asynchronous is more scalable)
Which library is the common one to use? Is it Apache HttpComponenets HttpClient or its asynch couterpart HttpAsynchClient - which is in Alpha...)? How about jfarcand's AsyncHttpClient?
Okay, let's say I will use threads.
After digging around I realize that spawning threads from within a servlet (in my case - a Struts action), may be a big No No :
related questions:
What is recommended way for spawning threads from a servlet in Tomcat
Need help with java web app design to perform background tasks
Can i spawn a thread from a servlet ?
The way I see it, these are my options:
use my own thread pool (container doesn't manage my threads)
use a WorkManager such as CommonJ (seems like an inactive product)
use a 3rd party scheduler such as Quartz (may be an overkill ... ?)
I would appreciate any recommendations for this specific use case - aggregating lotsa data from different web services (this aggregation is invoked by a single user's single request).
Good question. I would try an asynchronous solution first to see how everything works. The asynchronous solution would be the simplest to implement.
If that doesn't work, try a more threaded model.
I would use HttpClient for making your requests. I've worked with it a lot and use it for any http work that I have to do.
A single thread for each remote http connection, and using a synchronous http client will probably be easier. I would try this approach first, and see if it is fast/scalable enough. For the synchronous approach, apache http client is a good choice.
If a synchronous solution is not good enough, something like netty may be a good fit. It uses NIO so you won't get thousands of threads.
I do not know of any existing software to do this for you that will not be overkill. But you might try splitting things up. That is, separate the fetching of the data of the showing of the result. Since you do not provide no further details on the problem at hand I cannot say for you whether that would be feasible or not.
Basically the idea is to create a service that will fetch those 30 subsequent requests for you and if possible process it into a request. The client of this service is the service that is running on the web. It will receive a request from a user and subsequently put it's own request trough to your data service. When the data service is ready it will return it's response. Either synchronously or asynchronously.
You could program your data service in any language you want, even Java, without being bound to servlets ensuring that fetching the subsequent 30 request and combining them in a response isn't being done by the webserver. This could also enhance the responsiveness of your web server itself.
Nutshell: branch of the ``difficult" tasks to a specialised service where you can transparently handle parallelism.
I would use Jetty and in the servlets I'd use the continuation mechanism to free up the thread while waiting for the third party web request to complete. This will allow maximum concurrency on your server, as you can have a lot more suspended requests than threads.
You can use either continuations or the servlet 3.0 asynchronous API, the end result is the same.
I need to perform an asynchronous task when a RESTful web service endpoint is called. Effectively, the endpoint is asked to perform a body of work with a POST operation. It should immediately return a 200 OK to the caller, spawn a thread and perform it's resource intensive task. On completion, the thread would then POST to a corresponding endpoint on the caller (another REST server) indicating success (passing a token that represents the initial transaction request).
What are the best practice approaches for performing asynchronous actions inside a servlet that I should be aware of?
Servlet 3.0 has support for asynchronous operations. Tomcat 7.0 is already stable, so you can get it and try the new features.
If you don't need to output data continously, but to simply start a background process, then you can use any asynchronous mechanism available:
Executors framework
new Thread(new Runnable())
#Async (spring/ejb)
Aside from the complexities of async coding in Java, another "best practice" in RESTful web services is to use HTTP status codes to describe your server's response as accurately as possible. Unless you have a compelling reason to stick with 200 (ie a client which you can't change expects this), you should return HTTP 202:
202 Accepted
The request has been accepted for processing, but the processing has not been completed.
The only advice for your scenario would be to use thread pool rather than creating a new thread per request. In Java it is very easy, just create pool once during application startup (look at Executors class) and submit new tasks to it each time you need to perform some asynchronous operation. For your scenario this tasks will perform resource intensive operations and second REST call from within a different thread, long after the original request was served with 200.
In a Java EE container, the recommended way is to use the WorkManager API (JSR-237). Check this article for an overview.
A J2EE container can serve multiple
users at the same time (in parallel)
by managing a pool of threads, but for
various reasons opening independent
threads within a single J2EE container
is not recommended. Some containers'
security managers will not allow user
programs to open threads. Moreover, if
some containers allowed opening
threads, then they would not manage
those threads and therefore no
container-managed services would be
available for the threads.