I faced a requirement to process a large amount of data for every request and I'm looking for some clues. I came up with three ideas:
1) start a new thread from a service
2) use request scope for a service
3) use #Async
I realized I don't fully understand the basics:
If all Spring beans are Singletons, what happens in there is a
time-consuming operation in #Service? Will other users have to wait for #Service to complete?
If expected service behavior to call DB, fetch and process results
and do other no trivial operations, shouldn't it be scoped request by
default?
What #Async has to do with all this? Is it equivalent to AJAX call?
I would really appreciate some explanation on how to perform heavy calculations for every request in Spring Boot.
No, singleton bean doesn't mean other threads have to wait for a thread to finish executing some service offered by that bean.
I see that you mentioned about up to 1000 users requesting concurrent calculation. In that case, #Async may not be a good choice.
You can start with a simple ThreadPoolExecutor with some decent value for maximum amount of concurrent thread (16), and work queue size (10000). Since you claim that this is kind of 'start and forget' action, I assume it's OK to have an amount of calculation requests waiting in the work queue until some idle thread available.
Next, do some load test with the starting solution to estimate again the capability of your service. In case your single service doesn't have enough capability to handle such huge amount of heavy calculation requests in time, you would need to think about having dedicated worker service instances where actual calculations are done, while your server service only play as "request dispatcher".
This is not an actual answer but since I don't have enough point to add answer, consider this as some starting point for your problem.
Related
In my Jetty service, I have an endpoint which triggers a number of threads to be run. I'm currently using a ThreadExecutorPool to execute my Runnables. But I want to be able to know how many jobs are remaining for each individual request to the endpoint and allow for multiple concurrent requests.
My thought is to do something like have a single queue for each request and have some kind of executor which just pulls from each queue in a round robin manner. I'm wondering if something like this exists that works out of the box.
Note that I don't care about execution order. What I do care about is that each request (i.e. session) is given equal time and that, for each session, I can find out how many threads have not yet been started/completed.
So I guess a MultiQueueExecutor or something like that?
But I want to be able to know how many jobs are remaining for each individual request to the endpoint and allow for multiple concurrent requests.
I would just use your Futures appropriately in combination with a ExecutionCompletionService. If you use an ECS your web requests will be notified as soon as each of the tasks finishes so they will always know how many tasks are left to run.
What I do care about is that each request (i.e. session) is given equal time and that, for each session, I can find out how many threads have not yet been started/completed.
Well the "equal time" is not guaranteed and there is are race conditions around when the jobs are submitted to the thread-pool given the FIFO nature of it. Whether or not this solution works depends highly on what you will do with the information in question.
As I found, controllers in String are singletones Are Spring MVC Controllers Singletons?
The question is, how Spring handles multiple long time consuming requests, to the same mapping? For example, when we want to return a model which require long calculations or connection to other server- and there are a lot of users which are sending request to the same url?
Async threads I assume- are not a solution, because method need to end before next request will be maintained? Or not..?
Requests are handled using a thread-pool (Container-managed), so each request has an independent context, it does not matter whether if the Controller is Singleton or not.
One important thing is that Singleton instances MUST NOT share state between requests to avoid unexpected behaviours or race conditions.
The thread-pool capacity will define the number of requests the server could handle in a sync model.
If you want an async approach you coud use many options like:
Having a independent thread pool that processes tasks from container threads, or
Use a queue to push tasks and use an scheduler process tasks, or
Use Websockets to make requests and use (1) or (2) for processing and then receive the notification when done.
There is a servlet filter in my application from which I need to invoke a web service which takes some time to return response and then store the response in session to be used later. I want that by the time this time-taking process takes place my filter should proceed and should continue invoking the other filters too so that the performance is not affected.
So this is what I am thinking of doing inside doFilter(). Create a different thread for this purpose.
log.debug("start filter");
CustomThread ct=new CustomThread();
ct.start(); //invoke web service in run()
log.debug("continuing with filter");
Considering the fact that more that 1000 users will be hitting the application, will this approach work properly. Will this condition fail for some scenario?
Please suggest if I need to take a different route.
The main problem is that you start a new thread each time. This is time consuming, and it can bring the server to its knees if you get many concurrent requests, because you don't have any limit on the number of spawned threads.
I would use a ThreadPoolExecutor instead, which would solve those two problems.
You should use an Executor of some sort rather than worrying about thread management yourself. The Executors class provides a variety of simple ways to create executor instances.
I need to wait for a condition in a Spring MVC request handler while I call a third party service to update some entities for a user.
The wait averages about 2 seconds.
I'm calling Thread.sleep to allow the remote call to complete and for the entities to be updated in the database:
Thread.currentThread().sleep(2000);
After this, I retrieve the updated models from the database and display the view.
However, what will be the effect on parallel requests that arrive for processing at this controller/request handler?
Will parallel requests also experience a wait?
Or will they be spawned off into separate threads and so not be affected by the delay experienced by the current request?
What are doing may work sometimes, but it is not a reliable solution.
The Java Future interface, along with a configured ExecutorService allows you to begin some operation and have one or more threads wait until the result is ready (or optionally until a certain amount of time has passed).
You can find documentation for it here:
http://download.oracle.com/javase/6/docs/api/java/util/concurrent/Future.html
I am working on a servlet that can take a few hours to complete the request. However, the client calling the servlet is only interested in knowing whether the request has been received by the servlet or not. The client doesn't want to wait hours before it gets any kind of response from the servlet. Also since calling the servlet is a blocking call, the client cannot proceed until it receives the response from the servlet.
To avoid this, I am thinking of actually launching a new thread in the servlet code. The thread launched by the servlet will do the time consuming processing allowing the servlet to return a response to the client very quickly. But I am not sure if this an acceptable way of working around the blocking nature of servlet calls. I have looked into NIO but it seems like it is not something that is guaranteed to work in any servlet container as the servlet container has be NIO based also.
What you need is a job scheduler because they give assurance that a job will be finished, even in case a server is restarted.
Take a look at java OSS job schedulers, most notably Quartz.
Your solution is correct, but creating threads in enterprise applications is considered a bad practice. Better use a thread pool or JMS queue.
You have to take into account what should happen server goes down during processing, how to react when multiple requests (think: hundreds or even thousands) occur at the same time, etc. So you have chosen the right direction, but it is a bit more complicated.
A thread isn't bad but I recommend throwing this off to an executor pool as a task. Better yet a long running work manager. It's not a bad practice to return quickly like you plan. I would recommend providing some sort of user feedback indicating where the user can find information about the long running job. So:
Create a job representing the work task with a unique ID
Send the job to your background handler object (that contains an executor)
Build a url for the unique job id.
Return a page describing where they can get the result
The page with the result will have to coordinate with this background job manager. While it's computing you can have this page describe the progress. When its done the page can display the results of the long running job.