I have a JAX-RS/Jersey Rest API which gets a request and needs to do an additional job in a separate thread but I am not sure whether it would be advisable to use a threadpool or not. I expect a lot of requests to this API (a few thousands a day) but I only have a single additional job in the background.
Would it be bad to just create a new Thread each time like this? Any advice would be appreciated. I have not used a ThreadPool before.
#Get
#Path("/myAPI")
public Response myCall() {
// call load in the background
load();
...
// do main job here
mainJob();
...
}
private void load() {
new Thread(new Runnable() {
#Override
public void run() {
doSomethingInTheBackground();
}
}).start();
}
Edit:
Just to clarify. I only need a single additional job to run in the background. This job will call another API to log some info and that's it. But it has to do this for every request and I do not need to wait for a response. That's why I thought of just doing this in a new background thread.
Edit2:
So this is what I came up with now. Could anyone please tell me if this seems OK (it works locally) and if I need to shutdown the executor (see my comment in the code)?
// Configuration class
#Bean (name = "executorService")
public ExecutorService executorService() {
return Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors() + 1);
}
// Some other class
#Qualifier("executorService")
#Autowired
private ExecutorService executorService;
....
private void load() {
executorService.submit(new Runnable() {
#Override
public void run() {
doSomethingInTheBackground();
}
});
// If I enable this I will get a RejectedExecutionException
// for a next request.
// executorService.shutdown();
}
Threadpool is a good way of dealing with this for two reasons:
1) you will reuse existing threads in the pool, sort of less overhead
2) more importantly, your system will not get bog down if system goes under attack and some party tries to start zillions of sessions at once because of size of the pool will be preset.
Use of threadpools is not complicated at all. See here more about threadpools. And also take a look at oracle documentation.
It sounds to me you don't need to create multiple threads at all.
(although I might be wrong, I don't know the specifics of your task).
Could you perhaps create exactly 1 thread that does background work, and give that thread a LinkedBlockingQueue to store the parameters of the doSomethingInTheBackground call?
This solution wouldn't work if it is of the utmost importance that the background task starts right away, even when the server is under heavy load. But for example for my most recent task (retrieve text externally, return them to the API caller, then delayed-add the text to the SOLR layer) this was a perfectly fine solution.
I suggest using neither of the approaches you mention, but to use a JMS queue. You can easily embed an ActiveMQ instance in your application. First create one or more separate consumer threads in the background to pick up jobs from the queue.
Then when a request is received just push a message with the job details on the JMS queue. This is a much better architecture and more scalable than fiddling with low level threads or thread pools.
See also: this answer and the activeMQ site.
Related
We have a Spring application, I want to add a service that will handle 10Ks IDs with multiple threads but will be as background process without impact production realtime.
Service will update database and send external providers requests.
I don't want service to impact/effect production performance/timing, I want to execute operation on each ID in a low priority
I read previous post about setting priority in Executer, but I want low priority to all other threads that can be outside this specific Executer scope.
Is answer using ThreadPoolExecutor more relevant to my case?
ThreadPoolExecutor threadPool = new ThreadPoolExecutor(1, numOfWorkerThreads, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>());
threadPool.setThreadFactory(new OpJobThreadFactory(Thread.NORM_PRIORITY-2));
public final static class OpJobThreadFactory implements ThreadFactory {
private int priority;
public OpJobThreadFactory(int priority) {
this(priority, true);
}
#Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r, namePrefix + threadNumber.getAndIncrement());
t.setDaemon(daemon);
t.setPriority(priority);
}
}
maybe even use Thread.MIN_PRIORITY
Or I'm fine with using Executors.newCachedThreadPool()
Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available. These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks.
Also should I use Spring bean? because I need to create pool on demand/request so it seems not needed/wrong
EDIT
Should I use Spring Actuator to get this task or other monitoring tool?
Spring Boot Actuator module helps you monitor and manage your Spring Boot application by providing production-ready features like health check-up, auditing, metrics gathering, HTTP tracing etc. All of these features can be accessed over JMX or HTTP endpoints.
I would like to throw some light on the question
what is a thread priority? According to java SE Docs
Every thread has a priority. Threads with higher priority are executed in preference to threads with lower priority.
Even though you create threads with priority it does not completely guarantee that threads with lower priority get executed first you may have to block the thread with lower priority until other threads are executed
For small java programs you can handle the thread execution by yourself but for larger programs, it's recommended either you use the Executor Framework from vanilla Java which is from the package java.util.concurrent or use the spring TaskExecutor.By using both frameworks you can execute tasks asynchronously that is executing them in the background as part of the main task.
Impact on Production:
The main task, for example, will be a call to your rest endpoint i.e /account and on calling the account endpoint you want to send welcome emails to customers which are a third party API call which can be executed asynchronously either using Executor Framework or Spring TaskExecutor on executing them asynchronously i.e as background process they will not have an impact on the current API but it will surely have an impact on the production server since you are running the threads within the same JVM and they share common memory. if there are a number of threads created and not destroyed .you server will surely go down.
So using Executor Framework or Spring TaskExecutor does not guarantee you that it will not affect your current production it will surely increase the performance of the rest API that is called. since it's executed asynchronously and on your other questions
can i use Executors.newCachedThreadPool()
yes if you have a number of the short-lived task such as updating a single column in a database or triggering a rest endpoint only once and it's not suitable for bulk loading or executing some backend job which updates 10000 records because it will create larger number of threads for each task and you will surely face memory problems.
Also, should I use Spring bean? because I need to create a pool on-demand/request so it seems not needed/wrong
yes you can ThreadPoolTaskExecutor use https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/concurrent/ThreadPoolTaskExecutor.html
as per the docs Setting "queueCapacity" to 0 mimics Executors.newCachedThreadPool() .so you can use either Spring TaskExecutor or use Executor Framework from vanilla Java I personally recommend using Spring TaskExecutor which has more feature for a good start on using Spring you can refer the tutorial https://egkatzioura.com/2017/10/25/spring-and-async/ which is a good start
final verdict: if you are looking to execute the task only asynchronous or a background process you can use either use Executor Framework from java or Spring TaskExecutor but both will have an impact on production since they use the same JVM .if you do not want to impact production at all then I recommend creating separate spring boot app on a different server and make the database calls or service call from the new app and expose it as a rest endpoint and invoke these endpoints asynchronously from your main app using Spring Task Executor.
Spring Task Executor: https://egkatzioura.com/2017/10/25/spring-and-async/
Java Executor Framework : https://stackabuse.com/concurrency-in-java-the-executor-framework/
for using threads with low priority :
https://medium.com/#daniyaryeralin/priority-based-thread-pooling-in-spring-framework-d74b91b51dcb
Sorry if the answer is too long :)
Here there is a nice tutorial about priority based task execution in Spring. Maybe this may help you in some ways.
This is a method of creating a configurable ¨heap¨ of task and always keep your main task in the top of the heap.
To sum up this process you should create a custom Task Executor. Firstly you need to create a ThreadPoolTaskExecutor bean with one method being overidden. The properties that should be modified are: CorePoolSize(initial number of threads), QueueCapacity(the number of threads waiting in the queue), and MaxPoolSize(maximum number of threads). With these parameters you can configure your applications limitations in order for this service not to impact the production performance.
#Bean("CustomTaskExecutor")
public TaskExecutor threadPoolTaskExecutor(
#Value("${spring.async.core-pool-size}") int corePoolSize,
#Value("${spring.async.max-pool-size}") int maxPoolSize,
#Value("${spring.async.queue-capacity}") int queueCapacity) {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor() {
#Override
protected BlockingQueue<Runnable> createQueue(int queueCapacity) {
return new PriorityBlockingQueue<Runnable>(queueCapacity);
}
};
executor.setCorePoolSize(corePoolSize);
executor.setMaxPoolSize(maxPoolSize);
executor.setQueueCapacity(queueCapacity);
return executor;
}
After that,you need to make tasks with priorities that the task executor can
understand. For that we would need to create two classes:
1) A custom class that implements Runnable interface that will be running the task
2) A wrapper class that extends FutureTask and implementsComparable interface, so that the task executor could understand the priority picking logic of the tasks
public class Task implements Runnable {
private Consumer<Job> jobConsumer;
private Job job;
public Job getJob() {
return this.job;
}
public Task(Consumer<Job> jobConsumer, Job job) {
this.jobConsumer = jobConsumer;
this.job = job;
}
#Override
public void run() {
this.jobConsumer.accept(job);
}
}
Then you have the FutureCustomTask class:
public class FutureCustomTask extends FutureTask<FutureCustomTask> implements Comparable<FutureCustomTask> {
private Task task;
public FutureCustomTask(Task task) {
super(task, null);
this.task = task;
}
#Override
public int compareTo(FutureCustomTask o) {
return task.getJob().getPriority().compareTo(o.task.getJob().getPriority());
}
}
For the execution the TaskExecutor needs to be Autowired.
Then, you can create your Task object, wrap it inside FutureCustomTask, and pass it to TaskExecutor.The code should look like this:
#Autowired
private TaskExecutor taskExecutor;
#Autowired
private JobBusiness jobBusiness;
...
Task task = new Task(jobBusiness::performSomethingOn, job);
taskExecutor.execute(new FutureCustomTask(task));
I have one functionality in online application. I need to mail receipt to customer after generate receipt. My problem is mail function takes more time nearly 20 to 30 seconds, customer could not wait for long time during online transaction.
So i have used java ExecutorService to run independently mail service [sendMail] and return response PAGE to customer either mail sent or not.
Is it right to use ExecutorService in online application [Http request & Response]. Below is my code. Kindly advice.
#RequestMapping(value="/generateReceipt",method=RequestMethod.GET)
public #ResponseBody ReceiptBean generateReceipt(HttpServletRequest httpRequest,HttpServletResponse httpResponse) {
// Other codes here
...
...
I need run below line independently, since it takes more time. so commeneted and wrote executor service
//mailService.sendMail(httpRequest, httpResponse, receiptBean);
java.util.concurrent.ExecutorService executorService = java.util.concurrent.Executors.newFixedThreadPool(10);
executorService.execute(new Runnable() {
ReceiptBean receiptBean1;
public void run() {
mailService.sendMail(httpRequest, httpResponse, receiptBean);
}
public Runnable init(ReceiptBean receiptBean) {
this.receiptBean = receiptBean1;
return(this);
}
}.init(receiptBean));
executorService.shutdown();
return receiptBean;
}
You can do that, although I wouldn't expect this code in a controller class but in a separate on (Separation of Concerns and all).
However, since you seem to be using Spring, you might as well use their scheduling framework.
It is fine to use Executor Service to make an asynchronous mail sending request, but you should try to follow SOLID principles in your design. Let the service layer take care of running the executor task.
https://en.wikipedia.org/wiki/SOLID
I agree with both #daniu and #Ankur regarding the separation of concerns u should follow. So just create a dedicated service like "EmailService" and inject it where needed.
Moreover you are leveraging the Spring framework and u can take advantage of its Async feature.
If u prefer to write your own async code then I'll suggest to use maybe a CompletableFuture instead of the ExecutorService to better handling failure (maybe u want store messages not sent into a queue for achieving retry feature or some other behaviour).
Is it safe for my Java (Tomcat 8) web server to spawn threads in response to a HTTP request? I'm seeing posts and forums where some people say it's absolutely fine, and others say not to do it.
My use case would be something like this:
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
...
...
final MyResult res = new MyResult();
Thread first = new Thread(new Runnable() {
#Override
public void run() {
// put this into res
}
});
Thread second = new Thread(new Runnable() {
#Override
public void run() {
// put that into res
}
});
first.start();
second.start();
first.join(10000);
second.join(10000);
// return res
}
When I say safe, I mean is there anything inherently dangerous about what I'm proposing with regards to the stability of the web server. As #Burrman points out, a thread pool is good idea here, and I will do that. If I am using a thread pool, is there then any other potential issues with the servlet container that I should be concerned about or need to address?
I suppose what I'm thinking about is, for example, JDBC connections. I believe it's recommended to set that up using JNDI resource etc. and configuring that with Tomcat config. Is anything like that necessary or recommended for spawning arbitrary threads like in my example?
First, it looks you're modifying the result object in both threads. This is not thread safe because what the first and second threads do might not be visible to each other or to the thread the servlet is running on. See this article for more info.
Second, if you are modifying the response in these other threads, no, this will not be safe. Once you exit the doGet method, you should consider the response sent. In your example, there's a chance the response will get sent back to the client before those two threads have run.
Suppose the MyResult result affects the response object (you're either adding the result to the response, it's affecting the response code, etc). There are a few ways to handle this.
Use ExecutorService and Future:
public void doGet(HttpServletRequest request, HttpServletResponse response) {
// Creating a new ExecutorService for illustrative purposes.
// As mentioned in comments, it is better to create a global
// instance of ExecutorService and use it in all servlets.
ExecutorService executor = Executors.newFixedThreadPool(2);
Future<Result1> f1 = executor.submit(new Callable<Result1>() {
#Override
public Result1 call() throws Exception {
// do expensive stuff here.
return result;
}
});
Future<Result2> f2 = executor.submit(new Callable<Result2>() {
#Override
public Result2 call() throws Exception {
// do expensive stuff here.
return result;
}
});
// shutdown allows the executor to clean up its threads.
// Also prevents more Callables/Runnables from being submitted.
executor.shutdown();
// The call to .get() will block until the executor has
// completed executing the Callable.
Result1 r1 = f1.get();
Result2 r2 = f2.get();
MyResult result = new MyResult();
// add r1 and r2 to result.
// modify response based on result
}
A more advanced technique is Asynchronous Processing. Using async processing is a good idea if your requests take a long time to process. It does not improve the latency of any one request, but it does allow Tomcat to handle more requests at any given point in time.
A simple example would be:
#WebServlet(urlPatterns={"/asyncservlet"}, asyncSupported=true)
// Rather than #WebServlet, you can also specify these parameters in web.xml
public class AsyncServlet extends HttpServlet {
#Override
public void doGet(HttpServletRequest request, HttpServletResponse response) {
response.setContentType("text/html;charset=UTF-8");
final AsyncContext acontext = request.startAsync();
acontext.start(new Runnable() {
public void run() {
// perform time consuming steps here.
acontext.complete();
}
}
Also, besides excellent answer of kuporific I really advise you to think if your result computation could be expressed in terms of map / filter / group operations on lists or maps, because in 90% of cases it's possible to do so.
And if that's the case I would really advise you to use Java 8 Stream.parallelStream functionality as outlined in this official tutorial
Please ask separate question if you are interested if/how it's possible to express your computation in that way
Also, answering your initial question - it's perfectly fine to spawn threads to parallelise your computation anywhere (including servlets), however I would really advise to measure performance before and after optimization mainly because of reasons described in this superb article
In principle, yes. But you have to keep an eye on the total number of threads you are spawning to make sure you're not using too many resources.
Using a thread pool can help keep your number of threads under control.
In my opinion
this idea has not sense if thread should give effect in same request (cit. in order to satisfy a request)
may have sense (if done properly) when effect of thread (running / ended) will be visible in next requests (especially via AJAX). Most servlet frameworks have way how to do "in orthodox way" long operations.
To be clear: this is not normal way to do typical everyday AJAX tasks (must divide design to "main" request and next children ajax requests, using your framework), but AJAX can present state of background long thread.
In my conception Thread or equivalent can be started from few/rare request, far below limits etc, but if it is done from every request, it is very bad design.
JEE server stores some information in thread local variables. E.g. security context for JAAS, JTA transaction, ... New plain java thread has no access to such information. AsyncContext and JEE ExecuterService are integrated into server and can transparently spread request state to managed threads.
I am trying to get the FacesContext by calling FacesContext.getCurrentInstance() in the run() method of a Runnable class, but it returns null.
public class Task implements Runnable {
#Override
public void run() {
FacesContext context = FacesContext.getCurrentInstance(); // null!
// ...
}
}
How is this caused and how can I solve it?
The FacesContext is stored as a ThreadLocal variable in the thread responsible for the HTTP request which invoked the FacesServlet, the one responsible for creating the FacesContext. This thread usually goes through the JSF managed bean methods only. The FacesContext is not available in other threads spawned by that thread.
You should actually also not have the need for it in other threads. Moreover, when your thread starts and runs independently, the underlying HTTP request will immediately continue processing the HTTP response and then disappear. You won't be able to do something with the HTTP response anyway.
You need to solve your problem differently. Ask yourself: what do you need it for? To obtain some information? Just pass that information to the Runnable during its construction instead.
The below example assumes that you'd like to access some session scoped object in the thread.
public class Task implements Runnable {
private Work work;
public Task(Work work) {
this.work = work;
}
#Override
public void run() {
// Just use work.
}
}
Work work = (Work) FacesContext.getCurrentInstance().getExternalContext().getSessionMap().get("work");
Task task = new Task(work);
// ...
If you however ultimately need to notify the client e.g. that the thread's work is finished, then you should be looking for a different solution than e.g. adding a faces message or so. The answer is to use "push". This can be achieved with SSE or websockets. A concrete websockets example can be found in this related question: Real time updates from database using JSF/Java EE. In case you happen to use PrimeFaces, look at
<p:push>. In case you happen to use OmniFaces, look at <o:socket>.
Unrelated to the concrete problem, manually creating Runnables and manually spawning threads in a Java EE web application is alarming. Head to the following Q&A to learn about all caveats and how it should actually be done:
Spawning threads in a JSF managed bean for scheduled tasks using a timer
Is it safe to start a new thread in a JSF managed bean?
In simple description, I have a servlet and response time is long so I decided to divide it into two parts, one just composes a response to client, and second let's say performs some business logic and stores result in DB. To decrease response time I execute business logic asynchronously using ThreadPoolExecutor in combination with ArrayBlockingQueue. Using ArrayBlockingQueue I can ensure original FIFO ordering if requests were sequential for the same client. This is important prerequisite.
Here is a snippet:
Servlet
public class HelloServlet extends HttpServlet {
AsyncExecutor exe = new AsyncExecutor();
protected void doGet(HttpServletRequest req,
HttpServletResponse resp) throws ServletException, IOException {
PrintWriter w = resp.getWriter();
exe.executeAsync(exe.new Task(this));
w.print("HELLO TO CLIENT");
}
protected void someBusinessMethod(){
// long time execution here
}
}
and Executor
public class AsyncExecutor {
static final BlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(10, true);
static final Executor executor = new ThreadPoolExecutor(3, 5, 20L, TimeUnit.SECONDS, queue);
public void executeAsync(Task t){
boolean isTaskAccepted = false;
while(!isTaskAccepted){
try {
executor.execute(t);
isTaskAccepted = true;
} catch (RejectedExecutionException e){
}
}
}
class Task implements Runnable{
private HelloServlet servlet;
Task(HelloServlet servlet){
this.servlet = servlet;
}
#Override
public void run() {
// just call back to servlet's business method
servlet.someBusinessMethod();
}
}
}
This implementation works fine if I deploy it only to one Tomcat node, since I have only one ArrayBlockingQueue in application. But if I have several nodes and load balancer in front then I can not guarantee FIFO ordering of requests for async execution for the same client since I have already several Queues.
My question is, how it is possible to guarantee the same order of requests to be executed asynchronously for the same client in clustered (multi node) deployment? I think ActiveMQ probably a solution (not preferable for me), or load balancer configuration, or can it be implemented in code?
Hope some of these ideas help.
Thanks Sam for you prompt suggestions.
In the first post I described a problem in very simplified way so to clarify it better let's say I have a legacy web app deployed to Tomcat and it serves some Licensing Model(old one). Then we got a new Licensing Model (this is a GlassFish app) and we need to use it alongside with old one to be in sync. For the end user such integration must be transparent and not intrusive. So user request is served like this.
caller send a request (create subscription for example)
execute business logic of the the new licensing model
execute business logic of the the old licensing model
despite the result of the p.3 return response of p.2 in format of old licensing model back to caller
(optional) handle failure of p.3 if any
This was implemented with Aspect which intercepts requests of p.1 and executes the rest of stuff sequentially. And as I said in previous post p.3 execution time can be long that's why I want to make it asynchronous. Let's have a look at snippet of Aspect (instead of Servlet from the first post).
#Aspect #Component public class MyAspect {
#Autowired
private ApplicationContext ctx;
#Autowired
private AsyncExecutor asyncExecutor;
#Around("#annotation(executeOpi)")
public Object around(ProceedingJoinPoint jp, ExecuteOpi executeOpi) throws Throwable {
LegacyMapper newModelExecutor = ctx.getBean(executeOpi.legacyMapper());
// executes a new model and then return result in the format of old model
Object result = newModelExecutor.executeNewModelLogic(joinPoint.getArgs());
// executes old model logic asynchronously
asyncExecutor.executeAsync(asyncExecutor.new Task(this, jp)
return object
}
public void executeOldModelLogic(ProceedingJoinPoint jp) throws Throwable{
// long time execution here
jp.proceed();
}
}
With this implementation as in the first post, I can guarantee a FIFO order of executeOldModelLogic methods, if requests come to the same Tomcat node. But with multi-node deployment and round robin LB in front I can end-up with such a case when for the same caller "update subscription in old model" can come first to ArrayBlockingQueue than "create subscription in old model", which of course a bad logical bug.
And as for points you suggested:
p1, p2 and p4: I probably can't use it as a solution since I don't have a state of object as such. You see that I pass to Runnable task a references of Aspect and JoinPoint to make a call back of executeOldModelLogic from Runnable to Aspect
p3 Don't know about this might be worthwhile to investigate
p5 This is a direction I want go for further investigation, I have a gut feeling it is only way to solve my problem in the given conditions.
There are some solutions that come to mind off hand.
Use the database: post the jobs to be run in a database table, have a secondary server process run the jobs, and place results in an output table. Then when users call back to the web page, it can pick up any results waiting for them from the output table.
Use a JMS service: this is a pretty lightweight messaging service, which would integrate with your Tomcat application reasonably well. The downside here is that you have to run another server side component, and build the integration layer with your app. But that's not a big disadvantage.
Switch to a full J2EE container (Java App Server): and use an EJB Singleton. I have to admit, I don't have any experience with running a Singleton across separate server instances, but I believe that some of them may be able to handle it.
Use EHCache or some other distributed cache: I built a Queue wrapper around EHCache to enable it to be used like a FIFO queue, and it also has RMI (or JMS) replication, so multiple nodes will see the same data.
Use the Load Balancer: If your load balancer supports session level balancing, then all requests for a single user session can be directed to the same node. In a big web environment where I worked we were unable to share user session state across multiple servers, so we set up load balancing to ensure that the user's session always was directed to same web server.
Hope some of these ideas help.