I have three main classes: Controller, Service1, Service2
Controller: Simple controller that retrieves input (list) and passes it to Service1
Service1: Receives input from controller procceses and passes it to Service2
Service2: Receives processed input from Service 1 and sends a POST request to an external service
These three are all annotated with #Component and based from what I was reading, Spring beans, by default, are singletons unless specified to be prototype which creates a new instance every time the bean is utilized. It is important to note that these are all stateless.
The main flow works like this:
Controller -> Service 1 -> Service 2.
It's simple as is and works well synchronously. But I want asynchronous, fire and forget, behavior for these three classes.
My main concern is that Service1 can be a possible bottleneck since it's doing a lot of processing that takes (4-5 seconds). I mean, sure, the controller can spawn a lot of CompleteableFutures but since Service1 is a singleton, a single thread locks the whole method until it finishes which results to somewhat synchronous behavior.
My question is would setting the scope to 'prototype' for Service1 solve this issue?
since Service1 is a singleton, a single thread locks the whole method until it finishes
No, not at all. That would only be true if the method was synchronized. But there's no reason to make it synchronized, since the object is stateless. Several threads execute the methods of a given object concurrently. That's the whole point of threads.
Note that
prototype which creates a new instance every time the bean is utilized
is also wrong. A new instance is created every time a bean is requested to the application context. So if you have two controllers both using a prototype bean Foo, bith will have their own instance of Foo. But there will only be 2 instances, used and shared by all threads executing the methods of Foo. Read the documentation, which explains it.
If you really want your controller to send back the response before the processing is finished, then use Async, as described in the documentation.
Related
Apologies for the long question..
I'm fairly new to Spring and don't understand the inner working fully yet.
So, my current java project has Spring 4.x code written way back in 2015 that uses ThreadLocal variable to store some user permission data.
The flow starts as a REST call in a REST controller which then calls the backend code and checks for user permissions from the DB.
There is a #Repository class that has a static instance of ThreadLocal where this user permission is stored. The ThreadLocal variable is updated by the calling thread.
So, if the thread finds data in the ThreadLocal instance already present for it, it just reads that data from the ThreadLocal variable and works away. If not, it goes to DB tables and fetches new permission data and also updates the ThreadLocal variable.
So my understanding is that ThreadLocal variable was used as these user permissions are needed multiple times within the same REST Call. So the idea was for a given REST request since the thread is the same, it needn't fetch user permissions from DB and instead can refer to its entry in the ThreadLocal variable within the same REST request.
Now, this seems to work fine in Spring 4.3.29.RELEASE as every REST call was being serviced by a different thread.(I printed Thread IDs to confirm.)
Spring 4.x ThreadStack up to Controller method call:
com.xxx.myRESTController.getDoc(MyRESTController.java),
org.springframework.web.context.request.async.WebAsyncManager$5.run(WebAsyncManager.java:332),
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511),
java.util.concurrent.FutureTask.run(FutureTask.java:266),
java.lang.Thread.run(Thread.java:748)]
However, when I upgraded to Spring 5.2.15.RELEASE this breaks when calling different REST endpoints that try to fetch user permissions from the backend.
On printing the Stacktrace in the backend, I see there is a ThreadPoolExecutor being used in Spring 5.x.
Spring 5.x ThreadStack:
com.xxx.myRESTController.getDoc(MyRESTController.java),
org.springframework.web.context.request.async.WebAsyncManager.lambda$startCallableProcessing$4(WebAsyncManager.java:337),
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511),
java.util.concurrent.FutureTask.run(FutureTask.java:266),
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149),
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624),
java.lang.Thread.run(Thread.java:748)]
So in Spring 5.x, it looks like the same thread is being put back in the ThreadPool and later gets called for multiple different REST calls.
When this thread looks up the ThreadLocal instance, it finds stale data stored by it for an earlier unrelated REST call. So quite a few of my test cases fail due to stale data permissions being read by it.
I read that calling ThreadLocal's remove() clears the calling thread's entry from the variable (which wasn't implemented at the time).
I wanted to do this in a generic way so that all REST calls call the remove() before the REST Response is sent back.
Now, in order to clear the ThreadLocal entry, I tried
writing an Interceptor by implementing HandlerInterceptor but this didn't work.
I also wrote another Interceptor extending HandlerInterceptorAdapter and calling ThreadLocal's remove() in its afterCompletion().
I then tried implementing ServletRequestListener and called the ThreadLocal's remove() from its requestDestroyed() method.
In addition, I implemented a Filter and called remove() in doFilter() method.
All these 4 implementations failed cos when I printed the Thread IDs in their methods they were the exact same as each other, but different to the Thread ID being printed in RestController method.
So, the Thread calling the REST endpoint is a different thread from those being called by the above 4 classes. So the remove() call in the above classes never clears anything from ThreadLocal variable.
Can someone please provide some pointers on how to clear the ThreadLocal entry for a given thread in a generic way in Spring?
As you noticed, both the HandlerInterceptor and the ServletRequestListener are executed in the original servlet container thread, where the request is received. Since you are doing asynchronous processing, you need a CallableProcessingInterceptor.
Its preProcess and postProcess methods are executed on the thread where asynchronous processing will take place.
Therefore you need something like this:
WebAsyncUtils.getAsyncManager(request)//
.registerCallableInterceptor("some_unique_key", new CallableProcessingInterceptor() {
#Override
public <T> void postProcess(NativeWebRequest request, Callable<T> task,
Object concurrentResult) throws Exception {
// remove the ThreadLocal
}
});
in a method that has access to the ServletRequest and executes in the original servlet container thread, e.g. in a HandlerInterceptor#preHandle method.
Remark: Instead of registering your own ThreadLocal, you can use Spring's RequestAttributes. Use the static method:
RequestContextHolder.currentRequestAttributes()
to retrieve the current instance. Under the hood a ThreadLocal is used, but Spring takes care of setting it and removing it on every thread where the processing of your request takes place (asynchronous processing included).
I'm using Spring 4 and I've noticed an odd behaviour... if I'm calling an async method multiple times from a normal instance method then they are all called in different threads and finish at random times. But if I call multiple times an async method from another async method then they finish in order. I have something like this:
#Async
public void nonAsyncMethod() {
for (int i = 0; i < 30; i++) {
asyncMethod();
}
}
#Async
public void asyncMethod() {
... something here
}
I'm using the default async executor. Should I use a different one? However this executor do not reuse any threads and starts another one every time so it should be fine... Can it be just a coincidence? But I've tried like more than 10 times and if I revert back to non-async for the first method then they finish randomly
What you are describing is a classic pitfall of Spring AOP.
In short, for Spring to be able to provide the async behavior it needs to create a proxy for your class at runtime. The proxy then does whatever it needs to do before and/or after calling your code. But in your case, the proxy mechanism is not being applied for the second method.
When a bean of your class is injected via Spring into some other component, Spring really injects the proxy instead. Therefor the relevant method of the proxy is called. However, when you are calling a method from inside the class, the limitations of Spring AOP mean the proxy never comes into play, but instead the regular method is called - with no extra features.
That is why asyncMethod is always executing on the same thread as the other method in the same class that called it.
Check out this excellent blog post as well as this part of Spring documentation.
There are some ways around the problem (check out this) that don't require you to refactor your code, but if you want async to work on both methods no matter what, the simplest thing to do is refactor the second method into another class.
EDITED SHORT VERSION OF THE POST:
Haven't had enough views, so I'm summarizing the question:
My architecture is completely stateless and async, the front-end makes a petition to a REST API and then long-polls for the response. This Rest API queues petitions into a messaging queue, and each petition is dequeued and processed by the Back-end.
I want this Back-end to follow the "traditional" Spring #Service interface and ServiceImpl approach, however is kind of hard because of how I'm doing it.
One Thread dequeues the petition (Producer), spawns a new Thread (Consumer), and then it processes all the petition within that thread, which later sends back to a "responses pool" where it gets polled. That petition might need to use several #Service's and merge the responses from each, maybe even the same #Service twice.
How would you do it? For more information, check the description below!
ORIGINAL LONG POST:
I have a large application with 3 layers like this:
Front-end (Spring-MVC): Views and Controllers, "Model" are async requests to REST API in Middleware to queue the petition first and then long-polling for an answer
Middleware (Spring-MVC): The rest API. Two main functions: receives a petition from front-end and queues it, receives an answer from Backend and stores it on responses cache until retrieved by front-end
Back-End (Spring Standalone App): Producer/Consumer pattern, ONE Producer dequeues petition and creates a Prototype Consumer for each petition. The consumer implements InitializingBean, so it goes something like this: It is initialized, many Autowired fields are initialized and then afterPropertiesSet is executed and many fields which depends on the petition are set.
I also have a Repository Layer of HibernateDaos, which does all the querying to the database.
I'm missing a properly built Service Layer and that's what this question is all about.
Let me put a little bit more of context. What I have right now is like one only HUGE service with 221 functions (The Consumer's file is very long), and one petition may need to invoke several of this functions, and the result of each is merged into a List of DTOs, which is later received by the front-end.
I want to split this one and only service into several, in a logical match to "it's" corresponding Repository, however I've faced the following problems:
Keep this in mind:
One petition has many Actions, one action is a call to a function of a Service.
Each Consumer is a single and unique Thread.
Every time a Consumer Thread starts, a transaction is started and right before returning it is commited, unless rollbacked.
I need all the services of that petition to be executed in the same thread and transaction.
When the consumer runs afterPropertiesSet, several fields specific to that request are initialized by some parameters which are always sent.
With a good Service Layer I want to acomplish:
I don't want to have to initialize all these parameters always for each service of the petition, I want them to be global to the petition/Thread, however, I don't want to have to pass then as parameters to all the 221 functions.
I want to lazily initialize the new services, only if needed, and when it is initialized, I want to set all the parameters I mentioned above. Each service needs to be a Prototype to the petition, however I feel like is dumb initializing it twice if needed within the same petition (2 actions for the same service on one petition), i.e. I want it to behave like a "Request" scope, however, it is not a request since it is not Web Based, it is a new Thread initialized by the Producer when de-queuing a petition.
I was thinking of having a prototype ServicesFactory per Consumer which is initialized with all the parameters afterPropetiesSet in the Consumer, inside this ServicesFactory all possible Services are declared as Class fields, and when a specific service is requested, if it's field is null it is initialized and all fields are set, if not null, the same instance is returned. The problem with this approach, I that I'm losing Dependency Injection on all the Services. I've been reading about ServiceFactoryBean thinking maybe this is the way to go, however I really can't get a hold to it. The fact that it needs all the parameters of the Consumer, and that it needs to be an unique ServiceFactoryBean per Consumer is really confusing.
Any thoughts?
Thanks
Based on the description I don't think this is a good case for using the protoype scope, in this case the ideal scope seems to be thread scope.
As a solution, the simplest would be to make all services singleton. Then the consumer reads the petition from the inbound queue and starts processing.
One of the services that is also singleton and gets injected in all services needed, let's call it PetitionScopedService.
This service internally uses a ThreadLocal, which is a thread scoped holder for a variable of type PetitionContext. PetitionContext on it's turn contain all information that is global to that petition.
All the consumer needs to do is to set the initial values of the petition context, and any caller of PetitionScopedService on the same thread will be able to read those values in a transparent way. Here is some sample code:
public class PetitionContext {
... just a POJO, getters and setters etc.
}
#Service
public class PetitionScopedService {
private ThreadLocal<PetitionContext> = new ThreadLocal<PetitionContext>();
public doSomethingPetitionSpecific() {
... uses the petition context ...
}
}
#Service
public class SomeOtherService {
#Autowired
private PetitionScopedService petitionService;
... use petition service that is a singleton with thread scoped internal state, effectivelly thread scoped ...
}
Points 2 and 3 need more reorganizing, prefer to check "Spring Integration" for both "Middleware" and "(Spring Standalone App): Producer/Consumer pattern" actually spring integration made to solve these 2 points, and using publish/subscribe if you are doing 2 or more actions at same time, the other point why you are using REST in "Middleware" are these "Middleware" services exposed by another app rather than your front end, in this case you can integrate this part in your Spring-MVC front end app using "content negotiation", otherwise if you are going to use "Spring Integration" you will find multiple ways for communication.
I have a class which acts as a simple crawler and I want to invoke this class within a servlet.
My idea is to get an url from user then url request will be passed to the servlet and servelt pass the url to the class and class will start the crawling. and I want my servlet to create only one instance of this class.the retrieved data from crawlwer will be added to the DB directly by the class.
I want to control the behavior of the class like running/halting/stopping from servlet
(for this matter I think I am able to create a simple xml file which will be shared between servlet and class and if servlet change the status code class should response to the status change)
But I have some doubts about how to control the behavior of the class such as command it to run/halt/stop and since my class is not multithreaded I don't have any idea what will happen to invoked class after calling it from servlet and since this class needs to read from network obviously I'll have some gap/freezing phase during running it.
How can I solve the problem of concurrency in this situation?or in other word will I have any concurrency issue or not?
regards.
It depends on the Servlet container you are using. Some containers spawn a new Thread per user request (almost always this is the desired behavior), so you should definitely design for concurrency.
You can make the Servlet class implement SingleThreadModel, then in the service method you can directly call the crawler class code, as only a thread will enter service at a time.
This implies only an URL can be processed at a given time, which is probably not what you want, so instead of that, don't implement SingleThreadModel and create a singleton executor service in the init method:
ExecutorService ex = Executors.newFixedThreadPool(20); //Only 20 tasks at a given time
Then, in the service method create a new CrawlingTask (Runnable) with the URL specified in the request, then submit the task to the executor.
That way you could also shutdown it:
ex.shutdown();
As ExecutorService is thread-safe, you don't have to worry about concurrency when enqueuing tasks.
First, understand the difference between a Class and a Thread. A class is just code, a thread is where the code is executed. You don't stop/halt a class, you stop or halt a thread that is executing code in a class.
I would suggest you start reading up on Java concurrency programming. since what you are describing is very much about multithreading and thread synchronization.
My question is about threads being queued. For my example I have one Spring context. I have a method named CalculateTax in a stateless class. A request comes in, a thread is created (tA) and it eventually enters the CalculateTax method. Within the same "time frame" another request comes in and another thread is created (tB). Now, here is what I want to understand. AFAIK tB cannot execute CalculateTax until tA has exited the method. Is this true?
As long as CalculateTax only uses local variables (i.e. declared in the method), you will not have any thread sync issues and multiple threads can call the method without a problem.
However if for some reason CalculateTax uses variables defined at the class level, and you are using the Singleton pattern (you tagged your question with "singleton", so I guess you are), you may have thread sync issues.
No it is not true if they are parallel thread, each thread is in its own stack of execution so it should be able to execute while tA is executing.
This is what Threads are for.
Generally speaking the answer is undefined. If your 'request' comes from remote client the answer depends on implementation details of the mechanism used for service exposing.
However, I'm not aware about remote communication frameworks that really make the proxy serializing the requests, i.e. that is assumed to be addressed by target service developer (e.g. its your task to provide thread-safety for the service implementation OR serialize all requests using explicit synchronization etc).