I have an interface called processor:
public interface Processor {
MyResponse process(Request request, String id);
}
I have two implementations of this processor, one synchronous, the other asynchronous (both Spring Service classes):
This is Sync Version, completes the job and give a fully populated response:
#Service
public class SynchrnousProcess implements Processor {
#Override
public MyResponse process(Request request, String id) {
Job job = getJob(request);
JobParameters parameter = buildParams(request, id);
JobExecution jobExecution = kickOfJob(job, request, parameter);
return buildResponse(request, trackingId, jobExecution);
}
}
This is Async version, which adds requests to blocking queue:
#Service
public class AsyncProcess implements Processor {
private BlockingQueue<Pair> requestQueue = new ArrayBlockingQueue<>(100);
#Override
public MyResponse process(Request request, String id) {
//add request to requestQueue
}
public void completeProcess() {
//take data from queue and process
log.info("start process !!!!!!!!!!!!!!!");
if (!CollectionUtils.isEmpty(requestQueue)) {
requestQueue.stream().forEach(pair -> {
String trackingId = String.valueOf(pair.getFirst());
FeedDto feedDto = (FeedDto) pair.getSecond();
Request request = feedDto.getRequest();
Job job = getJob(request);
JobParameters parameter = getJobParameters(request, trackingId);
JobExecution jobExecution = runJob(job, request, parameter);
}
}
As you can see, completeProcess() takes data out of the queue and processes the job. Common code to do with getting job, building param is duplicated between synchronous run and async.
Another thread runs completeProcess() in background.
I want to achieve a clean design and move common code that are shared by the two interface implementations in one place. How can I achieve this for what I am trying to achieve? It will be really helpful to see an example of the design pattern to use in this case?
In Addition, any suggestion as to where the ExecutorService to process the queued requests should be initiated and how the threads to process these should be started will help a lot.
You could design the interface as it has to perform async operations. For example:
public interface Callback() {
void onResponse(MyResponse mr);
}
public void Processor {
void process(Request request, String id, Callback callback);
}
#Service
public class SynchrnousProcess implements Processor {
#Override
public void process(Request request, String id, Callback callback) {
Job job = getJob(request);
JobParameters parameter = buildParams(request, id);
JobExecution jobExecution = kickOfJob(job, request, parameter);
MyResponse r = buildResponse(request, trackingId, jobExecution);
callback.onResponse(r);
}
}
#Service
public class AsyncProcess implements Processor {
private BlockingQueue<Pair> requestQueue = new ArrayBlockingQueue<>(100);
#Override
public void process(Request request, String id, Callback callback) {
//add request and callback to requestQueue
}
}
I don't know who will process the queue, but surely who will use Processor does not have to worry if the call is synchronous or not.
this is the most similar approach to the one you would use in C# with async await.
In C# you have to return a Task < MyResponse > even in the synchronous case. Then the consumer will always think that the call will be asynchronous.
Related
I'm trying to convert the following method using WebFlux to avoid using #Async on the same.
#Async
#Override
public void notifyCallback(NotifyCallbackRequest request, String id) {
startProcess(id);
}
Controller class:
#ResponseStatus(HttpStatus.OK)
#PostMapping("/notify-status/{id}")
public void notifyCallback(#PathVariable("id") String id, #RequestBody NotifyCallbackRequest request) {
identityService.notifyCallback(request, id);
}
startProcess(id) is a method that makes an api call via webflux and returns a Mono type.
I am interested that the caller immediately receives a ResponseStatus (HttpStatus.OK) despite what happens next.
You can just subscribe to your "async" task independently of your main reactive chain to start to make it run in the background as soon as the endpoint is called.
#Override
public void notifyCallback(NotifyCallbackRequest request, String id) {
startProcess(id).subscribe();
}
#ResponseStatus(HttpStatus.OK)
#PostMapping("/notify-status/{id}")
public Mono<Void> notifyCallback(#PathVariable("id") String id, #RequestBody NotifyCallbackRequest request) {
identityService.notifyCallback(request, id);
return Mono.empty();
}
You can demo this behaviour using the following endpoint example. If you call the endpoint, it will a 200 straight away and 10 seconds later it will log to your console
#RequestMapping(path = "/async", method = RequestMethod.GET)
public Mono<Void> start() {
Mono.delay(Duration.ofSeconds(10))
.doOnNext(i -> LOGGER.info("we have waited 10 seconds"))
.subscribe();
return Mono.empty();
}
I am trying to write my own Async service implementation alongside my already existing Synchronous version.
I have the following so far:
#Service("asynchronousProcessor")
public class AsynchronousProcessor extends Processor {
private BlockingQueue<Pair<String, MyRequest>> requestQueue = new LinkedBlockingQueue<>();
public AsynchronousProcessor(final PBRequestRepository pbRequestRepository,
final JobRunner jobRunner) {
super(pbRequestRepository, jobRunner);
}
#Override
public MyResponse process(MyRequest request, String id) {
super.saveTheRequestInDB(request);
// add task to blocking queue and have it processed in the background
}
}
Basically I have an endpoint RestController class that calls process(). The async version should queue the request in a BlockingQueue and have it processed in the background.
I am unsure how to implement this code to solve this problem. Whether I should use ExecutorService and how best to fit with this current design.
It would be useful to have some controls such as before executing a task or after executing a task calls.
Any answer with some code samples to show design would be really helpful :)
If the only requirement is to process it asynchronously then I'd strongly recommend consider using spring inbuilt #Async for this purpose. Using this approach however will not be interface compatible with your existing process method of Processor since the return type MUST be either void or wrapped in Future type. This limitation is for good reasons since the async execution can not return the response immediately thus Future wrapper is the only way to get access to result should that be needed.
Following solution outline lays out what should be done in order to switch from sync execution to async execution while retaining interface compatibility. All important points are mentioned with inline comments. Please note, although this is interface compatible, the return type is null (for the reasons stated above). If you MUST need the return value within your controller than this approach (or any async approach for that matter) is NOT going to work unless you switch to async controller as well (a different topic with much wider change and design though). Following outline also include pre and post execution hooks.
/**
* Base interface extracted from existing Processor.
* Use this interfae as injection type in the controller along
* with #Qualifier("synchProcessor") for using sync processor.
* Once ready, switch the Qualifier to asynchronousProcessor
* to start using async instead.
*/
public interface BaseProcessor {
public MyResponse process(MyRequest request, String id);
}
#Service("synchProcessor")
#Primary
public class Processor implements BaseProcessor {
#Override
public MyResponse process(MyRequest request, String id) {
// normal existing sync logic
}
}
#Service("asynchronousProcessor")
public class AsynchronousProcessor implements BaseProcessor {
#Autowired
private AsynchQueue queue;
public MyResponse process(MyRequest request, String id) {
queue.process(request,id);
// async execution can not return result immediately
// this is a hack to have this implementation interface
// compatible with existing BaseProcessor
return null;
}
}
#Component
public class AsynchQueue {
#Autowired
#Qualifier("synchProcessor")
private BaseProcessor processor;
/**
* This method will be scheduled by spring scheduler and executd
* asynchronously using an executor. Presented outline will
* call preProcess and postProcess methods before actual method
* execution. Actual method execution is delegated to existing
* synchProcessor resuing it 100% AS-IS.
*/
#Override
#Async
public void process(MyRequest request, String id) {
preProcess(request, id);
MyResponse response = processor.process(request, id);
postProcess(request, id, response);
}
private void preProcess(MyRequest request, String id) {
// add logic for pre processing here
}
private void postProcess(MyRequest request, String id, MyResponse response) {
// add logic for post processing here
}
}
Another use case could be to batch process the db updates instead of processing them using one by one as you are doing already. This is especially useful if you have high volume and db updates are becoming bottleneck. For this case, using a BlockingQueue makes sense. Following is the solution outline that you can use for this purpose. Again, although this is interface compatible, the return type is still null. You can further fine tune this outline to have multiple processing threads (or spring executor for that matter) should that be needed for batch processing. For one similar use case, a single processing thread with batch updates was sufficient for my needs, concurrent db updates were presenting bigger problems due to db level locks in concurrent execution.
public class MyRequestAndID {
private MyRequest request;
prviate String id;
public MyRequestAndID(MyRequest request, String id){
this.request = request;
this.id = id;
}
public MyRequest getMyRequest() {
return this.request;
}
public String MyId() {
return this.id;
}
}
#Service("asynchronousProcessor")
public class BatchProcessorQueue implements BaseProcessor{
/* Batch processor which can process one OR more items using a single DB query */
#Autowired
private BatchProcessor batchProcessor;
private LinkedBlockingQueue<MyRequestAndID> inQueue = new LinkedBlockingQueue<>();
private Set<MyRequestAndID> processingSet = new HashSet<>();
#PostConstruct
private void init() {
Thread processingThread = new Thread(() -> processQueue());
processingThread.setName("BatchProcessor");
processingThread.start();
}
public MyResponse process(MyRequest request, String id) {
enqueu(new MyRequestAndID(request, id));
// async execution can not return result immediately
// this is a hack to have this implementation interface
// compatible with existing BaseProcessor
return null;
}
public void enqueu(MyRequestAndID job) {
inQueue.add(job);
}
private void processQueue() {
try {
while (true) {
processQueueCycle();
}
} catch (InterruptedException ioex) {
logger.error("Interrupted while processing queue", ioex);
}
}
private void processQueueCycle() throws InterruptedException {
// blocking call, wait for at least one item
MyRequestAndID job = inQueue.take();
processingSet.add(job);
updateSetFromQueue();
processSet();
}
private void processSet() {
if (processingSet.size() < 1)
return;
int qSize = processingSet.size();
preProcess(processingSet)
batchProcessor.processAll(processingSet);
postProcess(processingSet)
processingSet.clear();
}
private void updateSetFromQueue() {
List<MyRequestAndID> inData = Arrays.asList(inQueue.toArray(new MyRequestAndID[0]));
if (inData.size() < 1)
return;
inQueue.removeAll(inData);
processingSet.addAll(inData);
}
private void preProcess(Set<MyRequestAndID> currentSet) {
// add logic for pre processing here
}
private void postProcess(Set<MyRequestAndID> currentSet) {
// add logic for post processing here
}
}
I am trying to use BlockingQueue inside Spring Boot. My design was like this: user submit request via a controller and controller in turn puts some objects onto a blocking queue. After that the consumer should be able to take the objects and process further.
I have used Asnyc, ThreadPool and EventListener. However with my code below I found consumer class is not consuming objects. Could you please help point out how to improve?
Queue Configuration
#Bean
public BlockingQueue<MyObject> myQueue() {
return new PriorityBlockingQueue<>();
}
#Bean
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(3);
executor.setMaxPoolSize(3);
executor.setQueueCapacity(10);
executor.setThreadNamePrefix("Test-");
executor.initialize();
return executor;
}
Rest Controller
#Autowired
BlockingQueue<MyObject> myQueue;
#RequestMapping(path = "/api/produce")
public void produce() {
/* Do something */
MyObject myObject = new MyObject();
myQueue.put(myObject);
}
Consumer Class
#Autowired
private BlockingQueue<MyObject> myQueue;
#EventListener
public void onApplicationEvent(ContextRefreshedEvent event) {
consume();
}
#Async
public void consume() {
while (true) {
try {
MyObject myObject = myQueue.take();
}
catch (Exception e) {
}
}
}
Your idea is using Queue to store messages, consumer listens to spring events and consume.
I didn't see your code have actually publish the event, just store them in queue.
If you want to use Spring Events, producers could like this:
#Autowired
private ApplicationEventPublisher applicationEventPublisher;
public void doStuffAndPublishAnEvent(final String message) {
System.out.println("Publishing custom event. ");
CustomSpringEvent customSpringEvent = new CustomSpringEvent(this, message);
applicationEventPublisher.publishEvent(customSpringEvent);
}
check this doc
If you still want to use BlockingQueue, your consumer should be a running thread, continuously waiting for tasks in the queue, like:
public class NumbersConsumer implements Runnable {
private BlockingQueue<Integer> queue;
private final int poisonPill;
public NumbersConsumer(BlockingQueue<Integer> queue, int poisonPill) {
this.queue = queue;
this.poisonPill = poisonPill;
}
public void run() {
try {
while (true) {
Integer number = queue.take(); // always waiting
if (number.equals(poisonPill)) {
return;
}
System.out.println(Thread.currentThread().getName() + " result: " + number);
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
could check this code example
#Async doesn't actually start a new thread if the target method is called from within the same object instance, this could be the problem in your case.
Also note that you need to put #EnableAsync on a config class to enable the #Async annotation.
See Spring documentation: https://docs.spring.io/spring-framework/docs/current/reference/html/integration.html#scheduling-annotation-support
The default advice mode for processing #Async annotations is proxy which allows for interception of calls through the proxy only. Local calls within the same class cannot get intercepted that way. For a more advanced mode of interception, consider switching to aspectj mode in combination with compile-time or load-time weaving.
In the end I came up with this solution.
Rest Controller
#Autowired
BlockingQueue<MyObject> myQueue;
#RequestMapping(path = "/api/produce")
public void produce() {
/* Do something */
MyObject myObject = new MyObject();
myQueue.put(myObject);
Consumer.consume();
}
It is a little bit weird because you have to first put the object on queue yourself then consume that object by yourself. Any suggestions on improvement is highly appreciated.
we have a service that has one endpoint which needs to be restricted to process 2 requests at a time. These 2 requests can take a while to be completed.
Currently we use the tomcat properties to do so.
The problem we face is now is that - when these 2 threads are used up for that endpoint - our healthcheck does not work anymore.
So we would like to restrict the number of requests for that particular endpoint.
We pondered a while about it and one idea was to do so via filter, but that seems very hacky to me...
So I was hoping someone has another idea?
Here's an example of how to implement an asynchronous REST controller that will handle no more than 2 simultaneous requests at the same time. This implementation will not block any of your Tomcat servlet threads while the requests are being processed.
If another one arrives while the two are in progress then the caller will get an HTTP 429 (Too Many Requests).
This example immediately rejects requests that cannot be handled with a 429. If instead you'd like to queue pending requests until one of the 2 processing threads are available then replace SynchronousQueue with another implementation of BlockingQueue.
You might want to tidy up this sample, I've intentionally embedded all classes used to fit it in here:
#Configuration
#RestController
public class TestRestController {
static class MyRunnable implements Runnable {
DeferredResult<ResponseEntity<String>> deferredResult;
MyRunnable(DeferredResult<ResponseEntity<String>> dr) {
this.deferredResult = dr;
}
#Override
public void run() {
// do your work here and adjust the following
// line to set your own result for the caller...
this.deferredResult.setResult(ResponseEntity.ok("it worked"));
}
}
#SuppressWarnings("serial")
#ResponseStatus(HttpStatus.TOO_MANY_REQUESTS)
static class TooManyRequests extends RuntimeException {
}
private final ExecutorService executorService = new ThreadPoolExecutor(2, 2,
0L, TimeUnit.MILLISECONDS,
new SynchronousQueue<Runnable>(),
(runnable, executor) -> {
((MyRunnable) runnable).deferredResult.setErrorResult(new TooManyRequests());
});
#GetMapping(value = "/blah", produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
public DeferredResult<ResponseEntity<String>> yourRestService() {
final DeferredResult<ResponseEntity<String>> deferredResult = new DeferredResult<>();
this.executorService.execute(new MyRunnable(deferredResult));
return deferredResult;
}
}
By default RequestMappingHandlerAdapter handles #Controller 's #RequestMapping methods . So the most easiest way is to create your own RequestMappingHandlerAdapter and override its handleInternal to add your control logic.
Below is the pseudocode:
public static class MyRequestMappingHandlerAdapter extends RequestMappingHandlerAdapter {
//counter to keep track number of concurrent request for each HandlerMethod
//HandlerMethod represent a #RequestMapping method
private Map<HandlerMethod, Integer> requestCounterMap = new ConcurrentHashMap<>();
#Override
protected ModelAndView handleInternal(HttpServletRequest request, HttpServletResponse response,
HandlerMethod handlerMethod) throws Exception {
//Increase the counter for this handlerMethod by 1.
//Throw exception if the counter is more than 2 request
ModelAndView mv = super.handleInternal(request, response, handlerMethod);
//Method finish , decrease the counter by 1
return mv;
}
}
Assume you are using the spring boot MVC auto-configuration , you can replace RequestMappingHandlerAdapter with your customized one by creating a WebMvcRegistrations bean and override its getRequestMappingHandlerAdapter() methods:
#Bean
public WebMvcRegistrations webMvcRegistrations() {
return new WebMvcRegistrations() {
#Override
public RequestMappingHandlerAdapter getRequestMappingHandlerAdapter() {
return new MyRequestMappingHandlerAdapter();
}
};
}
Below is a simplified setup of my application. It has a class Foobar which calls on a facade method for fetching data. The facade then calls on a web service to actually get the data and then manipulates the data a bit and then returns it to Foobar.
Now because the web service might take a good while to run, the method call to the facade needs to be asynchronous. Hence the facade's method doesn't have a return value, but instead, the method uses a callback object. Look at the example and continue reading below.
public class Foobar {
private List<DTO> dtos;
#Autowired
private Facade facade;
public void refresh() {
facade.refreshFoobar(new CallBack() {
public void dataFetched(List<DTO> dtos) {
setDtos(dtos);
}
});
}
public void setDtos(List<DTO> dtos) {
this.dtos = dtos;
}
}
public class Facade {
...
public void refreshFoorbar(CallBack cb) {
// Fetch data from a web service
List<DTO> dtos = webService.getData();
// Manipulate DTOs
....
// call on the callback method
cb.dataFecthed(dtos);
}
}
I have two ways of making the facade's method asynchronous, either by creating a thread manually or by using springs #Async annotation.
public class Facade {
public void refreshFoorbar(CallBack cb) {
new Thread() {
#Override
public void run() {
....
}
}.start();
}
}
// ... OR ...
public class Facade {
#Async
public void refreshFoorbar(CallBack cb) {
....
}
}
My problem is that I now need to write an integration test for this chain of method calls. I think I need to force the async facade call to be synchronous when the integration test is ran, otherwise I won't know for sure when I can do the appropriate asserts. The only idea for making the method call synchronous is to use manually handled threads AND making the threading conditional (so, for testing purposes, I have an if clause which determines if the facade method should be ran in a separate thread or not).
However, I have a feeling that there could be a better solution to my problem, whether it be a better way of forcing the method to me synchronous, eg with spring, or by testing the multithreading on some way.
This is where I need your suggestions, how would you solve my problem? Note, I'm using junit for both unit and integration tests.
Simple solution would be to return a Future object like this,
#Async
public Future<String> refreshFoorbar(CallBack cb) {
yourHeavyLifting(); //asynchronous call
return new AsyncResult<String>("yourJobNameMaybe");
}
And in your test, take the future reference and call the get() method.
future.get(); // if its not already complete, waits for it to complete
assertTrue(yourTestCondition)
This blog post shows a sample.
When JUnit testing stuff like this, I use a testing callback with a CountDownLatch that gets counted down by the callback and await()ed by the test method.
private static class TestingCallback implements Callback {
private final CountDownLatch latch;
public TestingCallback(CountDownLatch latch) {
this.latch = latch;
}
#Override public void onEvent() {
this.latch.countDown();
}
}
#Test
public void testCallback() {
final CountDownLatch latch = new CountDownLatch(1);
classUnderTest.execute( new TestCallback(latch) );
assertTrue(latch.await(30, TimeUnit.SECONDS));
}
If the callback is invoked (asynchronously) by the code under test, the latch returns true and the test passes. If the callback doesn't get invoked, the test times out after thirty seconds and the assertion fails.