I'm writing the unit tests for the old method, which uses CompletableFuture.supplyAsync(). The method is not complex, but my unit test keeps running and does not stop when executing the join(). I believe supplyAsync() should return the value very quickly. Is it because I set up the taskExecutor incorrectly? I'm new to Spring and Java, so please advise anything. Thank you.
Code:
public Response getReponse(Request request) {
CompletableFuture<String> vip = CompletableFuture.supplyAsync(() -> {
if(StringUtils.isBlank(request.getAddress())
){ return "";}
Request newRequest = Request.builder().
userId(request.getUserId()).
address(request.getZipCode()).
build();
Response newResult = getResult(newRequest);
return ("12345".equals(newResult.getZipCode()) + "";
}, taskExecutor);
final Response result = getResult(request);
result.setVIP(zipCode.join());
return result;
}
My unit test:
#Mock
private ThreadPoolTaskExecutor taskExecutor;
#Test
void getReponseTest(){
SomeService someService = new SomeService(Constants, logService, taskExecutor);
final SomeService someServiceSpy = Mockito.spy(someService);
final Request request = TestHelper.buildRequest();
final Response response = TestTestHelper.buildResponse();
doReturn(response).when(someServiceSpy).getResult(any(Request.class));
Response result = taxServiceSpy.getQuotationV2(taxRequest);
// assertion
...
}
You are mocking the ThreadPoolTaskExecutor so when it's told to execute the task, it is not doing anything. The CompletableFuture will never be completed, which is why things are hanging. You would need to either use a real ThreadPoolTaskExecutor, or implement the execute() method in the mocked Executor so it calls the run() method on the runnable passed to it. I don't understand what you're actually trying to accomplish in this unit test though. There doesn't seem to be a point to using an Executor at all. Also there is an error in the code since you call join on a variable called zipCode which does not exist, the future is called vip.
Like Jared mentioned, I built an executor, and it worked.
#Spy
private ThreadPoolTaskExecutor spyTaskExecutor = new ThreadPoolTaskExecutor();
spyTaskExecutor.setCorePoolSize(1);
spyTaskExecutor.setWaitForTasksToCompleteOnShutdown(true);
spyTaskExecutor.initialize();
Related
For my studies, I'm working on a Spring Boot REST API. I'm supposed to reduce the execution time of the code when it received a request. So, I thought that make the code asynchronous would be a good idea. But, unfortunately, I face some problems with Spring for this, and despite the few hours of research online to find a solution, I didn't find anything. Let me explain :
To optimize my code, I decided to use #Async Spring Annotation. For this, I created an AsyncConfiguration class who looks like this :
#Configuration
#EnableAsync
public class AsyncConfiguration {
#Bean(name = "asyncExecutor")
public Executor asyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(1);
executor.setQueueCapacity(50);
executor.setThreadNamePrefix("AsynchThread-");
executor.initialize();
return executor;
}
}
Usually, the asyncExecutor() method returns the Executor class or sub-class that must be used by Spring for async calls. In this case, it's a ThreadPoolTaskExecutor. And the majority of my code is annotated with #Async to use my asyncExecutor, like this :
#Async("asyncExecutor")
public CompletableFuture<User> calculateRewards(User user) {
return CompletableFuture.supplyAsync(() -> {
logger.info("Calculating Reward for user : " + user.getUserName());
List<VisitedLocation> userLocations = user.getVisitedLocations();
List<Attraction> attractions = gps.getAllAttraction();
for(VisitedLocation visitedLocation : userLocations) {
for(Attraction attraction : attractions) {
if(user.getUserRewards().stream().filter(r -> r.attraction.attractionName.equals(attraction.attractionName)).count() == 0) {
if(nearAttraction(visitedLocation, attraction)) {
user.addUserReward(new UserReward(visitedLocation, attraction, reward.getAttractionRewardPoints(attraction.attractionId, user.getUserId())));
}
}
}
}
return user;
});
}
But, here's the point : when I run the code, Spring DON'T use my asyncExecutor() bean. How do I know that ? First, when I call an annotated method, in the terminal, here's what I see :
2022-10-27 00:26:24.688 INFO 41436 --- [onPool-worker-4] c.T.T.service.TourGuideMainService : Calculating Reward for user : internalUser919
The "[onPool-worker-4]" is the Thread name, or at least the end of the Thread name. But it's not supposed to be named like that. If you look at my asyncExecutor() method above, you can see that there is a executor.setThreadNamePrefix("AsynchThread-");. If the code was working as intended, the Thread should be called "AsynchThread-4", but It's not.
Secondly, I decided to run my code in Debug mode, and I went into the Debug menu of VS Code, and I dicovered two things :
1 - When I run my stress test who make 1000 calls of calculateRewards() simultaneously, there is only 11 Threads created. Considering the execution time of the calculateRewards() method AND the fact that the Executor have its maxPoolSize by default (i.e. Integer.MAX_VALUE), there should be more than 11 Threads ;
2 - The entire name of the Threads when they are created is "[ForkJoinPool.commonPool-worker-4]" ;
It seems that Spring is using the ForkJoinPool class to create Threads, and it don't ever consider the configuration of my Executor. I have no idea of why it's doing that, I never used ForkJoinPool at all, and like I said, I didn't find anything when searching about that online.
So, why Spring is using ForkJoinPool instead of ThreadPoolTaskExecutor for async methods in my code ? And most important : how can I address that ?
(I hope that it's understandable...)
EDIT 1 : During some random testing, I found that, if Spring seems to use some ForkJoinPool Threads to execute my code, it create a "AsynchThread" anyway but don't use it. That's even more confusing... -_-"
The problem here is that you are using CompletableFuture.supplyAsync to produce your CompletableFuture.
This method will fork off a task running in the ForkJoinPool.commonPool() and execute your Supplier in that task.
Since spring will supply the executor and run your entire method asynchronously, you shouldn't need to use the supplyAsync function with a lambda. Instead your async method should look something like this (inside of your service):
#Service
public class MyService {
...
#Async("asyncExecutor")
public CompletableFuture<User> calculateRewards(User user) {
logger.info("Calculating Reward for user : " + user.getUserName());
List<VisitedLocation> userLocations = user.getVisitedLocations();
List<Attraction> attractions = gps.getAllAttraction();
for(VisitedLocation visitedLocation : userLocations) {
for(Attraction attraction : attractions) {
if(user.getUserRewards().stream().filter(r -> r.attraction.attractionName.equals(attraction.attractionName)).count() == 0) {
if(nearAttraction(visitedLocation, attraction)) {
user.addUserReward(new UserReward(visitedLocation, attraction, reward.getAttractionRewardPoints(attraction.attractionId, user.getUserId())));
}
}
}
}
return CompletableFuture.completedFuture(user);
}
}
If you autowire your service into a CommandLineRunner that executes your method, you can use this runner to see that spring will handle executing your method asynchronously using the Executor you have defined in your configuration.
For example:
#Component
public class Runner implements CommandLineRunner {
private final MyService service;
public Runner(MyService service) {
this.service = service;
}
#Override
public void run(String... args) throws Exception {
CompletableFuture<User> user1 = service.calculateRewards(new User("user1"));
CompletableFuture<User> user2 = service.calculateRewards(new User("user2"));
CompletableFuture<User> user3 = service.calculateRewards(new User("user3"));
CompletableFuture.allOf(user1,user2,user3).join();
}
}
I am trying to mock two consumer interfaces inside subscribe method
ConsumerService.java
#Autowired
ConsumerDao consumerDao;
public void insertStatus(){
SchedularStatus schedularStatus = new SchedularStatus();
Mono<Object> savedConsumer = consumerDao.save(schedularStatus);
savedConsumer.subscribe(success -> Log.info("consumer data saved")),
(error-> Log.error("Error while saving schedular data));"
}
Below Junit test code I have tried
ConsumerServiceTest.java
#ExtendWith(MockitoExtension.class)
class ConsumerServiceTest {
#InjectMock
ConsumerService consumerService;
#Mock
Mono<Object> mono;
#Mock
ConsumerDao consumerDao;
#Test
void testInsertStatus(){
Mockito.when(consumerDao.save(any(SchedularStatus))).thenReturn(mono);
doAnswer(answer -> {
Consumer<Object> consumer1 = answer.getArgument(0);
Consumer<Object> consumer2 = answer.getArgument(1);
consumer1.accept(new Object());
consumer2.accept(new Object());
return null;
}).when(mono).subscribe(any(Consumer.class), any(Consumer.class));
Mockto.verify(consumerDao).save(any(SchedularStatus.class));
}
But I am getting Nullpointer Exception
java.lang.NullPointerException : errorConsumer
at java.util.Objects.requireNonNull(Ojbects.java:228)
at reactor.core.publisher.Mono.subscribe(Mono.java:4278)
Don't mock non-service classes. Mocking Monos or Optionals or CompletableFutures almost always leads to problems (I have also seen real test code which created and set up mocks for Lists and Maps). Simply return a real instance:
Mockito.when(consumerDao.save(any(SchedularStatus)))
.thenAnswer(a -> Mono.just(new Object()));
And a second test which sets up the mock to return a failed Mono (Mono.error(new Exception())).
But then again, your test is not really testing anything (or performing very questionable actions), because:
You mock the method that you are verifying
Your mocked mono is successful and failed at the same time
You are never calling a method on your class under test
Your consumers don't do anything really, so why bother calling them?
Your NPE comes from your mock configuration :
doAnswer(answer -> {
Consumer<Object> consumer1 = answer.getArgument(0);
Consumer<Object> consumer2 = answer.getArgument(1);
consumer1.accept(new Object());
consumer2.accept(new Object());
return null; // You return null instead of a valid Disposable
}).when(mono).subscribe(any(Consumer.class), any(Consumer.class));
subscribe method return a Disposable. You have to provide one.
I am new to Async in Spring boot.
I have a bean A as follows:
class A {
private final B b;
private final C c;
...
int x = b.f();
c.g(x);
...
}
Here I would like to call both f() and g() in async. I have got some ideas from different articles regarding how to make #Async work. But, being a newbie, I cannot understand how would I call g() with the return value of f() in async.
It's pretty straight forward ,Add #EnableAsync annotation to a configuration class or to the main application class then add the #Async annotation to the method that you want to be executed asynchronously on a separate thread. Springboot will setup a thread pool and using proxy will automatically start a new thread to handle the method call. But if you want to return something from the method then use Future. You can also control which thread pool to use by creating a thread pool executor bean like below and specifying it in the #Async annotation.
#Configuration
#EnableAsync
class GeneralConfiguration {
#Bean(name = "asyncTaskExecutor")
public Executor threadPoolTaskExecutor() {
ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();
threadPoolTaskExecutor.setCorePoolSize(10);
threadPoolTaskExecutor.setMaxPoolSize(20);
threadPoolTaskExecutor.setThreadNamePrefix("ScheduledTaskThread-");
threadPoolTaskExecutor.initialize();
return threadPoolTaskExecutor;
}
}
class B {
#Async("asyncTaskExecutor") //ask spring to use your thread pool for this async task.
public Future<Integer> f() {
//Do something
return new AsyncResult<Integer>(1);
}
}
class C {
#Async
public void g() {
//Do something
}
}
From your comments, in order to wait for thee result of method f of class B to be provided as input to the method g of class C then use CompletableFuture like :
class B {
#Async("asyncTaskExecutor") //ask spring to use your thread pool for this async task.
public CompletableFuture<Integer> f() {
//Do something
return CompletableFuture.completedFuture(1);
}
}
Then after while calling the method do something like :
...
CompletableFuture<Integer> result = b.f();
CompletableFuture.allOf(result).join(); //add other futures if you want to wait for other calls to complete.
c.g(result.get());
...
Obviously there are other optimized way to use Completable Future . But it depends on how you want to use this in your code. I suggest going through the completable future docs and finding out which suits your use case best.
I have Service.class with start() and asychronous() method :
public ResponseEntity<Object> start() throws APICommandConstructionException, APICommunicationException, APIReplyParseException,
APIErrorResponse, IOException {
List<Company> companiesList = dbHandler.retrieveCompaniesList();
Company company = null;
for (int i = 0; i < companiesList.size(); i++) {
asychronousMethod(companiesList, i, company);
}
return new ResponseEntity<Object>("Start method has Finished", HttpStatus.OK);
}
#Async("threadPoolTaskExecutor")
public void asychronousMethod(List<Company> companiesList, int i, Company company) throws APICommandConstructionException, APIReplyParseException, APICommunicationException, APIErrorResponse, IOException {
company = companiesList.get(i);
company = utils.websiteScrap(company);
companiesRepository.save(company);
}
Everything that is inside a loop doesn't run in parallel, but it starts second loop after first finished. Why is that? How to do it parallel?
In brief: you shouldn't call explicitly methods with Spring annotations.
More detailed:
Spring creates special proxies that on back-stage do 'magic' for you. So if you have async annotation, that means that (depending on compile- and runtime-configuration of Spring) there was some hidden part of code that is not executed when you invoke this.asynchronousMethod.
How to fix:
First of all method should match public CompletableFuture<Void> - such way you know when thread is complete.
Second instead of this you need resolve self-instance as Spring proxy.
The simplest way over #Autowired:
#Autowired
MyClass zhis;
.... //in for loop:
future = zhis.asychronousMethod(companiesList, i, company);
P.s please see good example at https://spring.io/guides/gs/async-method/
I'm using Java and Mockito to mock some methods for unit testing. I want to mock the producer in the below code so that I can test for the log messages that are sent when the exception is thrown. I tried mocking the future however I get the error that the future.get() method cannot be mocked, and the RecordMetadata class is final and cannot be mocked. Any help would be greatly appreciated.
The producer in the below example is a KafkaProducer.
public void send(Log log){
Future<RecordMetadata> future = producer.send(new ProducerRecord<(this.topic, record));
try {
RecordMetadata recordMetadata = send.get();
} catch (InterruptedException e) {
LOG.error("Sending the message to kafka was interrupted. "+e);
}
}
Kafka supplies a MockProducer class that you can inject into your class for testing. You can set the producer in your class to an instance of MockProducer in a JUnit #Before-annotated setup method. From the referenced documentation for MockProducer:
A mock of the producer interface you can use for testing code that uses Kafka.
By default this mock will synchronously complete each send call successfully. However it can be configured to allow the user to control the completion of the call and supply an optional error for the producer to throw.
You can use the #errorNext method to supply the exception that you want to throw in your test scenario.
It is difficult to be precise here without seeing your test code. Two issues
1) RecordMetadata cannot be used a Mockito mock, this is a known limitation of Mockito. Instead you can create a dummy instance of RecordMetadata using its public constructor.
2) KafkaProducer can be mocked by Mockito, but you need a two stage approach. Firstly the mock KafkaProducer returns a Future, and secondly that Future is also a mock that returns some known value.
public class ServiceTest {
#Mock
private KafkaProducer<String, Integer> producer;
#Mock
private Future<RecordMetadata> future;
#Before
public void setup() {
MockitoAnnotations.initMocks(this);
}
#Test
public void success() throws Exception {
RecordMetadata dummyRecord = new RecordMetadata(null, 0L, 0L, 0L, 0L, 0, 0);
when(producer.send(any())).thenReturn(future);
when(future.get()).thenReturn(dummyRecord);
producer.send(null);
}
#Test
public void timeout() throws Exception {
when(producer.send(any())).thenReturn(future);
when(future.get()).thenThrow(new TimeoutException("timeout"));
producer.send(null);
}
}