Spring sync vs async rest controller - java

I try to see a difference between Spring synchronous REST Controller vs async version of same controller.
Each controller do the same thing : take a RequestBody and save it in a Mongo database.
#RestController
#RequestMapping ("/api/1/ticks")
public class TickController {
#Autowired
private TickManager tickManager;
#RequestMapping (method = RequestMethod.POST)
public ResponseEntity save(#RequestBody List<Tick> ticks) {
tickManager.save(ticks);
return new ResponseEntity(HttpStatus.OK);
}
#RequestMapping (value = "/async", method = RequestMethod.POST)
public #ResponseBody Callable<ResponseEntity> saveAsync(#RequestBody List<Tick> ticks) {
return () -> {
tickManager.save(ticks);
return new ResponseEntity(HttpStatus.OK);
};
}
}
The tickManager has only a dependency on a tickRepository and just do call to sub-layer.
The tickRepository is based on Spring Data Mongodb:
#Repository
public interface TickRepository extends MongoRepository<Tick, String> {}
I use Gatling to test those controllers.
This is my scenario:
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._
class TicksSaveSyncSimulation extends Simulation {
val rampUpTimeSecs = 20
val testTimeSecs = 5
val noOfUsers = 1000
val minWaitMs = 1000 milliseconds
val maxWaitMs = 3000 milliseconds
val baseURL = "http://localhost:9080"
val requestName = "ticks-save-sync-request"
val scenarioName = "ticks-save-sync-scenario"
val URI = "/api/1/ticks"
val httpConf = http.baseURL(baseURL)
val http_headers = Map(
"Accept-Encoding" -> "gzip,deflate",
"Content-Type" -> "application/json;charset=UTF-8",
"Keep-Alive" -> "115"
)
val scn = scenario(scenarioName)
.repeat(100) {
exec(
http(requestName)
.post(URI)
.headers(http_headers)
.body(StringBody(
"""[{
| "type": "temperature",
| "datas": {}
|}]""".stripMargin))
.check(status.is(200))
)
}
setUp(scn.inject(rampUsers(1000) over (1 seconds))).protocols(httpConf)
}
I tried several situations and the sync version always handle 2 times more request by second than the async version.
When I increase the number of users the two versions crash.
I tried to override the taskExecutor for the async version with no more success:
#Configuration
public class TaskExecutorConfig implements AsyncConfigurer {
#Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setMaxPoolSize(1000);
taskExecutor.setThreadNamePrefix("LULExecutor-");
taskExecutor.initialize();
return taskExecutor;
}
#Override
public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
return new SimpleAsyncUncaughtExceptionHandler();
}
}
I thought see a difference in favor of the async implementation.
What am I doing wrong?

Your test looks to be flawed. It doesn't make any sense being non blocking at one end of the pipeline (here, your controllers), and being blocking at the other end (tickManager.save really looks like a blocking call). You're just paying the extra cost of jumping into a ThreadPoolTaskExecutor.
Then, generally speaking, you won't gain anything from a non blocking architecture when all your tasks are very fast, like a tick. You can expect gains when some tasks take some longer time, so you don't want to waste resources (threads, CPU cycles) just waiting for those to complete, and you want to use them to perform other tasks in the meanwhile.
Regarding your Too many open files exception, you probably haven't properly tuned your OS for load testing, check relevant documentation. There's also a good chance that you're running your app and Gatling (and possibly your database too) on the same host, which is bad as they'll compete for resources.

Related

Possible to add a delay in spring integration DSL?

I have the following adapter:
#Component
class MyCustomFlow : IntegrationFlowAdapter() {
fun singleThreadTaskExecutor(): TaskExecutor {
val executor = ThreadPoolTaskExecutor()
executor.maxPoolSize = 1
executor.initialize()
return executor
}
#Filter
fun filter(data: SomeData): Boolean = ...
#Transformer
fun transform(customer: Data): Message<SomeData> {
....
}
#ServiceActivator
fun handle(data: Data): SomeData {
....
}
#Bean(name = [PollerMetadata.DEFAULT_POLLER])
fun poller(): PollerSpec? {
return Pollers.fixedRate(500)
}
#Bean
override fun buildFlow(): IntegrationFlowDefinition<*> {
return from(MessageChannels.queue("updateCustomersLocation"))
.channel(MessageChannels.executor(singleThreadTaskExecutor()))
.split()
.filter(this)
.transform(this)
.handle(this)
.channel("customerLocationFetched")
}
}
And if I understand the Java DSL, the handle is:
handle → ServiceActivator
Given a ServiceActivator, outside of the IntegrationFlowAdapter, I have an option to define a poller. Within that poller I could add a delay before the next messages is processed.
#ServiceActivator(
poller = [Poller(fixedDelay = "3000", maxMessagesPerPoll = "1", fixedRate = "3000")]
)
Would it be possible, within the adapter, to add a delay for the ServiceActivator (the method named handle, similar to the logic I get if I add another channel.
#Bean
override fun buildFlow(): IntegrationFlowDefinition<*> {
return from(MessageChannels.queue("updateCustomersLocation"))
.channel(MessageChannels.executor(singleThreadTaskExecutor()))
.split()
.filter(this)
.transform(this)
.channel("myNewChannel")
//.handle(this)
//.channel("customerLocationFetched")
}
And then from outside the adapter I could just define a new ServiceActivator:
#ServiceActivator(
inputChannel = "myNewChannel"
poller = [Poller(fixedDelay = "3000", maxMessagesPerPoll = "1", fixedRate = "3000")]
)
The reason I want a delay between the messages in the handle/service activator is that the method sends requests and I wanna control the rate the request are sent.
First of all the buildFlow() method must not be marked with #Bean : the IntegrationFlowAdapter does the trick to register everything.
You can add poller to the endpoint spec. See a second arg of that handle(). Only the point that input channel for this polling endpoint must be pollable - the QueueChannel exists out-of-the-box and you can place it just before your handle().

Spring Boot RabbitMQ consistency issue

I'm using spring boot (2.5.5). I'm seeing inconsistent behavior using rabbitmq queue listener.
Issue is, accountService.incrementMediaCount executes, even if neoPostService.createNeoPost throws exeception, thus executes multiple times, as retry policy is set to 5, which I guess seems weird as accountService.incrementMediaCount should execute only if neoPostService.createNeoPost execute successfully (without exception). Also, loggingRepository.save() execute lesser times than number of messages coming to queue. What is causing such behavior? Is it relared to distributed nature of RabbitMQ?
Queue listener
#RabbitListener(queues = [RabbitConfiguration.Companion.Queues.POST_CREATION_QUEUE], concurrency = "3")
fun postCreationListener(postMessage: PostMessage) {
neoPostService.createNeoPost(postMessage.postId)
accountService.incrementMediaCount(postMessage.userId)
loggingRepository.save(
Log(
resourceType = ResourceType.Post,
entityEventType = EntityEventType.Created,
message = "Post ${postMessage.postId} created for user ${postMessage.userId}"
)
)
}
fun createNeoPost(postId: Long): NeoPost {
// Get post by id from postgresql
val post = postRepository.findByIdOrNull(postId)
?: throw ResourceNotFoundException("Post $postId not found")
val neoUser = neoUserService.getUserById(post.user.userId!!)
// Save post to Neo4j
return neoPostRepository.save(post.toNeoPost().copy(user = neoUser))
}
//Increment counter in postgresql
fun incrementMediaCount(userId: Long, amount: Int = 1) {
accountRepository.incrementPostCounter(userId, amount)
}
Rabbit Config
#EnableRabbit
#Configuration
class RabbitConfiguration {
// other code
....
#Bean
fun retryInterceptor(): RetryOperationsInterceptor? {
return RetryInterceptorBuilder.stateless()
.backOffOptions(100, 5.0, 10000)
.maxAttempts(5)
.recoverer(RejectAndDontRequeueRecoverer())
.build()
}
....
}

Debounce similar requests with reactor-grpc

In order to offload my database, I would like to debounce similar requests in a gRPC service (say for instance that they share the same id part of the request) that serves an API which does not have strong requirements in terms of latency. I know how to do that with vanilla gRPC but I am sure what kind of API of Mono I can use.
The API calling directly the db looks like this:
public Mono<Blob> getBlob(
Mono<MyRequest> request) {
return request.
map(reader.getBlob(request.getId()));
I have a feeling I should use delaySubscription but then it does not seem that groupBy is part of the Mono API that gRPC services handle.
It's perfeclty ok to detect duplicates not using reactive operators:
// Guava cache as example.
private final Cache<String, Boolean> duplicatesCache = CacheBuilder.newBuilder()
.expireAfterWrite(Duration.ofMinutes(1))
.build();
public Mono<Blob> getBlob(Mono<MyRequest> request) {
return request.map(req -> {
var id = req.getId();
var cacheKey = extractSharedIdPart(id);
if (duplicatesCache.getIfPresent(cacheKey) == null) {
duplicatesCache.put(cacheKey, true);
return reader.getBlob(id);
} else {
return POISON_PILL; // Any object that represents debounce hit.
// Or use flatMap() + Mono.error() instead.
}
});
}
If for some reason you absolutely want to use reactive operators, then first you need to convert incoming grpc requests into Flux. This can be achieved using thirdparty libs like salesforce/reactive-grpc or directly:
class MyService extends MyServiceGrpc.MyServiceImplBase {
private FluxSink<Tuple2<MyRequest, StreamObserver<MyResponse>>> sink;
private Flux<Tuple2<MyRequest, StreamObserver<MyResponse>>> flux;
MyService() {
flux = Flux.create(sink -> this.sink = sink);
}
#Override
public void handleRequest(MyRequest request, StreamObserver<MyResponse> responseObserver) {
sink.next(Tuples.of(request, responseObserver));
}
Flux<Tuple2<MyRequest, StreamObserver<MyResponse>>> getFlux() {
return flux;
}
}
Next you subscribe to this flux and use operators you like:
public static void main(String[] args) {
var mySvc = new MyService();
var server = ServerBuilder.forPort(DEFAULT_PORT)
.addService(mySvc)
.build();
server.start();
mySvc.getFlux()
.groupBy(...your grouping logic...)
.flatMap(group -> {
return group.sampleTimeout(...your debounce logic...);
})
.flatMap(...your handling logic...)
.subscribe();
}
But beware of using groupBy with lots of distinct shared id parts:
The groups need to be drained and consumed downstream for groupBy to work correctly. Notably when the criteria produces a large amount of groups, it can lead to hanging if the groups are not suitably consumed downstream (eg. due to a flatMap with a maxConcurrency parameter that is set too low).

Periodic work not reach running state when unit testing WorkManager

I created a PeriodicWorkRequest from my SyncDatabaseWorker, which like below:
class SyncDatabaseWorker(ctx: Context, params: WorkerParameters) : RxWorker(ctx, params) {
private val dataManager: DataManager = App.getDataManager()
override fun createWork(): Single<Result> {
return Single.create { emitter ->
dataManager.loadStoresFromServer()
.subscribe(object : SingleObserver<List<Store>> {
override fun onSubscribe(d: Disposable) {
}
override fun onSuccess(storeList: List<Store>) {
if (!storeList.isEmpty()) {
emitter.onSuccess(Result.success())
} else {
emitter.onSuccess(Result.retry())
}
}
override fun onError(e: Throwable) {
emitter.onSuccess(Result.failure())
}
})
}
}
companion object {
fun prepareSyncDBWorker(): PeriodicWorkRequest {
val constraints = Constraints.Builder()
.setRequiredNetworkType(NetworkType.CONNECTED)
.build()
val myWorkBuilder = PeriodicWorkRequest.Builder(SyncDatabaseWorker::class.java, 7, TimeUnit.DAYS)
.setBackoffCriteria(BackoffPolicy.EXPONENTIAL, 1, TimeUnit.DAYS) // Backoff retry after 1 day
.setConstraints(constraints)
return myWorkBuilder.build()
}
}
}
Then I write unit test base on Google's guide like this:
#RunWith(AndroidJUnit4::class)
class SyncDatabaseWorkerTest {
private lateinit var context: Context
#Before
fun setup() {
context = InstrumentationRegistry.getInstrumentation().targetContext
val config = Configuration.Builder()
.setMinimumLoggingLevel(Log.DEBUG)
.setExecutor(SynchronousExecutor())
.build()
// Initialize WorkManager for instrumentation tests.
WorkManagerTestInitHelper.initializeTestWorkManager(context, config)
}
#Test
#Throws(Exception::class)
fun testPeriodicWork_WithConstrains() {
// Create request
val request = SyncDatabaseWorker.prepareSyncDBWorker()
val workManager = WorkManager.getInstance(context)
val testDriver = WorkManagerTestInitHelper.getTestDriver(context)
// Enqueue and wait for result.
workManager.enqueue(request).result.get()
// Check work request is enqueued
var workInfo = workManager.getWorkInfoById(request.id).get()
assertThat(workInfo.state, `is`(WorkInfo.State.ENQUEUED))
// Tells the testing framework the period delay & all constrains is met
testDriver!!.setPeriodDelayMet(request.id)
testDriver.setAllConstraintsMet(request.id)
// Check work request is running
workInfo = workManager.getWorkInfoById(request.id).get()
assertThat(workInfo.state, `is`(WorkInfo.State.RUNNING))
}
}
It's always in state ENQUEUED, even when period delay & all constrains is met.
Expected: is <RUNNING>
but: was <ENQUEUED>
When I debugged the test & find out that the createWork(): Single<Result> method is also triggered, but why the state is not RUNNING?
May be I'm wrong about the approach, but the documents about unit testing WorkManager is very few now, and I don't know the right way to do it.
Since you're using a synchronous executor, you will never actually see your work in the RUNNING state - it should have already executed. I suspect your work is actually being marked for retry and therefore enters the ENQUEUED state again. You should be able to verify this by either setting breakpoints or looking at your logs.
Given you are executing a PeriodicWorkRequest with a SynchronousExecutor, you will never see the WorkRequest in RUNNING state. It will be done executing before you can assert that it was in RUNNING.
After you return a Result.success() or a Result.failure() in your doWork(), the WorkRequest goes back to ENQUEUED for the next period (given it's a periodic request).

Can we use #Async on a #HystrixCommand

I don't know how to test this and need a quick answer whether it is possible, and how could I possibly test this.
So i have an api-gateway call from a service, and what i want to know is that if following is possible with spring boot
#Component
class GatewayService {
#Autowired
private MicroserviceGateway microServiceGateway;
#Async
#HystrixCommand(threadPool = "microservice", groupKey = "microService", fallback = "getResponse2")
private Future<ResponseDTO> getResponse(RequestDTO request) {
try {
ResponseDTO response = new APIRequest(endpoint, api).body(request).post();
return new AsyncResult(response);
} catch (Exception e) {
throw new GatewayServiceException(e);
}
}
public Future<ResponseDTO> getResponse2(RequestDTO request) {
return Futures.immediateFuture(RequestDTO.empty());
}
}
Will the fallback work? will it all be async?
EDIT : Tried this, and hystrix is being ignored, since the future is returned immediately. Any work arounds? solutions?
I believe the approach to use Hystrix commands asynchronously using a thread pool would be:
If using #HystrixCommand annotation:
...
#HystrixCommand
public Future<Product> findProduct(String id) {
return new AsyncResult<Product>() {
#Override
public Product invoke() {
...
return productService.find ...
}
};
}
...
If using a Hystrix command class, for instance ProductCommandFind.java
...
ProductCommandFind command = new ProductCommandFind(...);
Future<Product> result = command.queue();
...
Hystrix command uses its own and configurable threadpool, maybe configuring a thread pool per command group, lets say a thread pool for command group named PRODUCT_GROUP used by commands: productCommandFind, productCommandCreate, ....
Basically I don't think there is a need to annotate a method with #Async and #HystrixCommand at the same time.
This should work. Your normal DTO method will be executed async and during fallback it will be synchronous.

Categories

Resources