I know that a reactive controller should return a Flux<T> or Mono<T>, which means it's reactive on http handler level.
But if we do an external http call in a controller with non-reactive code, which have to wait a long time IO until that http call response, what will happen if 10000 users use http to call this controller at the same time? Let's assume there is only one thread to handle the code inside controller, so will more requests be handled during the IO?
If no, do we have to use reactive code such as WebClient and ReactiveRepository to call external http API and CRUD on DB?
If yes, how can that be implemented? Because it's just lines of non-reactive code, how Java know like "hey, it's waiting for response, let's handle another event first"?
Doing blocking I/O within a reactive pipeline (e.g. a reactive controller method) is forbidden; doing so can lead to serious problems at runtime or even terminal errors if a block operator is used.
Reactor provides infrastructure to wrap blocking calls and schedule that work on specific threads. See the How do I wrap a synchronous, blocking call section in the reactor reference documentation. Doing that will work but is likely to have a negative effect on performance.
I did an experiment myself, looks like we have to use reactive code inside WebFlux so that everything is reactive and the performance is really high
Related
Hello I was reading the following article Quarkus reactive architecture
At the start of the article it says
Quarkus is reactive. It’s even more than this: Quarkus unifies reactive and imperative programming. You don’t even have to choose: you can implement reactive components and imperative components then combine them inside the very same application.
just below the middle the article mentions
Thanks to hints in your code (such as the #Blocking and #NonBlocking annotations), Quarkus extensions can decide when the application logic is blocking or non-blocking.
My questions is does it matter if i write reactive or imperative? For example
Reactive approach
#GET
public Uni<List<Fruit>> get() {
return sf.withTransaction((s,t) -> s
.createNamedQuery("Fruits.findAll", Fruit.class)
.getResultList()
);
}
Imperative approach
#GET
#NonBlocking
public List<Fruit> get() {
return entityManager.createNamedQuery("Fruits.findAll", Fruit.class)
.getResultList();
}
Will both these code snippets run with the same reactive benefits?
Assuming your second snippet uses classic blocking Hibernate ORM, it will not work just like the first snippet (which seemingly uses Hibernate Reactive). It contains a blocking call, so you will block the event loop, which is the worst thing you can do in a non-blocking architecture, as it basically works against all the benefits that a non-blocking architecture can give you (see https://vertx.io/docs/vertx-core/java/#golden_rule).
If your methods are blocking, which includes calls to other libraries that are blocking, you must not pretend that they are #NonBlocking. If you remove the #NonBlocking annotation, the method call will be offloaded to a thread pool and the event loop stays free to handle other requests. This has some overhead, plus the thread pool is not infinite, but I dare to say that it will work just fine in a vast majority of cases.
I have a scenario where I’m attempting to serve data in a non-blocking fashion which is sourced by a RxJava Observable (also non-blocking). I’m using the WriteListener callback provided by ServletOutputStream. I’m running into an issue where the write is throwing an IllegalStateException (java.lang.IllegalStateException: UT010035: Stream in async mode was not ready for IO operation) immediately after a successful isReady() check on the ServletOutputStream.
While looking deeper, I noticed this comment in the Undertow implementation of ServletOutputStream:
Once the write listener has been set operations must only be invoked on this stream from the write listener callback. Attempting to invoke from a different thread will result in an IllegalStateException.
Given that my data source is asynchronous there are scenarios where the onWritePossible() callback will reach a state where there no data immediately available and I would need to wait for more to be received from the source. In these cases, I would need to interact with the stream from the callback of my data source which is going to be a different thread. The only other option would be to suspend the thread used to call onWritePossible() and wait for more data to arrive, but this would be a blocking operation which defeats the whole purpose.
Is there another approach that I’m missing? The single thread requirement of Undertow doesn’t seem to be required by the Servlet 3.1 spec. From what I’ve read, other implementations appear to tolerate the multi-threaded approach given that the application coordinates the stream access synchronization.
Today I did some experiments on AsyncRestTemplate. Below is a piece of sample code:
ListenableFuture<ResponseEntity<MyObject[]>> result
= asyncRestTemplate.getForEntity(uri, MyObject[]);
List<MyObject> objects = Arrays.asList(result.get().getBody());
To my surprise, the request was not sent to uri in first line (i.e. after calling getForEntity) but sent after result.get() is called.
Isn't it a synchronous way of doing stuff?
The whole idea of doing async request is that either you do not want to wait for the async task to start/complete OR you want the main thread to do some other task before asking for the result from the Future instance. Internally, the AsyncRestTemplate prepares an AsyncRequest and calls executeAsync method.
AsyncClientHttpRequest request = createAsyncRequest(url, method);
if (requestCallback != null) {
requestCallback.doWithRequest(request);
}
ListenableFuture<ClientHttpResponse> responseFuture = request.executeAsync();
There are two different implementations - HttpComponentsAsyncClientHttpRequest ( which uses high performant async support provided in Apache http component library ) and SimpleBufferingAsyncClientHttpRequest (which uses facilities provided by J2SE classes). In case of HttpComponentsAsyncClientHttpRequest, internally it has a thread factory (which is not spring managed AFAIK) whereas in SimpleBufferingAsyncClientHttpRequest, there is a provision of Spring managed AsyncListenableTaskExecutor. The whole point is that in all cases there is some ExecutorService of some kind to be able to run the tasks asynchronously. Of course as is natural with these thread pools, the actual starting time of task is indeterminate and depends upon lots of factor like load, available CPU etc. and should not be relied upon.
When you call future.get() you're essentially turning an asynchronous operation into a synchronous one by waiting for the result.
It doesn't matter when the actual request is performed, the important thing is that since it's asynchronous, you don't need to worry about it unless/until you need the result.
The advantage is obvious when you need to perform other work before processing the result, or when you're not waiting for a result at all.
Short Version:
How can I create a Promise<Result> which is completed on a trigger of a callback?
Long Version:
I am working on an application which deals with third-party SOAP services. A request from user delegates to multiple SOAP services simultaneously, aggregates the results and sends back to the user.
The system needs to be scalable and should allow multiple concurrent users. As each user requests ends up triggering about 10 web service calls and each call blocking for about 1 second, the system needs to be designed with non-blocking I/O.
I am using Apache CXF within Play Framework (Java) for this system. I have managed to generate the Asynchronous WS Client proxies and enable the async transport. What I am unable to figure out is how to return a Future to Play's Thread when I have delegated to multiple Web Service proxies and the results will be obtained as callbacks.
Option 1: Using async method calls returning Java Future.
As described in this scala.concurrent.Future wrapper for java.util.concurrent.Future thread, there is no way we can convert a Java Future to a Scala Future. Only way to get a result from the Future is to do Future.get() which blocks the caller. Since CXF's generated proxies return Java Future, this option is ruled out.
Option 2: Use Scala Future.
Since CXF generates the proxy interfaces, I am not sure if there is any way I can intervene and return a Scala Future (AFAIK Akka uses Scala Futures) instead of Java Future?
Option 3: Use the callback approach.
The async methods generated by CXF which return Java Future also takes a callback object which I suppose will provide a callback when result is ready. To use this approach, I will need to return a Future which will wait until I receive a callback.
I think Option 3 is most promising, although I have no ideas about how I can return a Promise which will be completed on receiving a callback. I could possibly have a thread waiting in a while(true) and waiting in between until result is available. Again, I don't know how I can go into wait without blocking the thread?
In a nutshell, I am trying to build a system which is making a lot of SOAP web service calls, where each call blocks for significant time. The system may easily run out of threads in case of lot of concurrent web service calls. I am working on finding a solution which is non-blocking I/O based which can allow many ongoing web service calls at the same time.
Option 3 looks good :) A couple of imports to start with...
import scala.concurrent.{Await, Promise}
import scala.concurrent.duration.Duration
and, just to illustrate the point, here's a mocked CXF API that takes the callback:
def fetch(url: String, callback: String => Unit) = {
callback(s"results for $url")
}
Create a promise, call API with promise as callback:
val promise = Promise[String]
fetch("http://corp/api", result => promise.success(result))
Then you can take promise.future which is an instance of Future into your Play app.
To test it, you can do this:
Await.result(promise.future, Duration.Inf)
which will block awaiting the result, at which point you should see "results for http://corp/api" in the console.
I need to understand what approaches there are to handle asynchronous operations in REST and what their advantages and disadvantages are. Some approaches I found:
Resource Based: Where the status of an operation is modeled as a status. User makes an async REST call (PUT, POST etc.) gets Accepted or In-Progress response (202). Further a status URI is polled repeatedly via GET to check status/progress/messages from operation execution.
Question: How long should this resource be active at Server? If the client polls in large intervals where in between the operation completes, how do we return the status? Seems like persisting the execution status would work. But how long to persist, when to archive/delete, is this kind of standard approach?
Callback Based: Where an async request is required to have a callback URI. The request gets processed asynchronously and upon completion makes a call to the callback URI with the operation status/result.
Question: This seems more elegant and involving less overhead at the server side. But how to handle scenarios where the callback server is intermittently down, not responding, etc.? Implement a typical retries where the callback URI provides retries configuration as well? Is there any other downside to this approach?
Servlet 3.0 Asynchronous support: Where an HTTP client makes a connection to a Java Servlet, which remains open until it is explicitly closed and until closed client and server can communicate asynchronously over it.
Question: Since its Servlet 3.0 spec, I think Jersey, the Spring REST implementation, doesn't utilize this approach as of now. Is there any specific REST implementation which utilizes a similar approach or pointer on ways to make it possible?
Any other approaches, maybe commercial ones?
Spring 3.2+ supports the async features of Servlet 3.0. From the Spring Blog:
you can make any existing controller method asynchronous by changing it to return a Callable. For example a controller method that returns a view name, can return Callable instead. An #ResponseBody that returns an object called Person can return Callable instead. And the same is true for any other controller return value type.
Jersey 2+ also supports asyncronous servers. See the Asynchronous Services and Clients chapter in the reference docs.
I think, the approach depends on time gap between initial request and the end of operation.
For short-time operations ( < 10s ) I would just keep the request open and return response when operation finished;
For long operations ( < 30m ) I would use servlet 3.0 or Comet model;
For extremely long operations ( hours, days ) good enough way, as for me, is just client-based polling or Comet with big timeouts.
I'm dealing now with the same situation and found the common approach of using Location header response to give a resource that can be monitored to check status (by polling of course). That seems to be the best, but in my case, I'm not creating a resource so I don't have a location to check the status (my async process is just to build a cache page).
You can always use your own headers to give an estimated time to complete the operation. Anyway I'm thinking of using Retry-After header to give an estimated time. What do you guys think?
I know this is old but I thought I'd chime in here to say that if what you want is something that can scale out in a stateless environment then you should go with your first option. You can perform the underlying operation anywhere and if you put the result in something like redis it will not matter to what web server the client makes subsequent polling requests. I'll usually put the polling interval in the response I sent to the client. When there a result is ready I will return the client a SEE OTHER that includes the id of the result in the URI.