Today I did some experiments on AsyncRestTemplate. Below is a piece of sample code:
ListenableFuture<ResponseEntity<MyObject[]>> result
= asyncRestTemplate.getForEntity(uri, MyObject[]);
List<MyObject> objects = Arrays.asList(result.get().getBody());
To my surprise, the request was not sent to uri in first line (i.e. after calling getForEntity) but sent after result.get() is called.
Isn't it a synchronous way of doing stuff?
The whole idea of doing async request is that either you do not want to wait for the async task to start/complete OR you want the main thread to do some other task before asking for the result from the Future instance. Internally, the AsyncRestTemplate prepares an AsyncRequest and calls executeAsync method.
AsyncClientHttpRequest request = createAsyncRequest(url, method);
if (requestCallback != null) {
requestCallback.doWithRequest(request);
}
ListenableFuture<ClientHttpResponse> responseFuture = request.executeAsync();
There are two different implementations - HttpComponentsAsyncClientHttpRequest ( which uses high performant async support provided in Apache http component library ) and SimpleBufferingAsyncClientHttpRequest (which uses facilities provided by J2SE classes). In case of HttpComponentsAsyncClientHttpRequest, internally it has a thread factory (which is not spring managed AFAIK) whereas in SimpleBufferingAsyncClientHttpRequest, there is a provision of Spring managed AsyncListenableTaskExecutor. The whole point is that in all cases there is some ExecutorService of some kind to be able to run the tasks asynchronously. Of course as is natural with these thread pools, the actual starting time of task is indeterminate and depends upon lots of factor like load, available CPU etc. and should not be relied upon.
When you call future.get() you're essentially turning an asynchronous operation into a synchronous one by waiting for the result.
It doesn't matter when the actual request is performed, the important thing is that since it's asynchronous, you don't need to worry about it unless/until you need the result.
The advantage is obvious when you need to perform other work before processing the result, or when you're not waiting for a result at all.
Related
The application in question handles requests from clients which then requires a lot of calculation on the server side. This calculation is done piece-by-piece, so if the client is slow to read, this calculation should not progress (the calculation should respond to back-pressure).
The calculation is now represented as a Supplier<Buffer>, in which the get() call might take a long time and needs to be called multiple times until it responds with null (no more data). The get() should be called in a separate thread-pool (which is shared with other requests), and should only be called if the client is really able to accept the data.
My current code is:
ReadStream<Buffer> readStream = new MyComplicatedReadStream(supplier, executor)
.exceptionHandler(request::fail)
.endHandler(x -> request.response().end());
Pump.pump(readStream, request.response())).start();
I've made a custom implementation of ReadStream to do this, which sort-of works, but is long, clunky and has synchronization issues.
Instead of fixing that, I wonder if there is a idiomatic way in vert.x / rx to implement / instantiate a MyComplicatedReadStream. So, for a Supplier<Buffer> and an ExecutorService get a ReadStream<Buffer> which executes get() with the given executor and doesn't generate if it is paused.
I have near 0 experience with vert.x but I do have some experience with rxjava. So there might be a better way to do this but from rxjava perspective you can make use of generate method to create 'cold' flowables which only generate items on demand. I believe in this case when the stream is paused, no additional calls to supplier.get() will be made as there is no 'demand'
using kotlin syntax here but I think you can derive the java version easily.
Flowable.generate<Buffer> { emitter ->
val nextValue = supplier.get()
if (nextValue == null) {
emitter.onComplete()
} else {
emitter.onNext(nextValue)
}
}.subscribeOn(Schedulers.from(executor)) // this will make the above callback run in the given executor
Since it seems that the supplier is holding some state, you may in some cases want to generate a 'new supplier' for each consumer, in which case you can use the overload of the generate method that allows specifying another callback to get an instance of the state (supplier in your case). http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/Flowable.html#generate-java.util.concurrent.Callable-io.reactivex.functions.BiConsumer-
Looks like then you can convert the flowable to a read stream:
ReadStream<Buffer> readStream = FlowableHelper.toReadStream(observable);
based on the docs here: https://vertx.tk/docs/vertx-rx/java2/#_read_stream_support
We have a set of Java applications that were originally written using normal synchronous methods but have largely been converted to asynchronous Vert.x (the regular API, not Rx) wherever it makes sense. We're having some trouble at the boundaries between sync and async code, especially when we have a method that must be synchronous (reasoning explained below) and we want to invoke an async method from it.
There are many similar questions asked previously on Stack Overflow, but practically all of them are in a C# context and the answers do not appear to apply.
Among other things we are using Geotools and Apache Shiro. Both provide customization through extension using APIs they have defined that are strictly synchronous. As a specific example, our custom authorization realm for Shiro needs to access our user data store, for which we have created an async DAO API. The Shiro method we have to write is called doGetAuthorizationInfo; it is expected to return an AuthorizationInfo. But there does not appear to be a reliable way to access the authorization data from the other side of the async DAO API.
In the specific case that the thread was not created by Vert.x, using a CompletableFuture is a workable solution: the synchronous doGetAuthorizationInfo would push the async work over to a Vert.x thread and then block the current thread in CompletableFuture.get() until the result becomes available.
Unfortunately the Shiro (or Geotools, or whatever) method may be invoked on a Vert.x thread. In that case it is extremely bad to block the current thread: if it's the event loop thread then we're breaking the Golden Rule, while if it's a worker thread (say, via Vertx.executeBlocking) then blocking it will prevent the worker from picking up anything more from its queue - meaning the blocking will be permanent.
Is there a "standard" solution to this problem? It seems to me that it will crop up anytime Vert.x is being used under an extensible synchronous library. Is this just a situation that people avoid?
EDIT
... with a bit more detail. Here is a snippet from org.apache.shiro.realm.AuthorizingRealm:
/**
* Retrieves the AuthorizationInfo for the given principals from the underlying data store. When returning
* an instance from this method, you might want to consider using an instance of
* {#link org.apache.shiro.authz.SimpleAuthorizationInfo SimpleAuthorizationInfo}, as it is suitable in most cases.
*
* #param principals the primary identifying principals of the AuthorizationInfo that should be retrieved.
* #return the AuthorizationInfo associated with this principals.
* #see org.apache.shiro.authz.SimpleAuthorizationInfo
*/
protected abstract AuthorizationInfo doGetAuthorizationInfo(PrincipalCollection principals);
Our data access layer has methods like this:
void loadUserAccount(String id, Handler<AsyncResult<UserAccount>> handler);
How can we invoke the latter from the former? If we knew doGetAuthorizationInfo was being invoked in a non-Vert.x thread, then we could do something like this:
#Override
protected AuthorizationInfo doGetAuthorizationInfo(PrincipalCollection principals) {
CompletableFuture<UserAccount> completable = new CompletableFuture<>();
vertx.<UserAccount>executeBlocking(vertxFuture -> {
loadUserAccount((String) principals.getPrimaryPrincipal(), vertxFuture);
}, res -> {
if (res.failed()) {
completable.completeExceptionally(res.cause());
} else {
completable.complete(res.result());
}
});
// Block until the Vert.x worker thread provides its result.
UserAccount ua = completable.get();
// Build up authorization info from the user account
return new SimpleAuthorizationInfo(/* etc...*/);
}
But if doGetAuthorizationInfo is called in a Vert.x thread then things are completely different. The trick above will block an event loop thread, so that's a no-go. Or if it's a worker thread then the executeBlocking call will put the loadUserAccount task onto the queue for that same worker (I believe), so the subsequent completable.get() will block permanently.
I bet you know the answer already, but are wishing it wasn't so -- If a call to GeoTools or Shiro will need to block waiting for a response from something, then you shouldn't be making that call on a Vert.x thread.
You should create an ExecutorService with a thread pool that you should use to execute those calls, arranging for each submitted task to send a Vert.x message when it's done.
You may have some flexibility in the size of the chunks you move into the thread pool. Instead of tightly wrapping those calls, you can move something larger higher up the call stack. You will probably make this decision based on how much code you will have to change. Since making a method asynchronous usually implies changing all the synchronous methods in its call stack anyway (that's the unfortunate fundamental problem with this kind of async model), you will probably want to do it high on the stack.
You will probably end up with an adapter layer that provides Vert.x APIs for a variety of synchronous services.
Short Version:
How can I create a Promise<Result> which is completed on a trigger of a callback?
Long Version:
I am working on an application which deals with third-party SOAP services. A request from user delegates to multiple SOAP services simultaneously, aggregates the results and sends back to the user.
The system needs to be scalable and should allow multiple concurrent users. As each user requests ends up triggering about 10 web service calls and each call blocking for about 1 second, the system needs to be designed with non-blocking I/O.
I am using Apache CXF within Play Framework (Java) for this system. I have managed to generate the Asynchronous WS Client proxies and enable the async transport. What I am unable to figure out is how to return a Future to Play's Thread when I have delegated to multiple Web Service proxies and the results will be obtained as callbacks.
Option 1: Using async method calls returning Java Future.
As described in this scala.concurrent.Future wrapper for java.util.concurrent.Future thread, there is no way we can convert a Java Future to a Scala Future. Only way to get a result from the Future is to do Future.get() which blocks the caller. Since CXF's generated proxies return Java Future, this option is ruled out.
Option 2: Use Scala Future.
Since CXF generates the proxy interfaces, I am not sure if there is any way I can intervene and return a Scala Future (AFAIK Akka uses Scala Futures) instead of Java Future?
Option 3: Use the callback approach.
The async methods generated by CXF which return Java Future also takes a callback object which I suppose will provide a callback when result is ready. To use this approach, I will need to return a Future which will wait until I receive a callback.
I think Option 3 is most promising, although I have no ideas about how I can return a Promise which will be completed on receiving a callback. I could possibly have a thread waiting in a while(true) and waiting in between until result is available. Again, I don't know how I can go into wait without blocking the thread?
In a nutshell, I am trying to build a system which is making a lot of SOAP web service calls, where each call blocks for significant time. The system may easily run out of threads in case of lot of concurrent web service calls. I am working on finding a solution which is non-blocking I/O based which can allow many ongoing web service calls at the same time.
Option 3 looks good :) A couple of imports to start with...
import scala.concurrent.{Await, Promise}
import scala.concurrent.duration.Duration
and, just to illustrate the point, here's a mocked CXF API that takes the callback:
def fetch(url: String, callback: String => Unit) = {
callback(s"results for $url")
}
Create a promise, call API with promise as callback:
val promise = Promise[String]
fetch("http://corp/api", result => promise.success(result))
Then you can take promise.future which is an instance of Future into your Play app.
To test it, you can do this:
Await.result(promise.future, Duration.Inf)
which will block awaiting the result, at which point you should see "results for http://corp/api" in the console.
I'm trying to wrap my head around Hystrix and after reading their docs, still have a question about its usage patterns.
For one, I don't understand the use case for when to use their Asynchronous execution vs. their Reactive execution. The only difference I can see is that Asynchronous execution is always non-blocking, whereas Reactive can either be blocking or non-blocking. So I guess my real question is:
What's the difference between Synchronous and Blocking Reactive execution?; and
What's the difference between Asynchronous and Non-Blocking Reactive execution?
Let's assume you have wrapped two service calls A and B as a HystrixCommand. You now have three options:
use .execute(): pure synchronous call. You call the method and continue your program once the result has arrived. Your program's total execution time is the sum both calls. The main flow of your program is very linear.
use .queue(): receive a Future immediately for both commands. Both service calls are executed in parallel. Then use .get() to retrieve the results. These calls with block until the result is there. Your total execution time is faster than before: your execution time will be the length of the longest service call. Use this when you i.e. want to combine the results of the two services. The main flow of your program is still linear, although both calls are executed in parallel.
use .subscribe(): receive a Observable immediately for both commands. Both service calls are executed in parallel. Then use .subscribe() to register a call-back to act on the result once it is available. This is very useful if you don't want to combine the results and want to react independently on the results of service A and B once they arrive.
The main flow of your program is no linear, but reactive: the flow of the program will continue inside the callback for each command.
I hope it helps.
i'm developing an app that make requests to the Musicbrainz webservice. I read in the musicbrainz manual to not make more than one request per second to the webservice or the client IP will be blocked.
What architecture do you suggest in order to make this restriction transparent to the service client.
I would like to call a method (getAlbuns for example) and it should only make the request 1sec after the last request.
I also want to call 10 request at once and the service should handle the queueing, returning the results when avaiable (Non-blocking).
Thanks!
Because of the required delay between invocations, I'd suggest a java.util.Timer or java.util.concurrent.ScheduledThreadPoolExecutor. Timer is very simple, and perfectly adequate for this use case. But if additional scheduling requirements are identified later, a single Executor could handle all of them. In either case, use fixed-delay method, not a fixed-rate method.
The recurring task polls a concurrent queue for a request object. If there is a pending request, the task executes it, and returns the result via a callback. The query for the service and the callback to invoke are members of the request object.
The application keeps a reference to the shared queue. To schedule a request, simply add it to the queue.
Just to clarify, if the queue is empty when the scheduled task is executed, no request is made. The simple approach would be just to end the task, and the scheduler will invoke the task one second later to check again.
However, this means that it could take up to one second to start a task, even if no requests have been processed lately. If this unnecessary latency is intolerable, writing your own thread is probably preferable to using Timer or ScheduledThreadPoolExecutor. In your own timing loop, you have more control over the scheduling if you choose to block on an empty queue until a request is available. The built-in timers aren't guaranteed to wait a full second after the previous execution finished; they generally schedule relative to the start time of the task.
If this second case is what you have in mind, your run() method will contain a loop. Each iteration starts by blocking on the queue until a request is received, then recording the time. After processing the request, the time is checked again. If the time difference is less than one second, sleep for the the remainder. This setup assumes that the one second delay is required between the start of one request and the next. If the delay is required between the end of one request and the next, you don't need to check the time; just sleep for one second.
One more thing to note is that the service might be able to accept multiple queries in a single request, which would reduce overhead. If it does, take advantage of this by blocking on take() for the first element, then using poll(), perhaps with a very short blocking time (5 ms or so), to see if the application is making any more requests. If so, these can be bundled up in a single request to the service. If queue is a BlockingQueue<? extends Request>, it might look something like this:
Collection<Request> bundle = new ArrayList<Request>();
bundle.add(queue.take());
while (bundle.size() < BUNDLE_MAX) {
Request req = queue.poll(EXTRA, TimeUnit.MILLISECONDS);
if (req == null)
break;
bundle.add(req);
}
/* Now make one service request with contents of "bundle". */
You need to define a local "proxy service" which your local clients will call.
The local proxy will receive requests and pass it on to the real service. But only at the rate of one message per second.
How you do this depends very much on the tecnoligy available to you.
The simplest would be a mutithreaded java service with a static and synchronised LastRequestTime long;" timestamp variable. (Although you would need some code acrobatics to keep your requests in sequence).
A more sophisticated service could have worker threads receiving the requests and placing them on a queue with a single thread picking up the requests and passing them on to the real service.