Our app is a message processing system with multiple components connected with RabbitMQ queues. So the message processing are asynchronous. Now I need to add a HTTP adapter that communicates with the system. Since HTTP is synchronous with request/response, I need a way to connect synchronous and asynchronous flows. The current solution is:
HTTP requests are sent to one queue. Each request has a unique request ID for correlation.
HTTP request is blocked by a CompletableFuture.
The request is processed and response is sent back to another queue.
Queue consumer uses response to complete the CompletableFuture matching with request ID.
The HTTP adapter is implemented using Akka HTTP. Requests are handled using handleWithAsyncHandler() with a function of type Function<HttpRequest, CompletionStage<HttpResponse>>.
The problem is that the HTTP adapter needs to manage a map (Map<String, CompletableFuture>) of all pending requests. For each request, a new CompletableFuture object is created and put into the map. When a response is received in the queue, the matching CompletableFuture is completed to finish the request. This seems a bad smell in the code because I need to carefully manage this map. For example, if a response failed to generate for a request, the request needs to be removed from the map.
I wonder if there are other ways than using a map to track all pending requests.
Basically, akka-http could be async style. You do not need to implement that queue to map the request Id.
One thing needs to be considered that DO NOT use the default dispatcher.
Better to define a blocking dispatcher to handle CompletableFuture.supplyAsync
For example
my-blocking-dispatcher {
type = Dispatcher
executor = "thread-pool-executor"
thread-pool-executor {
fixed-pool-size = 16
}
throughput = 1
}
import static akka.http.javadsl.server.Directives.completeWithFuture;
import static akka.http.javadsl.server.Directives.post;
// GOOD (the blocking is now isolated onto a dedicated dispatcher):
final Route routes = post(() -> {
final MessageDispatcher dispatcher = system.dispatchers().lookup("my-blocking-dispatcher");
return completeWithFuture(CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(5000L);
} catch (InterruptedException e) {
}
return HttpResponse.create()
.withEntity(Long.toString(System.currentTimeMillis()));
}, dispatcher // uses the good "blocking dispatcher" that we
// configured, instead of the default dispatcher to isolate the blocking.
));
});
Related
Is it possible to decide at runtime whether a Jersey REST request to an resource endpoint should be handled synchronously or asynchronously? Let's take a simple example.
The synchronous version:
#Path("resource")
public class Resource {
#GET
#Produces({MediaType.TEXT_PLAIN})
public Response get() {
return Response.ok("Hello there!").build();
}
}
The asynchronous version:
#Path("resource")
public class Resource {
#GET
#Produces({MediaType.TEXT_PLAIN})
public void get(#Suspended final AsyncResponse r) {
r.resume(Response.ok("Hello there!").build()); // usually called somewhere from another thread
}
}
Depending on certain parameters, I would like to decide at runtime whether the GET request should be handled synchronously or asynchronously. The URL of the resource endpoint (http://server/resource) must be the same in both cases. Is this possible?
Of course, as you can see in the example above, the synchronous version can be faked in an asynchronous manner by simply calling AsyncResponse.resume(...). However, I would to avoid the overhead of creating the asynchronous response.
A step back
The JAX-RS Asynchronous Server API is all about how the container will manage the request. But it will still hold the request and won't affect the client experience.
Quoting the Jersey documentation about the Asynchronous Server API:
Note that the use of server-side asynchronous processing model will
not improve the request processing time perceived by the client. It
will however increase the throughput of the server, by releasing the
initial request processing thread back to the I/O container while the
request may still be waiting in a queue for processing or the
processing may still be running on another dedicated thread. The
released I/O container thread can be used to accept and process new
incoming request connections.
The approaches described below won't bring any benefits to your client.
Using a custom header
You could have different URLs for sync and async methods and create a pre-matching filter, which is executed before the request matching is started.
To do it, implement ContainerRequestFilter, annotate it with #PreMatching and, based on your conditions (headers, parameters, etc), change the requested URI:
#Provider
#PreMatching
public class PreMatchingFilter implements ContainerRequestFilter {
#Override
public void filter(ContainerRequestContext requestContext) throws IOException {
if (requestContext.getHeaders().get("X-Use-Async") != null) {
requestContext.setRequestUri(yourNewURI);
}
}
}
Have a look at the ContainerRequestContext API.
Using a custom media type
I haven't tested the following solution, but it should work. You can keep the same URL for both sync and async methods, just accepting a different content type for each method.
For example:
Sync method: #Consumes("application/vnd.example.sync+text")
Async method: #Consumes("application/vnd.example.async+text")
And use the PreMatchingFilter to change the Content-Type header based on your conditions, like the following:
if (useSync) {
requestContext.getHeaders().putSingle(
HttpHeaders.CONTENT_TYPE, "application/vnd.example.sync+text");
} else {
requestContext.getHeaders().putSingle(
HttpHeaders.CONTENT_TYPE, "application/vnd.example.async+text");
}
According to the documentation, ContainerRequestContext#getHeaders() returns a mutable map with the request headers.
You could use a custom MediaType...you can for example put #Produces("simple") on your simple get method and #Produces("asynch") on your asynchronous get method. In your client you then can set the Accept Header of your call to "simple" or "asynch" depending on what you need.
I am analyzing some jersey 2.0 code and i have a question on how the following method works:
#Stateless
#Path("/mycoolstuff")
public class MyEjbResource {
…
#GET
#Asynchronous //does this mean the method executes on child thread ?
public void longRunningOperation(#Suspended AsyncResponse ar) {
final String result = executeLongRunningOperation();
ar.resume(result);
}
private String executeLongRunningOperation() { … }
}
Lets say im at a web browser and i type in www.mysite/mycoolstuff
this will execute the method but im not understanding what the asyncResponse is used for neither the #Asynchronous annotation. From the browser how would i notice its asychnronous ? what would be the difference in removing the annotation ? Also the suspended annotation after reading the documentation i'm not clear its purpose.
is the #Asynchronous annotation simply telling the program to execute this method on a new thread ? is it a convenience method for doing "new Thread(.....)" ?
Update: this annotation relieves the server of hanging onto the request processing thread. Throughput can be better. Anyway from the official docs:
Request processing on the server works by default in a synchronous processing mode, which means that a client connection of a request is processed in a single I/O container thread. Once the thread processing the request returns to the I/O container, the container can safely assume that the request processing is finished and that the client connection can be safely released including all the resources associated with the connection. This model is typically sufficient for processing of requests for which the processing resource method execution takes a relatively short time. However, in cases where a resource method execution is known to take a long time to compute the result, server-side asynchronous processing model should be used. In this model, the association between a request processing thread and client connection is broken. I/O container that handles incoming request may no longer assume that a client connection can be safely closed when a request processing thread returns. Instead a facility for explicitly suspending, resuming and closing client connections needs to be exposed. Note that the use of server-side asynchronous processing model will not improve the request processing time perceived by the client. It will however increase the throughput of the server, by releasing the initial request processing thread back to the I/O container while the request may still be waiting in a queue for processing or the processing may still be running on another dedicated thread. The released I/O container thread can be used to accept and process new incoming request connections.
#Suspended have more definite if you used it, else it will not make any difference of using it.
Let's talk about benefits of it:
#Suspended will pause/Suspend the current thread until it gets response,by default #NO_TIMEOUT no suspend timeout set. So it doesn't mean your request response (I/O)thread will get free and be available for other request.
Now Assume you want your service to be a response with some specific time, but the method you are calling from resource not guarantee the response time, then how will you manage your service response time? At that time, you can set suspend timeout for your service using #Suspended, and even provide a fall back response when time get exceed.
Below is some sample of code for setting suspend/pause timeout
public void longRunningOperation(#Suspended AsyncResponse ar) {
ar.setTimeoutHandler(customHandler);
ar.setTimeout(10, TimeUnit.SECONDS);
final String result = executeLongRunningOperation();
ar.resume(result);
}
for more details refer this
The #Suspended annotation is added before an AsyncResponse parameter on the resource method to tell the underlying web server not to expect this thread to return a response for the remote caller:
#POST
public void asyncPost(#Suspended final AsyncResponse ar, ... <args>) {
someAsyncMethodInYourServer(<args>, new AsyncMethodCallback() {
#Override
void completed(<results>) {
ar.complete(Response.ok(<results>).build());
}
#Override
void failed(Throwable t) {
ar.failed(t);
}
}
}
Rather, the AsyncResponse object is used by the thread that calls completed or failed on the callback object to return an 'ok' or throw an error to the client.
Consider using such asynchronous resources in conjunction with an async jersey client. If you're trying to implement a ReST service that exposes a fundamentally async api, these patterns allow you to project the async api through the ReST interface.
We don't create async interfaces because we have a process that takes a long time (minutes or hours) to run, but rather because we don't want our threads to ever sleep - we send the request and register a callback handler to be called later when the result is ready - from milliseconds to seconds later - in a synchronous interface, the calling thread would be sleeping during that time, rather than doing something useful. One of the fastest web servers ever written is single threaded and completely asynchronous. That thread never sleeps, and because there is only one thread, there's no context switching going on under the covers (at least within that process).
The #suspend annotation makes the caller actually wait until your done work. Lets say you have a lot of work to do on another thread. when you use jersey #suspend the caller just sits there and waits (so on a web browser they just see a spinner) until your AsyncResponse object returns data to it.
Imagine you had a really long operation you had to do and you want to do it on another thread (or multiple threads). Now we can have the user wait until we are done. Don't forget in jersey you'll need to add the " true" right in the jersey servlet definition in web.xml to get it to work.
I have a lightweight Java application exposing a web service through Jersey/Grizzly. It's fairly simple as it just sends back JSON content:
#GET
public Response status() {
CacheControl cc = new CacheControl();
cc.setMaxAge(CLIENT_EXPIRY);
cc.setPrivate(true);
ResponseBuilder builder = Response.ok(someJsonString, MediaType.APPLICATION_JSON);
builder.cacheControl(cc);
return builder.build();
}
I would like to perform server-side operations when and only when a client requests this response, but without delaying the response itself.
These other methods will not influence the response, but will use up server resources, and take a significant time. That would make the end user experience less enjoyable if I simply pasted the call in the middle of response building.
What would be a good way to monitor the endpoint activity and trigger a server-side treatment without delaying the response?
Start a new thread that will do the work, and then return the response to the client.
Consider using a thread pull, not to exhaust your server's resources.
This way the client will not wait for response and the task will be executed.
Servlet 3.0 allows the 'request' thread (or 'main' thread) to delegate long-running processing to some other thread, so as to free itself to receive more requests. Agreed.
That is, we are achieving scalability (of requests) by utilizing multi-threading.
But this requires that my 'Servlet container JVM' is capable of such processing.
What if I have a multi-tiered architecture where the 'Servlet container JVM' is only the entry point, while the logic for servicing requests lies somewhere else in some other JVM (henceforth called as 'Service JVM' in this post).
What if I want to post the incoming 'request' (or atleast the relevant attributes of the request) to a JMS queue and let the 'request' be grabbed and processed by one out of a pool of 'Service JVMs' ? Wouldnt it be better to delegate the responsibility of sending the 'response' (say as JSON) also to this Service JVM ?
I dont think 'AsyncContext' can be passed meaningfully outside of the Servlet container JVM. So, how to really delegate request-processing and response-sending, to be done by distributed services (JVMs) ?
In terms of code/pseudo-code, my question is:
#WebServlet(urlPatterns = "/AsyncServlet", asyncSupported=true)
public class AsyncServlet extends HttpServlet {
protected void doGet(HttpServletRequest request,
HttpServletResponse response)
throws ServletException, IOException {
AsyncContext asyncCtx = request.startAsync();
// Put the asyncCtx in a JMS queue so as to be picked by another
// service JVM that can really service this request.
// return back to receiving requests and dont worry abt sending a response
// The service JVM will take care of sending the appropriate response
// as it has the data necessary for the response.
}
}
One option seems to be to have Worker threads (in the Servlet container JVM) wait for responses from the Service JVMs. After a Service JVM does the actual processing, it can communicate the results (thru messages or otherwise) to the respective Worker thread, and have the Worker thread send the GET response.
I want to know if there is (am sure there shd be !) a better alternative than this, as this seems so convoluted !
Set context as async
Store context inside singleton bean
Send a jms request
Process jms request
Send jms reply
Get context for reply from the singleton bean
Send reply to client
You might want to set a timer for cleanup and you can replace jms with async one-way ejb calls
HTTP controller (Spring) gets a requests and making a blocking call. Once the blocking call is over the controls get back to the controller.
At this point I don't want to send a response back to the client but rather wait for another event(e.g., completion of some processing) to happen. As soon as that event happens I need a way to collect the data from the event and then return the HTTP response with this data.
HTTPController doSomething( HTTPRequest )
{
makeBlockingCall();
waitForEventToHappen();
collectDataFromEvent();
return HTTPResponse();
}
You wouldn't do that from within one request, because that would probably time out eventually and until then wouldn't provide feedback to the customer that something is happening.
Instead, you would immediately show a page that asynchronously (probably per AJAX) polls the server to see whether the result is already available.
So the first request will return an id that will be used in the second (AJAX) request to lookup the result.