Develop Reference

Calling multiple asynchronous requests to Thrift Service from a single client

For my Code it is necessary to call multiple asynchronous Requests from the same client to a Thrift Service.
So I am using a Non blocking Server and Asynchronous Clients (see the code below) to allow asynchronous Calls, which means the execution of the code continues after the first call of the "checkForPrime()" Method, which I call on the Thrift Service.
Now this seems to work with only executing one call. If I make a second asynchronous call right after, I get the following error message:
Client is currently executing another method:
Interfaces.PrimeCheck$AsyncClient$checkForPrime_call
at
org.apache.thrift.async.TAsyncClient.checkReady(TAsyncClient.java:78)
at
Interfaces.PrimeCheck$AsyncClient.checkForPrime(PrimeCheck.java:110)
at ThriftClient.main(ThriftClient.java:40)
I need a smart solution to allow for multiple calls, but it has to be from the same client. Any suggestions are welcome. Please dont hesitate if you need further information.
org.apache.thrift.protocol.TBinaryProtocol.Factory factory = new TBinaryProtocol.Factory();
TAsyncClientManager manager;
TNonblockingSocket socket;
AsyncClient client;
try {
manager = new TAsyncClientManager();
socket =new TNonblockingSocket("localhost", 4711);
client = new AsyncClient(factory, manager, socket);
client.checkForPrime(5, resultHandler);
client.checkForPrime(7, resultHandler);
Thread.sleep(100);
} catch (IOException e2) ....

to allow asynchronous Calls, which means the execution of the code continues after the first call of the "checkForPrime()" Method,
Not quite. Asynchronous only means that the call is completed asynchronously and you don't have to wait for the completion until necessary.
It does not imply that you can use the same client to do another parallel request. Some implementations may support this but the current implementation does not.
Multiple outstanding calls require some bookkeeping, otherwise you will get lost with the responses:
call 1 made --->
call 2 made --->
response arrives <----
response arrives <----
Now, what call does the first response belong to: call 1 or call 2? Hard to say, it could be either one. Without more information a multi-call client would have a hard time trying to correlate the data.
The TAsyncClientManager handles that by restricting clients to allowing only one pending call at a time.
it is necessary to call multiple asynchronous Requests from the same client
Why do you think it is necessary?
The client is only a mediator, a means of transport. If you send two emails, do you require the emails follow the exact same path across the interwebs? No, because the relevant information the other side (server) should rely on is in the message content, not in the transport level.
If, however, you need to store data at the client, you should store it in a dedicated place outside of the client instance. Either way, the fact that we deal with one or two client instances should not really matter.

Java / Scala Future driven by a callback

Short Version:
How can I create a Promise<Result> which is completed on a trigger of a callback?
Long Version:
I am working on an application which deals with third-party SOAP services. A request from user delegates to multiple SOAP services simultaneously, aggregates the results and sends back to the user.
The system needs to be scalable and should allow multiple concurrent users. As each user requests ends up triggering about 10 web service calls and each call blocking for about 1 second, the system needs to be designed with non-blocking I/O.
I am using Apache CXF within Play Framework (Java) for this system. I have managed to generate the Asynchronous WS Client proxies and enable the async transport. What I am unable to figure out is how to return a Future to Play's Thread when I have delegated to multiple Web Service proxies and the results will be obtained as callbacks.
Option 1: Using async method calls returning Java Future.
As described in this scala.concurrent.Future wrapper for java.util.concurrent.Future thread, there is no way we can convert a Java Future to a Scala Future. Only way to get a result from the Future is to do Future.get() which blocks the caller. Since CXF's generated proxies return Java Future, this option is ruled out.
Option 2: Use Scala Future.
Since CXF generates the proxy interfaces, I am not sure if there is any way I can intervene and return a Scala Future (AFAIK Akka uses Scala Futures) instead of Java Future?
Option 3: Use the callback approach.
The async methods generated by CXF which return Java Future also takes a callback object which I suppose will provide a callback when result is ready. To use this approach, I will need to return a Future which will wait until I receive a callback.
I think Option 3 is most promising, although I have no ideas about how I can return a Promise which will be completed on receiving a callback. I could possibly have a thread waiting in a while(true) and waiting in between until result is available. Again, I don't know how I can go into wait without blocking the thread?
In a nutshell, I am trying to build a system which is making a lot of SOAP web service calls, where each call blocks for significant time. The system may easily run out of threads in case of lot of concurrent web service calls. I am working on finding a solution which is non-blocking I/O based which can allow many ongoing web service calls at the same time.

Option 3 looks good :) A couple of imports to start with...
import scala.concurrent.{Await, Promise}
import scala.concurrent.duration.Duration
and, just to illustrate the point, here's a mocked CXF API that takes the callback:
def fetch(url: String, callback: String => Unit) = {
callback(s"results for $url")
}
Create a promise, call API with promise as callback:
val promise = Promise[String]
fetch("http://corp/api", result => promise.success(result))
Then you can take promise.future which is an instance of Future into your Play app.
To test it, you can do this:
Await.result(promise.future, Duration.Inf)
which will block awaiting the result, at which point you should see "results for http://corp/api" in the console.

How Do Applications handle Asynchronous Responses - via Callback

I have been doing Java for a few years but I have not had much experience with Asynchronous programming.
I am working on an application that makes SOAP web service calls to some Synchronous web services and currently the implementation of my consuming application is Synchronous also ie. my applications threads block while waiting for the response.
I am trying to learn how to handle these SOAP calls in an asynchronous way - just for the hell of it but I have some high-level questions which I cant seem to find any answers to.
I am using CXF but my question is not specifically about CXF or SOAP, but higher-level, in terms of asynchronous application architecture I think.
What I want to know (working thru a scenario) - at a high level - is:
So I have a Thread (A) running in my JVM that makes a call to a remote web service
It registers a callback method and returns a Future
Thread (A) has done its bit and gets returned to its pool once it has returned the Future
The remote web service response returns and Thread (B) gets allocated and calls the callback method (which generally populates the Future with a result I believe)
Q1. I cant get my head off the blocking thread model - if Thread (A) is no longer listening to that network socket then how does the response that comes back from the remote service get allocated Thread (B) - is it simply treated as a new request coming into the server/container which then allocates a thread to service it?
Q2. Closely related to Q1 I imagine: if no Thread has the Future, or handler (with its callback method) on its stack, then how does the response from the remote web service get associated with the callback method it needs to call?
Or, in another way of asking, how does Thread B (now dealing with the response) get given a reference to the Future/Callback object?
Very sorry my question is so long - and thanks to anyone who gave their time to read through it! :)

I don't see why you'd add all this complexity using asynchronous Threading.
The way to design an asynchronous soap service:
You have one service sending out a response to a given client / clients.
Those clients work on the response given asynchronously.
When done, they would call another soap method to return their response.
The response will just be stored in a queue (e.g. a database table), without any extra logic. You'd have a "Worker" Service working on the incoming tasks. If a response is needed again another method on the other remote service would be called. The requests I would store as events in the database, which would later be asynchronously handled by an EventHandler. See
Hexagonal Architecture:
https://www.youtube.com/watch?v=fGaJHEgonKg

Your Q1 and Q2 seem to have more to do with multithreading than they have to do with asynchronous calls.
The magic of asynchronous web service calls is that you don't have to worry about multithreading to handle blocking while waiting for a response.
It's a bit unclear from the question what the specific problem statement is (i.e., what you are hoping to have your application do while blocking or rather than blocking), but here are a couple ways that you could use asynchronous web service calls that will allow you to do other work.
For the following cases, assume that the dispatch() method calls Dispatch.invokeAsync(T msg, AsyncHandler handler) and returns a Future:
1) Dispatch multiple web service requests, so that they run in parallel:
If you have multiple services to consume and they can all execute independently, dispatch them all at once and process the responses when you have received them all.
ArrayList<Future<?>> futures = new ArrayList<Future<?>>();
futures.add(serviceToConsume1.dispatch());
futures.add(serviceToConsume2.dispatch());
futures.add(serviceToConsume3.dispatch());
// now wait until all services return
for(Future f<?> : futures) {
f.get();
}
// now use responses to continue processing
2) Polling:
Future<?> f = serviceToConsume.dispatch();
while(!f.isDone()) {
// do other work here
}
// now use response to continue processing

Synchronous, Asynchronous and Command Client Requests with GWT and GAE

In designing my GWT/GAE app, it has become evident to me that my client-side (GWT) will be generating three types of requests:
Synchronous - "answer me right now! I'm important and require a real-time response!!!"
Asynchronous - "answer me when you can; I need to know the answer at some point but it's really not all that ugent."
Command - "I don't need an answer. This isn't really a request, it's just a command to do something or process something on the server-side."
My game plan is to implement my GWT code so that I can specify, for each specific server-side request (note: I've decided to go with RequestFactory over traditional GWT-RPC for reasons outside the scope of this question), which type of request it is:
SynchronousRequest - Synchronous (from above); sends a command and eagerly awaits a response that it then uses to update the client's state somehow
AsynchronousRequest - Asynchronous (from above); makes an initial request and somehow - either through polling or the GAE Channel API, is notified when the response is finally received
CommandRequest - Command (from above); makes a server-side request and does not wait for a response (even if the server fails to, or refuses to, oblige the command)
I guess my intention with SynchronousRequest is not to produce a totally blocking request, however it may block the user's ability to interact with a specific Widget or portion of the screen.
The added kicker here is this: GAE strongly enforces a timeout on all of its frontend instances (60 seconds). Backend instances have much more relaxed constraints for timeouts, threading, etc. So it is obvious to me that AsynchronousRequests and CommandRequests should be routed to backend instances so that GAE timeouts do not become an issue with them.
However, if GAE is behaving badly, or if we're hitting peak traffic, or if my code just plain sucks, I have to account for the scenario where a SynchronousRequest is made (which would have to go through a timeout-regulated frontend instance) and will timeout unless my GAE server code does something fancy. I know there is a method in the GAE API that I can call to see how many milliseconds a request has before its about to timeout; but although the name of it escapes me right now, it's what this "fancy" code would be based off of. Let's call it public static long GAE.timeLeftOnRequestInMillis() for the sake of this question.
In this scenario, I'd like to detect that a SynchronousRequest is about to timeout, and somehow dynamically convert it into an AsynchronousRequest so that it doesn't time out. Perhaps this means sending an AboutToTimeoutResponse back to the client, and force the client to decide about whether to resend as an AsynchronousRequest or just fail. Or perhaps we can just transform the SynchronousRequest into an AsynchronousRequest and push it to a queue where a backend instance will consume it, process it and return a response. I don't have any preferences when it comes to implementation, so long as the request doesn't fail or timeout because the server couldn't handle it fast enough (because of GAE-imposed regulations).
So then, here is what I'm actually asking here:
How can I wrap a RequestFactory call inside SynchronousRequest, AsynchronousRequest and CommandRequest in such a way that the RequestFactory call behaves the way each of them is intended? In other words, so that the call either partially-blocks (synchronous), can be notified/updated at some point down the road (asynchronous), or can just fire-and-forget (command)?
How can I implement my requirement to let a SynchronousRequest bypass GAE's 60-second timeout and still get processed without failing?
Please note: timeout issues are easily circumvented by re-routing things to backend instances, but backends don't/can't scale. I need scalability here as well (that's primarily why I'm on GAE in the first place!) - so I need a solution that deals with scalable frontend instances and their timeouts. Thanks in advance!

If the computation that you want GAE to do is going to take longer than 60 seconds, then don't wait for the results to be computed before sending a response. According to your problem definition, there is no way to get around this. Instead, clients should submit work orders, and wait for a notification from the server when the results are ready. Requests would consist of work orders, which might look something like this:
class ComputeDigitsOfPiWorkOrder {
// parameters for the computation
int numberOfDigitsToCompute;
// Used by the GAE app to contact the requester when results are ready.
ClientId clientId;
}
This way, your GAE app can respond as soon as the work order is saved (e.g. in Task Queue), and doesn't have to wait until it actually finishes calculating a billion digits of pi before responding. Your GWT client then waits for the result using the Channel API.
In order to give some work orders higher priority, you can use multiple task queues. If you want Task Queue work to scale automatically, you'll want to use push queues. Implementing priority using push queues is a little tricky, but you can configure high priority queues to have faster feed rate.
You could replace Channel API with some other notification solution, but that would probably be the most straightforward.

RMI - Remote Objects - Design question

I have been reading through the Java tutorial on RMI. I like the approach that is outlined here for implementing a remote interface:
http://download.oracle.com/javase/tutorial/rmi/implementing.html
What I would like to know are 2 things:
1) With regard to the executeTask method outlined in the aforementioned link, how would this design allow Remote Objects (tasks) access some sort of global state if the ComputeEngine is just calling the execute method of a Task?
2) Would this design be suitable for a multi-threaded environment?
Thanks indeed.

Ad. 1: Please note that remote client does not know anything about ComputeEngine class, only the Compute interface. Also, the server implementation might change completely, but if the interface does not change, client shouldn't notice. If you want to pass some context to the task coming from remote client, do it on interface layer:
public class ComputeEngine implements Compute {
private GlobalContext globalContext = //...
public <T> T executeTask(Task<T> t) {
return t.execute(globalContext);
}
This way each task has access to the globalContext and knows exactly what to expect from globalContext (what are the server capabilities, the context). GlobalContext would be a JavaBean or more likely some service interface.
On the client side it might look like this
Compute compute = //obtain RMI client stub somehow
compute.executeTask(new Task<String>() {
public String execute(GlobalContext globalContext) {
//Note that this code is executed on the server and
//getFoo() is implemented on the server side. We only know its interface
globalContext.getFoo();
//...
}
}
Ad. 2: It will work with multiple clients calling the service concurrently. However it is up to you to implement the server in thread-safe manner. The example from tutorial you mentioned in thread-safe, but my code using GlobalContext might not be. Please notice that several clients will use the same instance of globalContext concurrently, which might, but does not have to cause some issues. That's probably the most interesting part.
And finally remember that receiving unknown Task from remote client and running it on server is very impressive, but not quite safe.