I couldn't find anything in the GRPC documentation about this. Does GRPC expect my implementation of StreamObserver.onNext() to be non-blocking? What are the implications on GRPC if it does block (e.g. rejects new requests, queues up new requests, etc..)?
You can block if you need to block.
Since the callbacks for the RPC are considered non-thread-safe, blocking will delay other callbacks until you return. That includes setOnReadyHandler and setOnCancelHandler in ClientCallStreamObserver and ServerCallStreamObserver.
In streaming RPCs, gRPC automatically requests another message after you return from your onNext(), so if you block gRPC will avoid receiving too many more messages. gRPC will still allow some messages to be buffered, however.
Blocking has no impact on new RPCs.
Related
I would like to make client-side streaming blocking. The definition of that protocol can look like this:
rpc RecordRoute(stream Point) returns (RouteSummary) {}
As it's said in the documentation, for certain types of streaming call, it's only possible to use async stub:
a non-blocking/asynchronous stub that makes non-blocking calls to the server, where the response is returned asynchronously. You can make certain types of streaming call only using the asynchronous stub.
Then how can I make that call blocking/synchronous? Is it possible?
Blocking stub can only be used for RPCs that client sends only a single request. For client streaming calls, you can only use async stub. The generated code for blocking stub does not contain the RPC method for client-streaming or bidi-streaming methods.
If you want to avoid excessive buffering due to async requests, you can use the CallStreamObServer API to do manual flow control. With some external synchronizations such as a CountDownLatch, the async API can behave synchronously. See how gRPC's manual flow control example works.
In gRPC I would like some more information on the way the server handles requests.
Are requests executed in parallel? Or does the server spawn a new thread for each request, and execute them in parallel? Is there a way to modify this behavior? I understand that in client-streaming rpc's that message order is guaranteed.
If I send Request A followed by Request B to the same RPC, is it guaranteed that A will be executed first before B begins processing? Or are they each their own thread and executed in parallel with no guarantee that A finishes before B.
Ideally I would like to send a request to the server, the server acknowledges receipt of the request, and then the request is added to a queue to be processed sequentially, and returns a response once it's been processed. An approach I was exploring is to use an external task queue (like RabbitMQ) to queue the work done by the service but I want to know if there is a better approach.
Also -- on a somewhat related note -- does gRPC have a native retry counter mechanism? I have a particularly error-prone RPC that may have to retry up to 3 times (with an arbitrary delay between retries) before it is successful. This is something that could be implemented with RabbitMQ as well.
grpc-java passes RPCs to the service using the Executor provided by ServerBuilder.executor(Executor), or a cached thread pool if no executor is provided.
There is no ordering between simultaneous RPCs. RPCs can arrive in any order.
You could use a server-streaming RPC to allow the server to respond twice, once for acknowledgement and once for completion. You can use a oneof in the response message to allow sending the two different responses.
grpc-java as experimental retry support. gRFC A6 describes the support. The configuration is delivered to the client via service config. Retries are disabled by default, so overall you would want something like channelBuilder.defaultServiceConfig(serviceConfig).enableRetry(). You can also reference the hedging example which is very similar to retries.
For my Code it is necessary to call multiple asynchronous Requests from the same client to a Thrift Service.
So I am using a Non blocking Server and Asynchronous Clients (see the code below) to allow asynchronous Calls, which means the execution of the code continues after the first call of the "checkForPrime()" Method, which I call on the Thrift Service.
Now this seems to work with only executing one call. If I make a second asynchronous call right after, I get the following error message:
Client is currently executing another method:
Interfaces.PrimeCheck$AsyncClient$checkForPrime_call
at
org.apache.thrift.async.TAsyncClient.checkReady(TAsyncClient.java:78)
at
Interfaces.PrimeCheck$AsyncClient.checkForPrime(PrimeCheck.java:110)
at ThriftClient.main(ThriftClient.java:40)
I need a smart solution to allow for multiple calls, but it has to be from the same client. Any suggestions are welcome. Please dont hesitate if you need further information.
org.apache.thrift.protocol.TBinaryProtocol.Factory factory = new TBinaryProtocol.Factory();
TAsyncClientManager manager;
TNonblockingSocket socket;
AsyncClient client;
try {
manager = new TAsyncClientManager();
socket =new TNonblockingSocket("localhost", 4711);
client = new AsyncClient(factory, manager, socket);
client.checkForPrime(5, resultHandler);
client.checkForPrime(7, resultHandler);
Thread.sleep(100);
} catch (IOException e2) ....
to allow asynchronous Calls, which means the execution of the code continues after the first call of the "checkForPrime()" Method,
Not quite. Asynchronous only means that the call is completed asynchronously and you don't have to wait for the completion until necessary.
It does not imply that you can use the same client to do another parallel request. Some implementations may support this but the current implementation does not.
Multiple outstanding calls require some bookkeeping, otherwise you will get lost with the responses:
call 1 made --->
call 2 made --->
response arrives <----
response arrives <----
Now, what call does the first response belong to: call 1 or call 2? Hard to say, it could be either one. Without more information a multi-call client would have a hard time trying to correlate the data.
The TAsyncClientManager handles that by restricting clients to allowing only one pending call at a time.
it is necessary to call multiple asynchronous Requests from the same client
Why do you think it is necessary?
The client is only a mediator, a means of transport. If you send two emails, do you require the emails follow the exact same path across the interwebs? No, because the relevant information the other side (server) should rely on is in the message content, not in the transport level.
If, however, you need to store data at the client, you should store it in a dedicated place outside of the client instance. Either way, the fact that we deal with one or two client instances should not really matter.
I have pretty simple Jetty-based websockets server, responsible for streaming small binary messages to connect clients.
To avoid any blocking on server side I was using sendBytesByFuture method.
After increasing load from 2 clients to 20, they stop receive any data. During troubleshooting I decided to switch on synchronous send method and finally got potential reason:
java.lang.IllegalStateException: Blocking message pending 10000 for BLOCKING
at org.eclipse.jetty.websocket.common.WebSocketRemoteEndpoint.lockMsg(WebSocketRemoteEndpoint.java:130)
at org.eclipse.jetty.websocket.common.WebSocketRemoteEndpoint.sendBytes(WebSocketRemoteEndpoint.java:244)
Clients not doing any calculations upon receiving data so potentially they can't be slow joiners.
So I wondering what can I do to solve this problem?
(using Jetty 9.2.3)
If the error message occurs from a synchronous send, then you have multiple threads attempting to send messages on the same RemoteEndpoint - something that isn't allowed per the protocol. Only 1 message at a time may be sent. (There is essentially no queue for synchronous sends)
If the error message occurs from an asynchronous send, then that means you have messages sitting in a queue waiting to be sent, yet you are still attempting to write more async messages.
Try not to mix synchronous and asynchronous at the same time (it would be very easy to accidentally have output that become an invalid protocol stream)
Using Java Futures:
You'll want to use the Future objects that are provided on the return of the sendBytesByFuture() and sendStringByFuture() methods to verify that the message was actually sent or not (could have been an error), and if enough start to queue up unsent you back off on sending more messages until the remote endpoint can catch up.
Standard Future behavior and techniques apply here.
Using Jetty Callbacks:
There is also the WriteCallback behavior available in the sendBytes(ByteBuffer,WriteCallback) and sendString(String,WriteCallback) methods that would call your own code on success/error, at which you can put some logic around what you send (limit it, send it slower, queue it, filter it, drop some messages, prioritize messages, etc. whatever you need)
Using Blocking:
Or you can just use blocking sends to never have too many messages queue up.
My system consists of a "proxy" class that receives "request" packets, marshals them and sends them over the network to a server, which unmarshals them, processes, and returns some "response packet".
My "submit" method on the proxy side should block until a reply is received to the request (packets have ids for identification and referencing purposes) or until a timeout is reached.
If I was building this in early versions of Java, I would likely implement in my proxy a collection of "pending messages ids", where I would submit a message, and wait() on the corresponding id (with a timeout). When a reply was received, the handling thread would notify() on the corresponding id.
Is there a better way to achieve this using an existing library class, perhaps in java.util.concurrency?
If I went with the solution described above, what is the correct way to deal with the potential race condition where a reply arrives before wait() is invoked?
The simple way would be to have a Callable that talks to the server and returns the Response.
// does not block
Future<Response> response = executorService.submit(makeCallable(request));
// wait for the result (blocks)
Response r = response.get();
Managing the request queue, assigning threads to the requests, and notifying the client code is all hidden away by the utility classes.
The level of concurrency is controlled by the executor service.
Every network call blocks one thread in there.
For better concurrency, one could look into using java.nio as well (but since you are talking to same server for all requests, a fixed number of concurrent connections, maybe even just one, seems to be sufficient).