Develop Reference

Vert.x return results and store them from a query MongoDB

I am using Vert.x in my project, I used future() to get the results from a MongoDB query. However when I do future().result it returns "null". I want the result to be saved in the future and I will use it for other APIs. Is there any guide for me, I will be very grateful and appreciate if someone give me a solution. Thanks
router.class
rivate void getClazzById(RoutingContext rc) {
Future<JsonObject> future = Future.future();
String clazzId = rc.request().getParam("clazzId");
classService.getClazzById(clazzId, res -> {
System.out.println(res.result());
if (res.succeeded()) {
JsonObject result = res.result();
if (result != null) {
future.complete(res.result());
rc.response().setStatusCode(200).putHeader("Content-Type", "application/json; charset=utf-8")
.end(result.encodePrettily());
} else {
rc.response().setStatusCode(400).putHeader("Content-Type", "application/json; charset=utf-8")
.end(new JsonObject().put("error", "Class not found!").encodePrettily());
}
} else
rc.fail(res.cause());
});
future.setHandler(s -> {
if (s.succeeded()) {
System.out.println("sss: " +s.result()); // print: {"_id":"123", "name":"abc"}
}
else {
System.out.println("fail");
}
});
System.out.println("hhhhhh: " + future.result()); // I want to print {"_id":"123", "name":"abc"}
}
service.class
public void getClazzById(String clazzId, Handler<AsyncResult<JsonObject>> resultHandler) {
JsonObject query = new JsonObject().put("_id", clazzId);
client.findOne(Collection.CLAZZ, query, null, ar -> {
if (ar.succeeded()) {
if (ar.succeeded()) {
JsonObject result = ar.result();
if (result == null) {
resultHandler.handle(Future.failedFuture("Error"));
} else {
resultHandler.handle(Future.succeededFuture(result));
}
} else {
ar.cause();
}
}
});
}

When writing asynchronous code, you are carried to use the framework / runtime semantics and tools for communication.
You are already leveraging one of Vert.x's way of async communication, the Future - but in the wrong manner trying to retrieve its result inline.
Instead of having the Future result accessed within your method, you need to return it as your mean of communication to a caller, which would be able to set a completion handler (Handler) to it:
private Future<JsonObject> getClazzById(RoutingContext rc) {
Future<JsonObject> future = Future.future();
String clazzId = rc.request().getParam("clazzId");
classService.getClazzById(clazzId, res -> {
if (res.succeeded()) {
JsonObject result = res.result();
if (result != null) {
future.complete(res.result()); // set the retrieved result
rc.response().setStatusCode(200).putHeader("Content-Type", "application/json; charset=utf-8")
.end(result.encodePrettily());
} else {
future.complete(null); // you need to provide 'null' result to avoid caller blocking
rc.response().setStatusCode(400).putHeader("Content-Type", "application/json; charset=utf-8")
.end(new JsonObject().put("error", "Class not found!").encodePrettily());
}
} else
rc.fail(res.cause());
});
return future; // return the future to caller
}
An interested caller would be able to set a handler for Future completion as needed:
getClassById(rc).setHandler(s -> {
if (s.succeeded()) {
System.out.println("sss: " +s.result()); // print: {"_id":"123", "name":"abc"}
}
else {
System.out.println("fail");
}
});
As a side note: you are not setting the API boundaries properly in your business logic as you are trying to resolve the HTTP Response result which generally denotes the request processing end while still returning the query result to be handled in some other manner.

How to return an async result from a sync method in vertx?

I am using executeBlocking from vertx in a for loop to parallelise the processing of a result, and collating the results using a CompositeFuture. Based on all the results, I want to return some value from the method, but lambda function inside CompositeFuture's handler is not letting me do so. How to work with this usecase?
Code for reference:
public Response call() throws Exception {
List<Future> futureList = new ArrayList<>();
//services is a global variable, an arraylist of services
for(String service: services) {
Future serviceFuture = Future.future();
futureList.add(serviceFuture);
vertx.executeBlocking(implementTask(service, serviceFuture), false, asyncResult -> {
if (asyncResult.failed()) {
LOGGER.error("Task FAILED for service {}", service, asyncResult.cause());
}
});
}
CompositeFuture.all(futureList).setHandler(asyncResult -> {
if(asyncResult.succeeded()) {
LOGGER.debug("Task completed successfully");
return new Response(ResponseStatus.SUCCESS);
} else {
LOGGER.error("Task FAILED", asyncResult.cause());
return new Response(ResponseStatus.FAILED);
}
});
}

You can't do this.
Your call() method should return Future<Result> and not the Result. Then you would need to attach the callback handler on your original caller. This is the way that async methods propagate the result in Vert.x.
public Future<Response> call() throws Exception {
Promise<Response> promise = Promise.promise();
List<Future> futureList = new ArrayList<>();
//services is a global variable, an arraylist of services
for(String service: services) {
Future serviceFuture = Future.future();
futureList.add(serviceFuture);
vertx.executeBlocking(implementTask(service, serviceFuture), false, asyncResult -> {
if (asyncResult.failed()) {
LOGGER.error("Task FAILED for service {}", service, asyncResult.cause());
}
});
}
CompositeFuture.all(futureList).setHandler(asyncResult -> {
if(asyncResult.succeeded()) {
LOGGER.debug("Task completed successfully");
promise.complete(new Response(ResponseStatus.SUCCESS));
} else {
LOGGER.error("Task FAILED", asyncResult.cause());
promise.fail("Failed");
}
});
return promise.future();
}
Then the call would look like this:
object.call().onSuccess(resultHandler - > {
//Code when call succeeded
}).onFailure(failureHandler - > {
//Code when call failed
});
Note: This example is using Vert.x 4. If you are using a version of Vert.x less than 4, then the syntax is a bit different, but the point stays the same.

Why pulling message have nothing google pubsub?

I have subscription VIEW_TOPIC with pull strategy. Why I cannot see any message although have 7 delay messages? I cannot figure out what am I missing. By the way, I'm running subscriber on k8s GCP. I was also add GOOGLE_APPLICATION_CREDENTIALS variable environment.
Subscriber configuration
private Subscriber buildSubscriber() {
try (SubscriptionAdminClient subscriptionAdminClient = SubscriptionAdminClient.create()) {
TopicName topicName = TopicName.of(projectId, topic);
ProjectSubscriptionName subscriptionName =
ProjectSubscriptionName.of(projectId, subscriptionId);
// Create a pull subscription with default acknowledgement deadline of 10 seconds.
// Messages not successfully acknowledged within 10 seconds will get resent by the server.
Subscription subscription =
subscriptionAdminClient.createSubscription(
subscriptionName, topicName, PushConfig.getDefaultInstance(), 10);
System.out.println("Created pull subscription: " + subscription.getName());
} catch (IOException e) {
LOGGER.error("Cannot create pull subscription");
} catch (AlreadyExistsException existsException) {
LOGGER.warn("Subscription already created");
}
ProjectSubscriptionName subscriptionName = ProjectSubscriptionName.of(projectId, subscriptionId);
LOGGER.info("Subscribe topic: " + topic + " | SubscriptionId: " + subscriptionId);
// default is 4 * num of processor
ExecutorProvider executorProvider = InstantiatingExecutorProvider.newBuilder().build();
Subscriber.Builder subscriberBuilder = Subscriber.newBuilder(subscriptionName, new MessageReceiverImpl())
.setExecutorProvider(executorProvider);
// The subscriber will pause the message stream and stop receiving more messages from the
// server if any one of the conditions is met.
FlowControlSettings flowControlSettings =
FlowControlSettings.newBuilder()
.setMaxOutstandingElementCount(100)
// the maximum size of messages the subscriber
// receives before pausing the message stream.
// 10Mib
.setMaxOutstandingRequestBytes(10L * 1024L * 1024L)
.build();
subscriberBuilder.setFlowControlSettings(flowControlSettings);
Subscriber subscriber = subscriberBuilder.build();
subscriber.addListener(new ApiService.Listener() {
#Override
public void failed(ApiService.State from, Throwable failure) {
LOGGER.error(from, failure);
}
}, MoreExecutors.directExecutor());
return subscriber;
}
Subscriber
public void startSubscribeMessage() {
LOGGER.info("Begin subscribe topic " + topic);
this.subscriber.startAsync().awaitRunning();
LOGGER.info("Subscriber start successfully!!!");
}
public class MessageReceiverImpl implements MessageReceiver {
private static final Logger LOGGER = Logger.getLogger(MessageReceiverImpl.class);
private final LogSave logSave = MatchSave.getInstance();
#Override
public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
ByteString data = message.getData();
// Get the schema encoding type.
String encoding = message.getAttributesMap().get("googclient_schemaencoding");
Req.LogReq logReqMsg = null;
try {
switch (encoding) {
case "BINARY":
logReqMsg = Req.LogReq.parseFrom(data);
break;
case "JSON":
Req.LogReq.Builder msgBuilder = Req.LogReq.newBuilder();
JsonFormat.parser().merge(data.toStringUtf8(), msgBuilder);
logReqMsg = msgBuilder.build();
break;
}
LOGGER.info((JsonFormat.printer().omittingInsignificantWhitespace().print(logReqMsg)));
logSave.addLogMsg(battleLogMsg);
} catch (InvalidProtocolBufferException e) {
e.printStackTrace();
}
consumer.ack();
}
}
With Req.LogReq is a proto message. My dependency:
// google cloud
implementation platform('com.google.cloud:libraries-bom:22.0.0')
implementation 'com.google.cloud:google-cloud-pubsub'
implementation group: 'com.google.protobuf', name: 'protobuf-java-util', version: '3.17.2'
And the call function logSave.addLogMsg(battleLogMsg); is add message to CopyOnWriteArrayList

RxJava valve use case

Is there an operator in RxJava, an external library or a way I'm missing to create a flowable/observable that recieves a function that controls the emission of data, like a valve?
I have a huge json file I need to process but I have to get a portion of the file, a list of entities, process it and then get another portion, I have tried using windows(), buffer() but the BiFunction I pass to Flowable.generate() keeps executing after I recieved the first list and I haven't finished processing it. I also tried FlowableTransformers.valve() from hu.akarnokd.rxjava3.operators but it just piles up the items before the flatMap() function that process the list
private Flowable<T> flowable(InputStream inputStream) {
return Flowable.generate(() -> jsonFactory.createParser(new GZIPInputStream(inputStream)), (jsonParser, emitter) -> {
final var token = jsonParser.nextToken();
if (token == null) {
emitter.onComplete();
}
if (JsonToken.START_ARRAY.equals(token) || JsonToken.END_ARRAY.equals(token)) {
return jsonParser;
}
if (JsonToken.START_OBJECT.equals(token)) {
emitter.onNext(reader.readValue(jsonParser));
}
return jsonParser;
}, JsonParser::close);
}
Edit: I need to control de emission of items to don't overload the memory and the function that process the data, because that function reads and writes to database, also the processing needs to be sequentially. The function that process the data it's not entirely mine and it's written in RxJava and it's expected that I use Rx.
I managed to solve it like this but if there is another way let me know please:
public static <T> Flowable<T> flowable(InputStream inputStream, JsonFactory jsonFactory, ObjectReader reader, Supplier<Boolean> booleanSupplier) {
return Flowable.generate(() -> jsonFactory.createParser(new GZIPInputStream(inputStream)), (jsonParser, emitter) -> {
if (booleanSupplier.get()) {
final var token = jsonParser.nextToken();
if (token == null) {
emitter.onComplete();
}
if (JsonToken.START_ARRAY.equals(token) || JsonToken.END_ARRAY.equals(token)) {
return jsonParser;
}
if (JsonToken.START_OBJECT.equals(token)) {
emitter.onNext(reader.readValue(jsonParser));
}
}
return jsonParser;
}, JsonParser::close);
}
Edit2: This is one of the ways I'm currently consuming the function
public Flowable<List<T>> paging(Function<List<T>, Single<List<T>>> function) {
final var atomicInteger = new AtomicInteger(0);
final var atomicBoolean = new AtomicBoolean(true);
return flowable(inputStream, jsonFactory, reader, atomicBoolean::get)
.buffer(pageSize)
.flatMapSingle(list -> {
final var counter = atomicInteger.addAndGet(1);
if (counter == numberOfPages) {
atomicBoolean.set(false);
}
return function.apply(list)
.doFinally(() -> {
if (atomicInteger.get() == numberOfPages) {
atomicInteger.set(0);
atomicBoolean.set(true);
}
});
});
}

Managed to solve it like this
public static Flowable<Object> flowable(JsonParser jsonParser, ObjectReader reader, PublishProcessor<Boolean> valve) {
return Flowable.defer(() -> {
final var token = jsonParser.nextToken();
if (token == null) {
return Completable.fromAction(jsonParser::close)
.doOnError(Throwable::printStackTrace)
.onErrorComplete()
.andThen(Flowable.empty());
}
if (JsonToken.START_OBJECT.equals(token)) {
final var value = reader.readValue(jsonParser);
final var just = Flowable.just(value).compose(FlowableTransformers.valve(valve, true));
return Flowable.concat(just, flowable(jsonParser, reader, valve));
}
return flowable(jsonParser, reader, valve);
});
}

akka websocket with java, counting clients number, sending message to client

I'm following the akka java websocket tutorial in attempt to create a websocket server. I want to implement 2 extra features:
Being able to display the number of connected clients, but the result
is always 0 or 1 , even when I know I have 100's concurrently
connected clients.
Websocket communication is biDirectional. Currently the server only respond with a message when client sends a message. How do I initiate sending a message from server to client?
Here's original akka java server example code with minimum modification of my client counting implementation:
public class websocketServer {
private static AtomicInteger connections = new AtomicInteger(0);//connected clients count.
public static class MyTimerTask extends TimerTask {
//called every second to display number of connected clients.
#Override
public void run() {
System.out.println("Conncurrent connections: " + connections);
}
}
//#websocket-handling
public static HttpResponse handleRequest(HttpRequest request) {
HttpResponse result;
connections.incrementAndGet();
if (request.getUri().path().equals("/greeter")) {
final Flow<Message, Message, NotUsed> greeterFlow = greeter();
result = WebSocket.handleWebSocketRequestWith(request, greeterFlow);
} else {
result = HttpResponse.create().withStatus(413);
}
connections.decrementAndGet();
return result;
}
public static void main(String[] args) throws Exception {
ActorSystem system = ActorSystem.create();
TimerTask timerTask = new MyTimerTask();
Timer timer = new Timer(true);
timer.scheduleAtFixedRate(timerTask, 0, 1000);
try {
final Materializer materializer = ActorMaterializer.create(system);
final Function<HttpRequest, HttpResponse> handler = request -> handleRequest(request);
CompletionStage<ServerBinding> serverBindingFuture =
Http.get(system).bindAndHandleSync(
handler, ConnectHttp.toHost("****", 1183), materializer);
// will throw if binding fails
serverBindingFuture.toCompletableFuture().get(1, TimeUnit.SECONDS);
System.out.println("Press ENTER to stop.");
new BufferedReader(new InputStreamReader(System.in)).readLine();
timer.cancel();
} catch (Exception e){
e.printStackTrace();
}
finally {
system.terminate();
}
}
//#websocket-handler
/**
* A handler that treats incoming messages as a name,
* and responds with a greeting to that name
*/
public static Flow<Message, Message, NotUsed> greeter() {
return
Flow.<Message>create()
.collect(new JavaPartialFunction<Message, Message>() {
#Override
public Message apply(Message msg, boolean isCheck) throws Exception {
if (isCheck) {
if (msg.isText()) {
return null;
} else {
throw noMatch();
}
} else {
return handleTextMessage(msg.asTextMessage());
}
}
});
}
public static TextMessage handleTextMessage(TextMessage msg) {
if (msg.isStrict()) // optimization that directly creates a simple response...
{
return TextMessage.create("Hello " + msg.getStrictText());
} else // ... this would suffice to handle all text messages in a streaming fashion
{
return TextMessage.create(Source.single("Hello ").concat(msg.getStreamedText()));
}
}
//#websocket-handler
}

Addressing your 2 bullet points below:
1 - you need to attach your metrics to the Message flow - and not to the HttpRequest flow - to effectively count the active connections. You can do this by using watchTermination. Code example for the handleRequest method below
public static HttpResponse handleRequest(HttpRequest request) {
HttpResponse result;
if (request.getUri().path().equals("/greeter")) {
final Flow<Message, Message, NotUsed> greeterFlow = greeter().watchTermination((nu, cd) -> {
connections.incrementAndGet();
cd.whenComplete((done, throwable) -> connections.decrementAndGet());
return nu;
});
result = WebSocket.handleWebSocketRequestWith(request, greeterFlow);
} else {
result = HttpResponse.create().withStatus(413);
}
return result;
}
2 - for the server to independently send messages you could create its Message Flow using Flow.fromSinkAndSource. Example below (this will only send one message):
public static Flow<Message, Message, NotUsed> greeter() {
return Flow.fromSinkAndSource(Sink.ignore(),
Source.single(new akka.http.scaladsl.model.ws.TextMessage.Strict("Hello!"))
);
}

In the handleRequest method you increment and then decrement the counter connections, so at the end the value is always 0.
public static HttpResponse handleRequest(HttpRequest request) {
...
connections.incrementAndGet();
...
connections.decrementAndGet();
return result;
}