Develop Reference

Vert.x return results and store them from a query MongoDB

I am using Vert.x in my project, I used future() to get the results from a MongoDB query. However when I do future().result it returns "null". I want the result to be saved in the future and I will use it for other APIs. Is there any guide for me, I will be very grateful and appreciate if someone give me a solution. Thanks
router.class
rivate void getClazzById(RoutingContext rc) {
Future<JsonObject> future = Future.future();
String clazzId = rc.request().getParam("clazzId");
classService.getClazzById(clazzId, res -> {
System.out.println(res.result());
if (res.succeeded()) {
JsonObject result = res.result();
if (result != null) {
future.complete(res.result());
rc.response().setStatusCode(200).putHeader("Content-Type", "application/json; charset=utf-8")
.end(result.encodePrettily());
} else {
rc.response().setStatusCode(400).putHeader("Content-Type", "application/json; charset=utf-8")
.end(new JsonObject().put("error", "Class not found!").encodePrettily());
}
} else
rc.fail(res.cause());
});
future.setHandler(s -> {
if (s.succeeded()) {
System.out.println("sss: " +s.result()); // print: {"_id":"123", "name":"abc"}
}
else {
System.out.println("fail");
}
});
System.out.println("hhhhhh: " + future.result()); // I want to print {"_id":"123", "name":"abc"}
}
service.class
public void getClazzById(String clazzId, Handler<AsyncResult<JsonObject>> resultHandler) {
JsonObject query = new JsonObject().put("_id", clazzId);
client.findOne(Collection.CLAZZ, query, null, ar -> {
if (ar.succeeded()) {
if (ar.succeeded()) {
JsonObject result = ar.result();
if (result == null) {
resultHandler.handle(Future.failedFuture("Error"));
} else {
resultHandler.handle(Future.succeededFuture(result));
}
} else {
ar.cause();
}
}
});
}

When writing asynchronous code, you are carried to use the framework / runtime semantics and tools for communication.
You are already leveraging one of Vert.x's way of async communication, the Future - but in the wrong manner trying to retrieve its result inline.
Instead of having the Future result accessed within your method, you need to return it as your mean of communication to a caller, which would be able to set a completion handler (Handler) to it:
private Future<JsonObject> getClazzById(RoutingContext rc) {
Future<JsonObject> future = Future.future();
String clazzId = rc.request().getParam("clazzId");
classService.getClazzById(clazzId, res -> {
if (res.succeeded()) {
JsonObject result = res.result();
if (result != null) {
future.complete(res.result()); // set the retrieved result
rc.response().setStatusCode(200).putHeader("Content-Type", "application/json; charset=utf-8")
.end(result.encodePrettily());
} else {
future.complete(null); // you need to provide 'null' result to avoid caller blocking
rc.response().setStatusCode(400).putHeader("Content-Type", "application/json; charset=utf-8")
.end(new JsonObject().put("error", "Class not found!").encodePrettily());
}
} else
rc.fail(res.cause());
});
return future; // return the future to caller
}
An interested caller would be able to set a handler for Future completion as needed:
getClassById(rc).setHandler(s -> {
if (s.succeeded()) {
System.out.println("sss: " +s.result()); // print: {"_id":"123", "name":"abc"}
}
else {
System.out.println("fail");
}
});
As a side note: you are not setting the API boundaries properly in your business logic as you are trying to resolve the HTTP Response result which generally denotes the request processing end while still returning the query result to be handled in some other manner.

How to return an async result from a sync method in vertx?

I am using executeBlocking from vertx in a for loop to parallelise the processing of a result, and collating the results using a CompositeFuture. Based on all the results, I want to return some value from the method, but lambda function inside CompositeFuture's handler is not letting me do so. How to work with this usecase?
Code for reference:
public Response call() throws Exception {
List<Future> futureList = new ArrayList<>();
//services is a global variable, an arraylist of services
for(String service: services) {
Future serviceFuture = Future.future();
futureList.add(serviceFuture);
vertx.executeBlocking(implementTask(service, serviceFuture), false, asyncResult -> {
if (asyncResult.failed()) {
LOGGER.error("Task FAILED for service {}", service, asyncResult.cause());
}
});
}
CompositeFuture.all(futureList).setHandler(asyncResult -> {
if(asyncResult.succeeded()) {
LOGGER.debug("Task completed successfully");
return new Response(ResponseStatus.SUCCESS);
} else {
LOGGER.error("Task FAILED", asyncResult.cause());
return new Response(ResponseStatus.FAILED);
}
});
}

You can't do this.
Your call() method should return Future<Result> and not the Result. Then you would need to attach the callback handler on your original caller. This is the way that async methods propagate the result in Vert.x.
public Future<Response> call() throws Exception {
Promise<Response> promise = Promise.promise();
List<Future> futureList = new ArrayList<>();
//services is a global variable, an arraylist of services
for(String service: services) {
Future serviceFuture = Future.future();
futureList.add(serviceFuture);
vertx.executeBlocking(implementTask(service, serviceFuture), false, asyncResult -> {
if (asyncResult.failed()) {
LOGGER.error("Task FAILED for service {}", service, asyncResult.cause());
}
});
}
CompositeFuture.all(futureList).setHandler(asyncResult -> {
if(asyncResult.succeeded()) {
LOGGER.debug("Task completed successfully");
promise.complete(new Response(ResponseStatus.SUCCESS));
} else {
LOGGER.error("Task FAILED", asyncResult.cause());
promise.fail("Failed");
}
});
return promise.future();
}
Then the call would look like this:
object.call().onSuccess(resultHandler - > {
//Code when call succeeded
}).onFailure(failureHandler - > {
//Code when call failed
});
Note: This example is using Vert.x 4. If you are using a version of Vert.x less than 4, then the syntax is a bit different, but the point stays the same.

Thread blocked when calling dynamoDB in vert.x app

I am using vert.x and I am trying to list my DynamoDB tables. Here is how I build the DynamoDB client.
private static DynamoDbAsyncClient buildDynamoDBAsyncClient(final Vertx vertx) {
return VertxSdkClient.withVertx(DynamoDbAsyncClient.builder(), vertx.getOrCreateContext())
.build();
}
And here is the request I make
CompletableFuture<ListTablesResponse> response = client.listTables(ListTablesRequest.builder()
.build());
// Map the response to another CompletableFuture containing just the table names
CompletableFuture<List<String>> tableNames = response.thenApply(ListTablesResponse::tableNames);
// When future is complete (either successfully or in error) handle the response
tableNames.whenComplete((tables, err) -> {
if (tables != null) {
tables.forEach(System.out::println);
} else {
// Handle error
err.printStackTrace();
}
client.close();
});
tableNames.join();
I get warnings of blocked threads and then a request timeout. What am I doing wrong? Thank you in advance.

As Tim stated in the comment, tablesName.join() blocks the thread, you should let the CompletableFuture handle async.
Try this:
private Future<ListTablesResponse> listTables() {
Promise<ListTablesResponse> promise = Promise.promise();
CompletableFuture<ListTablesResponse> response = client.listTables();
response.handleAsync((result, error) -> {
if (error == null) {
promise.complete(result);
} else {
promise.fail(error);
}
return null;
});
return promise.future();
}
then in your vertx request handler:
this.listTables()
.onSuccess(listTablesResponse -> {...})
.onFailure(throwable -> {...});

Spring WebFlux - how to determine when my client has finished working

I need to call certain API with multiple query params simultaneously, in order to do that I wanted to use reactive approach. I ended up with reactive client that is able to call endpoint based on passed SearchQuery, handle pagination of that response and call for remaining pages and returns Flux<Item>. So far it works fine, however what I need to do now is to:
Collect data for all search queries and save them as initial state
Once the initial data is collected, I need to start repeating those calls in small time intervals and validate each item against initial data. Basically, I need to find new items from here.
But I'm running out of options how to solve that, I came up with probably the dirties solution ever, but I bet there are much better ways to do that.
So first of all, this is relevant code of my client
public Flux<Item> collectData(final SearchQuery query) {
final var iteration = new int[]{0};
return invoke(query, 0).expand(res ->
this.handleResponse(res, query, iteration))
.flatMap(response -> Flux.fromIterable(response.collectItems()));
}
private Mono<ApiResponse> handleResponse(final ApiResponse response, final SearchQuery searchQuery, final int[] iteration) {
return hasNextPage(response) ? invoke(searchQuery, calculateOffset(++iteration[0])) : Mono.empty();
}
private Mono<ApiResponse> invoke(final SearchQuery query, final int offset) {
final var url = offset == 0 ? query.toUrlParams() : query.toUrlParamsWithOffset(offset);
return doInvoke(url).onErrorReturn(ApiResponse.emptyResponse());
}
private Mono<ApiResponse> doInvoke(final String endpoint) {
return webClient.get()
.uri(endpoint)
.retrieve()
.bodyToMono(ApiResponse.class);
}
And here is my service that is using this client
private final Map<String, Item> initialItems = new ConcurrentHashMap<>();
void work() {
final var executorService = Executors.newSingleThreadScheduledExecutor();
queryRepository.getSearchQueries().forEach(query -> {
reactiveClient.collectData(query).subscribe(item -> initialItems.put(item.getId(), item));
});
executorService.scheduleAtFixedRate(() -> {
if(isReady()) {
queryRepository.getSearchQueries().forEach(query -> {
reactiveClient.collectData(query).subscribe(this::process);
});
}
}, 0, 3, TimeUnit.SECONDS);
}
/**
* If after 2 second sleep size of initialItems remains the same,
* that most likely means that initial population phase is over,
* and we can proceed with further data processing
**/
private boolean isReady() {
try {
final var snapshotSize = initialItems.size();
Thread.sleep(2000);
return snapshotSize == initialItems.size();
} catch (Exception e) {
return false;
}
}
I think the code speaks for itself, I just want to finish first phase, which is initial data population and then start processing all incomming data.

Unable to insert data into db via an Observable

I have the following Observable where I am expecting some DB insertions to occur upon subscribing to it.
But nothing happens, no DB inserts and same time no errors either.
But If I directly subscribe to the method that does the DB calls, the DB insert occurs as expected.
How can I fix this such that the subscription to the Observable call below will perform the DB insert?
Please advice. Thanks.
This is the Observable where no DB insert occurs and no errors. I want to change this such that the DB insertion occurs when I subscribe to this Observable.
public Observable<KafkaConsumerRecord<String, RequestObj>> apply(KafkaConsumerRecords<String, RequestObj> records) {
Observable.from(records.getDelegate().records().records("TOPIC_NAME"))
.buffer(2)
.map(this::convertToEventRequest)
.doOnNext(this::handleEventInsertions)
.doOnSubscribe(() -> System.out.println("Subscribed!"))
.subscribe(); // purposely subscribing here itself to test
return null; // even if I return this observable and subscribe at the caller, same outcome.
}
Just to test if the query works, if I were to directly subscribe to the method that does the insertion, it works as expected as follows.
Doing this in debug mode.
client.rxQueryWithParams(query, new JsonArray(params)).subscribe() // works
The following are references to see whats happening inside the convertToEventRequest and handleEventInsertions methods
private Map<String, List<?>> convertToEventRequest(Object records) {
List<ConsumerRecord<String, RequestObj>> consumerRecords = (List<ConsumerRecord<String, RequestObj>>) records;
List<AddEventRequest> addEventRequests = new ArrayList<>();
List<UpdateEventRequest> updateEventRequests = new ArrayList<>();
consumerRecords.forEach(record -> {
String eventType = new String(record.headers().headers("type").iterator().next().value(), StandardCharsets.UTF_8);
if("add".equals(eventType)) {
AddEventRequest request = AddEventRequest.builder()
.count(Integer.parseInt(new String(record.headers().headers("count").iterator().next().value(), StandardCharsets.UTF_8)))
.data(record.value())
.build();
addEventRequests.add(request);
} else {
UpdateEventRequest request = UpdateEventRequest.builder()
.id(new String(record.headers().headers("id").iterator().next().value(), StandardCharsets.UTF_8))
.status(Integer.parseInt(new String(record.headers().headers("status").iterator().next().value(), StandardCharsets.UTF_8)))
.build();
updateEventRequests.add(request);
}
});
return new HashMap<String, List<?>>() {{
put("add", addEventRequests);
put("update", updateEventRequests);
}};
}
private void handleEventInsertions(Object eventObject) {
Map<String, List<?>> eventMap = (Map<String, List<?>>) eventObject;
List<AddEventRequest> addEventRequests = (List<AddEventRequest>) eventMap.get("add");
List<UpdateEventRequest> updateEventRequests = (List<UpdateEventRequest>) eventMap.get("update");
if(addEventRequests != null && !addEventRequests.isEmpty()) {
insertAddEvents(addEventRequests);
}
if(updateEventRequests != null && !updateEventRequests.isEmpty()) {
insertUpdateEvents(updateEventRequests);
}
}
private Single<ResultSet> insertAddEvents(List<AddEventRequest> requests) {
AddEventRequest request = requests.get(0);
List<Object> params = Arrays.asList(request.getCount(), request.getData());
String query = "INSERT INTO mytable(count, data, creat_ts) " +
"VALUES (?, ?, current_timestamp)";
return client.rxQueryWithParams(query, new JsonArray(params));
}
private Single<ResultSet> insertUpdateEvents(List<UpdateEventRequest> requests) {
UpdateEventRequest request = requests.get(0);
return client.rxQueryWithParams(
"UPDATE mytable SET status=?, creat_ts=current_timestamp WHERE id=?",
new JsonArray(Arrays.asList(request.getStatus(), request.getId())));
}

Can you try to wrap it into Observable.defer?
Observable.defer(() -> Observable.from(records.getDelegate().records().records("TOPIC_NAME"))...