Develop Reference

Flux does not wait for completion of elements before 'then'

I am failing to understand the issue and I am not sure what am I doing wrong.
I want to wait for Flux to end and then return Mono of serverResponse
I have attached the code snippet, the doOnNext will populate the categoryIdToPrintRepository.
I have looked around on how to return mono after flux ends and found the 'then' but still the 'then' method is being executed way before the onNextSite is being processed, this results with the error:
java.lang.IllegalArgumentException: 'producer' type is unknown to ReactiveAdapterRegistry
What am I doing wrong?
public Mono<ServerResponse> retrieveCatalog(ServerRequest ignored) {
return Mono.just("start").flatMap(id ->
Flux.fromIterable(appSettings.getSites())
.subscribeOn(ForkJoinPoolScheduler.create("SiteCatalogScheduler"))
.doOnNext(this::onNextSite)
.then(Mono.from(ServerResponse.ok().body(categoryIdToPrintRepository.getSortedTreeValues(), String.class))));
}
private void onNextSite(Integer siteId) {
IntStream.range(1, appSettings.getCatalogMaxValue()).parallel().forEach(catalogId -> {
Optional<SiteCatalogCategoryDTO> cacheData =
siteCatalogCacheUseCaseService.getSiteCatalogResponseFromCache(siteId, catalogId);
cacheData.ifPresentOrElse(siteCatalogCategoryDTO -> {/*do nothing already exist in cache*/},
() -> {
Mono<SiteCatalogCategoryDTO> catalogCategoryDTOMono = WebClient.create(getUri(siteId, catalogId))
.get().retrieve().bodyToMono(SiteCatalogCategoryDTO.class);
catalogCategoryDTOMono.subscribe(siteCatalogCategoryDTO ->
handleSiteServerResponse(siteCatalogCategoryDTO, siteId, catalogId));
});
});
}
private void handleSiteServerResponse(SiteCatalogCategoryDTO siteCatalogCategoryDTO,
int siteId, int catalogId) {
if (siteCatalogCategoryDTO.getResponseStatus().equals(ResponseStatus.SUCCESS))
Flux.fromIterable(siteCatalogCategoryDTO.getMappingList())
.subscribe(mapSCC -> {
categoryIdToPrintRepository.insertIntoTree(mapSCC.getCategoryId(),
"Site " + siteId + " - Catalog " + catalogId + " is mapped to category " + "\"" +
mapSCC.getCategoryName() + "\" (" + mapSCC.getCategoryId() + ")");
siteCatalogCacheUseCaseService.insertIntoSiteCatalogCache(siteId, catalogId, siteCatalogCategoryDTO);
});
}

You are doing several things wrong you should not subscribe in your application, and you are having void methods, which should not be used in reactive programming unless in specific places.
here is some example code:
// Nothing will happen, we are not returning anything, we can't subscribe
private void doSomething() {
Mono.just("Foo");
}
// complier error
doSomething().subscribe( ... );
Your application is a publisher the calling client, is the subscriber, thats why we return a Mono or a Flux out to the calling client, they subscribe.
You have solved it this way:
private void doSomething() {
Mono.just("Foo").subscribe( ... );
}
doSomething();
Now you are subscribing to yourself to get things running, this is not the correct way, as mentioned before, the calling client is the subscriber, not you.
Correct way:
private Mono<String> doSomething() {
return Mono.just("Foo");
}
// This is returned out to the calling client, they subscribe
return doSomething();
As a Mono/Flux completes, it will emit a signal, this signal will trigger the next and the next and the next in the chain.
So my opinion of what you need to do is the following:
Remove all subscribes, if you want to do things there are functions like, flatmap, map, doOnSuccess etc. keep the chain instact all the way out to the client.
Remove all void functions, make sure they return a Flux or a Mono and if you want to not return something return a Mono<Void> by using the Mono.empty() function so that the chain will be complete.
As soon as you use a Mono/Flux you need to handle the return so that others can chain on.
Update:
In order for then to trigger, you must return something, it will return when the previous mono/flux completes.
example:
private Flux<String> doSomething() {
return Flux.just("Foo", "Bar", "FooBar")
.doOnNext(string -> {
return // return something
});
}
// Ignore what was return from doSomething and return something else when the flux has completed (so only trigger on the completed signal from the flux)
return doSomething().then( ... );

How do I conditionally chain webclient calls in spring webflux/webclient

I am trying to achieve the following scenario using WebClient. It is trivial using RestTemplate, but I can't do it anymore.
Relevant parts of a Spring controller in pseudo-java code:
Mono<T1> t1 = webClient.get()...retrieve()...;
Mono<T2> t2;
if (t1.getResult().getValue() > 0) {
t2 = webClient.get().buildUsing(t1.getResult().getValue())...retrieve()...);
} else {
t2 = Mono.empty();
}
return(Mono.zip(t1, t2, mergeFunction));
I am not asking how to use Webflux. I can also add error handling myself. My problem is how to pass data to the second call if the first call is successful and where to merge results of both calls one of which may or may not happen. The task is absolutely trivial if I could use RestTemplate.
There is a question with a very similar title, but it was not answered.

I think zipWhen fits well for this purpose. zipWhen waits for the result from first mono and then combines both results into a Tuple2
WebClient.builder().baseUrl("https://jsonplaceholder.typicode.com/todos/1")
.build()
.get()
.retrieve()
.bodyToMono(User.class)
.zipWhen(r -> {
if (r.getId() == 1) {
return WebClient.builder().baseUrl("https://jsonplaceholder.typicode.com/todos/2")
.build()
.get()
.retrieve()
.bodyToMono(User.class);
} else {
return Mono.empty();
}
});
The result is a Mono<Tuple2<T, T2>> holding both values.

as far as I could understand your problem, this is my reactive solution to this:
private static Mono<String> mono() {
Mono<Integer> t1 = Mono.just(0);
return t1.flatMap(outerResult -> outerResult > 0
? Mono.just("VALUE").map(innerResult -> outerResult + "" + innerResult)
: Mono.just(outerResult.toString())
);
}
So what's happening here:
With .flatMap you subscribe to a new Mono and take the result of that.
Inside the lambda of the .flatMap you still have the result of your t1, so you can use .map on t2, if you need to subscribe, or just do whatever you need to do with the result of t1 to bring it to the wanted return value.

Nesting CompletionStages in Java to make inner blocks run before outer blocks

I've written a method like below:
public static CompletionStage<Tuple2<ObjectNode, String>> calculateTemplateTreeAndKeys(
String content,
RequestContext context,
MetricsClient metricsClient,
JdbcSession jdbcSession) {
AtomicReference<ObjectNode> templateTreeHolder = new AtomicReference<>();
templateTreeHolder.set(Json.rootNode());
return getTemplateIds(context, metricsClient, jdbcSession, content)
.thenCompose(
templateIds -> {
templateIds.map(
id ->
// do something and return CompletionStage<String>
.thenAccept(
tree -> {
templateTreeHolder.set(
(ObjectNode)
templateTreeHolder.get().set(id, Json.readTree(tree)));
System.out.println(
"From inner function: " + templateTreeHolder.get());
}));
return CompletableFuture.completedFuture(NotUsed.getInstance());
})
.thenApply(
notUsed -> {
String includedTemplateIdsStr =
getKeysFromTemplateTree(templateTreeHolder.get()).toJavaList().toString();
System.out.println("From outer function: " + templateTreeHolder.get());
return Tuple.of(templateTreeHolder.get(), includedTemplateIdsStr);
});
I expect the inner block to process and update templateTreeHolder before
.thenApply is invoked, so that templateTreeHolder would hold correct data to return. But, .thenApply block is processing before the inner .thenAccept block.
From the console output sequence:
From outer function: {}
From inner function: {"f9406341-c62a-411a-9389-00a62bd63629":{}}
I'm not sure what I'm doing wrong on chaining CompletionStages, kindly advise me how can I make sure that the inner block completes before the outer block?

Your function passed to thenCompose is returning an already complete future, i.e. return CompletableFuture.completedFuture(NotUsed.getInstance()); which allows the dependent stages to proceed immediately. This seem to conflict with the evaluation of the function passed to templateIds.map(…), which happens asynchronously, apparently.
Generally, you should avoid such mixture of completion stages and dependencies to side effects, especially when their asynchronous evaluation is not modeled as prerequisite completion stage.
But you can work-around this, if you have no other choice:
return getTemplateIds(context, metricsClient, jdbcSession, content)
.thenCompose(
templateIds -> {
// create an initially uncompleted stage
CompletableFuture<Object> subStage = new CompletableFuture<>();
templateIds.map(
id ->
// do something and return CompletionStage<String>
.thenAccept(
tree -> {
templateTreeHolder.set(
(ObjectNode)
templateTreeHolder.get().set(id, Json.readTree(tree)));
System.out.println(
"From inner function: " + templateTreeHolder.get());
// complete when all work has been done
subStage.complete(null);
}));
// use this stage for dependent actions
return subStage;
})
.thenApply(
notUsed -> {
String includedTemplateIdsStr =
getKeysFromTemplateTree(templateTreeHolder.get()).toJavaList().toString();
System.out.println("From outer function: " + templateTreeHolder.get());
return Tuple.of(templateTreeHolder.get(), includedTemplateIdsStr);
});
In the code above, the future will never get completed if your action fails with an exception before the completion attempt. The general pattern would be like this:
CompletableFuture<Type> stage = new CompletableFuture<>();
…
try {
code that will eventually call complete on stage
}
catch(Throwable t) {
stage.completeExceptionally(t);
}
But, of course, it will get a bit more complicated when the code supposed to complete the stage also bears asynchronous processing, so you have to guard the code trying to submit the actual completion code, as well as the actual completion code.
So, a more elaborated version of the inner code would look like:
CompletableFuture<Object> subStage = new CompletableFuture<>();
try {
templateIds.map(
id ->
// do something and return CompletionStage<String>
.thenAccept(
tree -> {
templateTreeHolder.set(
(ObjectNode)
templateTreeHolder.get().set(id, Json.readTree(tree)));
System.out.println(
"From inner function: " + templateTreeHolder.get());
})
.whenComplete((v,t) -> {
// complete when all work has been done
if(t != null) subStage.completeExceptionally(t);
else subStage.complete(v);
}));
} catch(Throwable t) {
subStage.completeExceptionally(t);
}
// use this stage for dependent actions
return subStage;
(perhaps, the “do something and return CompletionStage” has to be guarded with try { … } catch(Throwable t) { subStage.completeExceptionally(t); } too)

rxjava2 - if else on Maybe

I am looking for what is the recommended practice in rxjava2 to handle a case where one flowable leads to conditional behaviors.
More concretely, I have a Maybe<String> for which I want to Update the String on the database if the String exists or, if it doesn't exists I want to create a new String and save it on the database.
I thought of the below but obviously it is not what I am looking for:
Maybe<String> source = Maybe.just(new String("foo")); //oversimplified source
source.switchIfEmpty(Maybe.just(new String("bar"))).subscribe(result ->
System.out.println("save to database "+result));
source.subscribe(result -> System.out.println("update result "+result));
The above obviously produces
save to database foo
update result foo
I tried also the below which gives the expected result but still feel it's... weird.
Maybe<String> source = Maybe.just(new String("foo")); //oversimplified source
source.switchIfEmpty(Maybe.just(new String("bar")).doOnSuccess(result ->
System.out.println("save to database "+result))).subscribe();
source.doOnSuccess(result -> System.out.println("update result "+result)).subscribe();
How can I have an action for when the result exists and when it doesn't exists? How is that use case supposed to be handled in rxjava2?
Update 01
I tried the below and it looks cleaner than what I came up with above. Note sure it is recommended rxjava2 practice however...
Maybe.just(new String("foo"))
.map(value -> Optional.of(value))
.defaultIfEmpty(Optional.empty())
.subscribe(result -> {
if(result.isPresent()) {
System.out.println("update result "+result);
}
else {
System.out.println("save to database "+"bar");
}
});

You have the isEmpty() operator that will return you Boolean if the Maybe source is empty or not, and then you can flatMap it and write a if else statement depending on that Boolean

This is a common pattern in our code as well, though in our case the choices are themselves async. You can't get quite the right semantic by simply composing flatMapX and switchIfEmpty (in either order), so I am curious why this isn't part of the API.
Here's what we're doing for now (this example for when the 2 options are both Completables, we have similar things for the other types as well):
public static <T> Completable flatMapCompletable(Maybe<T> target,
#ClosureParams(FirstParam.FirstGenericType.class)
Closure<? extends CompletableSource> completableSupplier,
Supplier<CompletableSource> emptySupplier) {
Maybe<T> result = target.cache();
return result.isEmpty().flatMapCompletable(empty -> {
if (empty) {
return emptySupplier.get();
} else {
return result.flatMapCompletable(completableSupplier::call);
}
});
}
We're using Groovy, so we package these up as extension methods. I'm not thrilled with the need to use cache() so I'm wondering if there is a better alternative. From looking at the code, an operator which basically combines flatMapX and switch looks like it wouldn't be too hard (but I feel like I'm missing something).

Try something like this. checkDB can return a Maybe or Single or whatever which emits either an optional or a wrapper Object.
checkDB(String)
.flatMap(s -> {
if (s.isPresent()) {
return updateDB(s.get());
} else {
return insertDB("new String");
}
})

There is an solution using the flatMap call with 3 params
fun addOrUpdate(message: LocalMessage): Single<LocalMessage> {
return getById(message.id) // returns Maybe
.flatMap(
Function {
update(message) // onSuccess update call returns Single
},
Function {
Single.error(it) // onError
},
Callable {
add(message) // onComplete add call returns Single
}
)
}
}
Or shorter version
fun addOrUpdate(message: LocalMessage): Single<LocalMessage> {
return getById(message.id) // returns Maybe
.flatMap(
{
update(message) // onSuccess update call returns Single
},
{
Single.error(it) // onError
},
{
add(message) // onComplete add call returns Single
}
)
}
}

How do you access completed futures passed to CompletableFuture allOf?

I am trying to get a grip of Java 8 CompletableFuture. How can I join these to person and return them after "allOf". The code under is not working but gives you an idea of what I have tried.
In javascript ES6 i would do
Promise.all([p1, p2]).then(function(persons) {
console.log(persons[0]); // p1 return value
console.log(persons[1]); // p2 return value
});
My efforts in Java so far
public class Person {
private final String name;
public Person(String name) {
this.name = name;
}
public String getName() {
return name;
}
}
#Test
public void combinePersons() throws ExecutionException, InterruptedException {
CompletableFuture<Person> p1 = CompletableFuture.supplyAsync(() -> {
return new Person("p1");
});
CompletableFuture<Person> p2 = CompletableFuture.supplyAsync(() -> {
return new Person("p1");
});
CompletableFuture.allOf(p1, p2).thenAccept(it -> System.out.println(it));
}

The CompletableFuture#allOf method does not expose the collection of completed CompletableFuture instances that were passed to it.
Returns a new CompletableFuture that is completed when all of the
given CompletableFutures complete. If any of the given
CompletableFutures complete exceptionally, then the returned
CompletableFuture also does so, with a CompletionException holding
this exception as its cause. Otherwise, the results, if any, of the
given CompletableFutures are not reflected in the returned
CompletableFuture, but may be obtained by inspecting them
individually. If no CompletableFutures are provided, returns a
CompletableFuture completed with the value null.
Note that allOf also considers futures that were completed exceptionally as completed. So you won't always have a Person to work with. You might actually have an exception/throwable.
If you know the amount of CompletableFutures you're working with, use them directly
CompletableFuture.allOf(p1, p2).thenAccept(it -> {
Person person1 = p1.join();
Person person2 = p2.join();
});
If you don't know how many you have (you're working with an array or list), just capture the array you pass to allOf
// make sure not to change the contents of this array
CompletableFuture<Person>[] persons = new CompletableFuture[] { p1, p2 };
CompletableFuture.allOf(persons).thenAccept(ignore -> {
for (int i = 0; i < persons.length; i++ ) {
Person current = persons[i].join();
}
});
If you wanted your combinePersons method (ignoring it's a #Test for now) to return a Person[] containing all the Person objects from the completed futures, you could do
#Test
public Person[] combinePersons() throws Exception {
CompletableFuture<Person> p1 = CompletableFuture.supplyAsync(() -> {
return new Person("p1");
});
CompletableFuture<Person> p2 = CompletableFuture.supplyAsync(() -> {
return new Person("p1");
});
// make sure not to change the contents of this array
CompletableFuture<Person>[] persons = new CompletableFuture[] { p1, p2 };
// this will throw an exception if any of the futures complete exceptionally
CompletableFuture.allOf(persons).join();
return Arrays.stream(persons).map(CompletableFuture::join).toArray(Person[]::new);
}

As pointed out in #Sotirios Delimanolis's answer, the CompletableFuture is not exception friendly, which means we cannot get the result of all futures easily when exceptions happen in one or several of the futures.
As OP's question is not limited to the successful situation, in this answer, I want to make an addition to this imperfect of CompletableFuture.
If we however want to know the results of all the futures, we can handle this separately when defining futures. For example:
CompletableFuture<Person> p1 = CompletableFuture.supplyAsync(() -> {
return new Person("p1");
});
p1.thenAccept(person -> {
// handle successful future
// for example: add p1 to success operation list
});
p1.exceptionally((exception) -> {
// handle fail future
// for example, log the parameter (not exist in this case) to create the person
System.out.println("Exception happen in creating p1");
return null;
});
and then after calling CompletableFuture.allOf(persons).join() (Please also pay attention to the exception handling here), we can figure out which future succeeds and which future fails.
Hope this simple tip can help newcomers write solid code in real-world business systems.

CompletableFuture.allOf(p1, p2); // .get can be ignored
List<Person> res =
List.of(p1, p2) // list of futures
.stream()
.map(future -> {
System.out.println("future " + future.join());
return future.join();
})
.collect(Collectors.toList());
Or you can fetch the values individually, using p1.get() and p2.get()