I have a bit more complex use case in Spring Gateway which is based on WebFlux and I ran into small issue with Mono usage. Long story short, my switchIfEmpty is called even if not required. I prepared small example which allows me to reproduce this problem:
public class ReactiveTest {
#Test
void test1() {
isOdd(2)
.flatMap(this::onNotEmpty)
.switchIfEmpty(Mono.defer(this::onEmpty))
.block();
}
Mono<String> isOdd(Integer number) {
return number % 2 != 0 ? Mono.just("Yes") : Mono.empty();
}
Mono<Void> onNotEmpty(String value) {
System.out.println("Value uppercased " + value.toUpperCase());
return Mono.empty();
}
Mono<Void> onEmpty() {
System.out.println("Value not present, this shouldn't been called if value was odd");
return Mono.empty();
}
}
I hope this is pretty self-explanatory, but just to be safe:
isOdd(Integer number) may produce Mono with data or empty Mono
I expect onNotEmpty to be called only if previous Mono had data
I expect onEmpty to be called only if isOdd produced empty Mono
Unfortunatelly, both onNotEmpty and onEmpty are called all the time, regardless if I pass odd or even number to isOdd.
How can I make sure that onEmpty is called only when isOdd produced Mono.empty()?
The onNotEmpty(String value) method is always returning Mono.empty(), meaning that .switchIfEmpty(Mono.defer(this::onEmpty)) will always be call either because isOdd(2) is already an empty Mono or because onNotEmpty(String value) method was called and returned an empty Mono.
In order to avoid this, you need to change your onNotEmpty(String value) method to return something else than an empty Mono.
Additionally, please avoid using block() since this defeats the whole purpose of using Spring WebFlux by blocking the thread waiting for something to be emitted by the reactive chain.
Related
We are given a Mono, that's handling some action(say a database update), and returns a value.
We want to add that Mono(transformed) to a special list that contains actions to be completed for example during shutdown.
That mono may be eagerly subscribed after adding to the list, to start processing now, or .subscribe() might not be called meaning it will be only subscribed during shutdown.
During shutdown we can iterate on the list in the following way:
for (Mono mono : specialList) {
Object value = mono.block(); // (do something with value)
}
How to transform the original Mono such that when shutdown code executes, and Mono was previously subscribed(), the action will not be triggered again but instead it will either wait for it to complete or replay it's stored return value?
OK, looks like it is as simple as calling mono.cache(), so this is how I used it in practice
public Mono<Void> addShutdownMono(Mono<Void> mono) {
mono = mono.cache();
Mono<Void> newMono = mono.doFinally(signal -> shutdownMonos.remove(mono));
shutdownMonos.add(mono);
return newMono;
}
public Function<Mono<Void>,Mono<Void>> asShutdownAwaitable() {
return mono -> addShutdownMono(mono);
}
database.doSomeAction()
.as(asShutdownAwaitable)
.subscribe() // Or don't subscribe at all, deferring until shutdown
Here is the actual shutdown code.
It was also important to me that they execute in order of being added, if user chose not to eagerly subscribe them, that's reason for Flux.concat instead of Flux.merge.
public void shutdown() {
Flux.concat(Lists.transform(new ArrayList<>(shutdownMonos), mono -> mono.onErrorResume(err -> {
logger.error("Async exception during shutdown, ignoring", err);
return Mono.empty();
}))
).blockLast();
}
What is the different between the following two executions?
Mono.justOrEmpty(someFunction())
.doOnNext(() -> doSomeTask()).subscribe();
Mono.fromCallable(() -> someFunction())
.doOnNext(() -> doSomeTask()).subscribe();
With Mono.fromCallable, the Callable is called lazily only when the resulting Mono is subscribed to.
(you can even rewrite your snippet to Mono.fromCallable(this::someFunction) if someFunction doesn't take any parameter)
With Mono.justOrEmpty, the value is captured immediately by the operator for future emission. So in effect here the someFunction() method is called immediately at construction.
Note that both of these variants will correctly deal with someFunction() returning null (unlike Mono.just(...)). In that case, the resulting Mono will simply complete (onComplete signal without onNext).
Note also that if the method is blocking and long running, it might be an antipattern in both cases. See https://projectreactor.io/docs/core/release/reference/#faq.wrap-blocking
Basically using Mono.fromCallable() Callable emits the resulting value as Mono. Mono.justOrEmpty() only emits the specified item as Mono if it exists.
In the official docs justOrEmpty and fromCallable are described as follows:
Mono.justOrEmpty()
Create a new Mono that emits the specified item if non null otherwise only emits onComplete.
Mono.fromCallable()
Create a Mono producing its value using the provided Callable. If the Callable resolves to null, the resulting Mono completes empty.
If you need more detailed information about Mono, you can check the official documentation.
With Mono.fromCallable, someFunction will be called when a subscription is made.
private static Integer someFunction() {
System.out.println("calling someFunction");
return 1;
}
public static void main(String[] args) {
Mono<Integer> mono = Mono.fromCallable(ReactorApp2::someFunction)
.doOnNext(System.out::println);
System.out.println("Subscribing...");
mono.subscribe();
mono.subscribe();
}
/*
Subscribing...
calling someFunction
1
calling someFunction
1
*/
With Mono.justOrEmpty, someFunction will be called only once.
private static Integer someFunction() {
System.out.println("calling someFunction");
return 1;
}
public static void main(String[] args) {
Mono<Integer> mono = Mono.justOrEmpty(someFunction())
.doOnNext(System.out::println);
System.out.println("Subscribing...");
mono.subscribe();
mono.subscribe();
}
/*
calling someFunction
Subscribing...
1
1
*/
I am working on a project reactor workshop and am stuck with the following task:
/**
* TODO 5
* <p>
* For each item call received in colors flux call the {#link #simulateRemoteCall} operation.
* Timeout in case the {#link #simulateRemoteCall} does not return within 400 ms, but retry twice
* If still no response then provide "default" as a return value
*/
The problem I can't wrap my head around is that Flux never actually throws the TimeOutException! I am able to observe this in the console log:
16:05:09.759 [main] INFO Part04HandlingErrors - Received red delaying for 300
16:05:09.781 [main] INFO Part04HandlingErrors - Received black delaying for 500
16:05:09.782 [main] INFO Part04HandlingErrors - Received tan delaying for 300
I tried to play around with the order of the statements, though it didn't seem to change the behaviour. Note: In addition, I tried the overloaded variant of timeout() which accepts a default value that should be returned, if no element is emitted.
public Flux<String> timeOutWithRetry(Flux<String> colors) {
return colors
.timeout(Duration.ofMillis(400))
//.timeout(Duration.ofMillis(400), Mono.just("default"))
.retry(2)
.flatMap(this::simulateRemoteCall)
.onErrorReturn(TimeoutException.class, "default");
}
Can someone clear up why the timeout doesn't occur? I suspect that the mechanism is somehow not "bound" to the method invoked by flatMap.
For completeness: The helper method:
public Mono<String> simulateRemoteCall(String input) {
int delay = input.length() * 100;
return Mono.just(input)
.doOnNext(s -> log.info("Received {} delaying for {} ", s, delay))
.map(i -> "processed " + i)
.delayElement(Duration.of(delay, ChronoUnit.MILLIS));
}
More completeness, this is the test I am given to verify the functionality:
#Test
public void timeOutWithRetry() {
Flux<String> colors = Flux.just("red", "black", "tan");
Flux<String> results = workshop.timeOutWithRetry(colors);
StepVerifier.create(results).expectNext("processed red", "default", "processed tan").verifyComplete();
}
The answer of Martin Tarjányi is correct, but you also asked why in your code
return colors
.timeout(Duration.ofMillis(400))
//.timeout(Duration.ofMillis(400), Mono.just("default"))
.retry(2)
.flatMap(this::simulateRemoteCall)
.onErrorReturn(TimeoutException.class, "default");
no timeout occurs.
The reason is that if the elements of the colors flux are available, then invoking .timeout(Duration.ofMillis(400)) has no effect as timeout only propagates a TimeoutException if no item is emitted within the given duration of 400ms, but this is not the case in this example.
As a consequence the element is emitted and retry(2) has no effect either. Next you invoke simulateRemoteCall on the emitted element which takes some time, but which does not return an error. The result of your code is (beyond timing differences) the same as if you simply apply a map on the given flux:
public Flux<String> timeOutWithRetry(Flux<String> colors) {
return colors.map(s -> "processed " + s);
}
If you want to see a timeout on invocation of simulateRemoteCall then you must add the timeout method after this invocation.
Instead of using flatMap you could also use concatMap. The difference is whether the order should be preserved or not, i.e. whether the default values may occur out of order or not.
Using concatMap the answer looks as follows:
public Flux<String> timeOutWithRetry(Flux<String> colors) {
return colors.concatMap(
color -> simulateRemoteCall(color)
.timeout(Duration.ofMillis(400))
.retry(2)
.onErrorReturn("default"));
}
You're right in that it's the order and place of the statements that is incorrect.
Since you want to retry/timeout/error-handle the remote call, you should put these operators on the Mono of the remote call instead of the Flux.
Timeout on Flux observes the time elapsed between subsequent elements. However, when you use flatMap you get concurrency out of the box and delay between elements practically will be zero (assuming the colors Flux is sourced by an in-memory list). So this operator should not be put directly on the Flux to achieve your goal.
Retry on Flux means it resubscribes to the source in case of error, which depending on the source could result in re-processing already processed elements. Instead, you want to retry failed elements only, so it also should be put on Mono.
public Flux<String> timeOutWithRetry(Flux<String> colors) {
return colors.flatMap(color -> simulateRemoteCall(color).timeout(Duration.ofMillis(400))
.retry(2)
.onErrorReturn("default"));
}
I have been working with some Reactor Core Java, because I want to figure out if this is possible to solve one problem I currently have using this framework.
At present I have a long, executing job that takes about 40-50 minutes to complete. The method looks more or less like this:
public void doLongTask(List<Something> list){
//instructions.
for(Something sm : list){
if(condition){
executeLongOperation();
}
//instructions
if(condition){
executeLongOperation();
}
}
}
in my controller I have something like this:
#GetMapping(path = "/integersReactor", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
#ResponseBody
public Flux<Integer> getIntegersReactor(){
logger.debug("Request getIntegersReactor initialized.");
return simpleSearchService.getIntegersReactor();
}
and in the service layer I have something like this:
#Override
public Flux<Integer> getIntegersReactor(){
return Flux.range(0, Integer.MAX_VALUE);
}
this is just a placeholder that I am using as a proof of concept. My real intentions are to somehow return a Flux of some object that I will define myself, this object will have a few fields that I will use to tell the consumer the status of the job.
Now, things get somewhat complicated now because I would like to send updates as the executeLongOperation(); are executed, and somehow instead of returning a flux of Integers, return a flux of an object that uses the return of executeLongOperation();
Can this be acomplished with Flux? How can I leverage Reactor Core java to push the return values of all of the times executeLongOperation(); is executed into a reactive stream that can be passed to the controller the same way that getIntegersReactor() does it in my example?
Yes it should be possible, but since the executeLongOperation is blocking, it will need to be offset on a dedicated thread (which reduces the benefits you get from a top-to-bottom reactive implementation).
Change your doLongTask to return a Flux<Foo>, make it concatenate Monos that wrap executeLongOperation on a dedicated thread (or better yet, change the executeLongOperation itself to return a Mono<Foo> and do the wrapping internally and subscribeOn another thread internally). Something like:
public Flux<Foo> doLongTask(List<Something> list) {
return Flux.fromIterable(list)
//ensure `Something` are published on a dedicated thread on which
//we can block
.publishOn(Schedulers.elastic()) //maybe a dedicated Scheduler?
//for each `Something`, perform the work
.flatMap(sm -> {
//in case condition is false, we'll avoid long running task
Flux<Foo> work = Flux.empty();
//start declaring the work depending on conditions
if(condition) {
Mono<Foo> op = Mono.fromCallable(this::executeLongOperation);
work = conditional.concatWith(op);
}
//all other instructions should preferably be non-blocking
//but since we're on a dedicated thread at this point, it should be ok
if(condition) {
Mono<Foo> op = Mono.fromCallable(this::executeLongOperation);
work = conditional.concatWith(op);
}
//let the flatMap trigger the work
return work;
});
}
Suppose I have the following RxJava code (which accesses a DB, but the exact use case is irrelevant):
public Observable<List<DbPlaceDto>> getPlaceByStringId(final List<String> stringIds) {
return Observable.create(new Observable.OnSubscribe<List<DbPlaceDto>>() {
#Override
public void call(Subscriber<? super List<DbPlaceDto>> subscriber) {
try {
Cursor c = getPlacseDb(stringIds);
List<DbPlaceDto> dbPlaceDtoList = new ArrayList<>();
while (c.moveToNext()) {
dbPlaceDtoList.add(getDbPlaceDto(c));
}
c.close();
if (!subscriber.isUnsubscribed()) {
subscriber.onNext(dbPlaceDtoList);
subscriber.onCompleted();
}
} catch (Exception e) {
if (!subscriber.isUnsubscribed()) {
subscriber.onError(e);
}
}
}
});
}
Given this code, I have the following questions:
If someone unsubscribes from the observable returned from this method (after a previous subscription), is that operation thread-safe? So are my 'isUnsubscribed()' checks correct in this sense, regardless of scheduling?
Is there a cleaner way with less boilerplate code to check for unsubscribed states than what I'm using here? I couldn't find anything in the framework. I thought SafeSubscriber solves the issue of not forwarding events when the subscriber is unsubscribed, but apparently it does not.
is that operation thread-safe?
Yes. You are receiving an rx.Subscriber which (eventually) checks against a volatile boolean that is set to true when the subscriber's subscription is unsubscribed.
cleaner way with less boilerplate code to check for unsubscribed states
The SyncOnSubscribe and the AsyncOnSubscribe (available as an #Experimental api as of release 1.0.15) was created for this use case. They function as a safe alternative to calling Observable.create. Here is a (contrived) example of the synchronous case.
public static class FooState {
public Integer next() {
return 1;
}
public void shutdown() {
}
public FooState nextState() {
return new FooState();
}
}
public static void main(String[] args) {
OnSubscribe<Integer> sos = SyncOnSubscribe.createStateful(FooState::new,
(state, o) -> {
o.onNext(state.next());
return state.nextState();
},
state -> state.shutdown() );
Observable<Integer> obs = Observable.create(sos);
}
Note that the SyncOnSubscribe next function is not allowed to call observer.onNext more than once per iteration nor can it call into that observer concurrently. Here are a couple of links to the SyncOnSubscribe implementation and tests on the head of the 1.x branch. It's primary usage is to simplify writing observables that iterate or parsing over data synchronously and onNext downstream but doing so in a framework that supports back-pressure and checks if unsubscribed. Essentially you would create a next function which would get invoked every time the downstream operators need a new data element onNexted. Your next function can call onNext either 0 or 1 time.
The AsyncOnSubscribe is designed to play nicely with back pressure for observable sources that operate asynchronously (such as off-box calls). The arguments to your next function include the request count and your provided observable should provide an observable that fulfills data up to that requested amount. An example of this behavior would be paginated queries from an external datasource.
Previously it was a safe practice to transform your OnSubscribe to an Iterable and use Observable.from(Iterable). This implementation gets an iterator and checks subscriber.isUnsubscribed() for you.