I need to combine the results from two reactive Publishers - Mono and Flux. I tried to do it with zip and join functions, but I was not able to fulfil two specific conditions:
result should contain as many element as Flux emits, but corresponding Mono source should be called only once (this condition alone can be implemented with join)
when Flux is empty, then chain should complete without waiting for Mono element
The solution for the first condition is presented in the Combine Mono with Flux entry (pasted below). But I was not able to achieve second condition without blocking the chain - and that I would like to avoid.
Flux<Integer> flux = Flux.concat(Mono.just(1).delayElement(Duration.ofMillis(100)),
Mono.just(2).delayElement(Duration.ofMillis(500))).log();
Mono<String> mono = Mono.just("a").delayElement(Duration.ofMillis(50)).log();
List<String> list = flux.join(mono, (v1) -> Flux.never(), (v2) -> Flux.never(), (x, y) -> {
return x + y;
}).collectList().block();
System.out.println(list);
If you want to cancel the entire operation if Flux is empty you could do the below
Flux<Integer> flux = Flux.concat(Mono.just(1).delayElement(Duration.ofMillis(100)),
Mono.just(2).delayElement(Duration.ofMillis(500))).log();
//Uncomment below and comment out above statement for empty flux
//Flux<Integer> flux = Flux.empty();
Mono<String> mono = Mono.just("a").delayElement(Duration.ofMillis(5000)).log();
//Throw exception if flux is empty
flux = flux.switchIfEmpty(Flux.error(IllegalStateException::new));
List<String> list = flux
.join(mono, s -> Flux.never() , s -> Flux.never(), (x, y) -> x + y)
//Catch exception and return nothing
.onErrorResume(s -> Flux.empty())
.collectList().block();
System.out.println(list);
You could do something like the following if you want Mono to complete but don't want join to hang
DirectProcessor<Integer> processor = DirectProcessor.create();
//Could omit sink, and use processor::onComplete in place of sink::complete
//But typically recommended as provides better thread safety
FluxSink<Integer> sink = processor.serialize().sink();
Flux<Integer> flux = Flux.concat(Mono.just(1).delayElement(Duration.ofMillis(100)),
Mono.just(2).delayElement(Duration.ofMillis(500))).log();
//Uncomment below and comment out above statement for empty flux
//Flux<Integer> flux = Flux.empty();
Mono<String> mono = Mono.just("a").delayElement(Duration.ofMillis(5000)).log();
List<String> list = flux
.doOnComplete(sink::complete)
.join(mono, s -> processor , s -> processor, (x, y) -> x + y).collectList().block();
System.out.println(list);
Related
I am doing a spring webflux reactive code
.flatMap(r -> save1)
.flatMapMany(r -> save2) //save a flux of elements
.flatMap(r -> save3) //need r.getId
.subscribe();
I want to control the emission of events and only save 3 event for first emitted element of the previous chain (flatmapMany) instead of N times.
So in resume, i need to save N elements and catch the first save and ignore the others. I need to result of save2 to pass argument for save3 method.
thanks,
If I understood the problem correctly you can use .next() to "convert" Flux into Mono by taking the first emitted element and then cancel the subscription.
.flatMap(r -> save1)
.flatMapMany(r -> save2) //save a flux of elements
.next()
.flatMap(r -> save3) //need r.getId
Here is a test for the above flow that shows that only "2.1" was handled in save3
#Test
void testFlow() {
var flow = Mono.just("1")
.flatMap(r -> save1(r))
.flatMapMany(r -> save2(r))
.next()
.flatMap(r -> save3(r));
StepVerifier.create(flow)
.expectNext("2.1")
.verifyComplete();
}
where
private Mono<String> save3(String r) {
return Mono.just(r);
}
private Flux<String> save2(String r) {
return Flux.just("2.1", "2.2", "2.3");
}
private Mono<String> save1(String r) {
return Mono.just(r);
}
but without a subscribe() it will not execute the entire chain (save1, save2) , right or i'm doing confusion?
Imagine that i have 3 mono objects to save on save2
-> save 2.1
-> save 2.2
-> save 2.3
after 2.1 save operation (first element) -> get response and save only once
-> save 3
instead of save 3 times
-> save 3
-> save 3
-> save 3
So ... the idea is stop chain propagation to not duplicate saved objects in DB.
Java version : 11
I have a List, which contains many sublist and for each sublist I want to perform certain transformation/operations.
I want to perform this operation in non-blocking asynchronous fashion, so I am using CompletableFuture.
This is my operation:
public static List<String> convertBusinessObjectJson(List<BusinessObject> businessObjList) {
List<Either> eitherValueOrException = {//omitted logic to convert to json}
return eitherValueOrException;
}
It returns a List of Either Objects, where Either holds, either runtime exception thrown by conversion logic or String result when conversion is successful.
This is my caller code:
mainList.forEach(sublist -> {
CompletableFuture<List<Either>> listCompletableFuture = CompletableFuture.supplyAsync(() -> FutureImpl.convertBusinessObjectJson(sublist));
});
Once the CompletableFuture<List<Either>> listCompletableFuture is received, I want to chain the operation,
As in
take CompletableFuture<List<Either>> listCompletableFuture, take exceptions only from list and, perform certain operation
take CompletableFuture<List<Either>> listCompletableFuture, take results only from list and, perform certain operation
Something like this (pseudo code):
mainList.forEach(sublist -> {
CompletableFuture<List<Either>> listCompletableFuture = CompletableFuture.supplyAsync(() -> FutureImpl.convertDSRowToJson(subDSRowList));
listCompletableFuture.thenApply(//function which pushes exception to say kafka)
listCompletableFuture.thenApply(//function which pushes result to say database)
});
Can it be done?
Any help is much appreciated :)
You could try smth like this:
var futureList = mainList.stream()
.map(sublist -> CompletableFuture.supplyAsync(() -> FutureImpl.convertBusinessObjectJson(sublist)))
.collect(Collectors.toList());
The above would collect a list of CompletableFutures. Now what needs to happen is we need to wait for the completion of all those futures. We do this by:
var joinedFutureList = futureList.stream()
.map(objectCompletableFuture -> {
try {
return objectCompletableFuture.get();
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
}).collect(Collectors.toList());
});
After that the separation would look smth like this:
var exceptionList = joinedFutureList.stream()
.filter(obj -> obj instanceof Exception)
.peek(System.out::println)
.collect(Collectors.toList());
var successList = joinedFutureList.stream()
.filter(obj -> obj instanceof String)
.peek(System.out::println)
.collect(Collectors.toList());
Existing code that I have:
private Flux<Integer> testGetFluxTestData() {
return Flux.just(new TestData(1), new TestData(2))
.collectList()
.map(list -> list.stream()
.map(TestData::getId)
.collect(Collectors.toList()))
.flatMapMany(Flux::fromIterable);
}
I want to enrich existing code and throw an exception when some not allowed data received, I made the following changes:
private Flux<Integer> testGetFluxTestData2() {
return Flux.just(new TestData(1), new TestData(2))
.collectList()
.map(list -> {
return !list.contains(new TestData(1)) ?
list.stream()
.map(TestData::getId)
.collect(Collectors.toList()) :
Flux.error(new IllegalTestDataException("illegal test data 1"));
})
.flatMapMany(Flux::fromIterable);
}
but my implementation even noncompilable due to the following line:
Flux.error(new IllegalTestDataException("illegal test data 1"));
Could you please suggest, how to handle exception throwing for my particular scenario?
You are attempting to map from a List<TestData> to either a List<Integer> or a Flux<?> (error), which makes the desired result type ambiguous. Returning a reactive type in a mapping function is generally not desired (you'd want to do that in a flatmapping function).
(side note: even if you were in a flatMap, it wouldn't work either because at that point you're in Mono API due to collectList, so Mono.flatMap expects a Mono result to the Function).
Note that the map operator catches exceptions from the lambda and turn them into an onError signal, so technically you could replace the Flux.error with a throw.
Otherwise, you'd need to turn the map into a flatMap and the Flux.error into a Mono.error, for the reasons stated above.
I am new to java reactive programming and i have started learning it with spring webflux.
One thing that is always bothering me that map is synchronous while flatmap is asynchronous.
I am learning it from book spring in action chapter 10 and i see the below example.
Flux<Player> playerFlux = Flux
.just("Michael Jordan", "Scottie Pippen", "Steve Kerr")
.map(n -> {
String[] split = n.split("\\s");
return new Player(split[0], split[1]);
});
and after very next line it says
What’s important to understand about map() is that the mapping is performed synchronously,
as each item is published by the source Flux. If you want to perform the
mapping asynchronously, you should consider the flatMap() operation.
and showed this example
Flux<Player> playerFlux = Flux
.just("Michael Jordan", "Scottie Pippen", "Steve Kerr")
.flatMap(n -> Mono.just(n)
.map(p -> {
String[] split = p.split("\\s");
return new Player(split[0], split[1]);
})
.subscribeOn(Schedulers.parallel())
);
Alright i think i got the point, but when i was practicing i think actually i didn't got it. lot of question started to rise in my head when i was trying to populate my class fields.
#Data
#Accessors(chain = true)
public class Response {
private boolean success;
private String message;
private List data;
}
Here is the code how i was trying to populate it
Mono<Response> response = Mono.just(new Response())
.map(res -> res.setSuccess(true))
.map(res -> res.setMessage("Success"))
.map(res -> res.setData(new ArrayList()));
after writing this code, one line from the book blinks in my head that map its synchronous will it be a blocking code, could it be bad for entire application because once i read that in a non-blocking application single blocking code could ruin the entire app.
So i decided to convert it into flatmap and according to book it should be look like this.
Mono<Response> response1 = Mono.just(new Response())
.flatMap(
m -> Mono.just(m)
.map(res -> res.setSuccess(true))
)
.flatMap(
m -> Mono.just(m)
.map(res -> res.setMessage("Success"))
)
.flatMap(
m -> Mono.just(m)
.map(res -> res.setData(new ArrayList()))
);
Both example output same but then what is the difference here. it so we should always use flatmap?
Thanks
I'm using Java Reactor Core, and I have a reactive Flux of objects. For each object of the Flux I need to do an external query that will return one different object for each input. The newly generated Flux needs then to be zipped with the original one - so the items of the 2 Flux must be synchronized and generated in the same order.
I'm just re-using the same flow twice, like this:
Flux<MyObj> aTest = Flux.fromIterable(aListOfObj);
Flux<String> myObjLists = aTest.map(o -> MyRepository.findById(o.name)).map(o -> {
if (!o.isPresent()) {
System.out.println("Fallback to empty-object");
return "";
}
List<String> l = o.get();
if (l.size() > 1) {
System.out.println("that's bad");
}
return l.get(0);
});
Flux.zip(aTest, myObjLists, (a, b) -> doSomethingWith(a,b))
Is it the right way to do it? If the myObjLists emits an error, how do I prevent the zip phase to skip the failing iteration?
I've finally opted for using Tuples and Optionals (to prevent null-items that would break the flux), so that I don't need to re-use the initial Flux:
Flux<Tuple<MyObj, Optional<String>>> myObjLists = Flux.fromIterable(aListOfObj)
.map(o -> Tuples.of(o, Optional.ofNullable(MyRepository.findById(o.name))
.flatMap(t -> {
if (!t.getT2().isPresent()) {
System.out.println("Discarding this item");
return Flux.empty();
}
List<String> l = t.getT2().get();
if (l.size() > 1) {
System.out.println("that's bad");
}
return Tuples.of(t.getT1(), l.get(0));
})
.map(t -> doSomethingWith(t.getT1(),t.getT2()))
Note that the flatMap could be replaced with a .map().filter(), removing tuples with missing Optional items