I'm new with webflux and I'm trying to execute multiple monos with Flux. But i think i'm doing it wrong.. is this the best approach to execute multiple Mono and collect it to list?
Here is my code:
mainService.getAllBranch()
.flatMapMany(branchesList -> {
List<Branch> branchesList2 = (List<Branch>) branchesList.getData();
List<Mono<Transaction>> trxMonoList= new ArrayList<>();
branchesList2.stream().forEach(branch -> {
trxMonoList.add(mainService.getAllTrxByBranchId(branch.branchId));
});
return Flux.concat(trxMonoList); // <--- is there any other way than using concat?
})
.collectList()
.flatMap(resultList -> combineAllList());
interface MainService{
Mono<RespBody> getAllBranch();
Mono<RespBody> getAllTrxByBranchId(String branchId); //will return executed url ex: http://trx.com/{branchId}
}
so far my with above code i can explain it like this:
Get all branches
iterate through all branchesList2 and add it to trxMonoList
return Flux.concat, this is where i'm not sure is this the right way or not. but it's working
combine all list
I'm just confuse is this the proper way to use Flux in my context? or is there any better way to achieve with what i'm trying to do?
You need to refactor a little bit your code to reactive.
mainService.getAllBranch()
.flatMapMany(branchesList -> Flux.fromIterable(branchesList.getData())) (1)
.flatMap(branch -> mainService.getAllTrxByBranchId(branch.branchId)) (2)
.collectList()
.flatMap(resultList -> combineAllList());
1) Create Flux of branches from List;
2) Iterate through the each element and call a service.
You shouldn't use Stream API in Reactor because it has the same methods but with adaptaptions and optimisations for multythreading.
The real problem here is that you shouldn't be hitting a Mono multiple times within a Flux. That will give you problems. If you are designing the API you should fix that to do what you want in a correct reactive manner.
interface MainService{
Flux<Branch> getAllBranch();
Flux<Transaction> getAllTrxByBranchId(Flux<String> branchIds);
}
Then your code becomes simpler and the reactive framework will work properly.
mainService.getAllTrxByBranchId(mainService.getAllBranch().map(Branch::getId));
Related
I am currently working on a project where I need to create a loop using spring webflux to generate a Flux for downstream processing. The loop should sequentially take batches of elements from a source (in this instance a repository) and pass the elements as signal in a Flux. To acquire the elements, we have a repository method which fetches the next batch. When all elements have been processed, the method yields an empty List.
I have identified that I can use Flux::generate in the following manner:
Flux.<List<Object>>generate(sink -> {
List<Object> batch = repository.fetch();
if (batch.isEmpty()) {
sink.complete();
} else {
sink.next(batch);
}
})
...
However, when I use this, the argument method runs continuously, buffering until I run out of memory.
I have also tried using Flux::create, but I am struggling to find an appropriate approach. I have found that I can do the following:
Consumer<Integer> sinker;
Flux<?> createFlux() {
return Flux.<List<Object>>create(sink -> sinker = integer -> {
List<Object> batch = repository.fetch();
if (batch.isEmpty()) {
sink.complete();
} else {
sink.next(batch);
}
})
...
.doOnNext(x -> sinker.accept(y))
...
}
Then I just need to call the Consumer initially to initiate the loop.
However, I feel like I am overly complicating a job which should have a fairly standard implementation. Also, this implementation requires secondary calls to get started, and I haven't found a decent way to initiate it within the pipeline (for instance, using .onSubscribe() doesn't work, as it attempts to call the Consumer before it has been assigned).
So in summary, I am looking for a simple way to create an unbounded loop while controlling the backpressure to avoid outOfMemory-errors.
I believe I have found a simpler solution which serves my need. The method Mono::repeat(BooleanSuplier) allows me to loop until the list is empty, simply by:
Mono.fromCallable(() -> repository.nextBatch())
.flatMap(/* do some stuff here */)
.repeat(() -> repository.hasNext())
If other more elegant solutions exist, I am still open for suggestions.
Please bear with me, i dont usually use spring and havent used newer versions of java (I say newer I mean anything past prob 1.4)
Anyway, I have an issue where I have to do rest calls to do a search using multiple parallel requests. Ive been looking around online and I see you can use CompletableFuture.
So I created my method to get the objects I need form the rest call like:
#Async
public CompletableFuture<QueryObject[]> queryObjects(String url)
{
QueryObject[] objects= restTemplate.getForObject(url, QueryObject[].class);
return CompletableFuture.completedFuture(objects);
}
Now I need to call that with something like:
CompletableFuture<QueryObject> page1 = QueryController.queryObjects("http://myrest.com/ids=[abc, def, ghi]);
CompletableFuture<QueryObject> page2 = QueryController.queryObjects("http://myrest.com/ids=[jkl, mno, pqr]);
The problem I have is that the call needs to only do three ids at a time and there could be a list of variable number ids. So I parse the idlist and create a query string like above. The problem with that I am having is that while I can call the queries I dont have separate objects that I can then call CompletableFuture.allOf on.
Can anyone tell me the way to do this? Ive been at it for a while now and Im not getting any further than where I am now.
Happy to provide more info if the above isnt sufficient
You are not getting any benefit of using the CompletableFuture in a way you're using it right now.
The restTemplate method you're using is a synchronous method, so it has to finish and return a result, before proceeding. Because of that wrapping the final result in a CompletableFuture doesn't cause it to be executed asynchronously (neither in parallel). You just wrap a response, that you have already retrieved.
If you want to benefit from the async execution, then you can use for example the AsyncRestTemplate or the WebClient .
A simplified code example:
public ListenableFuture<ResponseEntity<QueryObject[]>> queryForObjects(String url) {
return asyncRestTemplate.getForEntity(url, QueryObject[].class);
}
public List<QueryObject> queryForList(String[] elements) {
return Arrays.stream(elements)
.map(element -> queryForObjects("http://myrest.com/ids=[" + element + "]"))
.map(future -> future.completable().join())
.filter(Objects::nonNull)
.map(HttpEntity::getBody)
.flatMap(Arrays::stream)
.collect(Collectors.toList());
}
I have a workflow which resembles this:
flux1.subscribe(result1 -> {
flux2ThatDependsOnResult1.subscribe(result2 -> {
flux3TheDependsOnResult2.subscribe(result3 -> {
...
})
})
})
(Using Spring Webflux)
All the nesting makes this code hard to work with. Is there a better way to write these?
Take a look at concatMap or flatMap operators instead of subscribing on each "step"
While working with Spring Webflux, I'm trying to insert some data in the realm object server which interacts with Java apps via a Rest API. So basically I have a set of students, who have a set of subjects and my objective is to persist those subjects in a non-blocking manner. So I use a microservice exposed via a rest endpoint which provides me with a Flux of student roll numbers, and for that flux, I use another microservice exposed via a rest endpoint that gets me the Flux of subjects, and for each of these subjects, I want to persist them in the realm server via another rest endpoint. I wanted to make this all very nonblocking which is why I wanted my code to look like this.
void foo() {
studentService.getAllRollnumbers().flatMap(rollnumber -> {
return subjectDirectory.getAllSubjects().map(subject -> {
return dbService.addSubject(subject);
})
});
}
But this doesn't work for some reason. But once I call blocks on the things, they get into place, something like this.
Flux<Done> foo() {
List<Integer> rollNumbers = studentService.getAllRollnumbers().collectList().block();
rollNumbers.forEach(rollNumber -> {
List<Subject> subjects = subjectDirectory.getAllSubjects().collectList().block();
subjects.forEach(subject -> {dbService.addSubject(subject).block();});
});
return Flux.just(new NotUsed());
}
getAllRollnumbers() returns a flux of integers.
getAllSubjects() returns a flux of subject.
and addSubject() returns a Mono of DBResponse pojo.
What I can understand is that the thread executing this function is getting expired before much of it gets triggerred. Please help me work this code in an async non blocking manner.
You are not subscribing to the Publisher at all in the first instance that is why it is not executing. You can do this:
studentService.getAllRollnumbers().flatMap(rollnumber -> {
return subjectDirectory.getAllSubjects().map(subject -> {
return dbService.addSubject(subject);
})
}).subscribe();
However it is usually better to let the framework take care of the subscription, but without seeing the rest of the code I can't advise.
New to RxJava. Trying to achieve something trivial but can't seem to wrap my head around the entire RxJava paradigm so far. All i want is to make two service calls one after the other and simply return a final custom POJO consisting of the response of the two calls. Point to be noted here is that the two calls are independent of each other. Only if the first calls comes back with a success, the second one should fire. The second one does not use any data from the first service call. What i have so far.
RestWebClient.get().getFirstImage()
.flatMap(firstImage -> RestWebClient.get().getSecondImage())
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(firstImage -> mainFragmentPresenterInterface.showImages(firstImage));
I am not sure how can i chain the response of two separate network calls and return them as a single POJO back to the presenter.
Also how can we make sure that getSecondImage() is called only if getFirstImage() was a success?
Thanks.
You started off correctly, I'll just modify your snippet a little:
RestWebClient.get().getFirstImage()
.flatMap(firstImage -> /* first image success */ ? RestWebClient.get().getSecondImage() : Observable.just(null),
(firstImage, secondImage) -> Pair.create(firstImage, secondImage))
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(imagePair -> mainFragmentPresenterInterface.showImages(imagePair));