I am currently using RxJava on Android with Kotlin, but I have a problem and I can't solve without using toBlocking().
I have method in employee service which returns an Observable>:
fun all(): Observable<List<Employee>>
This is all and good since this Observable emits the new list of employees whenever an employee changes. But I'd like to generate a PDF file from the employees, which obviously doesn't need to run everytime an employee changes. Also, I'd like to return a Completable object from my PDF generator method. I want to add a header to my PDF, and then iterate through the employees and calculate the wage of each employee, which also returns an Observable, and this is the place where I am using toBlocking right now. My current approach is this:
private fun generatePdf(outputStream: OutputStream): Completable {
return employeeService.all().map { employees ->
try {
addHeaderToPDF()
for (i in employees) {
val calculated = employeeService.calculateWage(i.id).toBlocking().first()
// Print calculated to PDF....
}
addFooterToPDF()
return #map Completable.complete()
}
catch (e: Exception) {
return #map Completable.error(e)
}
}.first().toCompletable()
Is there any way to make this code a little cleaner using RxJava?
Thanks in advance!
Disclaimer: This answer is a work in progress.
Basic premise: If you have blocking in the stream, you're doing it wrong.
Note: No state must leave the observable lambda.
Step 1: Stream the whole data set
The input is a stream of employees. For each employee you need to get one wage. Let's make it into one stream.
/**
* #param employeesObservable
* Stream of employees we're interested in.
* #param wageProvider
* Transformation function which takes an employee and returns a [Single] of their wage.
* #return
* Observable stream spitting individual [Pair]s of employees and their wages.
*/
fun getEmployeesAndWagesObservable(
employeesObservable: Observable<Employee>,
wageProvider: Function<Employee, Single<Int>>
): Observable<Pair<Employee, Int>>? {
val employeesAndWagesObservable: Observable<Pair<Employee, Int>>
// Each Employee from the original stream will be converted
// to a Single<Pair<Employee, Int>> via flatMapSingle operator.
// Remember, we need a stream and Single is a stream.
employeesAndWagesObservable = employeesObservable.flatMapSingle { employee ->
// We need to get a source of wage value for current employee.
// That source emits a single Int or errors.
val wageForEmployeeSingle: Single<Int> = wageProvider.apply(employee)
// Once the wage from said source is loaded...
val employeeAndWageSingle: Single<Pair<Employee, Int> = wageForEmployeeSingle.map { wage ->
// ... construct a Pair<Employee, Int>
employee to wage
}
// This code is not executed now. It will be executed for each Employee
// after the original Observable<Employee> starts spitting out items.
// After subscribing to the resulting observable.
return#flatMapSingle employeeAndWageSingle
}
return employeesAndWagesObservable
}
What's going to happen when you subscribe:
Take an employee from source.
Fetch wage of an employee.
Spit out a pair of employee and their wage.
This repeats until employeesObservable signals onComplete or something fails with onError.
Used operators:
flatMapSingle: Converts an actual value into a new Single stream of some transformed value.
map: Converts an actual value into some other actual value (no nested streams).
Hee's how you'd hook it up to your code:
fun doStuff() {
val employeesObservable = employeeService.all()
val wageProvider = Function<Employee, Single<Int>> { employee ->
// Don't listen to changes. Take first wage and use that.
employeeService.calculateWage(employee.id).firstOrError()
}
val employeesAndWagesObservable =
getEmployeesAndWagesObservable(employeesObservable, wageProvider)
// Subscribe...
}
Used operators:
first: Take the first item from observable and turn it into a Single stream.
timeout: A good idea would be to .timeout the wage if you're getting it over network.
Next steps
Option 1: End here
Don't subscribe, call
val blockingIterable = employeesAndWagesObservable.blockingIterable()
blockingIterable.forEach { ... }
and process each item in a synchronous fashion. Sit back, figure out next steps, watch presentations, read examples.
Option 2: Add layers
.map each of these Pair<Employee, Int> to some abstract PDF building block.
Turn your header and footer printers to Observables via Observable.fromCallable { ... }, have them return PDF building blocks too.
Merge all of these in a sequential manner via Observable.concat(headerObs, employeeDataObs, footerObs)
.subscribe to this result and start writing the PDF building blocks to a PDF writer.
TODO:
Figure out a way to initialize the PDF writer lazily on subscription (not before building the stream),
Delete output on error,
Close output stream on complete or on error.
I came up with this:
return employeeService.all().first()
.doOnSubscribe { addHeaderToPDF() }
.flatMapIterable { it }
.flatMap { employeeService.calculateWage(it.id).first() }
.doOnNext { printEmployeeWage(it) }
.doOnCompleted { addFooterToPDF }
.toCompletable()
Is this how it is supposed to be done? :)
Related
I can't find a correct solution to this problem and I'm stuck. Let's say I have this method
#GET
#Path("/testingAsync")
public Uni<List<String>> testingMutiny() {
List<String> completeList = new ArrayList<>();
completeList.add("hello");
completeList.add("RestEasy");
List<String> finalList = new ArrayList<>();
completeList.forEach(e -> Uni.createFrom().item(e)
.onItem().delayIt().by(Duration.ofMillis(10000))
.map(value -> finalList.add(value.toUpperCase()))
.subscribe().asCompletionStage());
return Uni.createFrom().item(finalList);
}
As you see the method is simple it just takes the values from 1 list and adds them to the second one but what's the problem? When you add the waiting .onItem().delayIt().by(Duration.ofMillis(10000)) the program will return an empty list and after a while, it will just update the list. I created this method to simulate a request that the response that has some delay in it.
Let's say you hit 2 URLs with 2 different Unis after that you try to combine them and return it as one Uni. The problem is if one of those 2 URLs delay for some reason we will return the list empty but I don't want that to happen I either want the list to be completed 100% or return an error if it takes a while.
What is the best approach to that? I understand that if you add await() you are blocking the main thread and you lose all the value of using the reactive library but still, I can't find a way for this to work
EDIT
I have found out that the external URL I try to call takes about 5 seconds to do the job so I want my code to stop when creating the Uni and continue after I have received an answer from the server. I have seen in their docs (here) That I can also call await.indefinitely but I receive The current thread cannot be blocked: vert.x-eventloop-thread-14. How do I wait for a response from the server?
EDIT 2
I understand that with strings it doesn't make sense my question so much so let's say I have the following one
#GET
#Path("/testingAsync")
public Uni<List<Car>> testingMutiny() {
//ALL THIS IS IN A FOR EACH FOR EVERY CAR
//HIT ENDPOINT GET DOORS
Uni<List<JsonObjectCar>> carDoorsUni = getDoors(variable1,
variable2, variable3);
//HIT ENDPOINT GET WHEELS
Uni<List<JsonObjectCar>> carWheelsUni = getWheels(variable1,
variable2, variable3);
//HIT ENDPOINT GET WINDOWS
Uni<List<JsonObjectCar>> carWindowsUni = getWindows(variable1,
variable2, variable3);
Uni.combine()
.all()
.unis(carDoorsUni, carWheelsUni, carWindowsUni)
.combinedWith((carDoors, carWheels, carWindows) -> {
//Check if cardoors is present and set the doors into the car object
Optional.of(carDoors)
.ifPresent(val -> car.setDoors(val.getDoors()));
Optional.of(carWheels)
.ifPresent(val -> car.setWheels(val.getWheels()));
Optional.of(carWindows)
.ifPresent(val -> car.setWindows(val.getWindows()));
return car;
}).subscribe().with(e-> System.out.println("Okay it worked"));
//END OF FOR EACH
//Return car (Should have been returned with new doors / wheels/ windows but instead its empty)
return Uni.createFrom().item(car);
}
As you see in the above code It should have hit some endpoints for doors / wheels / windows and set them into the variable car but what happens, in reality, is that the car is empty because one of those endpoints has been delayed so i return a car without those values inside it. I want to first update the car object and then actually return it
You could rewrite the method like this:
#GET
#Path("/testingAsync")
public Uni<List<String>> testingMutiny() {
List<Uni<String>> unis = new ArrayList<>();
List.of("hello", "RestEasy").forEach( e -> {
unis.add( Uni.createFrom().item( e )
.onItem().delayIt().by( Duration.ofMillis( 10000 ) ) );
} );
return Uni.combine().all().unis( unis )
.combinedWith( list -> (List<String>) list);
}
Note that when you write reactive code, you want to avoid using .await().indefinetly. It shouldn't be needed anyway when using Quarkus, because it recognizes async API and interpret the results accordingly.
You also don't need to subscribe the Uni or Multi when using Quarkus, for the same reason.
Based on my previous example, you can rewrite your use case with endpoints as:
#GET
#Path("/testingAsync")
public Uni<Car> testingMutiny() {
Uni<List<JsonObjectCar>> carDoorsUni = getDoors(variable1, variable2, variable3);
Uni<List<JsonObjectCar>> carWheelsUni = getWheels(variable1,variable2, variable3);
Uni<List<JsonObjectCar>> carWindowsUni = getWindows(variable1,variable2, variable3);
return Uni.combine()
.all()
.unis(carDoorsUni, carWheelsUni, carWindowsUni)
.combinedWith(list -> {
// Result of carDoorsUni
List<JsonObjectCar> carDoors = list.get(0);
// Result of carWheelsUni
List<JsonObjectCar> carWheels = list.get(1);
// Result of carWindowsUni
List<JsonObjectCar> carWindows = list.get(2);
// Create a car instance with the previous results
Car car = createCar(...);
// You can also return a list of cars, but you need to change the return type of testingMutiny to Uni<List<Car>>
return car;
})
.invoke( () -> System.out.println("Okay it worked"));
}
You return a list, but the asynchronous processing on the Uni is delayed, so your list will be empty.
You should try returning a Uni from the pipeline that you create (and also see collect(), toUni() to put into lists) instead of doing a subscription, collect the results and re-wrap into a Uni.
Below function creates a Map, gets the count of passengers where passengers are > minTrips. The code works completely fine. Please see below
fun List<Trip>.filter(minTrips : Int): Set<Passenger> {
var passengerMap: HashMap<Passenger, Int> = HashMap()
this.forEach { it: Trip ->
it.passengers.forEach { it: Passenger ->
var count: Int? = passengerMap.get(it)
if (count == null) {
count = 1
passengerMap.put(it, count)
} else {
count += 1
passengerMap.put(it, count)
}
}
}
val filteredMinTrips: Map<Passenger, Int> = passengerMap.filterValues { it >= minTrips }
println (" Filter Results = ${filteredMinTrips}")
return filteredMinTrips.keys
}
Even though this is written in Kotlin, it seems like the code was first written in Java and then converted over to Kotlin. If it was truly written in Kotlin I am sure this wouldnt have been so many lines of code. How can I reduce the lines of Code? What would be a more funtional approach to solve this? What function or functions can I use to extract the Passengers Set directly where Passengers are > minTrips? This is too much of a code and seems crazy. Any pointers would be helpful here.
One way you could do this is to take advantage of Kotlin's flatmap and grouping calls. By creating a list of all passengers on all trips, you can group them, count them, and return the ones that have over a certain number.
Assuming you have data classes like this (essential details only):
data class Passenger(val id: Int)
data class Trip(val passengers: List<Passenger>)
I was able to write this:
fun List<Trip>.frequentPassengers(minTrips: Int): Set<Passenger> =
this
.flatMap { it.passengers }
.groupingBy { it }
.eachCount()
.filterValues { it >= minTrips }
.keys
This is nice because it is a single expression. Going through it, we look at each Trip and extract all of its Passengers. If we had just done map here, we would have List<List<Passenger>>, but we want a List<Passenger> so we flatmap to achieve that. Next, we groupBy the Passenger objects themselves, and call eachCount() on the returned object, giving us a Map<Passenger, Int>. Finally we filter the map down the Passengers we find interesting, and return the set of keys.
Note that I renamed your function, List already has a filter on it, and even though the signatures are different I found it confusing.
You basically want to count the trips for each passenger, so you can put all passengers in a list and then group by them and afterwards count the occurences in each group:
fun List<Trip>.usualPassengers(minTrips : Int) = // 1
flatMap(Trip::passengers) // 2
.groupingBy { it } // 3
.eachCount() // 4
.filterValues { it >= minTrips } // 5
.keys // 6
Explanation:
return type Set<Passenger> can be inferred
this can be ommitted, a list of the form [p1, p2, p1, p5, ...] is returned
a Grouping is created, which looks like this [p1=[p1, p1], p2=[p2], ...]]
the number of occurences in each group will be counted: [p1=2, p2=1, ...]
all elementes with values which less than minTrips will be filtered out
all keys that are left will be returned [p1, p2, ...]
p1...pn are Passenger instances
I am very new to Project reactor library and reactive programming with Kotlin, and trying to implement functions like flatmap, flatMapIterable, subscribe etc.
Now issue is I am trying to use the o/p of one function into another one using flatMapIterable, and after using I am trying to subscribe this, by passing the output of fist function and second one to another function of new class.
Now when I try to use the o/p of function 1, I am unable to see the value, I only see Mono<> or Flux<>.
Below is code snippet for more explanation
var result = employerService.getEmployee("Active") // return value is Mono<GetEmployeeStatusListResult>
result.flatMapIterable(GetEmployeeStatusListResult::emps)
.flatMap {
employerService.getUsersById(it.userId) // it is of type GetEmployeeStatusListResult.emps and value returned from employerService.getUsersById(it.userId) is of type GetUserResult class created
}.subscribe {
aService.createContact(result, it)
}
Now at line 4 I am getting expected userId out of it.userId, but when I inspect result at line 6, then I do not get the expected list of values, it just provides me MonomapFuesable, with mapper and source.
Can anyone please help me to understand what should I do, as my whole agenda is to pass the calculated value from line 1 and line 4 to line 6 function.
Please ask more question, if I haven't provided the required information, I am very new to this.
Thanks in advance !!
[UPDATE] : I have resolved the issue with the following way :
```
employerService.getEmployee("Active") // return value is Mono<GetEmployeeStatusListResult>
.flatMapIterable(GetEmployeeStatusListResult::emps)
.flatMap {
employerService.getUsersById(it.userId).map{x->Pair(it,x)} // it is of type GetEmployeeStatusListResult.emps and value returned from employerService.getUsersById(it.userId) is of type GetUserResult class created
}.subscribe {
aService.createContact(it.first, it.second)
}
```
It's a bit hard to know for sure from the information supplied above, but I think it looks like the call to employerService.getUsersById isn't returning a Publisher. From your comments I'm guessing it's returning an actual value, GetUserResult, rather than a Mono. Below is a mocked up set of classes which show the desired result, I believe. Maybe compare the below to what you've got and see if you can spot a difference?
data class Employee(val userId: String)
data class GetEmployeeStatusListResult(val emps: List<Employee>)
data class GetUserResult(val employee: Employee)
class EmployerService {
fun getEmployee(status: String) = Mono.just(GetEmployeeStatusListResult(listOf(Employee("a"))))
fun getUsersById(userId: String) = Mono.just(GetUserResult(Employee("a")))
}
fun test() {
val employerService = EmployerService()
employerService
.getEmployee("Active")
.flatMapIterable(GetEmployeeStatusListResult::emps)
.flatMap {
employerService.getUsersById(it.userId)
}.subscribe {
// Here "it" is a GetUserResult object
}
}
If, in the subscribe, you need both the initial value retrieved from the call to getEmployee and also the result of the call to getUsersById then you could wrap those two values in a Pair as shown below:
employerService
.getEmployee("Active")
.flatMapIterable(GetEmployeeStatusListResult::emps)
.flatMap { emp ->
employerService.getUsersById(emp.userId).map { emp to it }
}.subscribe {
// Here "it" is a Pair<Employee, GetUserResult>
}
employerService.getEmployee("Active") // return value is Mono<GetEmployeeStatusListResult>
.flatMapIterable(GetEmployeeStatusListResult::emps)
.flatMap {
employerService.getUsersById(it.userId).map{x->Pair(it,x)} // it is of type GetEmployeeStatusListResult.emps and value returned from employerService.getUsersById(it.userId) is of type GetUserResult class created
}.subscribe {
aService.createContact(it.first, it.second)
}
Adding pair function to fetch both the values and use it in subscribe block !!
Thanks everyone !!
What is the purpose of flatMapPublisher ?
return factory.retrieveDiskDataStore().isCached()
.flatMapPublisher {
factory.retrieveDataStore(it).getData(token)
}
.flatMap {
Flowable.just(if (it is PickupListDataModel) mapper.mapFromData(it) else null)
}
.flatMap {
saveData(it).toSingle { it }.toFlowable()
}
In this code, factory.retrieveDiskDataStore().isCached() checks whether the information is stored in database or not.
If not, then following code executes
.flatMapPublisher {
factory.retrieveDataStore(it).getData(token)
}
From the JavaDocs:
Returns a Flowable that emits items based on applying a specified function to the item emitted by the source Single, where that function returns a Publisher.
In other terms, a Single will succeed with a value which you'd like to turn into a sequence of values generated by a Publisher (i.e., some Flowable, Flux or other standard Reactive Streams source) and have the items of that.
Scenario: I have a customerID string that is used to query multiple different backend systems: calendar, helpdesk, ERP, CRM etc. I want to compile a single report.
So I have roughly (psydocode):
Result myResult = new Result();
Observable<Cal> cal = Calbackend.get(customerid);
cal.subscribe(calentry -> myResult.addCal(calentry));
Observable<Erp> erp = ERPbackend.get(customerid);
erp.subscribe(erpentry -> myResult.addErp(erpentry));
Observable<Help> help = Helpbackend.get(customerid);
help.subscribe(helpentry -> myResult.addHelp(helpentry));
Observable<Crm> crm = CRMbackend.get(customerid);
crm.subscribe(crmentry -> myResult.addCrm(crmentry));
// Magic here?
return result;
The approach I was thinking of: using defer() to prevent the start and then additionally subscribe to count() for each. Then I could ZIP the count elements since they only will emit a single item each (while the others will have different numbers of events). However that could lead to loss of data if the myResult.add is performing slower than the count().
The other option I was thinking of, is to set an array of boolean flags for each subscription and check in each completion (and error) event if all of them are done and do a callback or use blocking for that one.
I had a look here and here but that examples deal with constant numbers or data types.
Or is there a better / recommended way?
Operator toList can be used together with zip like this:
Observable<List<Cal>> cal = Calbackend.get(customerid).toList();
Observable<List<Erp>> erp = ERPbackend.get(customerid).toList();
Observable<List<Help>> help = Helpbackend.get(customerid).toList();
Observable<List<Crm>> crm = CRMbackend.get(customerid).toList();
Observable.zip(cal, erp, help, crm,
new Func4<List<Cal>, List<Erp>, List<Help>, List<Crm>, Result>() {
#Override
public Result call(List<Cal> cals, List<Erp> erps, List<Help> helps, List<Crm> crms) {
Result myResult = new Result();
// add all cals, erps, helps and crms to result
return myResult;
}
})
.subscribe(new Subscriber<Result>() {
#Override
public void onNext(Result result) {
// do something with the result
}
...
});
Explanation: As the name suggests, the toList operator creates a list of the items emitted by the source observable (the list is emitted just once, when the source observable completes) and zip is then used to combine the results of the observables.
Edit: In case of the possibility that those Observables can emit an error, you could use onErrorReturn to keep the normal flow going:
Observable<List<Cal>> cal = Calbackend.get(customerid)
.onErrorReturn(new Func1<Throwable, Cal>() {
#Override
public Cal call(Throwable throwable) {
// Return something in the error case
return null;
}
})
.toList();