I use combineLatest() to combine 3 streams of observables. All these are combined so that all data in the UI is shown at the same time. Now, there is a scenario in which one of the observables won't emit anything, since the data that gets fetched, can be null.
Is there a RxJava operator to let the subscriber know that there won't be any emits because of null data?
Edit
private fun retrieveData() {
Observable.combineLatest(getCurrentUser.execute(), getLatestGoal.execute(), getLatestLog.execute(),
Function3<User, Goal, Log, PersonalViewModel> { user, goal, log -> mapToViewModel(user, goal, log) })
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.doOnSubscribe { /*todo: animation*/ }
.doOnNext { view.setViewModel(it) }
.doOnComplete { view.stopLoading() }
.doOnError { /*todo: error message*/ }
.subscribe()
}
The third stream: getLatestLog.execute() emits nothing when the user has nog log. When this stream doesn't emit, the whole view will not be visible.
The data is fetched from FireBase Realtime database. The ChildEventListener has a method that looks like this:
override fun onChildAdded(dataSnapshot: DataSnapshot?, p1: String?) {
val log = dataSnapshot?.getValue(Log::class.java)
log?.let { subscriber.onNext(it) }
subscriber.onComplete()
firebaseDatabase.reference.removeEventListener(this)
}
If you have Java8 or some Optionals at your fingertips, you may use this construct:
#Test
void name() {
TestScheduler scheduler = new TestScheduler();
Observable<Optional<Integer>> o1$ =
Observable.just(Optional.ofNullable(4)).mergeWith(Observable.never());
Observable<Optional<Integer>> o2$ =
Observable.just(Optional.ofNullable(2)).mergeWith(Observable.never());
Observable<Optional<Integer>> o3$ =
Observable.<Optional<Integer>>never()
.timeout(1000, TimeUnit.MILLISECONDS, scheduler)
.onErrorResumeNext(
throwable -> {
return Observable.<Optional<Integer>>never()
.mergeWith(Observable.just(Optional.empty()));
});
Observable<Tuple3<Optional<Integer>, Optional<Integer>, Optional<Integer>>> result =
Observable.combineLatest(
o1$,
o2$,
o3$,
(integer, integer2, integer3) -> Tuple.of(integer, integer2, integer3))
.filter(t -> t._1.isPresent() && t._2.isPresent() && t._3.isPresent());
TestObserver<Tuple3<Optional<Integer>, Optional<Integer>, Optional<Integer>>> test =
result.test();
scheduler.advanceTimeTo(10000, TimeUnit.SECONDS);
test.assertNotComplete().assertNoErrors().assertNoValues();
}
As you may no, null values are not allowed to be emitted through observables-pipelines. Therfore we need some other construct to represent null. In Java8 there is a construct called Optional (vavr calls it Option -> also Java8).
In this example o3$-Observable will not emit anything. It could also error, maybe this resembles your case a little bit more. We will catch the error (in this case: timeout-exception) and return a Observable with Optional.empty.
In the combination-callback we combine alle three values. In a later step we filter out all Tuples, which all have valid values (Optional with Value).
You will only get a value emitted, when all three values have been emitted with a value.
When you can not use a Optional-class, you can also define a INVALID-Object like in the following example:
class So51217041 {
private static Integer INVALID_VALUE = 42;
#Test
void name() {
Observable<Integer> o1$ = Observable.just(4).mergeWith(Observable.never());
Observable<Integer> o2$ = Observable.just(2).mergeWith(Observable.never());
Observable<Integer> o3$ =
Observable.<Integer>never()
.onErrorResumeNext(
throwable -> {
return Observable.<Integer>never().mergeWith(Observable.just(INVALID_VALUE));
});
Observable<Tuple3<Integer, Integer, Integer>> result =
Observable.combineLatest(
o1$,
o2$,
o3$,
(integer, integer2, integer3) -> Tuple.of(integer, integer2, integer3))
.filter(t -> t._3 != INVALID_VALUE); // yeah I know, I want to compare reference, not the content
TestObserver<Tuple3<Integer, Integer, Integer>> test = result.test();
test.assertNotComplete().assertNoErrors().assertNoValues();
}
}
Also, when you want a stream to start with INVALID or NULL, that the CombineLatest emits at least one value, you may use Observable#startWith(INVALID) oder Observable#startWith(Optional.empty()). This will guarantee, that the observable will at least emit one value.
You can use public final Single first(T defaultItem) method. So the code may look like this
getLatestLog.execute()
.first(someDefaultNonNullLog)
.toObservable()
Related
The app is a simple processing - reading data, filtering it, process and write filtered data.
Here is a simple code which runs without any issue:
void run() {
Flux.interval(Duration.ofMillis(200))
.filter(value -> getPredicate().test(value))
.flatMap(this::writeData)
.subscribe();
}
private Predicate<Long> getPredicate() {
return value -> value % 2 == 0;
}
Is it possible to have dynamic predicate which will be retrieved from remote web service with periodic requests?
If possible - how to use Mono<Predicate> inside .filter() and keep it non-blocking
For example replacing getPredicate() with below:
private Mono<Predicate<Long>> getPredicateFromRemoteServer() {
return webClient.get()
.uri("/filters/1")
.retrieve()
.bodyToMono(Filter.class)
.map(this::mapToPredicate)
.cache(v -> Duration.ofMinutes(10), ex -> Duration.ZERO, () -> Duration.ZERO);
}
private Predicate<Long> mapToPredicate(Filter filter) {
// here will be converting filter object into predicate
return value -> value > 5;
}
Ideally I would like to avoid cache(Duration.ofMinutes(10)) because filter could be updated each minute, or each day... and once filter is updated my service get notified, but I didn't find a way to invalidate cache externally, that's why Duration.ofMinutes(10) is used for some approximate invalidation.
Well, perhaps you could write the pipeline a bit differently. Instead of aspiring to return a new Predicate every time your process an item in your stream by calling getPredicateFromRemoteServer(), you could make the function itself your predicate. Pass the value you are processing from the stream and make it return a Mono<Boolean> with the answer and use that in a filterWhen pipe in your pipeline.
For example, somewhat like this:
private Mono<Boolean> isWithinThreshold(int value) {
return webClient.get()
.uri("/filters/1")
.retrieve()
.bodyToMono(Filter.class)
.map(filter -> filter.threshold <= value)
.cache(v -> Duration.ofMinutes(10), ex -> Duration.ZERO, () -> Duration.ZERO);
}
Then in your main pipeline you can do:
Flux.interval(Duration.ofMillis(200))
.filterWhen(value -> isWithinThreshold(value))
.flatMap(this::writeData)
.subscribe();
}
I'm experimenting with RxJava. I need an Observable, which produces a HashSet<String>. Into Observable I want to be abele to put Pair<String, Boolean> in the way that false boolean value removes the String key from the resulting HashSet. Here's a code snippet of what I have:
private val selectionSubject = ReplaySubject.create<Pair<String, Boolean>>()
init {
selectionSubject.onNext(Pair("dd", false))
selectionSubject
.collect({HashSet<String>()}, {dest, value -> collectSelection(dest, value)})
.subscribe { t1, t2 -> Log.d(TAG, t1.toString())}
}
private fun collectSelection(dest: HashSet<String>, value: Pair<String, Boolean>): HashSet<String> {
if (value.second) {
dest.add(value.first)
} else {
dest.remove(value.first)
}
Log.d(TAG, "collectSelection, ${dest.toString()}")
return dest
}
In the logs I can see that collectSelection gets called but my subscribe listener doesn't.
How can it be fixed?
collect waits for onComplete event from the stream above before emitting a value. In your case, ReplaySubject never ends and thus no value is emitted.
Without knowing the context of selectionSubject I can't provide a solution to your problem, like, does it have to remain open? If there are limited onNext calls, you can use .take(X). If it has to remain open, you shouldn't depend on collect but add the item inside on HashSet in something like .doOnNext
I am looking for what is the recommended practice in rxjava2 to handle a case where one flowable leads to conditional behaviors.
More concretely, I have a Maybe<String> for which I want to Update the String on the database if the String exists or, if it doesn't exists I want to create a new String and save it on the database.
I thought of the below but obviously it is not what I am looking for:
Maybe<String> source = Maybe.just(new String("foo")); //oversimplified source
source.switchIfEmpty(Maybe.just(new String("bar"))).subscribe(result ->
System.out.println("save to database "+result));
source.subscribe(result -> System.out.println("update result "+result));
The above obviously produces
save to database foo
update result foo
I tried also the below which gives the expected result but still feel it's... weird.
Maybe<String> source = Maybe.just(new String("foo")); //oversimplified source
source.switchIfEmpty(Maybe.just(new String("bar")).doOnSuccess(result ->
System.out.println("save to database "+result))).subscribe();
source.doOnSuccess(result -> System.out.println("update result "+result)).subscribe();
How can I have an action for when the result exists and when it doesn't exists? How is that use case supposed to be handled in rxjava2?
Update 01
I tried the below and it looks cleaner than what I came up with above. Note sure it is recommended rxjava2 practice however...
Maybe.just(new String("foo"))
.map(value -> Optional.of(value))
.defaultIfEmpty(Optional.empty())
.subscribe(result -> {
if(result.isPresent()) {
System.out.println("update result "+result);
}
else {
System.out.println("save to database "+"bar");
}
});
You have the isEmpty() operator that will return you Boolean if the Maybe source is empty or not, and then you can flatMap it and write a if else statement depending on that Boolean
This is a common pattern in our code as well, though in our case the choices are themselves async. You can't get quite the right semantic by simply composing flatMapX and switchIfEmpty (in either order), so I am curious why this isn't part of the API.
Here's what we're doing for now (this example for when the 2 options are both Completables, we have similar things for the other types as well):
public static <T> Completable flatMapCompletable(Maybe<T> target,
#ClosureParams(FirstParam.FirstGenericType.class)
Closure<? extends CompletableSource> completableSupplier,
Supplier<CompletableSource> emptySupplier) {
Maybe<T> result = target.cache();
return result.isEmpty().flatMapCompletable(empty -> {
if (empty) {
return emptySupplier.get();
} else {
return result.flatMapCompletable(completableSupplier::call);
}
});
}
We're using Groovy, so we package these up as extension methods. I'm not thrilled with the need to use cache() so I'm wondering if there is a better alternative. From looking at the code, an operator which basically combines flatMapX and switch looks like it wouldn't be too hard (but I feel like I'm missing something).
Try something like this. checkDB can return a Maybe or Single or whatever which emits either an optional or a wrapper Object.
checkDB(String)
.flatMap(s -> {
if (s.isPresent()) {
return updateDB(s.get());
} else {
return insertDB("new String");
}
})
There is an solution using the flatMap call with 3 params
fun addOrUpdate(message: LocalMessage): Single<LocalMessage> {
return getById(message.id) // returns Maybe
.flatMap(
Function {
update(message) // onSuccess update call returns Single
},
Function {
Single.error(it) // onError
},
Callable {
add(message) // onComplete add call returns Single
}
)
}
}
Or shorter version
fun addOrUpdate(message: LocalMessage): Single<LocalMessage> {
return getById(message.id) // returns Maybe
.flatMap(
{
update(message) // onSuccess update call returns Single
},
{
Single.error(it) // onError
},
{
add(message) // onComplete add call returns Single
}
)
}
}
Scenario: I have a customerID string that is used to query multiple different backend systems: calendar, helpdesk, ERP, CRM etc. I want to compile a single report.
So I have roughly (psydocode):
Result myResult = new Result();
Observable<Cal> cal = Calbackend.get(customerid);
cal.subscribe(calentry -> myResult.addCal(calentry));
Observable<Erp> erp = ERPbackend.get(customerid);
erp.subscribe(erpentry -> myResult.addErp(erpentry));
Observable<Help> help = Helpbackend.get(customerid);
help.subscribe(helpentry -> myResult.addHelp(helpentry));
Observable<Crm> crm = CRMbackend.get(customerid);
crm.subscribe(crmentry -> myResult.addCrm(crmentry));
// Magic here?
return result;
The approach I was thinking of: using defer() to prevent the start and then additionally subscribe to count() for each. Then I could ZIP the count elements since they only will emit a single item each (while the others will have different numbers of events). However that could lead to loss of data if the myResult.add is performing slower than the count().
The other option I was thinking of, is to set an array of boolean flags for each subscription and check in each completion (and error) event if all of them are done and do a callback or use blocking for that one.
I had a look here and here but that examples deal with constant numbers or data types.
Or is there a better / recommended way?
Operator toList can be used together with zip like this:
Observable<List<Cal>> cal = Calbackend.get(customerid).toList();
Observable<List<Erp>> erp = ERPbackend.get(customerid).toList();
Observable<List<Help>> help = Helpbackend.get(customerid).toList();
Observable<List<Crm>> crm = CRMbackend.get(customerid).toList();
Observable.zip(cal, erp, help, crm,
new Func4<List<Cal>, List<Erp>, List<Help>, List<Crm>, Result>() {
#Override
public Result call(List<Cal> cals, List<Erp> erps, List<Help> helps, List<Crm> crms) {
Result myResult = new Result();
// add all cals, erps, helps and crms to result
return myResult;
}
})
.subscribe(new Subscriber<Result>() {
#Override
public void onNext(Result result) {
// do something with the result
}
...
});
Explanation: As the name suggests, the toList operator creates a list of the items emitted by the source observable (the list is emitted just once, when the source observable completes) and zip is then used to combine the results of the observables.
Edit: In case of the possibility that those Observables can emit an error, you could use onErrorReturn to keep the normal flow going:
Observable<List<Cal>> cal = Calbackend.get(customerid)
.onErrorReturn(new Func1<Throwable, Cal>() {
#Override
public Cal call(Throwable throwable) {
// Return something in the error case
return null;
}
})
.toList();
I'm learning RxJava and, as my first experiment, trying to rewrite the code in the first run() method in this code (cited on Netflix's blog as a problem RxJava can help solve) to improve its asynchronicity using RxJava, i.e. so it doesn't wait for the result of the first Future (f1.get()) before proceeding on to the rest of the code.
f3 depends on f1. I see how to handle this, flatMap seems to do the trick:
Observable<String> f3Observable = Observable.from(executor.submit(new CallToRemoteServiceA()))
.flatMap(new Func1<String, Observable<String>>() {
#Override
public Observable<String> call(String s) {
return Observable.from(executor.submit(new CallToRemoteServiceC(s)));
}
});
Next, f4 and f5 depend on f2. I have this:
final Observable<Integer> f4And5Observable = Observable.from(executor.submit(new CallToRemoteServiceB()))
.flatMap(new Func1<Integer, Observable<Integer>>() {
#Override
public Observable<Integer> call(Integer i) {
Observable<Integer> f4Observable = Observable.from(executor.submit(new CallToRemoteServiceD(i)));
Observable<Integer> f5Observable = Observable.from(executor.submit(new CallToRemoteServiceE(i)));
return Observable.merge(f4Observable, f5Observable);
}
});
Which starts to get weird (mergeing them probably isn't what I want...) but allows me to do this at the end, not quite what I want:
f3Observable.subscribe(new Action1<String>() {
#Override
public void call(String s) {
System.out.println("Observed from f3: " + s);
f4And5Observable.subscribe(new Action1<Integer>() {
#Override
public void call(Integer i) {
System.out.println("Observed from f4 and f5: " + i);
}
});
}
});
That gives me:
Observed from f3: responseB_responseA
Observed from f4 and f5: 140
Observed from f4 and f5: 5100
which is all the numbers, but unfortunately I get the results in separate invocations, so I can't quite replace the final println in the original code:
System.out.println(f3.get() + " => " + (f4.get() * f5.get()));
I don't understand how to get access to both those return values on the same line. I think there's probably some functional programming fu I'm missing here. How can I do this? Thanks.
It looks like all you really need is a bit more encouragement and perspective on how RX is used. I'd suggest you read more into the documentation as well as marble diagrams (I know they're not always useful). I also suggest looking into the lift() function and operators.
The entire point of an observable is to concatenate data flow and data manipulation into a single object
The point of calls to map, flatMap and filter are to manipulate the data in your data flow
The point of merges are to combine data flows
The point of operators are to allow you to disrupt a steady stream of observables and define your own operations on a data flow. For example, I coded a moving average operator. That sums up n doubles in an Observable of doubles to return a stream of moving averages. The code literally looked like this
Observable movingAverage = Observable.from(mDoublesArray).lift(new MovingAverageOperator(frameSize))
You'll be a relieved that a lot of the filtering methods that you take for granted all have lift() under the hood.
With that said; all it takes to merge multiple dependencies is:
changing all incoming data to a standard data type using map or flatMap
merging standard data-types to a stream
using custom operators if one object needs to wait on another, or if you need to order data in the stream. Caution: this approach will slow the stream down
using to list or subscribe to collect all of that data
Edit: someone converted the following text, which I had added as an edit on the question, into an answer, which I appreciate, and understand may be the proper SO thing to do, however I do not consider this an answer because it's clearly not the right way to do it. I would not ever use this code nor would I advise anyone to copy it. Other/better solutions and comments welcome!
I was able to solve this with the following. I didn't realize you could flatMap an observable more than once, I assumed results could only be consumed once. So I just flatMap f2Observable twice (sorry, I renamed some stuff in the code since my original post), then zip on all the Observables, then subscribe to that. That Map in the zip to aggregate the values is undesirable because of the type juggling. Other/better solutions and comments welcome! The full code is viewable in a gist. Thank you.
Future<Integer> f2 = executor.submit(new CallToRemoteServiceB());
Observable<Integer> f2Observable = Observable.from(f2);
Observable<Integer> f4Observable = f2Observable
.flatMap(new Func1<Integer, Observable<Integer>>() {
#Override
public Observable<Integer> call(Integer integer) {
System.out.println("Observed from f2: " + integer);
Future<Integer> f4 = executor.submit(new CallToRemoteServiceD(integer));
return Observable.from(f4);
}
});
Observable<Integer> f5Observable = f2Observable
.flatMap(new Func1<Integer, Observable<Integer>>() {
#Override
public Observable<Integer> call(Integer integer) {
System.out.println("Observed from f2: " + integer);
Future<Integer> f5 = executor.submit(new CallToRemoteServiceE(integer));
return Observable.from(f5);
}
});
Observable.zip(f3Observable, f4Observable, f5Observable, new Func3<String, Integer, Integer, Map<String, String>>() {
#Override
public Map<String, String> call(String s, Integer integer, Integer integer2) {
Map<String, String> map = new HashMap<String, String>();
map.put("f3", s);
map.put("f4", String.valueOf(integer));
map.put("f5", String.valueOf(integer2));
return map;
}
}).subscribe(new Action1<Map<String, String>>() {
#Override
public void call(Map<String, String> map) {
System.out.println(map.get("f3") + " => " + (Integer.valueOf(map.get("f4")) * Integer.valueOf(map.get("f5"))));
}
});
And this yields me the desired output:
responseB_responseA => 714000
I think what you are looking for is switchmap. We ran into a similar issue where we have a session service that handles getting a new session from an api, and we need that session before we can get more data. We can add to the session observable that returns the sessionToken for use in our data call.
getSession returns an observable;
public getSession(): Observable<any>{
if (this.sessionToken)
return Observable.of(this.sessionToken);
else if(this.sessionObservable)
return this.sessionObservable;
else {
// simulate http call
this.sessionObservable = Observable.of(this.sessonTokenResponse)
.map(res => {
this.sessionObservable = null;
return res.headers["X-Session-Token"];
})
.delay(500)
.share();
return this.sessionObservable;
}
}
and getData takes that observable and appends to it.
public getData() {
if (this.dataObservable)
return this.dataObservable;
else {
this.dataObservable = this.sessionService.getSession()
.switchMap((sessionToken:string, index:number) =>{
//simulate data http call that needed sessionToken
return Observable.of(this.dataResponse)
.map(res => {
this.dataObservable = null;
return res.body;
})
.delay(1200)
})
.map ( data => {
return data;
})
.catch(err => {
console.log("err in data service", err);
// return err;
})
.share();
return this.dataObservable;
}
}
You will still need a flatmap to combine the not dependent observables.
Plunkr: http://plnkr.co/edit/hiA1jP?p=info
Where I got the idea to use switch map: http://blog.thoughtram.io/angular/2016/01/06/taking-advantage-of-observables-in-angular2.html