I have UserConfig that I would like to download during splash screen.
class UserManager {
Single<UserConfig> loadConfig()
}
After downloading of the UserConfig, user is redirected to the next screen. I do something like these:
#Override
public void onResume(boolean isNewView) {
subscriptions.add(
userManager.loadConfig().subscribe(config -> {
applyConfig(config);
launchActivity(HomeActivity.class);
}, error -> {
//some error handling
})
);
}
However, I would like to show the splash screen for at least 1 second. (If loading took less than 1s add extra delay)
I think .delay(), .delaySubscription() will not work for my case, since they will delay every request (no matter was it shorter that 1s or not).
Try Zip operator
Returns a Single that emits the results of a specified combiner function > applied to two items emitted by two other Singles.
You can do something like
Single
.zip(
Single.timer(1, TimeUnit.SECONDS),
userManager.loadConfig(),
(time, config) -> config
)
.subscribe(
config -> {
applyConfig(config);
launchActivity(HomeActivity.class);
}, error -> {
//some error handling
}
);
My solution with kotlin extention function for Single type.
This delay work similarly with errors
/**
* sets the minimum delay on the success or error
*/
fun <T> Single<T>.minDelay(time: Long, unit: TimeUnit, scheduler: Scheduler = Schedulers.computation()): Single<T> {
val timeStart = scheduler.now(TimeUnit.MILLISECONDS)
val delayInMillis = TimeUnit.MILLISECONDS.convert(time, unit)
return Single.zip(
Single.timer(time, unit, scheduler),
this.onErrorResumeNext { error: Throwable ->
val afterError = scheduler.now(TimeUnit.MILLISECONDS)
val millisPassed = afterError - timeStart
val needWaitDelay = delayInMillis - millisPassed
if (needWaitDelay > 0)
Single.error<T>(error)
.delay(needWaitDelay, TimeUnit.MILLISECONDS, scheduler, true)
else
Single.error<T>(error)
},
BiFunction { _, t2 -> t2 }
)
}
Related
I am writing a little polling mechanism using Mutiny, part of me learning the library and i am kinda stuck in cancelling the polling when result is found.
I tried using the tick() and what i came up with looks like
Multi.createFrom().ticks().every(Duration.ofSeconds(5))
.onItem().transformToMultiAndMerge(tick -> {
System.out.println("Tick:" + tick);
return Multi.createFrom()
.<Transaction>emitter(
emitter -> {
service.getTransactions().toMulti()
.onItem().transformToMultiAndMerge(
transactions -> Multi.createFrom().iterable(transactions))
.subscribe().with(transaction -> {
if (!verification.isOngoing()) {
emitter.fail(new TransactionVerificationException());
} else {
boolean transactionFound = transaction.getAmount().stream().anyMatch(
amount -> amount.getQuantity()
.equals("test"));
if (transactionFound) {
emitter.emit(transaction);
emitter.complete();
}
}
});
});
})
.subscribe()
.with(transaction -> log.info(transaction),
x -> x.printStackTrace());
Problem here is that the Multi from ticks() is running forever and the only way i think of to cancel it would be to propagate somehow that the emitter has completed.
The case here is that i want to emit, and process only if certain conditions are met.
You approach is almost correct, though,
there is no need to create a custom MultiEmitter out of an existing Multi (or transformed Uni) as you can leverage the different Multi operators on top of your source service#getTransaction result
you missed the EmptyMulti source which will automatically complete downstream subscriber chain and which you can use to signal an absence of valid item (i.e. Transaction)
you need to select the first valid item (being non-null) then transform your Multi to Uni which will result in the upstream subscription being cancelled automatically once an item is received
Here down what the stream pipeline would look like:
Multi.createFrom()
.ticks()
.every(Duration.ofSeconds(5))
.onItem()
// flat map the ticks to the `service#getTransactions` result
.transformToMultiAndMerge(tick -> service.getTransactions()
.toMulti()
.onItem()
// flatten Collection<Transaction> to Multi<Transaction>
.transformToIterable(Function.identity())
.onItem()
.transformToMultiAndMerge(transaction -> {
if (!verification.isOngoing()) {
return Multi.createFrom().failure(new TransactionVerificationException());
} else {
boolean transactionFound = transaction.getAmount()
.stream()
.anyMatch(amount -> amount.getQuantity().equals("test"));
if (transactionFound) {
return Multi.createFrom().item(transaction);
} else {
return Multi.createFrom().empty();
}
}
})
)
.select()
.first(Objects::nonNull)
.toUni()
.subscribe()
.with(transaction -> log.info(transaction), x -> x.printStackTrace());
Have been banging my head over this for a while, I am lost here in managing a requirement where I have to use Rx in Kotlin.
Let me explain.
There is a set of ids whose equivalent items needs to be deleted from server and eventually in local based on server success.
Basically
Make network call to delete for a single id(Supported network call returns a Completable)
if complete(success) callback is received store the id in a list (memory)
Do step one and two for all id to delete
Once every network call is complete pass the list to delete from local DB
So these functions are available which cannot be modified.
fun deleteId(id: String): Completable { networkCall.deleteId(id) }
fun deleteIds(ids: List<String>): Unit { localDb.deleteId(ids) }
This is what I have tried but obviously incomplete and stuck...
val deleted = CopyOnWriteArrayList<String>()
val error = CopyOnWriteArrayList<String>()
items?.filter { it.isChecked }
?.map { Pair(it.id, dataManager.deleteId(it.id)) }
?.forEach { (Id, deleteOp) ->
deleteOp.subscribeOn(Schedulers.io())
.subscribe(object: CompletableObserver {
override fun onComplete() { deleted.add(Id) }
override fun onSubscribe(d: Disposable) { disposableManager += d }
override fun onError(e: Throwable) { error.add(Id) }
})
}
So now there are multiple problems here, One of them is the requirement where I am unable to find a place to know that all requests are completed so as to initiate a localDb delete.
Is there a way where I can use Flowable.fromIterable() or zip or merge somehow following the chain of commands like above to achieve the above scenario?
If I understood your use case correctly, then this should do:
// ids of items to delete, for illustration lets have some temp set
val ids = setOf<String>("1", "2", "3", "4")
val deleteIdSingles = mutableListOf<Single<String>>()
ids.forEach { id ->
deleteIdSingles.add(
api.deleteId(id)
// when request successfully completes, return its id wrapped in a Single, instead of Completable
.toSingle<String> { id }
// return a flag when this request fails, so that the stream is not closed and other requests would still be executed
.onErrorReturn { "FAILED" }
)
}
Single.merge(deleteIdSingles)
// collect the results of the singles (i.e. the ids of successful deletes), and emit a set of those ids once all the singles has completed
.collect(
{ mutableListOf() },
{ deletedIds: MutableList<String>, id: String -> if (id != "FAILED") deletedIds.add(id) }
)
.observeOn(Schedulers.io())
.subscribe(
{ deletedIds ->
db.deleteIds(deletedIds)
}, { error ->
// todo: onError
})
I am writing to an in-memory distributed database in the batch size of that is user-defined in multithreaded environment. But I want to limit the number of rows written to ex. 1000 rows/sec. The reason for this requirement is that my producer is writing too fast and consumer is running into leaf-memory error. Is there any standard practice to perform throttling while batch processing of the records.
dataStream.map(line => readJsonFromString(line)).grouped(memsqlBatchSize).foreach { recordSet =>
val dbRecords = recordSet.map(m => (m, Events.transform(m)))
dbRecords.map { record =>
try {
Events.setValues(eventInsert, record._2)
eventInsert.addBatch
} catch {
case e: Exception =>
logger.error(s"error adding batch: ${e.getMessage}")
val error_event = Events.jm.writeValueAsString(mapAsJavaMap(record._1.asInstanceOf[Map[String, Object]]))
logger.error(s"event: $error_event")
}
}
// Bulk Commit Records
try {
eventInsert.executeBatch
} catch {
case e: java.sql.BatchUpdateException =>
val updates = e.getUpdateCounts
logger.error(s"failed commit: ${updates.toString}")
updates.zipWithIndex.filter { case (v, i) => v == Statement.EXECUTE_FAILED }.foreach { case (v, i) =>
val error = Events.jm.writeValueAsString(mapAsJavaMap(dbRecords(i)._1.asInstanceOf[Map[String, Object]]))
logger.error(s"insert error: $error")
logger.error(e.getMessage)
}
}
finally {
connection.commit
eventInsert.clearBatch
logger.debug(s"committed: ${dbRecords.length.toString}")
}
}
I was hoping if I could pass a user defined arguments as a throttleMax and if total records written by each thread reaches the throttleMax, thread.sleep() will be called for 1 sec. But this is going to make the entire process very slow. Can there be any other effective method, that can be used for throttle the loading of the data to 1000 rows/sec?
As others have suggested (see the comments on the question), you have better options available to you than throttling here. However, you can throttle an operation in Java with some simple code like the following:
/**
* Given an Iterator `inner`, returns a new Iterator which will emit items upon
* request, but throttled to at most one item every `minDelayMs` milliseconds.
*/
public static <T> Iterator<T> throttledIterator(Iterator<T> inner, int minDelayMs) {
return new Iterator<T>() {
private long lastEmittedMillis = System.currentTimeMillis() - minDelayMs;
#Override
public boolean hasNext() {
return inner.hasNext();
}
#Override
public T next() {
long now = System.currentTimeMillis();
long requiredDelayMs = now - lastEmittedMillis;
if (requiredDelayMs > 0) {
try {
Thread.sleep(requiredDelayMs);
} catch (InterruptedException e) {
// resume
}
}
lastEmittedMillis = now;
return inner.next();
}
};
}
The above code uses Thread.sleep, so is not suitable for use in a Reactive system. In that case, you would want to use the Throttle implementation provided in that system, e.g. throttle in Akka
I use combineLatest() to combine 3 streams of observables. All these are combined so that all data in the UI is shown at the same time. Now, there is a scenario in which one of the observables won't emit anything, since the data that gets fetched, can be null.
Is there a RxJava operator to let the subscriber know that there won't be any emits because of null data?
Edit
private fun retrieveData() {
Observable.combineLatest(getCurrentUser.execute(), getLatestGoal.execute(), getLatestLog.execute(),
Function3<User, Goal, Log, PersonalViewModel> { user, goal, log -> mapToViewModel(user, goal, log) })
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.doOnSubscribe { /*todo: animation*/ }
.doOnNext { view.setViewModel(it) }
.doOnComplete { view.stopLoading() }
.doOnError { /*todo: error message*/ }
.subscribe()
}
The third stream: getLatestLog.execute() emits nothing when the user has nog log. When this stream doesn't emit, the whole view will not be visible.
The data is fetched from FireBase Realtime database. The ChildEventListener has a method that looks like this:
override fun onChildAdded(dataSnapshot: DataSnapshot?, p1: String?) {
val log = dataSnapshot?.getValue(Log::class.java)
log?.let { subscriber.onNext(it) }
subscriber.onComplete()
firebaseDatabase.reference.removeEventListener(this)
}
If you have Java8 or some Optionals at your fingertips, you may use this construct:
#Test
void name() {
TestScheduler scheduler = new TestScheduler();
Observable<Optional<Integer>> o1$ =
Observable.just(Optional.ofNullable(4)).mergeWith(Observable.never());
Observable<Optional<Integer>> o2$ =
Observable.just(Optional.ofNullable(2)).mergeWith(Observable.never());
Observable<Optional<Integer>> o3$ =
Observable.<Optional<Integer>>never()
.timeout(1000, TimeUnit.MILLISECONDS, scheduler)
.onErrorResumeNext(
throwable -> {
return Observable.<Optional<Integer>>never()
.mergeWith(Observable.just(Optional.empty()));
});
Observable<Tuple3<Optional<Integer>, Optional<Integer>, Optional<Integer>>> result =
Observable.combineLatest(
o1$,
o2$,
o3$,
(integer, integer2, integer3) -> Tuple.of(integer, integer2, integer3))
.filter(t -> t._1.isPresent() && t._2.isPresent() && t._3.isPresent());
TestObserver<Tuple3<Optional<Integer>, Optional<Integer>, Optional<Integer>>> test =
result.test();
scheduler.advanceTimeTo(10000, TimeUnit.SECONDS);
test.assertNotComplete().assertNoErrors().assertNoValues();
}
As you may no, null values are not allowed to be emitted through observables-pipelines. Therfore we need some other construct to represent null. In Java8 there is a construct called Optional (vavr calls it Option -> also Java8).
In this example o3$-Observable will not emit anything. It could also error, maybe this resembles your case a little bit more. We will catch the error (in this case: timeout-exception) and return a Observable with Optional.empty.
In the combination-callback we combine alle three values. In a later step we filter out all Tuples, which all have valid values (Optional with Value).
You will only get a value emitted, when all three values have been emitted with a value.
When you can not use a Optional-class, you can also define a INVALID-Object like in the following example:
class So51217041 {
private static Integer INVALID_VALUE = 42;
#Test
void name() {
Observable<Integer> o1$ = Observable.just(4).mergeWith(Observable.never());
Observable<Integer> o2$ = Observable.just(2).mergeWith(Observable.never());
Observable<Integer> o3$ =
Observable.<Integer>never()
.onErrorResumeNext(
throwable -> {
return Observable.<Integer>never().mergeWith(Observable.just(INVALID_VALUE));
});
Observable<Tuple3<Integer, Integer, Integer>> result =
Observable.combineLatest(
o1$,
o2$,
o3$,
(integer, integer2, integer3) -> Tuple.of(integer, integer2, integer3))
.filter(t -> t._3 != INVALID_VALUE); // yeah I know, I want to compare reference, not the content
TestObserver<Tuple3<Integer, Integer, Integer>> test = result.test();
test.assertNotComplete().assertNoErrors().assertNoValues();
}
}
Also, when you want a stream to start with INVALID or NULL, that the CombineLatest emits at least one value, you may use Observable#startWith(INVALID) oder Observable#startWith(Optional.empty()). This will guarantee, that the observable will at least emit one value.
You can use public final Single first(T defaultItem) method. So the code may look like this
getLatestLog.execute()
.first(someDefaultNonNullLog)
.toObservable()
I'm playing with the RxJava retryWhen operator. Very little is found about it on the internet, the only one worthy of any mention being this. That too falls short of exploring the various use cases that I'd like to understand. I also threw in asynchronous execution and retry with back-off to make it more realistic.
My setup is simple: I've a class ChuckNorrisJokesRepository that returns random number of Chuck Norris jokes from a JSON file. My class under test is ChuckNorrisJokesService which is shown below. The use cases I'm interested in are as follows:
Succeeds on 1st attempt (no retries)
Fails after 1 retry
Attempts to retry 3 times but succeeds on 2nd hence doesn't retry 3rd time
Succeeds on 3rd retry
Note: The project is available on my GitHub.
ChuckNorrisJokesService.java:
#Slf4j
#Builder
public class ChuckNorrisJokesService {
#Getter
private final AtomicReference<Jokes> jokes = new AtomicReference<>(new Jokes());
private final Scheduler scheduler;
private final ChuckNorrisJokesRepository jokesRepository;
private final CountDownLatch latch;
private final int numRetries;
private final Map<String, List<String>> threads;
public static class ChuckNorrisJokesServiceBuilder {
public ChuckNorrisJokesService build() {
if (scheduler == null) {
scheduler = Schedulers.io();
}
if (jokesRepository == null) {
jokesRepository = new ChuckNorrisJokesRepository();
}
if (threads == null) {
threads = new ConcurrentHashMap<>();
}
requireNonNull(latch, "CountDownLatch must not be null.");
return new ChuckNorrisJokesService(scheduler, jokesRepository, latch, numRetries, threads);
}
}
public void setRandomJokes(int numJokes) {
mergeThreadNames("getRandomJokes");
Observable.fromCallable(() -> {
log.debug("fromCallable - before call. Latch: {}.", latch.getCount());
mergeThreadNames("fromCallable");
latch.countDown();
List<Joke> randomJokes = jokesRepository.getRandomJokes(numJokes);
log.debug("fromCallable - after call. Latch: {}.", latch.getCount());
return randomJokes;
}).retryWhen(errors ->
errors.zipWith(Observable.range(1, numRetries), (n, i) -> i).flatMap(retryCount -> {
log.debug("retryWhen. retryCount: {}.", retryCount);
mergeThreadNames("retryWhen");
return Observable.timer(retryCount, TimeUnit.SECONDS);
}))
.subscribeOn(scheduler)
.subscribe(j -> {
log.debug("onNext. Latch: {}.", latch.getCount());
mergeThreadNames("onNext");
jokes.set(new Jokes("success", j));
latch.countDown();
},
ex -> {
log.error("onError. Latch: {}.", latch.getCount(), ex);
mergeThreadNames("onError");
},
() -> {
log.debug("onCompleted. Latch: {}.", latch.getCount());
mergeThreadNames("onCompleted");
latch.countDown();
}
);
}
private void mergeThreadNames(String methodName) {
threads.merge(methodName,
new ArrayList<>(Arrays.asList(Thread.currentThread().getName())),
(value, newValue) -> {
value.addAll(newValue);
return value;
});
}
}
For brevity, I'll only show the Spock test case for the 1st use case. See my GitHub for the other test cases.
def "succeeds on 1st attempt"() {
setup:
CountDownLatch latch = new CountDownLatch(2)
Map<String, List<String>> threads = Mock(Map)
ChuckNorrisJokesService service = ChuckNorrisJokesService.builder()
.latch(latch)
.threads(threads)
.build()
when:
service.setRandomJokes(3)
latch.await(2, TimeUnit.SECONDS)
Jokes jokes = service.jokes.get()
then:
jokes.status == 'success'
jokes.count() == 3
1 * threads.merge('getRandomJokes', *_)
1 * threads.merge('fromCallable', *_)
0 * threads.merge('retryWhen', *_)
1 * threads.merge('onNext', *_)
0 * threads.merge('onError', *_)
1 * threads.merge('onCompleted', *_)
}
This fails with:
Too few invocations for:
1 * threads.merge('fromCallable', *_) (0 invocations)
1 * threads.merge('onNext', *_) (0 invocations)
What I'm expecting is that fromCallable is called once, it succeeds, onNext is called once, followed by onCompleted. What am I missing?
P.S.: Full disclosure - I've also posted this question on RxJava GitHub.
I solved this after several hours of troubleshooting and with help from ReactiveX member David Karnok.
retryWhen is a complicated, perhaps even buggy, operator. The official doc and at least one answer here use range operator, which completes immediately if there are no retries to be made. See my discussion with David Karnok.
The code is available on my GitHub complete with the following test cases:
Succeeds on 1st attempt (no retries)
Fails after 1 retry
Attempts to retry 3 times but succeeds on 2nd hence doesn't retry 3rd time
Succeeds on 3rd retry