I have an Observable that at some point has to write things to the cache - and we would like to wait that writes are done before finishing the whole operation on the observable (for reporting purposes).
For the purpose of test, the cache write Completable looks like this:
Completable.create(
emitter ->
new Thread(
() -> {
try {
Thread.sleep(2000);
doSomething();
emitter.onComplete();
} catch (InterruptedException e) {
e.printStackTrace();
}
})
.start());
Since I have several cache writes, I try to merge them in a container class:
public class CacheInsertionResultsTracker {
private Completable cacheInsertResultsCompletable;
public CacheInsertionResultsTracker() {
this.cacheInsertResultsCompletable = Completable.complete();
}
public synchronized void add(Completable cacheInsertResult) {
this.cacheInsertResultsCompletable = this.cacheInsertResultsCompletable.mergeWith(cacheInsertResult);
}
public Completable getCompletable() {
return this.cacheInsertResultsCompletable;
}
}
And I try to merge it with Observable in a following way:
CacheInsertionResultsTracker tracker = new ...;
observable
.doOnNext(next->tracker.add(next.writeToCache(...)))
.mergeWith(Completable.defer(()->tracker.getCompletable()))
.subscribe(
// on next
this::logNextElement
// on error
this::finishWithError
// on complete
this::finishWithSuccess
);
How could I make sure that by the time finishWithSuccess is called the doSomething is completed?
The problem is that the Completable reference is updated every time I add a new one, and it happens after the mergeWith runs...
The solution that seems to work for our use case is to use concatWith + defer:
observable
.doOnNext(next->tracker.add(next.writeToCache(...)))
.concatWith(Completable.defer(()->tracker.getCompletable()))
.subscribe(
// on next
this::logNextElement
// on error
this::finishWithError
// on complete
this::finishWithSuccess
);
Concat assures that the subscription to the Completable happens only after the Observable is done, and defer defers getting the final Completable till this subscription (so all the objects are already added to the tracker).
Based on the comments, you could replace the completable cache with ReplaySubject<Completable>, do some timeout to detect inactivity and have the observable sequence end.
ReplaySubject<Completable> cache = ReplaySubject.create();
cache.onNext(completable);
observable.mergeWith(
cache.flatMapCompletable(v -> v)
.timeout(10, TimeUnit.MILLISECONDS, Completable.complete())
)
Edit:
Your updated example implies you want to run Completables in response to items in the main observable, isolated to that sequence, and wait for all of them to complete. This is a typical use case for flatMap:
observable.flatMap(
next -> next.writeToCache(...).andThen(Observable.just(next))
)
.subscribe(
this::logNextElement
// on error
this::finishWithError
// on complete
this::finishWithSuccess
);
Related
Following is my Current Code
private final List<Disposable> subscriptions = new ArrayList<>();
for (Instrument instrument : instruments) {
// Waiting for OrderBook to generate Reliable results.
GenericBook Book =
service
.getBook(instrument.getData())
.filter(gob -> onBookUpdate(gob))
.blockingFirst();
subscriptions.add(
service
.getBook(instrument.getData())
.subscribe(
gob -> {
try {
onBookUpdate(gob);
} catch (Exception e) {
logger.error("Error on subscription:", e);
}
},
e -> logger.error("Error on subscription:", e)));
}
So what it does is for each instrument it first Block wait till the output of onBookUpdate(gob) Becomes true. onBookUpdate(gob) returns boolean.
Once we have first onBookUpdate as true then i Will push that subscriber into subscriptions variable.
This slow down as I have to wait foreach instrument and then move on the next instrument.
My Goal is to run all these in parallel then wait all to finish and push them to subscriptions variable.
I tried zip but didn't work
List<Observable<GenericOrderBook>> obsList = null;
for (Instrument instrument : instruments) {
// This throws nullException.
obsList.add(service
.getBook(instrument.getData())
.filter(gob -> onBookUpdate(gob))
.take(1));
}
}
// Some how wait over here until all get first onBookUpdate as true.
String o = Observable.zip(obsList, (i) -> i[0]).blockingLast();
When using observables etc, one should embrace them wholeheartedly. One of the premises for embracing is to separate the configuration and construction of your pipeline from its execution.
In other words, configure your pipeline upfront and then, when the data is available, send the data through it.
Furthermore, embracing observables implies avoiding for-loops.
I'm not 100% what your use case is but what I'd suggest is to create a pipeline that takes an instrument as input and returns a subscription...
So something like
service.getBook(instrument.getData())
.flatMap(gob -> {
onBookUpdate(gob);
return gob;
});
That will return an Observable that you can subscribe to and add the result to the subscriptions.
Then create a seed observable that pumps the instrument objects into it.
Not sure of some of the details of your API, so come back to me if this is not clear or I've made a wrong assumption.
I am assuming instruments to be a List. If yes, then you can do something like this,
Observable
.fromIterable(instruments)
// Returns item from instrument list one by one and passes it to getBook()
.flatmap(
instrument -> getBook(instrument.getData())
)
.filter(
gob -> onBookUpdate(gob)
)
// onComplete will be called if no items from filter
.switchIfEmpty(Observable.empty())
.subscribe(
onBookUpdateResponse -> // Do what you want,
error -> new Throwable(error)
);
Hope this helps.
I have a list a want to refresh every minute.
For example the user list here : https://github.com/android10/Android-CleanArchitecture/blob/master/domain/src/main/java/com/fernandocejas/android10/sample/domain/interactor/GetUserList.java
I add a periodical refresh using repeatWhen :
public Observable<List<User>> buildUseCaseObservable(Void unused) {
return this.userRepository
.users()
.repeatWhen(new Function<Observable<Object>, ObservableSource<?>>() {
#Override
public ObservableSource<?> apply(Observable<Object> objectObservable) throws Exception {
return objectObservable.delay(1, TimeUnit.MINUTES);
}
});
}
It works fine this way, calling onNext every minute.
But if I want to refresh immediately this list (because of user's action or because of a notification), I don't know how to perform that.
Should I cancel/dispose the observable and restart a new one ?
Thanks
From your code I understand that the users list is generated and emitted upon subscription.
Here are some solutions I can think of, instead of unsubscribing and resubscribing upon the event to which you want to react immediately:
Instead of using the repeatWhen operator, use the interval creation operator combined with the flatMap to invoke the subscription to a new Observable every minute and use the merge operator to add reaction to the other event in which you are interested. Something like this:
#Test
public void intervalObservableAndImmediateReaction() throws InterruptedException {
Observable<String> obs = Observable.interval(1, TimeUnit.SECONDS)
.cast(Object.class)
.mergeWith(
Observable.just("mockedUserClick")
.delay(500, TimeUnit.MILLISECONDS))
.flatMap(
timeOrClick -> Observable.just("Generated upon subscription")
);
obs.subscribe(System.out::println);
Thread.currentThread().sleep(3000); //to see the prints before ending the test
}
or adjusted to your needs (but the principal is the same):
Observable.interval(1, TimeUnit.MINUTES)
.mergeWith(RxView.clicks(buttonView))
.flatMap(timeOrClick -> this.userRepository.users());
You can use the flatMap operator as before, even while keeping you working current implementation and without merging to an interval - just keep your working code and in another area of the programme chain it to the RxBinding of your choosing:
RxView.touches(yourViewVariable)
.flatMatp(motionEvent -> this.userRepository.users())
.subscribe(theObserver);
Note that in this solution the subscription is done independently to the two observables. You'll probably be better off if you use different observers, or manage a subject or something on that line. A small test I ran showed one subscriber handled subscribing to 2 different observables with no problem (in Rxjava1 - didn't check in Rxjava2 yet), but it feels iffy to me.
If you aren't concerned with adjusting the refresh time after one of the other observables emits data you can do something like the following:
// Specific example of a user manually requesting
val request = Observable.create<String> { emitter ->
refresh.setOnClickListener {
emitter.onNext("Click Request")
}
}
.observeOn(Schedulers.io())
.flatMap {
userRepository.users()
}
// Refresh based off of your original work, could use something like interval as well
val interval = userRepository.users()
.subscribeOn(Schedulers.io())
.repeatWhen { objectObservable ->
objectObservable.delay(1, TimeUnit.MINUTES)
}
// Combine them so that both emissions are received you can even add on another source
Observable.merge(request,interval)
.observeOn(AndroidSchedulers.mainThread())
.subscribe({
contents.text = it.toString()
}, {
contents.text = it.toString()
},{
println(contents.text)
})
Then you don't have to dispose and resubscribe every time
In the following code
public CompletableFuture<String> getMyFuture(String input)
{
CompletableFuture<String> future = new CompletableFuture<String>().thenApply((result) -> result+ "::");
ExecutorService service = Executors.newFixedThreadPool(6);
service.submit(() -> {
try {
future.complete(getResult(input));
} catch (InterruptedException e) {
e.printStackTrace();
}
});
return future;
}
public String getResult(String input) throws InterruptedException
{
Thread.sleep(3000);
return "hello "+ input +" :" + LocalTime.now();
}
I am expecting the output to contain trailing "::" but program doesn't is "hello first :16:49:30.231
" Is my implementation of apply correct ?
You're invoking complete() method of the CompletionStage that you got at the first line (where you call "thenApply" method).
If your intention is to complete the CompletableFuture with some string value (future.complete(getResult(input))) and then apply some function, you'd better place thenApply() at the end (where you return the future).
public CompletableFuture<String> getMyFuture(String input)
{
CompletableFuture<String> future = new CompletableFuture<String>();
ExecutorService service = Executors.newFixedThreadPool(6);
service.submit(() -> {
try {
future.complete(getResult(input));
} catch (InterruptedException e) {
e.printStackTrace();
}
});
return future.thenApply(result -> result+ "::");
}
I don't know how to explain it in a more understandable way. But in short: you're calling complete() method on the wrong object reference inside your Runnable.
You are creating two CompletableFuture instances. The first, created via new CompletableFuture<String>() will never get completed, you don’t even keep a reference to it that would make completing it possible.
The second, created by calling .thenApply((result) -> result+ "::") on the first one, could get completed by evaluating the specified function once the first one completed, using the first’s result as an argument to the function. However, since the first never completes, the function becomes irrelevant.
But CompletableFuture instances can get completed by anyone, not just a function passed to a chaining method. The possibility to get completed is even prominently displayed in its class name. In case of multiple completion attempts, one would turn out to be the first one, winning the race and all subsequent completion attempts will be ignored. In your code, you have only one completion attempt, which will successfully complete it with the value returned by getResult, without any adaptations.
You could change your code to keep a reference to the first CompletableFuture instance to complete it manually, so that the second gets completed using the function passed to thenApply, but on the other hand, there is no need for manual completion here:
public CompletableFuture<String> getMyFuture(String input) {
ExecutorService service = Executors.newFixedThreadPool(6);
return CompletableFuture.supplyAsync(() -> getResult(input), service)
.thenApply(result -> result + "::");
}
public String getResult(String input) {
LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(3));
return "hello "+ input +" :" + LocalTime.now();
}
When specifying the executor to supplyAsync, the function will be evaluated using that executor. More is not needed.
Needless to say, that’s just for example. You should never create a temporary thread pool executor, as the whole point of a thread pool executor is to allow reusing the threads (and you’re using only one of these six threads at all) and it should get shut down after use.
I'm new with RxJava. I'm trying to create an observable that when it completes it will start all over again until I call dispose, but I'm facing an OutofMemory error after a while, below is a simplified example of what I'm trying to do
public void start() throws RuntimeException {
log.info("\t * Starting {} Managed Service...", getClass().getSimpleName());
try {
executeObserve();
log.info("\t * Starting {} Managed Service...OK!", getClass().getSimpleName());
} catch (Exception e) {
log.info("Managed Service {} FAILED! Reason is {} ", getClass().getSimpleName(), e.getMessage(), e);
}
}
start is invoked at the initialization phase once, the executeObserve is as follows (in a simplified form..). Notice that on the onComplete I "resubmit" executeObserve
public void executeObserve() throws RuntimeException {
Observable<Book> booksObserve
= manager.getAsObservable();
booksObserve
.map(Book::getAllOrders)
.flatMap(Observable::fromIterable)
.toList()
.subscribeOn(Schedulers.io())
.subscribe(collectedISBN ->
Observable.fromIterable(collectedISBN)
.buffer(10)
// ...some more steps here...
.toList()
.toObservable()
// resubmit
.doOnComplete(this::executeObserve)
.subscribe(validISBN -> {
// do something with the valid ones
})
)
);
}
My guess is that this is not the way to go if I want to resubmit my tasks but it was not possible to find any documentation.
the booksObserve is implemented as follows
public Observable<Book> getAsObservable() {
return Observable.create(e -> {
try (CloseableResultSet<Book> rs = (CloseableResultSet<Book>) datasource.retrieveAll())) {
for (Book r : rs) {
e.onNext(r);
}
e.onComplete();
} catch (Exception ex) {
e.onError(ex);
}
});
}
What is the correct way to constantly resubmit an operation until we call dispose or equivalent? I'm using RxJava 2
You have created an endless recursion, the loop will create more and more resources and sometime it will blow with OutOfMemory/Stack overflow exception.
In order to repeat the Observable work you should use repeat() operator, it will resubscribes to the Observable when it receives onComplete().
Besides that, some general comments on your code:
why are you nesting the second Observable inside the subscriber? you are breaking the chain, you can just continue the chain instead of creating new Observable at the Subscriber.
Moreover, it's seems (assuming Observable.fromIterable(collectedBets) using the collectedISBN that gets with the onNext() o.w. from where does it comes?) you're collecting all items to a list, and then flatting it again using from iterable, so it's seems you can just continue on the stream , something like that:
booksObserve
.map(Book::getAllOrders)
.flatMap(Observable::fromIterable)
.buffer(10)
// ...some more steps here...
.toList()
.toObservable()
// resubmit
.doOnComplete(this::executeObserve)
.subscribeOn(Schedulers.io())
.subscribe(validISBN -> {
// do something with the valid ones
});
Anyhow, with the nested Observable, the repeat() operator will just repeat the nested one, and not the entire stream (which is what you want) as it is not connected to it.
In continuation to my question the repeat as #yosriz suggested is the proper way to go, the following simple snippet demonstrates that the observable source will be called on each repeat
Observable<Integer> recursiveObservable = Observable.create(emitter -> {
System.out.println("Calling to emit data");
Lists.newArrayList(1, 2, 3, 4, 5, 6, 7, 8, 9, 0).forEach(emitter::onNext);
emitter.onComplete();
});
recursiveObservable
.buffer(2)
.repeat()
.subscribe(integers -> {
System.out.println(integers);
TimeUnit.SECONDS.sleep(1);
});
I've been experimenting with different ways to handle blocking methods with disconnected results while maintaining state which might have been interrupted. I've found it to be frustrating having to deal with disparate classes and methods where sending and receiving are difficult to align.
In the following example, SomeBlockingMethod() normally returns void as a message is sent to some other process. But instead I've made it synchronized with a listener which receives the result. By spinning it off to a thread, I can wait() for the result with a timeout or indefinitely.
This is nice because once the result is returned, I can continue working with a particular state which I had to pause while waiting for the result of the threaded task.
This there anything wrong with my approach?
Although this question may seem generic, I am specifically looking for advice on threading in Java.
Example pseudocode:
public class SomeClass implements Command {
#Override
public void onCommand() {
Object stateObject = new SomeObjectWithState();
// Do things with stateObject
Runnable rasync = () -> {
Object r = SomeBlockingMethod();
// Blocking method timed out
if (r == null)
return;
Runnable rsync = () -> {
// Continue operation on r which must be done synchronously
// Also do things with stateObject
};
Scheduler().run(rsync);
};
Scheduler().run(rasync);
}
Update with CompletableFuture:
CompletableFuture<Object> f = CompletableFuture.supplyAsync(() -> {
return SomeBlockingMethod();
});
f.thenRun(() -> { () -> {
String r = null;
try {
r = f.get();
}
catch (Exception e) {
e.printStackTrace();
}
// Continue but done asynchronously
});
or better yet:
CompletableFuture.supplyAsync(() -> {
return SomeBlockingMethod();
}).thenAccept((
Object r) -> {
// Continue but done asynchronously
});
The problem with using strictly CompletableFuture is that CompletableFuture.thenAccept is run from the global thread pool and is not guaranteed to be synchronous with the calling thread.
Adding the scheduler back for the synchronous task fixes this:
CompletableFuture.supplyAsync(() -> {
return SomeBlockingMethod();
}).thenAccept((
Object r) -> {
Runnable rsync = () -> {
// Continue operation on r which must be done synchronously
};
Scheduler().run(rsync);
});
A caveat of using CompletableFuture compared to the complete scheduler method is that any previous state which exists outside must be final or effectively final.
You should check out RxJava, it uses stream manipulation and has threading support.
api.getPeople()
.observeOn(Schedulers.computation())
.filter(p -> return p.isEmployee();)
.map(p -> return String.format("%s %s - %s", p.firstName(), p.lastName(), p.payrollNumber());)
.toList()
.observerOn(<ui scheudler>)
.subscirbe(p -> screen.setEmployees(p);)