I have the following observable:
ScheduledExecutorService executorService = Executors.newScheduledThreadPool( 1 );
Observable<List<Widget>> findWidgetsObservable = Observable.create( emitter -> {
executorService.scheduleWithFixedDelay( emitFindWidgets( emitter, 0, 30, TimeUnit.SECONDS );
} );
private Runnable emitFindWidgets( ObservableEmitter<List<Widgets>> emitter ) {
return () -> {
emitter.onNext( Collections.emptyList() ); // dummy empty array
};
}
And I'm returning it in a graphql-java subscription resolver like so:
ConnectableObservable<List<Widget>> connectableObservable = findWidgetsObservable.share().publish();
Disposable connectionDisposable = connectableObservable.connect();
return connectableObservable.toFlowable( BackpressureStrategy.LATEST )
The graphql subscription works as expected and emits data to the JavaScript graphql client, but when the client unsubscribes, my Runnable continues seemingly infinitely. That said, the flowable's doOnCancel() event handler IS being run.
In order to remedy this problem, I've attempted to do the following within the flowable's doOnCancel():
Disposable connectionDisposable = connectableObservable.connect();
return connectableObservable.toFlowable( BackpressureStrategy.LATEST ).doOnCancel( () -> {
findWidgetsObservable.toFuture().cancel( true );
connectionDisposable.dispose();
})
However, the Runnable continues omitting indefinitely. Is there any way I can solve this problem and completely stop the emits?
I did have one thought: scheduleWithFixedDelay returns a ScheduledFuture, which has a cancel() method, but I'm not sure that there's anyway I can do that when the scheduling itself is scoped within an observable! Any help is appreciated.
The runnable keeps on emitting because you are scheduling the emission on a scheduler that is not known/bound to observable stream.
When you dispose your connection, you stop receiving the items from upstream because the connection to upstream observable is cut. But since you are scheduling the emitter to run repeatedly on a separate scheduler, the runnable keeps running.
You can describe the custom scheduling behavior using a custom scheduler and passing it in subscribeOn(Your-Custom-Scheduler)
Also, you mentioned you can invoke cancel() on ScheduledFuture in doOnDispose().
But you should switch schedulers explicitly in the observable chain. Otherwise, it becomes harder to debug.
Related
I have N tasks to execute, and the number of tasks is not fixed. The next task can only be executed after the previous task is completed. How can the entire task chain be executed asynchronously?
If the number of tasks is fixed, such as N=2, I can use the following code. How to do it if N is not fixed
public void futureTest() throws InterruptedException {
CompletableFuture<Integer> finalFuture = new CompletableFuture<>();
CompletableFuture<Integer> cf1 = doTask(1);
AtomicReference<CompletableFuture<Integer>> cf2 = new AtomicReference<>(new CompletableFuture<>());
cf1.whenComplete(((integer1, throwable1) -> {
if (throwable1 != null) {
finalFuture.completeExceptionally(throwable1);
return;
}
// when task1 complete then submit task2
cf2.set(doTask(2));
cf2.get().whenComplete(((integer2, throwable2) -> {
if (throwable2 != null) {
finalFuture.completeExceptionally(throwable2);
return;
}
finalFuture.complete(integer2);
}));
}));
finalFuture.whenComplete(((integer, throwable) -> {
System.out.println("all task is done");
}));
Thread.sleep(1000000);
}
private CompletableFuture<Integer> doTask(int index) {
CompletableFuture<Integer> cf = new CompletableFuture<>();
// Simulate task execution
THREAD_POOL.execute(() -> {
Thread.sleep(3_000);
cf.complete(index);
});
return cf;
}
I looked at Compeltable's API docs and none of them seem to solve my problem. I tried to use a loop to control the task submission, but all failed, unable to submit the next task after the previous task is completed
Refer to this answer on this thread Click here. Seems a duplicate of this question.
thenRun method is used to run the task after the previous future is completed successfully. This method will be skipped in case of any failures in previous stages.
whenComplete method is used as the final stage of execution chain. Here you will receive the composed result of all the other functions in the supply chain and you can choose to fail your future or handle exceptions accordingly inside this.
You can compose the futures for the individual tasks via CompletableFuture#thenCompose in a loop:
CompletableFuture<?> future = createFirstTask();
while (hasMoreTasks()) {
future = future.thenCompose(this::createNextTask);
}
Here every next tasks depends on the result of the previous one until no more task is left.
Conceptionally this is a fold operation, which unfortunately is not part of the API of CompletableFuture. But if you don't mind using my better future library (which is just a thin wrapper around CompleteableFuture), I just recently added support for folding streams of futures there.
We are given a Mono, that's handling some action(say a database update), and returns a value.
We want to add that Mono(transformed) to a special list that contains actions to be completed for example during shutdown.
That mono may be eagerly subscribed after adding to the list, to start processing now, or .subscribe() might not be called meaning it will be only subscribed during shutdown.
During shutdown we can iterate on the list in the following way:
for (Mono mono : specialList) {
Object value = mono.block(); // (do something with value)
}
How to transform the original Mono such that when shutdown code executes, and Mono was previously subscribed(), the action will not be triggered again but instead it will either wait for it to complete or replay it's stored return value?
OK, looks like it is as simple as calling mono.cache(), so this is how I used it in practice
public Mono<Void> addShutdownMono(Mono<Void> mono) {
mono = mono.cache();
Mono<Void> newMono = mono.doFinally(signal -> shutdownMonos.remove(mono));
shutdownMonos.add(mono);
return newMono;
}
public Function<Mono<Void>,Mono<Void>> asShutdownAwaitable() {
return mono -> addShutdownMono(mono);
}
database.doSomeAction()
.as(asShutdownAwaitable)
.subscribe() // Or don't subscribe at all, deferring until shutdown
Here is the actual shutdown code.
It was also important to me that they execute in order of being added, if user chose not to eagerly subscribe them, that's reason for Flux.concat instead of Flux.merge.
public void shutdown() {
Flux.concat(Lists.transform(new ArrayList<>(shutdownMonos), mono -> mono.onErrorResume(err -> {
logger.error("Async exception during shutdown, ignoring", err);
return Mono.empty();
}))
).blockLast();
}
The problem in that: I have Observable and Subscriber. I try to launch Observable in .io() thread, because it works with files and zip archivers (I won't show the code - is too large), but Observable do nothing!:
Observable<Double> creatingObservable = getCreatingObservable(image);
Subscriber<Double> creatingSubscriber = getCreatingSubscriber();
creatingObservable
.subscribeOn(Schedulers.io())
.subscribe(creatingSubscriber);
If I launch code without the subscribeOn() - all work. What is the problem and how to solve it
P.S. System.out.println() doesn't work too. Problem have all Scheduler's threads.
It seems the problem is that the main thread terminated before creatingObservable could emit any values.
The simple solution: make the main thread wait long enough to enable creatingObservable to emit/complete.
Observable<Double> creatingObservable = getCreatingObservable(image);
Subscriber<Double> creatingSubscriber = getCreatingSubscriber();
creatingObservable
.subscribeOn(Schedulers.io())
.subscribe(creatingSubscriber);
Thread.sleep(5000); //to wait 5 seconds while creatingObservable is running on IO thread
Try this one:
Subscriber<Double> creatingSubscriber = getCreatingSubscriber();
Observable.defer(new Func0<Observable<Double>>() {
#Override
public Observable<Double> call() {
return getCreatingObservable(image);
}
})
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(creatingSubscriber);
Don't forget to add:
compile 'io.reactivex:rxandroid:1.2.1'
From here: https://github.com/ReactiveX/RxAndroid
Explanation
getCreatingObservable(image); - most probably you use some operators which do 'hard' work in moment of call.
For example:
Observable.just(doSomeStuff())
.subscribeOn(...)
.observeOn(...)
So, the execution process will be:
1). Calculate doSomeStuff()
2). Pass result to Observable.just()
3). And only passing you are applying schedulers
In other words, you are doing 'hard' work firstly, and then applying schedulers.
That's why you need to use Observable.defer()
For more explanation, please read this article of Dan Lew:
http://blog.danlew.net/2014/10/08/grokking-rxjava-part-4/
Section Old, Slow Code
In this case you app create observable just once. You may try to either use
Observable.defer(()-> creatingObservable) so .defer operator will force observable creation every time.
Observable.defer(new Func0<Observable<Double>>() {
#Override
public Observable<Double> call() {
return getCreatingObservable();
}
})
.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.subscribe(getCreatingSubscriber);
Is there an ExecutorService that will allow me to submit tasks without beginning execution until I request it? I'm looking for something like ScheduledExecutorService, except that I want to trigger the execution manually without depending on a fixed time delay.
The reason I'm looking for this is because I want to create a set of tasks which may recursively use results of Futures generated from parallel tasks in the same set. So I would need to first submit all the tasks in order to get a set of Futures, and only then could I allow the tasks to begin executing.
It sounds like a job for CompletableFuture
fire execution of first portion of tasks as separe CompletableFutures
then using CompletableFuture.allOf(...furures) to create a barrier future that completes only when all are done
then using one of combinators like CompletableFuture.thenAccept to schedule next portion of tasks to perform on completion of barrier future
But more idiomatic way to use it would be to chain each next task based on future result of some previous one
CompletableFuture<FirstResult> firstTask = //....
CompletableFuture<SecondResult> secondTask = firstTask.thenApply(someTransformation);
CompletableFuture<Void> anotherTaks = firstTask.thenAccept(someConsumer);
CompletableFuture<ThirdResult> combined = firstTask.thenAcceptBoth(secondTask, someFunction);
Perhaps an alternate approach would be to simply use a FutureCallback or an AsyncFunction?
FutureCallback example:
final List<ListenableFuture<T>> futures = new ArrayList<ListenableFuture<T>>();
final Callable<T> callable = new Callable<T>() {
// Some task you want to complete
};
// Submit all your tasks for execution
futures.add(listeningExecutorService.submit(callable));
// ... add as many tasks as you have
futures.add(listeningExecutorService.submit(callable));
// Get a single Future to wait on
final ListenableFuture<List<T>> future = Futures.allAsList(futures);
Futures.addCallback(future, new FutureCallback<List<T>>() {
#Override
public void onSuccess(final List<T> result) {
// Begin other tasks using `result` (the set of results from the first tasks)
}
#Override
public void onFailure(final Throwable t) {
// ...
}
});
This would be helpful if you don't care about waiting around for the second set of tasks to complete, as Futures.addCallback doesn't return anything.
AsyncFunction example:
final ListenableFuture<O> result = Futures.transform(future, new AsyncFunction<List<T>, O>() {
#Override
public ListenableFuture<O> apply(final List<T> input) {
// Begin other tasks using `input` (the set of results from the first tasks)
}
});
This would be beneficial if you want to wait on the resulting ListenableFuture, or potentially add a third set of tasks that need to happen upon completion of the second set.
I have a Callable<String>. I want to run it periodically via ScheduledExecutorService.scheduleAtFixedRate(), and to get a list of all the Strings that were returned by the .call() invocations on my callable. As scheduleAtFixedRate does not take a Callable (only Runnables) I need to roll out a custom Runnable that wraps my Callable, something along these lines:
final Callable<String> myCallable = ....;
final ConcurrentLinkedQueue<String> results
= new ConcurrentLinkedQueue<String>();
Runnable r = new Runnable() {
#Override public void run() {
try {
results.add(myCallable.call());
} catch (Exception e) {
results.add(null); // Assuming I want to know that an invocation failed
}
}
};
ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);
executor.scheduleAtFixedRate(r, 0, 1, TimeUnit.SECONDS);
Naturally, I'd like to avoid rolling out my own custom thingies (especially in multi-threaded code), so I wonder there is a JDK class that does this kind of aggregation?
What you are doing above is treating your Callable implementation as just another normal class. You are not submitting the callable to a ThreadPool executor. Calling Callable.call() doesn't utilize the ThreadPoolExecutor.
You need to submit your Task (Runnable/Callable/ForkJoinTask,etc..) to a ThreadPool to utilize the thread pooling.
You can use the Futures to collect the results once executed.
ForkJoinPool is one option you can try thats part of JDK 7. Fork the tasks and Join them using ForkJoinTask
Why not use Futures? They are exactly meant for knowing the state of Task and its result.
Did you look at this : Using Callable to Return Results From Runnables