I'm a newbie in RxJava world. I started few days ago and I'm trying to do something very concrete.
I would like to use RxJava to communicate some events from one class to other classes. My emitter class do some work and when finish I want notify that event to every subscribed class.
Also, I would like to don't loose any event, so if emitter finish an action and notifies to subscribers but no one consumes that event it should keep it.
I have been looking Subjects because i read was the good approach for emit by demand but I can't find one that solves my problem, may be I could try PublishSubject in the way they say in that link:
Note that a PublishSubject may begin emitting items immediately upon creation (unless you have taken steps to prevent this), and so there is a risk that one or more items may be lost between the time the Subject is created and the observer subscribes to it. If you need to guarantee delivery of all items from the source Observable, you’ll need either to form that Observable with Create so that you can manually reintroduce “cold” Observable behavior (checking to see that all observers have subscribed before beginning to emit items), or switch to using a ReplaySubject instead.
At the moment I have an Observable that emits the complete list of events to every new Subscription:
protected List<T> commands = new ArrayList<>();
Observable<T> observe = Observable.defer(() -> Observable.fromIterable(commands));
Disposable subscribe(Consumer<T> onNext, Consumer<Throwable> onError) {
return observe.subscribe(onNext/*, onError*/);
}
But this is far away from what I want. As I said I'm pretty noob with Rx, if I can't find a way I will end up using ReplaySubject (The complete list of commands would be repeated on every subscription but at least I will never lose one event).
EDIT:
Finally I split my problem in two parts. For emit items manually I use PublishSubject
PublishSubject<T> actionSubject = PublishSubject.create();
actionSubject.subscribe(onceAction);
actionSubject.onNext(action);
You are probably looking for a BehaviorSubject which replays last value emitted for every new subscriber.
Anyhow, you can create ReplaySubject with the factory method called createWithSize(capacity) to define size-bounded buffer, which discards the oldest item on overflow. There is also a method to create time-bounded buffer with similar behavior.
Isnt it what are you looking for?
Observable<Integer> observable = Observable.create((ObservableEmitter<Integer> e) -> {
for (int i = 0; i < 10; i++) {
e.onNext(i);
}
e.onComplete();
});
System.out.println("FIRST");
observable
.doOnComplete(() -> System.out.println("FIRST COMPLETE"))
.subscribe(System.out::println);
System.out.println("SECOND");
observable
.doOnComplete(() -> System.out.println("SECOND COMPLETE"))
.subscribe(System.out::println);
Output:
FIRST
0
1
2
3
4
5
6
7
8
9
FIRST COMPLETE
SECOND
0
1
2
3
4
5
6
7
8
9
SECOND COMPLETE
Related
Consider the following code:
AtomicInteger counter1 = new AtomicInteger();
AtomicInteger counter2 = new AtomicInteger();
Flux<Object> source = Flux.generate(emitter -> {
emitter.next("item");
});
Executor executor1 = Executors.newFixedThreadPool(32);
Executor executor2 = Executors.newFixedThreadPool(32);
Flux<String> flux1 = Flux.merge(source).concatMap(item -> Mono.fromCallable(() -> {
Thread.sleep(1);
return "1_" + counter1.incrementAndGet();
}).subscribeOn(Schedulers.fromExecutor(executor1)));
Flux<String> flux2 = Flux.merge(source).concatMap(item -> Mono.fromCallable(() -> {
Thread.sleep(100);
return "2_" + counter2.incrementAndGet();
}).subscribeOn(Schedulers.fromExecutor(executor2)));
Flux.merge(flux1, flux2).subscribe(System.out::println);
You can see that one publisher is 100x faster than the other one. Still, when running the code it seems that all data is printed, but there's a huge gap between the two publishers which increases overtime.
What's interesting to note is that when changing the numbers so executer2 will have 1024 threads, and executer1 will have only 1 thread, then still we see a gap that is getting larger and larger overtime.
My expectation was that after tweaking the thread-pools accordingly the publishers will get balanced.
I'd like to achieve a balance between publishers (relative to the thread-pool sizes and processing time)
What would happen if I waited long enough? In other words, is a back-pressure can occur? (Which by default I guess it's a runtime exception, right?)
I don't want to drop items nor want to have a runtime exception. Instead, as I mentioned, I'd like the system to get balanced with respect to the resources it has and the processing times - Does the code above promise that?
Thanks!
Your Flux objects in this example are not ParallelFlux objects, so they'll only ever use one thread.
It doesn't matter if you create a scheduler that's capable of handling thousands of threads, and pass that to one of the Flux objects - they'll just sit there going unused, which is exactly what's happening in this example. There's no backpressure, and it won't result in an exception - it's just going as fast as it can using one thread.
If you want to make sure that the Flux takes full advantage of the 1024 threads available to it, then you need to call .parallel(1024):
ParallelFlux<String> flux1 = Flux.merge(source).parallel(1).concatMap(item -> Mono.fromCallable(() -> {
Thread.sleep(1);
return "1_" + counter1.incrementAndGet();
}).subscribeOn(Schedulers.fromExecutor(executor1)));
ParallelFlux<String> flux2 = Flux.merge(source).parallel(1024).concatMap(item -> Mono.fromCallable(() -> {
Thread.sleep(100);
return "2_" + counter2.incrementAndGet();
}).subscribeOn(Schedulers.fromExecutor(executor1)));
If you do that to your code, then you start to see results much closer to what you seem to be expecting, with 2_ sailing past 1_ despite the fact it's sleeping for 100 times as long:
...
2_17075
2_17076
1_863
1_864
2_17077
1_865
2_17078
2_17079
...
However, a word of warning:
I'd like to achieve a balance between publishers (relative to the thread-pool sizes and processing time)
You can't pick numbers here to balance the outputs, at least not reliably or in any meaningful way - the thread scheduling will be completely arbitrary. If you want to do that, then you could use this variant of the subscribe method, allowing you to explicitly call request() on the subscription consumer. This then allows you to provide backpressure by only requesting as many elements as you're prepared to deal with.
I call a list of methods in a loop, wherein each return an Observable & I add them to a List for processing later.
Code:
List<Observable> productViewObservables = new ArrayList<>();
for (ProductEnricher enricher : orchestrationStrategy.getEnrichers()) {
productViewObservables.add(enricher.asyncEnrich(productView, productId); }
But Im not sure if the Observable responses get added to that list in the same order that I invoke them, which is essential for my processing. Can someone clarify this?
Yes, they are added in the same order, because Observable is just a regular object. Your code is simple sequential code - you are not executing any Observable yet in your example, they are just objects which represent computation that might be done in the future.
If you would like to run these Observables (using subscribe method) then you get into asynchronous world and the results might not be that obvious.
If you'd want to run your List of Observables sequentially then you could use concat method on Observable which takes Iterable as an argument
My API makes about 100 downstream calls, in pairs, to two separate services. All responses need to be aggregated, before I can return my response to the client. I use hystrix-feign to make the HTTP calls.
I came up with what I believed was an elegant solution until on the rxJava docs I've found the following
BlockingObservable is a variety of Observable that provides blocking operators. It can be useful for testing and demo purposes, but is generally inappropriate for production applications (if you think you need to use a BlockingObservable this is usually a sign that you should rethink your design).
My code looks roughly as follows
List<Observable<C>> observables = new ArrayList<>();
for (RequestPair request : requests) {
Observable<C> zipped = Observable.zip(
feignClientA.sendRequest(request.A()),
feignClientB.sendRequest(request.B()),
(a, b) -> new C(a,b));
observables.add(zipped);
}
Collection<D> apiResponse = = new ConcurrentLinkedQueue<>();
Observable
.merge(observables)
.toBlocking()
.forEach(combinedResponse -> apiResponse.add(doSomeWork(combinedResponse)));
return apiResponse;
Few questions based on this setup:
Is toBlocking() justified given my use case
Am I correct in understanding that the actual HTTP calls do not get made until the main thread gets to the forEach()
I've seen that the code in the forEach() block is executed by different threads, but I was not able to verify if there can be more than one thread in the forEach() block. Is the execution there concurrent?
A better option is to return the Observable to be consumed by other operators but you may get away with blocking code (It should, however, run on a background thread.)
public Observable<D> getAll(Iterable<RequestPair> requests) {
return Observable.from(requests)
.flatMap(request ->
Observable.zip(
feignClientA.sendRequest(request.A()),
feignClientB.sendRequest(request.B()),
(a, b) -> new C(a,b)
)
, 8) // maximum concurrent HTTP requests
.map(both -> doSomeWork(both));
}
// for legacy users of the API
public Collection<D> getAllBlocking(Iterable<RequestPair> requests) {
return getAll(requests)
.toList()
.toBlocking()
.first();
}
Am I correct in understanding that the actual HTTP calls do not get made until the main thread gets to the forEach()
Yes, the forEach triggers the whole sequence of operations.
I've seen that the code in the forEach() block is executed by different threads, but I was not able to verify if there can be more than one thread in the forEach() block. Is the execution there concurrent?
Only one thread at a time is allowed to execute the lambda in forEach but you may indeed see different threads entering there.
I'm trying to get the latest value of a given Observable and get it to emit
immediately once it's called. Given the code below as an example:
return Observable.just(myObservable.last())
.flatMap(myObservable1 -> {
return myObservable1;
})
.map(o -> o.x) // Here I want to end up with a T object instead of Observable<T> object
This does not work because by doing this the flatMap will emit myObservable1 which in turn will have
to emit to reach the map.
I don't know if doing such a thing is even possible. Does anyone have any clue on how to achieve this goal? Thank you
last() method will not be of any help here as it waits for the Observable to terminate to give you the last item emitted.
Assuming that you do not have the control over the emitting observable you could simply create a BehaviorSubject and subscribe it to the observable that emits the data that you want to listen and then subscribe to the created subject. Since Subject is both Observable and Subscriber you will get what you want.
I think (do not have the time to check it now) you may have to manually unsubscribe from the original observable as the BehaviorSubject once all of his subscribers unsubscribe will not unsubscribe automatically.
Something like this:
BehaviorSubject subject = new BehaviorSubject();
hotObservable.subscribe(subject);
subject.subscribe(thing -> {
// Here just after subscribing
// you will receive the last emitted item, if there was any.
// You can also always supply the first item to the behavior subject
});
http://reactivex.io/RxJava/javadoc/rx/subjects/BehaviorSubject.html
In RxJava, subscriber.onXXX is called asynchronous.It means that if your Observable emit items in new thread, you can never get the last item before return, except you block the thread and wait for the item.But if the Observable emit item synchronously and you dont' change it's thread by subscribeOn and observOn,
such as the code:
Observable.just(1,2,3).subscribe();
In this case, you can get the last item by doing like this:
Integer getLast(Observable<Integer> o){
final int[] ret = new int[1];
Observable.last().subscribe(i -> ret[0] = i);
return ret[0];
}
It's a bad idea doing like this.RxJava prefer you to do asynchronous work by it.
What you actually want to achieve here is to take an asynchronous task and transform it to a synchronous one.
There are several ways to achieve it, each one with it's pros and cons:
Use toBlocking() - it means that this thread will be BLOCKED, until the stream is finish, in order to get only one item simply use first() as it will complete once an item is delivered.
let's say your entire stream is Observable<T> getData();
then a method that will get the last value immediately will look like this:
public T getLastItem(){
return getData().toBlocking().first();
}
please don't use last() as it will wait for the stream to complete and only then will emit the last item.
If your stream is a network request and it didn't get any item yet this will block your thread!, so only use it when you are sure that there is an item available immediately (or if you really want a block...)
another option is to simply cache the last result, something like this:
getData().subscribe(t-> cachedT = t;) //somewhere in the code and it will keep saving the last item delivered
public T getLastItem(){
return cachedT;
}
if there wasn't any item sent by the time you request it you will get null or whatever initial value you have set.
the problem with this approch is that the subscribe phase might happen after the get and might make a race condition if used in 2 different threads.
I am new to rxjava and I have the following problem:
Objects are irregularly dropped into a FIFO queue by an outside system. I need an Observable which runs every second, takes an item from the queue (if there is one) and emits it to subscribers.
Two problems:
The queue items are produced while the Observable is alive, it is not possible to provide all items upfront. The queue may run empty, in which case the Observable must standby and not emit anything. (It would be nice if the Observable would jump-start immediately when an item becomes available in the queue after a pause, but then the queue would probably need to be an Observable as well if we don't want to poll more frequently, no idea how.)
It must be possible for the outside system to complete the Observable. I could set a variable and read it from within the Observable, but I'd like to know if there is a more elegant way to do that.
LinkedList<Layer> queue = new LinkedList<Layer>(); // the queue
boolean stopObservable = false; // the variable to stop the observable
Observable.create(new Observable.OnSubscribe<Layer>() {
#Override public void call(Subscriber<? super Layer> subscriber) {
try {
if (!queue.isEmpty()) {
Layer layer = queue.poll();
subscriber.onNext(layer);
} else {
if (stopObservable) { subscriber.onCompleted(); }
}
} catch (Exception e) {
subscriber.onError(e);
}
}
}).somethingThatCreatesTheInterval().subscribeOnEtc.
For the interval, I cannot use .sample(), because it drops items, and it is important that all items are emitted.
.throttleWithTimeout() looks better, but it also seems to drop items.
rx is very cool, but tough to get into. Any input appreciated.
I did something similar when I needed to poll external web services at a regular time interval.
For the time interval you could proceed with a timer ; upon each tick with a granularity of 1s the observable chain will poll and maybe pick one layer, if that layer is null then nothing is emitted
Observable.timer(0, 1, TimeUnit.SECOND)
.flatMap(tick -> Observable.just(queue.poll()).filter(layer -> layer != null))
.subscribe(layer -> System.out.format("The layer is : %s", layer));
Now if you want to abort the whole the chain you may add takeUntil. So when your external system want to stop it will submit something in stopObservable which will stop subsequent subscription :
// somewhere before
PublishSubject stopNotifier = PublishSubject.create();
// somewhere process the queue
Observable.timer(0, 1, TimeUnit.SECOND)
.takeUntil(stopNotifier)
.flatMap(tick -> Observable.just(queue.poll()))
.subscribe(layer -> System.out.format("The layer is : %s", layer));
// when not anymore interested (calling onComplete works too)
stopNotifier.onNext("cancel everything about the queue");
I'm writing this response from a tablet so you may assume I may have misspell some words or made naive programming errors ;)
If possible, you should use a PublishSubject<Layer> instead of a LinkedList<Layer>. Then, the outside system can provide new items by calling publishSubject.onNext, and since PublishSubject is a subclass of Observable, your system can treat it as an Observable, and, depending on what semantics with regard to timing you want, apply one of these operators to it:
sample
debounce
throttleFirst/throttleLast/throttleWithTimeout
.zipWith(Observable.timer(1, TimeUnit.SECONDS), (value, tick) -> value) (might do a lot of buffering!)
no timing modification at all (consider this as well)