I have stupid problem with RxJava2.
I need to run two long operations at the same time. I know that I should use Observable.zip() and I use it.
The problem, that my long operations is run one after another and another problem that my long operations starting before I subscribe to them.
Let's imagine that this is my long operation that I should run async.
private String doSomethingLong() {
Random rand = new Random();
int value = rand.nextInt(5);
Timber.i("Do something for [%d] sec [%s]", value, Thread.currentThread().getName());
try {
Thread.sleep(value * 1000);
} catch (InterruptedException e) {
e.printStackTrace();
return String.format(Locale.getDefault(), "Exception [%s]", e.getMessage());
}
return String.format(Locale.getDefault(),"Job for [%d] seconds", value);
}
And let there is a method like test() that will try to make it parallel:
public void test() {
final long started = System.currentTimeMillis();
Observable<String> just1 = Observable.just(doSomethingLong()).subscribeOn(Schedulers.newThread());
Observable<String> just2 = Observable.just(doSomethingLong()).subscribeOn(Schedulers.newThread());
Observable.zip(just1, just2, new Func2<String, String, Combined>() {
#Override
public Combined call(String s, String s2) {
return new Combined(s, s2);
}
}).observeOn(AndroidSchedulers.mainThread()).subscribe(new Observer<Combined>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable e) {
}
#Override
public void onNext(Combined combined) {
long total = System.currentTimeMillis() - started;
Timber.i("TOTAL [%d]ms [%s]", total, combined.toString());
}
});
}
When I'm trying to run this I observe that two observables just1 and just2 runs one after another... And it's confused me...
But there is another staff that confused me more... I commented Observable.zip and noticed that just1 and just2 started method doSomethingLong() before I subscribed to them...
Let me show:
public void test() {
final long started = System.currentTimeMillis();
Observable<String> just1 = Observable.just(doSomethingLong()).subscribeOn(Schedulers.newThread());
Observable<String> just2 = Observable.just(doSomethingLong()).subscribeOn(Schedulers.newThread());
// Observable.zip(just1, just2, new Func2<String, String, Combined>() {
// #Override
// public Combined call(String s, String s2) {
// return new Combined(s, s2);
// }
// }).observeOn(AndroidSchedulers.mainThread()).subscribe(new Observer<Combined>() {
// #Override
// public void onCompleted() {
//
// }
//
// #Override
// public void onError(Throwable e) {
//
// }
//
// #Override
// public void onNext(Combined combined) {
// long total = System.currentTimeMillis() - started;
// Timber.i("TOTAL [%d]ms [%s]", total, combined.toString());
// }
// });
}
This code make almost same - it's run two times doSomethingLong() one after another...
What I'm expect:
1. I need that doSomethingLong() methods run parallel
2. I'm asking to explain why those methods runs before I start subscribe them.
3. How should I write me code well in this situation. I want that doSomethingLong() methods do not called before I subscribe to them.
Thanks a lot. Hope that I explain problem well.
Observable.just doesn't run anything when you subscribe. It emits the elements when you subscribe, but your doSomethingLong will run as soon as you pass it as an argument. That's normal and it's how the language works.
What you're looking for is a way to say return this when we subscribe, but also only run it at that time and hopefully on a background thread.
There are a couple of answers for this, here are some:
Using defer
There's an operator called defer which takes a lambda which will be executed once you subscribe:
Observable.defer(() -> doSomethingLong())
This will only execute doSomethingLong when you subscribe
Using fromCallable
You can create an observable from a lambda. This is known as fromCallable:
Observable.fromCallable(() -> doSomethingLong())
Similarly, this will only run doSomethingLong when you subscribe
Using create
I think this is perhaps the most discouraged way of doing it, since there's a couple of things you have to deal with, but I think for the she of completeness it's ok to mention:
Observable.create( emitter -> {
if(emitter.isDisposed()) return;
emitter.onNext(doSomethingLong());
emitter.onComplete();
});
Again, I'm sure there's more ways of doing this. I just wanted to explain the issue and give some options.
Create your Observables as Observable.fromCallable{}.
And instead of zip use combineLatest()
Docs:
http://reactivex.io/RxJava/javadoc/io/reactivex/Observable.html#fromCallable-java.util.concurrent.Callable-
http://reactivex.io/documentation/operators/combinelatest.html
Related
Here's a sample of the code I've been working on
items contains 100 elements, thus obtaining data using synchronous calling takes up a lot of time. Can someone suggest a way to increase the speed of this operation so that it takes less time.
Currently this takes 15-20 seconds to execute. I'm new to rxjava so please provide a detailed solution to this problem if possible. dataResponses contains RouteDistance objects for each of the 100 items.
for(int i = 0 ; i<items.size();i++){
Map<String, String> map2 = new HashMap<>();
map2.put("units", "metric");
map2.put("origin", currentLocation.getLatitude()+","+currentLocation.getLongitude());
map2.put("destination", items.get(i).getPosition().get(0)+","+items.get(i).getPosition().get(1));
map2.put("transportMode", "car");
requests.add(RetrofitClient4_RouteDist.getClient().getRouteDist(map2));
}
Observable.zip(requests, new Function<Object[], List<RouteDist>>() {
#Override
public List<RouteDist> apply(Object[] objects) throws Exception {
Log.i("onSubscribe", "apply: " + objects.length);
List<RouteDist> dataaResponses = new ArrayList<>();
for (Object o : objects) {
dataaResponses.add((RouteDist) o);
}
return dataaResponses;
}
})
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.subscribe(
new Consumer<List<RouteDist>>() {
#Override
public void accept(List<RouteDist> dataaResponses) throws Exception {
Log.i("onSubscribe", "YOUR DATA IS HERE: "+dataaResponses.toString());
recyclerViewAdapter_profile = new RecyclerViewAdapter_Profile(items,dataaResponses);
recyclerView.setAdapter(recyclerViewAdapter_profile);
}
},
new Consumer<Throwable>() {
#Override
public void accept(Throwable e) throws Exception {
Log.e("onSubscribe", "Throwable: " + e);
}
});
API
interface Client {
Observable<RouteDist> routeDist();
}
final class RouteDist {
}
final class ClientImpl implements Client {
#Override
public Observable<RouteDist> routeDist() {
return Observable.fromCallable(() -> {
// with this log, you see, that each subscription to an Observable is executed on the ThreadPool
// Log.e("---------------------", Thread.currentThread().getName());
return new RouteDist();
});
}
}
Apply threading via subscribeOn
final class ClientProxy implements Client {
private final Client api;
private final Scheduler scheduler;
ClientProxy(Client api, Scheduler scheduler) {
this.api = api;
this.scheduler = scheduler;
}
#Override
public Observable<RouteDist> routeDist() {
// apply #subscribeOn in order to move subscribeAcutal call on given Scheduler
return api.routeDist().subscribeOn(scheduler);
}
}
AndroidTest
#Test
public void name() {
// CachedThreadPool, in order to avoid creating 100-Threads or more. It is always a good idea to use own Schedulers (e.g. Testing)
ThreadPoolExecutor threadPool = new ThreadPoolExecutor(0, 10,
60L, TimeUnit.SECONDS,
new SynchronousQueue<>());
// wrap real client with Proxy, in order to move the subscribeActual call to the ThreadPool
Client client = new ClientProxy(new ClientImpl(), Schedulers.from(threadPool));
List<Observable<RouteDist>> observables = Arrays.asList(client.routeDist(), client.routeDist(), client.routeDist());
TestObserver<List<RouteDist>> test = Observable.zip(observables, objects -> {
return Arrays.stream(objects).map(t -> (RouteDist) t).collect(Collectors.toList());
})
.observeOn(AndroidSchedulers.mainThread())
.test();
test.awaitCount(1);
// verify that onNext in subscribe is called in Android-EventLoop
assertThat(test.lastThread()).isEqualTo(Looper.getMainLooper().getThread());
// verify that 3 calls were made and merged into one List
test.assertValueAt(0, routeDists -> {
assertThat(routeDists).hasSize(3);
return true;
});
}
Further reading:
http://tomstechnicalblog.blogspot.de/2016/02/rxjava-understanding-observeon-and.html
Note:
It is not recommanded to call an API 100-times concurrently at once. Furthermore when using Zip, this is what will acutally happen, when you have a ThreadPool, which is big enough. When one API-call times-out, an onError will probably emitted for this API-calls. The onError will be propagated further to the subscriber. You will not get any result, even if only on API-call fails. It is recommanded to have some onErrorResumeNext or some other error-handling operator, in order to ensure, that one API-call does not cancel the overall result.
I am new using java RX and I am facing a problem, hopefully someone can give me a clue what Im doing wrong.
The issue:
There are many events I am tracking, such events are triggered like this:
Observable<Long> otherObservable = Observable.empty();
public void myMethod(){
Observable<Long> observable1 = Observable.timer(VARIABLE_TIME, TimeUnit.SECONDS);
final Subscriber<Long> timeSubscriber = new Subscriber<Long>() {
#Override
public void onCompleted() {
// nothing really
}
#Override
public void onError(final Throwable throwable) {
// nothing really
}
#Override
public void onNext(final Long number) {
// Here i do something
}
};
return Observable.merge(timerObservable, otherObservable)
.first()
.subscribe(timeSubscriber);
}
So basically it fires an event after a VARIABLE_TIME.
It works great but now I am facing the fact I have too many events.
So I thought about using debounce and buffer.
What Im trying to do is this:
Still create many observables that emit an event after N seconds.
Collect info from each of them (a long or maybe a String)
After a delay time (buffer time) Send a list with all the collected info to the subscriber.
So far Ive done this:
Observable<List<Long>> otherObservable = Observable.empty();
otherObservable.debounce(10L, SECONDS).buffer(20L, SECONDS);
Observable<List<Long>> observable1 = Observable.timer(VARIABLE_TIME, TimeUnit.SECONDS).buffer(1);
Subscriber<List<Long> > observerSuscriber = new Subscriber<List<Long>>() {
#Override
public void onCompleted() {
}
#Override
public void onError(final Throwable throwable) {
}
#Override
public void onNext(final List<Long> ids ) {
// do something here
}
};
Observable.merge(otherObservable, observable1)
.first()
.subscribe(observerSuscriber);
But like this I still get the message instantly after emitted.
I am wondering if there is anyway to do this? Any ideas? I am using java RX 1.2
After a delay time (buffer time) Send a list with all the collected info to the subscriber.
You answered your own question there! Use the buffer operator:
Flowable<String> stream = ...
Flowable<List<String>> lists = stream.buffer(5, TimeUnit.SECONDS);
Consider we have a scheduled executor service:
ScheduledExecutorService threadPool = Executors.newScheduledThreadPool(...);
And for some logic we want to retry a task execution. The following approach seems to be smelling for me, but I can't understand why:
threadPool.submit(new Runnable() {
#Override
public void run() {
// ...
if (needToBeScheduled()) {
threadPool.schedule(this, delay, TimeUnit.MINUTES);
} else if (needToBeResubmitted()) {
threadPool.submit(this);
}
}
});
The one obvious problem I see is that this code is not possible to convert to lambda:
threadPool.submit(()-> {
// ...
if (needToBeScheduled()) {
threadPool.schedule(this, delay, TimeUnit.MINUTES);
} else if (needToBeResubmitted()) {
threadPool.submit(this);
}
});
^^ this won't compile, as we can not refer this from lambda. Though it can be solved by introducing a method which produces such an instance and provide it instead of this.
But this is only one disadvantage I see. Is anything else here which can cause any problems? Perhaps there is a more proper approach? Move this logic to ThreadPoolExecutor.afterExecute() (this causes type conversion though...)?
Assuming that object is stateless, i.e. there are no object variables in Runnable instance.
P.S. The logic of what to do (reschedule task or resubmit or do nothing) is based on some information retrieved from the database (or any external source). So Runnable is still stateless, but it calculates the outcome based on some results of its work.
Honestly, I don't like the approach where a task (a simple independent unit of work) decides whether it should put itself in the service or not and interacts with the ExecutorService directly. I believe // ... is the only part a task should execute.
I would convert a Runnable in a Callable<Boolean>:
Callable<Boolean> task = () -> {
// ...
return needToBeScheduled; // or sth more complex with several boolean fields
};
And I would definitely move that logic outside a task (for example, into a service method):
Future<Boolean> future = threadPool.submit(task);
try {
boolean needToBeScheduled = future.get();
if (needToBeScheduled) {
threadPool.schedule(task, delay, TimeUnit.MINUTES);
}
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
By something more complex I meant a class that comprises 2 boolean fields. It takes Supplier<Boolean>s to make things lazy.
final class TaskResult {
private final Supplier<Boolean> needToBeScheduled;
private final Supplier<Boolean> needToBeResubmitted;
private TaskResult(Supplier<Boolean> needToBeScheduled, Supplier<Boolean> needToBeResubmitted) {
this.needToBeScheduled = needToBeScheduled;
this.needToBeResubmitted = needToBeResubmitted;
}
public static TaskResult of(Supplier<Boolean> needToBeScheduled, Supplier<Boolean> needToBeResubmitted) {
return new TaskResult(needToBeScheduled, needToBeResubmitted);
}
public boolean needToBeScheduled() {
return needToBeScheduled != null && needToBeScheduled.get();
}
public boolean needToBeResubmitted() {
return needToBeResubmitted != null && needToBeResubmitted.get();
}
}
With a few changes to the above example, we have:
Callable<TaskResult> task = () -> {
// ...
return TaskResult.of(() -> needToBeScheduled(), () -> needToBeResubmitted());
};
final Future<TaskResult> future = threadPool.submit(task);
try {
final TaskResult result = future.get();
if (result.needToBeScheduled()) {
threadPool.schedule(task, delay, TimeUnit.MINUTES);
}
if (result.needToBeResubmitted()) {
threadPool.submit(task);
}
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
I would like to add tests for method, which contains CompletableFuture:
public void report(List<String> srcList) {
if (srcList != null) {
...
CompletableFuture.runAsync(() ->
....
srcList.forEach(src-> downloader.send(url)));
}
}
I would like to test, that method send is called. My test looks like:
#Test
public void _test() {
List<String> events = new ArrayList();
events.add("http://xxxx//");
events.add("http://xxxx//");
expect(downloader.send(events.get(0))).andReturn("xxx").times(2);
replay(downloader);
eventReporter.report(events);
verify(downloader);
}
And I get such error Downloader.send("http://xxxx//"): expected: 2, actual: 0
One way to avoid this error, is to set Thread.sleep(100); timeout. Then the thread will wait and verify that the method had called. But this will increase tests time.
Is there other way to test multithreading with EasyMock?
It is a bad practice to unit test asynchronoys code with Thread.sleep() method
because if it even works test will be unstable and flicker (run 3 times 2 pass and 1 fail)
if you set up big time of sleep and write few tests like this you meet a big time of execution
that may be exceed dozens of seconds. For complete this task you need to decouple asynchronous part
of you code from synchronous. Example how to do it:
class Service {
private Downloader downloader;
private ExecutorService service;
public Service (Downloader downloader, ExecutorService service) {
//set variables
}
public void doWork(List<String> list) {
for (String item : list) {
service.submit(() -> {
downloader.download(item);
});
}
}
}
ExecutorService is interface and we need to make our Service that will be synchronous
class SycnronousService impliments ExecutorService {
//methods empty implementations
public void submit(Runnable runnable) {
runnable.run(); //run immediately
}
//methods empty implementations
}
public class ServiceTest {
public void shouldPassAllItemsToDownloader() {
Downloader mockDownloader = AnyMockFramework.mockIt();
Service service = new Service(mockDownloader, new SycnronousService());
List<String> tasks = Arrays.asList("A", "B");
service.doWork(tasks);
verify(mockDownloader).download("A"); //verify in your way with EasyMock
verify(mockDownloader).download("B"); //verify in your way with EasyMock
// no more Timer.sleep() , test runs immeadetely
}
}
You need to replace CompletableFuture to something like in my example, because
unit testing this code not able in this way.
Later in you app you will be able to replace SycnronousService to asynchronous implementation and all will be work as expected.
I agree with #joy-dir 's answer. And you should probably do what she said to simplify your testing.
For the sake of completeness, your problem here is that the verify is called before your tasks being actually finished. There are many things you could do.
One is to loop on verify.
#Test
public void test() throws Exception {
List<String> events = new ArrayList();
events.add("http://xxxx//");
events.add("http://xxxx//");
expect(downloader.send(events.get(0))).andReturn("xxx").times(2);
replay(downloader);
report(events);
for (int i = 0; i < 10; i++) {
try {
verify(downloader);
return;
} catch(AssertionError e) {
// wait until it works
}
Thread.sleep(10);
}
verify(downloader);
}
It won't sleep a long time for nothing when successful. However, you indeed need to make sure you wait enough to prevent the test from being flaky.
Another solution is actually to use the CompletableFuture returned by runAsync. I prefer this solution.
public CompletableFuture<Void> report(List<String> srcList) {
if (srcList != null) {
return CompletableFuture.runAsync(() -> srcList.forEach(src-> downloader.send(src)));
}
return CompletableFuture.completedFuture(null);
}
#Test
public void test2() throws Exception {
List<String> events = new ArrayList();
events.add("http://xxxx//");
events.add("http://xxxx//");
expect(downloader.send(events.get(0))).andReturn("xxx").times(2);
replay(downloader);
CompletableFuture<Void> future = report(events);
future.get(100, TimeUnit.MILLISECONDS);
verify(downloader);
}
Finally, there is a hackish way. You ask the common pool if it is done. It is hackish because something else might use it. So it's cute but I wouldn't really recommend it.
#Test
public void test3() throws Exception {
List<String> events = new ArrayList();
events.add("http://xxxx//");
events.add("http://xxxx//");
expect(downloader.send(events.get(0))).andReturn("xxx").times(2);
replay(downloader);
report(events);
while(!ForkJoinPool.commonPool().isQuiescent()) {
Thread.sleep(10);
}
verify(downloader);
}
I have an application that uses a ConnectableObservable that runs for a long time. Mysteriously after some time its observer stopped getting notifications in its onNext() method.
I have written the following test that simplifies the example. It's just a ConnectableObservable with an infinite loop, with one subscriber using both observeOn and subscribeon. After 128 s.onNext(1) calls it stops notifying the observer.
#Test
public void testHotObservable() throws InterruptedException{
CountDownLatch latch = new CountDownLatch(1);
ConnectableObservable<Integer> observable = Observable.<Integer>create( (s) -> {
while(true){
try {
Thread.sleep(500);
} catch (Exception e) {
e.printStackTrace();
}
s.onNext(1);
}
})
.observeOn(Schedulers.io())
.subscribeOn(Schedulers.io())
.publish();
Observer<Integer> observer = new Observer<Integer>() {
#Override
public void onNext(Integer i) {
System.out.println("got "+i);
}
#Override
public void onCompleted() {
System.out.println("completed");
}
#Override
public void onError(Throwable e) {
e.printStackTrace();
}
};
observable.subscribe(observer);
observable.connect();
latch.await();
}
This is what I've seen debugging RxJava's code I have found out the reason why it doesn't call the Observer's onNext() method but I don't understand it:
1.- s.onNext(1); is called:
2.- The execution gets to rx.internal.operators.OperatorObserveOn.ObserveOnSubscriber.pollQueue():
void pollQueue() {
int emitted = 0;
final AtomicLong localRequested = this.requested;
final AtomicLong localCounter = this.counter;
do {
localCounter.set(1);
long produced = 0;
long r = localRequested.get();
for (;;) {
...
System.out.println("R: "+r);
if (r > 0) {
Object o = queue.poll();
if (o != null) {
child.onNext(on.getValue(o));
r--;
The problem is the value of r. The first time it executes its value is always 128. After each call it decrements by 1 (r--). This means that ConnectableObservable can only notify its observers 128 times when using both observeOn and subscribeOn. If I remove subscribeOn, r's value starts over each iteration and it works.
UPDATE:
I found a solution: the problem was caused by the order of the .observerOn().subscribeOn(). If I reverse it to .subscribeOn().observeOn() it works (I can see that the value of r is always reset to 128).
Anyway I'd appreciate an explanation.
Many async operators use internal, fixed size buffers and rely on subscribers requesting requently. In your case, something doesn't request properly which I can't say what it is. I suggest trying your use case with standard components to see what could be wrong, i.e., you can replace your custom Observable with a PublishSubject + sample:
Subject<Integer, Integer> source = PublishSubject.<Integer>create().toSerialized();
ConnectableObservable<Integer> co = source.sample(
500, TimeUnit.MILLISECONDS, Schedulers.io())
.onBackpressureBuffer().publish();
co.subscribe(yourSubscriber);
co.connect();
source.onNext(1);