Develop Reference

Android RxJava process http response rows in separate threads

I'm studying RxJava to see if I can use it to replace the deprecated AsynTasks in an application created several years ago.
my use case is as follows:
make an http request on Schedulers.io that returns some rows
process the rows separately, in parallel threads
update the UI on main thread only when all rows have been processed
is there a way to do step 2 easily in rx java?
Below is a code example.
Thanks
Observable.fromCallable(()-> {
// 1- get rows form server
ArrayList<HashMap<String, Object>> rows = new ArrayList<HashMap<String, Object>>();
// 2- process rows
for (HashMap row : rows) {
//manipulate row
row.put("test", "test"); <-- code that I want to parallelize
}
return rows;
})
.subscribeOn(Schedulers.io())// Execute in IO thread, i.e. background thread.
.observeOn(AndroidSchedulers.mainThread())// report or post the result to main thread.
.subscribeWith(new Observer<ArrayList<HashMap<String, Object>>>() {
#Override
public void onSubscribe(#NonNull Disposable d) {
}
#Override
public void onNext(#NonNull ArrayList<HashMap<String, Object>> hashMaps) {
}
#Override
public void onError(#NonNull Throwable e) {
}
#Override
public void onComplete() {
//3- update UI....
}
});

After several attempts I came to this solution.
By adding logs to the processRow function I saw that it is called in parallel for multiple rows, as always, at the end the onComplete is called.
Observable.fromCallable(() -> getListResponse()) // 1- get rows form server
.subscribeOn(Schedulers.io())
.flatMapIterable(rowItem -> rowItem)
.flatMap(val -> Observable.just(val) //paralelize
.subscribeOn(Schedulers.computation())
.map(i -> processRow(i) )) // 2- process rows in parallel threads
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Observer<Object>() {
#Override
public void onSubscribe(#NonNull Disposable d) {
}
#Override
public void onNext(#NonNull Object listResponse) { }
#Override
public void onError(#NonNull Throwable e) {
e.printStackTrace();
}
#Override
public void onComplete() {
//3- update UI....
}
});

RXJava retrofit waiting having the UI wait for data to be fetched from API

I'm learning android development I'm using RXJava with retrofit to interact with an API. So i've successfully managed to GET Data from my API. The problem now is that the program continues before the data has been fetched. How can I wait for the data to be downloaded from the API and then run some function?
I have a retrofit client as shown
public class RetrofitClient {
private static Retrofit retrofit = null;
private RetrofitClient() {
}
public static Retrofit getClient(String baseUrl) {
if (retrofit == null) {
retrofit = new Retrofit.Builder()
.baseUrl(baseUrl)
.addConverterFactory(SimpleXmlConverterFactory.create())
.addCallAdapterFactory(RxJavaCallAdapterFactory.create())
.build();
}
return retrofit;
}
}
I do an API call as such.
public void getData(MyFragment Fragment) {
mAPIService.getData()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Subscriber<Data>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable e) {
Log.i("ERROR IN GET DATA", e.toString());
}
#Override
public void onNext(Data response) {
Log.i("MY DATA", response.toString());
fragment.downloadData(response);
}
});
}
the problem is my android application does not wait for fragment.downloadData(response) to finish but instead continues executing code and then crashes because response is null.
I have a listener on a button that when clicked gets data from the API
button.setOnClickListener(v ->{
APICaller.getData(this);
Log.i("TEST", data.ToString()); //THIS IS NULL
});
This is the downloadData function that I run from the APICaller
public void downloadData(Data data) {
this.data = data;
}

You need to be waiting for your RxJava stream to emit a value (either error or response).
Firstly, for this, if you're expecting a "single" emission, success or failure, I would use a Single. At the moment it looks your mAPIService.getData() method is returning an Observable. These are meant for streams that are going to emit multiple values which in your cause I am assuming is not what is going to happen. You only have one item that is going to be emitted so I would look at returning a Single. Not part of your question but FYI.
What I like to do is to tell my UI that whatever I'm doing is "loading", normally in the doOnSubscribe. Then the UI knows to show a loading icon or to not allow user interactions or something. Something like this (nb. notice how its after the observeOn(AndroidSchedulers.mainThread). Any time you interact with UI elements, do it on the main thread. I think this will do it):
mAPIService.getData()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.doOnSubscribe(new Consumer<Disposable>() {
#Override
public void accept(Disposable disposable) throws Exception {
fragment.loading();
}
})
Then when either the onError or onSuccess returns in your subscription, that is where you tell the UI it's ready to proceed. You then either have a valid response or can show the error to the user.
EDIT: Reactive Programming
After your comments it looks like you do not understand reactive programming. It has a bit of a steep learning curve and I still struggle with it today.
Your APICaller class, whatever it is, should return the Observable itself. You shouldn't be passing in a Fragment to this and handling it within there as you're opening yourself up to memory leaks and its a bit of a code smell. A better option is to just return the Observable returned by mAPIService.getData(). That's it. At the moment you are pushing the Observable to another thread using Schedulers.io() which says to your main thread, carry on and don't wait for me. You then come back to this thread when a response is emitted using the code .observeOn(AndroidSchedulers.mainThread())
In your fragment is then where you handle the emission of a value or an error. Your fragment code then becomes:
button.setOnClickListener(v ->{
APICaller.getData()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Subscriber<Data>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable e) {
Log.i("ERROR IN GET DATA", e.toString());
}
#Override
public void onNext(Data response) {
Log.i("MY DATA", response.toString());
this.data = response
Log.i("TEST: "+data.toString());
}
});
});
I would like to recommend you watch some tutorials, do some reading and I'd also like to refer you back to my point about Singles at the start

Make multiple asynchronous calls(fire and forget calls) at once using Rx java Observable

I have a list of downstream api calls(about 10) that I need to call at once asynchronously. I was using callables till now where I was using
List<RequestContextPreservingCallable <FutureResponse>> callables
I would add the api calls to this list and submit it at the end using executeAsyncNoReturnRequestContextPreservingCallables.
Using Rx java Observables how do I do this?
List<RequestContextPreservingCallable<FutureResponse>> callables = new
ArrayList<RequestContextPreservingCallable<FutureResponse>>();
callables.add(apiOneConnector.CallToApiOne(name));
callables.add(apiTwoConnector.CallToApiTWO(sessionId));
....
//execute all the calls
executeAsyncNoReturnRequestContextPreservingCallables(callables);

You could make use of the zip operator. The zip operator can take multiple observables and execute them simultaneously, and it will proceed after all the results have arrived.
You could then transform these result into your needed form and pass to the next level.
As per your example. Say you have multiple API calls for getting name and session etc, as shown below
Observable.zip(getNameRequest(), getSessionIdRequest(), new BiFunction<String, String, Object>() {
#Override
public Object apply(String name, String sessionId) throws Exception {
// here you will get all the results once everything is completed. you can then take these
// results and transform into another object and returnm from here. I decided to transform the results into an Object[]
// the retuen type of this apply funtion is generic, so you can choose what to return
return new Object[]{name, sessionId};
}
})
.subscribeOn(Schedulers.io()) // will start this entire chain in an IO thread
.observeOn(AndroidSchedulers.mainThread()) // observeOn will filp the thread to the given one , so that the downstream will be executed in the specified thread. here I'm switching to main at this point onwards
.subscribeWith(new DisposableObserver<Object>() {
#Override
public void onNext(Object finalResult) {
// here you will get the final result with all the api results
}
#Override
public void onError(Throwable e) {
// any error during the entire process will be triggered here
}
#Override
public void onComplete() {
//will be called once the whole chain is completed and terminated
}
});
You could even pass a list of observables to the zip as follows
List<Observable<String>> requests = new ArrayList<>();
requests.add(getNameRequest());
requests.add(getSessionIdRequest());
Observable.zip(requests, new Function<Object[], Object[]>() {
#Override
public Object[] apply(Object[] objects) throws Exception {
return new Object[]{objects[0], objects[1]};
}
}).subscribeWith(new DisposableObserver<Object[]>() {
#Override
public void onNext(Object[] objects) {
}
#Override
public void onError(Throwable e) {
}
#Override
public void onComplete() {
}
})

Observable with buffer and multiple value updates in Java RX?

I am new using java RX and I am facing a problem, hopefully someone can give me a clue what Im doing wrong.
The issue:
There are many events I am tracking, such events are triggered like this:
Observable<Long> otherObservable = Observable.empty();
public void myMethod(){
Observable<Long> observable1 = Observable.timer(VARIABLE_TIME, TimeUnit.SECONDS);
final Subscriber<Long> timeSubscriber = new Subscriber<Long>() {
#Override
public void onCompleted() {
// nothing really
}
#Override
public void onError(final Throwable throwable) {
// nothing really
}
#Override
public void onNext(final Long number) {
// Here i do something
}
};
return Observable.merge(timerObservable, otherObservable)
.first()
.subscribe(timeSubscriber);
}
So basically it fires an event after a VARIABLE_TIME.
It works great but now I am facing the fact I have too many events.
So I thought about using debounce and buffer.
What Im trying to do is this:
Still create many observables that emit an event after N seconds.
Collect info from each of them (a long or maybe a String)
After a delay time (buffer time) Send a list with all the collected info to the subscriber.
So far Ive done this:
Observable<List<Long>> otherObservable = Observable.empty();
otherObservable.debounce(10L, SECONDS).buffer(20L, SECONDS);
Observable<List<Long>> observable1 = Observable.timer(VARIABLE_TIME, TimeUnit.SECONDS).buffer(1);
Subscriber<List<Long> > observerSuscriber = new Subscriber<List<Long>>() {
#Override
public void onCompleted() {
}
#Override
public void onError(final Throwable throwable) {
}
#Override
public void onNext(final List<Long> ids ) {
// do something here
}
};
Observable.merge(otherObservable, observable1)
.first()
.subscribe(observerSuscriber);
But like this I still get the message instantly after emitted.
I am wondering if there is anyway to do this? Any ideas? I am using java RX 1.2

After a delay time (buffer time) Send a list with all the collected info to the subscriber.
You answered your own question there! Use the buffer operator:
Flowable<String> stream = ...
Flowable<List<String>> lists = stream.buffer(5, TimeUnit.SECONDS);

How to send multiple asynchronous requests to different web services?

I need to send multiple requests to many different web services and receive the results. The problem is that, if I send the requests one by one it takes so long as I need to send and process all individually.
I am wondering how I can send all the requests at once and receive the results.
As the following code shows, I have three major methods and each has its own sub methods.
Each sub method sends request to its associated web service and receive the results;therefore, for example, to receive the results of web service 9 I have to wait till all web services from 1 to 8 get completed, it takes a long time to send all the requests one by one and receive their results.
As shown below none of the methods nor sub-methods are related to each other, so I can call them all and receive their results in any order, the only thing which is important is to receive the results of each sub-method and populate their associated lists.
private List<StudentsResults> studentsResults = new ArrayList();
private List<DoctorsResults> doctorsResults = new ArrayList();
private List<PatientsResults> patientsResults = new ArrayList();
main (){
retrieveAllLists();
}
retrieveAllLists(){
retrieveStudents();
retrieveDoctors();
retrievePatients();
}
retrieveStudents(){
this.studentsResults = retrieveStdWS1(); //send request to Web Service 1 to receive its list of students
this.studentsResults = retrieveStdWS2(); //send request to Web Service 2 to receive its list of students
this.studentsResults = retrieveStdWS3(); //send request to Web Service 3 to receive its list of students
}
retrieveDoctors(){
this.doctorsResults = retrieveDocWS4(); //send request to Web Service 4 to receive its list of doctors
this.doctorsResults = retrieveDocWS5(); //send request to Web Service 5 to receive its list of doctors
this.doctorsResults = retrieveDocWS6(); //send request to Web Service 6 to receive its list of doctors
}
retrievePatients(){
this.patientsResults = retrievePtWS7(); //send request to Web Service 7 to receive its list of patients
this.patientsResults = retrievePtWS8(); //send request to Web Service 8 to receive its list of patients
this.patientsResults = retrievePtWS9(); //send request to Web Service 9 to receive its list of patients
}

That is a simple fork-join approach, but for clarity, you can start any number of threads and retrieve the results later as they are available, such as this approach.
ExecutorService pool = Executors.newFixedThreadPool(10);
List<Callable<String>> tasks = new ArrayList<>();
tasks.add(new Callable<String>() {
public String call() throws Exception {
Thread.sleep((new Random().nextInt(5000)) + 500);
return "Hello world";
}
});
List<Future<String>> results = pool.invokeAll(tasks);
for (Future<String> future : results) {
System.out.println(future.get());
}
pool.shutdown();
UPDATE, COMPLETE:
Here's a verbose, but workable solution. I wrote it ad hoc, and have not compiled it.
Given the three lists have diffent types, and the WS methods are individual, it is not
really modular, but try to use your best programming skills and see if you can modularize it a bit better.
ExecutorService pool = Executors.newFixedThreadPool(10);
List<Callable<List<StudentsResults>>> stasks = new ArrayList<>();
List<Callable<List<DoctorsResults>>> dtasks = new ArrayList<>();
List<Callable<List<PatientsResults>>> ptasks = new ArrayList<>();
stasks.add(new Callable<List<StudentsResults>>() {
public List<StudentsResults> call() throws Exception {
return retrieveStdWS1();
}
});
stasks.add(new Callable<List<StudentsResults>>() {
public List<StudentsResults> call() throws Exception {
return retrieveStdWS2();
}
});
stasks.add(new Callable<List<StudentsResults>>() {
public List<StudentsResults> call() throws Exception {
return retrieveStdWS3();
}
});
dtasks.add(new Callable<List<DoctorsResults>>() {
public List<DoctorsResults> call() throws Exception {
return retrieveDocWS4();
}
});
dtasks.add(new Callable<List<DoctorsResults>>() {
public List<DoctorsResults> call() throws Exception {
return retrieveDocWS5();
}
});
dtasks.add(new Callable<List<DoctorsResults>>() {
public List<DoctorsResults> call() throws Exception {
return retrieveDocWS6();
}
});
ptasks.add(new Callable<List<PatientsResults>>() {
public List<PatientsResults> call() throws Exception {
return retrievePtWS7();
}
});
ptasks.add(new Callable<List<PatientsResults>>() {
public List<PatientsResults> call() throws Exception {
return retrievePtWS8();
}
});
ptasks.add(new Callable<List<PatientsResults>>() {
public List<PatientsResults> call() throws Exception {
return retrievePtWS9();
}
});
List<Future<List<StudentsResults>>> sresults = pool.invokeAll(stasks);
List<Future<List<DoctorsResults>>> dresults = pool.invokeAll(dtasks);
List<Future<List<PatientsResults>>> presults = pool.invokeAll(ptasks);
for (Future<List<StudentsResults>> future : sresults) {
this.studentsResults.addAll(future.get());
}
for (Future<List<DoctorsResults>> future : dresults) {
this.doctorsResults.addAll(future.get());
}
for (Future<List<PatientsResults>> future : presults) {
this.patientsResults.addAll(future.get());
}
pool.shutdown();
Each Callable returns a list of results, and is called in its own separate thread.
When you invoke the Future.get() method you get the result back onto the main thread.
The result is NOT available until the Callable have finished, hence there is no concurrency issues.

So just for fun I am providing two working examples. The first one shows the old school way of doing this before java 1.5. The second shows a much cleaner way using tools available within java 1.5:
import java.util.ArrayList;
public class ThreadingExample
{
private ArrayList <MyThread> myThreads;
public static class MyRunnable implements Runnable
{
private String data;
public String getData()
{
return data;
}
public void setData(String data)
{
this.data = data;
}
#Override
public void run()
{
}
}
public static class MyThread extends Thread
{
private MyRunnable myRunnable;
MyThread(MyRunnable runnable)
{
super(runnable);
setMyRunnable(runnable);
}
/**
* #return the myRunnable
*/
public MyRunnable getMyRunnable()
{
return myRunnable;
}
/**
* #param myRunnable the myRunnable to set
*/
public void setMyRunnable(MyRunnable myRunnable)
{
this.myRunnable = myRunnable;
}
}
public ThreadingExample()
{
myThreads = new ArrayList <MyThread> ();
}
public ArrayList <String> retrieveMyData ()
{
ArrayList <String> allmyData = new ArrayList <String> ();
if (isComplete() == false)
{
// Sadly we aren't done
return (null);
}
for (MyThread myThread : myThreads)
{
allmyData.add(myThread.getMyRunnable().getData());
}
return (allmyData);
}
private boolean isComplete()
{
boolean complete = true;
// wait for all of them to finish
for (MyThread x : myThreads)
{
if (x.isAlive())
{
complete = false;
break;
}
}
return (complete);
}
public void kickOffQueries()
{
myThreads.clear();
MyThread a = new MyThread(new MyRunnable()
{
#Override
public void run()
{
// This is where you make the call to external services
// giving the results to setData("");
setData("Data from list A");
}
});
myThreads.add(a);
MyThread b = new MyThread (new MyRunnable()
{
#Override
public void run()
{
// This is where you make the call to external services
// giving the results to setData("");
setData("Data from list B");
}
});
myThreads.add(b);
for (MyThread x : myThreads)
{
x.start();
}
boolean done = false;
while (done == false)
{
if (isComplete())
{
done = true;
}
else
{
// Sleep for 10 milliseconds
try
{
Thread.sleep(10);
}
catch (InterruptedException e)
{
e.printStackTrace();
}
}
}
}
public static void main(String [] args)
{
ThreadingExample example = new ThreadingExample();
example.kickOffQueries();
ArrayList <String> data = example.retrieveMyData();
if (data != null)
{
for (String s : data)
{
System.out.println (s);
}
}
}
}
This is the much simpler working version:
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class ThreadingExample
{
public static void main(String [] args)
{
ExecutorService service = Executors.newCachedThreadPool();
Set <Callable<String>> callables = new HashSet <Callable<String>> ();
callables.add(new Callable<String>()
{
#Override
public String call() throws Exception
{
return "This is where I make the call to web service A, and put its results here";
}
});
callables.add(new Callable<String>()
{
#Override
public String call() throws Exception
{
return "This is where I make the call to web service B, and put its results here";
}
});
callables.add(new Callable<String>()
{
#Override
public String call() throws Exception
{
return "This is where I make the call to web service C, and put its results here";
}
});
try
{
List<Future<String>> futures = service.invokeAll(callables);
for (Future<String> future : futures)
{
System.out.println (future.get());
}
}
catch (InterruptedException e)
{
e.printStackTrace();
}
catch (ExecutionException e)
{
e.printStackTrace();
}
}
}

You can ask your jax-ws implementation to generate asynchronous bindings for the web service.
This has two advantages that I can see:
As discussed in Asynchronous web services calls with JAX-WS: Use wsimport support for asynchrony or roll my own? , jax-ws will generate well-tested (and possibly fancier) code for you, you need not instantiate the ExecutorService yourself. So less work for you! (but also less control over the threading implementation details)
The generated bindings include a method where you specify a callback handler, which may suit your needs better than synchronously get() ting all response lists on the thread calling retrieveAllLists(). It allows for per-service-call error handling and will process the results in parallel, which is nice if processing is non-trivial.
An example for Metro can be found on the Metro site. Note the contents of the custom bindings file custom-client.xml :
<bindings ...>
<bindings node="wsdl:definitions">
<enableAsyncMapping>true</enableAsyncMapping>
</bindings>
</bindings>
When you specify this bindings file to wsimport, it'll generate a client which returns an object that implements javax.xml.ws.Response<T>. Response extends the Future interface that others also suggest you use when rolling your own implementation.
So, unsurprisingly, if you go without the callbacks, the code will look similar to the other answers:
public void retrieveAllLists() throws ExecutionException{
// first fire all requests
Response<List<StudentsResults>> students1 = ws1.getStudents();
Response<List<StudentsResults>> students2 = ws2.getStudents();
Response<List<StudentsResults>> students3 = ws3.getStudents();
Response<List<DoctorsResults>> doctors1 = ws4.getDoctors();
Response<List<DoctorsResults>> doctors2 = ws5.getDoctors();
Response<List<DoctorsResults>> doctors3 = ws6.getDoctors();
Response<List<PatientsResults>> patients1 = ws7.getPatients();
Response<List<PatientsResults>> patients2 = ws8.getPatients();
Response<List<PatientsResults>> patients3 = ws9.getPatients();
// then await and collect all the responses
studentsResults.addAll(students1.get());
studentsResults.addAll(students2.get());
studentsResults.addAll(students3.get());
doctorsResults.addAll(doctors1.get());
doctorsResults.addAll(doctors2.get());
doctorsResults.addAll(doctors3.get());
patientsResults.addAll(patients1.get());
patientsResults.addAll(patients2.get());
patientsResults.addAll(patients3.get());
}
If you create callback handers such as
private class StudentsCallbackHandler
implements AsyncHandler<Response<List<StudentsResults>>> {
public void handleResponse(List<StudentsResults> response) {
try {
studentsResults.addAll(response.get());
} catch (ExecutionException e) {
errors.add(new CustomError("Failed to retrieve Students.", e.getCause()));
} catch (InterruptedException e) {
log.error("Interrupted", e);
}
}
}
you can use them like this:
public void retrieveAllLists() {
List<Future<?>> responses = new ArrayList<Future<?>>();
// fire all requests, specifying callback handlers
responses.add(ws1.getStudents(new StudentsCallbackHandler()));
responses.add(ws2.getStudents(new StudentsCallbackHandler()));
responses.add(ws3.getStudents(new StudentsCallbackHandler()));
...
// await completion
for( Future<?> response: responses ) {
response.get();
}
// or do some other work, and poll response.isDone()
}
Note that the studentResults collection needs to be thread safe now, since results will get added concurrently!

Looking at the problem, you need to integrate your application with 10+ different webservices.While making all the calls asynchronous. This can be done easily with Apache Camel. It is a prominent framework for enterprise integration and also supports async processing. You can use its CXF component for calling webservices and its routing engine for invocation and processing results. Look at the following page regarding camel's async routing capability. They have also provided a complete example invoking webservices async using CXF, it available at its maven repo. Also see the following page for more details.

You might consider the following paradigm in which you create work (serially), but the actual work is done in parallel. One way to do this is to: 1) have your "main" create a queue of work items; 2) create a "doWork" object that queries the queue for work to do; 3) have "main" start some number of "doWork" threads (can be same number as number of different services, or a smaller number); have the "doWork" objects put add their results to an object list (whatever construct works Vector, list...).
Each "doWork" object would mark their queue item complete, put all results in the passed container and check for new work (if no more on the queue, it would sleep and try again).
Of course you will want to see how well you can construct your class model. If each of the webservices is quite different for parsing, then you may want to create an Interface that each of your "retrieveinfo" classes promises to implement.

It has got various option to develop this.
JMS : quality of service and management, e.g. redelivery attempt, dead message queue, load management, scalability, clustering, monitoring, etc.
Simply using the Observer pattern for this. For more details OODesign and How to solve produce and consumer follow this Kodelog**