I have a webapp in which I have to return the results from a mongodb find() to the front-end from my java back-end.
I am using the Async Java driver, and the only way I think I have to return the results from mongo is something like this:
public String getDocuments(){
...
collection.find(query).map(Document::toJson)
.into(new HashSet<String>(), new SingleResultCallback<HashSet<String>>() {
#Override
public void onResult(HashSet<String> strings, Throwable throwable) {
// here I have to get all the Json Documents in the set,
// make a whole json string and wake the main thread
}
});
// here I have to put the main thread to wait until I get the data in
// the onResult() method so I can return the string back to the front-end
...
return jsonString;
}
Is this assumption right or there´s another way to do it?
Asynchronous APIs (any API based on callbacks, not necessarily MongoDB) can be a true blessing for multithreaded applications. But to really benefit from them, you need to design your whole application architecture in an asynchronous fashion. This is not always feasible, especially when it is supposed to fit into a given framework which isn't built on callbacks.
So sometimes (like in your case) you just want to use an asynchronous API in a synchronous fashion. In that case, you can use the class CompletableFuture.
This class provides (among others) two methods <T> get() and complete(<T> value). The method get will block until complete is called to provide the return value (should complete get called before get, get returns immediately with the provided value).
public String getDocuments(){
...
CompletableFuture<String> result = new CompletableFuture<>(); // <-- create an empty, uncompleted Future
collection.find(query).map(Document::toJson)
.into(new HashSet<String>(), new SingleResultCallback<HashSet<String>>() {
#Override
public void onResult(HashSet<String> strings, Throwable throwable) {
// here I have to get all the Json Documents in the set and
// make a whole json string
result.complete(wholeJsonString); // <--resolves the future
}
});
return result.get(); // <-- blocks until result.complete is called
}
The the get()-method of CompletableFuture also has an alternative overload with a timeout parameter. I recommend using this to prevent your program from accumulating hanging threads when the callback is not called for whatever reason. It will also be a good idea to implement your whole callback in a try { block and do the result.complete in the finally { block to make sure the result always gets resolved, even when there is an unexpected error during your callback.
Yes, you're right.
That's the correct behaviour of Mongo async driver (see MongoIterable.into).
However, Why don't you use sync driver in this situation? Is there any reason to use async method?
Related
The idea is that for REST API I have to return the updated data from a server: e.g., my server method should return Single<Data> but for DB it's have a method that return Completable, it results in the following code:
public Single<Player> updateShirtNumber(int number) {
Player player = new Player();
player.setShirtNumber(number);
return playersDao.savePlayerCompletable(player).andThen(Single.just(player));
}
How can I avoid this andThen(Single.just(player)) of things?
You can't avoid this unless you want/can change DB API (and, basically, do not use reactive approach in the first place, but in this case you don't need rx-java).
Think savePlayerCompletable as asynchronous method. Before you return the value from REST API you want to make sure that save operation finishes. That's what andThen does.
I'm currently checking out the following guide: https://developer.android.com/topic/libraries/architecture/guide.html
The networkBoundResource class:
// ResultType: Type for the Resource data
// RequestType: Type for the API response
public abstract class NetworkBoundResource<ResultType, RequestType> {
// Called to save the result of the API response into the database
#WorkerThread
protected abstract void saveCallResult(#NonNull RequestType item);
// Called with the data in the database to decide whether it should be
// fetched from the network.
#MainThread
protected abstract boolean shouldFetch(#Nullable ResultType data);
// Called to get the cached data from the database
#NonNull #MainThread
protected abstract LiveData<ResultType> loadFromDb();
// Called to create the API call.
#NonNull #MainThread
protected abstract LiveData<ApiResponse<RequestType>> createCall();
// Called when the fetch fails. The child class may want to reset components
// like rate limiter.
#MainThread
protected void onFetchFailed() {
}
// returns a LiveData that represents the resource
public final LiveData<Resource<ResultType>> getAsLiveData() {
return result;
}
}
I'm a bit confused here about the use of threads.
Why is #MainThread applied here for networkIO?
Also, for saving into the db, #WorkerThread is applied, whereas #MainThread for retrieving results.
Is it bad practise to use a worker thread by default for NetworkIO and local db interaction?
I'm also checking out the following demo (GithubBrowserSample): https://github.com/googlesamples/android-architecture-components
This confuses me from a threading point of view.
The demo uses executors framework, and defines a fixed pool with 3 threads for networkIO, however in the demo only a worker task is defined for one call, i.e. the FetchNextSearchPageTask. All other network requests seem to be executed on the main thread.
Can someone clarify the rationale?
It seems you have a few misconceptions.
Generally it is never OK to call network from the Main (UI) thread but unless you have a lot of data it might be OK to fetch data from DB in the Main thread. And this is what Google example does.
1.
The demo uses executors framework, and defines a fixed pool with 3 threads for networkIO, however in the demo only a worker task is defined for one call, i.e. the FetchNextSearchPageTask.
First of all, since Java 8 you can create simple implementation of some interfaces (so called "functional interfaces") using lambda syntax. This is what happens in the NetworkBoundResource:
appExecutors.diskIO().execute(() -> {
saveCallResult(processResponse(response));
appExecutors.mainThread().execute(() ->
// we specially request a new live data,
// otherwise we will get immediately last cached value,
// which may not be updated with latest results received from network.
result.addSource(loadFromDb(),
newData -> result.setValue(Resource.success(newData)))
);
});
at first task (processResponse and saveCallResult) is scheduled on a thread provided by the diskIO Executor and then from that thread the rest of the work is scheduled back to the Main thread.
2.
Why is #MainThread applied here for networkIO?
and
All other network requests seem to be executed on the main thread.
This is not so. Only result wrapper i.e. LiveData<ApiResponse<RequestType>> is created on the main thread. The network request is done on a different thread. This is not easy to see because Retrofit library is used to do all the network-related heavy lifting and it nicely hides such implementation details. Still, if you look at the LiveDataCallAdapter that wraps Retrofit into a LiveData, you can see that Call.enqueue is used which is actually an asynchronous call (scheduled internally by Retrofit).
Actually if not for "pagination" feature, the example would not need networkIO Executor at all. "Pagination" is a complicated feature and thus it is implemented using explicit FetchNextSearchPageTask and this is a place where I think Google example is done not very well: FetchNextSearchPageTask doesn't re-use request parsing logic (i.e. processResponse) from RepoRepository but just assumes that it is trivial (which it is now, but who knows about the future...). Also there is no scheduling of the merging job onto the diskIO Executor which is also inconsistent with the rest of the response processing.
I have a couple domain objects, Message and Contact:
public class Contact {
String name;
}
public class Message {
String body;
Contact contact;
}
I'm populating a list of Message and showing them the user. Contact info for each of the Messages is retrieved asynchronously, and then the list is updated. This is basically how I have it set up:
listAdapter.setDataSet(listOfMessages);
for (Message message : listOfMessages) {
fetchContactDetails(message);
}
...
fetchContactDetails(Message message) {
contactService
.fetchContactDetails(listOfMessages)
.subscribeOn... // observe, etc
.observe(new Observable<Contact>() {
onNext(Contact contact) {
message.setContact(contact);
list.notifyChanged(message);
}
});
This feels like a lot of code, given that I could instead just update each Message Contact within the service function contactService.fetchContactDetails. On the contrary, it feels unclean to use a service method to modify the object passed in without returning anything.
Is it a bad practice to use a service function to update a Object passed in as an argument, without returning anything?
Is it a bad practice to use a service function to update a Object passed in as an argument, without returning anything?
No, you can implement void method Observable, by using Completable, but in your case you will still need to update the UI on the main thread, so you will subscribe and handle the onNext on mainThread (by adding observeOn(AndroidSchedulers.mainThread())).
So in this case your method not seems to be pure void method that returns nothing, but method that updated some data and return it to the UI for processing.
So, I think it is more general architectural question, of how to separate the code right. I would defiantly gather the model/domain logic together in separate Observable, and let the UI/presentation handle the just the UI updates.
BTW, you can loop more elegantly with Rx using from() and flatMap()
Observable.from(listOfMessages)
.flatMap(msg -> contactService.fetchContactDetails(msg))
then you can have control on the enitre process, like do something when all updates done, or limit the parallelism or whatever.
I need to geocode an Address object, and then store the updated Address in a search engine. This can be simplified to taking an object, performing one long-running operation on the object, and then persisting the object. This means there is an order of operations requirement that the first operation be complete before persistence occurs.
I would like to use Akka to move this off the main thread of execution.
My initial thought was to use a pair of Futures to accomplish this, but the Futures documentation is not entirely clear on which behavior (fold, map, etc) guarantees one Future to be executed before another.
I started out by creating two functions, defferedGeocode and deferredWriteToSearchEngine which return Futures for the respective operations. I chain them together using Future<>.andThen(new OnComplete...), but this gets clunky very quickly:
Future<Address> geocodeFuture = defferedGeocode(ec, address);
geocodeFuture.andThen(new OnComplete<Address>() {
public void onComplete(Throwable failure, Address geocodedAddress) {
if (geocodedAddress != null) {
Future<Address> searchEngineFuture = deferredWriteToSearchEngine(ec, addressSearchService, geocodedAddress);
searchEngineFuture.andThen(new OnComplete<Address>() {
public void onComplete(Throwable failure, Address savedAddress) {
// process search engine results
}
});
}
}
}, ec);
And then deferredGeocode is implemented like this:
private Future<Address> defferedGeocode(
final ExecutionContext ec,
final Address address) {
return Futures.future(new Callable<Address>() {
public Address call() throws Exception {
log.debug("Geocoding Address...");
return address;
}
}, ec);
};
deferredWriteToSearchEngine is pretty similar to deferredGeocode, except it takes the search engine service as an additional final parameter.
My understand is that Futures are supposed to be used to perform calculations and should not have side effects. In this case, geocoding the address is calculation, so I think using a Future is reasonable, but writing to the search engine is definitely a side effect.
What is the best practice here for Akka? How can I avoid all the nested calls, but ensure that both the geocoding and the search engine write are done off the main thread?
Is there a more appropriate tool?
Update:
Based on Viktor's comments below, I am trying this code out now:
ExecutionContext ec;
private Future<Address> addressBackgroundProcess(Address address) {
Future<Address> geocodeFuture = addressGeocodeFutureFactory.defferedGeocode(address);
return geocodeFuture.flatMap(new Mapper<Address, Future<Address>>() {
#Override
public Future<Address> apply(Address geoAddress) {
return addressSearchEngineFutureFactory.deferredWriteToSearchEngine(geoAddress);
}
}, ec);
}
This seems to work ok except for one issue which I'm not thrilled with. We are working in a Spring IOC code base, and so I would like to inject the ExecutionContext into the FutureFactory objects, but it seems wrong for this function (in our DAO) to need to be aware of the ExecutionContext.
It seems odd to me that the flatMap() function needs an EC at all, since both futures provide one.
Is there a way to maintain the separation of concerns? Am I structuring the code badly, or is this just the way it needs to be?
I thought about creating an interface in the FutureFactory's that would allow chaining of FutureFactory's, so the flatMap() call would be encapsulated in a FutureFactory base class, but this seems like it would be deliberately subverting an intentional Akka design decision.
Warning: Pseudocode ahead.
Future<Address> myFutureResult = deferredGeocode(ec, address).flatMap(
new Mapper<Address, Future<Address>>() {
public Future<Address> apply(Address geocodedAddress) {
return deferredWriteToSearchEngine(ec, addressSearchService, geocodedAddress);
}
}, ec).map(
new Mapper<Address, SomeResult>() {
public SomeResult apply(Address savedAddress) {
// Create SomeResult after deferredWriteToSearchEngine is done
}
}, ec);
See how it is not nested. flatMap and map is used for sequencing the operations. "andThen" is useful for when you want a side-effecting-only operation to run to full completion before passing the result on. Of course, if you map twice on the SAME future-instance then there is no ordering guaranteed, but since we are flatMapping and mapping on the returned futures (new ones according to the docs), there is a clear data-flow in our program.
I'm integrating Akka in an existing software, mainly to make things asynchronous where they shouldn't be synchronous.
I've a service that is making some database calls, at the moment, everything is synchronous, the calling thread is just sitting here and waiting for the result...
My idea is to replace the DAO interface to have Future<T> (Akka's) as results instead of actual result types, thus, my DAO implementation is actually transforming those call in messages and routes them to appropriate actors (local and/or remote).
Now I'm a bit puzzled when it comes to how to return a Future<T> when I call the actor. Is there any other way than using Patterns.ask() ? Is it the best solution performance-wise (without rewriting everything using actors) ?
Using Patterns.ask(), how could I return an error without waiting for the timeout ? If the actor I call simply tells back the error, it would trigger a success, when I want to trigger a failure.
Edits
I'm using Java.
Right now, I came up with a construct like the one below, but, it implies my actor has to tell() the exception back to the sender.
final Future<Object> f = Patterns.ask(..., ..., ...);
f.flatMap(new Mapper<Object, Future<List<Element>>>() {
public Future<List<Element>> apply(Object response) {
if (response instanceof SuccessfulResult) {
return Futures.successful(response, f.executor());
} else if (response instanceof Throwable) {
return Futures.failed((Throwable) response, f.executor());
} else {
return Futures.failed(..., f.executor());
}
}
To differentiate between success or failure you would use scala.Either. So the answer type could look something like this:
type Result = Future[Either[SQLException, MyDataType]]