Suppose I have the following RxJava code (which accesses a DB, but the exact use case is irrelevant):
public Observable<List<DbPlaceDto>> getPlaceByStringId(final List<String> stringIds) {
return Observable.create(new Observable.OnSubscribe<List<DbPlaceDto>>() {
#Override
public void call(Subscriber<? super List<DbPlaceDto>> subscriber) {
try {
Cursor c = getPlacseDb(stringIds);
List<DbPlaceDto> dbPlaceDtoList = new ArrayList<>();
while (c.moveToNext()) {
dbPlaceDtoList.add(getDbPlaceDto(c));
}
c.close();
if (!subscriber.isUnsubscribed()) {
subscriber.onNext(dbPlaceDtoList);
subscriber.onCompleted();
}
} catch (Exception e) {
if (!subscriber.isUnsubscribed()) {
subscriber.onError(e);
}
}
}
});
}
Given this code, I have the following questions:
If someone unsubscribes from the observable returned from this method (after a previous subscription), is that operation thread-safe? So are my 'isUnsubscribed()' checks correct in this sense, regardless of scheduling?
Is there a cleaner way with less boilerplate code to check for unsubscribed states than what I'm using here? I couldn't find anything in the framework. I thought SafeSubscriber solves the issue of not forwarding events when the subscriber is unsubscribed, but apparently it does not.
is that operation thread-safe?
Yes. You are receiving an rx.Subscriber which (eventually) checks against a volatile boolean that is set to true when the subscriber's subscription is unsubscribed.
cleaner way with less boilerplate code to check for unsubscribed states
The SyncOnSubscribe and the AsyncOnSubscribe (available as an #Experimental api as of release 1.0.15) was created for this use case. They function as a safe alternative to calling Observable.create. Here is a (contrived) example of the synchronous case.
public static class FooState {
public Integer next() {
return 1;
}
public void shutdown() {
}
public FooState nextState() {
return new FooState();
}
}
public static void main(String[] args) {
OnSubscribe<Integer> sos = SyncOnSubscribe.createStateful(FooState::new,
(state, o) -> {
o.onNext(state.next());
return state.nextState();
},
state -> state.shutdown() );
Observable<Integer> obs = Observable.create(sos);
}
Note that the SyncOnSubscribe next function is not allowed to call observer.onNext more than once per iteration nor can it call into that observer concurrently. Here are a couple of links to the SyncOnSubscribe implementation and tests on the head of the 1.x branch. It's primary usage is to simplify writing observables that iterate or parsing over data synchronously and onNext downstream but doing so in a framework that supports back-pressure and checks if unsubscribed. Essentially you would create a next function which would get invoked every time the downstream operators need a new data element onNexted. Your next function can call onNext either 0 or 1 time.
The AsyncOnSubscribe is designed to play nicely with back pressure for observable sources that operate asynchronously (such as off-box calls). The arguments to your next function include the request count and your provided observable should provide an observable that fulfills data up to that requested amount. An example of this behavior would be paginated queries from an external datasource.
Previously it was a safe practice to transform your OnSubscribe to an Iterable and use Observable.from(Iterable). This implementation gets an iterator and checks subscriber.isUnsubscribed() for you.
Related
It's a simplified function.
It keeps returning 0 while I expect this to return 5. How come?
public int accessKey() {
a = 0;
mSubscription = mAccountManager.getLoginPassword()
.flatMap(loginPassword -> mServerAPI
.getProfilesList((new BaseRequest(
loginPassword.getLogin(),
loginPassword.getPassword(),
ClientGetter.getClientFromManager(),
CodeSnippets.getSha256(ClientGetter.getClientFromManager()))
)))
.doOnNext(profilesListe -> mProfilesList = profilesListe.getItems())
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.subscribe(new Subscriber<BaseResponse>() {
#Override
public void onCompleted() {
a =5;
}
return a; }
It's because of the asynchronous or non blocking behaviour , Normally during the scenarios of API calls fetching some data from database ,these behaviours we handle asynchronously,because otherwise your application will get stuck This post will help you to solve this matter. Refer this
you're executing an asynchronous operation here. your code doesn't execute "top down" here but will be executed on a different thread - Rxjava shifts this entire operation over to another thread and then returns the result to the thread specified, but this doesn't happen immediately. by the time your subscribe code has executed (we don't know when that will be) your return statement has already executed.
you could try change your code to something like this (just as an idea, i don't have any code similar to yours to create a working example):
return mAccountManager.getLoginPassword()
.flatMap(loginPassword -> mServerAPI
.getProfilesList((new BaseRequest(
loginPassword.getLogin(),
loginPassword.getPassword(),
ClientGetter.getClientFromManager(),
CodeSnippets.getSha256(ClientGetter.getClientFromManager()))
)))
.doOnNext(profilesListe -> mProfilesList = profilesListe.getItems())
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
this means that your method will now return an observable, which you can then subscribe on and subscribe to those changes where they are used.
this means that the signature of your method will have to change to support this new return type and the calling method will have to subscribe to this observable.
you describe your question as a simplified function, but I'll give you an even simpler (complete) example:
public Observable<Integer> foo() {
return Observable.just(5);
}
public void usage(){
Disposable disposable = foo().subscribeOn(something).observeOn(something).subscribe(
//inside this subscribe, like you'd normally do, you'd find the result of 5
)
...
//handle disposable
}
I used the Callback interface method and it worked! Many thanks
I have been working with some Reactor Core Java, because I want to figure out if this is possible to solve one problem I currently have using this framework.
At present I have a long, executing job that takes about 40-50 minutes to complete. The method looks more or less like this:
public void doLongTask(List<Something> list){
//instructions.
for(Something sm : list){
if(condition){
executeLongOperation();
}
//instructions
if(condition){
executeLongOperation();
}
}
}
in my controller I have something like this:
#GetMapping(path = "/integersReactor", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
#ResponseBody
public Flux<Integer> getIntegersReactor(){
logger.debug("Request getIntegersReactor initialized.");
return simpleSearchService.getIntegersReactor();
}
and in the service layer I have something like this:
#Override
public Flux<Integer> getIntegersReactor(){
return Flux.range(0, Integer.MAX_VALUE);
}
this is just a placeholder that I am using as a proof of concept. My real intentions are to somehow return a Flux of some object that I will define myself, this object will have a few fields that I will use to tell the consumer the status of the job.
Now, things get somewhat complicated now because I would like to send updates as the executeLongOperation(); are executed, and somehow instead of returning a flux of Integers, return a flux of an object that uses the return of executeLongOperation();
Can this be acomplished with Flux? How can I leverage Reactor Core java to push the return values of all of the times executeLongOperation(); is executed into a reactive stream that can be passed to the controller the same way that getIntegersReactor() does it in my example?
Yes it should be possible, but since the executeLongOperation is blocking, it will need to be offset on a dedicated thread (which reduces the benefits you get from a top-to-bottom reactive implementation).
Change your doLongTask to return a Flux<Foo>, make it concatenate Monos that wrap executeLongOperation on a dedicated thread (or better yet, change the executeLongOperation itself to return a Mono<Foo> and do the wrapping internally and subscribeOn another thread internally). Something like:
public Flux<Foo> doLongTask(List<Something> list) {
return Flux.fromIterable(list)
//ensure `Something` are published on a dedicated thread on which
//we can block
.publishOn(Schedulers.elastic()) //maybe a dedicated Scheduler?
//for each `Something`, perform the work
.flatMap(sm -> {
//in case condition is false, we'll avoid long running task
Flux<Foo> work = Flux.empty();
//start declaring the work depending on conditions
if(condition) {
Mono<Foo> op = Mono.fromCallable(this::executeLongOperation);
work = conditional.concatWith(op);
}
//all other instructions should preferably be non-blocking
//but since we're on a dedicated thread at this point, it should be ok
if(condition) {
Mono<Foo> op = Mono.fromCallable(this::executeLongOperation);
work = conditional.concatWith(op);
}
//let the flatMap trigger the work
return work;
});
}
I have a connectable observer with multiple subscribers.
Each subscriber computes some business logic. For example one of subscribers stores results in database on every onNext call, other subscriber accumulates it's results in memory and when onCompleted called writes them to file. I want to know when they all finished their work, so I can proceed in doing other stuff (aggregating with other connectable observers, read outputed data from database etc).
This is how I'm observing termination. It's only working because subscribers execute in the same thread as observer.
public Observable<Boolean> observeTermination() {
return Observable.defer(() -> {
try {
start();
return Observable.just(true);
} catch (RuntimeException e) {
return Observable.just(false);
}
});
}
void start() {
Observable<List<Foo>> fooBatchReaderObservable = fooBatchReader.createObservable(BATCH_SIZE);
ConnectableObservable<List<Foo>> connectableObservable = fooBatchReaderObservable.publish();
subscribers.forEach(s -> connectableObservable.subscribe(s));
connectableObservable.connect();
}
So when observeTermination gets called I don't want to execute logic in start method, but only when someone subscribes to it.
Is there a way to make observation better ?
Well, it's all bad. The problem is that I need to call connect on observable somewhere and also return boolean results as inication of termination.
Not a proper answer, but it needs the space to explain properly. It would be much easier if you could deal with Observables instead of Subscribers; that gives you much more flexibility in composing them;
Given that you have components:
Collection<Function<Observable<T>, Observable<?>>> components:
Observable<T> tObs = ... .publish().autoConnect(components.size());
Observable
.from(components)
.flatMap(component -> component.apply(tObs))
.ignoreElements()
.doOnTerminate() // or .defaultIfEmpty(...), or .switchIfEmpty(...)
.subscribe(...);
In fact, I'd say you should not even subscribe at all here, just create the observable and return it, let it be usable for composition in other parts of your code.
Producer-Consumer blog post states that:
"2) Producer doesn't need to know about who is consumer or how many consumers are there. Same is true with Consumer."
My problem is that I have an array of data that I need to get from the Webserver to clients as soon as possible. The clients can appear mid-calculation. Multiple clients at different times can request the array of data. Once the calculation is complete it is cached and then it can simply be read.
Exmaple Use Case: While the calculation is occurring I want to serve each and every datum of the array as soon as possible. I can't use a BlockingQueue because say if a second client starts to request the array while the first one has already used .take() on the first half of the array. Then the second client missed half the data! I need a BlockingQueue where you don't have to take(), but you could instead just read(int index).
Solution? I have a good amount of writes on my array, so I wouldn't want to use CopyOnWriteArrayList? The Vector class should work but would be inefficient?
Is it preferable to use a ThreadSafeList like this and just add a waitForElement() function? I just don't want to reinvent the wheel and I prefer crowd tested solutions for multi-threaded problems...
As far as I understand you need to broadcast data to subscribers/clients.
Here are some ways that I know for approaching it.
Pure Java solution, every client has a BlockingQueue and every time you broadcast a message you put it every queue.
for(BlockingQueue client: clients){
client.put(msg);
}
RxJava provides a reactive approach. Clients will be subscribers and ever time you emit a message, subscribers will be notified and they can choose to cancel their subscription
Observable<String> observable = Observable.create(sub->{
String[] msgs = {"msg1","msg2","msg3"};
for (String msg : msgs) {
if(!sub.isUnsubscribed()){
sub.onNext(msg);
}
}
if (!sub.isUnsubscribed()) { // completes
sub.onCompleted();
}
});
Now multiple subscribers can choose to receive messages.
observable.subscribe(System.out::println);
observable.subscribe(System.out::println);
Observables are a bit functional, they can choose what they need.
observable.filter(msg-> msg.equals("msg2")).map(String::length)
.subscribe(msgLength->{
System.out.println(msgLength); // or do something useful
});
Akka provides broadcast routers
This is not exactly a trivial problem; but not too hard to solve either.
Assuming your producer is an imperative program; it generates data chunk by chunk, adding each chunk to the cache; the process terminates either successfully or with an error.
The cache should have this interface for the produce to push data in it
public class Cache
public void add(byte[] bytes)
public void finish(boolean error)
Each consumer obtains a new view from the cache; the view is a blocking data source
public class Cache
public View newView()
public class View
// return null for EOF
public byte[] read() throws Exception
Here's a straightforward implementation
public class Cache
{
final Object lock = new Object();
int state = INIT;
static final int INIT=0, DONE=1, ERROR=2;
ArrayList<byte[]> list = new ArrayList<>();
public void add(byte[] bytes)
{
synchronized (lock)
{
list.add(bytes);
lock.notifyAll();
}
}
public void finish(boolean error)
{
synchronized (lock)
{
state = error? ERROR : DONE;
lock.notifyAll();
}
}
public View newView()
{
return new View();
}
public class View
{
int index;
// return null for EOF
public byte[] read() throws Exception
{
synchronized (lock)
{
while(state==INIT && index==list.size())
lock.wait();
if(state==ERROR)
throw new Exception();
if(index<list.size())
return list.get(index++);
assert state==DONE && index==list.size();
return null;
}
}
}
}
It can be optimized a little; most importantly, after state=DONE, consumers should not need synchronized; a simple volatile read is enough, which can be achieved by a volatile state
I have below code which tells me whether my data is PARTIAL or FULL. It works fine most of the time.
public static String getTypeOfData {
DataType type = new SelectTypes().getType();
if (type == DataType.partial || type == DataType.temp) {
return "partial";
}
return "full";
}
But sometimes, this line DataType type = new SelectTypes().getType(); just hangs and it keeps on waiting forever. This code is not in my control as it is developed by some other teams.
What I want to do is if this line DataType type = new SelectTypes().getType(); takes more than 10 second (or any default number of second), my method should return back a default string which can be partial.
Is this possible to do by any chance? Any example will help me to understand better.
I am using Java 7.
The ExecutorService provides methods which allow you to schedule tasks and invoke them with timeout options. This should do what you are after, however, please pay attention since terminating threads could leave your application in an inconsistent state.
If possible, you should contact the owners of the API and ask for clarification or more information.
EDIT: As per your comment, would caching be a possibility? Meaning that on start up, or some other point, you application goes through the SelectTypes and gets their type and stores them. Assuming that these do not change often, you can save them/update them periodically.
EDIT 2: As per your other comment, I cannot really add much more detail. You would need to add a method call which would allow your application to set these up the moment it is launched (this will depend on what framework you are using, if any).
A possible way would be to make the class containing the getTypeOfData() method as a Singleton. You would then amend the class to pull this information as part of its creation mechanism. Lastly, you would then create a Map<String, Type> in which you would throw in all your types. You could use getClass().getName() to populate the key for your map, and what you are doing now for the value part.
If you are not well aware of executor service then the easiest way to achieve this is by using Thread wait and notify mechanism:
private final static Object lock = new Object();
private static DataType type = null;
public static String getTypeOfData {
new Thread(new Runnable() {
#Override
public void run() {
fetchData();
}
}).start();
synchronized (lock) {
try {
lock.wait(10000);//ensures that thread doesn't wait for more than 10 sec
if (type == DataType.partial || type == DataType.temp) {
return "partial";
}else{
return "full";
}
} catch (InterruptedException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
}
return "full";
}
private static void fetchData() {
synchronized (lock) {
type = new SelectTypes().getType();
lock.notify();
}
}
You might have to do some little changes to make it work and looks better like instead of creating new thread directly you can use a Job to do that and some other changes based on your requirement. But the main idea remains same that Thread would only wait for max 10 sec to get the response.