I've been attempting to determine why the following code inside the .flatMap() operator was shown to be running on the main thread:
public Observable<Void> getObservable() {
return jobServiceObservable
.flatMap(jobService -> {
if (Looper.myLooper() == Looper.getMainLooper()) {
Log.d("LooperTest", "On main thread.");
} else {
Log.d("LooperTest", "Not on main thread.");
}
return jobService.syncReservations(accountUtil.getCurrentAccount());
})
.subscribeOn(Schedulers.io()).observeOn(foregroundScheduler);
}
As you can see, .subscribeOn() is being called with Schedulers.io(), however the log statements show that the code inside the .flatMap() was run on the main thread:
LooperTest: On main thread.
As a sanity check, I added extra calls to .subscribeOn(Schedulers.io()) to various parts of this code:
public Observable<Void> getObservable() {
return jobServiceObservable.subscribeOn(Schedulers.io())
.flatMap(jobService -> {
if (Looper.myLooper() == Looper.getMainLooper()) {
Log.d("LooperTest", "On main thread.");
} else {
Log.d("LooperTest", "Not on main thread.");
}
return jobService.syncReservations(accountUtil.getCurrentAccount()).subscribeOn(Schedulers.io());
})
.subscribeOn(Schedulers.io()).observeOn(foregroundScheduler);
}
However, the log statements appear to show the same result. Next, without any code changes, I cleaned the build and restarted my emulator. Upon the next run, the following was printed:
LooperTest: Not on main thread.
This was odd, as no code changes had been made. Again, without code changes, I cleaned the build and restarted the emulator. On the next run, the following was printed:
LooperTest: On main thread.
Once again, I cleaned the build, and then closed and opened a new emulator of a different type. Upon running, the following was printed:
LooperTest: Not on main thread.
Why is this occurring? I suspect there is some odd caching mechanism at play.
Additionally, note that jobService.syncReservations() returns a BehaviorSubject. Through various searches, it appears that Subjects may or may not respect calls to .subscribeOn().
Finally, note that jobServiceObservable is injected into the file where the above code is defined. The jobServiceObservable is created via the following code:
public Observable<JobService> getObservable() {
return Observable.create(e -> {
if (jobServiceBound && jobService != null) {
e.onNext(jobService);
e.onComplete();
} else {
jobServiceConnection = new ServiceConnection() {
#Override
public void onServiceConnected(ComponentName name, IBinder service) {
JobService.JobServiceBinder binder = (JobService.JobServiceBinder) service;
jobService = binder.getService();
jobServiceBound = true;
e.onNext(jobService);
e.onComplete();
}
#Override
public void onServiceDisconnected(ComponentName name) {
reset();
}
};
try {
boolean success = context.bindService(new Intent(context, JobService.class), jobServiceConnection, Context.BIND_AUTO_CREATE);
if (!success) {
e.onError(new Throwable("The service failed to be bound."));
}
} catch (SecurityException exception) {
e.onError(exception);
}
}
});
}
An authoritative answer on why the above behavior is occurring is needed.
Because onServiceConnected is called on the main thread by the system way after you subscribed to the wrapping Observable on the io() scheduler. subscribeOn tells where further subscription, or in this case, the body of Observable.create() should execute. You should use observeOn before the flatMap so that the mapper function gets executed on the desired thread:
public Observable<Void> getObservable() {
return jobServiceObservable
.observeOn(Schedulers.io())
.flatMap(jobService -> {
if (Looper.myLooper() == Looper.getMainLooper()) {
Log.d("LooperTest", "On main thread.");
} else {
Log.d("LooperTest", "Not on main thread.");
}
return jobService.syncReservations(
accountUtil.getCurrentAccount()).subscribeOn(Schedulers.io());
})
.observeOn(foregroundScheduler);
}
(In contrast, with typical Retrofit network calls, subscribeOn works because the network library executes its blocking call on the given scheduler and stays there for the emission of the network response.)
Related
I want to check the status of sending records to Kafka. I need to make sure that all records were successfully stored in Kafka. I was thinking about using the callback mechanism, e.g. creating a callback class like
public class MyCallback implememts Callback {
private AtomicReference<Exception> exceptionRef;
public MyCallback(){
exceptionRef=new AtomicReference<>(null);
}
#Override
public void onCompletion(final RecordMetadata metadata,final Exception exception){
if (exception!=null){
exceptionRef.set(exception);
}
}
public void check()
throws Exception
{
Exception exception=exceptionRef.get();
if (exception!=null) throw exception;
}
}
and then have a main program like
try{
Producer<Object,Object> producer=new KafkaProducer<>(props);
MyCallback callback=new MyCallback();
for (ProducerRecord<Object,Object> rec:myRecords){
producer.send(rec,callback);
}
producer.flush();
callback.check();
}
catch(Exception e){
handle(e);
}
My question: Can I be sure that the callback has been called for all sent records when flush() returns?
I should add that the setting acks=all is used.
I found that the answer is "yes". According to the javadoc:
The post-condition of flush() is that any previously sent record will have completed (e.g. Future.isDone() == true).
The question is then "Has the callback been called before the future completes?"
To answer that, you need to go into the internals code, i.e. the package
org.apache.kafka.clients.producer.internals. The future itself is implemented in the class FutureRecordMetadata, and the method is:
#Override
public boolean isDone() {
if (nextRecordMetadata != null)
return nextRecordMetadata.isDone();
return this.result.completed();
}
So we learn that the futures are linked, and the actual completion is delegated to the result, which is a ProduceFuture.
If you look in the class ProducerBatch, you will see that a ProduceFuture is on the batch level, in other words, a future on the record level is done, when the future of its batch is completed. Now looking at what sets batch level future to completed, we find that this method in ProducerBatch does that:
private void completeFutureAndFireCallbacks(long baseOffset, long logAppendTime, RuntimeException exception) {
// Set the future before invoking the callbacks as we rely on its state for the `onCompletion` call
produceFuture.set(baseOffset, logAppendTime, exception);
// execute callbacks
for (Thunk thunk : thunks) {
try {
if (exception == null) {
RecordMetadata metadata = thunk.future.value();
if (thunk.callback != null)
thunk.callback.onCompletion(metadata, null);
} else {
if (thunk.callback != null)
thunk.callback.onCompletion(null, exception);
}
} catch (Exception e) {
log.error("Error executing user-provided callback on message for topic-partition '{}'", topicPartition, e);
}
}
produceFuture.done();
}
The thunks are the record level futures, so it is clear that the callbacks are called prior to setting the future to done.
I'm starting a new thread from my activity, this thread does a 10 second operation and then reports back to the UI with runOnUiThread()
During the 10 second operation, the UI becomes unresponsive and does not respond to any user interaction. In this case I am attempting to close the activity using a button in the toolbar. An ANR error is thrown but the button click is processed after the worker thread has finished.
Although, while the thread is working, the app is still able to display a spinning ProgressBar which wouldn't happen if the work was being done on the UI thread.
The Profiler shows that the UI thread is sleeping during this work, so to my understanding it should be responsive?. I've tried using AsyncTask instead but that also doesn't work. Anyway here is some code:
The new Thread is started when the window comes into focus:
Activity:
#Override
public void onWindowFocusChanged(boolean hasFocus) {
super.onWindowFocusChanged(hasFocus);
if(hasFocus && !recyclerSetup){
progressBar.setIndeterminate(true);
progressBar.setVisibility(View.VISIBLE);
WorkThread thread = new WorkThread();
thread.start();
}
}
Thread:
private class WorkThread extends Thread {
#Override
public void run() {
getViewModelAndWords();
runOnUiThread(() -> setupRecycler());
}
}
private void getViewModelAndWords() {
viewModel = ViewModelProviders.of(this).get(WordViewModel.class);
adapter = new WordDetailedAdapter(this, viewModel, this, this, !favGroup.equals(ANY_WORD_PARAM));
allWords = viewModel.getAllWords();
}
I'm not sure if the viewModel has anything to do with the issue or not, but it's the viewModel.getAllWords() method which performs a heavy 10 second Room db operation.
Here's a snapshot of the Profiler showing the sleeping UI thread and worker Thread (AsyncTask #6):
EDIT:
Okay, so I think the issue lies within the room DB operation / viewModel. Replacing the contents of getAllWords() with Thread.sleep(10000); free'd up the UI thread for user interaction, therefore it's the following code which is (for some reason) preventing user input:
EDIT 2:
As suggested, I now use onPostExecute() along with an interface to retrieve the words:
public static class GetAllWordsWithCallBackTask extends AsyncTask<Void, Void, List<Word>>{
WordViewModel.iGetWords listener;
WordDao wordDao;
public GetAllWordsWithCallBackTask(WordDao wordDao, WordViewModel.iGetWords listener) {
this.listener = listener;
this.wordDao = wordDao;
}
#Override
protected List<Word> doInBackground(Void... voids) {
return wordDao.getAllWords();
}
#Override
protected void onPostExecute(List<Word> words) {
listener.gotWords(words);
}
}
get() has been removed and I simply execute the task, passing in listener to handle the call back:
public void getAllWordsWithCallBack(WordViewModel.iGetWords listener) {
try {
new GetAllWordsWithCallBackTask(wordDao, listener).execute();
} catch (Exception e) {
Crashlytics.log("Getting all words exception: "+e.getMessage());
e.printStackTrace();
}
}
This works well and the words are returned to my activity successfully, but the UI is still unresponsive while the operation is being executed.
Edit 1
You call .get() on a AsyncTask. The calling thread waits for the AsyncTask to complete. You could implement interface callbacks to fix this problem.
Here is a solution for you're problem
Edit 2:
I took a closer look at your code, and again, there is no error in the code you posted here.
Using AsyncTask with callbacks is a possible solution. Your code runs in the background thread and the result is passed to the main thread without blocking it.
I think that your error lies in transferring the data from the callback to ViewModel or MainActivity.
The best solution to get around this is using LiveData.
I tried to rebuild your code as closely as possible. Maybe it will help you to find the mistake.
WordDb
#Database(entities = {Word.class}, version = 3)
#TypeConverters(DateConverter.class)
public abstract class WordDb extends RoomDatabase {
private static WordDb INSTANCE;
public abstract WordDao wordDao();
static synchronized WordDb getInstance(Context contextPassed){
if(INSTANCE == null){
INSTANCE = Room.databaseBuilder(contextPassed.getApplicationContext(),WordDb.class,"word_db")
.fallbackToDestructiveMigration()
.build();
}
return INSTANCE;
}
}
WordRepo
class WordRepo {
private WordDao wordDao;
WordRepo(Context applicationContext) {
WordDb wordDb = WordDb.getInstance(applicationContext);
wordDao = wordDb.wordDao();
}
void getAllWords(WordRepo.iGetWords listener) {
try {
Log.i("WordRepo", String.format("getAllWords() called from %s", Thread.currentThread().getName()));
new GetAllWordsWithCallBackTask(wordDao, listener).execute();
} catch (Exception e) {
e.printStackTrace();
}
}
public static class GetAllWordsWithCallBackTask extends AsyncTask<Void, Void, List<Word>> {
WordRepo.iGetWords listener;
WordDao wordDao;
GetAllWordsWithCallBackTask(WordDao wordDao, WordRepo.iGetWords listener) {
this.listener = listener;
this.wordDao = wordDao;
}
#Override
protected List<Word> doInBackground(Void... voids) {
Log.i("WordRepo", String.format("GetAllWordsWithCallBackTask.doInBackground() called from %s", Thread.currentThread().getName()));
return wordDao.getAll();
}
#Override
protected void onPostExecute(List<Word> words) {
Log.i("WordRepo", String.format("GetAllWordsWithCallBackTask.onPostExecute() called from %s", Thread.currentThread().getName()));
listener.gotWords(words);
}
}
public interface iGetWords {
void gotWords(List<Word> words);
}
}
MainViewModel
public class MainViewModel extends AndroidViewModel {
MutableLiveData<List<Word>> wordList = new MutableLiveData<>();
private static final String TAG = "MainViewModel";
public MainViewModel(#NonNull Application application) {
super(application);
}
void getAllWords() {
Log.i(TAG, String.format("getAllWords() called from %s", Thread.currentThread().getName()));
WordRepo repo = new WordRepo(getApplication());
repo.getAllWords(new WordRepo.iGetWords() {
#Override
public void gotWords(List<Word> words) {
wordList.setValue(words);
}
});
}
}
getViewModelAndWords() in MainActivity
private void getViewModelAndWords() {
Log.i(TAG, String.format("getViewModelAndWords() called from %s", Thread.currentThread().getName()));
viewModel = ViewModelProviders.of(this).get(MainViewModel.class);
viewModel.wordList.observe(this, new Observer<List<Word>>() {
#Override
public void onChanged(List<Word> words) {
//Do something with youre result
Log.i(TAG, String.format("viewModel.wordList livedata returned %d results", words != null ? words.size() : -1));
}
});
viewModel.getAllWords();
Log.i(TAG, "viewModel.getAllWords() done");
}
If you find out what is going wrong with youre code, please leave a comment
As #mayokun already mentioned i would recommend to use RxJava or migrating your project to Kotlin + Coroutines to keep your code nice an clean.
Here you can find more:
Medium - Coroutines on Android (part I): Getting the background
CodeLabs - Using Kotlin Coroutines in your Android App
Medium - RxAndroid Basics: Part 1
Medium - RxJava VS. Coroutines In Two Use Cases
I have successfully tested this code with about 300,000 records. Running this operation has blocked the Async Task on my emulator for about 30 sec. The main thread was accessible during this process.
I hope this works for you this time as well
return new GetAllWordAsyncTask(wordDao).execute().get();
By calling get(), you are forcing the current invoking thread to synchronously wait for the result to come back, which makes your background query block the main thread while it executes.
The solution is to use a callback and onPostExecute rather than blocking the main thread to obtain your query results.
I defined this method, in MyActivity class that allow me to download in memory some data from a Firebase storage.
public void submitDownload() {
Task<byte[]> downloadTask=FirebaseStorage.getInstance.
getReference(DATA_PATH_TO_DOWNLOAD).getBytes(MAX_BYTES);
isTaskActive=true;
//remove eventually a previous callback from the handler
timeoutHandler.removeCallbacks(timeoutCallback);
downloadTask.addOnSuccessListener(MyActivity.this, onSuccessListener);
downloadTask.addOnFailureListener(MyActivity.this, onFailureListener);
timeoutHandler.postDelayed(timeoutCallback, 5000);
}
This is, instead, the onCreate() method:
protected void onCreate() {
super.onCreate();
onSuccessListener=new OnSuccessListener<byte[]>() {
public void onSuccess(byte[] bytes) {
if(isTaskActive) {
isTaskActive=false;
Log.d("DOWNLOAD_TASK", "SUCCESS");
}
}
};
onFailureListener=new OnFailureListener() {
public void onFailure(Exception e) {
if(isTaskActive) {
isTaskActive=false;
Log.d("DOWNLOAD_TASK", "FAILURE");
}
}
};
timeoutHandler=new Handler();
timeoutCallback=new Runnable() {
public voi run() {
if(isTaskActive) {
isTaskActive=false;
Log.d("DOWNLOAD_TASK", "TIMEOUT");
submitDownload(); //retry download
}
}
};
submitDownload();
}
Obviously, onSuccessListener, onFailureListener, timeoutHandler, timeoutCallback and isTaskActive are instance variable.
As you can see in the run() method defined in timeoutCallback, in addition to a log message, is also called the sumbitDownload(). Pratically, if a timeout occurs and the task is still active, a new download is started.
Now, imagine this scenario.
When Activity is created, a download task is started. Suppose that downloadTask doesn't complete, and neither onSuccessListener nor onFailureListener are called, but timeout occurs. So, from the run() method of timeoutCallback a new download is started.
Now, what happens to the previous downloadTask? Is it canceled? Is it replaced by the current task? or does it continue to be active and potentially could trigger its attached listeners?
If the latter question is true, how to remove the listeners from a Task<T> object?
Does the getResult() method, however, complete (i.e finish) the task?
I am writing a Play2 application service method in Java that should do the following. Asynchronously call method A, and if that fails, asynchronously call method B.
To illustrate assume this interface for the backend called by the service:
public interface MyBackend {
CompletionStage<Object> tryWrite(Object foo);
CompletionStage<Object> tryCleanup(Object foo);
}
So in my service method, I want to return a Future that can complete with these:
Success of tryWrite completed
Fail of tryWrite and Success of tryCleanup completed and failing with exception of tryWrite()
(Note: Of course tryWrite() could do any cleanup itself, this is a simplified example to illustrate a problem)
The implementation of a service calling the backend like this seems difficult to me because the CompletionStage.exceptionally() method does not allow Composing.
Version 1:
public class MyServiceImpl {
public CompletionStage<Object> tryWriteWithCleanup(Object foo) {
CompletionStage<Object> writeFuture = myBackend.tryWrite(foo)
.exceptionally((throwable) -> {
CompletionStage<Object> cleanupFuture = myBackend.tryCleanup(foo);
throw new RuntimeException(throwable);
});
return writeFuture;
}
}
So version 1 calls tryCleanup(foo) in a non-blocking way, but the CompletionStage returned by tryWriteWithCleanup() will not wait for cleanupFuture to complete. How to change this code to return a future from the service that would also wait for completion of cleanupFuture?
Version 2:
public class MyServiceImpl {
public CompletionStage<Object> tryWriteWithCleanup(Object foo) {
final AtomicReference<Throwable> saveException = new AtomicReference<>();
CompletionStage<Object> writeFuture = myBackend
.tryWrite(foo)
.exceptionally(t -> {
saveException.set(t);
// continue with cleanup
return null;
})
.thenCompose((nil) -> {
// if no cleanup necessary, return
if (saveException.get() == null) {
return CompletableFuture.completedFuture(null);
}
return CompletionStage<Object> cleanupFuture = myBackend.tryCleanup(foo)
.exceptionally(cleanupError -> {
// log error
return null;
})
.thenRun(() -> {
throw saveException.get();
});
});
return writeFuture;
}
}
Version2 uses an external AtomicReference to store the failure, and makes the asynchronous second call in another thenCompose() block, if there was a failure.
All my other attempts to do so ended up so unwieldy that I don't want to paste them here.
Unfortunately CompletionStage/CompletableFuture does not provide exception handling API's with composition.
You can work around this though by relying on a handle() with a BiFunction that returns a CompletionStage. This will give you nested stages (CompletionStage<CompletionStage<Object>>) that you can the "unnest" using compose(identity()):
public CompletionStage<Object> tryWriteWithCleanup(Object foo) {
return myBackend.tryWrite(foo)
.handle((r, e) -> {
if (e != null) {
return myBackend.tryCleanup(foo)
.handle((r2, e2) -> {
// Make sure we always return the original exception
// but keep track of new exception if any,
// as if run in a finally block
if (e2 != null) {
e.addSuppressed(e2);
}
// wrapping in CompletionException behaves as if
// we threw the original exception
throw new CompletionException(e);
});
}
return CompletableFuture.completedFuture(r);
})
.thenCompose(Function.identity());
}
You may simply wait for the completion inside the handler:
public CompletionStage<Object> tryWriteWithCleanup(Object foo) {
return myBackend.tryWrite(foo).exceptionally(throwable -> {
myBackend.tryCleanup(foo).toCompletableFuture().join();
throw new CompletionException(throwable);
});
}
This will defer the completion of the result CompletionStage to the completion of the cleanup stage. Using CompletionException as wrapper will make the wrapping transparent to the caller.
However, it has some drawbacks. While the framework might utilize the thread while waiting or spawn a compensation thread, if it is a worker thread, the blocked thread might be the caller thread if the stage returned by tryWrite happens to be already completed when entering exceptionally. Unfortunately, there is no exceptionallyAsync method. You may use handleAsync instead, but it will complicate the code while still feeling like a kludge.
Further, exceptions thrown by the cleanup may shadow the original failure.
A cleaner solution may be a bit more involved:
public CompletionStage<Object> tryWriteWithCleanup(Object foo) {
CompletableFuture<Object> writeFuture = new CompletableFuture<>();
myBackend.tryWrite(foo).whenComplete((obj,throwable) -> {
if(throwable==null)
writeFuture.complete(obj);
else
myBackend.tryCleanup(foo).whenComplete((x,next) -> {
try {
if(next!=null) throwable.addSuppressed(next);
}
finally {
writeFuture.completeExceptionally(throwable);
}
});
});
return writeFuture;
}
This simply creates a CompletableFuture manually, allowing to control its completion, which will happen either directly by the action chained to tryWrite’s stage in the successful case, or by the action chained to the cleanup stage in the exceptional case. Note that the latter takes care about chaining a possible subsequent cleanup exception via addSuppressed.
Observable.create(new Observable.OnSubscribe<Integer>() {
#Override
public void call(Subscriber<? super Integer> subscriber) {
subscriber.onStart();
subscriber.onNext(1);
subscriber.onCompleted();
}
}).delaySubscription(5, TimeUnit.SECONDS).subscribe(new Subscriber<Integer>() {
#Override
public void onCompleted() {
Log.e("TAG", String.format("(%s) - onCompleted", System.currentTimeMillis()));
}
#Override
public void onError(Throwable e) {
Log.e("TAG", String.format("(%s) - onError", System.currentTimeMillis()), e);
}
#Override
public void onNext(Integer integer) {
Log.e("TAG", String.format("(%s) - onNext: %s", System.currentTimeMillis(), integer));
}
#Override
public void onStart() {
super.onStart();
Log.e("TAG", String.format("(%s) - onStart", System.currentTimeMillis()));
}
});
output:
(1485004553817) - onStart
(1485004558818) - onNext: 1
(1485004558819) - onCompleted
why onStart event not waiting to delaySubscription and calling soon ?
i want aware when call method called
Documentation says -
onStart -
This method is invoked when the Subscriber and Observable have been connected but the Observable has not yet begun to emit items or send notifications to the Subscriber.
delaySubscription:
Returns an Observable that delays the subscription to the source Observable by a given amount of time.
onNext is invoked only when the subscription is achieved. onStart is called the moment a connection is established. Thus, it works as expected according to the definition.
You can try commenting the code subscriber.onStart(); and execute the same again to notice that onStart is still called at the beginning. The intentional execution did not really invoke the said method because this was executed not on the real subscriber we created, but the one which was a result of delaySubscription (of type OnSubscribeDelaySubscription).
Below is a snippet which can probably help you achieve what you're looking for:
public static void main(String[] args) throws UnsupportedEncodingException, IOException {
Observable.timer(5, TimeUnit.SECONDS).flatMap(val -> {
System.out.println("Initialize");
return Observable.create(subscriber -> {
System.out.println("onsubscribe");
doMyAsyncStuff(subscriber);
});
}).subscribe(val -> System.out.println(val));
Observable.timer(10, TimeUnit.SECONDS).toBlocking().first();
}
We initialize a timer, once timer is executed, we perform some task in flatMap which should be the same as what you earlier did with onStart. Once that task is executed, we emit a Observable which emits all the elements that you could have consumed earlier with onNext calls.