TL; DR - is it guaranteed that by default, only one thread is ever used in any given time when observing events emitted by Observable?
It seems to me that RxJava2 is generally sequential unless expressed otherwise via things like parallel(). Even with observeOn/subscribeOn, I see that there are e.g. never two threads running simultaneously for doOnNext():
AtomicInteger counter = new AtomicInteger();
PublishSubject<Integer> testSubject = PublishSubject.create();
testSubject
.observeOn(Schedulers.io())
.doOnNext(val -> {
if(counter.incrementAndGet() > 1)
System.out.println("Whoa!!!!"); // <- never happens
Thread.sleep(20);
counter.decrementAndGet();
})
.subscribe();
for (int i = 0; i < 10000; i++) {
Thread.sleep(10);
testSubject.onNext(i);
}
No matter how I change this example - unless I go hardcore with .toFlowable(...).parallel().runOn(...), I don't see doOnNext running on different threads simultaneously.
I'd like to rely on this feature so I can ignore synchronisation issues in my operators, however I never saw it explicitly specified in the documentation for either RxJava2, RxJava1 or even just RX in general. Maybe I just missed it, could anyone pls point me to where this part of the contract is described?
Thanks!
From http://reactivex.io/documentation/contract.html
Observables must issue notifications to observers serially (not in
parallel). They may issue these notifications from different threads,
but there must be a formal happens-before relationship between the
notifications.
In your example you are not breaks this observable contract. But if you implement Observable wrong, two threads will run simultaneously:
AtomicInteger counter = new AtomicInteger();
Observable.create(emitter -> {
new Thread(() -> {
for (int i = 0; i < 10000; i++) {
try {
Thread.sleep(1);
emitter.onNext(i);
} catch (InterruptedException e1) {
e1.printStackTrace();
}
}
}).start();
for (int i = 0; i < 10000; i++) {
Thread.sleep(1);
emitter.onNext(i);
}
}).doOnNext(integer -> {
if (counter.incrementAndGet() > 1)
System.out.println("Whoaa!");
counter.decrementAndGet();
Thread.sleep(1);
}).subscribe();
Seems you can fix this behavior with observeOn https://github.com/ReactiveX/RxJava/issues/5550#issuecomment-325114185
remove Thread.sleep(10); to bombard the subject
Sleep for longer in the worker to simulate longer running task.
Add Thread.sleep(10000) under the loop, so that your main thread waits for the background threads to do their work
Change your thread pool .subscribeOn(Schedulers.newThread()) if you want to force each worker to run on new thread.
Related
Is it possible in standard java8 to execute multiple callables on single thread concurrently?
i.e. when one callable sleeps, start working on other callable.
My current experiment, which does not work:
ExecutorService executor = Executors.newSingleThreadExecutor();
List<Future> fs = new ArrayList<>();
for (int i = 0; i < 2; i++) {
final int nr = i;
fs.add(executor.submit(() -> {
System.out.println("callable-" + nr + "-start");
try { Thread.sleep(10_000); } catch (InterruptedException e) { }
System.out.println("callable-" + nr + "-end");
return nr;
}));
}
try { executor.awaitTermination(5, TimeUnit.SECONDS); } catch (InterruptedException e) { }
Results in:
callable-0-start
callable-0-end
callable-1-start
callable-1-end
I want to have:
callable-0-start
callable-1-start
callable-0-end
callable-1-end
Notes:
I kind of expect an answer: "No it's not possible. This is not how threads work. Once thread is assigned to some executable code it runs until completion, exception or cancellation. There can be no midflight-switching between callables/runnables. Thread.sleep only allows other threads to run on CPU/core." (explicit confirmation would put my mind to rest)
Naturally, this is "toy" example.
This is about understanding, not some specific problem that I have.
What you attempt to do is to emulate deprecated functionality from older java versions. Back then it was possible to stop, suspend or resume a Thread. But from the javadoc of Thread.stop:
This method is inherently unsafe. Stopping a thread with Thread.stop causes it to unlock all of the monitors that it has locked (as a natural consequence of the unchecked ThreadDeath exception propagating up the stack). If any of the objects previously protected by these monitors were in an inconsistent state, the damaged objects become visible to other threads, potentially resulting in arbitrary behavior. Many uses of stop should be replaced by code that simply modifies some variable to indicate that the target thread should stop running. The target thread should check this variable regularly, and return from its run method in an orderly fashion if the variable indicates that it is to stop running. If the target thread waits for long periods (on a condition variable, for example), the interrupt method should be used to interrupt the wait.
As described by this outtake, the risks of doing what you want were critical, and therefore this behavior has been deprecated.
I would suggest, that instead of trying to force a running thread into some sort of halting position from the outside, you should maybe think about a ThreadPool API that allows you to package your code segments properly, so that their state can be unloaded from a thread, and later resumed. e.g. create Ticket, which would be an elementary job, which a thread would always complete before beginning another, a TicketChain that sequentially connects tickets and stores the state. Then make a handler that handles tickets one by one. In case a Ticket cannot be currently done (e.g. because not all data is present, or some lock cannot be acquired) the thread can skip it until a later point in time, when said conditions might be true.
Building on answer from #TreffnonX
One way to achieve desired stdout result is using CompletableFuture
(callable code must be explicitly split into separate functions):
ExecutorService executor = Executors.newSingleThreadExecutor();
CompletableFuture<Integer>[] fs = new CompletableFuture[2];
for(int i=0; i<2; i++) {
final Integer ii = i;
fs[i] = (CompletableFuture.completedFuture(ii)
.thenApply((Integer x) -> { System.out.println("callable-" + x + "-start");return x; })
.thenApplyAsync((Integer x) -> { try { Thread.sleep(1_000); } catch (InterruptedException e) {Thread.currentThread().interrupt();} return x; }, executor)
.thenApply((Integer x) -> { System.out.println("callable-" + x + "-end");return x; }));
}
CompletableFuture.allOf(fs).join();
try { executor.awaitTermination(5, TimeUnit.SECONDS); } catch (InterruptedException e) { }
Result:
callable-0-start
callable-1-start
callable-0-end
callable-1-end
I have method that is checking the CompletableFuture execution time. If such CompletableFuture is executing for more than 2 seconds i want to kill this task. But how can I doit if i don't have control overy thread where CompletableFuture methods are executed ?
final CompletableFuture<List<List<Student>>> responseFuture = new CompletableFuture<>();
responseFuture.supplyAsync(this::createAllRandomGroups)
.thenAccept(this::printGroups)
.exceptionally(throwable -> {
throwable.printStackTrace();
return null;
});
createAllRandomGroups()
private List<List<Student>> createAllRandomGroups() {
System.out.println("XD");
List<Student> allStudents = ClassGroupUtils.getActiveUsers();
Controller controller = Controller.getInstance();
List<List<Student>> groups = new ArrayList<>();
int groupSize = Integer.valueOf(controller.getGroupSizeComboBox().getSelectionModel().getSelectedItem());
int numberOfGroupsToGenerate = allStudents.size() / groupSize;
int studentWithoutGroup = allStudents.size() % groupSize;
if (studentWithoutGroup != 0) groups.add(this.getListOfStudentsWithoutGroup(allStudents, groupSize));
for(int i = 0; i < numberOfGroupsToGenerate; i++) {
boolean isGroupCreated = false;
while (!isGroupCreated){
Collections.shuffle(allStudents);
List<Student> newGroup = this.createNewRandomGroupOfStudents(allStudents, groupSize);
groups.add(newGroup);
if (!DataManager.isNewGroupDuplicated(newGroup.toString())) {
isGroupCreated = true;
allStudents.removeAll(newGroup);
}
}
}
DataManager.saveGroupsToCache(groups);
return groups;
}
printGroups()
private void printGroups(List<List<Student>> lists) {
System.out.println(lists);
}
This statement responseFuture.cancel(true); does not kill thread where responseFuture is doing the methods. So what is the most elegant way to terminate CompletableFuture thread ?
When you create a chain of CompletableFuture stages like b = a.thenApply(function), this handy method creates a setup of different components. Basically, these components refer to each other as a → function → b, so the completion of a will trigger the evaluation of function which will first pre-check whether b still is not completed, then evaluate your function and attempt to complete b with the result.
But b itself has no knowledge of function or the thread that will evaluate it. In fact, function is not special to b, anyone could call complete, completeExceptionally or cancel on it from any thread, the first one winning. Hence, the completable in the class name.
The only way to get hands on the threads evaluating the functions, is to be in control of them right from the start, e.g.
ExecutorService myWorkers = Executors.newFixedThreadPool(2);
CompletableFuture<FinalResultType> future
= CompletableFuture.supplyAsync(() -> generateInitialValue(), myWorkers)
.thenApplyAsync(v -> nextCalculation(v), myWorkers)
.thenApplyAsync(v -> lastCalculation(v), myWorkers);
future.whenComplete((x,y) -> myWorkers.shutdownNow());
Now, the completion of future, e.g. via cancellation, will ensure that no new evaluation will be triggered by this chain and further makes an attempt to interrupt ongoing evaluations, if any.
So you can implement a timeout, e.g.
try {
try {
FinalResultType result = future.get(2, TimeUnit.SECONDS);
System.out.println("got "+result);
}
catch(TimeoutException ex) {
if(future.cancel(true)) System.out.println("cancelled");
else System.out.println("got "+future.get());
}
}
catch(ExecutionException|InterruptedException ex) {
ex.printStackTrace();
}
Not that the rejection of tasks due to the shutdown of the thread pool may cause some of the intermediate future to never complete, but for this chain of stages, this is irrelevant. All that matters, is, that the final stage future is completed, which is guaranteed, as it is its completion which triggers the shutdown.
The only way to terminate a thread is via interruption, which is a cooperative mechanism. This means the the thread must implement interruption logic, by handling the InterruptedException.
But it is a really bad practice to interrupt threads that you don't own, which I think is your case.
I have method that is checking the CompletableFuture execution time. If such CompletableFuture is executing for more than 2 seconds i want to kill this task. But how can I doit if i don't have control overy thread where CompletableFuture methods are executed ?
final CompletableFuture<List<List<Student>>> responseFuture = new CompletableFuture<>();
responseFuture.supplyAsync(this::createAllRandomGroups)
.thenAccept(this::printGroups)
.exceptionally(throwable -> {
throwable.printStackTrace();
return null;
});
createAllRandomGroups()
private List<List<Student>> createAllRandomGroups() {
System.out.println("XD");
List<Student> allStudents = ClassGroupUtils.getActiveUsers();
Controller controller = Controller.getInstance();
List<List<Student>> groups = new ArrayList<>();
int groupSize = Integer.valueOf(controller.getGroupSizeComboBox().getSelectionModel().getSelectedItem());
int numberOfGroupsToGenerate = allStudents.size() / groupSize;
int studentWithoutGroup = allStudents.size() % groupSize;
if (studentWithoutGroup != 0) groups.add(this.getListOfStudentsWithoutGroup(allStudents, groupSize));
for(int i = 0; i < numberOfGroupsToGenerate; i++) {
boolean isGroupCreated = false;
while (!isGroupCreated){
Collections.shuffle(allStudents);
List<Student> newGroup = this.createNewRandomGroupOfStudents(allStudents, groupSize);
groups.add(newGroup);
if (!DataManager.isNewGroupDuplicated(newGroup.toString())) {
isGroupCreated = true;
allStudents.removeAll(newGroup);
}
}
}
DataManager.saveGroupsToCache(groups);
return groups;
}
printGroups()
private void printGroups(List<List<Student>> lists) {
System.out.println(lists);
}
This statement responseFuture.cancel(true); does not kill thread where responseFuture is doing the methods. So what is the most elegant way to terminate CompletableFuture thread ?
When you create a chain of CompletableFuture stages like b = a.thenApply(function), this handy method creates a setup of different components. Basically, these components refer to each other as a → function → b, so the completion of a will trigger the evaluation of function which will first pre-check whether b still is not completed, then evaluate your function and attempt to complete b with the result.
But b itself has no knowledge of function or the thread that will evaluate it. In fact, function is not special to b, anyone could call complete, completeExceptionally or cancel on it from any thread, the first one winning. Hence, the completable in the class name.
The only way to get hands on the threads evaluating the functions, is to be in control of them right from the start, e.g.
ExecutorService myWorkers = Executors.newFixedThreadPool(2);
CompletableFuture<FinalResultType> future
= CompletableFuture.supplyAsync(() -> generateInitialValue(), myWorkers)
.thenApplyAsync(v -> nextCalculation(v), myWorkers)
.thenApplyAsync(v -> lastCalculation(v), myWorkers);
future.whenComplete((x,y) -> myWorkers.shutdownNow());
Now, the completion of future, e.g. via cancellation, will ensure that no new evaluation will be triggered by this chain and further makes an attempt to interrupt ongoing evaluations, if any.
So you can implement a timeout, e.g.
try {
try {
FinalResultType result = future.get(2, TimeUnit.SECONDS);
System.out.println("got "+result);
}
catch(TimeoutException ex) {
if(future.cancel(true)) System.out.println("cancelled");
else System.out.println("got "+future.get());
}
}
catch(ExecutionException|InterruptedException ex) {
ex.printStackTrace();
}
Not that the rejection of tasks due to the shutdown of the thread pool may cause some of the intermediate future to never complete, but for this chain of stages, this is irrelevant. All that matters, is, that the final stage future is completed, which is guaranteed, as it is its completion which triggers the shutdown.
The only way to terminate a thread is via interruption, which is a cooperative mechanism. This means the the thread must implement interruption logic, by handling the InterruptedException.
But it is a really bad practice to interrupt threads that you don't own, which I think is your case.
Given a cold example:
Observable<Integer> cold = Observable.create(subscriber -> {
try {
for (int i = 0; i <= 42; i++) {
// avoid doing unnecessary work
if (!subscriber.isUnsubscribed()) {
break;
}
subscriber.onNext(i);
}
subscriber.onCompleted();
} catch (Throwable cause) {
subscriber.onError(cause);
}
});
it starts executing from scratch for every new subscriber :
// starts execution
cold.subscribe(...)
and can stop execution if subscriber unsubscribes early :
// stops execution
subscription.unsubscribe();
now if instead of the sample for loop we have some actual business logic going on (which does not need to replay for each subscriber but be realtime instead) then we are dealing with hot observable...
PublishSubject<Integer> hot = PublishSubject.create();
Thread thread = new Thread(() -> {
try {
for (int i = 0; i < 42; i++) {
// how to avoid unnecessary work when no one is subscribed?
hot.onNext(i);
}
hot.onCompleted();
} catch (Throwable cause) {
hot.onError(cause);
}
});
when we want it to start we might do
// stats work (although no one is subscribed)
thread.start();
hence the first question : how to start work only when first observer subscribes? (connectable observable maybe?)
and the important question : how to stop work when last subscriber unsubscribes? (I cant figure out how to access current subscriptions for that subject, and would like to find clean solution without shared global state if such solution exists)
One solution i can think of is to lift subject with custom operator which will manage subscribers...
see operator refCount - http://reactivex.io/documentation/operators/refcount.html. This Operator turns your Observable into ConnectableObservable, and connects it when first subscriber subscribes, and disconnects when there are no more subscriptions
I have a main process main. It creates 10 threads (say) and then what i want to do is the following:
while(required){
Thread t= new Thread(new ClassImplementingRunnable());
t.start();
counter++;
}
Now i have the list of these threads, and for each thread i want to do a set of process, same for all, hence i put that implementation in the run method of ClassImplementingRunnable.
Now after the threads have done their execution, i wan to wait for all of them to stop, and then evoke them again, but this time i want to do them serially not in parallel.
for this I join each thread, to wait for them to finish execution but after that i am not sure how to evoke them again and run that piece of code serially.
Can i do something like
for(each thread){
t.reevoke(); //how can i do that.
t.doThis(); // Also where does `dothis()` go, given that my ClassImplementingRunnable is an inner class.
}
Also, i want to use the same thread, i.e. i want the to continue from where they left off, but in a serial manner.
I am not sure how to go about the last piece of pseudo code.
Kindly help.
Working with with java.
You can't restart a thread.
What you could do is use the java.util.concurrent package to wait for the threads to finish and rerun you runnables in the main thread to run them sequentially - by putting your runnables in a list, you can access them during the sequential run.
ExecutorService executor = Executors.newFixedThreadPool(10);
List<Runnable> runnables = new ArrayList<Runnable> ();
for (int i = 0; i < 10; i++) {
Runnable r = new ClassImplementingRunnable();
runnables.add(r);
executor.submit(r);
}
executor.shutdown();
//wait until all tasks are finished
executor.awaitTermination(Integer.MAX_VALUE, TimeUnit.SECONDS);
//re run the tasks sequentially
for (ClassImplementingRunnable r : runnables) {
//the method below can access some variable in
//your ClassImplementingRunnable object, that was
//set during the first parallel run
r.doSomethingElse();
}
If you want serial execution, just use
for (int i = 0; i < 10; i++)
new ClassImplementingRunnable().run();
all the tasks will run in the same thread, one after the other. This is the cleanest way to achieve what you want.
Update
After your comment it is clear that you in fact don't want to run the same tasks again, but to print the results that were calculated by them. This would be even simpler:
add the ClassImplementingRunnable instances into a list of tasks;
run each task in its own thread;
join all the threads;
write a for loop that prints the results from each ClassImplementingRunnable instance.
You already have 2 and 3.
I guess you want something like
ExecutorCompletionService
Example copied from Java doc.
Usage Examples. Suppose you have a set of solvers for a certain problem, each returning a value of some type Result, and would like to run them concurrently, processing the results of each of them that return a non-null value, in some method use(Result r). You could write this as:
void solve(Executor e,
Collection<Callable<Result>> solvers)
throws InterruptedException, ExecutionException {
CompletionService<Result> ecs
= new ExecutorCompletionService<Result>(e);
for (Callable<Result> s : solvers)
ecs.submit(s);
int n = solvers.size();
for (int i = 0; i < n; ++i) {
Result r = ecs.take().get();
if (r != null)
use(r);
}
}
Although there are some great answers here, I'm not sure your initial questions have been answered.
Now after the threads have done their execution, i wan to wait for all of them to stop, and then evoke them again, but this time i want to do them serially not in parallel.
You are confusing the running thread from it's object. It is a very common pattern (although usually made better with the ExecutiveService classes) to do something like the following:
List<ClassExtendingThread> threads = new ArrayList<ClassExtendingThread>();
// create your list of objects
for (int i = 0; i < 10; i++) {
ClassExtendingThread thread = new ClassExtendingThread(...);
thread.start();
threads.add(thread);
}
for (ClassExtendingThread thread : threads) {
// now wait for each of them to finish in turn
thread.join();
// call some method on them to get their results
thread.doThis();
}
Notice that I changed your class to extending Thread. It is usually better to implement Runnable like you did but if you are going to be joining and calling back to the objects, extending Thread makes the code easier.
So you create your object instances, start them as threads, and then join() with them which both waits for them to finish and synchronizes their memory. Once you join() with the thread, you can call any of the methods on your objects that you'd like. That doesn't "re-evoke" the thread at all. It is just accessing the fields inside of your objects. If you try to do this while the thread is running then you need to worry about synchronization.