I'm learning Reactive programming with project-reactor.
I have the following test case:
#Test
public void createAFlux_just() {
Flux<String> fruitFlux = Flux.just("apple", "orange");
fruitFlux.subscribe(f -> {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(f);
});
System.out.println("hello main thread");
}
By executing the test it seems that the main thread is stuck for 5 seconds.
I would expect that the subscribed consumer should run asynchronously in its own thread, that is, the subscribe invoke should return immediately to the main thread and consequently the hello main thread should print instantly.
The main thread is stuck because the subscription happens on the main thread. If you want it to run asynchronously, you need to the subscription to happen on a thread other than main. You could do this as:
#Test
public void createAFlux_just() {
Flux<String> fruitFlux = Flux.just("apple", "orange");
fruitFlux.subscribeOn(Schedulers.parallel()).subscribe(f -> {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(f);
});
System.out.println("hello main thread");
}
Note: I have used the parallel thread pool. You could use whatever pool you like. Reactor's pipelines are executed on the calling thread by default (unlike CompletableFuture<T> which runs in the ForkJoin pool by default).
This behavior would be the case if you had an observable (Flux) that was asynchronous. You chose to use a Flux with two readily available values by using the just method. They were passed to the subscription object right away since they were immediately available.
from spring.io documentation
The Threading Model
Reactor operators generally are concurrent agnostic: they don’t impose a particular threading model and just run on the Thread on which their onNext method was invoked.
The Scheduler abstraction
In Reactor, a Scheduler is an abstraction that gives the user control about threading. A Scheduler can spawn Worker which are conceptually Threads, but are not necessarily backed by a Thread (we’ll see an example of that later). A Scheduler also includes the notion of a clock, whereas the Worker is purely about scheduling tasks.
so you should subscribe on different thread by subscribeOn method and the Thread.sleep(5000) will sleep thread of the scheduler. You can see more examples like this one in the documentation.
Flux.just("hello")
.doOnNext(v -> System.out.println("just " + Thread.currentThread().getName()))
.publishOn(Scheduler.boundedElastic())
.doOnNext(v -> System.out.println("publish " + Thread.currentThread().getName()))
.delayElements(Duration.ofMillis(500))
.subscribeOn(Schedulers.elastic())
.subscribe(v -> System.out.println(v + " delayed " + Thread.currentThread().getName()));
Related
I'm comparing the behaviour of CompletableFuture.supplyAsync() in the two cases in which I set a custom ExecutorService or I want my Supplier to be executed by the default executor (if not specified) which is ForkJoinPool.commonPool()
Let's see the difference:
public class MainApplication {
public static void main(final String[] args) throws ExecutionException, InterruptedException {
Supplier<String> action1 = () -> {
try {
Thread.sleep(3000);
}finally {
return "Done";
}
};
Function<String, String> action2 = (input) -> {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}finally {
return input + "!!";
}
};
final ExecutorService executorService = Executors.newFixedThreadPool(4);
CompletableFuture.supplyAsync(action1, executorService)
.thenApply (action2)
.thenAccept (res -> System.out.println(res));
System.out.println("This is the end of the execution");
}
}
In this case I'm passing executorService to my supplyAsync() and it prints:
This is the end of the execution
Done!!
So "Done" gets printed after the end of the main execution.
BUT if I use instead:
CompletableFuture.supplyAsync(action1)
so I don't pass my custom executorService and the CompletableFuture class uses under the hood the ForkJoinPool.commonPool() then "Done" is not printed at all:
This is the end of the execution
Process finished with exit code 0
Why?
In both cases when you do
CompletableFuture.supplyAsync(action1, executorService)
.thenApply (action2)
.thenAccept (res -> System.out.println(res));
you don't wait for task completition. But then you program is going to exit and there is differences how common fork join pool:
ForkJoinPool.commonPool()
and regular executor service:
final ExecutorService executorService = Executors.newFixedThreadPool(4);
..react on attempt to call System.exit(...) equivalent.
This is what doc says about fork join common pool, you should point attention to that:
However this pool and any ongoing processing are automatically
terminated upon program System.exit(int). Any program that relies on
asynchronous task processing to complete before program termination
should invoke commonPool().awaitQuiescence, before exit.
That is link to ExecutorService docs, you may point attention to:
The shutdown() method will allow previously submitted tasks to execute
before terminating
I think that may be a difference you asking about.
ForkJoinPool uses daemon threads that does not prevent JVM from exiting. On the other hand the threads in the ExecutorService created by Executors are non-daemon threads, hence it keeps JVM from exiting until you explicitly shutdown the thread pool.
Also notice that in your example you need to shutdown the pool at the end in order to terminate the JVM.
executorService.shutdown();
So, one solution would be to keep the main thread waiting for few seconds until your computation is completed like so,
Thread.sleep(4000);
I have some legacy, non-RX code that does some networking work by spawning a new Thread.
When the work finishes, it invokes one method on a callback.
I don't have control over the thread this code runs on. It's legacy, and it spawns a new Thread by itself.
This can be simplified in something like:
interface Callback {
void onSuccess();
}
static void executeRequest(String name, Callback callback) {
new Thread(() -> {
try {
System.out.println(" Starting... " + name);
Thread.sleep(2000);
System.out.println(" Finishing... " + name);
callback.onSuccess();
} catch (InterruptedException ignored) {}
}).start();
}
I want to convert this to an RX Completable. To do so I use Completable#create().
The implementation of the CompletableEmitter calls the executeRequest passing of implementation of the Callback that signals when the request has finished.
I also print a log trace when subscribed to help me debugging.
static Completable createRequestCompletable(String name) {
return Completable.create(e -> executeRequest(name, e::onComplete))
.doOnSubscribe(d -> System.out.println("Subscribed to " + name));
}
This works as expected. The Completable completes only after the "request" finishes and the callback is invoked.
Problem is that when subscribing to this these completables in a trampoline scheduler, it does not wait for the first request to finish before subscribing to the second request.
This code:
final Completable c1 = createRequestCompletable("1");
c1.subscribeOn(Schedulers.trampoline()).subscribe();
final Completable c2 = createRequestCompletable("2");
c2.subscribeOn(Schedulers.trampoline()).subscribe();
Outputs:
Subscribed to 1
Starting... 1
Subscribed to 2
Starting... 2
Finishing... 1
Finishing... 2
As you see, it subscribes to the second Completable before the first Completable has completed, even if I'm subscribing in trampoline.
I'd like to queue the completables, so that the second waits for the first to finish, outputting this:
Subscribed to 1
Starting... 1
Finishing... 1
Subscribed to 2
Starting... 2
Finishing... 2
I'm sure the problem is related to the work being done in a worker thread. If the implementation of the Completable does not spawn a new thread it works as expected.
But this is legacy code and what I'm trying to do is adapting it to RX without modifying.
NOTE: the requests are executed in different points of the program - I cannot use andThen or concat to implement the serialised execution.
I have managed to execute the Completables sequentially by explicitly blocking the subscription Thread with a Latch.
But I don't think this is the idiomatic way to do it in RX, and I still don't understand why I need to do this and the Thread is not blocked until the Completable completes.
static Completable createRequestCompletable(String name) {
final CountDownLatch latch = new CountDownLatch(1);
return Completable.create(e -> {
executeRequest(name, () -> {
e.onComplete();
latch.countDown();
});
latch.await();
})
.doOnSubscribe(disposable -> System.out.println("Subscribed to " + name));
}
I am reading this article
And here code to finish the consumer thread:
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
System.out.println("Starting exit...");
consumer.wakeup(); 1
try {
mainThread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
As I understand ShutdownHook invokes when all non-daemon threads have finished but before the process was destroyed by OS.
1. From my point of view mainThread.join() is useless. Main thread always will finished to the moment of ShutdownHook execution. Is it correct or I misunderstood something?
2. Actually I don't understand why do we need to await main thred? we need to await close method execution?
P.S.
Book provides following main method code:
try {
// looping until ctrl-c, the shutdown hook will cleanup on exit
while (true) {
ConsumerRecords<String, String> records =
movingAvg.consumer.poll(1000);
System.out.println(System.currentTimeMillis() + "-- waiting for data...");
for (ConsumerRecord<String, String> record :
records) {
System.out.printf("offset = %d, key = %s,
value = %s\n",
record.offset(), record.key(),
record.value());
}
for (TopicPartition tp: consumer.assignment())
System.out.println("Committing offset at position:" + consumer.position(tp));
movingAvg.consumer.commitSync();
}
} catch (WakeupException e) {
// ignore for shutdown 2
} finally {
consumer.close(); 3
System.out.println("Closed consumer and we are done");
}
}
You do consumer.wakeup() to interrupt current consumer's operation (that might be long-running (e.g. a poll) or even blocked (what could happen in case of beginningOffsets(...).
The mainThread.join() is put in there to ensure that main thread actually finishes and is not shut down in middle of processing after wakeup. Please remember that shutdownHook is responsible for handling interrupts as well, not only ordinary program shutdown.
So if you interrupt with e.g. ctrl-C:
1. shutdown hook gets called
2. main thread is still running, most often waiting for data in `poll`
3. shutdown hook `wakeup`-s the main thread
4. main thread enters the exception handler, breaks the loop
5. main thread closes the consumer with `.close()`
6. shutdown hook waits for 5. and finishes
Without waiting you might have not performed the consumer shutdown steps in steps 4 & 5.
I have wrote following code:
System.out.println("Main thread:" + Thread.currentThread().getId());
CompletableFuture<Void> future = CompletableFuture.runAsync(() -> {
try {
System.out.println("Before sleep thread:" + Thread.currentThread().getId(), + " isDaemon:" + Thread.currentThread().isDaemon());
Thread.sleep(100);
System.out.println("After sleep");
} catch (InterruptedException e) {
e.printStackTrace();
}
});
future.whenComplete((r, e) -> System.out.println("whenCompleted thread:" + Thread.currentThread().getId()));
and this one prints:
Main thread:1
Before sleep thread:11 isDaemon:true
and finishes.
How can I change this behaviour?
P.S. I don't see anything related in runAsync java doc
The javadoc for runAsync() says:
Returns a new CompletableFuture that is asynchronously completed by a task running in the ForkJoinPool.commonPool() after it runs the given action.
There is another version of runAsync() where you can pass an ExecutorService.
Thus: when the default commonPool() doesn't do what you want - then create your own ExecutorService instead.
Add this line:
ForkJoinPool.commonPool().awaitTermination(5, TimeUnit.SECONDS);
to the main method after running your future. I'll block until all tasks in the pool have been completed.
Blocks until all tasks have completed execution after a shutdown request, or the timeout occurs, or the current thread is interrupted, whichever happens first.
I have the following code:
public void run()
{
try
{
logger.info("Looking for new tasks to fetch... ");
// definitions ..
for(Task t: tasks)
{
logger.info(" Task " + t.getId() + " is being fetched ");
// processing ... fetching task info from db using some methods
}
Thread.sleep(FREQUENCY);
//t.start();
} catch (Exception e)
{
logger.info("FetcherThread interrupted: "+e.getMessage());
}
}
I'm trying to make the thread to sleep for a specific time "FREQUENCY" and then work again. when I execute this code in eclipse, the thread works only once and then nothing happens and process terminates. If I remove the comment from the statement: t.start(), I get "FetcherThread interrupted: null".
Can anyone tell me where I'm going wrong?
N.B.: I want the thread to be working all the time, but fetching on periods (say every 5 minutes)
You're missing any sort of loop in that code.
It seems that the thread is actually doing what you tell it to do: it runs all the tasks, then sleeps for a bit - then it has no more work to do, and so exits. There are several ways to address this, in ascending order of complexity and correctness:
The simple (and naive) way to address this is to wrap the try-catch block in an infinite loop (while(true) { ... }). This way after the thread finishes sleeping, it will loop back to the top and process all the tasks again.
However this isn't ideal, as it's basically impossible to stop the thread. A better approach is to declare a boolean field (e.g. boolean running = true;), and change the loop to while(running). This way, you have a way to make the thread terminate (e.g. expose a method that sets running to false.) See Sun's Why is Thread.stop() deprecated article for a longer explanation of this.
And taking a step further back, you may be trying to do this at too low a level. Sleeping and scheduling isn't really part of the job of your Runnable. The actual solution I would adopt is to strip out the sleeping, so that you have a Runnable implementation that processes all the tasks and then terminates. Then I would create a ScheduledExecutorService, and submit the "vanilla" runnable to the executor - this way it's the job of the executor to run the task periodically.
The last solution is ideal from an engineering perspective. You have a class that simply runs the job once and exits - this can be used in other contexts whenever you want to run the job, and composes very well. You have an executor service whose job is the scheduling of arbitrary tasks - again, you can pass different types of Runnable or Callable to this in future, and it will do the scheduling bit just as well. And possibly the best part of all, is that you don't have to write any of the scheduling stuff yourself, but can use a class in the standard library which specifically does this all for you (and hence is likely to have the majority of bugs already ironed out, unlike home-grown concurrency code).
Task scheduling has first-class support in Java, don't reinvent it. In fact, there are two implementations: Timer (old-school) and ScheduledExecutorService (new). Read up on them and design your app aroud them.
Try executing the task on a different thread.
You need some kind of loop to repeat your workflow. How shall the control flow get back to the fetching part?
You can put the code inside a loop.( May be while)
while(condition) // you can make it while(true) if you want it to run infinitely.
{
for(Task t: tasks)
{
logger.info(" Task " + t.getId() + " is being fetched ");
// processing ... fetching task info from db using some methods
}
Thread.sleep(FREQUENCY);
}
Whats happening in your case its running the Task loop then sleeping for some time and exiting the thread.
Put the thread in a loop as others have mentioned here.
I would like to add that calling Thread.start more than once is illegal and that is why you get an exception.
If you would like to spawn multiple thread create one Thread object per thread you want to start.
See http://docs.oracle.com/javase/6/docs/api/java/lang/Thread.html#start()
public void run()
{
while (keepRunning) {
try
{
logger.info("Looking for new tasks to fetch... ");
// definitions ..
for(Task t: tasks)
{
logger.info(" Task " + t.getId() + " is being fetched ");
// processing ... fetching task info from db using some methods
t.start();
}
Thread.sleep(FREQUENCY);
} catch (Exception e) {
keepRunning = false;
logger.info("FetcherThread interrupted: "+e.getMessage());
}
}
}
Add a member call keepRunning to your main thread and implement an accessor method for setting it to false (from wherever you need to stop the thread from executing the tasks)
You need to put the sleep in an infinite loop (or withing some condition specifying uptill when you want to sleep). As of now the sleep method is invoked at the end of the run method and behavior you observe is correct.
The following demo code will print "Sleep" on the console after sleeping for a second. Hope it helps.
import java.util.concurrent.TimeUnit;
public class Test implements Runnable {
/**
* #param args
*/
public static void main(String[] args) {
Test t = new Test();
Thread thread = new Thread(t);
thread.start();
}
public void run() {
try {
// logger.info("Looking for new tasks to fetch... ");
// definitions ..
// for(Task t: tasks)
// {
// logger.info(" Task " + t.getId() + " is being fetched ");
// // processing ... fetching task info from db using some methods
// }
while (true) { // your condition here
TimeUnit.SECONDS.sleep(1);
System.out.println("Sleep");
}
// t.start();
} catch (Exception e) {
// logger.info("FetcherThread interrupted: "+e.getMessage());
}
}
}
You could try ScheduledExecutorService (Javadoc).
And us it's scheduleAtFixedRate, which:
Creates and executes a periodic action that becomes enabled first after the given initial delay, and subsequently with the given period; that is executions will commence after initialDelay then initialDelay+period, then initialDelay + 2 * period, and so on.