I am new to RxJava and was trying to execute an example of parallel execution for multiple Observables from link :
RxJava Fetching Observables In Parallel
Though the example provided in the above link is executing observables in parallel, but when I added a Thread.sleep(TIME_IN_MILLISECONDS) in the forEach method then the system started executing one Observable at a time. Please help me to understand that why Thread.sleep is stopping the parallel execution of Observables.
Below is the modified example which is causing synchronous execution of multiple observables :
import rx.Observable;
import rx.Subscriber;
import rx.schedulers.Schedulers;
public class ParallelExecution {
public static void main(String[] args) {
System.out.println("------------ mergingAsync");
mergingAsync();
}
private static void mergingAsync() {
Observable.merge(getDataAsync(1), getDataAsync(2)).toBlocking()
.forEach(x -> { try{Thread.sleep(4000);}catch(Exception ex){};
System.out.println(x + " " + Thread.currentThread().getId());});
}
// artificial representations of IO work
static Observable<Integer> getDataAsync(int i) {
return getDataSync(i).subscribeOn(Schedulers.io());
}
static Observable<Integer> getDataSync(int i) {
return Observable.create((Subscriber<? super Integer> s) -> {
// simulate latency
try {
Thread.sleep(1000);
} catch (Exception e) {
e.printStackTrace();
}
s.onNext(i);
s.onCompleted();
});
}
}
In the above example we are using the subscribeOn method of observable and providing a ThreadPool(Schedules.io) for execution, so subscription for each Observable will happen on separate thread.
There is a possibility that Thread.sleep is locking any shared object between threads but I am still not clear on it. Please help.
Actually, with your example parallel execution is do happening, you are just looking at it incorrectly, there is a difference between where the work is executed and where the notification are emitted.
if you will put the log with the thread id at Observable.create, you will notice each Observable is executed at different thread simultaneously. but the notification is happens serially. this behavior is as expected as part of Observable contract is that observables must issue notifications to observers serially (not in parallel).
Related
subscribe documentation states:
Keep in mind that since the sequence can be asynchronous, this will
immediately return control to the calling thread. This can give the
impression the consumer is not invoked when executing in a main thread
or a unit test for instance.
So why do I see every element printed out? It means that lambda is executed in the main thread
import reactor.core.publisher.Flux;
import java.util.concurrent.TimeUnit;
public class ReactorMain {
public static void main(String[] args) {
Flux.just(1,2,3,4,5,6,7,8,9,10)
.limitRequest(5)
.skip(3)
.subscribe(value -> {
try {
TimeUnit.SECONDS.sleep(2);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Value: " + value);
})
;
}
}
This is because the generation of values does not contain any asynchronous element. So when you subscribe to the Flux, it will use the current thread to do as many thing it can do, until an asynchronous element makes it wait and use the thread for something else.
If to produce your elements you are for example calling a web service in an asynchronous way, you will receive values in an asynchronous way.
You can test it by making the sequence delayed:
public static void main(String[] args) {
Flux.just(1,2,3,4,5,6,7,8,9,10)
.limitRequest(5)
.skip(3)
.delayElements(Duration.ofSeconds(2))
.subscribe(value -> {
System.out.println("Value: " + value);
})
;
}
Using Java 8.
I have a Logger class that calls an API whenever it needs to log something. I realized that if the API is somehow badly configured, or the API just does not answer, my log action takes an awefull lot of time.
Example of synchronous logging:
public void debug(String message) {
MDC.clear();
MDC.put(SOME_KEY, "SOME_VALUE");
super.debug(message);
MDC.clear();
}
I was able to pinpoint that the problem is here because if I just comment everything and stop logging or doing anything, everything runs as fast as it should:
public void debug(String message) {
// MDC.clear();
// MDC.put(SOME_KEY, "SOME_VALUE");
// super.debug(message);
// MDC.clear();
}
So I thought to make this an asynchronous call, since I don't care if it's logged synchronously :
public void debug(String message) {
CompletableFuture.runAsync(() -> {
MDC.clear();
MDC.put(SOME_KEY, "SOME_VALUE");
super.debug(message);
MDC.clear();
});
}
But this asynchronous call is just as bad, performance wise for my main application, as the synchronous call. What am I missing ?
Your problem is that you don't provide an executor. This can cause Java to provide you with less threads than you have currently waiting debug calls, which means that you still get some blocking. On my Intel Core i7-4790 with 4 cores and hyperthreading on Java 8, I seem to get 7 threads running at the same time (number logical CPUs - 1 for the main thread). You can fix this by supplying an unlimited number of threads using a cached thread pool:
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class Test
{
public static void main(String[] args) throws InterruptedException
{
Executor ex = Executors.newCachedThreadPool();
for(int i=0;i<100;i++)
{
CompletableFuture.runAsync(() -> {
try {
TimeUnit.SECONDS.sleep(1);
} catch (InterruptedException e) {
throw new IllegalStateException(e);
}
System.out.println("completed");
},ex);
}
TimeUnit.SECONDS.sleep(2);
}
}
See the example above, which prints "completed" 100 times. If you remove the ex parameter, it will print far less.
However the underlying cause of slow debug calls may still need to be fixed as this may fill your memory if it is a long running task.
See also: (https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html):
All async methods without an explicit Executor argument are
performed using the ForkJoinPool.commonPool() (unless it does not
support a parallelism level of at least two, in which case, a new
Thread is created to run each task). [...]
Vertx docs suggests to use executeBlocking() method when one needs to call blocking API. On the other hand Vertx also offers a notion of Future which basically do the same thing. But the executeBlocking() method isn't static. It is also not a simple wrapper around Future, and if you look at its implementation you'll see that it's pretty complex. What's the difference between these two?
Assume that I want to execute some long running task in an async way. Is there any difference between these two methods?
method 1:
doTheJob() {
Future<Void> future = Future.future();
executeLongRunningBlockingOperation();
future.complete();
return future;
}
doTheJob().setHandler(asyncResult -> {
// ... handle result
});
method 2:
vertx.executeBlocking(future -> {
executeLongRunningBlockingOperation();
future.complete();
}, res -> {
// ... handle result
});
Your first example is not a correct usage of Future. The call to executeLongRunningBlockingOperation() will block the main thread until that method has completed — i.e. nothing else can happen until the blocking operation finishes. In your second example the blocking call is spun off into a background thread and other things continue to happen while it executes.
To illustrate this with a more complete example, this code:
public void executeLongRunningBlockingOperation() {
Thread.sleep(5000);
}
public Future<Void> doTheJob() {
System.out.println("Doing the job...");
Future<Void> future = Future.future();
executeLongRunningBlockingOperation();
// this line will not be called until executeLongRunningBlockingOperation returns!
future.complete();
// nor will this method! This means that the method won't return until the long operation is done!
return future;
}
public static void main(String[] args) {
doTheJob().setHandler(asyncResult -> {
System.out.println("Finished the job");
});
System.out.println("Doing other stuff in the mean time...");
}
Will produce the following output:
Doing the job...
Finished the job
Doing other stuff in the mean time...
Whereas this code (using the executeBlocking):
...
public Future<Void> doTheJob() {
System.out.println("Doing the job...");
Future<Void> future = Future.future();
Vertx vertx = Vertx.vertx();
vertx.executeBlocking(call -> {
executeLongRunningBlockingOperation();
call.complete;
}, result -> {
// this will only be called once the blocking operation is done
future.complete();
});
// this method returns immediately since we are not blocking the main thread
return future;
}
...
Will produce:
Doing the job...
Doing other stuff in the mean time...
Finished the job
If you'd like to develop a better understanding of Vert.x I'd recommend the following hands-on tutorials:
https://vertx.io/docs/guide-for-java-devs/
http://escoffier.me/vertx-hol/
In Java, is there any way to call and handle asynchronous method inside a thread?
Consider an scenario in which one of the method inside thread body takes more time to execute it. Because of that, thread completion takes more time.
I have tried some examples by using concurrency package classes like FutureTask and Executors.
Is it possible to implement and handle all exceptions inside asynchronous method? and Is it possible to get success or error responses like AJAX success and error handlers in JavaScript?
How will we ensure that asynchronous method successfully executed or not (with or without parent thread context)?
Most natural way of communication between async method and parent thread is standard class CompletableFuture:
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
public class AsyncExample {
String input; // common data
// async method
public String toLower() {
return input.toLowerCase();
}
// method on main thread
public void run() {
input = "INPUT"; // set common data
try {
// start async method
CompletableFuture<String> future = CompletableFuture.supplyAsync(this::toLower);
// here we can work in parallel
String result = future.get(); // get the async result
System.out.println("input="+input+"; result="+result);
} catch (InterruptedException | ExecutionException e) {
}
}
public static void main(String[] args) {
new AsyncExample().run();
}
}
Note that creation and warming of an Executor, including the default executor used in the example, requires some time (50 ms on my computer), so you may want to create and warm one beforehand, e.g. by supplying an empty method:
CompletableFuture.supplyAsync(()->null).get();
In a loop i need to make some checks, performed actually in some another verticle. In each iteration of my loop i need to check the response code, returned from those verticle and make some decision accordingly. In some other words i need to stop the execution of my loop and somehow wait till asynch. call returns. But such execution stop violates the vert.x philosophy, which states that main thread execution should be never stopped. How can i do it in the scope of Vert.x? So far i don't know how to do this. Any suggestions/code samples/urls to smth. like a solution would b highly appreciated.
Thanks!
When working with Vert.x you need to think less in terms of loops, and more in terms of callbacks.
You should use eventBus to communicate between vertices.
Let's say that what you want is something similar to this pseudocode:
for (int i = 0; i < 4; i++) {
int result = getVerticleResult();
System.out.println(result);
}
So, just a very basic example
class LooperVerticle extends AbstractVerticle {
private int i = 4;
#Override
public void start() throws Exception {
doWork();
}
private void doWork() {
vertx.eventBus().send("channel", "", (o) -> {
if (o.succeeded()) {
System.out.println(o.result().body());
i--;
if (i > 0) {
doWork();
}
}
});
}
}
class WorkerVerticle extends AbstractVerticle {
#Override
public void start() throws Exception {
vertx.eventBus().consumer("channel", (o) -> {
// Generate some random number
int num = ThreadLocalRandom.current().nextInt(0, 9);
// Simulate slowness
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
o.reply(num);
});
}
}
To test:
public class EventBusExample {
public static void main(String[] args) {
Vertx vertx = Vertx.vertx();
vertx.deployVerticle(new LooperVerticle());
vertx.deployVerticle(new WorkerVerticle());
}
}
I think you need to use FutureTask and store them in a Collection and use FutureTask.get() to retrieve the result when needed which is a blocking call.
It sounds like a use case for reactive steam processing.
In general such problem could be solved using 2 parties:
a producer that executes tasks and returns asynchronous results
a handler that subscribes to results and performs another tasks
There is a way to configure producer to perform tasks only when there is a subscriber. And on other side subscriber can decide to unsubscribe from producer on some condition.
I'm not familiar with vertx capabilities for reactive streams. But I would start from RxJava integration
http://vertx.io/docs/vertx-rx/java/