I need to run subscribeOn action in the same thread as Observable. I found a solution but it seems it's not the best. Basically I use the same scheduler for initial observable and observable which doOnSubscribe returns:
Scheduler singleThreadScheduler = Schedulers.from(Executors.newSingleThreadExecutor());
Observable<Object> obs = Observable.create(subscriber -> {
System.out.println("onNext at: " + Thread.currentThread());
subscriber.onNext(new Object());
subscriber.onCompleted();
}).subscribeOn(singleThreadScheduler).observeOn(Schedulers.newThread());
obs = obs.doOnSubscribe(() -> System.out.println("doOnSubscribe at: " + Thread.currentThread())).subscribeOn(singleThreadScheduler);
obs.subscribe(o -> {
System.out.println("result at: " + Thread.currentThread());
});
Log:
doOnSubscribe at: Thread[pool-1-thread-1] //runs on the same thread
onNext at: Thread[pool-1-thread-1] //runs on the same thread
result at: Thread[RxNewThreadScheduler-1]
Is there any better or maybe more standard way to achieve it?
As akarnokd noted the solution which I mention in the question is only one.
Related
I am playing around with Project Loom for the first time and I have some code
try (var executor = Executors.newVirtualThreadExecutor()) {
IntStream.range(0, 16).forEach(i -> {
System.out.println("i = " + i + ", Thread ID = " + Thread.currentThread());
executor.submit(() -> {
System.out.println("Thread ID = " + Thread.currentThread());
});
});
}
with output like
Thread ID = VirtualThread[#37]/runnable#ForkJoinPool-1-worker-4
Thread ID = VirtualThread[#33]/runnable#ForkJoinPool-1-worker-5
i = 9, Thread ID = Thread[#1,main,5,main]
Thread ID = VirtualThread[#43]/runnable#ForkJoinPool-1-worker-9
Thread ID = VirtualThread[#46]/runnable#ForkJoinPool-1-worker-11
i = 10, Thread ID = Thread[#1,main,5,main]
i = 11, Thread ID = Thread[#1,main,5,main]
Is there a way I can tell what Carrier Thread each Virtual Thread is running on?
Does ForkJoinPool-1-worker-11 represent a particular Carrier (Platform) Thread, or does it mean something else?
Yes, this suffix is the name of the current carrier thread.
When I use the following code
public static void main(String[] args) throws InterruptedException {
Set<String> threadStrings = ConcurrentHashMap.newKeySet();
try(var executor = Executors.newVirtualThreadPerTaskExecutor()) {
executor.invokeAll(Collections.nCopies(16,
() -> threadStrings.add(Thread.currentThread().toString())));
}
System.out.println("\tSimple Run");
threadStrings.stream().sorted().forEachOrdered(System.out::println);
threadStrings.clear();
try(var executor = Executors.newVirtualThreadPerTaskExecutor()) {
executor.invokeAll(Collections.nCopies(16, () -> {
threadStrings.add(Thread.currentThread().toString());
Thread.sleep(100);
return threadStrings.add(Thread.currentThread().toString());
}));
}
System.out.println("\tWith wait");
threadStrings.stream().sorted().forEachOrdered(System.out::println);
}
It prints
Simple Run
VirtualThread[#15]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#17]/runnable#ForkJoinPool-1-worker-2
VirtualThread[#18]/runnable#ForkJoinPool-1-worker-3
VirtualThread[#19]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#20]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#21]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#22]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#23]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#24]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#25]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#26]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#27]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#28]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#29]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#30]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#31]/runnable#ForkJoinPool-1-worker-4
With wait
VirtualThread[#36]/runnable#ForkJoinPool-1-worker-2
VirtualThread[#37]/runnable#ForkJoinPool-1-worker-3
VirtualThread[#37]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#38]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#38]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#39]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#39]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#40]/runnable#ForkJoinPool-1-worker-5
VirtualThread[#40]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#41]/runnable#ForkJoinPool-1-worker-6
VirtualThread[#41]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#42]/runnable#ForkJoinPool-1-worker-7
VirtualThread[#42]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#43]/runnable#ForkJoinPool-1-worker-5
VirtualThread[#43]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#44]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#44]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#45]/runnable#ForkJoinPool-1-worker-5
VirtualThread[#45]/runnable#ForkJoinPool-1-worker-6
VirtualThread[#46]/runnable#ForkJoinPool-1-worker-5
VirtualThread[#46]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#47]/runnable#ForkJoinPool-1-worker-2
VirtualThread[#49]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#49]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#50]/runnable#ForkJoinPool-1-worker-2
VirtualThread[#50]/runnable#ForkJoinPool-1-worker-6
VirtualThread[#51]/runnable#ForkJoinPool-1-worker-3
VirtualThread[#51]/runnable#ForkJoinPool-1-worker-5
VirtualThread[#52]/runnable#ForkJoinPool-1-worker-2
VirtualThread[#52]/runnable#ForkJoinPool-1-worker-8
(results may vary)
demonstrating how the carrier thread might change when performing a sleep. But in the current snapshot (“build 18-loom+6-282”) it’s not possible to specify your own Executor anymore and there is no method for querying the virtual thread about the carrier thread it uses (other than the implicit hint via toString()). So, the management of the underlying host threads is mostly a black box in this version.
Keep in mind that this is an ongoing development. It’s not clear whether and how this will change.
I am playing with Replaying Reactor Sinks, I am trying to achieve a mix of a unicast and a replay processor. I would like it to emit to only one subscriber at the same (UnicastProcessor), but that it can also emit a default value on subscribe (ReplayProcessor). Here is something similar to the real case:
Flux<Boolean> monoC = Sinks.many().replay().latestOrDefault(true).asFlux().doOnNext(integer -> System.out.println(new Date() + " - " + Thread.currentThread().getName() + " emiting next"));
for(int i = 0; i < 5; i++) {
new Thread(() -> {
monoC.flatMap(unused ->
webClientBuilder.build()
.get()
.uri("https://www.google.com")
.retrieve()
.toEntityFlux(String.class)
.doOnSuccess(stringResponseEntity -> {
System.out.println(new Date() + " - " + Thread.currentThread().getName() + " finished processing");
})
).subscribe();
}).start();
}
That is printing:
emiting next
...
emiting next
finished processing
...
finished processing
Instead, I would like it to print:
emiting next
finished processing
...
emiting next
finished processing
Update, some more clarifications on the real case scenario:
The real case scenario is: I have a Spring WebFlux application that acts like a relay, it receives a request on a specific endpoint A, and it relays it to another microservice B. This microservice can then reply with a 429 if I go too fast, and in a header with how long I have to wait before retrying again. The retrying thing I have already achieved it with a .retry operator and a Mono.delay, but in the meantime, I can receive another request on my first endpoint A which will have to be blocked until the Mono.delay finishes.
I am trying to achieve this with a Replay Sink, so that after receiving a 429, I emit a "false" to the sink and after Mono.delay is over, it emits a true to the sink, so if in the mean time I receive any further request on A it can filter out all the falses and wait for a true to be emitted.
The problem i have on top of that is that, when I receive too many request to relay on A, microservice B starts responding slow, and getting overloaded. Therefore, i would like to limit the rate that the Sink is emitting. To be precise, i would like the publisher to emit a value, but don't emit any more until the subscriber hits onCompleted.
As soon as I understood your issue correctly, you want the requests to B being processed sequentially. In that case you should have a look at https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Flux.html#flatMap-java.util.function.Function-int-
public final <V> Flux<V> flatMap(Function<? super T, ? extends Publisher<? extends V>> mapper, int concurrency)
I think your case should look like
//sinks should be global variable for your controller, initialized in #PostConstruct
var sinks = Sinks
//unsafe is required for multithreading
.unsafe()
.many()
.replay()
.latest();
sinks.asFlux()
.doOnNext(it -> System.out.printf("%s is emitting %s\n", Thread.currentThread().getName(), it))
.flatMap(counter -> {
return webClientBuilder.build()
.get()
.uri("https://www.google.com")
.retrieve()
.toEntityFlux(String.class)
.doOnSuccess(stringResponseEntity -> {
System.out.println(counter + " " + new Date() + " - " + Thread.currentThread().getName() + " finished processing with " + stringResponseEntity.getStatusCode());
})
.then(Mono.just(counter));
//concurrency = 1 causes the flatMap being handled only once in parallel
}, 1)
.doOnError(Throwable::printStackTrace)
//this subscription also must be done in #PostConstruct
.subscribe(counter -> System.out.printf("%s completed in %s\n", counter, Thread.currentThread().getName()));
//and this is your endpoint method
for (int i = 0; i < 5; i++) {
int counter = i;
new Thread(() -> {
var result = sinks.tryEmitNext(counter);
if (result.isFailure()) {
//mb in that case you should retry
System.out.printf("%s emitted %s. with fail: %s\n", Thread.currentThread().getName(), counter, result);
} else {
System.out.printf("%s successfully emitted %s\n", Thread.currentThread().getName(), counter);
}
}).start();
}
I am new to RxJava and am trying to use RxJava 2 to handle a stream of action events. I have two types in the stream (list) - "Immediate" which have to be executed in a blocking manner and "Deferred" which have to be executed non-blocking - both groups have to execute their set of actions in parallel. After reading about the options, I am thinking the groupBy operator is in play here but I am stuck on how to handle the subscription to the groups so the handling is appropriate. This is the code snippet I have come up with (obviously wrong) but I cannot think of the approach. I am looking for a suggestion on how to consume the two groups of observables correctly (so that immediate group executes in a blocking manner and the deferred group as non-blocking). Thanks in advance.
List<Map<String,String>> actionMaps = getActionMaps();
if(actionMaps != null && !actionMaps.isEmpty()){
actionMaps.removeIf(Objects::isNull);
Observable.fromIterable(actionMaps)
.subscribeOn(Schedulers.computation())
.map(actionMap -> {
String configForAction = jsonBasedConfigProvider.getValueForKey(Constants.CLASSIFIER_ACTIONS, getType() + "." + actionMap.get("name"));
if(configForAction != null){
Utilities.convertJsonStringToMap(configForAction).forEach(actionMap::putIfAbsent);
}
return actionMap;
})
.groupBy(actionMap -> actionMap.get("executionType"))
.subscribe(group -> {
//Group all the immediates for special processing
group
.toList()
.flatMapObservable(Observable::fromIterable)
.takeWhile(actionMap -> "immediate".equals(actionMap.get("executionType")))
.subscribeOn(Schedulers.computation())
.blockingSubscribe(actionMap -> {
System.out.println("Key is: " + group.getKey() + ", thread is: " + Thread.currentThread().getName() + ",item is:" + actionMap );
//ActionFactory.getAction(actionMap.get("name")).execute(actionMap);
//Any error handling required here?
});
//Group all the deferreds
group
.toList()
.flatMapObservable(Observable::fromIterable)
.takeWhile(actionMap -> "deferred".equals(actionMap.get("executionType")))
.subscribeOn(Schedulers.computation())
.subscribe(actionMap -> {
System.out.println("Key is: " + group.getKey() + ", thread is: " + Thread.currentThread().getName() + ",item is:" + actionMap );
//ActionFactory.getAction(actionMap.get("name")).execute(actionMap);
});
});
}
I have a use case where I need to aggregate the finished thread responses from multiple Observable objects and return back to the client. My question is how to achieve it with using the rX Java. Here I have written a code snippet but the issue of this one is that this won't return anything after the timeout.
Observable<AggregateResponse> aggregateResponse = Observable.
zip(callServiceA(endpoint), callServiceB(endpoint), callServiceC(endpoint),
(Mashup resultA, Mashup resultB, Mashup resultC) -> {
AggregateResponse result = new AggregateResponse();
result.setResult(resultA.getName() + " " + resultB.getName() + " " + resultC.getName());
return result;
}).timeout(5, TimeUnit.SECONDS);
Subscriber
aggregateResponse.subscribe(new Subscriber<AggregateResponse>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable throwable) {
//Timeout execute this rather than aggregating the finished tasks
System.out.println(throwable.getMessage());
System.out.println(throwable.getClass());
}
#Override
public void onNext(AggregateResponse response) {
asyncResponse.resume(response);
}
});
You need to put the timeout operator on each Observable, zip will wait for all Observables to emit a value before emitting a result, so if only one of them take longer while others already emitted, you will cut down the stream with the timeout (with onError) before the zipped Observable will have a chance to emit.
What you should do, assuming you want to ignore timed out sources while keeping the rest, is to add timeout operator to each Observable and also add error handling like onErrorReturn to each one, the error return can return some kind of 'empty' result (you can't use null in RxJava2), and when you aggregate result ignore those empty results:
Observable<AggregateResponse> aggregateResponse = Observable.
zip(callServiceA(endpoint)
.timeout(5, TimeUnit.SECONDS)
.onErrorReturn(throwable -> new Mashup()),
callServiceB(endpoint)
.timeout(5, TimeUnit.SECONDS)
.onErrorReturn(throwable -> new Mashup()),
callServiceC(endpoint)
.timeout(5, TimeUnit.SECONDS)
.onErrorReturn(throwable -> new Mashup()),
(Mashup resultA, Mashup resultB, Mashup resultC) -> {
AggregateResponse result = new AggregateResponse();
result.setResult(resultA.getName() + " " + resultB.getName() + " " + resultC.getName());
return result;
});
In my Android Application I used Rxjava2,But some strange situation was appeared.
In my Disposable I print to log current thread name:
//1
Observable
.create((ObservableOnSubscribe<UserModel>) e -> {
//mock io
if (phoneNumber.equals("HolyHigh") && password.equals("111111")) {
e.onNext(new UserModel());
e.onComplete();
} else {
e.onError(new RuntimeException("Error."));
}
})
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.delay(1, TimeUnit.SECONDS)
.subscribe(
r -> {
view.onLoginSuccess(new UserModel());
//test
String name = Thread.currentThread().getName();
Log.e("Thread Name", " Success Current Thread Name: " + name);
}
, e -> {
e.printStackTrace();
view.onLoginFailed(e.getMessage());
//test
String name = Thread.currentThread().getName();
Log.e("Thread Name", " Error Current Thread Name: " + name);
}
);
then logged:
Thread Name: Error Current Thread Name: RxComputationThreadPool-3
It looks like observeOn and subscribeOn not working...
why not main thread?
However,I wrote some simple ...
//2
Single.timer(1, TimeUnit.SECONDS)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(r -> {
Log.e("Single Thread Name", "Single Thread Name: " + Thread.currentThread().getName());
CommonUtil.showToast(r + "~");
});
and this logged:
Single Thread Name: Single Thread Name: main
where is my mistake?...
delay() operator operates by default on the computation scheduler, so it is changed upstream events to get notify on computation thread.
You simply need to change it right before the subscribe, just move the observeOn .observeOn(AndroidSchedulers.mainThread()) after the delay() operator.
BTW, delay() also has overload that gets Scheduler param that lets you change the default Scheduler.