How do you wait for a single value to come on an observable with a timout?
I am looking for something like:
Observable<Acknowledgement> acknowledgementObservable;
port.send(new Message());
Optional<Acknowledgement> ack = acknowledgementObservable.getFirst(100, TimeUnit.MILLISECONDS);
First, convert Observable to CompletableFuture as described in Converting between Completablefuture and Observable:
Observable<Acknowledgement> acknowledgementObservable;
port.send(new Message());
CompletableFuture<T> future = new CompletableFuture<>();
acknowledgementObservable
.doOnError(future::completeExceptionally)
.single()
.forEach(future::complete);
Then, wait for the event using timeout:
Acknowledgement ack = future.get(100, TimeUnit.MILLISECONDS);
It throws TimeoutException if timeout occurs.
You can do so by adding .timeout() and .onErrorComplete() .blockingGet()
So in this example it would be:
Acknowledgement ack = acknowledgementObservable
.firstElement()
.timeout(3, TimeUnit.SECONDS)
.onErrorComplete()
.blockingGet();
If the timeout is hit, ack will be null.
Related
Below code returned a timeout in client (Elasticsearch Client) when number of records are higher.
CompletableFuture<BulkByScrollResponse> future = new CompletableFuture<>();
client.reindexAsync(request, RequestOptions.DEFAULT, new ActionListener<BulkByScrollResponse>() {
#Override
public void onResponse(BulkByScrollResponse bulkByScrollResponse) {
future.complete(bulkByScrollResponse);
}
#Override
public void onFailure(Exception e) {
future.completeExceptionally(e);
}
});
BulkByScrollResponse response = future.get(10, TimeUnit.MINUTES); // client timeout occured before this timeout
Below is the client config.
connectTimeout: 60000
socketTimeout: 600000
maxRetryTimeoutMillis: 600000
Is there a way to wait indefinitely until the re-indexing complete?
submit the reindex request as a task:
TaskSubmissionResponse task = esClient.submitReindexTask(reindex, RequestOptions.DEFAULT);
acquire the task id:
TaskId taskId = new TaskId(task.getTask());
then check the task status periodically:
GetTaskRequest taskQuery = new GetTaskRequest(taskId.getNodeId(), taskId.getId());
GetTaskResponse taskStatus;
do {
Thread.sleep(TimeUnit.MINUTES.toMillis(1));
taskStatus = esClient.tasks()
.get(taskQuery, RequestOptions.DEFAULT)
.orElseThrow(() -> new IllegalStateException("Reindex task not found. id=" + taskId));
} while (!taskStatus.isCompleted());
Elasticsearch java api doc about task handling just sucks.
Ref
I don't think its a better choice to wait indefinitely to complete the re-indexing process and give very high value for timeout as this is not a proper fix and will cause more harm than good.
Instead you should examine the response, add more debugging logging to find the root-cause and address them. Also please have a look at my tips to improve re-indexing speed, which should fix some of your underlying issues.
https://pulsar.apache.org/api/client/2.4.0/org/apache/pulsar/client/api/Consumer.html#seek-long-
When calling seek(long timestamp) method on the consumer, does timestamp have to equal the exact time a message was published?
For example, if i sent three messages at t=1, 5, 7 and if i call consumer.seek(3), will i get an error? or will my consumer get reset to t=3, so that if i call consumer.next(), i'll get my second message?
Thanks in advance,
The Consumer#seek(long timestamp) allows you to reset your subscription to a given timestamp. After seeking the consumer will start receiving messages with a publish time equal to or greater than the timestamp passed to the seek method.
The below example show how to reset a consumer to the previous hour:
try (
// Create PulsarClient
PulsarClient client = PulsarClient
.builder()
.serviceUrl("pulsar://localhost:6650")
.build();
// Create Consumer subscription
Consumer<String> consumer = client.newConsumer(Schema.STRING)
.topic("my-topic")
.subscriptionName("my-subscription")
.subscriptionMode(SubscriptionMode.Durable)
.subscriptionType(SubscriptionType.Key_Shared)
.subscriptionInitialPosition(SubscriptionInitialPosition.Latest)
.subscribe()
) {
// Seek consumer to previous hour
consumer.seek(Instant.now().minus( Duration.ofHours(1)).toEpochMilli());
while (true) {
final Message<String> msg = consumer.receive();
System.out.printf(
"Message received: key=%s, value=%s, topic=%s, id=%s%n",
msg.getKey(),
msg.getValue(),
msg.getTopicName(),
msg.getMessageId().toString());
consumer.acknowledge(msg);
}
}
Note that if you have multiple consumers that belong to the same subscriptio ( e.g., Key_Shared) then all consumers will be reset.
I have a problem with Resilience4j RateLimiter
public static void main(final String[] args) throws InterruptedException {
final ExternalService service = new ExternalService();
final ExecutorService executorService = Executors.newFixedThreadPool(30);
final RateLimiterConfig config = RateLimiterConfig.custom()
.limitRefreshPeriod(Duration.ofSeconds(10))
.limitForPeriod(3)
.timeoutDuration(Duration.ofSeconds(12))
.build();
final RateLimiter rateLimiter = RateLimiter.of("RateLimiter", config);
final Callable<Response<String>> callable = RateLimiter.decorateCallable(
rateLimiter, () -> service.get(200, "OK")
);
executorService.submit(callable); //fine in first period
executorService.submit(callable); //fine in first period
executorService.submit(callable); //fine in first period
executorService.submit(callable); //should wait 10 sec and fine in second period
executorService.submit(callable); //should wait 10 sec and fine in second period
executorService.submit(callable); //should wait 10 sec and fine in second period
executorService.submit(callable); //should exit with timeout after 12 seconds
executorService.submit(callable); //should exit with timeout after 12 seconds
executorService.submit(callable); //should exit with timeout after 12 seconds
Thread.sleep(Duration.ofSeconds(40).toMillis());
executorService.shutdown();
}
In ExternalService I have some basic logging with localTime of responses. I think that it should work as I explained in comments, but my response is:
> Task :Main.main()
[12:24:53.5] Return standard response
[12:24:53.5] Return standard response
[12:24:53.5] Return standard response
[12:25:03.5] Return standard response
[12:25:03.5] Return standard response
[12:25:03.5] Return standard response
[12:25:03.5] Return standard response
[12:25:03.5] Return standard response
BUILD SUCCESSFUL in 40s
So it seems that the first cycle is good, but after that, FIVE next threads are allowed by RateLimiter, and the last thread is never called.
Unfortunately it was a bug which was introduced in a PR #672 which is part of release v1.2.0. The PR added the possibility to requests multiple permits per call. The bug was fixed now.
I'm thinking how to use RXJava for the scenario described bellow.
A List<Object>,each object will be sent to k8s and checked the status till the respone return true,so my polling active is that：
private Observable<Boolean> startPolling(String content) {
log.info("start polling "+ content);
return Observable.interval(2, TimeUnit.SECONDS)
.take(3)
.observeOn(Schedulers.newThread())
.flatMap(aLong -> Observable.just(new CheckSvcStatus().check(content)))
.takeUntil(checkResult -> checkResult)
.timeout(3000L, TimeUnit.MILLISECONDS, Observable.just(false))
;
}
Function of sent action:
Observable<Compo> sentYamlAndGet() {
log.info("sent yaml");
sentYaml()
return Observable.just(content);
}
I try to use the foreach to get each object status which like this:
public void rxInstall() throws JsonProcessingException {
List<Boolean>observables = Lists.newArrayList();
Observable.from(list)
.subscribeOn(Schedulers.newThread())
.concatMap(s -> sendYamlAndGet())
.timeout(3000l, TimeUnit.MILLISECONDS)
.subscribe()
;
Observable.from(list).forEach(s -> {
observables.add(Observable.just(s)
.flatMap(this::startPolling)
.toBlocking()
.last()
)
;
System.out.println(new ObjectMapper().writeValueAsString(observables));
}
Objects of outputs list is :{"o1","o2","o3","o4","o5"}
the last status of objest which I want is : [false,true,false,false,true].
All above style is not much 'ReactX',check object status action do not affect to each other.
How to throw foreach? I trid toIterable(),toList() but failed.
Observable.from(list)
.concatMap(s -> sentYamlAndGet())
.concatMap(this::startPolling)
....
;
Wanted to know if it's good practice to do that and what would be the best way to do that?
Thanks in advance.
pps: currentlly I'm using rxjava1 <version>1.2.0</version> but can change to 2(´▽`)ﾉ
I am trying to use a thread pool to make blocking requests.
The problem is, each request is blocking the whole pool and items are process sequentially.
Not sure if this is even possible. Somebody please help
city-dispatcher {
type = Dispatcher
executor = "thread-pool-executor"
thread-pool-executor {
fixed-pool-size = 16
}
throughput = 100
}
And Java
Props props = Props.create(CityDataProcessorActor.class, psRespHolder).withDispatcher("akka.actor.city-dispatcher");
SmallestMailboxPool pool = new SmallestMailboxPool(10);
ActorRef cityRequestActorPool = actorSystem.actorOf(pool.props(props), "city-request-route");
for (String city : citiesArray) {
Future<Object> future = Patterns.ask(cityRequestActorPool, new CityCommand(city.trim()), timeout);
Object results = Await.result(future, duration);
log.info(results.toString());
}
As #Mon Calamari mentioned
Object results = Await.result(future, duration); is a blocking call. you can try future with callback
future onComplete{
case Success()=> println(result)
case Failure()=> println("some error")
}
Mon Calamari's comment is exactly correct. Here's an implementation. It will create a List of Futures as you create them. Then it blocks on the collected Futures sequentially to log each one. The awaits should become trivial as the iteration progresses, providing later Futures have completed in similar time.
....
Array<Future<Object>> futures = new ArrayList<>();
for (String city : citiesArray) {
Future<Object> future = Patterns.ask(cityRequestActorPool, new CityCommand(city.trim()), timeout);
futures.add(future);
}
for (<Future<Object>> f :futures){
Object results = Await.result(f, duration);
log.info(results.toString());
}