I have a situation in which I wanted to implement an API retry mechanism.
Let say I have an API that calls third party API where normal response time comes under 2 seconds but sometimes we got an error saying "Service Not available", "Gateway Timeout" Etc.
So I went online to see if we have a library to handle these things and I found out https://jodah.net/failsafe/
Purpose Of using Library:-
If under 5 seconds, I don't get the result, I will cancel the execution of the current call and try one more time.
For that, In Library I can see we have timeout and retry policy.
First I am trying the timeout.
Timeout<Object> timeout = Timeout.of(Duration.ofMillis(1000)).withCancel(true)
.onFailure(e -> logger.error("Connection attempt timed out {} {} ",new DateTime(), e.getFailure()))
.onSuccess(e -> logger.info("Execution completed on time"));
try {
logger.info("TIme: {}", new DateTime());
result = Failsafe.with(timeout).get(() -> restTemplate.postForEntity(messageSendServiceUrl, request, String.class));
} catch (TimeoutExceededException | HttpClientErrorException e) {
logger.info("TIme: {}", new DateTime());
logger.error("Timeout exception", e);
} catch (Exception e) {
logger.error("Exception", e);
}
But while calculating the time I am getting 20 seconds delay between calling the API and receiving TimeoutExceededException, which should be 1 second as duration is Duration.ofMillis(1000). Below you can see a difference of 21 seconds.
TIme: 2020-06-11T10:00:17.964+05:30
Connection attempt timed out 2020-06-11T10:00:39.037+05:30 {}
Can you please let me know what I am doing wrong here.
Second is the retry policy
RetryPolicy<Object> retryPolicy = new RetryPolicy<>()
.handle(HttpClientErrorException.class, TimeoutExceededException.class, Exception.class)
.withDelay(Duration.ofSeconds(1))
.withMaxRetries(3);
I want once TimeoutExceededException exception occurs after let's say 3 seconds, with a delay of 1 second, again the request is fired with max 3 retries.
I am using it as
result = Failsafe.with(retryPolicy,timeout).get(() -> restTemplate.postForEntity(messageSendServiceUrl, request, String.class));
get is a blocking or synchronous operation and it uses the calling thread. There is virtually no way for Failsafe to stop early. Timeout is best used in conjunction with an asynchronous operation, usually indicated by *Async methods. Make sure you read https://jodah.net/failsafe/schedulers/ because the default has some implications and is usually a poor choice for IO-bound operations.
Related
This question already has answers here:
Spring RestTemplate timeout
(11 answers)
Closed 5 months ago.
Below is our call to a third party service:
try {
ResponseEntity<String> response = new RestTemplate().exchange(
requestUrl,
HttpMethod.POST,
new HttpEntity<String>(null, new HttpHeaders()),
String.class);
// Log the response received (and request sent)
log.info(String.format("3rd party response: %s for request: %s" + response, requestUrl));
} catch (CustomTimeoutException e) {
log.error(String.format("Call to 3rd party with %s failed with: %s", requestUrl, e));
}
If the requested service is not available, it takes 30 seconds to timeout. We have no control over their timeout time, so we'd like to throw, catch and handle a custom exception if we don't hear back in 3 seconds.
Not very experienced with threading so a concrete example of how this specific code would fit into the solution would be greatly appreciated.
We have no control over their timeout time
Do not care about the server timeouts. As an HTTP client, you can define your own timeouts, on two distinct levels:
a connection timeout : how long max to reach the target server
a read timeout : allowing you to give up after a certain amount of time (and potentially earlier than the server will abort) in case the HTTP response does not come
See Spring doc on how to configure these two timeouts.
I have a WebClient that I want to stop and provide a fallback value after a certain timeout.
webClient.post()
.uri(path)
.bodyValue(body)
.retrieve()
.bodyToMono(type)
.timeout(Duration.ofSeconds(15))
.onErrorResume(ex -> Mono.just(provideFallbackValue())); //not only timeout, but any failure
Anyways, I would like to continue waiting for the response in another background thread in case of a TimeoutException (at least for let's say for another 60s), and still process the response then async.
Is that possible?
What you can do is change the point of view of this code. Instead of wait for additional 60 seconds in another thread start immediately the new thread and work asynchronously.
If the answer from the second thread arrives in 15 seconds or less you can reply immediately. Otherwise, send the message related to the error, but you still have active the call until 60 seconds.
Here is a code that can do that:
private Future<YourClass> asyncCall() {
...
// Put here the async call with a timeout of 60 seconds
// This call can be sync because you are already in a secondary thread
}
// Main thread
try {
Future<YourClass> future = executors.submit(() -> asyncCall());
YourClass result = future.get(15, TimeUnit.SECOND);
// Result received in less than 15 seconds
} catch (TimeoutException e) {
// Handle timeout of 15 seconds
}
Interesting, I would think have 255 concurrent users, an async API would have better performance. Here are 2 of my endpoints in my Spring server:
#RequestMapping("/async")
public CompletableFuture<String> g(){
CompletableFuture<String> f = new CompletableFuture<>();
f.runAsync(() -> {
try {
Thread.sleep(500);
f.complete("Finished");
} catch (InterruptedException e) {
e.printStackTrace();
}
});
return f;
}
#RequestMapping("/sync")
public String h() throws InterruptedException {
Thread.sleep(500);
return "Finished";
}
In the /async it runs it on a different thread. I am using Siege for load testing as follows:
siege http://localhost:8080/sync --concurrent=255 --time=10S > /dev/null
For the async endpoint, I got a transaction number of 27 hits
For the sync endpoint, I got a transaction number of 1531 hits
So why is this? Why isnt the async endpoint able to handle more transactions?
Because the async endpoint is using a shared (the small ForkJoinPool.commonPool()) threadpool to execute the sleeps, whereas the sync endpoint uses the larger threadpool of the application server. Since the common pool is so small, you're running maybe 4-8 operations (well, if you call sleeping an operation) at a time, while others are waiting for their turn to even get in the pool. You can use a bigger pool with CompletableFuture.runAsync(Runnable, Executor) (you're also calling the method wrong, it's a static method that returns a CompletableFuture).
Async isn't a magical "make things faster" technique. Your example is flawed as all the requests take 500ms and you're only adding overhead in the async one.
I am trying to implement exponential backoff for consumer failures. To that end I have three queues with DLX thus: RETRY -> MAIN -> FAILED.
Anything rejected from MAIN goes FAILED, and anything added to RETRY goes into MAIN after a per-message TTL. The consumer receives from MAIN.
I've implemented an ErrorHandler and set it on the SimpleRabbitListenerContainerFactory. This handler either computes a new TTL and sends the message to the RETRY queue, or throws AmqpRejectAndDontRequeueException if that's not possible or retries are exceeded in order to DLX it to FAILED. The problem is, I cannot work out how to get rid of the original message.
As far as I can see I have to ack it, but the Channel is not available in the error handler, and there are no other exceptions to throw that would trigger an ack.
If instead I remove the MAIN -> FAILED DLX and switch to manually adding messages to FAILED, then if that doesn't work I've lost the message.
#Override
public void handleError(Throwable t) {
log.warn("Execution of Rabbit message listener failed.", t);
try {
queueForExponentialRetry(((ListenerExecutionFailedException) t).getFailedMessage());
// what to do here?
} catch (RuntimeException ex) {
t.addSuppressed(ex);
log.error("Not requeueing after failure", t);
throw new AmqpRejectAndDontRequeueException(t);
}
// or here?
}
I think I immediately found the answer. Missed it before because I was throwing from the the wrong place.
#Override
public void handleError(Throwable t) {
log.warn("Execution of Rabbit message listener failed.", t);
try {
queueForExponentialRetry(((ListenerExecutionFailedException) t).getFailedMessage());
} catch (RuntimeException ex) {
t.addSuppressed(ex);
log.error("Not requeueing after failure", t);
throw new AmqpRejectAndDontRequeueException(t);
}
throw new ImmediateAcknowledgeAmqpException("Queued for retry");
}
ImmediateAcknowledgeAmqpException
Special exception for listener implementations that want to signal that the current batch of messages should be acknowledged immediately (i.e. as soon as possible) without rollback, and without consuming any more messages within the current transaction.
This should be safe as I'm not using batches or transactions, only publisher returns.
Side note: I should also be aware that exponential backoff isn't going to actually work properly:
While consumers never see expired messages, only when expired messages reach the head of a queue will they actually be discarded (or dead-lettered). When setting a per-queue TTL this is not a problem, since expired messages are always at the head of the queue. When setting per-message TTL however, expired messages can queue up behind non-expired ones until the latter are consumed or expired.
I am trying to use a Third Party Internal Library which is processing a given request. Unfortunately it is synchronous in nature. Also I have no control on the code for the same. Basically it is a function call. This function seems to a bit erratic in behavior. Sometimes this function takes 10 ms to complete processing and sometimes it takes up to 300 secs to process the request.
Can you suggest me a way to write a wrapper around this function so that it would throw an interrupted exception if the function does not complete processing with x ms/secs. I can live with not having the results and continue processing, but cannot tolerate a 3 min delay.
PS: This function internally sends an update to another system using JMS and waits for that system to respond and sends apart from some other calculations.
Can you suggest me a way to write a wrapper around this function so that it would throw an interrupted exception if the function does not complete processing with x ms/secs.
This is not possible. InterruptException only gets thrown by specific methods. You can certainly call thread.stop() but this is deprecated and not recommended for a number of reasons.
A better alternative would be for your code to wait for the response for a certain amount of time and just abandon the call if doesn't work. For example, you could submit a Callable to a thread pool that actually makes the call to the "Third Party Internal Library". Then your main code would do a future.get(...) with a specific timeout.
// allows 5 JMS calls concurrently, change as necessary or used newCachedThreadPool()
ExecutorService threadPool = Executors.newFixedThreadPool(5);
...
// submit the call to be made in the background by thread-pool
Future<Response> future = threadPool.submit(new Callable<Response>() {
public Response call() {
// this damn call can take 3 to 3000ms to complete dammit
return thirdPartyInternalLibrary.makeJmsRequest();
}
});
// wait for some max amount of time
Response response = null;
try {
response = future.get(TimeUnit.MILLISECONDS, 100);
} catch (TimeoutException te) {
// log that it timed out and continue or throw an exception
}
The problem with this method is that you might spawn a whole bunch of threads waiting for the library to respond to the remote JMS query that you would not have a lot of control over.
No easy solution.
This will throw a TimeoutException if the lambda doesn't finish in the time allotted:
CompletableFuture.supplyAsync(() -> yourCall()).get(1, TimeUnit.SECONDS)
Being that this is 3rd party you cannot modify the code. As such you will need to do two things
Launch the execution in a new thread.
Wait for execution in current thread, with timeout.
One possible way would be to use a Semaphore.
final Semaphore semaphore = new Semaphore(0);
Thread t = new Thread(new Runnable() {
#Override
public void run() {
// do work
semaphore.release();
}
});
t.start();
try {
semaphore.tryAcquire(1, TimeUnit.SECONDS); // Whatever your timeout is
} catch (InterruptedException e) {
// handle cleanup
}
The above method is gross, I would suggest instead updateing your desing to use a dedicated worker queue or RxJava with a timeout if possible.