How to handler errors/exceptions while using Spring Kafka framework? - java

I am not able to find how to do custom error handling in for a spring kafka consumer.
My requirement is:
For any deserialization errors, just write the error and message to database.
For any errors in execution under #KafkaListener method, retry 3 times and then write the error and message to database.
From the spring docs, I found that,
For 1, I will have to use ErrorHandlingDeserializer and it will then call the #KafkaListener error handler.
For 2, framework provides SeekToCurrentErrorHandler which handles message retries.
I am not understanding where can I add the code to write the exception/message to database in addition to enable configured retries.

Add a recoverer to the SeekToCurrentErrorHandler
new SeekToCurrentErrorHandler((rec, ex) -> {
Throwable cause = ex.getCause();
if (cause instanceof DeserializationException) {
...
}
else {
...
}, new FixedBackOff(2000L, 2L));
By default, deserialization exceptions are not retried; most others are retried before calling the recoverer.

Related

Retrying with Spring Retry based on error message or status code?

I am using the spring-retry dependency and currently have a Java Spring Boot application class annotated with #EnableRetry, and a function annotated with the below
#Retryable(value = MongoException.class,
maxAttempts = 4, backoff = #Backoff(delay = 100))
This allows me to retry for all exceptions of type MongoException for the function. However, id like to retry based on a specific status code or even an error message that is thrown. Is this possible to achieve with Spring Retry?
In the #Retryable method, catch the exception; then:
RetrySynchronizationManager.getContext().setExhaustedOnly()`
and rethrow it (or another exception type).
will disable retries and go straight to the #Recover method, if supplied, otherwise it will be thrown to the caller.
Or, as indicated in #geobreze comment above, use an exceptionExpression https://docs.spring.io/spring-retry/docs/api/current/org/springframework/retry/annotation/Retryable.html#exceptionExpression--

How to retry if web services are down?

I tried to retry when the web service failed.
I was using spring retry, but with spring retry I can't specify the status code based retry
This is my code
#retryable(value=Exception.class,maxAttampts=2)
Public void retry() throws exception {
System.out.println("retry started")
throw new exception}
I invoked the retry() method in my webservice class.
Spring retry has the following structure.
#Retryable(
value = {IllegalArgumentException.class},
maxAttempts = 5,
backoff = #Backoff(1000)
)
It will work when there is a specific exception. It does not work based upon the custom value. In your case, if the status code like 200 or 201 is not retrieved from a service, then you can throw an exception or your custom exception for which you can perform this retry operation. It works only on exceptional cases I mean for exception.Any value can be transformed into an exception to perform retry operation.
you can refer the following link.
https://github.com/debjava/spring-retry/blob/master/src/main/java/com/ddlab/rnd/MyServiceImpl.java

spring-boot log runtime exceptions from new thread

I have a rest API on my spring-boot 1.3.5.RELEASE application, from which a process is started asynchronously using a ThreadPoolExecutor.
My problem is that the async process happens to throw runtime exceptions for scenarios that were not handled but the stacktrace does not go in my logs. I am using log4j.
This is a big issue, because it takes a lot of time to debug and see what the actual problem is.
I am aware of the #ControllerAdvice, but I think it's context ends when the new thread is started and spring no longer intercepts the Exceptions.
How can I configure the ThreadPoolExecutor to redirect the uncought exceptions to the spring context or how should I approach this issue?
I am looking for a general solution to catch these types of exceptions and add their stacktrace to the appender, as I don't know where a developer will start a new thread which will crash with an uncaught exception
Let's say that the rest looks like this:
#RequestMapping("/api/test")
public void doSomething() {
ThreadPoolExecutor threadPoolExecutor = new ThreadPoolExecutor(10, 10, 10 , TimeUnit.SECONDS, new ArrayBlockingQueue<>(10, true));
threadPoolExecutor.execute(() -> {
throw new RuntimeException("some exception");
});
}
Have you tried creating a #ControllerAdvice, injecting Throwable into it and call logging?

How to ask RabbitMQ to retry when business Exception occurs in Spring Asynchronous MessageListener use case

I have a Spring AMQP message listener running.
public class ConsumerService implements MessageListener {
#Autowired
RabbitTemplate rabbitTemplate;
#Override
public void onMessage(Message message) {
try {
testService.process(message); //This process method can throw Business Exception
} catch (BusinessException e) {
//Here we can just log the exception. How the retry attempt is made?
} catch (Exception e) {
//Here we can just log the exception. How the retry attempt is made?
}
}
}
As you can see, there could be exception coming out during process. I want to retry because of a particular error in Catch block. I cannot through exception in onMessage.
How to tell RabbitMQ to there is an exception and retry?
Since onMessage() doesn't allow to throw checked exceptions you can wrap the exception in a RuntimeException and re-throw it.
try {
testService.process(message);
} catch (BusinessException e) {
throw new RuntimeException(e);
}
Note however that this may result in the message to be re-delivered indefinitely. Here is how this works:
RabbitMQ supports rejecting a message and asking the broker to requeue it. This is shown here. But RabbitMQ doesn't natively have a mechanism for retry policy, e.g. setting max retries, delay, etc.
When using Spring AMQP, "requeue on reject" is the default option. Spring's SimpleMessageListenerContainer will by default do this when there is an unhandled exception. So in your case you just need to re-throw the caught exception. Note however that if you cannot process a message and you always throw the exception this will be re-delivered indefinitely and will result in an infinite loop.
You can override this behaviour per message by throwing a AmqpRejectAndDontRequeueException exception, in which case the message will not be requeued.
You can also switch off the "requeue on reject" behavior of SimpleMessageListenerContainer entirely by setting
container.setDefaultRequeueRejected(false)
When a message is rejected and not requeued it will either be lost or transferred to a DLQ, if one is set in RabbitMQ.
If you need a retry policy with max attempts, delay, etc the easiest is to setup a spring "stateless" RetryOperationsInterceptor which will do all retries within the thread (using Thread.sleep()) without rejecting the message on each retry (so without going back to RabbitMQ for each retry). When retries are exhausted, by default a warning will be logged and the message will be consumed. If you want to send to a DLQ you will need either a RepublishMessageRecoverer or a custom MessageRecoverer that rejects the message without requeuing (in that latter case you should also setup a RabbitMQ DLQ on the queue). Example with default message recoverer:
container.setAdviceChain(new Advice[] {
org.springframework.amqp.rabbit.config.RetryInterceptorBuilder
.stateless()
.maxAttempts(5)
.backOffOptions(1000, 2, 5000)
.build()
});
This obviously has the drawback that you will occupy the Thread for the entire duration of the retries. You also have the option to use a "stateful" RetryOperationsInterceptor, which will send the message back to RabbitMQ for each retry, but the delay will still be implemented with Thread.sleep() within the application, plus setting up a stateful interceptor is a bit more complicated.
Therefore, if you want retries with delays without occupying a Thread you will need a much more involved custom solution using TTL on RabbitMQ queues. If you don't want exponential backoff (so delay doesn't increase on each retry) it's a bit simpler. To implement such a solution you basically create another queue on rabbitMQ with arguments: "x-message-ttl": <delay time in milliseconds> and "x-dead-letter-exchange":"<name of the original queue>". Then on the main queue you set "x-dead-letter-exchange":"<name of the queue with the TTL>". So now when you reject and don't requeue a message RabbitMQ will redirect it to the second queue. When TTL expires it will be redirected to the original queue and thus redelivered to the application. So now you need a retry interceptor that rejects the message to RabbitMQ after each failure and also keeps track of the retry count. To avoid the need to keep state in the application (because if your application is clustered you need to replicate state) you can calculate the retry count from the x-death header that RabbitMQ sets. See more info about this header here. So at that point implementing a custom interceptor is easier than customising the Spring stateful interceptor with this behaviour.
Also check the section about retries in the Spring AMQP reference.

How to handle badly formatted messages on the consumer side when working with Spring Integration and RabbitMQ

I am currently working on a project involves consuming messages from RabbitMQ brocker. However, I am still new to Spring Integration, AMQP and RabbitMQ.
I have an issue with consuming malformed messages formats. When my consumer receives a malformed message it returns it back the queue then RabbitMQ sends it back which creates an endless cycle.
In Spring Integration documentation there are some configuration that can be implemented to that this kind of message are no returned back to the queue.
However I could not understand how to implement that.
What I want is to be able to configure some kind of bean that has a format like
class ExceptionHandler {
public void handle(Throwable e ) {
Logger.log("Some log ... we don't give a Sh** ... ") ;
}
}
I've checked section 3.9 Exception Handling
and 3.15.3 Message Listeners and the Asynchronous Case
but unfortunately I could not understand anything.
So, if you have an example code or a link to one send it I will be greateful.
Yes, that's is one of the correct solution - to throw AmqpRejectAndDontRequeueException, when you decide that the message should not be requeued.
There is also defaultRequeueRejected on the SimpleMessageListenerContainer, which is true by default.
You maybe should to take a look to the DLX/DLQ solution to not lose those malformed messages.
Please, share the StackTrace which bothers you.
There is such a code in the SimpleMessageListenerContainer:
catch (AmqpRejectAndDontRequeueException rejectEx) {
/*
* These will normally be wrapped by an LEFE if thrown by the
* listener, but we will also honor it if thrown by an
* error handler.
*/
}
After a lot of try-fails attempts I was able to handle the error. However I am struggling with harboring the exception log now. I don't understand why this is implemented this way. I was able to handle the log issue too.
It turns that there is another way to say that you don't want to return the message back it is with acknowledge-mode="NONE" attribute. Checkout 10.2 Inbound Channel Adapter section.This way you don't even need to throw that ugly exception.
< bean id="handler" class="MessagesErrorHandler"/>
< int-amqp:inbound-channel-adapter
error-handler="handler"
id="idActivityAdapter"
channel="channelName"
queue-names="activityQueue"
/>
import org.springframework.util.ErrorHandler;
import org.springframework.amqp.AmqpRejectAndDontRequeueException;
public class MessagesErrorHandler implements ErrorHandler {
#Override
public void handleError(Throwable throwable) {
System.out.println("YESSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ERROR IS HANDLED !!!!");
throw new AmqpRejectAndDontRequeueException(throwable);// this very important
//so that message don't go back to the queue.
}
}
The AmqpRejectAndDontRequeueException is a signal to the container to reject and not requeue the message; by default, it requeues any exception.
Alternatively, you can manually wire up a SimpleMessageListenerContainer bean; set defaultRequeueRejected to false and add it to the adapter using the container attribute. Then, all exceptions will cause messages to be rejected and not requeued.
Also, instead of an error-handler, you can use an error-channel and throw the AmqpRejectAndDontRequeueException from the error flow.

Categories

Resources