I have the following listener method:
#Override
public void onMessage(Message message, Channel channel) {
try {
// do something bad :)
} catch (Exception e){
try {
long dt = null != message.getMessageProperties()
? message.getMessageProperties().getDeliveryTag()
: 0;
channel.basicReject(dt, true);
} catch(IOException io) {
logger.error("IO-COMMON", io);
}
}
}
The issue is basic reject doesn't work, I don't know why. How to reject it gracefully? I think that if I reject a message, it should be requeued and reside is sth like cache, before going to next worker. But in fact this message just seems to be lost.
You need to set the acknowledgemode to MANUAL if you are doing your own acks. I am not sure why it's not working for you; DEBUG/TRACE logging might help.
You should consider letting the container handle the acks - use acknowledgemode=AUTO; the container will normally requeue the message for any exception thrown or ack it if the listener returns normally.
You can set defaultRequeueRejected to false (it is true by default) and the message will be discarded (or routed to a DLX/DLQ).
You can also throw an AmqpRejectAndDontRequeueException to override the default mechanism of requeuing failed messages.
If the ack mode is NONE - there are no acks and RabbitMQ automatically acks the message as soon as it's sent.
Related
I'm trying to know when a message has been accepted (ack) or not (nack) using RabbitMQ and Spring Boot.
I want to send a message into a queue (via exchange) and check if the queue has been accepted the message. Actually I want to send to two different queues, but it is not important, I'm assuming if it works for one of them will work for the other too.
So I've tried something like this using CorrelationData:
public boolean sendMessage(...) {
CorrelationData cd = new CorrelationData();
this.rabbitTemplate.convertAndSend(exchange, routingKey, message, cd);
try {
return cd.getFuture().get(3, TimeUnit.SECONDS).isAck();
} catch (InterruptedException | ExecutionException | TimeoutException e ) {
e.printStackTrace();
return false;
}
}
The line cd.getFuture().get(3, TimeUnit.SECONDS).isAck() should get false is value has not been ack into the queue I think. But this is always true, even if routingKey doesn't exists.
So I'm assuming this piece of code is checking the message has been send into the exchange and exchange says "yes, I've recived the message, it has not been routed, but I've recived it".
So, I've looked for other ways into Rabbit/Spring documentation but I can't get the way.
And, explaining a little more, that I want is:
Into Spring Boot code I receive a message. This message has to been send to other queues/exchange, but can't be removed from the current queue (i.e. acked) until other two queues confirm the ack.
I have manual ack and as a little pseudo-code I have this:
#RabbitListener(queues = {queue})
public void receiveMessageFromDirect(Message message, Channel channel,
#Header(AmqpHeaders.DELIVERY_TAG) long tag){
boolean sendQueue1 = sendMessage(...);
boolean sendQueue2 = sendMessage(...);
if(sendQueue1 && sendQueue2){
//both messages has been readed; now I can ack this message
channel.basicAck(tag, false);
}else{
//nacked; I can't remove the message util both queue ack the message
channel.basicNack(tag,false,true);
}
I've tested this structure and, even if the queues don't exists, values sendQueue1 and sendQueue2 are always true.
The confirm is true; even for unroutable messages (I am not entirely sure why).
You need to enable returned messages (and check that it is null in the CorrelationData after the future completes - correlationData.getReturnedMessage()). If it's not null, the message wasn't routable to any queue.
You only get nacks if there is a bug in the broker, or if you are using a queue with x-max-length and overflow behavior reject-publish.
We had an network issue in the middle of sending a message and this was causing all the threads to be in Blocked state. We're using org.springframework.cloud:spring-cloud-stream:2.0.1.RELEASE and org.springframework:spring-messaging:5.0.8.RELEASE for sending message to RabbitMQ broker. Binding interface:
interface MessagingSource {
#Output("bindingTargetName")
fun messageChannelOutput(): MessageChannel
}
Usage:
val isSent = messageSource.messageChannelOutput().send(message)
MessageChannel#send(Message, long) method also has the timeout in milliseconds as the second parameter, but it further ignored in org.springframework.integration.channel.AbstractSubscribableChannel#doSend method:
#Override
protected boolean doSend(Message<?> message, long timeout) { // timeout is ignored in this method
try {
return getRequiredDispatcher().dispatch(message);
}
catch (MessageDispatchingException e) {
String description = e.getMessage() + " for channel '" + this.getFullChannelName() + "'.";
throw new MessageDeliveryException(message, description, e);
}
}
Can you explain why the timeout parameter is ignored and how i can configure it to avoid long blocking state?
Thanks!
The channel sendTimeout only applies if the channel itself can block, e.g. a QueueChannel with a bounded queue that is currently full; the caller will block until either space becomes available in the queue, or the timeout occurs.
In this case, the block is downstream of the channel so the sendTimeout is irrelevant (in any case, it's a DirectChannel which can't block anyway, the subscribed handler is called directly on the calling thread).
The actual blocking you are seeing is most likely in the socket.write() in the rabbitmq client, which does not have a timeout and is not interruptible; there is nothing that can be done by the calling thread to "time out" the write.
The only possible solution I am aware of is to force close the rabbit connection by calling resetConnection() on the connection factory.
When having declared a method like this using Spring AMQP:
#RabbitListener(..)
public void myMethod(#Header(AmqpHeaders.CHANNEL) Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) Long tag, ...)
and using manual acknowledge mode, how should one properly deal with the IOException that may be thrown when doing ACK:
try {
channel.basicAck(tag, false);
} catch (IOException e) {
// What to do here?
}
Should the exception be rethrown? Should the "basicAck" operation be retried? What's the proper way to handle it?
The standard way of doing this is using retry mechanism & to come out if none of them succeeds.
However, based on my experience, if channel throws an exception, it more or less means the channel is useless & you might have to redo the whole thing again. I normally log the error along with the required details so that I can track which message processing failed so that I can verify the same later to see if its processed or I need to do anything about it.
I am trying to implement exponential backoff for consumer failures. To that end I have three queues with DLX thus: RETRY -> MAIN -> FAILED.
Anything rejected from MAIN goes FAILED, and anything added to RETRY goes into MAIN after a per-message TTL. The consumer receives from MAIN.
I've implemented an ErrorHandler and set it on the SimpleRabbitListenerContainerFactory. This handler either computes a new TTL and sends the message to the RETRY queue, or throws AmqpRejectAndDontRequeueException if that's not possible or retries are exceeded in order to DLX it to FAILED. The problem is, I cannot work out how to get rid of the original message.
As far as I can see I have to ack it, but the Channel is not available in the error handler, and there are no other exceptions to throw that would trigger an ack.
If instead I remove the MAIN -> FAILED DLX and switch to manually adding messages to FAILED, then if that doesn't work I've lost the message.
#Override
public void handleError(Throwable t) {
log.warn("Execution of Rabbit message listener failed.", t);
try {
queueForExponentialRetry(((ListenerExecutionFailedException) t).getFailedMessage());
// what to do here?
} catch (RuntimeException ex) {
t.addSuppressed(ex);
log.error("Not requeueing after failure", t);
throw new AmqpRejectAndDontRequeueException(t);
}
// or here?
}
I think I immediately found the answer. Missed it before because I was throwing from the the wrong place.
#Override
public void handleError(Throwable t) {
log.warn("Execution of Rabbit message listener failed.", t);
try {
queueForExponentialRetry(((ListenerExecutionFailedException) t).getFailedMessage());
} catch (RuntimeException ex) {
t.addSuppressed(ex);
log.error("Not requeueing after failure", t);
throw new AmqpRejectAndDontRequeueException(t);
}
throw new ImmediateAcknowledgeAmqpException("Queued for retry");
}
ImmediateAcknowledgeAmqpException
Special exception for listener implementations that want to signal that the current batch of messages should be acknowledged immediately (i.e. as soon as possible) without rollback, and without consuming any more messages within the current transaction.
This should be safe as I'm not using batches or transactions, only publisher returns.
Side note: I should also be aware that exponential backoff isn't going to actually work properly:
While consumers never see expired messages, only when expired messages reach the head of a queue will they actually be discarded (or dead-lettered). When setting a per-queue TTL this is not a problem, since expired messages are always at the head of the queue. When setting per-message TTL however, expired messages can queue up behind non-expired ones until the latter are consumed or expired.
I am using ActiveMQ to send the message.
So when I sent a message, the message comes to receive message. On successful insertion, it is acknowledged.
But I have code after acknowledgement, which can throw NullPointerException.
So to produce that exception intentionally, I have thrown NullPointerException.
So when it does that:
Message is not dequeued and the same message comes again to the onMessage function.
My code is:
public void onMessage(Message message) {
String msg = null;
try
{
msg = receiveMessage(message);
// Other code to insert message in db
message.acknowledge();
if(true)
{
throw new NullPointerException("npe"));
}
** // Other code which might produce a null pointer exception **
}
catch(Exception ex)
{
}
}
Why is the message again coming to onMessage() function as I have acknowledge() it also.
Since I have already inserted the message in db.
Doesn't the message inside queue will be removed on acknowledge()?
How I can achieve this?
You use AUTO acknowledge mode with message listners, then by specification, a message is redelivered if the message listeners fails to return successfully (for instance if there is an exception thrown).
In your case, you are trying to manually acknowledge the message, but that is not possible using a session created with createSession(false, Session.AUTO_ACKNOWLEDGE).
Your code would have worked with Session.CLIENT_ACKNOWLEDGE.
Otherwise, you want to catch the exceptions inside the onMessage method, while using AUTO_ACKNOWLEDGE.
To get a more fine grained controll over your messages, please consider using transacted sessions and use session.commit(); to confirm a message has been read.
Have you checked that you are not using transacted sessions?. When using transacted sessions,the acknowledge mode is ignored, so:
Your message.acknowledge() would effectively be a no-op
Your uncaught exception would be triggering a "session rollback" when escaping your message listener, forcing redelivery of the message.
NOTE: Your published code has a catch (Exception ex) { }, so I don't know exactly how your exception escapes outside.
You can create a separate method for processing the message, by which I mean that in the onMessage() function write code for only insertion of that message into the database.
And create a separate function for the processing of that message.
So that if you get any error during processing, the message will not come to onMessage() again.
When you use a transacted JMS acknowledge mode, your message will be received by JMS-listener several times (in AMQ by default it is approximately eight) till be processed without exception or will be moved by JMS-container to DQL-queue. See Message Redelivery and DLQ Handling for details.
Managing transactions depends on the framework used by you. I prefer to use Spring Framework, so my Spring XML configuration is looks like:
<jms:listener-container container-type="default"
connection-factory="calendarConnectionFactory"
acknowledge="transacted"
destination-type="queue"
cache="consumer"
concurrency="1-5">
<jms:listener destination="${jms.calendar.destination}" ref="calendarListener"/>
</jms:listener-container>
And the Java code of my message listener is
#Override
#Transactional(propagation = Propagation.REQUIRED,
noRollbackFor =
{ClassCastException.class, IllegalArgumentException.class})
public void onMessage(Message message) {
....
}
So I can manage what exceptions will rollback the transaction or not.