I did read this reference: https://www.rabbitmq.com/dlx.html, however it doesn't resolve my doubts, namely:
In case of accepting message there is no problem - spring-rabbitmq send ack and everthing is fine, DLX doesn't know about acked message.
The problem is in case rejecting answer, namely what about throwing MessageConverterException ? This message is removed or moved to DLX?
And what about in case other exception ? For example Exception ? It is removed/requeued/moved to DLX ?
Edit after answer of #Gary
I think, that after answer's #Gary I should add more details about my case and some summary of #Gary's answer. #Gary exactly grasped my use case.
I wouldn't like requeue - never (I am afraid of looping), but I wouldn't like to lose messages when an exception was thrown (for example lost connection to database) - this message should be resend to DLX. On the other hand, conversion of message should be treated as fatal error - no requeue, no resend to DLX - simply permanent removing message. Generally, in depends on exception either reject (=resend to DLX if configured) or accept, never requeue.
To sum up in a nutshell approach proposed by #Gary.
First: We can override ExceptionHandler to manage of sending nack/ack, which gives to us a full control.
Second: IMO simpler, solution is to set defaultRequeueRejected=false and in converter throw ImmediateAcknowledgeAmqpException. It makes that RabbitMQ think that answer was accepted (the same thing as in case of first solution), moreover listener wouldn't be invoked.
**Conclusion**: UsingImmediateAcknowledgeAmqpExceptionorExceptionHandler` exception we have a full control on permanent rejecting message (under hood ack) and resending to DLX.
RabbitMQ knows nothing about the exceptions.
When the container catches an exception it calls channel.basicReject(deliveryTag, requeue).
If requeue is true, the message is requeued.
By default, for any exception other than those mentioned here
o.s.amqp...MessageConversionException
o.s.messaging...MessageConversionException
o.s.messaging...MethodArgumentNotValidException
o.s.messaging...MethodArgumentTypeMismatchException
java.lang.NoSuchMethodException
java.lang.ClassCastException
requeue is set to true, so the message is requeued.
For those exceptions, the delivery is considered fatal and the message is NOT requeued, it will go to a DLX/DLQ if one is configured.
The container has a flag defaultRequeueRejected which is true by default; if you set it to false; no exceptions will be requeued.
For application-level exceptions, generally, messages will be requeued. To dynamically reject (and not requeue) a message, make sure there is an AmqpRejectAndDontRequeueException in the cause chain. This instructs the container to not requeue the message, and it will go to the DLX/DLQ (if configured). This behavior is enabled by the defaultRequeueRejected flag mentioned above.
This is all explained in the documentation and, as I have discussed in other answers to you, you can change this behavior by using a custom error handler; that, too, is explained in the documentation.
It is not possible to send some exceptions to the DLX/DLQ and not others; rabbit only has a binary option, requeue or don't requeue and, for the latter, if a DLX/DLQ is configured all such rejected messages go to the DLX/DLQ.
Spring AMQP provides one more exception, ImmediateAcknowledgeAmqpException. If your listener throws this exception, the message will be ack'd as if it was processed successfully (channel.basicAck()). That is the only technique provided by the container, to discard a bad message without sending it to the DLX/DLQ.
Of course, your application can drop such messsages itself.
If you want to DLX/DLQ all business exceptions but drop conversion exceptions, throw AmqpRejectAndDontRequeueException (or set defaultRequeueRejected to false), and throw ImmediateAcknowledgeAmqpException from your converter.
Related
Hi I am using a jmslistener annotation to recieve messages from tibco queue. I am DefaultJmsListenerContainer factory with sessionTransacted = true. What I want to do is
When we get a RunTimeException I want to retry the specific message specific no of times(lets say x)
When we get cannot get jdbc connection I want to shutdown the system and want to make sure that this message is sent back to the queue to be redelivered the next time system is brought up.
What I am facing is
When I am setting sessionTransacted as true and I am throwing a RunTimeException the message is redelivered indefinitely . How can I set this configuration to redeliver the message only x times.( I have tried using message header property JMSXDeliveryCount but that does not give me the correct no of times a specific message is redelivered.)
I tried shutting down the system using System.exit(1) but this leads to deadlock and application hangs. I added another piece of code where I am shutting down the application in a different thread and making sure if in between the shutting down of the container another message is read by the listener I throw a RunTimeException so that I am able to get that message again once my system is brought up. However what I want is the 1st message for which we did not get the jdbc connection to be redelivered and no other messages to be read when I stop the container.How can we achieve this.
I have to implement an error handler that uses the Camel Redelivery Policy over a business process which send a SOAP request and process its response. During the process part, a special exception (RetryException) could be thrown. This exception is caught (thanks to onException(RetryException.class)) on the error handler level.
That was the easy part.
Now I want to ignore exceptions that could be thrown by the cxf endpoint (in case of connection error per example) and process them.
So I try :
(1) One main route that has the onException clause with retry strategy
(2) One sub route that aggregates 2 routes (and has noErrorHandler) to be sure to replay the 2 routes and not only the processing one that throw the exception :
(3) The route which send the SOAP request
(4) The route which process the response and can throw the retryException.
In order to ignore the exception thrown by the cxf endpoint I implement the route (3) like that :
public void configure() {
from(ROUTE_NAME).
.handleFault() // To handle Soap fault
.onException(Fault.class)
.continued(true)
.end()
.to("cxf:[...]")
;
}
It works like a charm, the response processor perform some checks before throwing the RetryException... But the continued instruction throws away all the informations about redelivery that I previously had and Camel believe that this exception is the first one. So the route enters into a kind of retry forever loop.
Exchange headers before onException(Fault) :
Headers: {breadcrumbId=ID-ITEM-S28636-63117-1469800853403-0-1, CamelRedelivered=true, CamelRedeliveryCounter=1, CamelRedeliveryMaxCounter=2, operationName=[...]}
Exchange headers after onException(Fault) :
Headers: {breadcrumbId=ID-ITEM-S28636-63117-1469800853403-0-1, operationName=[...]}
Did you have any solution to ignore some sub-route exception without erasing the upper route retry strategy?
I guess you are connecting the routes via direct endpoint; if that's the case, I think your main issue is that you're marking Fault as continued which implies handled, which is most likely what's causing your redelivery headers to be cleared. You could try using seda or a message queue, just not the direct endpoint since it will basically link your routes into one.
I am trying to implement a process consisting of several webservice-calls, initiated by a JMS-message read by Spring-integration. Since there are no transactions across these WS-calls, I would like to keep track of how far my process has gone, so that steps that are already carried out are skipped when retrying message processing.
Example steps:
Retrieve A (get A.id)
Create new B for A (using A.id, getting B.id)
Create new C for B (using B.id, getting C.id)
Now, if the first attempt fails in step 3, I already have a created a B, and know it's id. So if I want to retry the message, it will skip the second step, and not leave me with an incomplete B.
So, to the question: Is it possible to decorate a JMS-message read by Spring-integration with additional header properties upon message processing failures? If so, how could I do this?
The way it works at the moment:
Message is read
Some exception is thrown
Message processing halts, and ActiveMQ places the message on DLQ
How I would like it to work:
Message is read
Some exception is thrown
The exception is handled, with the result of this handling being an extra header property added to the original message
ActiveMQ places the message on DLQ
One thing that might achieve this is the following:
Read the message
Start processing, wrapped in try-catch
On exception, get the extra information from the exception, create a new message based on the original one, add extra info to header and send it directly to the DLQ
Swallow the exception so the original message dissappears
This feels kinda hackish though, hopefully there is a more elegant solution.
It's hard to generalize without more information about your flow(s) but you could consider adding a custom request handler advice to decorate and/or re-route failed messages. See Adding Behavior to Endpoints.
As the other answer says, you can't modify the message but you can build a new one from it.
EDIT:
So, to the question: Is it possible to decorate a JMS-message read by Spring-integration with additional header properties upon message processing failures? If so, how could I do this?
Ahhh... now I think I know what you are asking; no, you can't "decorate" the existing message; you can republish it with additional headers instead of throwing an exception.
You can republish in the advice, or in the error flow.
It might seem like a "hack" to you, but the JMS API provides no mechanism to do what you want.
From the spring forum:
To place new header to the MessageHeaders you should use
MessageBuilder, because not only headers, but entire Message is
immutable.
return MessageBuilder.fromMessage(message).setHeader(updateflag, message.getHeaders().get("Lgg_Rid") == "ACK" ? "CONF" : "FAIL").build();
In an asynchronous context, errors will go to an error channel - either one you configure yourself and indicate in the message headers with errorChannel, or a global error channel if none is specified. See for more details here.
I'm running a 0.8 Kafka, and build a producer using the provided Java API.
The API functions of sending a message (or messages) return void.
Is there a way to get the status of the sent message? If it sent or failed?
This is extremely important to us since we are reading the messages from a file and we want to delete the file after all messages were sent. But if there were errors and some messages weren't sent and I delete the file it will cause a loss of a very important data.
You can configure your producer to wait until it gets n acks from the Kafka cluster (request.required.acks) so that you have some kind of guarantee that the data has been committed properly before deleting your source file.
If really you need to be sure that the message sent succeeded, you might want to consider the alternative of making the producer to be synchronous (producer.type=sync). This way, you would be able to catch any exception thrown by the blocking invocation and act accordingly. The exception thrown by send() is kafka.common.FailedToSendMessageException.
Kafka's Java API is not ideal, hope this helps you.
We are using Camel fluent builders to set up a series of complex routes, in which we are using dynamic routing using the RecipientList functionality.
We've encountered issues where in some cases, the recipient list contains a messaging endpoint that doesn't exist (for example, something like seda:notThere).
A simple example is something like this:
from("seda:SomeSource")....to("seda:notThere");
How can I configure the route so that if the exchange tries to route to an endpoint that doesn't already exist, an error is thrown?
I'm using Camel 2.9.x, and I've already experimented with the Dead Letter Channel and various Error Handler implementations, with (seemingly) no errors or warnings logged.
The only logging I see indicates that Camel is (attempting to) send to the endpoint which doesn't exist:
2013-07-03 16:07:08,030|main|DEBUG|o.a.c.p.SendProcessor|>>>> Endpoint[seda://notThere] Exchange[Message: x.y.Z#293b9fae]
Thanks in advance!
All endpoints behave differently in this case.
If you attempt to write to a ftp server that does not exist, you certainly get an error (connection refused or otherwise)..
This is also true for a number of endpoints.
SEDA queues gets created if the do not exist and the message will be left there. So your route actually sends to "notThere" and the message will still be there until the application restarts or someone starts to consume messages from seda:notThere. This is the way seda queues are designed. If you set the size of the seda queue by to("seda:notThere?size=100"), then if there is noone reading (or reading slowly) you will get exceptions on message 101 and forward.
If you need to be sure some route is consuming your messages, use "direct" instead of "seda". You can even have some middle layer to use the features of seda with respect to staging and the features of direct knowing there is a consumer active (if sent from recipient list with perhaps user input (god forbid).
from("whatever").recipentList( ... ); // "direct:ep1" work, "direct:ep2" throws exception
from("direct:ep1").to("seda:ep1");
from("seda:ep1").doRealStagedStuffHere();