I am trying to implement a process consisting of several webservice-calls, initiated by a JMS-message read by Spring-integration. Since there are no transactions across these WS-calls, I would like to keep track of how far my process has gone, so that steps that are already carried out are skipped when retrying message processing.
Example steps:
Retrieve A (get A.id)
Create new B for A (using A.id, getting B.id)
Create new C for B (using B.id, getting C.id)
Now, if the first attempt fails in step 3, I already have a created a B, and know it's id. So if I want to retry the message, it will skip the second step, and not leave me with an incomplete B.
So, to the question: Is it possible to decorate a JMS-message read by Spring-integration with additional header properties upon message processing failures? If so, how could I do this?
The way it works at the moment:
Message is read
Some exception is thrown
Message processing halts, and ActiveMQ places the message on DLQ
How I would like it to work:
Message is read
Some exception is thrown
The exception is handled, with the result of this handling being an extra header property added to the original message
ActiveMQ places the message on DLQ
One thing that might achieve this is the following:
Read the message
Start processing, wrapped in try-catch
On exception, get the extra information from the exception, create a new message based on the original one, add extra info to header and send it directly to the DLQ
Swallow the exception so the original message dissappears
This feels kinda hackish though, hopefully there is a more elegant solution.
It's hard to generalize without more information about your flow(s) but you could consider adding a custom request handler advice to decorate and/or re-route failed messages. See Adding Behavior to Endpoints.
As the other answer says, you can't modify the message but you can build a new one from it.
EDIT:
So, to the question: Is it possible to decorate a JMS-message read by Spring-integration with additional header properties upon message processing failures? If so, how could I do this?
Ahhh... now I think I know what you are asking; no, you can't "decorate" the existing message; you can republish it with additional headers instead of throwing an exception.
You can republish in the advice, or in the error flow.
It might seem like a "hack" to you, but the JMS API provides no mechanism to do what you want.
From the spring forum:
To place new header to the MessageHeaders you should use
MessageBuilder, because not only headers, but entire Message is
immutable.
return MessageBuilder.fromMessage(message).setHeader(updateflag, message.getHeaders().get("Lgg_Rid") == "ACK" ? "CONF" : "FAIL").build();
In an asynchronous context, errors will go to an error channel - either one you configure yourself and indicate in the message headers with errorChannel, or a global error channel if none is specified. See for more details here.
Related
Using spring-camel, I have built a route that consumes from a JMS topic (with JMSReplyTo expected to be set for each input message), splits the message into smaller chunks, sends them to a REST processsor, then aggregates the answers and should produce an output message to the destination pointed by JMSReplyTo. Unfortunately, camel implicitly utilises the JMSReplyTo destination in one of the intermediate steps (producing an unmarshalled POJO).
We have a functional requirement to adapt JMSReplyTo in order to provide a request-reply messaging service.
I am able to read the JMSReplyTo header before ending the route and I am explicitly converting it to CamelJmsDestinationName, which successfully overrides the destination for JMS component and produces the message on the output topic. I am not sure if this is the best approach and the problem is that camel still utilises the JMSReplyTo on its own.
My RouteBuilder configuration is as follows:
from("jms:topic:T.INPUT")
.process(requestProcessor)
.unmarshal().json(JsonLibrary.Jackson, MyRequest.class)
.split(messageSplitter)
.process(restProcessor)
.aggregate(messagesAggregator)
.unmarshal().json(JsonLibrary.Jackson, BulkResponses.class)
.process(responseProcessor)
.to("jms:topic:recipientTopic");
T.INPUT is the name of the input topic, while recipientTopic is just a placeholder that will be replaced by CamelJmsDestinationName.
I'm not keen on using CamelJmsDestinationName and a sort of a mocked up topic name in route configuration so I'm open to find a better solution. It would be great if camel utilised the JMSReplyTo automatically to produce the output message to the output topic.
Currently, the problem is that camel produces an intermediate output on the JMSReplyTo topic BUT the output is an unmarshalled MyRequest object, which results in an exception saying "ClassNotFoundException: (package name).MyRequest", which is obvious since this is only a class used in my internal processing - I don't want to produce this to the output topic. It seems like Camel does implicitly use the JMSReplyTo destination between requestProcessor and messageSplitter processing... Why? What am I doing wrong? What are the best practices?
Use "disableReplyTo=true" in Endpoint. Camel will not try to use any reply option.
Refer: https://camel.apache.org/jms.html for more details
I have found the answer... this is absurdly easy but I haven't seen it anywhere in the documentation.
You just need to call .stop() to mark the route as completed, and Camel will reply the body you configured in the last step to the destination configured in ${header.JMSReplyTo}. It's that simple.
So you can do:
from("jms:my-queue")
.unmarshall().json(JsonLibrary.Jsonb, InboundMessage.class)
.bean(SomeProcessingBean.class)
....
.log(LoggingLevel.INFO, "Sending reply to: " + simple("${header.JMSReplyTo}").getExpression().toString())
.marshall().json(JsonLibrary.Jsonb, ReplyMessage.class)
.stop();
And you will receive reply.
I wonder why no one has found this before... Nothing when I search the doc or here.... I must be dumb, or the doc is incomplete...but I am not dumb, so.
I did read this reference: https://www.rabbitmq.com/dlx.html, however it doesn't resolve my doubts, namely:
In case of accepting message there is no problem - spring-rabbitmq send ack and everthing is fine, DLX doesn't know about acked message.
The problem is in case rejecting answer, namely what about throwing MessageConverterException ? This message is removed or moved to DLX?
And what about in case other exception ? For example Exception ? It is removed/requeued/moved to DLX ?
Edit after answer of #Gary
I think, that after answer's #Gary I should add more details about my case and some summary of #Gary's answer. #Gary exactly grasped my use case.
I wouldn't like requeue - never (I am afraid of looping), but I wouldn't like to lose messages when an exception was thrown (for example lost connection to database) - this message should be resend to DLX. On the other hand, conversion of message should be treated as fatal error - no requeue, no resend to DLX - simply permanent removing message. Generally, in depends on exception either reject (=resend to DLX if configured) or accept, never requeue.
To sum up in a nutshell approach proposed by #Gary.
First: We can override ExceptionHandler to manage of sending nack/ack, which gives to us a full control.
Second: IMO simpler, solution is to set defaultRequeueRejected=false and in converter throw ImmediateAcknowledgeAmqpException. It makes that RabbitMQ think that answer was accepted (the same thing as in case of first solution), moreover listener wouldn't be invoked.
**Conclusion**: UsingImmediateAcknowledgeAmqpExceptionorExceptionHandler` exception we have a full control on permanent rejecting message (under hood ack) and resending to DLX.
RabbitMQ knows nothing about the exceptions.
When the container catches an exception it calls channel.basicReject(deliveryTag, requeue).
If requeue is true, the message is requeued.
By default, for any exception other than those mentioned here
o.s.amqp...MessageConversionException
o.s.messaging...MessageConversionException
o.s.messaging...MethodArgumentNotValidException
o.s.messaging...MethodArgumentTypeMismatchException
java.lang.NoSuchMethodException
java.lang.ClassCastException
requeue is set to true, so the message is requeued.
For those exceptions, the delivery is considered fatal and the message is NOT requeued, it will go to a DLX/DLQ if one is configured.
The container has a flag defaultRequeueRejected which is true by default; if you set it to false; no exceptions will be requeued.
For application-level exceptions, generally, messages will be requeued. To dynamically reject (and not requeue) a message, make sure there is an AmqpRejectAndDontRequeueException in the cause chain. This instructs the container to not requeue the message, and it will go to the DLX/DLQ (if configured). This behavior is enabled by the defaultRequeueRejected flag mentioned above.
This is all explained in the documentation and, as I have discussed in other answers to you, you can change this behavior by using a custom error handler; that, too, is explained in the documentation.
It is not possible to send some exceptions to the DLX/DLQ and not others; rabbit only has a binary option, requeue or don't requeue and, for the latter, if a DLX/DLQ is configured all such rejected messages go to the DLX/DLQ.
Spring AMQP provides one more exception, ImmediateAcknowledgeAmqpException. If your listener throws this exception, the message will be ack'd as if it was processed successfully (channel.basicAck()). That is the only technique provided by the container, to discard a bad message without sending it to the DLX/DLQ.
Of course, your application can drop such messsages itself.
If you want to DLX/DLQ all business exceptions but drop conversion exceptions, throw AmqpRejectAndDontRequeueException (or set defaultRequeueRejected to false), and throw ImmediateAcknowledgeAmqpException from your converter.
I'm running a 0.8 Kafka, and build a producer using the provided Java API.
The API functions of sending a message (or messages) return void.
Is there a way to get the status of the sent message? If it sent or failed?
This is extremely important to us since we are reading the messages from a file and we want to delete the file after all messages were sent. But if there were errors and some messages weren't sent and I delete the file it will cause a loss of a very important data.
You can configure your producer to wait until it gets n acks from the Kafka cluster (request.required.acks) so that you have some kind of guarantee that the data has been committed properly before deleting your source file.
If really you need to be sure that the message sent succeeded, you might want to consider the alternative of making the producer to be synchronous (producer.type=sync). This way, you would be able to catch any exception thrown by the blocking invocation and act accordingly. The exception thrown by send() is kafka.common.FailedToSendMessageException.
Kafka's Java API is not ideal, hope this helps you.
We are using Camel fluent builders to set up a series of complex routes, in which we are using dynamic routing using the RecipientList functionality.
We've encountered issues where in some cases, the recipient list contains a messaging endpoint that doesn't exist (for example, something like seda:notThere).
A simple example is something like this:
from("seda:SomeSource")....to("seda:notThere");
How can I configure the route so that if the exchange tries to route to an endpoint that doesn't already exist, an error is thrown?
I'm using Camel 2.9.x, and I've already experimented with the Dead Letter Channel and various Error Handler implementations, with (seemingly) no errors or warnings logged.
The only logging I see indicates that Camel is (attempting to) send to the endpoint which doesn't exist:
2013-07-03 16:07:08,030|main|DEBUG|o.a.c.p.SendProcessor|>>>> Endpoint[seda://notThere] Exchange[Message: x.y.Z#293b9fae]
Thanks in advance!
All endpoints behave differently in this case.
If you attempt to write to a ftp server that does not exist, you certainly get an error (connection refused or otherwise)..
This is also true for a number of endpoints.
SEDA queues gets created if the do not exist and the message will be left there. So your route actually sends to "notThere" and the message will still be there until the application restarts or someone starts to consume messages from seda:notThere. This is the way seda queues are designed. If you set the size of the seda queue by to("seda:notThere?size=100"), then if there is noone reading (or reading slowly) you will get exceptions on message 101 and forward.
If you need to be sure some route is consuming your messages, use "direct" instead of "seda". You can even have some middle layer to use the features of seda with respect to staging and the features of direct knowing there is a consumer active (if sent from recipient list with perhaps user input (god forbid).
from("whatever").recipentList( ... ); // "direct:ep1" work, "direct:ep2" throws exception
from("direct:ep1").to("seda:ep1");
from("seda:ep1").doRealStagedStuffHere();
I want to understand a java program and need to modify which was developed using jms spring framework. Typically it has JMS receiver & sender, it receives a message from request queue and will invoke a job (another java program) once the job is completed the sender will send response to response queue. Have couple of questions which are below,
The request message is not deleted until response posted into response queue successfully. How its been achieved what is the logic behind it.
I want to write a functionality of writing response into flat file when sender fails to send message (by catching JMS exception). Once the sender queue is up and running i will read flat file and will send responses. The reason i need is because its involved in job processing could be in hours if job failed then input message will be read again by receiver. I want to avoid duplicate processing. Please suggest your ideas here.
Without seeing the configuration it's hard to answer these questions, but best guess is that #1 is because the app is using a transactional session. This means all updates on that session are not completed until the transaction is committed.
Just catch the exception and write the data; as long as the transaction commits (because you caught the exception) the input message will be removed.