Is there any way a publisher can be acknowledged that a published message has been delivered to a listener when using Spring AQMP? I have a number of queues where I set x-message-ttl = 0, which means messages will be discarded if they cannot be immediately delivered, but as I'm using this in a request/reply scenario, I'd like to be able to abort the request and handle an error immediately.
You could publish a message with the mandatory flag.
If this flag is set, the server will return an undeliverable message
with a Return method. If this flag is zero, the server will queue the
message, but with no guarantee that it will ever be consumed.
And set a return callback which will be called if the message in unroutable.
Another solution should be to use an alternate exchange associated to your exchange. The cons are that you need to bind a queue to this AE and consume messages to be able to know if a request has failed.
Related
Is there a way we can push messages to RabbitMQ and have an expiry time for it and once it expires, it should provide a notification.
Or
Is there a way we can deliver the messages in RabbitMQ after a certain amount of time. For example, I want to push a message in the queue and wants it to get delivered after 10 seconds..and simultaneously next messages.
Regarding the first part of your question, the routing of messages that have expired due to a per-message TTL is a feature of the RabbitMQ dead letter exchange (DLX).
Regarding a delay, this is not something supported by RabbitMQ out of the box, nor in my opinion should it be a feature of a message broker. I can't imagine a legitimate use case where you would deliberately want to introduce a delay into a message queue. In fact, it is a design goal of any message broker to minimize delay with enqueued messages. If you find a delay to be appropriate, then it is also likely that a message queue is not the appropriate means of conveyance.
The RabbitMQ Delayed Message Plugin adds a new exchange type to RabbitMQ where messages routed by that exchange can be delayed if the users chooses to do so.
You can use it in a way like described below.
// ... elided code ...
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-delayed-type", "direct");
channel.exchangeDeclare("my-exchange", "x-delayed-message", true, false, args);
// ... more code ...
I'm new to RabbitMQ and want to implement asynchronous messaging of SAGA with RabbitMQ.So I used RPC example of RabbitMQ to do the task. I've one orchestrator ( RPCClient) and multiple microservices ( RPCServer). Orchestrator uses unique queues to command microservices.And each microservice uses a common queue ( Reply_ Queue) to reply orchestrator. To keep log I want to get notifications in orchestrator side, when any microservice is down for any configurable time.
I read about consumer cancellation,but it only works when I delete the queue.How to get notifications in JAVA with keeping queue messages? And is it correct way to implement saga asynchronous messaging?
To implement a reliable RPC is hard, I can't give a detail guide about how to do this. If we ignore same special failure situation, I can give a simple workaround:
First, we assume that RPCClient never fail, RPCServer may fail anytime.
RPCClient need to know which request is timeout, so it can send request message with a TTL. After RPCServer receive request message and send response message, it should ACK the request message.
If RPCServer:
has failed before consume request message
OR
has failed before send response message
The request message will be republish to Dead Letter Exchange, so RPCClient can consume to some queue binded with that exchange, it can know which request is timeout.
I am using Amazon SQS with Amazon SQS-JMS java library with Java EE 7. What I want to achieve is after receiving a message, depending on business logic of the application either confirm (consume) the message or resend it to the queue again and after 3 failed retries move it to DLQ.
I though about using CLIENT_Acknowledge mode in JMS and only acknowledging the messages that were successfully processed, but this is from their official documentation:
In this mode, when a message is acknowledged, all messages received before this message are implicitly acknowledged as well. For example, if 10 messages are received, and only the 10th message is acknowledged (in the order the messages are received), then all of the previous nine messages are also acknowledged.
This example also seems to confirm this: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/code-examples.html#example-synchronous-receiver-client-acknowledge-mode.
For me this is kind of a strange behavior and opposite what I would expect from a client_acknowledge. Is there a more elegant solution here than just manually sending message throughout the code to main SQS queue or DLQ depending on process status?
You can use:
UNORDERED_ACKNOWLEDGE
SQSSession.UNORDERED_ACKNOWLEDGE
Which comes from 'com.amazon.sqs.javamessaging;' and as it states in the documentation it is a variation of Client_Acknowledge which only acknowledges the message for which it is called.
/**
* Non standard acknowledge mode. This is a variation of CLIENT_ACKNOWLEDGE
* where Clients need to remember to call acknowledge on message. Difference
* is that calling acknowledge on a message only acknowledge the message
* being called.
*/
dependency example:
"com.amazonaws:amazon-sqs-java-messaging-lib:1.0.3"
To handle this case you can use RedrivePolicy attribute for the DLQ that you created. Solution for this case can be:
Create a 2 sqs Qs say my_q and my_q_dl (latter one is for DLQ)
Set DLQ my_q_dl as the DLQ of my_q by using RedrivePolicy.
Here, care should be taken to specify deadLetterTargetArn and maxReceiveCount. This maxReceiveCount is the number of times you want to process any message without acknowledging before sending it to the DLQ. If you set maxReceiveCount=3 then, the msg will remain in my_q up to 3rd pull by the consumer with no ack.
2 cases here:
Normal case: msg gets deleted as soon as ack is received.
If no ack (msg delete) for that msg upto third time then the msg gets deleted from my_q and pushed to
my_q_dl itself.
*RedrivePolicy - The string that includes the parameters for the deadletter queue functionality of the source queue.
deadLetterTargetArn - The Amazon Resource Name (ARN) of the dead-letter queue to which Amazon SQS moves messages after the value
of maxReceiveCount is exceeded.
maxReceiveCount - The number of times a message is delivered to the source queue before being moved to the dead-letter queue.
Note
The dead-letter queue of a FIFO queue must also be a FIFO queue. Similarly, the dead-letter queue of a standard queue must also be a
standard queue.*
RabbitMQ's Channel#basicConsume method gives us the following arguments:
channel.basicConsume(queueName, autoAck, consumerTag, noLocal,
exclusive, arguments, callback);
Giving us the ability to tell RabbitMQ exactly which queue we want to consume from.
But Channel#basicPublish has no such equivalency:
channel.basicPublish(exchangeName, routingKey, mandatory, immediateFlag,
basicProperties, messageAsBytes);
Why can't I specify the queue to publish to here?!? How do I get a Channel publishing to, say, a queue named logging? Thanks in advance!
To expand on #Tien Nguyen's answer, there is a "cheat" in RabbitMQ that effectively lets you publish directly to a queue. Each queue is automatically bound to the AMQP default exchange, with the queue's name as the routing key. The default exchange is also known as the "nameless exchange" - ie its name is the empty string. So if you publish to the exchange named "" with routing key equal to your queue's name, the message will go to just that queue. It is going through an exchange as #John said, it's just not one that you need to declare or bind yourself.
I don't have the Java client handy to try this code, but it should work.
channel.basicPublish("", myQueueName, false, false, null, myMessageAsBytes);
That said, this is mostly contrary to the spirit of how RabbitMQ works. For normal application flow you should declare and bind exchanges. But for exceptional cases the "cheat" can be useful. For example, I believe this is how the Rabbit Admin Console allows you to manually publish messages to a queue without all the ceremony of creating and binding exchanges.
Basically queues can be binded to an exchange based on routingKeys.
Assume that you have 3 different publishers.
Publisher1 sending message to exchange with routingKey "events"
Publisher2 sending message to exchange with routingKey "tasks"
Publisher3 sending message to exchange with routingKey "jobs"
You can have a consumer that consumes only messages with specific routhingKey.
For example in order to have a consumer for "events" messages you declare like this
channel.queueBind(queueName, exchangeName, "events");
If you want to consume all the messages coming to the exchange you give the routing as '#'
So in short what i can say is,
1. Messages will be published to an exchange.
2. Queues will be bound to exchange based on routingKeys.
3. RabbitMQ will forward messages with matching routing keys to the corresponding queues.
Please see the tutorial - http://www.rabbitmq.com/tutorials/tutorial-three-java.html
The core idea in the messaging model in RabbitMQ is that the producer never sends any messages directly to a queue. Actually, quite often the producer doesn't even know if a message will be delivered to any queue at all. Instead, the producer can only send messages to an exchange
please try this:
channel.basicPublish("", yourQueueName, null,
message.getBytes((Charset.forName("UTF-8"))));
It worked for my project.
I have a situation in which the I'm reading messages from a queue using a message consumer (javax.jms.MessageConsumer) .
The session used is using AUTO_ACKNOWLEDGE mode.
From what I've read so far on AUTO_ACK mode:
In auto acknowledgement if the consumer's onMessage() method completes without error the message is considered received and processed successfully, it'll be then removed from the JMS server.
My question is when is the message considered to be ACK by the JMS producer considering I`m not using an MDB that has an onMessage() method but reading the messages by using the message consumer described earlier.
Is the message ACK'ed once I successfully read it using the messageConsumer ?
What will happen if further down the logic-chain a method that uses the respective message will throw an error ? Will the message already be ACK'ed by that time ?
The Javadoc for the AUTO_ACKNOWLEDGE constant says this:
With this acknowledgment mode, the session automatically acknowledges
a client's receipt of a message either when the session has
successfully returned from a call to receive or when the message
listener the session has called to process the message successfully
returns.
I suspect you are calling receive on the MessageConsumer (although you don't explicitly state that) so if you set AUTO_ACKNOWLEDGE it is acknowledged by the time receive returns.
Of course if you have a transacted session then the acknowledge mode is ignored and the message isn't considered received until the session is committed.