Adding "Constant" Queues to a Topic Exchange in RabbitMQ - java

I've recently learned RabbitMQ with hopes of implementing it in my work flow. (I will be implementing it in Java) I just finished all the tutorials and was curious how I would implement a "constant" queue instead of a "temporary" queue. Or at least allow the consumer to get the message that the exchange sent. For example if I send a topic of "kern.overflow" but a consumer is offline, as soon as my consumer comes online as long as it is listening for something related to "kern.#" or "#.overflow" I want it to receive un-received messages.

Here is the code to:
create a persistent queue
bind the queue to the topic with "kern.#" as routing-key:
code:
String myPersistentQueue = "myPersistentQueue";
boolean isDurable = true;
boolean isExclusive = false;
boolean isAutoDelete = false;
channel.queueDeclare(myPersistentQueue, isDurable, isExclusive, isAutoDelete, null);
channel.queueBind(myPersistentQueue, "myTopic", "kern.#");
final QueueingConsumer consumer = new QueueingConsumer(channel);
boolean autoAck = true;
String tag1 = channel.basicConsume(myPersistentQueue, autoAck, consumer);
executorService.execute(new Runnable() {
#Override
public void run() {
while (true) {
Delivery delivery;
try {
delivery = consumer.nextDelivery();
String message = new String(delivery.getBody());
System.out.println("Received: " + message);
} catch (Exception ex) {
Logger.getLogger(TestMng.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
});
System.out.println("Consumers Ready");
When you publish a message to myTopic using kern.overflow as routing-key the message is stored to the myPersistentQueue queue. The client can be off-line, when the client is on-line can get the messages.

Related

How can we not receive a new message from the queue in Java rabbitmq until I have not responded to a message?

Receive single message from the queue in RabbitMQ using java:
I'm new to RabbitMQ and was wondering of a good approach to this problem I'm mulling over. I want to create a service that subscribes to a queue and Receive only one message and receive the next one after processing.
DeliverCallback deliverCallback = new DeliverCallback() {
public void handle(String s, Delivery delivery) throws IOException {
System.out.println("Tag: "+delivery.getEnvelope().getDeliveryTag());
String message = new String(delivery.getBody(), "UTF-8");
}
};
channel.basicConsume(QUEUE_NAME, false, deliverCallback, new CancelCallback() {
public void handle(String consumerTag) throws IOException {}
});
To receive a single message from the queue, I did the following steps:
Step 1: Accept only one unack-ed message at a time:
channel.basicQos(1);
Step 2: Setting autoAck to false:
boolean autoAck = false;
Step 3: Work Queues using the Java Client
DeliverCallback deliverCallback = (consumerTag, delivery) -> {
try {
System.out.println("Tag: "+delivery.getEnvelope().getDeliveryTag());
String message = new String(delivery.getBody(), "UTF-8");
} finally {
System.out.println(" [Message] Task Done");
channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
}
};
channel.basicConsume(QUEUE_NAME, autoAck, deliverCallback, consumerTag -> { });
see: rabbitmq documentation
RabbitMQ supports polling a single message from a Queue at a time, but the official document doesnt encourage this approach.
Poll the message by disabling auto acknowledgement, finally send an ack after processing.
boolean autoAck = false;
GetResponse response = channel.basicGet(queueName, autoAck);
if (response != null) {
//process the message and acknowledge it
}
For more details please refer to the official document section 'Retrieving Individual Messages ("Pull API")'
https://www.rabbitmq.com/api-guide.html#getting

RabbitMQ in spring boot with priority queue

My problem because I try to implement priority queue with rabbitMQ but its always random. Even when I set priority #RabbitListener(queues = QUEUE_MESSAGES, priority = "10").
I send 100 messages to two queus :
public void sendRequest() {
for (int i = 0; i < 100; i++) {
try {
rabbitTemplate.convertAndSend(ProducerConfig.QUEUE_MESSAGES2,
new MessageDTO("Subject Two", "content2"), message -> {
message.getMessageProperties().setPriority(Integer.valueOf(10));
return message;
});
rabbitTemplate.convertAndSend(ProducerConfig.QUEUE_MESSAGES,
new MessageDTO("Subject One", "content1"), message -> {
message.getMessageProperties().setPriority(Integer.valueOf(1));
return message;
});
System.out.println("messages has been send");
} catch (AmqpException ex) {
System.out.println(ex.getMessage());
}
}
}
So I have two listeners :
#RabbitListener(queues = QUEUE_MESSAGES, priority = "1")
public void receiveMessage(MessageDTO message) throws BusinessException, InterruptedException {
try {
System.out.println(message.getSubject());
} catch (Exception ex) {
System.out.println("exception" + ex.getMessage());
}
}
#RabbitListener(queues = QUEUE_MESSAGES2, priority = "10")
public void receiveMessage2(MessageDTO message) throws BusinessException, InterruptedException {
try {
System.out.println(message.getSubject());
} catch (Exception ex) {
System.out.println("exception" + ex.getMessage());
}
}
My output is random like this :
Subject One
Subject Two
Subject One
Subject Two
Subject One
Subject Two
Subject One
Subject Two
Subject One
Subject Two
Subject One
Subject Two
Subject One
Subject Two
Subject One
I need to receive all messages from first queue then receive messages from seconds queue. Can anybody help ?
I already even try with this in application.properties :
spring.rabbitmq.listener.simple.prefetch=1
My version is : RabbitMQ 3.8.12 Erlang 23.2.6
#EDIT
I set priority in producer config to queue and in sending request priority to messages but it deosnt helps
Producer config :
#Bean
public Declarables fanoutBindings() {
Queue messageQueue = QueueBuilder.durable(QUEUE_MESSAGES)
.withArgument("x-dead-letter-exchange", DLX_EXCHANGE_MESSAGES)
.withArgument("x-priority", Integer.valueOf(1))
.build();
Queue messageQueue2 = QueueBuilder.durable(QUEUE_MESSAGES2)
.withArgument("x-dead-letter-exchange", DLX_EXCHANGE_MESSAGES)
.withArgument("x-priority", Integer.valueOf(10))
.build();
Queue deadLetterQueue = QueueBuilder.durable(QUEUE_MESSAGES_DLQ).build();
Queue parkingLotQueue = QueueBuilder.durable(QUEUE_PARKING_LOT).build();
FanoutExchange deadLetterExchange = new FanoutExchange(DLX_EXCHANGE_MESSAGES);
return new Declarables(
messageQueue,
parkingLotQueue,
deadLetterQueue,
messageQueue2,
deadLetterExchange,
BindingBuilder.bind(deadLetterQueue).to(deadLetterExchange));
}
The priority property on #RabbitListener is the consumer priority. Consumers with higher priority will receive messages when they are active, while lower priority consumers will only get messages when higher priority consumers block. This assumes those consumers are consuming from the same queue, which is not your case.
If you want to implement priority messages, you need to define a Priority Queue with a max priority and set the priority property when sending the message (messages without priority will be treated as 0 priority).

Kafka Java Consumer SDK long pull without using while

I try to use Kafka Java SDK to implement a consumer however most consumer examples I saw are using while(true) loop and inside the loop call consume method to get a message.
while (true) {
final ConsumerRecords<Long, String> consumerRecords =
consumer.poll(1000);
if (consumerRecords.count()==0) {
noRecordsCount++;
if (noRecordsCount > giveUp) break;
else continue;
}
consumerRecords.forEach(record -> {
System.out.printf("Consumer Record:(%d, %s, %d, %d)\n",
record.key(), record.value(),
record.partition(), record.offset());
});
consumer.commitAsync();
}
I am wondering are there any elegant way to handle this without using while loop which is similar to RabbitMQ implementation following:
Consumer consumer = new DefaultConsumer(channel) {
#Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body)
throws IOException {
String message = new String(body, "UTF-8");
System.out.println(" [x] Received '" + message + "'");
}
};
channel.basicConsume(QUEUE_NAME, true, consumer);
You can try using Spring-kafka which has #KafkaListener annotation and makes the method to listen topic, for more info here
Because in apache-kafka there is no elegant way to make method as a listener to topic, since consumer need to poll records for certain intervals, needed that code in loop
#KafkaListener(topics = "topicName", group = "foo")
public void listen(String message) {
System.out.println("Received Messasge in group foo: " + message);
}
Poll Loop is the only way to consume the messages in Kafka. The elegant code to handle the message should be within the loop.

Read without removing message from JMS queue

How to read a message from WebSphere MQ without deleting the original message from queue?
I have spring application which reads the message from the WebSphere MQ.
After reading, I have a process method which will process the data retrieved from queue.
Step 1:
response = jmsTemplate.receive();
//Message automatically removed from queue.
Step 2:
process(response);
There are chances of throwing exceptions in process method. In case of exceptions, I need to retain the message in the queue.
Is it possible? Is their any way to delete the message only on user acknowledgement?
I tried adding the following:
jmsTemplate.setSessionAcknowledgeMode(javax.jms.Session.CLIENT_ACKNOWLEDGE);
...but still the message is getting deleted.
JmsTemplate creating code snippet:
JndiConnectionFactorySupport connectionFactoryBean = new JndiConnectionFactorySupport();
connectionFactoryBean.setBindingsDir(this.bindingDir);
connectionFactoryBean
.setConnectionFactoryName(connectionFactoryName);
connectionFactoryBean.afterPropertiesSet();
jmsTemplate.setConnectionFactory(connectionFactoryBean.getObject());
JndiDestinationResolver destinationResolver = new JndiDestinationResolver();
destinationResolver.setJndiTemplate(connectionFactoryBean
.getJndiTemplate());
jmsTemplate.setDestinationResolver(destinationResolver);
jmsTemplate.setReceiveTimeout(20000);
jmsTemplate.setDefaultDestinationName(this.defaultDestinationName);
Tried the jmsTemplate.execute() method as below:
#SuppressWarnings({ "unused", "unchecked" })
Message responseMessage = (Message) jmsTemplate.execute(
new SessionCallback() {
public Object doInJms(Session session)
throws JMSException {
MessageConsumer consumer = session
.createConsumer(jmsTemplate.getDestinationResolver().resolveDestinationName(session, "QUEUE_NAME", false));
Message response = consumer.receive(1);
try {
testMethod();//this method will throw exception.
response.acknowledge();
consumer.close();
} catch(Exception e){
consumer.close();//control will come here.
}
return response;
}
}, true);
You can't do that with receive() methods because the operation is complete (from the session perspective) when the receive method returns.
You need to run the code that might fail within the scope of the session; e.g. with a JmsTemplate.execute() with a SessionCallback - something like this...
this.jmsTemplate.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
this.jmsTemplate.convertAndSend("foo", "bar");
try {
String value = this.jmsTemplate.execute(session -> {
MessageConsumer consumer = session.createConsumer(
this.jmsTemplate.getDestinationResolver().resolveDestinationName(session, "foo", false));
String result;
try {
Message received = consumer.receive(5000);
result = (String) this.jmsTemplate.getMessageConverter().fromMessage(received);
// Do some stuff that might throw an exception
received.acknowledge();
}
finally {
consumer.close();
}
return result;
}, true);
System.out.println(value);
}
catch (Exception e) {
e.printStackTrace();
}
You have to browse the queue.
Example of real code that was executed making usage of Websphere MQ
public void browseMessagesAndJiraCreation(String jiraUserName, String jiraPassword) {
int counterMessages = jmsTemplate.browse(destinationQueueName, new BrowserCallback<Integer>() {
#Override
public Integer doInJms(final Session session, final QueueBrowser queueBrowser) throws JMSException {
Enumeration<TextMessage> enumeration = queueBrowser.getEnumeration();
int counterMessages = 0;
while (enumeration.hasMoreElements()) {
counterMessages += 1;
TextMessage msg = enumeration.nextElement();
logger.info("Found : {}", msg.getText());
JiraId jiraId = jiraManager.createIssue(jiraUserName, jiraPassword);
jiraManager.attachFileToJira(jiraId, msg.getText(), jiraUserName, jiraPassword);
}
return counterMessages;
}
});
logger.info("{}:messages were browsed and processed from queue:{}.", counterMessages, destinationQueueName);
}
Explanations:
usage of the Spring Framework JmsTemplate
you pass the String gestinationQueueName (example destinationQueueName=QL.PREFCNTR.USER.REPLY)
Java enumeration of Text messages
counterMessages is the counter of messages that were processed
messages are NOT consumed!
You can add transactional processing of JMS messages. See the example
Your listener should be "transacted". Like this
<jms:listener-container connection-factory="connectionFactory" acknowledge="transacted">
<jms:listener ref="notificationProcessor" destination="incoming.queue"/>
</jms:listener-container>

ActiveMQ Queue And consumers

I am having a scenerio that , i am having more than 4 clients and i want to send a single queue messages to all of that clients. I didnt acknowledge for the client side. So anyone can get that messages from the queue. But the case is that i want to know the number of consumers who consumed that message. Can anyone help me to get the consumer count.
Here below is code that i wrote.
public static boolean sendMessage(String messageText)
{
try {
StompConnection connection = new StompConnection();
HashMap<String, String> header = new HashMap<String, String>();
header.put(PERSISTENT, "true");
connection.open(URLhost, port);
connection.connect("", "");
connection.begin("MQClient");
Thread.sleep(100);
connection.send(queuePath, messageText, "MQClient", header);
connection.commit("MQClient");
connection.disconnect();
return true;
} catch (Exception e) {
throw new BasicException(AppLocal.getIntString("ActiveMQ service ERROR"), e);
}
}
public static String receiveMessage() {
try {
StompConnection connection = new StompConnection();
connection.open(URLhost, port);
connection.connect("", "");
connection.subscribe(queuePath, Subscribe.AckModeValues.INDIVIDUAL);
connection.begin("MQClient");
Thread.sleep(1000);//below not a good NO DATA test .. worked by making thread sleep a while
if (connection.getStompSocket().getInputStream().available() > 1)
{
StompFrame message = connection.receive();
connection.commit("MQClient");
connection.disconnect();
return message.getBody();
}
else
return "";
} catch (Exception e) {
e.printStackTrace();
}
return "";
}
If you are writing to a Queue, then exactly one consumer will receive the message. The whole goal of point-to-point messaging is that only one of the consumers will receive the message.
If you want to send a message and have it be received by all of the consumers, then you'd want to use a Topic instead of a Queue.
If you switch to a topic, multiple clients can consume that same message.
You can probably figure out how many consumed your message by subscribing to the ActiveMQ.Advisory.MessageConsumed.Topic

Categories

Resources