I'm working on an application which processes messages from multiple Azure Service Bus queues. In order to optimize resources, I need to prioritize one queue and process its messages first. Once it's empty, process the next queue. In order to do that, I'm searching for a method to find if particular queue is empty. It would be really good if there's any way to set an empty-queue-listener.
Currently I have implemented my code to process just one queue with ServiceBusProcessorClient.
this.processor = new ServiceBusClientBuilder()
.connectionString(connectionString)
.processor()
.queueName(queueName)
.processMessage(onMessage)
.processError(onError)
.disableAutoComplete()
.prefetchCount(1)
.maxConcurrentCalls(1)
.receiveMode(ServiceBusReceiveMode.PEEK_LOCK)
.buildProcessorClient();
As Gaurav Mantri mentioned in the comment, you can use getTotalMessageCount() to get the number of messages in the queue.
Related
I'm using simple test project with Spring's JmsTemplate that sends synchronous messages with:
jmsTemplate.sendAndReceive(...)
Code snippet of JmsTemplate to do this:
Message requestMessage = messageCreator.createMessage(session);
responseQueue = session.createTemporaryQueue();
producer = session.createProducer(destination);
consumer = session.createConsumer(responseQueue);
requestMessage.setJMSReplyTo(responseQueue);
if (logger.isDebugEnabled()) {
logger.debug("Sending created message: " + requestMessage);
}
doSend(producer, requestMessage);
return receiveFromConsumer(consumer, getReceiveTimeout());
All work fine but when I'm going to Jolokia console I can see all my temp queues at address level:
In standard ActiveMQ console temporary queues are not shown (deleted?).
Because of my application use many synchronous message, list can grow up rapidly.
I try to use
<temporary-queue-namespace>temp</temporary-queue-namespace>
with
<address-setting match="temp.#">
<enable-metrics>false</enable-metrics>
</address-setting>
But my temp-queue are not under temp addresses...
Does it possible to don't show temp queue in console? (because when JmsTemplate has received response or time-out, consumer is closed and temp queue is marked as deleted).
If not, how can I regroup them into one addresses folder?
or something else useful to achieve this.
My application work with about 30-40 queues, and possibly 1000 or more temp queues by day. ActiveMQ "Classic" doesn't show temp queue in web console so its easy to administer durable queue. We plan to migrate to Artemis, and during my simple test case I see that temp queue are by default shown in the web console next to all other queues, and if I have 1000 or more temp queues I need to scroll down a very long time to show the queues that I want to see. After each refresh the scroll is reinitialized. So i want to find a solution to regroup all temp queue in one folder like namespace or other solution.
There are two main ways to deal with a large number of queues and problems with refreshing the JMX "tree" view.
Use the "Queues" tab to view the queues you're interested in rather than the JMX "tree" view. You can even filter out temporary queues, e.g.:
Disable refresh of the JMX "tree" view via the "Preferences" available by clicking on the user icon in the top right of the web console, e.g.:
It's worth noting that the enable-metrics only deals with metrics as they are related to metrics plugins. Setting this to false does not disable their MBeans.
In the future the JMX "tree" like likely be removed from the web console due, in part, to the issues you're observing.
I have a message queue which doesn't have Actor concepts or anything so on the application startup I want to start the message queue consumer which will then keep getting messages from the queue. Now, Play framework creates an Actor for every Web Socket Connection and I want to be able to group all the Actors that hold the Web Socket connection for a particular ws endpoint so that I can broadcast all the messages that I received from message queue on particular topic to those group of Actors.
For Example the following end points will have an Actor created every time a client initiates a request to any of the end point below. so lets call them Foo actors and Bar Actors.
ws://localshost/foo
ws://localshost/bar
https://www.playframework.com/documentation/2.5.x/JavaWebSockets
Now all I want to do is this
Pseudo code:
messages = ReceiveMessagesFromQueue; // This is a live stream and it never stops.
for message in messages:
if message has key1:
List<FooActors> foo_list = getAllFooActors
broadcast(message, foo_list)
else if message has key2:
List<BarActors> bar_list = getAllBarActors
broadcast(message, bar_list)
I am using the latest version of Play framework using Java.
I would avoid the burden of maintaining a list of actors, but rather go for a more decoupled approach using Akka's Event Bus.
Doing so, you could logically group your wensocket actors per topic (foo and bar). This is loosely coupled as neither your websocket actors, nor your message queue consumer needs to know about the other side. They would simply need to subscribe or publish to a specific topic.
The code below builds on the example shown under lookup classification.
When starting up, your websocket actors would just need to subscribe to the appropriate topic, roughly:
LookupBusImpl lookupBus = new LookupBusImpl();
lookupBus.subscribe(getSelf(), "foo"); // or "bar"
Your queue consumer would simply need to publish the messages to the appropriate topic, building on your pseudo code:
LookupBusImpl lookupBus = new LookupBusImpl();
messages = ReceiveMessagesFromQueue;
for message in messages:
if message has key1:
lookupBus.publish(new MsgEnvelope("foo", System.currentTimeMillis()))
else if message has key2:
lookupBus.publish(new MsgEnvelope("bar", System.currentTimeMillis()))
For a distributed version, use Distributed Publish Subscribe in Cluster.
I am trying to fetch a message with a particular correlation id like explained in rabbitmq docs. However I see that the irrelevant messages gets dequeued. I do not want it to happen. How can I tell rabbitmq to not dequeue after I get message and get to know that this is not the one I was looking for. Please help me.
`
.
.
replyQueueName = channel.queueDeclare().getQueue();
consumer = new QueueingConsumer(channel);
channel.basicConsume(replyQueueName, false, consumer);
while (true) {
QueueingConsumer.Delivery delivery = consumer.nextDelivery();
System.out.println(delivery.getProperties().getCorrelationId());
if (delivery.getProperties().getCorrelationId().equals(corrId)) {
response = new String(delivery.getBody());
break;
}
}
`
You can't do what you want, the way you want. The "selective consumer" is an anti-pattern in RabbitMQ.
Instead, you should design your RabbitMQ setup so that your messages are routed to a queue that only contains messages for the intended consumer.
I wrote more about this, here: http://derickbailey.com/2015/07/22/airport-baggage-claims-selective-consumers-and-rabbitmq-anti-patterns/
If you can afford to lose the order of messages you can use the re-queueing mechanism.
Try turning off auto ack.
If not, you have to redesign your application to inject headers or routing keys to route to a particular queue.
String queueA = "rabbitmq://host:5672/queue-a.exchange?queue=queue-a.exchange..etc
from(queueA)
.routeId("idForQueueA")
.onException(Exception.class)
.maximumRedeliveries(0)
// .processRef("sendEmailAlert") * not sure this belongs here*
.to(deadLetterQueueA)
.useOriginalMessage()
.end()
.processRef("dataProcessing")
.processRef("dataExporting")
.end();
Explaining the code above:
Messages are taken from queueA. Upon various processes being successful the message is consumed. If it fails its added to the dead letter queue "deadLetterQueueA". This all works ok.
My question is
When messages arrive in the deadletter queue I want to add alerts so we know to do something about it... How could I to add an email alert when a message arrives in the dead letter queue. I dont want to lose the original message if the alert fails - nor do I want the alert to consume the message.
My thoughts are.. I would need to split the message on an exception so its sent to two different queues? One for the alert which then sends out an email alert and then consumes itself. Then one for the dead letter queue that just sites there? However I'm not sure how to do this?
You can split a message to go to multiple endpoints using a multicast (details here):
.useOriginalMessage().multicast().to(deadLetterQueueA, "smtp://username#host:port?options")
This uses the camel mail component endpoints described here. Alternatively, you can continue processing the message after the to. So something like:
.useOriginalMessage()
.to(deadLetterQueueA)
.transform().simple("Hi <name>, there has been an error on the object ${body.toString}")
.to("smtp://username#host:port?options")
If you had multiple recipients, you could use a recipients list
public class EmailListBean {
#RecipientList
public String[] emails() {
return new String[] {"smtp://joe#host:port?options",
"smtp://fred#host:port?options"};
}
}
.useOriginalMessage()
.to(deadLetterQueueA)
.transform().simple("...")
.bean(EmailListBean.class)
Be careful of using JMS queues to store messages while waiting for a human to action them. I don't know what sort of message traffic you're getting. I'm assuming if you want to send an email for every failure, it's not a lot. But I would normally be wary of this sort of thing, and chose to use logging or database persistence to store the results of errors, and only use a JMS error queue to notify other processes or consumers of the error or to schedule a re-try.
There are two ways you can do this , but based on your message volume you might not want to send email on every failed message.
You can use the solution provided by AndyN , or you can use the Advisory Topics ActiveMQ.Advisory.MessageDLQd.Queue.* , whenever a message gets in to the DLQ the enqueue count of the topic will increase by 1 . By monitoring the Queue Depth you might now be able to send a mail to based on the number of the errors that ocurred.
If you want to do it at the producer end. You can use any one of the solutions provided by AndyN
I am intercepting messages that are sent through JBossESB. I am using pipeline interceptors to do so.
The problem is, that altough the sender is a service (for example PortReference < logical:BlueServiceESB#BlueListener >), the name of the receiver is a queue (not a service). That is logical because in some case, multiple services can receive messages from a given queue, but usually, each queue is mapped to only one service.
I would like to know which queue is mapped to which service, so I can display/save this information and have it displayed like message: service ---> service (not service ---> queue).
I know that I can get the name of the queue mapped to a service using the registry like this:
System.setProperty("javax.xml.registry.ConnectionFactoryClass", "org.apache.ws.scout.registry.ConnectionFactoryImpl");
// Retrieving information from the ESB Registry
Registry reg = RegistryFactory.getRegistry();
System.out.println(reg.findAllServices());
List<EPR> eprs = reg.findEPRs("FirstServiceESB", "SimpleListener");
System.out.println(eprs);
I would like to reverse this approach - queue is the input and service (EPR = end point reference = service) is the output. Is there any way how to do this or am I just trying to do the impossible here. I have found no tutorials or questions on this topic whatsoever.
Thanks for any tips!
As this question has 25 up-votes, this seems to be an useful feature. JBossESB is open source software. Thus, implement the feature yourself and commit it to the community! Or just create a change request hopping that somebody else will do it...
Try querying for all of the queues and building a reverse-lookup map. But I don't think there is any function that allows searching for services using a queue.