Develop Reference

How to move error message to Azure dead letter queue(Topics - Subscription) using Java?

I need to send my messages to Dead letter queue from azure topic subscription incase of any error while reading and processing the message from topic. So I tried testing pushing message directly to DLQ.
My sample code will be like
static void sendMessage()
{
// create a Service Bus Sender client for the queue
ServiceBusSenderClient senderClient = new ServiceBusClientBuilder()
.connectionString(connectionString)
.sender()
.topicName(topicName)
.buildClient();
// send one message to the topic
senderClient.sendMessage(new ServiceBusMessage("Hello, World!"));
}
static void resceiveAsync() {
ServiceBusReceiverAsyncClient receiver = new ServiceBusClientBuilder()
.connectionString(connectionString)
.receiver()
.topicName(topicName)
.subscriptionName(subName)
.buildAsyncClient();
// receive() operation continuously fetches messages until the subscription is disposed.
// The stream is infinite, and completes when the subscription or receiver is closed.
Disposable subscription = receiver.receiveMessages().subscribe(message -> {
System.out.printf("Id: %s%n", message.getMessageId());
System.out.printf("Contents: %s%n", message.getBody().toString());
}, error -> {
System.err.println("Error occurred while receiving messages: " + error);
}, () -> {
System.out.println("Finished receiving messages.");
});
// Continue application processing. When you are finished receiving messages, dispose of the subscription.
subscription.dispose();
// When you are done using the receiver, dispose of it.
receiver.close();
}
I tried getting the deadletter queue path
String dlq = EntityNameHelper.formatDeadLetterPath(topicName);
I got path of dead letter queue like = "mytopic/$deadletterqueue"
But It's not working while passing path as topic name. It throwing a Entity topic not found exception.
Any one can you please advise me on this
Reference :
How to move error message to Azure dead letter queue using Java?
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dead-letter-queues#moving-messages-to-the-dlq
How to push the failure messages to Azure service bus Dead Letter Queue in Spring Boot Java?
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-java-how-to-use-topics-subscriptions-legacy#receive-messages-from-a-subscription

You probably know that a message will be automatically moved to the deadletter queue if you throw exceptions during processing, and the maximum delievery count is exceeded. If you want to explicitly move the message to the DLQ, you can do so as well. A common case for this is if you know that the message can never succeed because of its contents.
You cannot send new messages directly to the DLQ, because then you would have two messages in the system. You need to call a special operation on the parent entity. Also, <topic path>/$deadletterqueue does not work, because this would be the DLQ of all subscriptions. The correct entity path is built like this:
<queue path>/$deadletterqueue
<topic path>/Subscriptions/<subscription path>/$deadletterqueue
https://github.com/Azure/azure-service-bus/blob/master/samples/Java/azure-servicebus/DeadletterQueue/src/main/java/com/microsoft/azure/servicebus/samples/deadletterqueue/DeadletterQueue.java
This sample code is for queues, but you should be able to adapt it to topics quite easily:
// register the RegisterMessageHandler callback
receiver.registerMessageHandler(
new IMessageHandler() {
// callback invoked when the message handler loop has obtained a message
public CompletableFuture<Void> onMessageAsync(IMessage message) {
// receives message is passed to callback
if (message.getLabel() != null &&
message.getContentType() != null &&
message.getLabel().contentEquals("Scientist") &&
message.getContentType().contentEquals("application/json")) {
// ...
} else {
return receiver.deadLetterAsync(message.getLockToken());
}
return receiver.completeAsync(message.getLockToken());
}
// callback invoked when the message handler has an exception to report
public void notifyException(Throwable throwable, ExceptionPhase exceptionPhase) {
System.out.printf(exceptionPhase + "-" + throwable.getMessage());
}
},
// 1 concurrent call, messages are auto-completed, auto-renew duration
new MessageHandlerOptions(1, false, Duration.ofMinutes(1)),
executorService);

Unable to push message into ActiveMQ

I'm successfully pushing message into ActiveMQ from local Eclipse setup. However, the same code does not push message when I try to execute from server as a cron job. It does not even throw an exception during code execution.
Java environment - 1.8
Supporting jars used:
slf4j-api-1.8.0-beta2.jar
javax.annotation-api-1.2.jar
javax.jms-api-2.0.1.jar
management-api-1.1-rev-1.jar
activemq-core-5.7.0.jar
Code:
try {
map = getMessageDetails(session,"MessageQueueEmail");
userName = map.get("userName");
password = map.get("password");
hostName = map.get("mqHostName");
queue = map.get("queueName");
// Create a ConnectionFactory
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(userName, password, hostName);
// Create a Connection
connection = factory.createConnection();
// start the Connection
connection.start();
System.out.println("MQ started connection");
// Create a Session
sessionMQ = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
// Create the destination Queue
Destination destination = sessionMQ.createQueue(queue);
// Create a MessageProducer from the Session to the Queue
messageProducer = sessionMQ.createProducer(destination);
messageProducer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
// Create a message
Message message = sessionMQ.createTextMessage(textMsg);
System.out.println("MQ Message sent successfully");
// Tell the producer to send the message
messageProducer.send(message);
} catch(Exception e) {
e.printStackTrace();
System.out.println("\n::::::::::::Error occurred sendEmailMessageToIntranet::::::::::::: " + e.getMessage());
}

Thanks everyone for response. The issue is resolved after importing correct certificate file to the server. Wondering, why MQ attempts failure notification had not logged

Your code looks ok except you might have expiration going. Try with PERSISTENT and most likely is the issues that you are not redirecting stderr in your cronjob ? Make sure you do something like this:
*/1 * * * * /something/send.sh &>> /something/out.log
And then check in the morning.

Vertx services not accepting messages continuously when running on local JVM over a finite set of data when deployed as separate fat-jars

I am getting started with vertx and was trying out point to point messaging on event bus. I have 2 services both created as separate maven projects and deployed as fat-jars
1) Read from a file and send the content as a message over an address - ContentParserService.java
2) Read the message and reply to the incoming message- PingService.java
Both these services are deployed as separate jars kind of a microservice fashion
The code is as follows: ContentParserService.java
#Override
public void start(Future<Void> startFuture) throws Exception {
super.start(startFuture);
// Reference to the eventbus running on JVM
EventBus eventBus = vertx.eventBus();
// Read file using normal java mechanism
try {
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(ClassLoader.getSystemResourceAsStream(
(config().getString("filename")))));
bufferedReader.readLine(); //read first line
String line = null;
while ((line = bufferedReader.readLine()) != null) {
String[] data = line.split(",");
// Create RealEstate Object
RealEstateTransaction realEstateData = createTransactionObject(data);
// Construct Message JSON
JsonObject messageJSON = constructMessageJson(realEstateData);
// Send message to PING address over the Event Bus
eventBus.send("PING", Json.encode(messageJSON), reply -> {
if (reply.succeeded())
System.out.println("Received Reply: " + reply.result().body());
else {
System.out.println("No reply");
}
});
}
} catch (IOException e) {
startFuture.fail(e.getMessage());
}
The code is as follows: PingService.java
#Override
public void start(Future<Void> startFuture) throws Exception {
super.start(startFuture);
System.out.println("Referencing event bus");
// Reference to the event bus running on the JVM
EventBus eventBus = vertx.eventBus();
System.out.println("Creating HttpServer");
// Create HTTP Server to handle incoming requests
HttpServer httpServer = vertx.createHttpServer();
System.out.println("Creating Router");
// Create Router for routing to appropriate endpoint
Router router = Router.router(vertx);
System.out.println("Starting to consume message sent over event bus");
// Consume the incoming message over the address PING
eventBus.consumer("PING", event -> {
System.out.println("Received message: " + event.body());
event.reply("Received at PING address");
});
System.out.println("Receiver ready and receiving messages");
When i run both the services I run on the same machine with the java -jar command for each of the service. What i observed was when i deploy the first jar of ContentParserService, it immediately starts and sends messages over the event bus, but by the time i start the pingservice jar , it is not able to receive any message sent over the event bus because my pingService is a separate fatjar and a microservice in itself. The file that i am reading is a finite lenght csv file of around 200 entries. This case would work if i bundle both the services in a single fat jar.
How should i achieve the different fat jars services able to send message to each other in my case.

This case works when both verticles in the same jar only because there's no network delay. But your usecase for EventBus is incorrect, since it doesn't persist messages, hence cannot replay them. You should start sending messages only when the other side is ready to receive them.
You need to reverse the dependency. In your ContentParserService register for some "ready" event, then start your while loop only when you get it:
vertx.eventBus().consumer("ready", (message) -> {
while ((line = bufferedReader.readLine()) != null) {
...
}
});
Now, what will happen if ContentParserService is actually slower and misses the "ready" event? Use vertx.setPeriodic() for that. So you start your PingService, and periodically tell ContentParserService that you're ready to receive some messages.
Or, as an option, just don't use EventBus at all between you services, and switch to something with persistence, like RabbitMQ or Kafka.

ActiveMQ browser needs long time for last .hasMoreElements()

I try to implement a queue browser for ActiveMQ.
The code shown below should display the text messages in the queue named 'Q1'. There are two messages in there. In general it works but the last e.hasMoreElements() call needs up to 20 seconds. I wanted to update the list every 500 millis. Why is that so slow?When i press 'update' in the browser view for http://localhost:8161/admin/browse.jsp?JMSDestination=Q1 e.hasMoreElements() returns immediately. What's going on here? How to achieve a 'realtime' view?
//init:
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("tcp://localhost:61616");
Connection connection = connectionFactory.createConnection();
connection.start();
Session session = connection.createSession(true, Session.CLIENT_ACKNOWLEDGE);
Queue queue = session.createQueue("Q1");
boolean run = true;
while (run) {
LOG.info("--------------------------------------------------------------------------------");
QueueBrowser browser = session.createBrowser(queue);
Enumeration e = browser.getEnumeration();
while (e.hasMoreElements()) { //<- very slow before returning false after last message. why?
Object o = e.nextElement();
if (o instanceof ActiveMQTextMessage) {
LOG.info(((ActiveMQTextMessage) o).getText());
} else {
LOG.info(o.toString());
}
}
Thread.sleep(500);
browser.close();
}
session.close();
connection.close();

After my previous comment, I've discovered that calling setTransactedIndividualAck(true) on the connection factory solves the problem
ActiveMQConnectionFactory cf2 = new ActiveMQConnectionFactory(...);
cf2.setTransactedIndividualAck(true);
I'm not sure that this is the right thing to do for the problem but at least it works now. See the message on the ActiveMQ user forum here:
http://activemq.2283324.n4.nabble.com/JMS-browser-getEnumeration-hasMoreElements-takes-15s-on-last-element-td4709969.html

I had the same issue. But changing the acknowledgment on the session eliminated the delay.
Try this:
Session session = connection.createSession(true, Session.AUTO_ACKNOWLEDGE);

I found that calling Session.commit() within my hasMoreElements() loop stopped the hanging using activemq-broker version 5.14.5:
while(enumeration.hasMoreElements()) {
final Message message = (Message)enumeration.nextElement();
final TextMessage textMessage = (TextMessage)message;
session.commit();
}
I did more research to see if this was a bug with ActiveMQ or not and found that activemq-broker version 5.15.1 did not hang, even without calling commit() after each iteration. All prior versions of the broker hanged on the final call to hasMoreElements(). It doesn't seem like the contributors deliberately fixed this particular issue, since the bug report on JIRA that the change referenced was for something different. The change that fixed this issue changed part of the iterate() method of the org.apache.activemq.broker.region.Queue class from:
// are we done browsing? no new messages paged
if (!added || browser.atMax()) {
browser.decrementQueueRef();
browserDispatches.remove(browserDispatch);
}
to
// are we done browsing? no new messages paged
if (!added || browser.atMax()) {
browser.decrementQueueRef();
browserDispatches.remove(browserDispatch);
} else {
wakeup();
}
To confirm this was the change that fixed the issue, I went to the previous version, 5.15.0 and forced wakeup() to be called using a debugger and the call to hasMoreElements() did not hang.

After some research and try i switched to the more up-to-date JMX technology. There are no performance issues while walking through the queues.
Some code:
JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi");
JMXConnector connector = JMXConnectorFactory.connect(url, null);
connector.connect();
connection = connector.getMBeanServerConnection();
ObjectName name = new ObjectName(getObjectNameByBrokerName(brokerName));
brokerMBean = (BrokerViewMBean) MBeanServerInvocationHandler.newProxyInstance(connection, name, BrokerViewMBean.class, true);
ObjectName[] objNames = brokerMBean.getQueues();
for (ObjectName objName : objNames) {
QueueViewMBean queueMBean = (QueueViewMBean) MBeanServerInvocationHandler.newProxyInstance(connection, objName, QueueViewMBean.class, true);
System.out.println(queueMBean.getName());
}
You have to activate jmx in the configuration. It's de-activated by default.

Delete top message from MQQueue

I am constructing a messaging system using MQSeries. For some reason, when I perform q.get(...), I am getting an exception thrown (I don't know the specific MQException). Below is the code causing the error:
private static MQGetMessageOptions GMO = new MQGetMessageOptions();
private static int GMO_OPTIONS = MQC.MQGMO_SYNCPOINT | MQC.MQGMO_WAIT;
GMO.options = GMO.options | GMO_OPTIONS;
GMO.waitInterval = MQC.MQWI_UNLIMITED;
MQEnvironment.hostname = args[0];
MQEnvironment.channel = args[2];
MQEnvironment.port = Integer.parseInt(args[1]);
MQQueueManager queueManager = new MQQueueManager(args[3])
MQMessage msg = new MQMessage();
MQQueue q = queueManager.accessQueue("qName1",MQC.MQOO_OUTPUT);
q.get(msg, GMO);
My plan is, when this error occurs, skip the message and delete it. To perform the delete I will call the following function:
private void deleteMsg(MQQueueManager queueManager, String queueName) throws MQException {
MQGetMessageOptions tempGmo = new MQGetMessageOptions();
tempGmo.options |= MQC.MQGMO_WAIT;
tempGmo.waitInterval = 1000;
MQQueue remover = queueManager.accessQueue(queueName, MQC.MQOO_INPUT_AS_Q_DEF);
remover.get(new MQMessage(), tempGmo);
queueManager.commit();
}
Would the remover.get() in my deleteMsg function also, in this specific scenario, fail for the same reason? Or does the option used to construct the MQQueue(MQC.MQOO_INPUT_AS_Q_DEF vs MQC.MQOO_OUTPUT) prevent it from also failing? If I am having trouble accessing my queue's message, how do I discard the top message and move to the next?
To shorten my question:
If I am unable to perform a get() on a given queue to retrieve a message, how can we delete that corrupt message on the same queue?
Thank you!

OMG!
MQQueue q = queueManager.accessQueue("qName1",MQC.MQOO_OUTPUT);
q.get(msg, GMO);
Your are opening a queue for output (writing) but you are trying to get a message. You have your shoes on the wrong feet!! Secondly, why aren't you catching the MQException that MQ would be throwing?? The exception would have included the reason code which would have given you the exact explanation to your issue.
Here's how you should be opening the queue for reading:
try
{
int oo = MQC.MQOO_INPUT_SHARED + MQC.MQOO_FAIL_IF_QUIESCING;
MQQueue q = queueManager.accessQueue("qName1",oo);
MQGetMessageOptions gmo = new MQGetMessageOptions();
gmo.options = MQC.MQGMO_NO_WAIT + MQC.MQGMO_FAIL_IF_QUIESCING;
q.get(msg, gmo);
}
catch (MQException e)
{
System.err.println(e.getLocalizedMessage() );
System.err.println("CC = " + e.completionCode + " - RC = " + e.reasonCode);
}
Also, make sure you use the appropriate "Fail if quiescing" option for the particular MQ API call.
Finally, look up "backout queue". If your application is having an issue with a message then the message should be moved to a backout queue and not simply deleted.

I do not why what you are doing does not work for you but I wonder why are you using proprietary API of MQ Series instead of using JMS API. In JMS terms remove top message just means receive the message, so call of session.receieve() does the work.
Using common JMS API has a lot of advantages. The main of them is that you can easily move from MQ Series to any other messaging solution without changing even one line of your code.

I wonder if the program compiled because there is no option called GMO_OPTIONS. All MQ constants are prefixed MQC