Develop Reference

ActiveMQ Consumer failed to consume message on request queue

Process:
App A is creating a consumer connection on startup to "REQUEST QUEUE"
in remote Active MQ Broker. App B is pushing request into the "REQUEST
QUEUE", and the same is consumed by App A. App A produces response
into another queue "RESPONSE QUEUE".
App A is using ActiveMQ connection factory and Spring DMLC for
consuming the message.
What I understand from Spring DMLC is, it continuously polls on queue
for message.
Problem:
App A initially consumes request on app restart. But fails to consume the request second time when an another request is made a day after. Even the consumer goes down on the arrival of the second request.
So far tried:
The issue exists only in LIVE.And as per observation consumer is not able process/receive incoming message after a certain period(30 minutes).
I tried with same broker setup in other environments, but unfortunately couldn't re-produce. I have tried every other link but all in vain.
Looking out for pointers:
Is there something fundamentally wrong with the way I'm making use of
ActiveMQ connection for both consuming and producing ?
Possible pointers on what configuration to look out for debugging the issue.
Code:
Message Poller Configuration Code:
#Configuration
#EnableJms
#ComponentScan
#Slf4j
public class MessagePoller
{
#Value("${outbound.queue}")
private String outboundQueue;
private final BrokerProperties brokerProperties;
#Autowired
public MessagePoller(BrokerProperties brokerProperties) {this.brokerProperties = brokerProperties;}
#Bean(name = CONNECTION_FACTORY)
#Primary
public ActiveMQConnectionFactory connectionFactory()
{
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(brokerProperties.getUrl());
connectionFactory.setUserName(brokerProperties.getUser());
connectionFactory.setPassword(brokerProperties.getPassword());
return connectionFactory;
}
#Bean
public JmsListenerContainerFactory jmsListenerContainerFactory()
{
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrency("1-1");
factory.setErrorHandler(getErrorHandler());
factory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
factory.setMessageConverter(new SimpleMessageConverter());
factory.setPubSubDomain(false);
return factory;
}
#Bean(name = OUTBOUND_JMS_TEMPLATE)
public JmsTemplate outboudJmsTemplate(
#Qualifier(CONNECTION_FACTORY)
ConnectionFactory connectionFactory)
{
JmsTemplate template = new JmsTemplate(connectionFactory);
template.setPubSubDomain(false);
template.setDefaultDestinationName(outboundQueue);
return template;
}
private ErrorHandler getErrorHandler()
{
return exception -> log.error("Exception thrown from consumer", exception);
}
}
Message Listener Code:
#JmsListener(
destination = "requestQueue",
containerFactory = "jmsListenerContainerFactory"
)
public void onMessage(Message<String> jmsMessage)
{
log.info("TriggerJobOnRequestService.onMessage() starts");
log.info("Incoming request message: {}", jmsMessage);
Integer deliveryCount = jmsMessage.getHeaders().get("JMSXDeliveryCount", Integer.class);
log.info("Payload : {}", jmsMessage.getPayload());
log.info("Headers : {}", jmsMessage.getHeaders());
log.info("Delivery Count : {}", deliveryCount);
//Processing Code logic
log.info("TriggerJobOnRequestService.onMessage() ends");
}
ActiveMQ connection url:
spring.activemq.url=failover://(tcp://mqueue05.net:61616,tcp://mqueue06.net:61616,tcp://mqueue07.net:61616,tcp://mqueue08.net:61616)?randomize=false&timeout=100&initialReconnectDelay=1000&maxReconnectDelay=1000&maxReconnectAttempts=10

How to listen to an existing queue in Spring AMQP?

I have a remote RabbitMQ server which has some queues I want to listen to. I tried this:
#RabbitListener(queues = "queueName")
public void receive(String message) {
System.out.println(message);
}
But it tried to create a new queue. Result is predictable - access denied.
o.s.a.r.listener.BlockingQueueConsumer : Failed to declare queue: queueName
I didn't declare any queue in any other way.
How can I listen to an existing queue on a remote server? Also, is there a way to check if this queue exists? And I saw this line
#RabbitListener(queues = "#{autoDeleteQueue2.name}")
in a tutorial. What does #{queueName.name} mean?
Logs and the beginning of the stack trace:
2018-08-30 22:10:21.968 WARN 12124 --- [cTaskExecutor-1] o.s.a.r.listener.BlockingQueueConsumer : Failed to declare queue: queueName
2018-08-30 22:10:21.991 WARN 12124 --- [cTaskExecutor-1] o.s.a.r.listener.BlockingQueueConsumer : Queue declaration failed; retries left=3
org.springframework.amqp.rabbit.listener.BlockingQueueConsumer$DeclarationException: Failed to declare queue(s):[queueName]
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.attemptPassiveDeclarations(BlockingQueueConsumer.java:711) ~[spring-rabbit-2.0.5.RELEASE.jar:2.0.5.RELEASE]
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:588) ~[spring-rabbit-2.0.5.RELEASE.jar:2.0.5.RELEASE]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:996) [spring-rabbit-2.0.5.RELEASE.jar:2.0.5.RELEASE]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]

Even if you don't have configuration permission on the broker, the queueDeclarePassive used by the listener is allowed (it checks for the presence of the queue).
o.s.a.r.listener.BlockingQueueConsumer : Failed to declare queue: queueName
That just means that the queue doesn't exist.
#RabbitListener(queues = "#{autoDeleteQueue2.name}")
That is used to get the queue name at runtime (when you have permission to create queues).
e.g.
#Bean
public AnonymousQueue autoDeleteQueue2() {
return new AnonymousQueue();
}
Spring will add that queue to the broker with a random, unique name. The listener is then configured with the actual queue name.

Here is an example on how to listen to a queue with rabbitMq :
#Component
public class RabbitConsumer implements MessageListener {
#RabbitListener(bindings =
#QueueBinding(
value = #Queue(value = "${queue.topic}", durable = "true"),
exchange = #Exchange(value = "${queue.exchange}", type = ExchangeTypes.FANOUT, durable = "true")
)
)
#Override
public void onMessage(Message message) {
// ...
}
}
And the config (application.yaml) :
queue:
topic: mytopic
exchange: myexchange
In rabbitmq, consumer are associated with exchanges. It allow you to define how the messages must be consumed (are all consumer listen to all message ? Is this enought if only one consumer read the message ? ...)

Here is an example of how to listen to a specific 'queue' using Spring Integration:
SpringIntegrationConfiguration.java
#Configuration
public class SpringIntegrationConfiguration {
#Value("${rabbitmq.queueName}")
private String queueName;
#Bean
public IntegrationFlow ampqInbound(ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Amqp.inboundAdapter(connectionFactory, queueName))
.handle(System.out::println)
.get();
}
}
ApplicationConfiguration.java
#Configuration
public class ApplicationConfiguration {
#Value("${rabbitmq.topicExchangeName}")
private String topicExchangeName;
#Value("${rabbitmq.queueName}")
private String queueName;
#Value("${rabbitmq.routingKey}")
private String routingKey;
#Bean
Queue queue() {
return new Queue(queueName, false);
}
#Bean
TopicExchange exchange() {
return new TopicExchange(topicExchangeName);
}
#Bean
Binding binding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(routingKey);
}
}
Application.yml
rabbitmq:
topicExchangeName: spring-boot-exchange
queueName: spring-boot
routingKey: foo.bar.#

Apache Camel ?transacted=true

Hi Camel/jms developers.
Using Apache Camel amqp jms connector. And as a Broker ActiveMQ.
My configuration is quite default.
Here is a consumer code example:
public static void main(String[] args) throws Exception {
AMQPComponent amqpComponent = AMQPComponent.amqpComponent(HOST, USER, PWD);
CamelContext context = new DefaultCamelContext();
context.addComponent("amqp", amqpComponent);
context.start();
context.addRoutes(new RouteBuilder() {
#Override
public void configure() {
from("amqp:queue:1test.queue?transacted=true")
.to("stream:out")
.end();
}
});
Thread.sleep(20*1000);
context.stop();
}
Easy to see, I have configured transacted consumer. for 1test.queue.
When Im running it, in log see:
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route1 started and consuming from: amqp://queue:1test.queue?transacted=true
[AmqpProvider :(1):[amqp:HOST2]] INFO org.apache.qpid.jms.sasl.SaslMechanismFinder - Best match for SASL auth was: SASL-PLAIN
[AmqpProvider :(1):[amqp:HOST2]] INFO org.apache.qpid.jms.JmsConnection - Connection ID:...:1 connected to remote Broker: amqp:HOST2
[AmqpProvider :(2):[amqp:HOST2]] INFO org.apache.qpid.jms.sasl.SaslMechanismFinder - Best match for SASL auth was: SASL-PLAIN
[AmqpProvider :(2):[amqp:HOST2]] INFO org.apache.qpid.jms.JmsConnection - Connection ID:...:2 connected to remote Broker: amqp:HOST2
[AmqpProvider :(3):[amqp:HOST2]] INFO org.apache.qpid.jms.sasl.SaslMechanismFinder - Best match for SASL auth was: SASL-PLAIN
[AmqpProvider :(3):[amqp:HOST2]] INFO org.apache.qpid.jms.JmsConnection - Connection ID:...:3 connected to remote Broker: amqp:HOST2
If I removing ?transacted=true from consumer
[AmqpProvider :(1):[amqp:HOST2]] INFO org.apache.qpid.jms.sasl.SaslMechanismFinder - Best match for SASL auth was: SASL-PLAIN
[AmqpProvider :(1):[amqp:HOST2]] INFO org.apache.qpid.jms.JmsConnection - Connection ID:...:1 connected to remote Broker: amqp:HOST2
It appears only once.
How to explain this behavior? This is normally for transacted consumers in camel?
Thank you.
P.S Checked this topic but not sure how to map it to Camel reality.

AMQP Camel Component has not defined a pooled connection factory so it is creating a new connection for each iteration to check if there are messages in the broker.
To avoid it you should define a CachingConnectionFactory to the AMQPComponent as:
import org.apache.camel.component.amqp.AMQPComponent;
import org.apache.camel.component.jms.JmsConfiguration;
import org.apache.qpid.jms.JmsConnectionFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Primary;
import org.springframework.jms.connection.CachingConnectionFactory;
#Bean
public JmsConnectionFactory jmsConnectionFactory() {
JmsConnectionFactory jmsConnectionFactory = new JmsConnectionFactory(brokerUser, brokerPassword, brokerUrl);
return jmsConnectionFactory;
}
#Bean
#Primary
public CachingConnectionFactory jmsCachingConnectionFactory(JmsConnectionFactory jmsConnectionFactory) {
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory(jmsConnectionFactory);
return cachingConnectionFactory;
}
#Bean
public JmsConfiguration jmsConfig(CachingConnectionFactory cachingConnectionFactory) {
JmsConfiguration jmsConfiguration = new JmsConfiguration();
jmsConfiguration.setConnectionFactory(cachingConnectionFactory);
jmsConfiguration.setCacheLevelName("CACHE_CONSUMER");
return jmsConfiguration;
}
#Bean
public AMQPComponent amqpComponent(JmsConfiguration jmsConfiguration) {
AMQPComponent amqpComponent = new AMQPComponent();
amqpComponent.setConfiguration(jmsConfiguration);
return amqpComponent;
}
You will get the same behavior as JMS Camel Component.
More info at AMQP Camel Component page

Use jms insted of amqp.
I had similar problem. But when I use JMS instead of AMQP it worked well there was log only one time i.e single connection got created.
Seems some issue is there with AMQ component.
Thanks,
Rahul

Load Balancing Client Sessions

I have a 2 server ActiveMQ Artemis cluster setup with discovery using JGroups working correctly. Then in my applications the ConnectionFactory is created via the ActiveMQJMSClient:
final ActiveMQConnectionFactory cf = ActiveMQJMSClient.createConnectionFactory(
"jgroups://test-cluster?file=jgroups-file-ping.xml&connectionLoadBalancingPolicyClassName=org.apache.activemq.artemis.api.core.client.loadbalance.RoundRobinConnectionLoadBalancingPolicy", "test-cluster");
Then the producers and consumers are handled with Spring. However, when Spring creates 10 consumers on start-up I'm seeing that all 10 consumers go to the same Artemis server.
Here is the Spring JMS Configuration:
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() throws Exception {
final ConnectionFactory cf = getActiveMQConnectionFactory();
final DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(cf);
factory.setConcurrency("10-20");
factory.setSessionTransacted(true);
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
return factory;
}
Then on application startup I call this within the JmsListenerConfigurer:
public static void registerEndPoint(final JmsListenerEndpointRegistrar registrar,
String endPointName, String dest, MessageListener listener) {
final SimpleJmsListenerEndpoint endpoint = new SimpleJmsListenerEndpoint();
endpoint.setId(endPointName);
endpoint.setDestination(dest);
endpoint.setMessageListener(listener);
registrar.registerEndpoint(endpoint);
}
Is there a way of setting this up so that there are 5 consumers on each Artemis server?

Spring's DefaultJmsListenerContainerFactory has a cache level that can be changed to CACHE_NONE,CACHE_CONNECTION,CACHE_SESSION or CACHE_AUTO.
I had to set it to CACHE_NONE for Spring to create more than a single connection in the connection factory.

Create durable topic and subscriber spring boot jms with ActiveMQ

I need to create a topic and a durable subscriber for ActiveMQ, my problem is that I don't know where to specify that. I am able to create the topic and consume the messages, but when I turn off the subscriber then keep sending messages and turn on the subscriber again, it won't read them.
This is what I have so far:
Sending the message :
JmsTemplate jmsTemplate = context.getBean(JmsTemplate.class);
jmsTemplate.setPubSubDomain(true);
jmsTemplate.setDeliveryMode(DeliveryMode.PERSISTENT);
jmsTemplate.setDeliveryPersistent(true);
jmsTemplate.convertAndSend("venta.topic",venta);
Receiving the message :
#JmsListener(destination = "venta.topic",id = "comercial",subscription = "venta.topic")
public void receiveMessage(Venta venta) {
logger.log(Level.INFO, "RECEIVED : {0}",venta);
repository.save(venta);
}
I have read this article and I understand that I need to create the durable subscriber.
I've also read the spring docs
And I think it has something to do with the DefaultJmsListenerContainerFactory (which I didn't implement, I am using the default configuration), the docs shows:
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setDestinationResolver(destinationResolver());
factory.setConcurrency("3-10");
return factory;
}
But I can't seem to find where to create the durable session. Both my producer and my subscriber are connected to a standalone activemq binary.
I hope you can help me , thanks in advance.

As the previous answers pointed out, it was necessary to set the client id and the durable subscription on the factory :
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setDestinationResolver(destinationResolver());
factory.setConcurrency("3-10");
factory.setClientID("brokerClientId");
factory.setSubscriptionDurable(true);
return factory;
}
but that alone did not register the client as a durable subscriber, that is because the JMSListener needed the containerFactory specified, otherwise it would just take the defaults :
#JmsListener(
destination = "venta.topic",
id = "comercial",
subscription = "venta.topic",
//this was also needed with the same name as the bean above
containerFactory = "jmsListenerContainerFactory"
)
public void receiveMessage(Venta venta) {
logger.log(Level.INFO, "RECEIVED : {0}",venta);
repository.save(venta);
}
It's worth mentioning that, this post was the one that made me realize my mistake.
I Hope this will help someone else

The DefaultJmsListenerContainerFactory should have unique clientId and durable sub. true set like below code :
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setDestinationResolver(destinationResolver());
factory.setConcurrency("3-10");
factory.setClientID("brokerClientId");
factory.setSubscriptionDurable(true);
return factory;
}

Hard to tell for sure, but a common cause to this problem is to forget configuring a unique clientId on the connectionFactory bean. It has to be unique and is the way the broker can tell each client apart.