Develop Reference

ActiveMQ Consumer failed to consume message on request queue

Process:
App A is creating a consumer connection on startup to "REQUEST QUEUE"
in remote Active MQ Broker. App B is pushing request into the "REQUEST
QUEUE", and the same is consumed by App A. App A produces response
into another queue "RESPONSE QUEUE".
App A is using ActiveMQ connection factory and Spring DMLC for
consuming the message.
What I understand from Spring DMLC is, it continuously polls on queue
for message.
Problem:
App A initially consumes request on app restart. But fails to consume the request second time when an another request is made a day after. Even the consumer goes down on the arrival of the second request.
So far tried:
The issue exists only in LIVE.And as per observation consumer is not able process/receive incoming message after a certain period(30 minutes).
I tried with same broker setup in other environments, but unfortunately couldn't re-produce. I have tried every other link but all in vain.
Looking out for pointers:
Is there something fundamentally wrong with the way I'm making use of
ActiveMQ connection for both consuming and producing ?
Possible pointers on what configuration to look out for debugging the issue.
Code:
Message Poller Configuration Code:
#Configuration
#EnableJms
#ComponentScan
#Slf4j
public class MessagePoller
{
#Value("${outbound.queue}")
private String outboundQueue;
private final BrokerProperties brokerProperties;
#Autowired
public MessagePoller(BrokerProperties brokerProperties) {this.brokerProperties = brokerProperties;}
#Bean(name = CONNECTION_FACTORY)
#Primary
public ActiveMQConnectionFactory connectionFactory()
{
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(brokerProperties.getUrl());
connectionFactory.setUserName(brokerProperties.getUser());
connectionFactory.setPassword(brokerProperties.getPassword());
return connectionFactory;
}
#Bean
public JmsListenerContainerFactory jmsListenerContainerFactory()
{
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrency("1-1");
factory.setErrorHandler(getErrorHandler());
factory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
factory.setMessageConverter(new SimpleMessageConverter());
factory.setPubSubDomain(false);
return factory;
}
#Bean(name = OUTBOUND_JMS_TEMPLATE)
public JmsTemplate outboudJmsTemplate(
#Qualifier(CONNECTION_FACTORY)
ConnectionFactory connectionFactory)
{
JmsTemplate template = new JmsTemplate(connectionFactory);
template.setPubSubDomain(false);
template.setDefaultDestinationName(outboundQueue);
return template;
}
private ErrorHandler getErrorHandler()
{
return exception -> log.error("Exception thrown from consumer", exception);
}
}
Message Listener Code:
#JmsListener(
destination = "requestQueue",
containerFactory = "jmsListenerContainerFactory"
)
public void onMessage(Message<String> jmsMessage)
{
log.info("TriggerJobOnRequestService.onMessage() starts");
log.info("Incoming request message: {}", jmsMessage);
Integer deliveryCount = jmsMessage.getHeaders().get("JMSXDeliveryCount", Integer.class);
log.info("Payload : {}", jmsMessage.getPayload());
log.info("Headers : {}", jmsMessage.getHeaders());
log.info("Delivery Count : {}", deliveryCount);
//Processing Code logic
log.info("TriggerJobOnRequestService.onMessage() ends");
}
ActiveMQ connection url:
spring.activemq.url=failover://(tcp://mqueue05.net:61616,tcp://mqueue06.net:61616,tcp://mqueue07.net:61616,tcp://mqueue08.net:61616)?randomize=false&timeout=100&initialReconnectDelay=1000&maxReconnectDelay=1000&maxReconnectAttempts=10

Load Balancing Client Sessions

I have a 2 server ActiveMQ Artemis cluster setup with discovery using JGroups working correctly. Then in my applications the ConnectionFactory is created via the ActiveMQJMSClient:
final ActiveMQConnectionFactory cf = ActiveMQJMSClient.createConnectionFactory(
"jgroups://test-cluster?file=jgroups-file-ping.xml&connectionLoadBalancingPolicyClassName=org.apache.activemq.artemis.api.core.client.loadbalance.RoundRobinConnectionLoadBalancingPolicy", "test-cluster");
Then the producers and consumers are handled with Spring. However, when Spring creates 10 consumers on start-up I'm seeing that all 10 consumers go to the same Artemis server.
Here is the Spring JMS Configuration:
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() throws Exception {
final ConnectionFactory cf = getActiveMQConnectionFactory();
final DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(cf);
factory.setConcurrency("10-20");
factory.setSessionTransacted(true);
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
return factory;
}
Then on application startup I call this within the JmsListenerConfigurer:
public static void registerEndPoint(final JmsListenerEndpointRegistrar registrar,
String endPointName, String dest, MessageListener listener) {
final SimpleJmsListenerEndpoint endpoint = new SimpleJmsListenerEndpoint();
endpoint.setId(endPointName);
endpoint.setDestination(dest);
endpoint.setMessageListener(listener);
registrar.registerEndpoint(endpoint);
}
Is there a way of setting this up so that there are 5 consumers on each Artemis server?

Spring's DefaultJmsListenerContainerFactory has a cache level that can be changed to CACHE_NONE,CACHE_CONNECTION,CACHE_SESSION or CACHE_AUTO.
I had to set it to CACHE_NONE for Spring to create more than a single connection in the connection factory.

Spring RabbitTemplate setRetryTemplate & setRecoveryCallback ignored

Question: After a fixed amount of retries I would like to send the message to an error queue and consume it from its original queue. I want to find a generic solution because I handle a lot of different messages and depending on the exception they raise I want to act differently. How can I set the RecoveryCallback so that Spring triggers it after maxRetries?
What I currently do
I try to set a RetryTemplate and a RecoveryCallback.
When I run the application and publish a message to the test queue I expect the processing in EListener#receive to fail 3 times and then trigger my configured RecoveryCallback which then routes the message based on the context to a specific error queue.
What actually happens
What actually happens is that Spring Boot initializes the RabbitAdmin object with its own RabbitTemplate and therefore does not use my configured RabbitTemplate bean.
I have the following directory structure:
rabbit
|___ EListener.java
|___ Rabbit.java
test
|___ Test.java
I have the following code in Rabbit.java
#Configuration
public class Rabbit {
#Autowired
ConnectionFactory connectionFactory;
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setRetryTemplate(createRetryTemplate());
rabbitTemplate.setRecoveryCallback(createRecoveryCallback());
return rabbitTemplate;
}
createRecoveryCallback() // Omitted for brevity
createRetryTemplate() // Omitted for brevity
}
The EListener.java file contains:
#Component
public class EListener {
#Autowired
RabbitTemplate rabbitTemplate;
#RabbitListener(bindings = #QueueBinding(value = #Queue(value = "test", durable = "true"), exchange = #Exchange(value = "test", type = ExchangeTypes.TOPIC, durable = "true", autoDelete = "true"), key = "test"))
public void receive(Message m) throws Exception {
System.out.println(m);
throw new Exception();
}
}
Test.java contains:
#SpringBootApplication
#ComponentScan("rabbit")
public class Test {
public static void main(String[] args) {
new SpringApplicationBuilder(Test.class).application().run(args);
}
}

Adding a RetryTemplate to the RabbitTemplate is to retry publishing messages.
To add retry on the consumer side, you have to add a retry interceptor to the listener container's advice chain.
Since you are using #RabbitListener, the advice chain goes on the listener container factory #Bean, which means you'll have to declare one yourself instead of relying on the default one created by boot.
Stateless retry does the retries in-memory; statefull retry requeues the message; it requires a messageId property (or some other mechanism to uniquely identify messages).

Spring JMS with Atomikos, poolsize and concurrent consumers

Can someone explain the semantic differences between the poolsize of a Atomikos connection factory, the pool size of the nested connection factory and concurrent consumers of a JMS Container?
Consider the following Spring config:
// ActiveMq vendor-specific Connection Factory
#Bean
public ConnectionFactory activeMQXAConnectionFactoryBean() {
ActiveMQXAConnectionFactory xaCF = new ActiveMQXAConnectionFactory(brokerUrl);
xaCF.setMaxThreadPoolSize(10);
return xaCF;
}
// Wrap it in an Atomikos ConnectionFactory
#Bean(initMethod = "init", destroyMethod = "close")
public ConnectionFactory atomikosConnectionFactoryBean(ConnectionFactory activeMQXAConnectionFactoryBean) {
AtomikosConnectionFactoryBean atomikosCF = new AtomikosConnectionFactoryBean();
atomikosCF.setUniqueResourceName("amq1");
atomikosCF.setXaConnectionFactory(xaConnectionFactory);
atomikosCF.setPoolSize(10);
return atomikosCF;
}
// wire it into Spring's MessageListenerContainer
#Bean
public MessageListenerContainer defaultMessageListenerContainer (ConnectionFactory atomikosConnectionFactoryBean) {
DefaultMessageListenerContainer mlc = new DefaultMessageListenerContainer();
mlc.setConnectionFactory(atomikosConnectionFactoryBean);
mlc.setConcurrency("3-10");
return mlc;
}
Can someone explain how the three settings affect JMS connections? For example should the poolsize of the connection factories here always be the same? What happens if they are not the same?

Load spring context even if rabbitmq is down

Is it possible to load spring context with #RabbitListener even if RabbitMQ message broker is down?
The behavior should be similar as in case of broker disconnection. Application is waiting for broker and when it is restored, then listerner is automaticaly reconnected.
Spring Boot 1.3.2.RELEASE
GitHub demo project
spring-amqp configuration:
#Bean
public MessageConverter jsonMessageConverter() {
return new JsonMessageConverter();
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(host);
connectionFactory.setUsername(username);
connectionFactory.setPassword(password);
return connectionFactory;
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrentConsumers(10);
factory.setMaxConcurrentConsumers(10);
factory.setMessageConverter(jsonMessageConverter());
return factory;
}
#Bean
public AmqpAdmin amqpAdmin() {
RabbitAdmin rabbitAdmin = new RabbitAdmin(connectionFactory());
rabbitAdmin.setIgnoreDeclarationExceptions(true); // useless
return rabbitAdmin;
}
listener configuration:
#Service
public class CalculatorServiceV2 {
#RabbitListener(bindings = #QueueBinding(
value = #Queue(value = "calc_service_v2.multiply", durable = "false", autoDelete = "true"),
exchange = #Exchange(value = "calc_service_v2", durable = "false", autoDelete = "true"),
key = "multiply"))
public Result multiply(Operands operands) {
// do something..
}
}

What version of Spring AMQP are you using?
Can you provide a stack trace? (edit the question, don't try to put it in a comment).
I just ran a test and it works as expected; the listener container attempts to reconnect every 5 seconds (the default recovery interval).
rabbitAdmin.setIgnoreDeclarationExceptions(true); // useless
In 1.6 (currently at milestone 1) we've changed that boolean to skip all exceptions; previously it only applied to exceptions caused by errors returned from the broker.
However, the admin won't attempt to declare the elements until the connection is established so that should be moot in this context.
I need to see a stack trace to understand what's going on in your case.