Develop Reference

Spring Boot App not consuming queue messages

I have the Rabbit MQ broker for communicating asynchronously between services. Service A is sending messages to the queue. I checked the queue and the messages from Service A have arrived:
I am trying to create a listener in the Service B in order to consume the messages produced by Service A. I verified like below to check if Service B is connected with RabbitMQ and it seems to be connected successfully.
The problem is that Service B started successfully but it is receiving messages from Rabbit MQ.
Below is the implementation of the listener:
#Slf4j
#Component
public class EventListener {
public static final String QUEUE_NAME = "events";
#RabbitListener(
bindings = {
#QueueBinding(
value = #Queue(QUEUE_NAME),
exchange = #Exchange("exchange")
)
}
)
public void handleTaskPayload(#Payload String payload) {
System.out.println(payload);
}
}
I verified the queue and exchange information in the Rabbit MQ and they are correct.
Everything is working correctly and there is no error thrown in service A or service B which makes this problem much harder to debug.
I tried to retrieve the message from the queue getMessage of RabbitMQ the message is like the below:
{"id":"1",:"name:"Test","created":null}
I will appreciate any help or guidance towards the solution of this problem.
Best Regards,
Rando.
P.S
I created a new test queue like the below and published some messages:
Modified the listener code like below and still wasn't able to trigger listener to listen to the queue events:
#Slf4j
#Component
public class RobotRunEventListener {
public static final String QUEUE_NAME = "test";
#RabbitListener(
bindings = {
#QueueBinding(
value = #Queue(QUEUE_NAME),
key = "test",
exchange = #Exchange("default")
)
}
)
public void handleTaskPayload(#Payload String payload) {
System.out.println(payload);
}

Try this approach:
#RabbitListener(queues = "test")
public void receive(String in, #Headers Map<String, Object> headers) throws IOException {
}

The problem was that the spring boot app that I was working on had a #Conditional(Config.class) that prevented the creation of the bean below:
#Slf4j
#Conditional(Config.class)
#EnableRabbit
public class InternalRabbitBootstrapConfiguration {
#Bean
public RabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setMaxConcurrentConsumers(5);
return factory;
}
...
which resulted in the spring boot app not listening to Rabbit MQ events. The Config.class required a specific profile in order to enable the app to listen to Rabbit MQ events.
public class DexiModeCondition implements Condition {
#Override
public boolean matches(ConditionContext context, AnnotatedTypeMetadata metadata) {
String[] activeProfiles = context.getEnvironment().getActiveProfiles();
return activeProfiles[0].equalsIgnoreCase(mode);
}
}

ActiveMQ Consumer failed to consume message on request queue

Process:
App A is creating a consumer connection on startup to "REQUEST QUEUE"
in remote Active MQ Broker. App B is pushing request into the "REQUEST
QUEUE", and the same is consumed by App A. App A produces response
into another queue "RESPONSE QUEUE".
App A is using ActiveMQ connection factory and Spring DMLC for
consuming the message.
What I understand from Spring DMLC is, it continuously polls on queue
for message.
Problem:
App A initially consumes request on app restart. But fails to consume the request second time when an another request is made a day after. Even the consumer goes down on the arrival of the second request.
So far tried:
The issue exists only in LIVE.And as per observation consumer is not able process/receive incoming message after a certain period(30 minutes).
I tried with same broker setup in other environments, but unfortunately couldn't re-produce. I have tried every other link but all in vain.
Looking out for pointers:
Is there something fundamentally wrong with the way I'm making use of
ActiveMQ connection for both consuming and producing ?
Possible pointers on what configuration to look out for debugging the issue.
Code:
Message Poller Configuration Code:
#Configuration
#EnableJms
#ComponentScan
#Slf4j
public class MessagePoller
{
#Value("${outbound.queue}")
private String outboundQueue;
private final BrokerProperties brokerProperties;
#Autowired
public MessagePoller(BrokerProperties brokerProperties) {this.brokerProperties = brokerProperties;}
#Bean(name = CONNECTION_FACTORY)
#Primary
public ActiveMQConnectionFactory connectionFactory()
{
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(brokerProperties.getUrl());
connectionFactory.setUserName(brokerProperties.getUser());
connectionFactory.setPassword(brokerProperties.getPassword());
return connectionFactory;
}
#Bean
public JmsListenerContainerFactory jmsListenerContainerFactory()
{
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrency("1-1");
factory.setErrorHandler(getErrorHandler());
factory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
factory.setMessageConverter(new SimpleMessageConverter());
factory.setPubSubDomain(false);
return factory;
}
#Bean(name = OUTBOUND_JMS_TEMPLATE)
public JmsTemplate outboudJmsTemplate(
#Qualifier(CONNECTION_FACTORY)
ConnectionFactory connectionFactory)
{
JmsTemplate template = new JmsTemplate(connectionFactory);
template.setPubSubDomain(false);
template.setDefaultDestinationName(outboundQueue);
return template;
}
private ErrorHandler getErrorHandler()
{
return exception -> log.error("Exception thrown from consumer", exception);
}
}
Message Listener Code:
#JmsListener(
destination = "requestQueue",
containerFactory = "jmsListenerContainerFactory"
)
public void onMessage(Message<String> jmsMessage)
{
log.info("TriggerJobOnRequestService.onMessage() starts");
log.info("Incoming request message: {}", jmsMessage);
Integer deliveryCount = jmsMessage.getHeaders().get("JMSXDeliveryCount", Integer.class);
log.info("Payload : {}", jmsMessage.getPayload());
log.info("Headers : {}", jmsMessage.getHeaders());
log.info("Delivery Count : {}", deliveryCount);
//Processing Code logic
log.info("TriggerJobOnRequestService.onMessage() ends");
}
ActiveMQ connection url:
spring.activemq.url=failover://(tcp://mqueue05.net:61616,tcp://mqueue06.net:61616,tcp://mqueue07.net:61616,tcp://mqueue08.net:61616)?randomize=false&timeout=100&initialReconnectDelay=1000&maxReconnectDelay=1000&maxReconnectAttempts=10

Sending messages to two ActiveMQ brokers using springBoot & camel

I'm struggling in creating a Spring Boot application that uses Apache Camel and sends messages to two separate ActiveMQ brokers.
I've created two ActiveMQComponent beans, each is configured to a different broker. I then defined a router that gets a message from the queue in the first broker, processes it and transfers it to a queue in the second broker.
When I try to run the application I get the following error:
org.apache.camel.FailedToCreateRouteException: Failed to create route route1: Route(route1)[From[{{primary.input.queue}}] -> [Log[New mess... because of No endpoint could be found for: primary://MyInputQueue
My configuration code is:
#Bean("primary")
public ActiveMQComponent primaryAMQComponent(final ConnectionFactory connectionFactory, final
JmsTransactionManager jmsTransactionManager ) {
ActiveMQComponent component = new ActiveMQComponent();
component.setBrokerURL("tcp://localhost:61616");
component.setPreserveMessageQos(true);
component.setConnectionFactory(connectionFactory);
component.setTransactionManager(jmsTransactionManager);
return component;
}
#Bean("secondary")
public ActiveMQComponent primaryAMQComponent(final ConnectionFactory connectionFactory, final
JmsTransactionManager jmsTransactionManager ) {
ActiveMQComponent component = new ActiveMQComponent();
component.setBrokerURL("tcp://remotehost:61616");
component.setPreserveMessageQos(true);
component.setConnectionFactory(connectionFactory);
component.setTransactionManager(jmsTransactionManager);
return component;
}
My router is:
#Component
public class JmsTestRouter extends RouteBuilder {
static final Logger log = LoggerFactory.getLogger(JmsTestRouter.class);
#Override
public void configure() throws Exception {
from("{{primary.input.queue}}")
.log("New message received")
.process(exchange -> {
System.out.println("New message received - message text :" + exchange.getMessage().getBody());
String convertedMessage = exchange.getMessage().getBody() + " is processed ";
exchange.getMessage().setBody(convertedMessage);
})
.to("{{secondary.output.queue}}")
.log("Message is successfully sent to the output queue")
.end();
}
}
The properties' definition is:
primary.input.queue=primary:MyInputQueue
secondary.output.queue=secondary:MyOutputQueue
Apparently I'm doing something wrong. Any help will be appreciated.

SQS Message visibility timeout being set to 0 when exception is thrown and #JMSListener

I have a simple Spring Boot service that listens to an AWS SQS queue using JMSTemplate. Everything works as expected when the message is properly handled.
I am using CLIENT_ACKNOWLEDGE so when an exception is thrown during processing, the message is received again. However the Default Visibility Timeout setting on the SQS queue is being ignored and the message is being received again immediately.
The SQS queue is configured with a 30 second Default Visibility Timeout and a re-drive policy of 20 receives before putting the message on a DLQ.
I have disabled the service and used the SQS Console to verify that the Default Visibility Timeout is properly set. I have also tried adding the JMS Message to the method signature and performing manual validation.
Here is code for the JMS Configuration:
#Configuration
#EnableJms
class JmsConfig
{
#Bean
#Conditional(AWSEnvironmentCondition.class)
public SQSConnectionFactory connectionFactory(#Value("${AWS_REGION}") String awsRegion)
{
return new SQSConnectionFactory(
new ProviderConfiguration(),
AmazonSQSClientBuilder.standard()
.withRegion(Regions.fromName(awsRegion))
.withCredentials(new DefaultAWSCredentialsProviderChain())
);
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(ConnectionFactory connectionFactory)
{
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setDestinationResolver(new DynamicDestinationResolver());
factory.setConcurrency("3-10");
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
factory.setErrorHandler(defaultErrorHandler());
return factory;
}
#Bean
public ErrorHandler defaultErrorHandler()
{
return new ErrorHandler()
{
#Override
public void handleError(Throwable throwable)
{
LOG.error("JMS message listener error: {}", throwable.getMessage());
}
};
}
#Bean
public JmsTemplate defaultJmsTemplate(ConnectionFactory connectionFactory)
{
return new JmsTemplate(connectionFactory);
}
}
And here is code for the Listener:
#Component
public class MessagingListener
{
#Autowired
private MessageService _messageService;
#Autowired
private Validator _validator;
#JmsListener(destination = "myqueue")
public void receiveMessage(String messageJson)
{
try
{
LOG.info("Received message");
// The following line throws an IOException is the message is not JSON.
MyMessage myMessage = MAPPER.readvalue(messageJson, MyMessage.class);
Set<ConstraintViolation<MyMessage>> _validator.validate(myMessage);
if (CollectionUtils.isNotEmpty(violations))
{
String errorMessage = violations.stream()
.map(v -> String.join(" : ", v.getPropertyPath().iterator().next().getName(),
v.getMessage()))
LOG.error("Exception occurred while validating the model, details: {}", errorMessage)
throw new ValidationException(errorMessage);
}
}
catch (IOException e)
{
LOG.error("Error parsing message", e);
throw new ValidationException("Error parsing message, details: " + e.getMessage());
}
}
}
When a message is placed on the SQS queue with either invalid JSON or JSON that that does not pass validation, the message is received 20 times very quickly and then ends up on the DLQ. What needs to be done so that the Default Visibility Timeout setting in SQS is respected?

In case of an exception, visibility timeout of the failed message is set to 0 via ChangeMessageVisibility so SQS will send this message immediately even though queue has a different visibilityTimeout setting.
How does that happen?
As you can see here, Spring JMS' AbstractMessageListenerContainer briefly does this:
try {
invokeListener(session, message); // This is your #JMSListener method
}
catch (JMSException | RuntimeException | Error ex) {
rollbackOnExceptionIfNecessary(session, ex);
throw ex;
}
commitIfNecessary(session, message);
On rollbackOnExceptionIfNecessary method, session.recover() will be invoked because:
session.getTransacted() will always be false since SQS does not support transactions. See here.
isClientAcknowledge(session) will return true because you're using CLIENT_ACKNOWLEDGE mode.
And lastly recover() of SQSSession negative acknowledges the message, which means setting visibilityTimeout of that specific message to 0, causes SQS to try sending that message immediately.
The easiest way to override this behavior would be implementing a CustomJmsListenerContainerFactory & CustomMessageListenerContainer instead of using DefaultJmsListenerContainerFactory & DefaultMessageListenerContainer.
public class CustomMessageListenerContainer extends DefaultMessageListenerContainer {
public CustomMessageListenerContainer() {
super();
}
#Override
protected void rollbackOnExceptionIfNecessary() {
// do nothing, so that "visibilityTimeout" will stay same
}
}
public class CustomJmsListenerContainerFactory {
#Override
protected DefaultMessageListenerContainer createContainerInstance() {
return new CustomMesageListenerContainer();
}
}
And make it a Spring bean either with #Component or just like you did in JmsConfig:
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(ConnectionFactory connectionFactory) {
DefaultJmsListenerContainerFactory factory = new CustomJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
// and set other stuff on factory
return factory;
}
NOTE:
If your application is consuming other type of data sources along SQS with JMS, make sure to use different Container and ContainerFactory for them so that rollbackOnExceptionIfNecessary behaves as expected.

How to listen to an existing queue in Spring AMQP?

I have a remote RabbitMQ server which has some queues I want to listen to. I tried this:
#RabbitListener(queues = "queueName")
public void receive(String message) {
System.out.println(message);
}
But it tried to create a new queue. Result is predictable - access denied.
o.s.a.r.listener.BlockingQueueConsumer : Failed to declare queue: queueName
I didn't declare any queue in any other way.
How can I listen to an existing queue on a remote server? Also, is there a way to check if this queue exists? And I saw this line
#RabbitListener(queues = "#{autoDeleteQueue2.name}")
in a tutorial. What does #{queueName.name} mean?
Logs and the beginning of the stack trace:
2018-08-30 22:10:21.968 WARN 12124 --- [cTaskExecutor-1] o.s.a.r.listener.BlockingQueueConsumer : Failed to declare queue: queueName
2018-08-30 22:10:21.991 WARN 12124 --- [cTaskExecutor-1] o.s.a.r.listener.BlockingQueueConsumer : Queue declaration failed; retries left=3
org.springframework.amqp.rabbit.listener.BlockingQueueConsumer$DeclarationException: Failed to declare queue(s):[queueName]
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.attemptPassiveDeclarations(BlockingQueueConsumer.java:711) ~[spring-rabbit-2.0.5.RELEASE.jar:2.0.5.RELEASE]
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:588) ~[spring-rabbit-2.0.5.RELEASE.jar:2.0.5.RELEASE]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:996) [spring-rabbit-2.0.5.RELEASE.jar:2.0.5.RELEASE]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]

Even if you don't have configuration permission on the broker, the queueDeclarePassive used by the listener is allowed (it checks for the presence of the queue).
o.s.a.r.listener.BlockingQueueConsumer : Failed to declare queue: queueName
That just means that the queue doesn't exist.
#RabbitListener(queues = "#{autoDeleteQueue2.name}")
That is used to get the queue name at runtime (when you have permission to create queues).
e.g.
#Bean
public AnonymousQueue autoDeleteQueue2() {
return new AnonymousQueue();
}
Spring will add that queue to the broker with a random, unique name. The listener is then configured with the actual queue name.

Here is an example on how to listen to a queue with rabbitMq :
#Component
public class RabbitConsumer implements MessageListener {
#RabbitListener(bindings =
#QueueBinding(
value = #Queue(value = "${queue.topic}", durable = "true"),
exchange = #Exchange(value = "${queue.exchange}", type = ExchangeTypes.FANOUT, durable = "true")
)
)
#Override
public void onMessage(Message message) {
// ...
}
}
And the config (application.yaml) :
queue:
topic: mytopic
exchange: myexchange
In rabbitmq, consumer are associated with exchanges. It allow you to define how the messages must be consumed (are all consumer listen to all message ? Is this enought if only one consumer read the message ? ...)

Here is an example of how to listen to a specific 'queue' using Spring Integration:
SpringIntegrationConfiguration.java
#Configuration
public class SpringIntegrationConfiguration {
#Value("${rabbitmq.queueName}")
private String queueName;
#Bean
public IntegrationFlow ampqInbound(ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Amqp.inboundAdapter(connectionFactory, queueName))
.handle(System.out::println)
.get();
}
}
ApplicationConfiguration.java
#Configuration
public class ApplicationConfiguration {
#Value("${rabbitmq.topicExchangeName}")
private String topicExchangeName;
#Value("${rabbitmq.queueName}")
private String queueName;
#Value("${rabbitmq.routingKey}")
private String routingKey;
#Bean
Queue queue() {
return new Queue(queueName, false);
}
#Bean
TopicExchange exchange() {
return new TopicExchange(topicExchangeName);
}
#Bean
Binding binding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(routingKey);
}
}
Application.yml
rabbitmq:
topicExchangeName: spring-boot-exchange
queueName: spring-boot
routingKey: foo.bar.#