Develop Reference

Spring Boot App not consuming queue messages

I have the Rabbit MQ broker for communicating asynchronously between services. Service A is sending messages to the queue. I checked the queue and the messages from Service A have arrived:
I am trying to create a listener in the Service B in order to consume the messages produced by Service A. I verified like below to check if Service B is connected with RabbitMQ and it seems to be connected successfully.
The problem is that Service B started successfully but it is receiving messages from Rabbit MQ.
Below is the implementation of the listener:
#Slf4j
#Component
public class EventListener {
public static final String QUEUE_NAME = "events";
#RabbitListener(
bindings = {
#QueueBinding(
value = #Queue(QUEUE_NAME),
exchange = #Exchange("exchange")
)
}
)
public void handleTaskPayload(#Payload String payload) {
System.out.println(payload);
}
}
I verified the queue and exchange information in the Rabbit MQ and they are correct.
Everything is working correctly and there is no error thrown in service A or service B which makes this problem much harder to debug.
I tried to retrieve the message from the queue getMessage of RabbitMQ the message is like the below:
{"id":"1",:"name:"Test","created":null}
I will appreciate any help or guidance towards the solution of this problem.
Best Regards,
Rando.
P.S
I created a new test queue like the below and published some messages:
Modified the listener code like below and still wasn't able to trigger listener to listen to the queue events:
#Slf4j
#Component
public class RobotRunEventListener {
public static final String QUEUE_NAME = "test";
#RabbitListener(
bindings = {
#QueueBinding(
value = #Queue(QUEUE_NAME),
key = "test",
exchange = #Exchange("default")
)
}
)
public void handleTaskPayload(#Payload String payload) {
System.out.println(payload);
}

Try this approach:
#RabbitListener(queues = "test")
public void receive(String in, #Headers Map<String, Object> headers) throws IOException {
}

The problem was that the spring boot app that I was working on had a #Conditional(Config.class) that prevented the creation of the bean below:
#Slf4j
#Conditional(Config.class)
#EnableRabbit
public class InternalRabbitBootstrapConfiguration {
#Bean
public RabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setMaxConcurrentConsumers(5);
return factory;
}
...
which resulted in the spring boot app not listening to Rabbit MQ events. The Config.class required a specific profile in order to enable the app to listen to Rabbit MQ events.
public class DexiModeCondition implements Condition {
#Override
public boolean matches(ConditionContext context, AnnotatedTypeMetadata metadata) {
String[] activeProfiles = context.getEnvironment().getActiveProfiles();
return activeProfiles[0].equalsIgnoreCase(mode);
}
}

Update: Spring Boot JMS static reply queue on IBM MQ Series

In my use case I need to do request-reply call to a remote system via managed queues. Using Spring Boot and IBM's MQ starter I have the problem that the application wants to create dynamic/temporary reply queues instead of using the already existing managed queue.
Configuration is set up here
#EnableJms
#Configuration
public class QueueConfiguration {
#Bean
public MQQueueConnectionFactory connectionFactory() throws JMSException {
MQQueueConnectionFactory factory = new MQQueueConnectionFactory();
factory.setTransportType(CT_WMQ); // is 1
factory.setHostName(queueProperties.getHost());
factory.setPort(queueProperties.getPort());
factory.setChannel(queueProperties.getChannel()); // combo of ${queueManager}%${channel}
return factory;
}
#Bean
public JmsMessagingTemplate messagingTemplate(ConnectionFactory connectionFactory) {
JmsMessagingTemplate jmt = new JmsMessagingTemplate(connectionFactory);
jmt.setDefaultDestinationName(queueProperties.getQueueName());
return jmt;
}
#Bean
public Jaxb2Marshaller marshaller() {
Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
marshaller.setPackagesToScan("com.foo.model");
return marshaller;
}
#Bean
public MessageConverter messageConverter(Jaxb2Marshaller marshaller) {
MarshallingMessageConverter converter = new MarshallingMessageConverter();
converter.setMarshaller(marshaller);
converter.setUnmarshaller(marshaller);
return converter;
}
}
Usage is pretty straight forward: Take the object convert and send it. Wait for response receive
and convert it.
#Component
public class ExampleSenderReceiver {
#Autowired
private JmsMessagingTemplate jmsMessagingTemplate;
#Override
#SneakyThrows
public ResponseExample sendAndReceive(RequestExample request, String correlationId) {
MessagePostProcessor mpp = message -> {
message = MessageBuilder.fromMessage(message)
.setHeader(JmsHeaders.CORRELATION_ID, correlationId)
// .setHeader(JmsHeaders.REPLY_TO, "DEV.QUEUE.3") this triggers queue creation
.build();
return message;
};
String destination = Objects.requireNonNull(jmsMessagingTemplate.getDefaultDestinationName());
return jmsMessagingTemplate.convertSendAndReceive(destination, request, ResponseExample.class, mpp);
}
I read already a lot of IBM documentation and think, I need to set the message type to "MQMT_REQUEST" but I do not find the right spot to do so.
Update
Added Spring Integration as Gary proposed and added a configuration for JmsOutboundGateway
#Bean
public MessageChannel requestChannel() {
return new DirectChannel();
}
#Bean
public QueueChannel responseChannel() {
return new QueueChannel();
}
#Bean
#ServiceActivator(inputChannel = "requestChannel" )
public JmsOutboundGateway jmsOutboundGateway( ConnectionFactory connectionFactory) {
JmsOutboundGateway gateway = new JmsOutboundGateway();
gateway.setConnectionFactory(connectionFactory);
gateway.setRequestDestinationName("REQUEST");
gateway.setReplyDestinationName("RESPONSE");
gateway.setReplyChannel(responseChannel());
gateway.setCorrelationKey("JMSCorrelationID*");
gateway.setIdleReplyContainerTimeout(2, TimeUnit.SECONDS);
return gateway;
}
And adapted my ExampleSenderReceiver class
#Autowired
#Qualifier("requestChannel")
private MessageChannel requestChannel;
#Autowired
#Qualifier("responseChannel")
private QueueChannel responseChannel;
#Override
#SneakyThrows
public ResponseExample sendAndReceive(RequestExample request, String correlationId) {
String xmlContent = "the marshalled request object";
Map<String, Object> header = new HashMap<>();
header.put(JmsHeaders.CORRELATION_ID, correlationId);
GenericMessage<String> message1 = new GenericMessage<>(xmlContent, header);
requestChannel.send(message1);
log.info("send done" );
Message<?> receive = responseChannel.receive(1500);
if(null != receive){
log.info("incoming: {}", receive.toString());
}
}
The important part is gateway.setCorrelationKey("JMSCorrelationID*");
Without that line the correlationId was not propagated correct.
Next step is re-adding MessageConverters and make it nice again.
Thank you.

The default JmsTemplate (used by the JmsMessagingTemplate) always uses a temporary reply queue. You can subclass it and override doSendAndReceive(Session session, Destination destination, MessageCreator messageCreator) to use your managed queue instead.
However, it will only work if you have one request outstanding at a time (e.g. all run on a single thread). You will also have to add code for discarding "late" arrivals by checking the correlation id.
You can use async sends instead and handle replies on a listener container and correlate the replies to the requests.
Consider using spring-integration-jms and its outbound gateway instead - it has much more flexibility in reply queue handling (and does all the correlation for you).
https://docs.spring.io/spring-integration/reference/html/jms.html#jms-outbound-gateway

You are missing the queue manager.
ibm:
mq:
queueManager: QM1
channel: chanel
connName: localhost(1414)
user: admin
password: admin

Sending messages to two ActiveMQ brokers using springBoot & camel

I'm struggling in creating a Spring Boot application that uses Apache Camel and sends messages to two separate ActiveMQ brokers.
I've created two ActiveMQComponent beans, each is configured to a different broker. I then defined a router that gets a message from the queue in the first broker, processes it and transfers it to a queue in the second broker.
When I try to run the application I get the following error:
org.apache.camel.FailedToCreateRouteException: Failed to create route route1: Route(route1)[From[{{primary.input.queue}}] -> [Log[New mess... because of No endpoint could be found for: primary://MyInputQueue
My configuration code is:
#Bean("primary")
public ActiveMQComponent primaryAMQComponent(final ConnectionFactory connectionFactory, final
JmsTransactionManager jmsTransactionManager ) {
ActiveMQComponent component = new ActiveMQComponent();
component.setBrokerURL("tcp://localhost:61616");
component.setPreserveMessageQos(true);
component.setConnectionFactory(connectionFactory);
component.setTransactionManager(jmsTransactionManager);
return component;
}
#Bean("secondary")
public ActiveMQComponent primaryAMQComponent(final ConnectionFactory connectionFactory, final
JmsTransactionManager jmsTransactionManager ) {
ActiveMQComponent component = new ActiveMQComponent();
component.setBrokerURL("tcp://remotehost:61616");
component.setPreserveMessageQos(true);
component.setConnectionFactory(connectionFactory);
component.setTransactionManager(jmsTransactionManager);
return component;
}
My router is:
#Component
public class JmsTestRouter extends RouteBuilder {
static final Logger log = LoggerFactory.getLogger(JmsTestRouter.class);
#Override
public void configure() throws Exception {
from("{{primary.input.queue}}")
.log("New message received")
.process(exchange -> {
System.out.println("New message received - message text :" + exchange.getMessage().getBody());
String convertedMessage = exchange.getMessage().getBody() + " is processed ";
exchange.getMessage().setBody(convertedMessage);
})
.to("{{secondary.output.queue}}")
.log("Message is successfully sent to the output queue")
.end();
}
}
The properties' definition is:
primary.input.queue=primary:MyInputQueue
secondary.output.queue=secondary:MyOutputQueue
Apparently I'm doing something wrong. Any help will be appreciated.

SQS Message visibility timeout being set to 0 when exception is thrown and #JMSListener

I have a simple Spring Boot service that listens to an AWS SQS queue using JMSTemplate. Everything works as expected when the message is properly handled.
I am using CLIENT_ACKNOWLEDGE so when an exception is thrown during processing, the message is received again. However the Default Visibility Timeout setting on the SQS queue is being ignored and the message is being received again immediately.
The SQS queue is configured with a 30 second Default Visibility Timeout and a re-drive policy of 20 receives before putting the message on a DLQ.
I have disabled the service and used the SQS Console to verify that the Default Visibility Timeout is properly set. I have also tried adding the JMS Message to the method signature and performing manual validation.
Here is code for the JMS Configuration:
#Configuration
#EnableJms
class JmsConfig
{
#Bean
#Conditional(AWSEnvironmentCondition.class)
public SQSConnectionFactory connectionFactory(#Value("${AWS_REGION}") String awsRegion)
{
return new SQSConnectionFactory(
new ProviderConfiguration(),
AmazonSQSClientBuilder.standard()
.withRegion(Regions.fromName(awsRegion))
.withCredentials(new DefaultAWSCredentialsProviderChain())
);
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(ConnectionFactory connectionFactory)
{
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setDestinationResolver(new DynamicDestinationResolver());
factory.setConcurrency("3-10");
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
factory.setErrorHandler(defaultErrorHandler());
return factory;
}
#Bean
public ErrorHandler defaultErrorHandler()
{
return new ErrorHandler()
{
#Override
public void handleError(Throwable throwable)
{
LOG.error("JMS message listener error: {}", throwable.getMessage());
}
};
}
#Bean
public JmsTemplate defaultJmsTemplate(ConnectionFactory connectionFactory)
{
return new JmsTemplate(connectionFactory);
}
}
And here is code for the Listener:
#Component
public class MessagingListener
{
#Autowired
private MessageService _messageService;
#Autowired
private Validator _validator;
#JmsListener(destination = "myqueue")
public void receiveMessage(String messageJson)
{
try
{
LOG.info("Received message");
// The following line throws an IOException is the message is not JSON.
MyMessage myMessage = MAPPER.readvalue(messageJson, MyMessage.class);
Set<ConstraintViolation<MyMessage>> _validator.validate(myMessage);
if (CollectionUtils.isNotEmpty(violations))
{
String errorMessage = violations.stream()
.map(v -> String.join(" : ", v.getPropertyPath().iterator().next().getName(),
v.getMessage()))
LOG.error("Exception occurred while validating the model, details: {}", errorMessage)
throw new ValidationException(errorMessage);
}
}
catch (IOException e)
{
LOG.error("Error parsing message", e);
throw new ValidationException("Error parsing message, details: " + e.getMessage());
}
}
}
When a message is placed on the SQS queue with either invalid JSON or JSON that that does not pass validation, the message is received 20 times very quickly and then ends up on the DLQ. What needs to be done so that the Default Visibility Timeout setting in SQS is respected?

In case of an exception, visibility timeout of the failed message is set to 0 via ChangeMessageVisibility so SQS will send this message immediately even though queue has a different visibilityTimeout setting.
How does that happen?
As you can see here, Spring JMS' AbstractMessageListenerContainer briefly does this:
try {
invokeListener(session, message); // This is your #JMSListener method
}
catch (JMSException | RuntimeException | Error ex) {
rollbackOnExceptionIfNecessary(session, ex);
throw ex;
}
commitIfNecessary(session, message);
On rollbackOnExceptionIfNecessary method, session.recover() will be invoked because:
session.getTransacted() will always be false since SQS does not support transactions. See here.
isClientAcknowledge(session) will return true because you're using CLIENT_ACKNOWLEDGE mode.
And lastly recover() of SQSSession negative acknowledges the message, which means setting visibilityTimeout of that specific message to 0, causes SQS to try sending that message immediately.
The easiest way to override this behavior would be implementing a CustomJmsListenerContainerFactory & CustomMessageListenerContainer instead of using DefaultJmsListenerContainerFactory & DefaultMessageListenerContainer.
public class CustomMessageListenerContainer extends DefaultMessageListenerContainer {
public CustomMessageListenerContainer() {
super();
}
#Override
protected void rollbackOnExceptionIfNecessary() {
// do nothing, so that "visibilityTimeout" will stay same
}
}
public class CustomJmsListenerContainerFactory {
#Override
protected DefaultMessageListenerContainer createContainerInstance() {
return new CustomMesageListenerContainer();
}
}
And make it a Spring bean either with #Component or just like you did in JmsConfig:
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(ConnectionFactory connectionFactory) {
DefaultJmsListenerContainerFactory factory = new CustomJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
// and set other stuff on factory
return factory;
}
NOTE:
If your application is consuming other type of data sources along SQS with JMS, make sure to use different Container and ContainerFactory for them so that rollbackOnExceptionIfNecessary behaves as expected.

Load spring context even if rabbitmq is down

Is it possible to load spring context with #RabbitListener even if RabbitMQ message broker is down?
The behavior should be similar as in case of broker disconnection. Application is waiting for broker and when it is restored, then listerner is automaticaly reconnected.
Spring Boot 1.3.2.RELEASE
GitHub demo project
spring-amqp configuration:
#Bean
public MessageConverter jsonMessageConverter() {
return new JsonMessageConverter();
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(host);
connectionFactory.setUsername(username);
connectionFactory.setPassword(password);
return connectionFactory;
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrentConsumers(10);
factory.setMaxConcurrentConsumers(10);
factory.setMessageConverter(jsonMessageConverter());
return factory;
}
#Bean
public AmqpAdmin amqpAdmin() {
RabbitAdmin rabbitAdmin = new RabbitAdmin(connectionFactory());
rabbitAdmin.setIgnoreDeclarationExceptions(true); // useless
return rabbitAdmin;
}
listener configuration:
#Service
public class CalculatorServiceV2 {
#RabbitListener(bindings = #QueueBinding(
value = #Queue(value = "calc_service_v2.multiply", durable = "false", autoDelete = "true"),
exchange = #Exchange(value = "calc_service_v2", durable = "false", autoDelete = "true"),
key = "multiply"))
public Result multiply(Operands operands) {
// do something..
}
}

What version of Spring AMQP are you using?
Can you provide a stack trace? (edit the question, don't try to put it in a comment).
I just ran a test and it works as expected; the listener container attempts to reconnect every 5 seconds (the default recovery interval).
rabbitAdmin.setIgnoreDeclarationExceptions(true); // useless
In 1.6 (currently at milestone 1) we've changed that boolean to skip all exceptions; previously it only applied to exceptions caused by errors returned from the broker.
However, the admin won't attempt to declare the elements until the connection is established so that should be moot in this context.
I need to see a stack trace to understand what's going on in your case.