Question: After a fixed amount of retries I would like to send the message to an error queue and consume it from its original queue. I want to find a generic solution because I handle a lot of different messages and depending on the exception they raise I want to act differently. How can I set the RecoveryCallback so that Spring triggers it after maxRetries?
What I currently do
I try to set a RetryTemplate and a RecoveryCallback.
When I run the application and publish a message to the test queue I expect the processing in EListener#receive to fail 3 times and then trigger my configured RecoveryCallback which then routes the message based on the context to a specific error queue.
What actually happens
What actually happens is that Spring Boot initializes the RabbitAdmin object with its own RabbitTemplate and therefore does not use my configured RabbitTemplate bean.
I have the following directory structure:
rabbit
|___ EListener.java
|___ Rabbit.java
test
|___ Test.java
I have the following code in Rabbit.java
#Configuration
public class Rabbit {
#Autowired
ConnectionFactory connectionFactory;
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setRetryTemplate(createRetryTemplate());
rabbitTemplate.setRecoveryCallback(createRecoveryCallback());
return rabbitTemplate;
}
createRecoveryCallback() // Omitted for brevity
createRetryTemplate() // Omitted for brevity
}
The EListener.java file contains:
#Component
public class EListener {
#Autowired
RabbitTemplate rabbitTemplate;
#RabbitListener(bindings = #QueueBinding(value = #Queue(value = "test", durable = "true"), exchange = #Exchange(value = "test", type = ExchangeTypes.TOPIC, durable = "true", autoDelete = "true"), key = "test"))
public void receive(Message m) throws Exception {
System.out.println(m);
throw new Exception();
}
}
Test.java contains:
#SpringBootApplication
#ComponentScan("rabbit")
public class Test {
public static void main(String[] args) {
new SpringApplicationBuilder(Test.class).application().run(args);
}
}
Adding a RetryTemplate to the RabbitTemplate is to retry publishing messages.
To add retry on the consumer side, you have to add a retry interceptor to the listener container's advice chain.
Since you are using #RabbitListener, the advice chain goes on the listener container factory #Bean, which means you'll have to declare one yourself instead of relying on the default one created by boot.
Stateless retry does the retries in-memory; statefull retry requeues the message; it requires a messageId property (or some other mechanism to uniquely identify messages).
Related
In my use case I need to do request-reply call to a remote system via managed queues. Using Spring Boot and IBM's MQ starter I have the problem that the application wants to create dynamic/temporary reply queues instead of using the already existing managed queue.
Configuration is set up here
#EnableJms
#Configuration
public class QueueConfiguration {
#Bean
public MQQueueConnectionFactory connectionFactory() throws JMSException {
MQQueueConnectionFactory factory = new MQQueueConnectionFactory();
factory.setTransportType(CT_WMQ); // is 1
factory.setHostName(queueProperties.getHost());
factory.setPort(queueProperties.getPort());
factory.setChannel(queueProperties.getChannel()); // combo of ${queueManager}%${channel}
return factory;
}
#Bean
public JmsMessagingTemplate messagingTemplate(ConnectionFactory connectionFactory) {
JmsMessagingTemplate jmt = new JmsMessagingTemplate(connectionFactory);
jmt.setDefaultDestinationName(queueProperties.getQueueName());
return jmt;
}
#Bean
public Jaxb2Marshaller marshaller() {
Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
marshaller.setPackagesToScan("com.foo.model");
return marshaller;
}
#Bean
public MessageConverter messageConverter(Jaxb2Marshaller marshaller) {
MarshallingMessageConverter converter = new MarshallingMessageConverter();
converter.setMarshaller(marshaller);
converter.setUnmarshaller(marshaller);
return converter;
}
}
Usage is pretty straight forward: Take the object convert and send it. Wait for response receive
and convert it.
#Component
public class ExampleSenderReceiver {
#Autowired
private JmsMessagingTemplate jmsMessagingTemplate;
#Override
#SneakyThrows
public ResponseExample sendAndReceive(RequestExample request, String correlationId) {
MessagePostProcessor mpp = message -> {
message = MessageBuilder.fromMessage(message)
.setHeader(JmsHeaders.CORRELATION_ID, correlationId)
// .setHeader(JmsHeaders.REPLY_TO, "DEV.QUEUE.3") this triggers queue creation
.build();
return message;
};
String destination = Objects.requireNonNull(jmsMessagingTemplate.getDefaultDestinationName());
return jmsMessagingTemplate.convertSendAndReceive(destination, request, ResponseExample.class, mpp);
}
I read already a lot of IBM documentation and think, I need to set the message type to "MQMT_REQUEST" but I do not find the right spot to do so.
Update
Added Spring Integration as Gary proposed and added a configuration for JmsOutboundGateway
#Bean
public MessageChannel requestChannel() {
return new DirectChannel();
}
#Bean
public QueueChannel responseChannel() {
return new QueueChannel();
}
#Bean
#ServiceActivator(inputChannel = "requestChannel" )
public JmsOutboundGateway jmsOutboundGateway( ConnectionFactory connectionFactory) {
JmsOutboundGateway gateway = new JmsOutboundGateway();
gateway.setConnectionFactory(connectionFactory);
gateway.setRequestDestinationName("REQUEST");
gateway.setReplyDestinationName("RESPONSE");
gateway.setReplyChannel(responseChannel());
gateway.setCorrelationKey("JMSCorrelationID*");
gateway.setIdleReplyContainerTimeout(2, TimeUnit.SECONDS);
return gateway;
}
And adapted my ExampleSenderReceiver class
#Autowired
#Qualifier("requestChannel")
private MessageChannel requestChannel;
#Autowired
#Qualifier("responseChannel")
private QueueChannel responseChannel;
#Override
#SneakyThrows
public ResponseExample sendAndReceive(RequestExample request, String correlationId) {
String xmlContent = "the marshalled request object";
Map<String, Object> header = new HashMap<>();
header.put(JmsHeaders.CORRELATION_ID, correlationId);
GenericMessage<String> message1 = new GenericMessage<>(xmlContent, header);
requestChannel.send(message1);
log.info("send done" );
Message<?> receive = responseChannel.receive(1500);
if(null != receive){
log.info("incoming: {}", receive.toString());
}
}
The important part is gateway.setCorrelationKey("JMSCorrelationID*");
Without that line the correlationId was not propagated correct.
Next step is re-adding MessageConverters and make it nice again.
Thank you.
The default JmsTemplate (used by the JmsMessagingTemplate) always uses a temporary reply queue. You can subclass it and override doSendAndReceive(Session session, Destination destination, MessageCreator messageCreator) to use your managed queue instead.
However, it will only work if you have one request outstanding at a time (e.g. all run on a single thread). You will also have to add code for discarding "late" arrivals by checking the correlation id.
You can use async sends instead and handle replies on a listener container and correlate the replies to the requests.
Consider using spring-integration-jms and its outbound gateway instead - it has much more flexibility in reply queue handling (and does all the correlation for you).
https://docs.spring.io/spring-integration/reference/html/jms.html#jms-outbound-gateway
You are missing the queue manager.
ibm:
mq:
queueManager: QM1
channel: chanel
connName: localhost(1414)
user: admin
password: admin
Process:
App A is creating a consumer connection on startup to "REQUEST QUEUE"
in remote Active MQ Broker. App B is pushing request into the "REQUEST
QUEUE", and the same is consumed by App A. App A produces response
into another queue "RESPONSE QUEUE".
App A is using ActiveMQ connection factory and Spring DMLC for
consuming the message.
What I understand from Spring DMLC is, it continuously polls on queue
for message.
Problem:
App A initially consumes request on app restart. But fails to consume the request second time when an another request is made a day after. Even the consumer goes down on the arrival of the second request.
So far tried:
The issue exists only in LIVE.And as per observation consumer is not able process/receive incoming message after a certain period(30 minutes).
I tried with same broker setup in other environments, but unfortunately couldn't re-produce. I have tried every other link but all in vain.
Looking out for pointers:
Is there something fundamentally wrong with the way I'm making use of
ActiveMQ connection for both consuming and producing ?
Possible pointers on what configuration to look out for debugging the issue.
Code:
Message Poller Configuration Code:
#Configuration
#EnableJms
#ComponentScan
#Slf4j
public class MessagePoller
{
#Value("${outbound.queue}")
private String outboundQueue;
private final BrokerProperties brokerProperties;
#Autowired
public MessagePoller(BrokerProperties brokerProperties) {this.brokerProperties = brokerProperties;}
#Bean(name = CONNECTION_FACTORY)
#Primary
public ActiveMQConnectionFactory connectionFactory()
{
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(brokerProperties.getUrl());
connectionFactory.setUserName(brokerProperties.getUser());
connectionFactory.setPassword(brokerProperties.getPassword());
return connectionFactory;
}
#Bean
public JmsListenerContainerFactory jmsListenerContainerFactory()
{
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrency("1-1");
factory.setErrorHandler(getErrorHandler());
factory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
factory.setMessageConverter(new SimpleMessageConverter());
factory.setPubSubDomain(false);
return factory;
}
#Bean(name = OUTBOUND_JMS_TEMPLATE)
public JmsTemplate outboudJmsTemplate(
#Qualifier(CONNECTION_FACTORY)
ConnectionFactory connectionFactory)
{
JmsTemplate template = new JmsTemplate(connectionFactory);
template.setPubSubDomain(false);
template.setDefaultDestinationName(outboundQueue);
return template;
}
private ErrorHandler getErrorHandler()
{
return exception -> log.error("Exception thrown from consumer", exception);
}
}
Message Listener Code:
#JmsListener(
destination = "requestQueue",
containerFactory = "jmsListenerContainerFactory"
)
public void onMessage(Message<String> jmsMessage)
{
log.info("TriggerJobOnRequestService.onMessage() starts");
log.info("Incoming request message: {}", jmsMessage);
Integer deliveryCount = jmsMessage.getHeaders().get("JMSXDeliveryCount", Integer.class);
log.info("Payload : {}", jmsMessage.getPayload());
log.info("Headers : {}", jmsMessage.getHeaders());
log.info("Delivery Count : {}", deliveryCount);
//Processing Code logic
log.info("TriggerJobOnRequestService.onMessage() ends");
}
ActiveMQ connection url:
spring.activemq.url=failover://(tcp://mqueue05.net:61616,tcp://mqueue06.net:61616,tcp://mqueue07.net:61616,tcp://mqueue08.net:61616)?randomize=false&timeout=100&initialReconnectDelay=1000&maxReconnectDelay=1000&maxReconnectAttempts=10
I want to write an application wherein I need a producer sending messages using spring boot rabbitmq and the receiver is written in python. The receiver part was easy-
receive.py
#!/usr/bin/env python
import pika
import sys
connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
channel = connection.channel()
channel.exchange_declare(exchange='topic_logs',
exchange_type='topic')
result = channel.queue_declare(exclusive=True)
queue_name = result.method.queue
binding_keys = sys.argv[1:]
if not binding_keys:
sys.stderr.write("Usage: %s [binding_key]...\n" % sys.argv[0])
sys.exit(1)
for binding_key in binding_keys:
channel.queue_bind(exchange='topic_logs',
queue=queue_name,
routing_key=binding_key)
print(' [*] Waiting for logs. To exit press CTRL+C')
def callback(ch, method, properties, body):
print(" [x] %r:%r" % (method.routing_key, body))
channel.basic_consume(callback,
queue=queue_name,
no_ack=True)
channel.start_consuming()
How to write a spring boot rabbitmq sender code for this? What are the necessary things to be declared there? Please help.
Spring Boot is Java application, first of all.
So, you need to make yourself familiar with that language.
And the choice is correct: you really can send to the RabbitMQ from Java and receive from the queue in any other client.
Spring Boot provides for you RabbitTemplate bean. So, if the story is about to send you just need to inject such a bean and use its API to send:
#Autowired
RabbitTemplate rabbitTemplate;
...
this.rabbitTemplate.convertAndSend("topic_logs", binding_key, data);
See Reference Manual for more info.
With spring integration, first you need to create a configuration class:
#Configuration
public class RabbitConfig {
#Autowired
private ConnectionFactory connectionFactory;
#Bean
DirectExchange dropfileExchange() {
return new DirectExchange("exchange_name", true, false);
}
#Bean
public Queue dropfileQueue() {
return new Queue("queue_name", true);
}
#Bean
Binding dropfileExchangeBinding(DirectExchange dropfileExchange, Queue dropfileQueue) {
return BindingBuilder.bind(dropfileQueue).to(dropfileExchange).with("key_name");
}
#Bean
public RabbitTemplate dropfileExchangeTemplate() {
RabbitTemplate rt = new RabbitTemplate(connectionFactory);
rt.setExchange("exchange_name");
rt.setRoutingKey("key_name");
rt.setConnectionFactory(connectionFactory);
return rt;
}
#Bean
public RabbitMessagingTemplate rabbitMessagingTemplate() {
return new RabbitMessagingTemplate(dropfileExchangeTemplate());
}
}
Then create a gateway service:
#MessagingGateway
public interface DropfileMessageGateway {
#Gateway(requestChannel = "channel_name")
void generate(String payload);
}
And then specify the producer flow using Java DSL as follows:
#Bean
public IntegrationFlow toOutboundQueueFlow() {
return IntegrationFlows.from("channel_name")
.transform(Transformers.toJson())
.handle(Amqp.outboundAdapter(rabbitConfig.dropfileExchangeTemplate())).get();
}
That will read a message from the channel, transform it to JSON and then dispatch it using an outbound adapter to the rabbit exchange.
Note that the messages go to the channel via the messaging gateway, so the channel name in both should be the same.
Do not forget to include the appropriate dependencies.
Is it possible to load spring context with #RabbitListener even if RabbitMQ message broker is down?
The behavior should be similar as in case of broker disconnection. Application is waiting for broker and when it is restored, then listerner is automaticaly reconnected.
Spring Boot 1.3.2.RELEASE
GitHub demo project
spring-amqp configuration:
#Bean
public MessageConverter jsonMessageConverter() {
return new JsonMessageConverter();
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(host);
connectionFactory.setUsername(username);
connectionFactory.setPassword(password);
return connectionFactory;
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrentConsumers(10);
factory.setMaxConcurrentConsumers(10);
factory.setMessageConverter(jsonMessageConverter());
return factory;
}
#Bean
public AmqpAdmin amqpAdmin() {
RabbitAdmin rabbitAdmin = new RabbitAdmin(connectionFactory());
rabbitAdmin.setIgnoreDeclarationExceptions(true); // useless
return rabbitAdmin;
}
listener configuration:
#Service
public class CalculatorServiceV2 {
#RabbitListener(bindings = #QueueBinding(
value = #Queue(value = "calc_service_v2.multiply", durable = "false", autoDelete = "true"),
exchange = #Exchange(value = "calc_service_v2", durable = "false", autoDelete = "true"),
key = "multiply"))
public Result multiply(Operands operands) {
// do something..
}
}
What version of Spring AMQP are you using?
Can you provide a stack trace? (edit the question, don't try to put it in a comment).
I just ran a test and it works as expected; the listener container attempts to reconnect every 5 seconds (the default recovery interval).
rabbitAdmin.setIgnoreDeclarationExceptions(true); // useless
In 1.6 (currently at milestone 1) we've changed that boolean to skip all exceptions; previously it only applied to exceptions caused by errors returned from the broker.
However, the admin won't attempt to declare the elements until the connection is established so that should be moot in this context.
I need to see a stack trace to understand what's going on in your case.
I have two separate applications running either side of an ActiveMQ broker; application 1 sends synchronous requests to application 2 which returns the response back to application 1. At present the replies are via temporary queues and I am now trying to create a named reply destination to avoid the overhead of creating multiple temporary queues.
Application 1
#MessagingGateway
public interface OrderGateway {
#Gateway(requestChannel = "requestChannel", replyChannel = "responseChannel")
public OrderDto fetchOrder(OrderRequest orderRequest);
}
#Bean
public IntegrationFlow outgoingRequestFlow(ConnectionFactory connectionFactory) {
return IntegrationFlows.from("requestChannel")
.handle(Jms.outboundGateway(connectionFactory)
.requestDestination("request.queue")
.replyDestination("response.topic")
.correlationKey("JMSCorrelationID"))
.channel("responseChannel")
.get();
}
Application 2
#Bean
public IntegrationFlow incomingRequestFlow(ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Jms.inboundGateway(connectionFactory)
.destination("request.queue")
.correlationKey("JMSCorrelationID"))
.channel("requestChannel")
.handle("requestServiceActivator", "handleRequest")
.channel("responseChannel")
.get();
}
#Component
public class OrderServiceActivator {
#Autowired
OrderService orderService;
#ServiceActivator
public OrderDto fetchOrder(OrderRequest orderRequest) {
return orderService.getById(orderRequest.getId());
}
}
When I start both applications request.queue gets created and has one consumer (application 2). response.topic gets created but for some reason it has no consumers. Consequently when I send a request in to application 1 it reaches application 2, but after 5 seconds application 1 does not receive a reply and times out and the following errors are logged:
Application 2
org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'org.springframework.web.context.WebApplicationContext:/application-2.responseChannel'
Application 1
org.springframework.integration.MessageTimeoutException: failed to receive JMS response within timeout of: 5000ms
I presume I've made some simple configuration error, any help would be appreciated.
With your configuration, there is no long-lived consumer for the reply queue - a consumer is created for each request (with a message selector for the specific correlation id).
If you add .replyContainer() there will be a permanent consumer.
However, it should make no difference functionally.
I just ran tests similar to yours with and without replyContainer() and it all worked fine for me...
#Bean
public IntegrationFlow jmsOutboundGatewayFlow() {
return f -> f.handleWithAdapter(a ->
a.jmsGateway(this.jmsConnectionFactory)
// .replyContainer()
.replyDestination("pipereplies")
.correlationKey("JmsCorrelationID")
.requestDestination("jmsPipelineTest"));
}
I suggest you turn on debug logging to see if that sheds some light.
#Bean
public IntegrationFlow jmsInboundGatewayFlow() {
return IntegrationFlows.from((MessagingGateways g) ->
g.jms(this.jmsConnectionFactory)
.correlationKey("JmsCorrelationID")
.destination("jmsPipelineTest"))
.<String, String>transform(String::toUpperCase)
.get();
}
I'm failing to find how the #ServiceActivator is getting wired...
Usually it's something like:
#ServiceActivator(inputChannel = "requestChannel", outputChannel = "responseChannel")
public .....
Perhaps that is what you are missing.