Its like "Houston we have a problem here" where I need to schedule/delay a message for 5 minutes after it fails on the first attempt to process an event.
I have implemented dead letter exchange in this scenario.
The messages on failing, route to the DLX --> Retry Queue and comes back to work queue after a TTL of 5 minutes for another attempt.
Here is the configuration I am using:
public class RabbitMQConfig {
#Bean(name = "work")
#Primary
Queue workQueue() {
return new Queue(WORK_QUEUE, true, false, false, null);
}
#Bean(name = "workExchange")
#Primary
TopicExchange workExchange() {
return new TopicExchange(WORK_EXCHANGE, true, false);
}
#Bean
Binding workBinding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(workQueue()).to(workExchange()).with("#");
}
#Bean(name = "retryExchange")
FanoutExchange retryExchange() {
return new FanoutExchange(RETRY_EXCHANGE, true, false);
}
#Bean(name = "retry")
Queue retryQueue() {
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-dead-letter-exchange", WORK_EXCHANGE);
args.put("x-message-ttl", RETRY_DELAY); //delay of 5 min
return new Queue(RETRY_QUEUE, true, false, false, args);
}
#Bean
Binding retryBinding(Queue queue,FanoutExchange exchange) {
return BindingBuilder.bind(retryQueue()).to(retryExchange());
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
return factory;
}
#Bean
Consumer receiver() {
return new Consumer();
}
#Bean
MessageListenerAdapter listenerAdapter(Consumer receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
}
Producer.java:
#GetMapping(path = "/hello")
public String sayHello() {
// Producer operation
String messages[];
messages = new String[] {" hello "};
for (int i = 0; i < 5; i++) {
String message = util.getMessage(messages)+i;
rabbitTemplate.convertAndSend("WorkExchange","", message);
System.out.println(" Sent '" + message + "'");
}
return "hello";
}
Consumer.java:
public class Consumer {
#RabbitListener(queues = "WorkQueue")
public void receiveMessage(String message, Channel channel,
#Header(AmqpHeaders.DELIVERY_TAG) Long tag) throws IOException, InterruptedException {
try {
System.out.println("message to be processed: " + message);
doWorkTwo(message);
channel.basicAck(tag, false);
} catch (Exception e) {
System.out.println("In the exception catch block");
System.out.println("message in dead letter exchange: " + message);
channel.basicPublish("RetryExchange", "", null, message.getBytes());
}
}
private void doWorkTwo(String task) throws InterruptedException {
int c = 0;
int b = 5;
int d = b / c;
}
}
Is it the correct way to use a dead letter exchange for my scenario and after waiting once in the RETRY QUEUE for 5 min, on the second time attempt it does not wait for 5 min in the RETRY QUEUE (I have mentioned TTL as 5 min) and moves to the WORK QUEUE immediately.
I am running this application by hitting localhost:8080/hello url.
Here is my updated configuration.
RabbitMQConfig.java:
#EnableRabbit
public class RabbitMQConfig {
final static String WORK_QUEUE = "WorkQueue";
final static String RETRY_QUEUE = "RetryQueue";
final static String WORK_EXCHANGE = "WorkExchange"; // Dead Letter Exchange
final static String RETRY_EXCHANGE = "RetryExchange";
final static int RETRY_DELAY = 60000; // in ms (1 min)
#Bean(name = "work")
#Primary
Queue workQueue() {
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-dead-letter-exchange", RETRY_EXCHANGE);
return new Queue(WORK_QUEUE, true, false, false, args);
}
#Bean(name = "workExchange")
#Primary
DirectExchange workExchange() {
return new DirectExchange(WORK_EXCHANGE, true, false);
}
#Bean
Binding workBinding(Queue queue, DirectExchange exchange) {
return BindingBuilder.bind(workQueue()).to(workExchange()).with("");
}
#Bean(name = "retryExchange")
DirectExchange retryExchange() {
return new DirectExchange(RETRY_EXCHANGE, true, false);
}
// Messages will drop off RetryQueue into WorkExchange for re-processing
// All messages in queue will expire at same rate
#Bean(name = "retry")
Queue retryQueue() {
Map<String, Object> args = new HashMap<String, Object>();
//args.put("x-dead-letter-exchange", WORK_EXCHANGE);
//args.put("x-message-ttl", RETRY_DELAY);
return new Queue(RETRY_QUEUE, true, false, false, null);
}
#Bean
Binding retryBinding(Queue queue, DirectExchange exchange) {
return BindingBuilder.bind(retryQueue()).to(retryExchange()).with("");
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setDefaultRequeueRejected(false);
/*factory.setAdviceChain(new Advice[] {
org.springframework.amqp.rabbit.config.RetryInterceptorBuilder
.stateless()
.maxAttempts(2).recoverer(new RejectAndDontRequeueRecoverer())
.backOffOptions(1000, 2, 5000)
.build()
});*/
return factory;
}
#Bean
Consumer receiver() {
return new Consumer();
}
#Bean
MessageListenerAdapter listenerAdapter(Consumer receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
}
Consumer.java:
public class Consumer {
#RabbitListener(queues = "WorkQueue")
public void receiveMessage(String message, Channel channel,
#Header(AmqpHeaders.DELIVERY_TAG) Long tag,
#Header(required = false, name = "x-death") HashMap<String, String> xDeath)
throws IOException, InterruptedException {
doWorkTwo(message);
channel.basicAck(tag, false);
}
private void doWorkTwo(String task) {
int c = 0;
int b = 5;
if (c < b) {
throw new AmqpRejectAndDontRequeueException(task);
}
}
}
If you reject the message so the broker routes it to a DLQ, you can examine the x-death header. In this scenario, I have a DLQ with a TTL of 5 seconds and the consumer of the message from the main queue rejects it; the broker routes it to the DLQ, then it expires and is routed back to the main queue - the x-death header shows the number of re-routing operations:
Related
I've implemented RabbitMQ publisher and consumer in reactive manner with Java, but my publishing functionality hangs channel. The queue itself, declaring a queue and consuming however works fine, I've tested it with admin's management UI. When attempting to send more messages I don't see any more of logs like "queue declare success" or "delivering message to exchange...". By the way I know I do not need declareQueue in deliver(), but I added it to verify if communication in this particular channel works.
My code is:
#Slf4j
#Component
public class RabbitConfigurator {
private TasksQueueConfig cfg;
private ReceiverOptions recOpts;
private List<Address> addresses;
private Utils.ExceptionFunction<ConnectionFactory, Connection> connSupplier;
public RabbitConfigurator(TasksQueueConfig cfg) {
this.cfg = cfg;
addresses = cfg
.getHosts()
.stream()
.map(Address::new)
.collect(Collectors.toList());
connSupplier = cf -> {
LOG.info("initializing new RabbitMQ connection");
return cf.newConnection(addresses, "dmTasksProc");
};
}
#Bean
public ConnectionFactory rabbitMQConnectionFactory() {
ConnectionFactory cf = new ConnectionFactory();
cf.setHost(cfg.getHosts().get(0));
cf.setPort(5672);
cf.setUsername(cfg.getUsername());
cf.setPassword(cfg.getPassword());
return cf;
}
#Bean
public Sender sender(ConnectionFactory connFactory) {
SenderOptions sendOpts = new SenderOptions()
.connectionClosingTimeout(Duration.parse(cfg.getConnectionTimeout()))
.connectionFactory(connFactory)
.connectionSupplier(connSupplier)
.connectionSubscriptionScheduler(Schedulers.elastic());
return RabbitFlux.createSender(sendOpts);
}
#Bean
public Receiver receiver(ConnectionFactory connFactory) {
ReceiverOptions recOpts = new ReceiverOptions()
.connectionClosingTimeout(Duration.parse(cfg.getConnectionTimeout()))
.connectionFactory(connFactory)
.connectionSupplier(connSupplier)
.connectionSubscriptionScheduler(Schedulers.elastic());
return RabbitFlux.createReceiver(recOpts);
}
#Bean
public Flux<Delivery> deliveryFlux(Receiver receiver) {
return receiver.consumeAutoAck(cfg.getName(), new ConsumeOptions().qos(cfg.getPrefetchCount()));
}
#Bean
public AmqpAdmin rabbitAmqpAdmin(ConnectionFactory connFactory) {
return new RabbitAdmin(new CachingConnectionFactory(connFactory));
}
}
and the consumer/publisher:
#Slf4j
#Service
public class TasksQueue implements DisposableBean {
private TasksQueueConfig cfg;
private ObjectMapper mapper;
private Flux<Delivery> deliveryFlux;
private Receiver receiver;
private Sender sender;
private Disposable consumer;
public TasksQueue(TasksQueueConfig cfg, AmqpAdmin amqpAdmin, ObjectMapper mapper, Flux<Delivery> deliveryFlux,
Receiver receiver, Sender sender) {
this.cfg = cfg;
this.mapper = mapper;
this.deliveryFlux = deliveryFlux;
this.receiver = receiver;
this.sender = sender;
amqpAdmin.declareQueue(new Queue(cfg.getName(), false, false, false));
consumer = consume();
}
public Mono<Void> deliver(Flux<Task> tasks) {
var pub = sender.sendWithPublishConfirms(
tasks.map(task -> {
try {
String exchange = "";
LOG.debug("delivering message to exchange='{}', routingKey='{}'", exchange, cfg.getName());
return new OutboundMessage(exchange, cfg.getName(), mapper.writeValueAsBytes(task));
} catch(JsonProcessingException ex) {
throw Exceptions.propagate(ex);
}
}));
return sender.declareQueue(QueueSpecification.queue(cfg.getName()))
.flatMap(declareOk -> {
LOG.info("queue declare success");
return Mono.just(declareOk);
})
.thenMany(pub)
.doOnError(JsonProcessingException.class, ex -> LOG.error("Cannot prepare queue message:", ex))
.doOnError(ex -> LOG.error("Failed to send task to the queue:", ex))
.map(res -> {
if(res.isAck()) {
LOG.info("Message {} sent successfully", new String(res.getOutboundMessage().getBody()));
return res;
} else {
LOG.info("todo");
return res;
}
})
.then();
}
private Disposable consume() {
return deliveryFlux
.retryWhen(Retry.fixedDelay(10, Duration.ofSeconds(1)))
.doOnError(err -> {
LOG.error("tasks consumer error", err);
})
.subscribe(m -> {
LOG.info("Received message {}", new String(m.getBody()));
});
}
#Override
public void destroy() throws Exception {
LOG.info("Cleaning up tasks queue resources");
consumer.dispose();
receiver.close();
sender.close();
}
}
Five minutes after attempting to send message I get log:
r.r.ChannelCloseHandlers$SenderChannelCloseHandler:47: closing channel 1 by signal cancel
r.r.ChannelCloseHandlers$SenderChannelCloseHandler:53: Channel 1 didn't close normally: null
Big thanks for input in advance!
I have a rabbitListener which continuously listens to user messages of a queue "user-messages" asynchronously. Everything is OK until unless queue is loaded with bulk messages. When messages in bulk published to queue, messages of the same user are processing first thereby messages of other users are waiting for their turn.
I can't use Priority Queue because all the users have equal priority. So I want to create new queues and listen to them at runtime. All the queues will be short-lived as soon as messages consumed. (the queue will be deleted)
On browsing, I found a queue can be dynamically created using RabbitAdmin. But the issues are
How can I make my listener listen to a new short-live (TTL) queue created at runtime?
How can I make the listener stop listening to a deleted queue (after TTL time) to avoid exceptions?
Currently, I'm using SimpleMessageListenerContainerFactory. I've no issues to use DirectMessageListenerContainer as well. My only concern is how to communicate about dynamic queue creation & deletion to Listener. Thinking about to https://www.rabbitmq.com/event-exchange.html (event exchange plugin).
Is there any way that spring-amqp supporting start/stop listening dynamic queues. Thanks in advance.
#Bean
public SimpleRabbitListenerContainerFactory myRabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(config.getConnectionFactory());
factory.setAcknowledgeMode(AcknowledgeMode.MANUAL);
factory.setConcurrentConsumers(1);
factory.setMaxConcurrentConsumers(3);
return factory;
}
#RabbitListener(id = "listener", queues = {
"#{receiver.queues()}" }, containerFactory = "myRabbitListenerContainerFactory")
public void listen(QueueMessage message, Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) long tag,
MessageHeaders headers) {
//process message
}
[1]: https://www.rabbitmq.com/event-exchange.html
this geezer seems to be doing exactly that => https://karadenizfaruk28.medium.com/rabbitmq-dynamic-queue-add-and-listen-at-runtime-with-springboot-c7d42f0447c
code from the link:
rabbitMQ config
#Configuration
public class RabbitMqConfiguration implements RabbitListenerConfigurer {
#Autowired
private ConnectionFactory connectionFactory;
#Bean
public Jackson2JsonMessageConverter producerJackson2MessageConverter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public MappingJackson2MessageConverter consumerJackson2MessageConverter() {
return new MappingJackson2MessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(producerJackson2MessageConverter());
return rabbitTemplate;
}
#Bean
public RabbitAdmin rabbitAdmin() {
return new RabbitAdmin(connectionFactory);
}
#Bean
public RabbitListenerEndpointRegistry rabbitListenerEndpointRegistry() {
return new RabbitListenerEndpointRegistry();
}
#Bean
public DefaultMessageHandlerMethodFactory messageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(consumerJackson2MessageConverter());
return factory;
}
#Bean
public MessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
#Override
public void configureRabbitListeners(final RabbitListenerEndpointRegistrar registrar) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setPrefetchCount(1);
factory.setConsecutiveActiveTrigger(1);
factory.setConsecutiveIdleTrigger(1);
factory.setConnectionFactory(connectionFactory);
registrar.setContainerFactory(factory);
registrar.setEndpointRegistry(rabbitListenerEndpointRegistry());
registrar.setMessageHandlerMethodFactory(messageHandlerMethodFactory());
}
}
interface
public interface RabbitQueueService {
void addNewQueue(String queueName,String exchangeName,String routingKey);
void addQueueToListener(String listenerId,String queueName);
void removeQueueFromListener(String listenerId,String queueName);
Boolean checkQueueExistOnListener(String listenerId,String queueName);
}
service
#Service
#Log4j2
public class RabbitQueueServiceImpl implements RabbitQueueService {
#Autowired
private RabbitAdmin rabbitAdmin;
#Autowired
private RabbitListenerEndpointRegistry rabbitListenerEndpointRegistry;
#Override
public void addNewQueue(String queueName, String exchangeName, String routingKey) {
Queue queue = new Queue(queueName, true, false, false);
Binding binding = new Binding(
queueName,
Binding.DestinationType.QUEUE,
exchangeName,
routingKey,
null
);
rabbitAdmin.declareQueue(queue);
rabbitAdmin.declareBinding(binding);
this.addQueueToListener(exchangeName,queueName);
}
#Override
public void addQueueToListener(String listenerId, String queueName) {
log.info("adding queue : " + queueName + " to listener with id : " + listenerId);
if (!checkQueueExistOnListener(listenerId,queueName)) {
this.getMessageListenerContainerById(listenerId).addQueueNames(queueName);
log.info("queue ");
} else {
log.info("given queue name : " + queueName + " not exist on given listener id : " + listenerId);
}
}
#Override
public void removeQueueFromListener(String listenerId, String queueName) {
log.info("removing queue : " + queueName + " from listener : " + listenerId);
if (checkQueueExistOnListener(listenerId,queueName)) {
this.getMessageListenerContainerById(listenerId).removeQueueNames(queueName);
log.info("deleting queue from rabbit management");
this.rabbitAdmin.deleteQueue(queueName);
} else {
log.info("given queue name : " + queueName + " not exist on given listener id : " + listenerId);
}
}
#Override
public Boolean checkQueueExistOnListener(String listenerId, String queueName) {
try {
log.info("checking queueName : " + queueName + " exist on listener id : " + listenerId);
log.info("getting queueNames");
String[] queueNames = this.getMessageListenerContainerById(listenerId).getQueueNames();
log.info("queueNames : " + new Gson().toJson(queueNames));
if (queueNames != null) {
log.info("checking " + queueName + " exist on active queues");
for (String name : queueNames) {
log.info("name : " + name + " with checking name : " + queueName);
if (name.equals(queueName)) {
log.info("queue name exist on listener, returning true");
return Boolean.TRUE;
}
}
return Boolean.FALSE;
} else {
log.info("there is no queue exist on listener");
return Boolean.FALSE;
}
} catch (Exception e) {
log.error("Error on checking queue exist on listener");
log.error("error message : " + ExceptionUtils.getMessage(e));
log.error("trace : " + ExceptionUtils.getStackTrace(e));
return Boolean.FALSE;
}
}
private AbstractMessageListenerContainer getMessageListenerContainerById(String listenerId) {
log.info("getting message listener container by id : " + listenerId);
return ((AbstractMessageListenerContainer) this.rabbitListenerEndpointRegistry
.getListenerContainer(listenerId)
);
}
}
This is my current code:
#Bean
public IntegrationFlow someFlow() {
return IntegrationFlows
.from(someInboundAdapter())
.transform(new JsonToObjectTransformer(SomeObject.class))
.filter((SomeObject s) -> s.getId()!=null && s.getId().isRealId(), f -> f.discardChannel(manualNackChannel()))
.channel(amqpInputChannel())
.get();
}
#ServiceActivator(inputChannel = "manualNackChannel")
public void manualNack(#Header(AmqpHeaders.CHANNEL) Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) Long tag) throws IOException {
channel.basicNack(tag, false, false);
}
#Bean
public AmqpInboundChannelAdapter someInboundAdapter() {
AmqpInboundChannelAdapter adapter = new AmqpInboundChannelAdapter(someListenerContainer());
adapter.setErrorChannel(manualNackChannel()); //NOT WORKING
return adapter;
}
#Bean
public SimpleMessageListenerContainer someListenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer(commonConfig.connectionFactory());
listenerContainer.setQueues(someQueue());
listenerContainer.setConcurrentConsumers(4);
listenerContainer.setMessageConverter(jackson2JsonConverter());
listenerContainer.setAcknowledgeMode(AcknowledgeMode.MANUAL);
listenerContainer.setConsumerTagStrategy(consumerTagStrategy());
listenerContainer.setAfterReceivePostProcessors(new GUnzipPostProcessor());
listenerContainer.setAdviceChain(commonConfig.retryInterceptor()); //reties 3 times and RejectAndDontRequeueRecoverer
return listenerContainer;
}
Here I use MANUAL ACK-ing, since I want to ACK/NACK message only if processed sucesfully in last part of IntegrationFlow.
Here, in case that message cannot be deserialized, retryInterceptor is invoked, but after exausting all the retries, I need to be able to manually NACK the message. I expected to do it with setErrorChannel method on adapter, but I cannot get AMQP channel headers in manualNack.
Is this proper way to manually NACK message from AmqpInboundChannelAdapter?
UPDATE
I guess this is my current solution, but don't know if good enough:
private ErrorMessageStrategy nackStrategy(){
return (throwable, attributes) -> {
Object inputMessage = attributes.getAttribute(ErrorMessageUtils.INPUT_MESSAGE_CONTEXT_KEY);
return new ErrorMessage(throwable, ((Message)inputMessage).getHeaders());
};
}
#Bean
public AmqpInboundChannelAdapter someInboundAdapter() {
AmqpInboundChannelAdapter adapter = new AmqpInboundChannelAdapter(someListenerContainer());
adapter.setRecoveryCallback(new ErrorMessageSendingRecoverer(manualNackChannel(), nackStrategy()));
adapter.setRetryTemplate(commonConfig.retryTemplate());
return adapter;
}
in case that message cannot be deserialized
Since AMQP message cannot be deserialized, the Spring Message isn't created and therefore no AmqpHeaders.CHANNEL header.
I'm not sure though how that ErrorMessageSendingRecoverer can help you here because deserialization really happens on the SimpleMessageListenerContainer level a bit earlier than onMessage() in the AmqpInboundChannelAdapter.
Not sure yet how to help you but maybe you can share some simply Spring Boot project to play from our side? Thanks
Here is the full working code for this example. You can test ACK/NACK on 3 REST endpoints:
http://localhost:8080/sendForAck -> will send Object SomeObject to queue proba, transform it, forward to exchange probaEx and ACK it after that
http://localhost:8080/sendForNack -> will send malformed byte[] message which cannot be deserialized and will be NACK-ed.
http://localhost:8080/sendForNack2 -> will create malformed json message and will be NACK-ed with InvalidFormatException
#Controller
#EnableAutoConfiguration
#Configuration
public class SampleController {
#Autowired
public RabbitTemplate rabbitTemplate;
#RequestMapping("/sendForAck")
#ResponseBody
String sendForAck() {
SomeObject s = new SomeObject();
s.setId(2);
rabbitTemplate.convertAndSend("", "proba", s);
return "Sent for ACK!";
}
#RequestMapping("/sendForNack")
#ResponseBody
String sendForNack() {
rabbitTemplate.convertAndSend("", "proba", new byte[]{1,2,3});
return "Sent for NACK!";
}
#RequestMapping("/sendForNack2")
#ResponseBody
String sendForNack2() {
MessageProperties p = new MessageProperties();
p.getHeaders().put("__TypeId__", "SampleController$SomeObject");
p.setDeliveryMode(MessageDeliveryMode.PERSISTENT);
p.setPriority(0);
p.setContentEncoding("UTF-8");
p.setContentType("application/json");
rabbitTemplate.send("", "proba", new org.springframework.amqp.core.Message("{\"id\":\"abc\"}".getBytes(), p));
return "Sent for NACK2!";
}
static class SomeObject{
private Integer id;
public Integer getId(){return id;}
public void setId(Integer id){ this.id=id; }
#Override
public String toString() {
return "SomeObject{" +
"id=" + id +
'}';
}
}
#Bean
public IntegrationFlow someFlow() {
return IntegrationFlows
.from(someInboundAdapter())
.transform(new JsonToObjectTransformer(SomeObject.class))
.filter((SomeObject s) -> s.getId()!=null, f -> f.discardChannel(manualNackChannel()))
.transform((SomeObject s) -> {s.setId(s.getId()*2); return s;})
.handle(amqpOutboundEndpoint())
.get();
}
#Bean
public MessageChannel manualNackChannel() {
return new DirectChannel();
}
#Bean
public MessageChannel manualAckChannel() {
return new DirectChannel();
}
#ServiceActivator(inputChannel = "manualNackChannel")
public void manualNack(#Header(AmqpHeaders.CHANNEL) Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) Long tag, #Payload Object p) throws IOException {
channel.basicNack(tag, false, false);
System.out.println("NACKED " + p);
}
#ServiceActivator(inputChannel = "manualAckChannel")
public void manualAck(#Header(AmqpHeaders.CHANNEL) Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) Long tag, #Payload Object p) throws IOException {
channel.basicAck(tag, false);
System.out.println("ACKED " + p);
}
private ErrorMessageStrategy nackStrategy() {
return (throwable, attributes) -> {
Message inputMessage = (Message)attributes.getAttribute(ErrorMessageUtils.INPUT_MESSAGE_CONTEXT_KEY);
return new ErrorMessage(throwable, inputMessage.getHeaders());
};
}
#Bean
public AmqpInboundChannelAdapter someInboundAdapter() {
AmqpInboundChannelAdapter adapter = new AmqpInboundChannelAdapter(someListenerContainer());
adapter.setRecoveryCallback(new ErrorMessageSendingRecoverer(manualNackChannel(), nackStrategy()));
adapter.setRetryTemplate(retryTemplate());
return adapter;
}
#Bean
public RetryTemplate retryTemplate() {
RetryTemplate template = new RetryTemplate();
ExponentialBackOffPolicy backOffPolicy = new ExponentialBackOffPolicy();
backOffPolicy.setInitialInterval(10);
backOffPolicy.setMaxInterval(5000);
backOffPolicy.setMultiplier(4);
template.setBackOffPolicy(backOffPolicy);
SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy();
retryPolicy.setMaxAttempts(4);
template.setRetryPolicy(retryPolicy);
return template;
}
#Bean
public AmqpOutboundEndpoint amqpOutboundEndpoint() {
AmqpOutboundEndpoint outboundEndpoint = new AmqpOutboundEndpoint(ackTemplate());
outboundEndpoint.setConfirmAckChannel(manualAckChannel());
outboundEndpoint.setConfirmCorrelationExpressionString("#root");
outboundEndpoint.setExchangeName("probaEx");
return outboundEndpoint;
}
#Bean
public MessageConverter jackson2JsonConverter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public RabbitTemplate ackTemplate() {
RabbitTemplate ackTemplate = new RabbitTemplate(connectionFactory());
ackTemplate.setMessageConverter(jackson2JsonConverter());
return ackTemplate;
}
#Bean
public Queue someQueue() {
return QueueBuilder.nonDurable("proba").build();
}
#Bean
public Exchange someExchange(){
return ExchangeBuilder.fanoutExchange("probaEx").build();
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory factory = new CachingConnectionFactory();
factory.setHost("10.10.121.137");
factory.setPort(35672);
factory.setUsername("root");
factory.setPassword("123456");
factory.setPublisherConfirms(true);
return factory;
}
#Bean
public SimpleMessageListenerContainer someListenerContainer() {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer(connectionFactory());
listenerContainer.setQueues(someQueue());
listenerContainer.setMessageConverter(jackson2JsonConverter());
listenerContainer.setAcknowledgeMode(AcknowledgeMode.MANUAL);
return listenerContainer;
}
public static void main(String[] args) throws Exception {
SpringApplication.run(SampleController.class, args);
}
}
Still, the question remains if this private ErrorMessageStrategy nackStrategy() could be written in a better way?
My aim: i have multiple jobs(Processes) running parallely (seperate threads). i want to implement messaging so that each process can send message(if required) to rabbitmq Server.
now i have this
#Configuration
public class SenderConfiguration {
String content = "";
String host = "";
String port = "";
String userName = "";
String password = "";
String queueName = "";
InputStream input = null;
public SenderConfiguration() {
init();
}
private void init() {
Properties prop = new Properties();
try {
input = new FileInputStream("R.CONFIGURATION_FILE_PATH");
host = prop.getProperty("messaging.host");
port = prop.getProperty("messaging.port");
userName = prop.getProperty("messaging.userName");
password = prop.getProperty("messaging.password");
queueName = prop.getProperty("messaging.queue");
} catch (FileNotFoundException e) {
e.printStackTrace();
} finally {
if (input != null) {
try {
input.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
template.setRoutingKey(this.queueName);
return template;
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(
this.host);
connectionFactory.setUsername(userName);
connectionFactory.setPassword(password);
return connectionFactory;
}
#Bean
public ScheduledProducer scheduledProducer() {
return new ScheduledProducer();
}
#Bean
public BeanPostProcessor postProcessor() {
return new ScheduledAnnotationBeanPostProcessor();
}
static class ScheduledProducer {
#Autowired
private volatile RabbitTemplate rabbitTemplate;
private final AtomicInteger counter = new AtomicInteger();
#Scheduled(fixedRate = 1000)
public void sendMessage(String message) {
rabbitTemplate.convertAndSend("Roxy " + counter.incrementAndGet());
}
}
}
and to call this from one of my operation
new AnnotationConfigApplicationContext(SenderConfiguration.class);
shall i make it abstract class and my every operation /process should extend it ? what would be best approach ?
and can i make above process any better?
Just use single class with property placeholders...
Use
#Value("${messaging.host}")
String host;
etc.
No need for a subclass for each.
I am getting spurious correlation errors using TcpOutboundGateway with CachingClientConnectionFactory in a multithreaded context.
The log message is:
2015-05-26 14:50:38.406 ERROR 3320 --- [pool-2-thread-2] o.s.i.ip.tcp.TcpOutboundGateway : Cannot correlate response - no pending reply
I do not get the error when sending from a single thread, and I have tested and 2 physical machines - Windows 7 and Fedora 20. I am using Spring boot
It results in a timeout error for on the send that does not recieve its response.
Below is my simplified code:
Note it does not always produce the error - it is spurious
The code Uses a TcpOutboundGateway and TcpInboundGateway, but in my actual application the server is legacy (not Spring) Java code, so I use CachingClientConnectionFactory to enhance performance
#Configuration
#ComponentScan
#EnableAutoConfiguration
public class Test {
//**************** Client **********************************************
#Bean
public MessageChannel replyChannel() {
return new DirectChannel();
}
#Bean
public MessageChannel sendChannel() {
MessageChannel directChannel = new DirectChannel();
return directChannel;
}
#Bean
AbstractClientConnectionFactory tcpNetClientConnectionFactory() {
AbstractClientConnectionFactory tcpNetClientConnectionFactory = new TcpNetClientConnectionFactory("localhost", 9003);
CachingClientConnectionFactory cachingClientConnectionFactory = new CachingClientConnectionFactory(tcpNetClientConnectionFactory, 4);
return cachingClientConnectionFactory;
}
#Bean
#ServiceActivator(inputChannel = "sendChannel")
TcpOutboundGateway tcpOutboundGateway() {
TcpOutboundGateway tcpOutboundGateway = new TcpOutboundGateway();
tcpOutboundGateway.setConnectionFactory(tcpNetClientConnectionFactory());
tcpOutboundGateway.setReplyChannel(replyChannel());
return tcpOutboundGateway;
}
//******************************************************************
//**************** Server **********************************************
#Bean
public MessageChannel receiveChannel() {
return new DirectChannel();
}
#Bean
TcpNetServerConnectionFactory tcpNetServerConnectionFactory() {
TcpNetServerConnectionFactory tcpNetServerConnectionFactory = new TcpNetServerConnectionFactory(9003);
tcpNetServerConnectionFactory.setSingleUse(false);
return tcpNetServerConnectionFactory;
}
#Bean
TcpInboundGateway tcpInboundGateway() {
TcpInboundGateway tcpInboundGateway = new TcpInboundGateway();
tcpInboundGateway.setConnectionFactory(tcpNetServerConnectionFactory());
tcpInboundGateway.setRequestChannel(receiveChannel());
return tcpInboundGateway;
}
//******************************************************************
#Bean
#Scope("prototype")
Worker worker() {
return new Worker();
}
public volatile static int lc = 4;
public volatile static int counter = lc;
public volatile static long totStartTime = 0;
public volatile static int messageCount = 0;
public static synchronized int incMessageCount(){
return ++messageCount;
}
public static void main(String args[]) {
//new LegaServer();
ConfigurableApplicationContext applicationContext = SpringApplication.run(Test.class, args);
totStartTime = System.currentTimeMillis();
for (int z = 0; z < lc; z++) {
new Thread((Worker) applicationContext.getBean("worker")).start();
}
try {
Thread.currentThread().sleep(20000);
} catch (InterruptedException e) {
e.printStackTrace();
}
applicationContext.stop();
}
}
#MessageEndpoint
class RequestHandler {
#ServiceActivator(inputChannel = "receiveChannel")
public String rxHandler(byte [] in) {
String s = new String(in);
System.out.println("rxHandler:"+s);
return "Blah blah " + s;
}
}
#MessageEndpoint
class ResponseHandler {
#ServiceActivator(inputChannel = "replyChannel")
public void replyHandler(byte [] in) {
System.out.println("replyHandler:"+new String(in));
}
}
class Worker implements Runnable {
#Autowired
#Qualifier("sendChannel")
MessageChannel dc;
#Override
public void run() {
Test.counter--;
int locMessageCount=0;
long startTime = System.currentTimeMillis();
for (int t = 0; t < 20; t++) {
locMessageCount = Test.incMessageCount();
Map hs = new HashMap<String, String>();
hs.put("context", new Integer(Test.counter));
GenericMessage message = new GenericMessage("this is a test message " + locMessageCount, hs);
try {
boolean sent = dc.send(message);
} catch (Exception e) {
//e.printStackTrace();
System.out.println("locMessageCount:"+locMessageCount);
}
}
if (locMessageCount == (Test.lc*20)) {
long totfinTime = System.currentTimeMillis();
System.out.println("Tot. Time taken: " + (totfinTime - Test.totStartTime));
System.out.println("Tot. TPS: " + (1000 * 20* Test.lc) / (totfinTime - Test.totStartTime));
System.out.println("Tot. messages: " + Test.messageCount);
}
}
}
Any suggestions would be greatly appreciated, as is the assistance I have received so far. TY
Thanks; this is a bug with the combo of the outbound gateway and caching connection factory; please open a JIRA Issue.
The problem is that the connection is added back to the pool (and reused) before the first thread (Thread-5) removes the pending reply; he ends up removing the new pending reply (for Thread-2) instead of his own.
Unfortunately, I don't have a simple work-around for you; it needs code changes in the gateway to fix it.