I have developed a simple client for receiving messages from a rabbitMQ queue.
#RabbitListener(queues = "#{'${rabbitmq.queues}'.split(',')}")
public void receiveMessage(Message message, #Header(AmqpHeaders.CONSUMER_QUEUE) String queue) {
logger.debug("message payload as string {}", new String(message.getBody()));
I am having a standard SSL handshake error at the moment because I haven't installed my private key.
Since there is an error, the message is not consumed, it stays in the queue.
The problem is that my client continues to try to consume the same message indefinitely. Could you advice me with a solution to this?
What I want is, in case of a serious global error, to stop consuming messages for a "while". Maybe a Thread sleep? Is there a spring solution to this? Am I missing something here?
By default, the listener container will attempt to reconnect every 5 seconds.
You can reconfigure the container(s), by adding a BackOff to the listener container factory; something like...
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrentConsumers(3);
final ExponentialBackOff backOff = new ExponentialBackOff();
backOff.setInitialInterval(5_000);
backOff.setMultiplier(1.5);
backOff.setMaxInterval(30_000);
backOff.setMaxElapsedTime(120_000);
factory.setRecoveryBackOff(backOff);
return factory;
}
Related
I have a RabbitListener that looks like:
#RabbitListener
public class ProductListener {
#Queue(value = "product", executor = "product-listener")
public void receive(MyType message) {
// process message
}
}
In certain cases, I want to shut down my application. However, before doing so I would like to stop the RabbitListner from consuming new messages and give it some time to finish the processing of already received messages.
In spring this should be possible via RabbitListenerEndpointRegistry#getListenerContainer(String) according to How to stop consuming messages with #RabbitListener and How to gracefully stop consuming messages with #RabbitListener .
I didn’t find something similar for Micronaut. Is this possible with Micronaut?
EDIT
Just found out how to run multiple consumers inside one service:
#Bean
SimpleMessageListenerContainer container(ConnectionFactory connectionFactory, MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(RENDER_QUEUE);
container.setConcurrentConsumers(concurrentConsumers); // setting this in env
container.setMessageListener(listenerAdapter);
return container;
}
#Bean
MessageListenerAdapter listenerAdapter(RenderMessageConsumer receiver) {
return new MessageListenerAdapter(receiver, "reciveMessageFromRenderQueue");
}
Now the only question that remains is: how can I have a global limit? So how do multiple instances of the AMQP receiver share the total number of consumers? So I want to set a global number of concurrentConsumers to 10, run 2 instances of the consumerSerivce and have each instance run around 5 consumers. Can this be managed by rabbitMq?
I have a Spring service that consumes AMQP messages and calls a http resource for each message.
After the http call completes another queue is called to either report error or done. Only then will message handling complete and the next message be taken from the queue.
// simplified
#RabbitListener(queues = RENDER_QUEUE)
public void reciveMessageFromRenderQueue(String message) {
try {
RenderMessage renderMessage = JsonUtils.stringToObject(message, RenderMessage.class);
String result = renderService.httpCallRenderer(renderMessage);
messageProducer.sendDoneMessage(result);
} catch (Exception e) {
logError(type, e);
messageProducer.sendErrorMessage(e.getMessage());
}
}
There are at times hundreds or thousands of render messages in the queue but the http call is rather long running and not doing much. This becomes obvious as I can improve the message handling rate by running multiple instances of the service thus adding more consumers and calling the http endpoint multiple times. One instance has exactly one consumer for the channel so the number of instances is equal to the number of consumers. However that heavily increases memory usage (since the service uses spring) for just forwarding a message and handling the result.
So I thought, I'd do the http call asynchronously and return immediatly after accepting the message:
.httpCallRendererAsync(renderMessage)
.subscribeOn(Schedulers.newThread())
.subscribe(new Observer<String >() {
public void onNext(String result) {
messageProducer.sendDoneMessage(result);
}
public void onError(Throwable throwable) {
messageProducer.sendErrorMessage(throwable.getMessage());
}
});
That however overloads the http endpoint which cannot deal with 1000 or more simultanous requests.
What I need is for my amqp service to take a certain amount of messages from the queue, handle them in separate threads, make the http call in each of them and return with "message handled". The amount of messages taken from the queue however needs to be shared between multiple instances of that service, so if maximum is 10, message consumption is round robin, the first 5 odd messages should be handled by instance one and the first 5 even messages by instance 2 and as soon as one instance finishes handling the message it should take another one from the queue.
What I found are things like prefetch with limts by consumer and by channel as described by rabbitmq. And the spring-rabbit implementation which uses prefetchCount and the transactionSize described here. That however does not seem to do anything for a single running instance. It will not spawn additional threads to handle more messages concurrently. And of course it will not reduce the number of messages handled in my async scenario since those messages are immediatly considered "handled".
#Bean
public RabbitListenerContainerFactory<SimpleMessageListenerContainer> prefetchContainerFactory(ConnectionFactory rabbitConnectionFactory) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(rabbitConnectionFactory);
factory.setPrefetchCount(5);
factory.setTxSize(5);
return factory;
}
// and then using
#RabbitListener(queues = RENDER_QUEUE, containerFactory = "prefetchContainerFactory")
The most important requirement for me seems to be that multiple messages should be handled in one instance while the maximum of concurrently handled messages should be shared between instances.
Can that be done using rabbitMq and spring? Or do I have to implemenent something in between.
In an early stage it might be acceptable to just have concurrent message handling in one instance and not share that limit. Then I'll have to configure the limit manually using environment variables while scaling the number of instances.
Now the only question that remains is: how can I have a global limit? So how do multiple instances of the AMQP receiver share the total number of consumers? So I want to set a global number of concurrentConsumers to 10, run 2 instances of the consumerSerivce and have each instance run around 5 consumers. Can this be managed by rabbitMq?
There is no mechanism in either RabbitMQ or Spring to support such a scenario automatically. You can, however, change the concurrency at runtime (setConcurrentConsumers() on the container) so you could use some external agent to manage the concurrency on each instance.
I got a load-test setup of spring websocket server (based on Jetty and spring version 4.3.2.RELEASE) and client, that generates many connections (based on spring's sample java websocket client). The code below sends data to given websocket session: the snippet exploits the case where sessionId can be used instead of User ID (Spring WebSocket #SendToSession: send message to specific session). I may execute this code very often, every 2-3 milliseconds. I use SimpleMessageBroker.
public void publishToSessionUsingTopic(String sessionId, String subscriptionTopic, Map<String, CacheRowModel> payload) {
String subscriptionTopicWithoutUser = subscriptionTopic.replace(USER_ENDPOINT, "");
// necessary message headers for per-session send
SimpMessageHeaderAccessor headerAccessor = SimpMessageHeaderAccessor.create(SimpMessageType.MESSAGE);
headerAccessor.setSessionId(sessionId);
headerAccessor.setLeaveMutable(true);
simpMessagingTemplate.convertAndSendToUser(sessionId, subscriptionTopicWithoutUser, Collections.singletonList(payload), headerAccessor.getMessageHeaders());
}
When this code is executed very frequently (every 2-3 milliseconds) for ~100 sessions, while I see in my logs that it was run and called the convertAndSendToUser, some of the sessions won't receive the message. I appreciate any suggestions about how this could be cleared.
Well, I think your problem is with the:
#Bean
public ThreadPoolTaskExecutor clientOutboundChannelExecutor() {
TaskExecutorRegistration reg = getClientOutboundChannelRegistration().getOrCreateTaskExecRegistration();
ThreadPoolTaskExecutor executor = reg.getTaskExecutor();
executor.setThreadNamePrefix("clientOutboundChannel-");
return executor;
}
where it uses this config for the Executor:
protected ThreadPoolTaskExecutor getTaskExecutor() {
ThreadPoolTaskExecutor executor = (this.taskExecutor != null ? this.taskExecutor : new ThreadPoolTaskExecutor());
executor.setCorePoolSize(this.corePoolSize);
executor.setMaxPoolSize(this.maxPoolSize);
executor.setKeepAliveSeconds(this.keepAliveSeconds);
executor.setQueueCapacity(this.queueCapacity);
executor.setAllowCoreThreadTimeOut(true);
return executor;
}
See, there is no RejectedExecutionHandler configured. And by default it is like:
private RejectedExecutionHandler rejectedExecutionHandler = new ThreadPoolExecutor.AbortPolicy();
So, when you have enough many messages and tasks for them exceed the ThreadPool, any extra are just aborted.
To fix the issue you should implement WebSocketMessageBrokerConfigurer and override its configureClientOutboundChannel() to provide some custom taskExecutor(ThreadPoolTaskExecutor taskExecutor) for example with the new ThreadPoolExecutor.CallerRunsPolicy().
I am currently using DefaultMessageListenerContainer to create listeners and JmsTemplate to send messages (producer) to queues.
Spring Configuration Snippet:
#Bean
public ActiveMQConnectionFactory connectionFactory() {
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_BROKER_URL);
factory.setRedeliveryPolicy(desiredRedeliveryPolicy());
return factory;
}
#Bean
public DefaultMessageListenerContainer requestMessageListenerContainer() {
DefaultMessageListenerContainer requestMessageListenerContainer = new DefaultMessageListenerContainer();
requestMessageListenerContainer.setConcurrentConsumers(noOfconcurrentConsumers);
requestMessageListenerContainer.setConnectionFactory(connectionFactory());
requestMessageListenerContainer.setDestinationName(requestQueueName);
requestMessageListenerContainer.setMessageListener(requestMessageListener());
requestMessageListenerContainer.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
requestMessageListenerContainer.setSessionTransacted(false);
return requestMessageListenerContainer;
}
#Bean
public JmsTemplate requestJmsTemplate() {
JmsTemplate jmsTemplate = new JmsTemplate();
jmsTemplate.setConnectionFactory(connectionFactory());
jmsTemplate.setDefaultDestination(requestMqQueue());
return jmsTemplate;
}
The issue that I am currently having is that my spring container loading process gets stuck if the ActiveMQ is not started before running the application.
I believe that the DefaultMessageListenerContainer and JmsTemplate are trying to create their connections and session to the ActiveMQConnectionFactory.
Outside of spring, I know if the activemq provided isnt running,
activeMQConnection.createSession()
is where the execution would get stuck. In a regular Java code I can timeout some long processing/potentially stuck process. But how can I do something like that in spring container?
I would like to know if there is any better way to declare these beans so that I would know if the activemq is stuck and the container doesn't get stuck?
Thanks in advance for any help.
Update 1:
I updated my Connection URL for Connection Factory and also added an ExceptionListener:
#Bean
public ActiveMQConnectionFactory connectionFactory() {
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(String.format("failover://(%s)?startupMaxReconnectAttempts=1&maxReconnectAttempts=2", ActiveMQConnectionFactory.DEFAULT_BROKER_BIND_URL));
factory.setRedeliveryPolicy(desiredRedeliveryPolicy());
factory.setExceptionListener(factoryExceptionListener());
return factory;
}
public FactoryExceptionListener factoryExceptionListener(){
return new FactoryExceptionListener();
}
public class FactoryExceptionListener implements ExceptionListener {
private static XLogger LOG = XLoggerFactory.getXLogger(FactoryExceptionListener.class);
#Override
public void onException(JMSException exception) {
LOG.error("Factory Exception Caught: "+exception.getMessage());
System.exit(1);
}
}
Now a stupid Question.
I can see the error log getting printed, but the application isn't exiting after System.exit(1). Am I doing something wrong here?
This change helped with the blocking call not being blocking anymore. But I am not able to exit and that means application starts execution and throws a bunch of exceptions as activeMQ isnt available.
What I instead want it to (for now) crash the application. How can I do that?
Update 2:
Instead of exiting out of the application (which is still not working - maybe something to do with the listener) I changed the Exception Listener to make a little more sense for my implementation. I am now trying to get the Broker Up if exception listener is triggered.
public void onException(JMSException exception) {
LOG.error("Factory Exception Caught: "+exception.getMessage());
try {
BrokerService brokerService = new BrokerService();
brokerService.addConnector("tcp://localhost:61616");
brokerService.setDataDirectory("C:/temp/data");
brokerService.setEnableStatistics(true);
brokerService.setPersistent(true);
brokerService.start();
} catch (Exception e) {
e.printStackTrace();
}
}
But I am getting following exception:
2014-07-16 10:24:35.009 [ActiveMQ Task-1] ERROR o.a.a.t.failover.FailoverTransport - Failed to connect to [tcp://localhost:61616] after: 1 attempt(s)
2014-07-16 10:24:35.012 [ActiveMQ Connection Executor: unconnected] ERROR c.b.s.o.b.m.FactoryExceptionListener - Factory Exception Caught: Connection refused: connect
Exception in thread "ActiveMQ Connection Executor: unconnected" java.lang.NoSuchMethodError: org.apache.activemq.transport.TransportFactory.bind(Lorg/apache/activemq/broker/BrokerService;Ljava/net/URI;)Lorg/apache/activemq/transport/TransportServer;
at org.apache.activemq.broker.BrokerService.createTransportConnector(BrokerService.java:2249)
at org.apache.activemq.broker.BrokerService.addConnector(BrokerService.java:291)
at org.apache.activemq.broker.BrokerService.addConnector(BrokerService.java:281)
at com.bhn.service.ordermgmt.bulkorder.mq.FactoryExceptionListener.onException(FactoryExceptionListener.java:19)
at org.apache.activemq.ActiveMQConnection$5.run(ActiveMQConnection.java:1998)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
However when I tried to execute the same code from another project. I was successfully able to get the BrokerService up and running.
I am not sure what this error means and how to resolve it?
Update 3:
Not sure what was wrong earlier but the same code is working now. Thanks for your help #Tim
The reason for this is that the default URL that you are specifying is using the Failover transport. By default the transport will try to connect to the broker until you shut down your app. The createSession call is triggering the client to try and send its connection information request to the Broker but that can't happen until the client connects to the Broker.
One solution which was stated in the comments is to disable the auto startup feature so that the session create calls doesn't get executed on startup. You will however still run into the hang if you later if you trigger a session create while the broker is still down. You can configure the failover transport with a set number of connection attempts using the options shown on the failover transport page.
If you just want to stop the process of connecting if first instance failed,
You can set parameter useKeepAlive=false
I tried other parameters Timeout etc. does not seems to be working.
useKeepAlive works for me.
Newbie to RabbitMQ and new to Java.
I'm attempting to write a listener that will use manual acks and handle consumer cancellation notifications using the java Spring AMQP abstraction. Can I accomplish both tasks by using the Spring abstraction?
I want to write a listener that will pull messages from a queue and process that message (maybe write to a database or something). I planned on using manual acknowledgements so that if processing of the message fails or can't be completed for some reason, I can reject and requeue. So far I think I've found that in order to manually ack/nack/reject using Spring AMQP I have to use a ChannelAwareMessageListener.
I realize that I should be handling Consumer Cancellation Notifications from RabbitMQ, however using the ChannelAwareMessageListener I don't really see a way to code for this. The only way I see to handle CCN is to write code using the lower level java client api by calling channel.basicConsume() and passing a new DefaultConsumer instance which allows you to handle message delivery and cancels.
I also don't see how I would set the clientProperties on the ConnectionFactory (to tell the broker I can handle the CCN) since I am getting the factory from a bean in config.
My pseudo code of the listener and creation of container is below.
public class MyChannelAwareListener implements ChannelAwareMessageListener
{
#Override
public void onMessage(Message message, Channel channel) throws Exception
{
msgProcessed = processMessage(message);
if(msgProcessed)
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
else
channel.basicReject(message.getMessageProperties().getDeliveryTag(), true);
}
}
public static void main(String[] args) throws Exception
{
ConnectionFactory rabbitConnectionFactory;
ClassPathXmlApplicationContext ctx = new ClassPathXmlApplicationContext (MY_CONTEXT_PATH);
rabbitConnectionFactory = (ConnectionFactory)ctx.getBean("rabbitConnectionFactory");
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
MyChannelAwareListener listener = new MyChannelAwareListener();
container.setMessageListener(listener);
container.setQueueNames("myQueue");
container.setConnectionFactory(rabbitConnectionFactory);
container.setAcknowledgeMode(AcknowledgeMode.MANUAL);
container.start();
}
For setting the client properties you need to use the setClientProperties method in the ConnectionFactory (assuming this ConnectionFactory is the object from the RabbitMQ Java library). This method is expecting a Map<String, Object> which contains the properties of the client and the capabilities. The following lines are the default values inside the RabbitMQ Java library:
Map<String,Object> props = new HashMap<String, Object>();
props.put("product", LongStringHelper.asLongString("RabbitMQ"));
props.put("version", LongStringHelper.asLongString(ClientVersion.VERSION));
props.put("platform", LongStringHelper.asLongString("Java"));
props.put("copyright", LongStringHelper.asLongString(Copyright.COPYRIGHT));
props.put("information", LongStringHelper.asLongString(Copyright.LICENSE));
Map<String, Object> capabilities = new HashMap<String, Object>();
capabilities.put("publisher_confirms", true);
capabilities.put("exchange_exchange_bindings", true);
capabilities.put("basic.nack", true);
capabilities.put("consumer_cancel_notify", true);
props.put("capabilities", capabilities);
For managing ACKs and Consumer cancelling I am not sure how to do it with the Spring AMQP abstraction, however it is perfectly doable with channel.basicConsume which gives you the possibility to handle all the scenarios through all the callback methods:
http://www.rabbitmq.com/releases/rabbitmq-java-client/v3.1.5/rabbitmq-java-client-javadoc-3.1.5/
Hope this helps!