Spring JMS with Atomikos, poolsize and concurrent consumers - java

Can someone explain the semantic differences between the poolsize of a Atomikos connection factory, the pool size of the nested connection factory and concurrent consumers of a JMS Container?
Consider the following Spring config:
// ActiveMq vendor-specific Connection Factory
#Bean
public ConnectionFactory activeMQXAConnectionFactoryBean() {
ActiveMQXAConnectionFactory xaCF = new ActiveMQXAConnectionFactory(brokerUrl);
xaCF.setMaxThreadPoolSize(10);
return xaCF;
}
// Wrap it in an Atomikos ConnectionFactory
#Bean(initMethod = "init", destroyMethod = "close")
public ConnectionFactory atomikosConnectionFactoryBean(ConnectionFactory activeMQXAConnectionFactoryBean) {
AtomikosConnectionFactoryBean atomikosCF = new AtomikosConnectionFactoryBean();
atomikosCF.setUniqueResourceName("amq1");
atomikosCF.setXaConnectionFactory(xaConnectionFactory);
atomikosCF.setPoolSize(10);
return atomikosCF;
}
// wire it into Spring's MessageListenerContainer
#Bean
public MessageListenerContainer defaultMessageListenerContainer (ConnectionFactory atomikosConnectionFactoryBean) {
DefaultMessageListenerContainer mlc = new DefaultMessageListenerContainer();
mlc.setConnectionFactory(atomikosConnectionFactoryBean);
mlc.setConcurrency("3-10");
return mlc;
}
Can someone explain how the three settings affect JMS connections? For example should the poolsize of the connection factories here always be the same? What happens if they are not the same?

Related

ActiveMQ Consumer failed to consume message on request queue

Process:
App A is creating a consumer connection on startup to "REQUEST QUEUE"
in remote Active MQ Broker. App B is pushing request into the "REQUEST
QUEUE", and the same is consumed by App A. App A produces response
into another queue "RESPONSE QUEUE".
App A is using ActiveMQ connection factory and Spring DMLC for
consuming the message.
What I understand from Spring DMLC is, it continuously polls on queue
for message.
Problem:
App A initially consumes request on app restart. But fails to consume the request second time when an another request is made a day after. Even the consumer goes down on the arrival of the second request.
So far tried:
The issue exists only in LIVE.And as per observation consumer is not able process/receive incoming message after a certain period(30 minutes).
I tried with same broker setup in other environments, but unfortunately couldn't re-produce. I have tried every other link but all in vain.
Looking out for pointers:
Is there something fundamentally wrong with the way I'm making use of
ActiveMQ connection for both consuming and producing ?
Possible pointers on what configuration to look out for debugging the issue.
Code:
Message Poller Configuration Code:
#Configuration
#EnableJms
#ComponentScan
#Slf4j
public class MessagePoller
{
#Value("${outbound.queue}")
private String outboundQueue;
private final BrokerProperties brokerProperties;
#Autowired
public MessagePoller(BrokerProperties brokerProperties) {this.brokerProperties = brokerProperties;}
#Bean(name = CONNECTION_FACTORY)
#Primary
public ActiveMQConnectionFactory connectionFactory()
{
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(brokerProperties.getUrl());
connectionFactory.setUserName(brokerProperties.getUser());
connectionFactory.setPassword(brokerProperties.getPassword());
return connectionFactory;
}
#Bean
public JmsListenerContainerFactory jmsListenerContainerFactory()
{
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrency("1-1");
factory.setErrorHandler(getErrorHandler());
factory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
factory.setMessageConverter(new SimpleMessageConverter());
factory.setPubSubDomain(false);
return factory;
}
#Bean(name = OUTBOUND_JMS_TEMPLATE)
public JmsTemplate outboudJmsTemplate(
#Qualifier(CONNECTION_FACTORY)
ConnectionFactory connectionFactory)
{
JmsTemplate template = new JmsTemplate(connectionFactory);
template.setPubSubDomain(false);
template.setDefaultDestinationName(outboundQueue);
return template;
}
private ErrorHandler getErrorHandler()
{
return exception -> log.error("Exception thrown from consumer", exception);
}
}
Message Listener Code:
#JmsListener(
destination = "requestQueue",
containerFactory = "jmsListenerContainerFactory"
)
public void onMessage(Message<String> jmsMessage)
{
log.info("TriggerJobOnRequestService.onMessage() starts");
log.info("Incoming request message: {}", jmsMessage);
Integer deliveryCount = jmsMessage.getHeaders().get("JMSXDeliveryCount", Integer.class);
log.info("Payload : {}", jmsMessage.getPayload());
log.info("Headers : {}", jmsMessage.getHeaders());
log.info("Delivery Count : {}", deliveryCount);
//Processing Code logic
log.info("TriggerJobOnRequestService.onMessage() ends");
}
ActiveMQ connection url:
spring.activemq.url=failover://(tcp://mqueue05.net:61616,tcp://mqueue06.net:61616,tcp://mqueue07.net:61616,tcp://mqueue08.net:61616)?randomize=false&timeout=100&initialReconnectDelay=1000&maxReconnectDelay=1000&maxReconnectAttempts=10

Artemis durable subscriptions for multicast address with JMS

With JMS I want to create some durable subscriptions for a topic (multicast address). In case of one duarble subscriptions it works, but in case of more, it does not and errors occur.
These are my listeners: maybe the properties are not correctly filled?
#JmsListener(destination = "VirtualTopic.test", id = "c1", subscription = "Consumer.A.VirtualTopic.test", containerFactory = "queueConnectionFactory")
public void receive1(String m) {
}
#JmsListener(destination = "VirtualTopic.test", id = "c2", subscription = "Consumer.B.VirtualTopic.test", containerFactory = "queueConnectionFactory")
public void receive2(String m) {
}
This is the listenerFactory: I'm not sure about the last property.
#Bean
public DefaultJmsListenerContainerFactory queueConnectionFactory() {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setClientId("brokerClientId");
factory.setSubscriptionDurable(true);
factory.setSubscriptionShared(true); **<-- needed for my case?**
return factory;
}
#Bean
public ActiveMQConnectionFactory connectionFactory() {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(brokerUrl);
return connectionFactory;
}
These are the error logs, when I set "factory.setSubscriptionShared(true);":
2020-04-17 11:23:44.485 WARN 7900 --- [enerContainer-3] o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener invoker failed for destination 'VirtualTopic.test' - trying to recover. Cause: org.apache.activemq.ActiveMQSession.createSharedDurableConsumer(Ljavax/jms/Topic;Ljava/lang/String;Ljava/lang/String;)Ljavax/jms/MessageConsumer;
2020-04-17 11:23:44.514 ERROR 7900 --- [enerContainer-3] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'VirtualTopic.test' - retrying using FixedBackOff{interval=5000, currentAttempts=0, maxAttempts=unlimited}. Cause: Broker: d1 - Client: brokerClientId already connected from /127.0.0.1:59979
As noted by the JMS specification, only one client with the same ID can connect. You're apparently using the same client ID for all your connections, i.e.:
factory.setClientId("brokerClientId");
Try not setting the client ID and see how that goes.
Also, ensure you're using a JMS client implementation that actually supports JMS 2.0 (e.g. the ActiveMQ Artemis core JMS client).

Spring Boot multiple JMS connections

I'm developing Spring Boot application which must connect to several WebSphere JMS connections with different ports or even ip addresses. I need receive and send messages to different queues.
I took example of connection from this source - https://github.com/lzp4ever/IBM_WebSphere_MQ_Spring_Boot_JMS
But when i add second connectionFactory Spring Boot failes to start, it just don't know which once to use.
My question is How should i configure my config file to listen several queues? Is it good idea connecting SpringBoot app to several different JMS servers?
Solution
i just copy and paste same beans(like at git link above) second time and add Bean(name) to separate them. It was not work and then i added new JmsListenerContainerFactory bean to each of my config file.
One of my config file is:
#Bean(name = "mqQueueConnectionFactory2")
public MQQueueConnectionFactory mqQueueConnectionFactory2() {
MQQueueConnectionFactory mqQueueConnectionFactory = new MQQueueConnectionFactory();
mqQueueConnectionFactory.setHostName(host);
try {
mqQueueConnectionFactory.setTransportType(WMQConstants.WMQ_CM_CLIENT);
mqQueueConnectionFactory.setCCSID(1208);
mqQueueConnectionFactory.setChannel(channel);
mqQueueConnectionFactory.setPort(port);
mqQueueConnectionFactory.setQueueManager(queueManager);
} catch (Exception e) {
logger.error("MQQueueConnectionFactory bean exception", e);
}
return mqQueueConnectionFactory;
}
#Bean(name = "userCredentialsConnectionFactoryAdapter2")
UserCredentialsConnectionFactoryAdapter userCredentialsConnectionFactoryAdapter2(#Qualifier("mqQueueConnectionFactory2") MQQueueConnectionFactory mqQueueConnectionFactory) {
UserCredentialsConnectionFactoryAdapter userCredentialsConnectionFactoryAdapter = new UserCredentialsConnectionFactoryAdapter();
userCredentialsConnectionFactoryAdapter.setUsername(username);
userCredentialsConnectionFactoryAdapter.setPassword(password);
userCredentialsConnectionFactoryAdapter.setTargetConnectionFactory(mqQueueConnectionFactory);
return userCredentialsConnectionFactoryAdapter;
}
#Bean(name = "cachingConnectionFactory2")
//#Primary
public CachingConnectionFactory cachingConnectionFactory2(#Qualifier("userCredentialsConnectionFactoryAdapter2") UserCredentialsConnectionFactoryAdapter userCredentialsConnectionFactoryAdapter) {
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory();
cachingConnectionFactory.setTargetConnectionFactory(userCredentialsConnectionFactoryAdapter);
cachingConnectionFactory.setSessionCacheSize(500);
cachingConnectionFactory.setReconnectOnException(true);
return cachingConnectionFactory;
}
#Bean(name = "jmsTransactionManager2")
public PlatformTransactionManager jmsTransactionManager2(#Qualifier("cachingConnectionFactory2") CachingConnectionFactory cachingConnectionFactory) {
JmsTransactionManager jmsTransactionManager = new JmsTransactionManager();
jmsTransactionManager.setConnectionFactory(cachingConnectionFactory);
return jmsTransactionManager;
}
#Bean(name = "jmsOperations2")
public JmsOperations jmsOperations2(#Qualifier("cachingConnectionFactory2") CachingConnectionFactory cachingConnectionFactory) {
JmsTemplate jmsTemplate = new JmsTemplate(cachingConnectionFactory);
jmsTemplate.setReceiveTimeout(receiveTimeout);
return jmsTemplate;
}
#Bean
public JmsListenerContainerFactory<?> myFactory2(#Qualifier("cachingConnectionFactory2") CachingConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
// This provides all boot's default to this factory, including the message converter
configurer.configure(factory, connectionFactory);
// You could still override some of Boot's default if necessary.
return factory;
}
Then i change my sender code from this:
#Autowired
private JmsOperations jmsOperations;
to this
#Autowired
#Qualifier("jmsOperations2")
private JmsOperations jmsOperations;
also i change my receiver to:
#JmsListener(destination = "${project.queues.uzb.recieve}", containerFactory = "myFactory2")
public void receiveMessage(JMSTextMessage data) {
}
it seems to me it worked!!!
But one of my CachingConnectionFactory must be marked as #Primary. If i delete #Primaty from one of my config files then i am gettig this error:
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2018-03-28 12:28:37 -
APPLICATION FAILED TO START
Description:
Parameter 1 of method myFactory in com.config.UzbConnection required a bean of type 'org.springframework.boot.autoconfigure.jms.DefaultJmsListenerContainerFactoryConfigurer' that could not be found.
Action:
Consider defining a bean of type 'org.springframework.boot.autoconfigure.jms.DefaultJmsListenerContainerFactoryConfigurer' in your configuration.
Thanks
Just my 2 cents. If you have problems with multiple JMS connections because you have a project using a mix of Spring-boot and JMS with Spring xml configuration to make your connection factory, you can disable autostart of spring-boot-jms with this in your application:
#SpringBootApplication(exclude = {JmsAutoConfiguration.class})
this way you can mix both.

Load Balancing Client Sessions

I have a 2 server ActiveMQ Artemis cluster setup with discovery using JGroups working correctly. Then in my applications the ConnectionFactory is created via the ActiveMQJMSClient:
final ActiveMQConnectionFactory cf = ActiveMQJMSClient.createConnectionFactory(
"jgroups://test-cluster?file=jgroups-file-ping.xml&connectionLoadBalancingPolicyClassName=org.apache.activemq.artemis.api.core.client.loadbalance.RoundRobinConnectionLoadBalancingPolicy", "test-cluster");
Then the producers and consumers are handled with Spring. However, when Spring creates 10 consumers on start-up I'm seeing that all 10 consumers go to the same Artemis server.
Here is the Spring JMS Configuration:
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() throws Exception {
final ConnectionFactory cf = getActiveMQConnectionFactory();
final DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(cf);
factory.setConcurrency("10-20");
factory.setSessionTransacted(true);
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
return factory;
}
Then on application startup I call this within the JmsListenerConfigurer:
public static void registerEndPoint(final JmsListenerEndpointRegistrar registrar,
String endPointName, String dest, MessageListener listener) {
final SimpleJmsListenerEndpoint endpoint = new SimpleJmsListenerEndpoint();
endpoint.setId(endPointName);
endpoint.setDestination(dest);
endpoint.setMessageListener(listener);
registrar.registerEndpoint(endpoint);
}
Is there a way of setting this up so that there are 5 consumers on each Artemis server?
Spring's DefaultJmsListenerContainerFactory has a cache level that can be changed to CACHE_NONE,CACHE_CONNECTION,CACHE_SESSION or CACHE_AUTO.
I had to set it to CACHE_NONE for Spring to create more than a single connection in the connection factory.

Load spring context even if rabbitmq is down

Is it possible to load spring context with #RabbitListener even if RabbitMQ message broker is down?
The behavior should be similar as in case of broker disconnection. Application is waiting for broker and when it is restored, then listerner is automaticaly reconnected.
Spring Boot 1.3.2.RELEASE
GitHub demo project
spring-amqp configuration:
#Bean
public MessageConverter jsonMessageConverter() {
return new JsonMessageConverter();
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(host);
connectionFactory.setUsername(username);
connectionFactory.setPassword(password);
return connectionFactory;
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrentConsumers(10);
factory.setMaxConcurrentConsumers(10);
factory.setMessageConverter(jsonMessageConverter());
return factory;
}
#Bean
public AmqpAdmin amqpAdmin() {
RabbitAdmin rabbitAdmin = new RabbitAdmin(connectionFactory());
rabbitAdmin.setIgnoreDeclarationExceptions(true); // useless
return rabbitAdmin;
}
listener configuration:
#Service
public class CalculatorServiceV2 {
#RabbitListener(bindings = #QueueBinding(
value = #Queue(value = "calc_service_v2.multiply", durable = "false", autoDelete = "true"),
exchange = #Exchange(value = "calc_service_v2", durable = "false", autoDelete = "true"),
key = "multiply"))
public Result multiply(Operands operands) {
// do something..
}
}
What version of Spring AMQP are you using?
Can you provide a stack trace? (edit the question, don't try to put it in a comment).
I just ran a test and it works as expected; the listener container attempts to reconnect every 5 seconds (the default recovery interval).
rabbitAdmin.setIgnoreDeclarationExceptions(true); // useless
In 1.6 (currently at milestone 1) we've changed that boolean to skip all exceptions; previously it only applied to exceptions caused by errors returned from the broker.
However, the admin won't attempt to declare the elements until the connection is established so that should be moot in this context.
I need to see a stack trace to understand what's going on in your case.

Categories

Resources