I am using Jboss-fuse-6.3, with external Apache-activemq-5.15.2.
I have bind 50 consumers on a queue, and on Active MQ portal in "Active Consumers" page, I have noticed that all 50 consumers are bind but message distribution on queues are not the same.
Screen shot is attached. On session id "1" En-queued message count is around 1010 but on other consumer sessions En-queued messages are only 10.
Im en-queuing messages from Apache Camel Route. Below is my Blueprint xml (Am i doing some thing wrong)
<bean class="org.apache.activemq.spring.ActiveMQConnectionFactory" id="connectionFactory">
<property name="brokerURL" value="tcp://localhost:61616"/>
<property name="userName" value="admin"/>
<property name="password" value="admin"/>
<property name="trustAllPackages" value="true"/>
</bean>
<bean class="org.apache.camel.component.jms.JmsConfiguration" id="jmsConfig">
<property name="connectionFactory" ref="connectionFactory"/>
</bean>
<bean class="org.apache.activemq.camel.component.ActiveMQComponent" id="activemq">
<property name="configuration" ref="jmsConfig"/>
</bean>
<bean class="org.apache.activemq.spring.ActiveMQConnectionFactory" id="connectionFactory">
<property name="brokerURL" value="tcp://localhost:61616"/>
<property name="userName" value="admin"/>
<property name="password" value="admin"/>
<property name="trustAllPackages" value="true"/>
</bean>
<bean class="org.apache.camel.component.jms.JmsConfiguration" id="jmsConfig">
<property name="connectionFactory" ref="connectionFactory"/>
</bean>
<bean class="org.apache.activemq.camel.component.ActiveMQComponent" id="activemq">
<property name="configuration" ref="jmsConfig"/>
</bean>
<!-- ENQUEUEING MESSAGES -->
<to pattern="InOnly" uri="activemq:queue:MyQueue"/>
<!-- DEQUEUEING MESSAGES -->
<fromuri="activemq:queue:MyQueue?concurrentConsumers=50"/>
This appears to be normal prefetching behavior on the part of the JMS client. If you want competing consumers with fair dispatching then you need to lower the prefetch level as the first consumer that connects will often get a higher number of message dispatched to it because the default prefetch is 1000 for a Queue consumer.
Read more about ActiveMQ consumer prefetch here.
Related
I am using Apache Camel(with Spring) and ActiveMQ in project. Here are the settings related to JMS/ActiveMQ:
Camel version: activemq-camel-5.15.3.jar (all ActiveMQ related jars)
ActiveMQ version : 5.15.0
<!-- language: lang-xml -->
<bean id="defaultActiveMQRedeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
</bean>
<util:list id="redeliveryPolicyEntries">
<bean id="activeMQRedeliveryPolicy1" class="org.apache.activemq.RedeliveryPolicy">
<property name="queue" value="inbox"></property>
</bean>
</util:list>
<bean id="amqRedeliveryPolicyMap"
class="org.apache.activemq.broker.region.policy.RedeliveryPolicyMap">
<property name="defaultEntry" ref="defaultActiveMQRedeliveryPolicy"></property>
<property name="redeliveryPolicyEntries" ref="redeliveryPolicyEntries"></property>
</bean>
<bean id="amqPrefetchPolicy" class="org.apache.activemq.ActiveMQPrefetchPolicy">
</bean>
<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
<property name="maxConnections" value="20" />
<property name="maximumActiveSessionPerConnection" value="40" />
<property name="connectionFactory" ref="jmsConnectionFactory">
</property>
</bean>
<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="${jmsConnectionFactory.brokerURL}" />
<property name="userName" value="admin" />
<property name="password" value="admin" />
<property name="prefetchPolicy" ref="amqPrefetchPolicy" />
<property name="redeliveryPolicyMap" ref="amqRedeliveryPolicyMap" />
</bean>
<bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="pooledConnectionFactory" />
<property name="concurrentConsumers" value="15" />
<property name="maxConcurrentConsumers" value="30" />
<property name="asyncConsumer" value="false" />
<property name="cacheLevelName" value="CACHE_CONSUMER" />
</bean>
<!-- this bean actually represents a jms component to be used in our camel-integration
setup.make endpoints by using name(id) of this bean. -->
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration" ref="jmsConfig" />
<property name="transacted" value="false" />
<property name="transactionManager">
<bean class="org.springframework.jms.connection.JmsTransactionManager">
<property name="connectionFactory" ref="jmsConnectionFactory" />
</bean>
</property>
</bean>
As you see I am using PooledConnectionFactory so I am expecting a fixed no of connections to connect with ActiveMQ. But unexpectedly I see a large no of TCP connections being opened in TIME_WAIT even when my application is idle and no messages are being produced/consumed at time. I confirmed this situation with infra team that confirmed all the Operating System level configuration are fine.
Here I tried debugging the doReceiveAndExecute method in AbstractPollingMessageListenerContainer- sessionToUse is not null, consumerToUse is also not null and code flows through receiveMessage(line number 304).I could not find the problem in debug trace as attached in debug screenshots:
and
and my actual problem
Is it a problem with MessageListenerContainer or with ConnectionFactory?? Am I missing some configuration which would prevent this from happening or is this an existing issue? If so is there a workaround?
Just spotted in your configuration that you configured the jmsConnectionFactory (not the pooled factory) in your transaction manager. Not sure if this could raise the issue because the pooled factory is simply not used.
<property name="transactionManager">
<bean class="org.springframework.jms.connection.JmsTransactionManager">
<property name="connectionFactory" ref="jmsConnectionFactory" />
</bean>
</property>
I'm using ActiveMQ with the spring framework.
I have two consumers setup in the jms container. When I send 4 message to the queue, and some of the message are transferred to the "Dispatched Queue", because it takes a long time to the consumer to process the message.
I'm trying to find the way to prevent the message from going to the "Dispatched Queue", that is, I want them to be available to any consumer that is ready to consume the message.
I tried to set pre-fetch to 0, but it doesn't seem to work at all.
<bean id="prefetchPolicy" class="org.apache.activemq.ActiveMQPrefetchPolicy">
<property name="queuePrefetch" value="0"/>
</bean>
<bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<constructor-arg index="0" value="tcp://localhost:61616" />
<property name="prefetchPolicy" ref="prefetchPolicy"/>
</bean>
The following is the setup for my jms container:
<bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="destination" ref="defaultDestination" />
<property name="messageListener" ref="messageListener" />
<property name="concurrentConsumers" value="2" />
</bean>
I found the problem. I had the same beans declared twice at two different places. The second bean that was loaded did not have pre-fetch set to 0 and therefore it didn't work out.
The above setup I posted works!
Thank you!
Message producer component has this :
session = connection.createSession(false,
Session.CLIENT_ACKNOWLEDGE);
producer = session.createProducer(session.createQueue(queueName));
I am using AWS SQS to send messages async
I was under the assumption that the message will be in the queue until it is acknowledged explicitly.(because of the client acknowledgement mode)
However it is removed off the queue.
Message consumer component:
I have written an async consumer to pull messages from the queue.
Here is the spring config file;
<bean id="ConnectionFactoryBuilder"
class="com.amazon.sqs.javamessaging.SQSConnectionFactory$Builder">
<property name="regionName" value="us-east-1" />
<property name="numberOfMessagesToPrefetch" value="1" />
<property name="awsCredentialsProvider" ref="CredentialsProviderBean" />
</bean>
<bean id="ConnectionFactory" class="com.amazon.sqs.javamessaging.SQSConnectionFactory"
factory-bean="ConnectionFactoryBuilder" factory-method="build" />
<bean id="Connection" class="javax.jms.Connection" factory-bean="ConnectionFactory"
factory-method="createConnection" init-method="start" destroy-method="close" />
<bean id="QueueName" class="java.lang.String">
<constructor-arg value="${sqs.queueName}" />
</bean>
<bean id="amazonMessageListener" class="com.myapp.AsyncMessageListener" />
<bean id="messageListener"
class="org.springframework.jms.listener.adapter.MessageListenerAdapter">
<property name="delegate" ref="amazonMessageListener" />
<property name="defaultListenerMethod" value="onMessage" />
<property name="messageConverter">
<null />
</property>
</bean>
<bean id="jmsContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="ConnectionFactory" />
<property name="destinationName" ref="QueueName" />
<property name="messageListener" ref="messageListener" />
<property name="sessionAcknowledgeMode" value="2"/>
</bean>
i configured my consumer with client acknowledgemode using the property sessionAcknowledgeMode. But still the message is removed off the queue without the consumer explicitly acknowledging using message.acknowledge()
Per AbstractMessageListenerContainer (super class of DefaultMessageListenerContainer) doc:
"sessionAcknowledgeMode" set to "CLIENT_ACKNOWLEDGE": Automatic message acknowledgment after successful listener execution; best-effort redelivery in case of a user exception thrown as well as in case of other listener execution interruptions (such as the JVM dying).
So when using DefaultMessageListenerContainer, if onMessage() method in your listener returns without exception, spring acknowledges the message automatically.
I have a component which reads messages from a queue and meanwhile sends the processed messages to another queue. Therefore, this component is both a message consumer and producer. For configuring them, I need a connection factory for consuming and another connection factory for the producing. Here is part of the spring configuration.
<!-- Configuration for listener -->
<bean id="mdc.TargetConnectionFactory4Listener" class="com.tibco.tibjms.TibjmsConnectionFactory">
<property name="serverUrl" value="tcp://localhost:7222"/>
</bean>
<bean id="mdc.ConnectionFactory4Listener" class="org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter">
<property name="targetConnectionFactory" ref="mdc.TargetConnectionFactory"/>
<property name="username" value="admin" />
<property name="password" value="test" />
</bean>
<bean id="mdc.InputQueue" class="com.tibco.tibjms.TibjmsQueue">
<constructor-arg>
<value>INPUT_QUEUE</value>
</constructor-arg>
</bean>
<bean id="mdc.JmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="mdc.ConnectionFactory4Listener" />
<property name="destination" ref="mdc.InputQueue" />
<property name="messageListener" ref="mdc.MessageReceiver" />
......
</bean>
<!-- Configuration for sender -->
<bean id="mdc.TargetConnectionFactory4Sender" class="com.tibco.tibjms.TibjmsQueue">
<property name="serverUrl" value="tcp://localhost:7222"/>
</bean>
<bean id="mdc.CachingConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="mdc.TargetConnectionFactory4Sender" />
<property name="sessionCacheSize" value="50" />
</bean>
<bean id="mdc.OutputQueue" class="com.tibco.tibjms.TibjmsQueue">
<constructor-arg>
<value>DISCOVERY_QUEUE</value>
</constructor-arg>
</bean>
<bean id="mdc.JmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="mdc.CachingConnectionFactory" />
</bean>
<bean id="mdc.MessageReceiver" class="net.siemens.discovery.queue.QueueMessageListener">
<property name="jmsTemplate" ref="mdc.JmsTemplate" />
<property name="destination" ref="mdc.OutputQueue" />
......
</bean>
The two queues are running on the same EMS server. Some has opinions about this configurations: they can be configured by using only one ConnectionFactory and two instances are not necessary. However, if I use one ConnectionFactory instance, then the instance is used both in the DefaultMessageListenerContainer and the CachingConnectionFactory (further used in JmsTemplate). I don't know whether they have impact on each other.
It is perfectly normal to use a single connection factory; it is very unusual to use 2 factories in this case.
In fact, if you want to perform JmsTemplate operations on the container thread and you want the interactions to run in a transaction (sessionTransacted = true in the container), then you must use the same connection factory. This allows everything to roll back if there's an exception.
When using a caching connection factory in the listener container you should set the connection factory cacheConsumers to false. (See this answer for more information.
So far i've only been able to find concurrency setting in the jms connection factory:
<jms:listener-container connection-factory="myConnectionFactory"
task-executor="myTaskExecutor"
destination-resolver="myDestinationResolver"
transaction-manager="myTransactionManager"
concurrency="10">
Is it possible to configure the number of consumers for a single queue? i.e something like:
<jms:listener destination="playerStatsQueue" ref="playerStatsService"
method="onMessage" concurrency="100" />
Thanks!~
Do not use the namespace but an abstract parent DefaultMessageListenerContainer and create one child instance per listener. That way you can tweak all the properties you need.
<bean id="parentContainer" abstract="true"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="messageListener" ref="messageListener"/>
<property name="transactionManager" ref="transactionManager"/>
</bean>
<bean id="playerStatsListener parent="parentContainer">
<property name="destination" ref="playerStatsQueue"/>
<property name="listener" ref="playerStatsService"/>
<property name="concurrency" value="100"/>
</bean>