I'm using ActiveMQ with the spring framework.
I have two consumers setup in the jms container. When I send 4 message to the queue, and some of the message are transferred to the "Dispatched Queue", because it takes a long time to the consumer to process the message.
I'm trying to find the way to prevent the message from going to the "Dispatched Queue", that is, I want them to be available to any consumer that is ready to consume the message.
I tried to set pre-fetch to 0, but it doesn't seem to work at all.
<bean id="prefetchPolicy" class="org.apache.activemq.ActiveMQPrefetchPolicy">
<property name="queuePrefetch" value="0"/>
</bean>
<bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<constructor-arg index="0" value="tcp://localhost:61616" />
<property name="prefetchPolicy" ref="prefetchPolicy"/>
</bean>
The following is the setup for my jms container:
<bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="destination" ref="defaultDestination" />
<property name="messageListener" ref="messageListener" />
<property name="concurrentConsumers" value="2" />
</bean>
I found the problem. I had the same beans declared twice at two different places. The second bean that was loaded did not have pre-fetch set to 0 and therefore it didn't work out.
The above setup I posted works!
Thank you!
Related
I am using Jboss-fuse-6.3, with external Apache-activemq-5.15.2.
I have bind 50 consumers on a queue, and on Active MQ portal in "Active Consumers" page, I have noticed that all 50 consumers are bind but message distribution on queues are not the same.
Screen shot is attached. On session id "1" En-queued message count is around 1010 but on other consumer sessions En-queued messages are only 10.
Im en-queuing messages from Apache Camel Route. Below is my Blueprint xml (Am i doing some thing wrong)
<bean class="org.apache.activemq.spring.ActiveMQConnectionFactory" id="connectionFactory">
<property name="brokerURL" value="tcp://localhost:61616"/>
<property name="userName" value="admin"/>
<property name="password" value="admin"/>
<property name="trustAllPackages" value="true"/>
</bean>
<bean class="org.apache.camel.component.jms.JmsConfiguration" id="jmsConfig">
<property name="connectionFactory" ref="connectionFactory"/>
</bean>
<bean class="org.apache.activemq.camel.component.ActiveMQComponent" id="activemq">
<property name="configuration" ref="jmsConfig"/>
</bean>
<bean class="org.apache.activemq.spring.ActiveMQConnectionFactory" id="connectionFactory">
<property name="brokerURL" value="tcp://localhost:61616"/>
<property name="userName" value="admin"/>
<property name="password" value="admin"/>
<property name="trustAllPackages" value="true"/>
</bean>
<bean class="org.apache.camel.component.jms.JmsConfiguration" id="jmsConfig">
<property name="connectionFactory" ref="connectionFactory"/>
</bean>
<bean class="org.apache.activemq.camel.component.ActiveMQComponent" id="activemq">
<property name="configuration" ref="jmsConfig"/>
</bean>
<!-- ENQUEUEING MESSAGES -->
<to pattern="InOnly" uri="activemq:queue:MyQueue"/>
<!-- DEQUEUEING MESSAGES -->
<fromuri="activemq:queue:MyQueue?concurrentConsumers=50"/>
This appears to be normal prefetching behavior on the part of the JMS client. If you want competing consumers with fair dispatching then you need to lower the prefetch level as the first consumer that connects will often get a higher number of message dispatched to it because the default prefetch is 1000 for a Queue consumer.
Read more about ActiveMQ consumer prefetch here.
I have a component which reads messages from a queue and meanwhile sends the processed messages to another queue. Therefore, this component is both a message consumer and producer. For configuring them, I need a connection factory for consuming and another connection factory for the producing. Here is part of the spring configuration.
<!-- Configuration for listener -->
<bean id="mdc.TargetConnectionFactory4Listener" class="com.tibco.tibjms.TibjmsConnectionFactory">
<property name="serverUrl" value="tcp://localhost:7222"/>
</bean>
<bean id="mdc.ConnectionFactory4Listener" class="org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter">
<property name="targetConnectionFactory" ref="mdc.TargetConnectionFactory"/>
<property name="username" value="admin" />
<property name="password" value="test" />
</bean>
<bean id="mdc.InputQueue" class="com.tibco.tibjms.TibjmsQueue">
<constructor-arg>
<value>INPUT_QUEUE</value>
</constructor-arg>
</bean>
<bean id="mdc.JmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="mdc.ConnectionFactory4Listener" />
<property name="destination" ref="mdc.InputQueue" />
<property name="messageListener" ref="mdc.MessageReceiver" />
......
</bean>
<!-- Configuration for sender -->
<bean id="mdc.TargetConnectionFactory4Sender" class="com.tibco.tibjms.TibjmsQueue">
<property name="serverUrl" value="tcp://localhost:7222"/>
</bean>
<bean id="mdc.CachingConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="mdc.TargetConnectionFactory4Sender" />
<property name="sessionCacheSize" value="50" />
</bean>
<bean id="mdc.OutputQueue" class="com.tibco.tibjms.TibjmsQueue">
<constructor-arg>
<value>DISCOVERY_QUEUE</value>
</constructor-arg>
</bean>
<bean id="mdc.JmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="mdc.CachingConnectionFactory" />
</bean>
<bean id="mdc.MessageReceiver" class="net.siemens.discovery.queue.QueueMessageListener">
<property name="jmsTemplate" ref="mdc.JmsTemplate" />
<property name="destination" ref="mdc.OutputQueue" />
......
</bean>
The two queues are running on the same EMS server. Some has opinions about this configurations: they can be configured by using only one ConnectionFactory and two instances are not necessary. However, if I use one ConnectionFactory instance, then the instance is used both in the DefaultMessageListenerContainer and the CachingConnectionFactory (further used in JmsTemplate). I don't know whether they have impact on each other.
It is perfectly normal to use a single connection factory; it is very unusual to use 2 factories in this case.
In fact, if you want to perform JmsTemplate operations on the container thread and you want the interactions to run in a transaction (sessionTransacted = true in the container), then you must use the same connection factory. This allows everything to roll back if there's an exception.
When using a caching connection factory in the listener container you should set the connection factory cacheConsumers to false. (See this answer for more information.
I would like to configure number of consumers for a single queue in jms template. I tried to implement this: JmsTemplate - define concurrency per queue? but in the ActiveMQQueue is stiil shows 1.
my messanging context:
<bean id="parentContainer" abstract="true"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory" />
</bean>
<bean id="parentContainer" abstract="true"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory" />
</bean>
<bean id="playerStatsListener" parent="parentContainer">
<property name="destination" ref="playerStatsQueue" />
<property name="messageListener" ref="playerStatsService" />
<property name="concurrency" value="100" />
</bean>
<!-- Listeners -->
<bean id="playerStatsService" class="com.XXX.service.PlayerStatsService" />
<!-- Destinations -->
<bean id="playerStatsQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="playerStatsQueue" />
another related question: Is there a difference between the "listener" to "messageListener" property?
In general, you can set concurrentConsumers and maxConcurrentConsumers on the DefaultMessageListenerContainer. The concurrency setting gives this as a convenient range.
But according to the docs
Specify concurrency limits via a "lower-upper" String, e.g. "5-10", or
a simple upper limit String, e.g. "10" (the lower limit will be 1 in
this case).
So if you just say concurrency=100, it's still equivalent to 1-100, which would explain why you're seeing 1 consumer.
I have two Java standalone applications. I want to send message from one application and asynchronously receive message by two clients: one is in the same application as the sender. The other is in a different application. Both are connected with ActiveMQ broker. But I can only see that the first client got the message while the other didn't get the message. What is the general approach to connect two applications by JMS? I think I must have some unclear concept about JMS. I looked up around but couldn't figure out how to set up my Spring bean configuration file to publish/subscribe message between two Java applications.
Here is my sender's bean configuration file in the first application and also the first listener's bean which is in the same application:
<bean id="customerMessageSender" class="com.example.message.CustomerStatusSender">
<property name="jmsTemplate" ref="jsmTemplateBean" />
<property name="topic" ref="topicBean" />
</bean>
<bean id="jsmTemplateBean" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="connectionFactoryBean"/>
<property name="pubSubDomain" value="true"/>
</bean>
<bean id="topicBean" class="org.apache.activemq.command.ActiveMQTopic">
<constructor-arg value="CustomerStatusTopic" />
</bean>
<bean id="connectionFactoryBean" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616" />
</bean>
<bean id="customerStatusListener" class="com.example.message.CustomerStatusListener" />
<bean id="listenerContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactoryBean" />
<property name="destination" ref="topicBean" />
<property name="messageListener" ref="customerStatusListener" />
</bean>
Here is the bean configuration file for the second listener which is in a different application:
<bean id="topicBean" class="org.apache.activemq.command.ActiveMQTopic">
<constructor-arg value="CustomerStatusTopic" />
</bean>
<bean id="connectionFactoryBean" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616" />
</bean>
<bean id="anotherCustomerStatusListener" class="com.mydomain.jms.CustomerStatusMessageListener" />
<bean id="listenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactoryBean" />
<property name="destination" ref="topicBean" />
<property name="messageListener" ref="anotherCustomerStatusListener" />
</bean>
As you can see, both customerStatusListener bean and anotherCustomerStatusListener bean subscribe to the topicBean. But only the first listener gets the message since it is in the same application as the sender while the second does not. What is the general principle to connect two Java applications by JMS so that message can be sent/received between two separate applications?
Edit: I can't add the following listener bean in the first XML file since the class CustomerStatusMessageListener is in a different application therefore is not visible in the classpath of the first (sender's) application.
<bean id="anotherCustomerStatusListener" class="com.mydomain.jms.CustomerStatusMessageListener" />
Edit again: The following is the second listener in the second application which is separate from the first application. It contains a main method to instantiate the listener bean (jms-beans.xml is the bean configuration file for the second listener as listed above).
public class CustomerStatusMessageListener implements MessageListener {
public void onMessage(Message message) {
if (message instanceof TextMessage) {
try {
System.out.println("Subscriber 2 got you! The message is: "
+ ((TextMessage) message).getText());
} catch (JMSException ex) {
throw new RuntimeException(ex);
}
} else {
throw new IllegalArgumentException(
"Message must be of type TextMessage");
}
}
public static void main(String[] args) {
ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext("jms-beans.xml");
CustomerStatusMessageListener messageListener = (CustomerStatusMessageListener) context.getBean("anotherCustomerStatusListener");
context.close();
}
}
Your understanding of JMS is correct. If you want two listeners to receive the same message, then a topic is the way to do it. It shouldn't matter if one listener is running in the same VM as the sender and the other one is not. Without seeing your code, your Spring configuration also looks correct. That leaves a few more things to check:
Are both listeners running on the same host? That is, is localhost one place for one of the listeners and a different place for the other listener?
Is your second listener running at the time the message was sent? If your second listener isn't active when the message is sent, it won't see if it starts later, unless you're using durable topics AND your subscriber has connected to the broker at least once.
Based on your comments, the second item is where you are having problems.
This blog post tells how to set up durable topics (and persist messages if you need messages to persist through broker restarts). Basically, add this configuration to your message listeners:
<property name="subscriptionDurable" value="true">
<property name="clientId" value="Some_unique_id">
<property name="durableSubscriptionName" value="Some_unique_id">
The client id and durable subscription name must be different for every subscriber, so your first listener would have something like:
<bean id="listenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactoryBean" />
<property name="destination" ref="topicBean" />
<property name="messageListener" ref="anotherCustomerStatusListener" />
<property name="subscriptionDurable" value="true">
<property name="clientId" value="listener1">
<property name="durableSubscriptionName" value="listener1">
</bean>
And second should have:
<bean id="listenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactoryBean" />
<property name="destination" ref="topicBean" />
<property name="messageListener" ref="anotherCustomerStatusListener" />
<property name="subscriptionDurable" value="true">
<property name="clientId" value="listener2">
<property name="durableSubscriptionName" value="listener2">
</bean>
Note that you must start your second listener at least once to register itself with the broker so the broker knows its clientId and will store messages for it, but you can then shut it down and start it later to get any messages it missed while it was down.
If your listener stays down for a long time on a high-volume system, the broker will store all messages for it, which might eventually fill the disk or slow the broker down. See the ActiveMQ documentation for automatically removing durable messages.
<bean id="myTopic" class="org.apache.activemq.command.ActiveMQTopic">
<property name="physicalName" value="feed.topic" />
</bean>
<bean id="myConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="failover:tcp://localhost:61616" />
</bean>
<bean id="myJmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="myConnectionFactory" />
<property name="defaultDestination" ref="myTopic" />
</bean>
<bean id="sender" class="com.feed.publish.PublishMessages">
<property name="jmsTemplate" ref="myJmsTemplate" />
</bean>
I have the above set up using the spring framework that allows me to publish messages to a queue. However, if the activemq instance is terminated mid process, I would like it to write to disk/file the messages until a connection can be reestablished. I have found sample code off of the website of activemq however I am unsure how I integrate this in to my current setup
<amq:broker useJmx="true" persistent="true" brokerName="localhost">
<amq:persistenceAdapter>
<amq:kahaPersistenceAdapter directory="activemq-data"
maxDataFileLength="33554432" />
</amq:persistenceAdapter>
<amq:transportConnectors>
<amq:transportConnector name="vm" uri="vm://localhost" />
</amq:transportConnectors>
</amq:broker>
Can someone please tell me how I go about merging these two styles? Thanks
the AMQ persistenceAdapter configuration is to allow the AMQ Broker to persist messages to disk, not the client. If the broker connection is terminated, your client code should catch the exception, write the messages to disk and provide a means to replay them at a later time.
along these lines, I generally use Apache Camel's exception handling and file components to handle these type of scenarios...