<bean id="myTopic" class="org.apache.activemq.command.ActiveMQTopic">
<property name="physicalName" value="feed.topic" />
</bean>
<bean id="myConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="failover:tcp://localhost:61616" />
</bean>
<bean id="myJmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="myConnectionFactory" />
<property name="defaultDestination" ref="myTopic" />
</bean>
<bean id="sender" class="com.feed.publish.PublishMessages">
<property name="jmsTemplate" ref="myJmsTemplate" />
</bean>
I have the above set up using the spring framework that allows me to publish messages to a queue. However, if the activemq instance is terminated mid process, I would like it to write to disk/file the messages until a connection can be reestablished. I have found sample code off of the website of activemq however I am unsure how I integrate this in to my current setup
<amq:broker useJmx="true" persistent="true" brokerName="localhost">
<amq:persistenceAdapter>
<amq:kahaPersistenceAdapter directory="activemq-data"
maxDataFileLength="33554432" />
</amq:persistenceAdapter>
<amq:transportConnectors>
<amq:transportConnector name="vm" uri="vm://localhost" />
</amq:transportConnectors>
</amq:broker>
Can someone please tell me how I go about merging these two styles? Thanks
the AMQ persistenceAdapter configuration is to allow the AMQ Broker to persist messages to disk, not the client. If the broker connection is terminated, your client code should catch the exception, write the messages to disk and provide a means to replay them at a later time.
along these lines, I generally use Apache Camel's exception handling and file components to handle these type of scenarios...
Related
I use the Listener to read the messages from MQ as mentioned in the link here
ExampleMessageListener :
This implements the onMessage method of Interface javax.jms.MessageListener
Listener :
<bean id="messageListener" class="sample.ExampleMessageListener" />
<bean id="msgListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsConnectionFactory" />
<property name="destination" ref="jmsQueue" />
<property name="messageListener" ref="messageListener" />
</bean>
<jee:jndi-lookup id="jmsConnectionFactory" jndi-name="jms/CF1" />
<jee:jndi-lookup id="jmsQueue" jndi-name="jms/jmsQueue" />
I want to stop the listener when all the messages in MQ are read? (In other words, stop the listener when no meessages in MQ)
Any best way to identify the unavailability of messages in MQ?
using jms Browser ?
session.createBrowser(session.createQueue("Q"))
https://docs.oracle.com/javaee/7/api/javax/jms/Session.html#createBrowser-javax.jms.Queue-
https://docs.oracle.com/javaee/7/api/javax/jms/QueueBrowser.html#getEnumeration--
to start and stop the DMLC see here control-message-listener-container-to-stop-for-certain-period-and-start-again
Due to scaling reasons we recently switched from a single ActiveMQ broker to a network of brokers. While for the most part everything works exactly as intended there is one weird issue we have encountered after a fresh deployment of the brokers during the day:
First just for the tech stack we are using ActiveMQ 5.12.1 and Camel 2.13.4 for the integration between the java method and a JMS endpoint. The broker side is a network of brokers currently consisting of 3 members using the following configuration
<broker useJmx="${activemq.expose.jmx}" persistent="false"
brokerName="${activemq.brokerName}" xmlns="http://activemq.apache.org/schema/core">
<sslContext>
<amq:sslContext keyStore="${activemq.broker.keyStore}"
keyStorePassword="${activemq.broker.keyStorePassword}"
trustStore="${activemq.broker.trustStore}"
trustStorePassword="${activemq.broker.trustStorePassword}" />
</sslContext>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="${activemq.memoryUsage}" />
</memoryUsage>
<tempUsage>
<tempUsage limit="${activemq.tempUsage}" />
</tempUsage>
</systemUsage>
</systemUsage>
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry queue=">" enableAudit="false">
<networkBridgeFilterFactory>
<conditionalNetworkBridgeFilterFactory
replayWhenNoConsumers="true" />
</networkBridgeFilterFactory>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<networkConnectors>
<networkConnector name="queues"
uri="static:(${activemq.otherBrokers})"
networkTTL="2" dynamicOnly="true"
decreaseNetworkConsumerPriority="true"
conduitSubscriptions="false">
<excludedDestinations>
<topic physicalName=">" />
</excludedDestinations>
</networkConnector>
<networkConnector name="topics"
uri="static:(${activemq.otherBrokers})"
networkTTL="1" dynamicOnly="true"
decreaseNetworkConsumerPriority="true"
conduitSubscriptions="true">
<excludedDestinations>
<queue physicalName=">" />
</excludedDestinations>
</networkConnector>
</networkConnectors>
<transportConnectors>
<transportConnector
uri="${activemq.protocol}${activemq.host}:${activemq.tcp.port}?needClientAuth=true"
updateClusterClients="true" rebalanceClusterClients="true" />
<transportConnector
uri="${activemq.websocket.protocol}${activemq.websocket.host}:${activemq.websocket.port}?needClientAuth=true"
updateClusterClients="true" rebalanceClusterClients="true" />
</transportConnectors>
</broker>
with the following placeholder values
activemq.tcp.port=9000
activemq.protocol=ssl://
activemq.brokerName=activemq-server1.com
activemq.expose.jmx=true
activemq.otherBrokers=ssl://server2.com:9000,ssl://server3.com:9000
activemq.websocket.port=9001
activemq.websocket.protocol=stomp+ssl://
activemq.websocket.host=server1.com
activemq.memoryUsage=1gb
activemq.tempUsage=1gb
On the client side the following camel configuration is being used
<bean id="xxx.activemq.redeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
<property name="maximumRedeliveries" value="0" />
</bean>
<bean id="xxx.activemq.jmsConnectionFactory" class="org.apache.activemq.ActiveMQSslConnectionFactory">
<property name="trustStore" value="${activemq.broker.trustStore}" />
<property name="trustStorePassword" value="${activemq.broker.trustStorePassword}" />
<property name="keyStore" value="${activemq.broker.keyStore}" />
<property name="keyStorePassword" value="${activemq.broker.keyStorePassword}" />
<property name="brokerURL" value="${activemq.broker.url}" />
<property name="redeliveryPolicy" ref="xxx.activemq.redeliveryPolicy" />
</bean>
<bean id="xxx.activemq.jmsConfiguration" class="org.apache.activemq.camel.component.ActiveMQConfiguration">
<property name="receiveTimeout" value="6000" />
<property name="connectionFactory" ref="xxx.activemq.pooledConnectionFactory" />
</bean>
<bean id="xxx.activemq.pooledConnectionFactory"
class="org.apache.activemq.pool.PooledConnectionFactory"
init-method="start" destroy-method="stop">
<property name="maxConnections" value="8" />
<property name="idleTimeout" value="0" />
<property name="timeBetweenExpirationCheckMillis"
value="10000" />
<property name="connectionFactory"
ref="xxx.activemq.jmsConnectionFactory" />
</bean>
<bean id="xxx.activemq.jms.abstractComponent" abstract="true"
class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration"
ref="xxx.activemq.jmsConfiguration" />
<property name="connectionFactory"
ref="xxx.activemq.pooledConnectionFactory" />
<property name="allowNullBody" value="true" />
<property name="transferException" value="true" />
<property name="defaultTaskExecutorType"
value="#{T(org.apache.camel.component.jms.DefaultTaskExecutorType).ThreadPool}" />
<property name="requestTimeout" value="5000" />
</bean>
<bean id="xxx.activemq.jms.queue"
parent="xxx.activemq.jms.abstractComponent">
<property name="concurrentConsumers" value="2" />
<property name="maxConcurrentConsumers" value="2" />
</bean>
with a connection url of
activemq.broker.url=failover:(ssl://server1.com:9000,ssl://server2.com:9000,ssl://server3.com:9000)?randomize=true
The request/reply EIP is achieved by having the producer set an according jmsReplyTo header and having camel default to the InOut using temp-queues.
Before the deployment all messaging was working as intended, however afterwards for some request/reply queues we would start to get timeouts on the producer side. The following entries showed up in the logs:
On producer side
Caused by: org.apache.camel.ExchangeTimedOutException: The OUT message was not received within: 5000 millis due reply message with correlationID: Camel-ID-xxx-intranet-phs-49404-1457684675710-8-11 not received on destination: temp-queue://ID:xxx.intranet.phs-41986-1457684806758-1:3:1.
Exchange[Message: BeanInvocation public abstract xxx.xxx.rapi.dto.RemoteDTO xxx.xxx.xxx.facade.RemoteFacade.findRemoteDTO(java.lang.String,java.lang.Long) with [xxx, 31333]]]
and on consumer side:
Caused by: javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-queue://ID:xxx.intranet.phs-41986-1457684806758-1:3:1
We have since then done some research and found that the problem would show up whenever an arbitrary broker of the network is shut down and then only for those producers that had a temp-queue open for a reply when the shutdown hit and they failover to a new broker. Afterwards the problem would persist for this producer until he is restarted. Once he joins back after a restart everything is back to normal. The problem is also described on grokbase as well as on two topics here activemq-failover-with-temporary-queues-on-a-network-of-brokers and activemq-how-to-handle-broker-failovers-while-using-temporary-queues. We have tried the one solution given in activemq-how-to-handle-broker-failovers-while-using-temporary-queues to set the cache timeout but did not get any results from it, the other suggested option to turn of advisory listening for clients is not really an option in our setup since we want to make use of features such as clusterRebalancing for easier adding of additional brokers to the network during runtime.
We have also found some JIRA issues on camel and ActiveMQ side like CAMEL-3193 that describe this issue and apparently fix them for versions where ours are newer so we are quite puzzled. Currently we are considering switching to exclusive reply queues over temporary queues to address this issue, but first wanted to ask if maybe we are just missing some configuration somewhere.
If you need any additional information please just ask!
See AMQ-5469: Temporary queues get deleted when the connection that established them drops. This happens when you shutdown a broker.
The solution can be one of two options:
Recreate temporary queues by the client on re-establishment of the broker-connection
Don't use temporary queues, use persistent queues
I'm using ActiveMQ with the spring framework.
I have two consumers setup in the jms container. When I send 4 message to the queue, and some of the message are transferred to the "Dispatched Queue", because it takes a long time to the consumer to process the message.
I'm trying to find the way to prevent the message from going to the "Dispatched Queue", that is, I want them to be available to any consumer that is ready to consume the message.
I tried to set pre-fetch to 0, but it doesn't seem to work at all.
<bean id="prefetchPolicy" class="org.apache.activemq.ActiveMQPrefetchPolicy">
<property name="queuePrefetch" value="0"/>
</bean>
<bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<constructor-arg index="0" value="tcp://localhost:61616" />
<property name="prefetchPolicy" ref="prefetchPolicy"/>
</bean>
The following is the setup for my jms container:
<bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="destination" ref="defaultDestination" />
<property name="messageListener" ref="messageListener" />
<property name="concurrentConsumers" value="2" />
</bean>
I found the problem. I had the same beans declared twice at two different places. The second bean that was loaded did not have pre-fetch set to 0 and therefore it didn't work out.
The above setup I posted works!
Thank you!
I have two Java standalone applications. I want to send message from one application and asynchronously receive message by two clients: one is in the same application as the sender. The other is in a different application. Both are connected with ActiveMQ broker. But I can only see that the first client got the message while the other didn't get the message. What is the general approach to connect two applications by JMS? I think I must have some unclear concept about JMS. I looked up around but couldn't figure out how to set up my Spring bean configuration file to publish/subscribe message between two Java applications.
Here is my sender's bean configuration file in the first application and also the first listener's bean which is in the same application:
<bean id="customerMessageSender" class="com.example.message.CustomerStatusSender">
<property name="jmsTemplate" ref="jsmTemplateBean" />
<property name="topic" ref="topicBean" />
</bean>
<bean id="jsmTemplateBean" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="connectionFactoryBean"/>
<property name="pubSubDomain" value="true"/>
</bean>
<bean id="topicBean" class="org.apache.activemq.command.ActiveMQTopic">
<constructor-arg value="CustomerStatusTopic" />
</bean>
<bean id="connectionFactoryBean" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616" />
</bean>
<bean id="customerStatusListener" class="com.example.message.CustomerStatusListener" />
<bean id="listenerContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactoryBean" />
<property name="destination" ref="topicBean" />
<property name="messageListener" ref="customerStatusListener" />
</bean>
Here is the bean configuration file for the second listener which is in a different application:
<bean id="topicBean" class="org.apache.activemq.command.ActiveMQTopic">
<constructor-arg value="CustomerStatusTopic" />
</bean>
<bean id="connectionFactoryBean" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616" />
</bean>
<bean id="anotherCustomerStatusListener" class="com.mydomain.jms.CustomerStatusMessageListener" />
<bean id="listenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactoryBean" />
<property name="destination" ref="topicBean" />
<property name="messageListener" ref="anotherCustomerStatusListener" />
</bean>
As you can see, both customerStatusListener bean and anotherCustomerStatusListener bean subscribe to the topicBean. But only the first listener gets the message since it is in the same application as the sender while the second does not. What is the general principle to connect two Java applications by JMS so that message can be sent/received between two separate applications?
Edit: I can't add the following listener bean in the first XML file since the class CustomerStatusMessageListener is in a different application therefore is not visible in the classpath of the first (sender's) application.
<bean id="anotherCustomerStatusListener" class="com.mydomain.jms.CustomerStatusMessageListener" />
Edit again: The following is the second listener in the second application which is separate from the first application. It contains a main method to instantiate the listener bean (jms-beans.xml is the bean configuration file for the second listener as listed above).
public class CustomerStatusMessageListener implements MessageListener {
public void onMessage(Message message) {
if (message instanceof TextMessage) {
try {
System.out.println("Subscriber 2 got you! The message is: "
+ ((TextMessage) message).getText());
} catch (JMSException ex) {
throw new RuntimeException(ex);
}
} else {
throw new IllegalArgumentException(
"Message must be of type TextMessage");
}
}
public static void main(String[] args) {
ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext("jms-beans.xml");
CustomerStatusMessageListener messageListener = (CustomerStatusMessageListener) context.getBean("anotherCustomerStatusListener");
context.close();
}
}
Your understanding of JMS is correct. If you want two listeners to receive the same message, then a topic is the way to do it. It shouldn't matter if one listener is running in the same VM as the sender and the other one is not. Without seeing your code, your Spring configuration also looks correct. That leaves a few more things to check:
Are both listeners running on the same host? That is, is localhost one place for one of the listeners and a different place for the other listener?
Is your second listener running at the time the message was sent? If your second listener isn't active when the message is sent, it won't see if it starts later, unless you're using durable topics AND your subscriber has connected to the broker at least once.
Based on your comments, the second item is where you are having problems.
This blog post tells how to set up durable topics (and persist messages if you need messages to persist through broker restarts). Basically, add this configuration to your message listeners:
<property name="subscriptionDurable" value="true">
<property name="clientId" value="Some_unique_id">
<property name="durableSubscriptionName" value="Some_unique_id">
The client id and durable subscription name must be different for every subscriber, so your first listener would have something like:
<bean id="listenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactoryBean" />
<property name="destination" ref="topicBean" />
<property name="messageListener" ref="anotherCustomerStatusListener" />
<property name="subscriptionDurable" value="true">
<property name="clientId" value="listener1">
<property name="durableSubscriptionName" value="listener1">
</bean>
And second should have:
<bean id="listenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactoryBean" />
<property name="destination" ref="topicBean" />
<property name="messageListener" ref="anotherCustomerStatusListener" />
<property name="subscriptionDurable" value="true">
<property name="clientId" value="listener2">
<property name="durableSubscriptionName" value="listener2">
</bean>
Note that you must start your second listener at least once to register itself with the broker so the broker knows its clientId and will store messages for it, but you can then shut it down and start it later to get any messages it missed while it was down.
If your listener stays down for a long time on a high-volume system, the broker will store all messages for it, which might eventually fill the disk or slow the broker down. See the ActiveMQ documentation for automatically removing durable messages.
I have a simple Spring config for unit tests that includes ActiveMQ embedded broker, JMS producer and JMS listener. Tests are passed successfully, but when shutting everything down I see some exceptions. Like this:
[ WARN] [ActiveMQ Connection Executor: tcp://localhost/127.0.0.1:61616 03:16:23] (SingleConnectionFactory.java:onException:301) Encountered a JMSException - resetting the underlying JMS Connection
javax.jms.JMSException: java.io.EOFException
My configuration:
<beans>
<amq:queue id="defaultDestination" physicalName="example.A"/>
<bean id="jmsProducerTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="defaultDestination" ref="defaultDestination"/>
</bean>
<bean id="jmsMessageSender" class="package.JMSMessageSender"/>
<amq:broker id="broker" useJmx="false" persistent="false" destroyApplicationContextOnStop="true" >
<amq:transportConnectors>
<amq:transportConnector uri="tcp://localhost:61616"/>
</amq:transportConnectors>
</amq:broker>
<bean id="messageListener" class="package.MessageReceiver"/>
<bean id="connectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory" depends-on="broker">
<property name="targetConnectionFactory">
<amq:connectionFactory brokerURL="tcp://localhost:61616"/>
</property>
</bean>
<jms:listener-container connection-factory="connectionFactory" container-type="default">
<jms:listener destination="example.A" ref="messageListener"/>
</jms:listener-container>
</beans>
I supposed that broker is closing and then connection factory loses connection. But depends-on has not helped.
Thanks for your help.
Ad far as I can tell, there is no way to shut down the embedded AMQ without these exceptions. The AMQ code treats that EOF as a relatively harmless event, but it does log that exception. You could try opening a JIRA at the Apache ActiveMQ project.