I am trying to make a queue with activemq and spring boot using this link and it looks fine. What I am unable to do is to make this queue persistent after application goes down. I think that SimpleJmsListenerContainerFactory should be durable to achieve that but when I set factory.setSubscriptionDurable(true) and factory.setClientId("someid") I am unable to receive messages any more. I would be greatfull for any suggestions.
I guess you are embedding the broker in your application. While this is ok for integration tests and proof of concepts, you should consider having a broker somewhere in your infrastructure and connect to it. If you choose that, refer to the ActiveMQ documentation and you should be fine.
If you insist on embedding it, you need to provide a brokerUrl that enables message persistence.
Having said that, it looks like you misunderstand durable subscriber and message persistence. The latter can be achieved by having a broker that actually stores the content of the queue somewhere so that if the broker is stopped and restarted, it can restore the content of its queue. The former is to be able to receive a message even if the listener is not active at a period of time.
you can enable persistence of messages using ActiveMQConnectionFactory.
as mentioned in the spring boot link you provided, this ActiveMQConnectionFactory gets created automatically by spring boot.so you can have this bean in your application configuration created manually and you can set various property as well.
ActiveMQConnectionFactory cf = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=true");
Here is the link http://activemq.apache.org/how-do-i-embed-a-broker-inside-a-connection.html
Related
I'm writing some core features for developers who will use my library.
One of the features is to provide an ability to switch the message consumption between two different sources by flag configuration, the handling of these messages should remain as is, no mater to the source - for example switching message consumption from kafka to rabbitmq. the same business logic will be executed with income message.
**I try to figure out how to configure the consumer without using the #RabbitListener, is it possible? **
RabbitAdmin - responsible for connect to the source.
RabbitTemplete - responsible for publishing messages.
The only clue that I found is to use springs SimpleMessageListenerContainer, but the issue with him is that there is no ability to set multiple onMessage handlers?
Also I saw the option to use MessageListenerAdapter in this answer
The main issue with these answers is that I'm going to deal with multiple number of queues and bindings.. and here it seem like solution for single consumer in whole application. - am I wrong here?
You need a listener container for each listener.
You can use Boot's auto configured listener container factory to created each container and add a listener to it.
If the same listener can consume from multiple queues, you can configure the container to listen to those queues.
We have a spring boot application and we are using apache camel as a framework for message processing. We are trying to best optimize our application settings to make the enqueue of messages on the ActiveMQ queue fast which is received by the Logstash on the other end of the queue as consumers.
The documentation is scattered at many places and there are too many configurations available.
For example, the camel link for spring boot specifies 102 options. Similarly, the activemq apache camel link details these with much more.
This is what we have currently configured:
Application.properties:
################################################
# Spring Active MQ
################################################
spring.activemq.broker-url=tcp://localhost:61616
spring.activemq.packages.trust-all=true
spring.activemq.user=admin
spring.activemq.password=admin
Apache Camel
.to("activemq:queue:"dataQueue"?messageConverter=#queueMessageConverter");
Problem:
1 - We suspect that we have to use poolConnectionFactory and not default Spring JMS Template bean which is somehow auto picked up.
2 - We also want the process to be asynchornous. We just want to put the message on queue and dont want to wait for any ACK from activemq or do anyretry or something.
3 - We want to wait for retry only if queue is full.
4 - Where should we set the settings for ActiveMq size? and also the activemq is putting things in Dead letter queue in case no consumer availaible? We want to override that behaviour and want to keep the message in there. (Is this have to be configured in Activemq and not in Our app/apache camel)
Update
Here is we have solved it after some more investigation and based on feedback for now. Note: this does not involve retrying, for that we will try the option suggested in the answer.
For Seda queues:
producer:
.to("seda:somequeue?waitForTaskToComplete=Never");
consumer:
.from("seda:somequeue?concurrentConsumers=20");
Active MQ:
.to("activemq:queue:dataQueue?disableReplyTo=true);
Application.Properties:
#Enable poolconnection factory
spring.activemq.pool.enabled=true
spring.activemq.pool.blockIfFull=true
spring.activemq.pool.max-connections=50
Yes, you need to use pooledConnectionFactory. Especially with Camel+Spring Boot. Or look to use the camel-sjms component. The culprit is Spring's JMSTemplate. Super high latency.
Send NON_PERSISTENT and AUTO_ACK, also turn on sendAsync on the connection factory
You need to catch javax.jms.ResourceAllocationException in your route to do retries when Producer Flow Control kicks in (aka queue or broker is full)
ActiveMQ does sizing based on bytes, not message count. See the SystemUsage settings in Producer Flow Control docs and Per-Destination Policies policies for limiting queue size based on bytes.
My questions is as follows:
I have a service, which queries the DB and retrieves some records. Then it updates an external system using that information and in the end in updates back the DB.
Using spring trascations and weblogic jta transaction manager i was able with sample code below not to loose any messages in case
No records are retrived. (these are mandatory for the External System)
External System Error
Failed to update the DB
So in all aboce cases the JMS Listener puts the errror message back to the queue.
My question, is there any better way, using Spring with all its goodies, to manage that? The sample code below throws explicilty a RuntimeException which i dont think is a good design...
Please for your comments.
EDIT:
The queue is being polled by the submissionListener MDP that its configuration is shown below. After the message is consumed it invokes the registerDocument() of the service. (another Spring bean). That service invokes 2 times the DAO and the external system.
Check out Spring's documentation on JmsTemplate and Message Driven POJOs for patterns in the core Spring framework.
Spring Integration models higher-level abstractions related to message-orientated patterns.
I have one question regarding JMS in Camel.
So I'm using JMS provided by some firm. But this JMS implementation does not provide Pooled Connection Factory.
So does camel has default pooled connection implementation?
Or it do smth trivial like:
1) Open connection
2) Open Session
3) Read/Write message
4) Close Session
5) Close connection
Because if believe my logs camel works like I mention in second case.
Thanks.
Camel pretty much uses JmsTemplate (from Spring Framework) for sending messages.
ActiveMQs thoughts of JmsTemplate
Essentially, you are true for the "producing" scenario, unless the underlying Jms provider features a pooling Connection Factory. This is usually the case if you run Spring or Camel inside App Servers.
If you set up something like
from("jms:queue:QUEUE.IN").to("somewhere:over/the/rainbow");
Then one or more on going consumers will be active, not destroying the session for each message (only commiting the message if you set transactions up). There is also a possiblity to pool the response listener for JMS request/response. Refer to camel.apache.org/jms for info.
But you are right, if you have a remote (non pooling) JMS provider and fires of frequent outgoing messages from Camel, this could be somewhat of a performance issue.
Use the spring CachingConnectionFactory. Btw. which JMS provider do you use?
http://static.springsource.org/spring/docs/2.5.x/api/org/springframework/jms/connection/CachingConnectionFactory.html
It's simple: I have an MDB and an EJB that sends messages to a topic (or queue). JBoss complains that the topic is not bound to the JNDI context.
I want to have the topic/queue to be automatically created at best, or at least to have a standard way to define it, per application (say, in ejb-jar/META-INF)
this question and this blogpost show us how to do it in an application server specific way. This surely works, but:
I want to use the #MessageDriven annotation
I want the setting not to be global for the application server
I want the setting to be portable
It seems impossible to do this, with JavaEE 5 at least.