I use Apache Camel module deployed inside ActiveMQ service.
Given I use Spring DSL and I have route definition ( implemented as routeContext) in the FilteringRouteContext.xml file (simplified):
<routeContext id="filteringRouteContext" xmlns="http://camel.apache.org/schema/spring">
<route id="myFilteringRoute">
<from uri="direct:filteringRoute"/>
<idempotentConsumer messageIdRepositoryRef="idempotentRepository" skipDuplicate="false">
<simple>${header.JMSType}</simple>
<filter>
<property>CamelDuplicateMessage</property>
<stop/>
</filter>
</idempotentConsumer>
</route>
</routeContext>
Next, I have configured Camel Context in other XML file (simplified):
<import resource="classpath:FilteringRouteContext.xml"/>
<camelContext xmlns="http://camel.apache.org/schema/spring">
<routeContextRef ref="filteringRouteContext"/>
<route id="myRoute1">
<from uri="activemq:topic:source1" />
<to uri="direct:filteringRoute" />
<to uri="activemq:topic:target1" />
</route>
<route id="myRoute2">
<from uri="activemq:topic:source2" />
<to uri="direct:filteringRoute" />
<to uri="activemq:topic:target2" />
</route>
<route id="myRoute3">
<from uri="activemq:topic:source3" />
<to uri="direct:filteringRoute" />
<to uri="activemq:topic:target3" />
</route>
</camelContext>
<bean id="idempotentRepository"
class="org.apache.camel.processor.idempotent.MemoryIdempotentRepository">
<property name="cacheSize" value="10"/>
</bean>
I would like to have shared route (with id=myFilteringRoute) from filteringRouteContext declared as, using IoC terminology, instance per dependency, so each route from single Camel Context (with id=myRoute1, myRoute2, myRoute3) should use it's own instance of that shared route (with id=myFilteringRoute), with separate internal state, bean instances, etc.
In other words, each route from Camel Context (with id=myRoute1, myRoute2, myRoute3) should not use the same instance of shared route (with id=myFilteringRoute), but should has its own completely separate instances (with completely separated internal states and bean instances )
Please consider that my shared route (with id=myFilteringRoute) may use more beans, which may have various scopes (singleton, prototype, request etc.).
My questions are: Can I achieve this goal using single Camel Context, or do I need to place my routes (with id=myRoute1, myRoute2, myRoute3) in separate Camel Contexts? What is the best solution of my problem?
Is there important performance impact If I use more than one Camel Contexts, and each context uses beans to communicate with ActiveMQ (org.apache.activemq.camel.component.ActiveMQComponent), or other beans that consume internal or system resources?
Or maybe it's better to resolve my problem by using Java DSL instead of Spring DSL?
Thank You.
Current Camel Spring DSL definitions are created by JAXB when it unmarshals the xml. This definitions help camel runtime to to build up processors and assemble them to route the message. In this way routeContextRef has nothing do with the scopes you mentioned.
But for the beans which you created by Spring with bean element, you can define the scopes if you like, and camel just grab it from Spring Application Context if there is bean reference in the Camel Spring DSL.
To answer you question, routeContextRef just give a way to share the route definition instance across the camel context, if you don't want share their instance you need to create the camel context in different spring application context which could hold the different instance of routeContext.
The answer is that Camel doesn't not provide an automatic mechanism for doing what you want to do. All routes in your example will share the same instance of idempotentRepository. The only solution is to provide a level of indirection.
For example, extend AbstractJdbcMessageIdRepository to provide your own implementation. This could then include the routeid in the look up to determine whether a message had already been processed.
Or you could have a set of repositories and look up which one to use from within the main idempotentRepository based on the route id.
Whatever you do, you'll need write code that uses the the route id of the outermost route to distinguish the messages.
Related
I have a pair of camel routes that receive messages from two different queues, whose process is executed by the same bean (by reference). This is done via Spring & Camel XML.
The configuration looks as such:
<route id "route-1" xmlns="http://camel.apache.org/schema/spring">
<from uri="queue:IN1" />
<process ref = "myProcessBean />
</route>
<route id "route-2" xmlns="http://camel.apache.org/schema/spring">
<from uri="queue:IN2" />
<process ref = "myProcessBean />
</route>
If I get messages on both IN1 and IN2, will those messages be processed in parallel?
Yes it would. A processor is a singleton bean and it can get processed in parallel. Just ensure that it doesn't store any state information within it, which is in fact one of the best practices suggested(Link below).
https://www.3riverdev.com/apache-camel-processors-should-never-be-stateful/
I am getting the below error when I try to post IDOC.
Dispatcher has no subscribers for channel
'DataHubWebApplicationContext.DEBMAS-DEBI'.; nested exception is
org.springframework.integration.MessageDispatchingException:
Dispatcher has no subscribers.
Xml mapping looks like this:
<int-xml:xpath-router id="splitKTOKD" input-channel="DEBMAS" evaluate-as-string="true" resolution-required="false"default-output-channel="DEBMAS-NOTSUPPORTED-KTOKD">
<int-xml:xpath-expression id="splitKTOKDExpression" expression="//KTOKD" />
<int-xml:mapping value="DEBI" channel="DEBMAS-DEBI" />
<int-xml:mapping value="0170" channel="DEBMAS-0170" />
<int-xml:mapping value="Z001" channel="DEBMAS-Z001" />
</int-xml:xpath-router>
<int:service-activator id="sapcustomerDEBMASCustomerServiceActivator" input-channel="DEBMAS-DEBI" output-channel="rawFragmentDataInputChannel" ref="sapcustomerDEBMASCustomerMappingService" method="map" />
<int:service-activator id="sapcustomerDEBMASCustomerServiceActivator" input-channel="DEBMAS-Z001" output-channel="rawFragmentDataInputChannel" ref="sapcustomerDEBMASCustomerMappingService" method="map" />
Though I have configured proper input-channel and output-channel I am getting the message delivery Exception only for channel DEBMAS-DEBI. DEBMAS-Z001, DEBMAS-0170 works fine.
Looks like this is something related to spring framework issue.
How can I resolve this spring issue?
You have two beans declared with the same sapcustomerDEBMASCustomerServiceActivator Id. By default the second wins. There is no the first bean in the application context. That's why you are observing that issue.
Use different ids or even without it to fix the problem.
Using Camel 2.15.2, Spring 4.1.7.RELEASE.
Project: camel-example-cxf-tomcat
Route is created:
<bean id="myRoutes" class="org.apache.camel.example.cxf.CamelRoute"/>
<camelContext xmlns="http://camel.apache.org/schema/spring">
<routeBuilder ref="myRoutes"/>
</camelContext>
Route is not created:
<bean id="myRoutes" class="org.apache.camel.example.cxf.CamelRoute"/>
<camelContext xmlns="http://camel.apache.org/schema/spring">
<packageScan>
<package>org.apache.camel.example.cxf</package>
<excludes>*Hide*</excludes>
</packageScan>
</camelContext>
Issue because, camel package scan will ignore already instantiated classes. So remove your below code and try.
<bean id="myRoutes" class="org.apache.camel.example.cxf.CamelRoute"/>
Hope it helps!!
Please share the error you are getting while creating routes.
The <package> and <packageScan> will skip any classes which has already been created by Spring etc. So if you define a route builder as a spring bean tag then that class will be skipped. You can include those beans using <routeBuilder ref="theBeanId"/> or the <contextScan> feature.
Try to use
<packageScan>
<package>---specify package containing route here---
</package>
</packageScan>
I have been struggling to find a work around to be able to dynamically read the polling frequency in mule flow. Currently I am reading that from a file using spring's Propertyplaceholder at the start up and value remains the same even if the fie is changed(as we all know)..
Since poll tag needs to be the first component in the flow, There is nothing much i could do to read the "live" file update.
Is there any way I could set the polling frequency dynamically read from a file(without requiring restart)?
For Reference:
<spring:beans>
<context:property-placeholder location="file:///C:/Users/test/config.properties" />
</spring:beans>
<flow name="querying-database-pollingFlow1" doc:name="querying-database-pollingFlow1">
<poll doc:name="Poll3e3">
<fixed-frequency-scheduler frequency="${pollinginterval}"/>
<db:select config-ref="MySQL_Configuration1" doc:name="Perform a query in MySQL">
<db:dynamic-query><![CDATA[select empId,empName from employer where status='active';]]></db:dynamic-query>
</db:select>
</poll>
....</flow>
There is absolutely no issue with <fixed-frequency-scheduler frequency="${pollinginterval}"/> as you can dynamically read polling frequency from a properties file ...
The only thing I am concern here is :- <context:property-placeholder location="file:///C:/Users/test/config.properties" />
Since you are reading from a properties file outside your classpath, better try with the following :-
<context:property-placeholder
location="file:C:/Users/test/config.properties" />
One more thing .. if you are using Spring beans for properties file use the following way :-
<spring:beans>
<spring:bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<spring:property name="locations">
<spring:list>
<spring:value>file:C:/Users/test/config.properties</spring:value>
</spring:list>
</spring:property>
</spring:bean>
</spring:beans>
The clean way using FixedFrequencyScheduler is not there. You could potentially go to the registry, fetch your flow by name, then get the MessageSource and cast it to FixedFrequencyScheduler set the new interval and stop-start, however if you take a look to the code you'll see there is no setter for it and reflexion it's just too dirty.
My first choice would probably be to leverage a quartz endpoint and then leverage the quartz abilities to expose the configuration throught jmx/rmi.
I would definitely advise against using hot deploy to solve this problem especially if you need to change the frequency often. There is a risk that this will lead to problems with permgen running out of memory.
Instead you could use a flow with a quartz endpoint that fires at a relatively low frequency. Then add a filter that only lets through the message at the required frequency.
The filter can either watch a properties file for changes or expose attributes over JMX to allow you to change the frequency. Something like this.
<spring:beans>
<spring:bean id="frequencyFilter" class="FrequencyFilter" />
</spring:beans>
<flow name="trigger-polling-every-second" doc:name="trigger-polling-every-second">
<quartz:inbound-endpoint repeatInterval="1000" doc:name="Quartz" responseTimeout="10000" jobName="poll-trigger">
<quartz:event-generator-job>
<quartz:payload>Scheduled Trigger</quartz:payload>
</quartz:event-generator-job>
</quartz:inbound-endpoint>
<filter ref="frequencyFilter" />
<vm:outbound-endpoint path="query-database" />
</flow>
<flow name="query-database">
<vm:inbound-endpoint path="query-database" />
<db:select config-ref="databaseConfig" doc:name="Perform a query in database">
<db:dynamic-query><![CDATA[select empId,empName from employer where status='active']]></db:dynamic-query>
</db:select>
<logger level="ERROR" message="#[payload]"/>
</flow>
Here is an existing spring rabbit config from a project that I inherited -
<rabbit:connection-factory id="rabbitConnectionFactory"
host="${rabbitmq.host}"
port="${rabbitmq.port}"
virtual-host="${rabbitmq.virtualHost}"
username="${rabbitmq.username}"
password="${rabbitmq.password}"/>
<rabbit:template id="ampqTemplate" connection-factory="rabbitConnectionFactory" routing-key="" />
<rabbit:admin connection-factory="rabbitConnectionFactory" />
<rabbit:queue name="${rabbitmq.queueName}" />
I dont have experience using Rabbit and with my limited reference,
I understand an exchange is an important piece in the setup since it relays messages to the queue internally.
However, the above configuration does not contain any exchange information.
My Questions are :
Is an exchange absolutely important for even a simple queue configuration.?
Is there any implication of not defining an exchange
Is there anyother configuration obviously missing from the above configuration?
If you don't define an exchange the default exchange will be used. It is a direct exchange which will use the name of the queue as its routing key.
It doesn't look like there is anything missing from your configuration. Mine is:
<rabbit:connection-factory id="connectionFactory" host="${rabbit.host}" username="${rabbit.username}" password="${rabbit.password}" virtual-host="${rabbit.vhost}"/>
<rabbit:template id="rabbitTemplate" connection-factory="connectionFactory" message-converter="jsonMessageConverter" routing-key="event-queue"/>
You may be able to take the rabbit:queue definition out and use the queue name as the routing key for the rabbit:template.