How do I use Camel File Component Consumer using multiple threads ?
meaning I have this code :
<route id="incomingFile">
<from
uri="file://{{incomingFileBaseFolder}}?filter=#fileFilter&recursive=true&readLock=changed&move=${file:parent}/.backup/${date:now:yyyy}/backup_${exchangeId}_${file:onlyname.noext}.${file:name.ext}&sortBy=file:modified&delay={{incomingFileDelay}}" />
<transacted />
<threads poolSize="10">
<convertBodyTo type="java.lang.String" />
<setHeader headerName="{{incoming_file_backup_date_header_name}}">
<simple>$simple{date:now:yyyy}
</simple>
</setHeader>
<bean ref="saveFile" method="duplicateCeck" />
<to uri="direct:validateFileDirect" />
<to uri="direct:inputFileContentHandle" />
</threads>
</route>
but it does not seems to work on more than one file at a time.
How do I make it happens ?
Remove <transacted/> as it does not support asynchronous routing. Also transactions only works with component/resources that support JTA transactions natively, which typically is only JMS and JDBC.
Related
I'm having a bit of an issue with preventing the original file from being renamed when I throw/catch an Exception in a Processor.
I have a route like this:
<route customId="true" id="localRoute">
<from uri="{{ftp.pull.LOCAL.server}}" />
<process ref="Processor" />
<to uri="{{ftp.push.LOCAL.route}}" />
</route>
The from URI includes the option of: &move=${file:name.noext}.${file:name.ext}.old
And during my Process, I stop the Exchange from being routed to the end under certain conditions and also throw an Exception that I catch using:
<onException>
<exception>com.myException.ThrownException</exception>
<handled><constant>true</constant></handled>
</onException>
Is there any way to prevent the renaming of the original file I'm pulling from if I throw and catch that exception?
(I throw and catch that Exception in order to prevent a file from entering a Idempotent File Repository for other routes. Many routes use this Processor.)
File will be renamed unless you remove the <handled> or mark that as false.
If you handle the exception it won't propagate and your file component thinks that the Camel exchange processed successfully and it will rename the file.
So I think I may have found an answer.
So what I have found out is that by doing a global <onException> block that catches my custom Exception, it will prevent logging a stack trace and entering in a file name into the File Repo for routes that use a Idempotent File Repository. But that global block will not prevent the routes that rename files from renaming the files.
In order to catch the Exception and prevent the logging of the stack trace on routes that rename files, I needed to use the doTry/doCatch blocks. I have to do something like this:
<route customId="true" id="localRoute">
<from uri="{{ftp.pull.LOCAL.server}}" />
<doTry>
<process ref="Processor" />
<doCatch><exception>com.myException.ThrownException</exception></doCatch>
</doTry>
<to uri="{{ftp.push.LOCAL.route}}" />
</route>
So my solution ends up looking like this:
<camelContext xmlns="http://camel.apache.org/schema/spring">
<onException>
<exception>com.myException.ThrownException</exception>
<handled><constant>true</constant></handled>
</onException>
<route customId="true" id="localRoute">
<from uri="{{ftp.pull.LOCAL.server}}" />
<doTry>
<process ref="Processor" />
<doCatch><exception>com.myException.ThrownException</exception></doCatch>
</doTry>
<to uri="{{ftp.push.LOCAL.route}}" />
</route>
<route customId="true" id="otherRoute">
<from uri="{{ftp.pull.OTHER.server}}" />
<setHeader headerName="otherFileRepoKey">
<simple>${file:name}-${file:modified}</simple>
</setHeader>
<idempotentConsumer messageIdRepositoryRef="otherFileStore">
<header>otherFileRepoKey</header>
<process ref="Processor" />
<to uri="{{ftp.push.OTHER.route}}"/>
</idempotentConsumer>
</route>
</camelContext>
I have a Camel route that looks something like the one below. If all records parse successfully, then I get an email from the onCompletion step. If one record gets an exception then the rest of the records will process, which is fine, but the onCompletion step does not fire.
What I'd like is for the onCompletion step to run even if there are errors and to be able to send a message saying "route completed with errors". How can I do this?
<route id="route1">
<from uri="file://C:/TEMP/load?noop=true&idempotentRepository=#sysoutStore&sorter=#externalDataFilesSorter"/>
<choice>
<when>
<simple>${file:name} regex '*file.*.(txt)'</simple>
<to uri="direct:RouteFile" />
</when>
</choice>
</route>
<route id="testRouteDirect">
<from uri="direct:RouteFile" />
<onException>
<exception>java.lang.IllegalArgumentException</exception>
<redeliveryPolicy maximumRedeliveries="1" />
<handled>
<constant>true</constant>
</handled>
<to uri="log:java.lang.IllegalArgumentException"></to>
</onException>
<onException>
<exception>java.text.ParseException</exception>
<redeliveryPolicy maximumRedeliveries="1" />
<handled>
<constant>true</constant>
</handled>
<to uri="log:java.text.ParseException"></to>
</onException>
<split parallelProcessing="false" strategyRef="exchangePropertiesAggregatorStrategy" >
<tokenize token="\r\n"/>
<to uri="log:Record"></to>
</split>
<onCompletion>
<to uri="log:completion"></to>
<to uri="smtp://mail.com?contentType=text/html&to=done#test.com&from=route#test.com&subject=we're done" />
</onCompletion>
</route>
The best part of your route is, you have onException inside your route with handled=true. So move your onCompletion to the parent route(route1), It should work !
There are a bunch of tickets related to oncompletion on the camel site: Camel Jira URL. I upgraded to a newer version of camel & I don't get this issue any more.
im using camel to translate a h2db into java-objects. It works! But now i dont know how to configurate the camel to read each table entry only once, except it was updated since the last read-in.
I cant read every 2 seconds a db with more than 1 million entrys, so i only want the new entries or the updated.
That's my camel-route:
<route id="db-import-route">
<from uri="timer://queryTimer?period=2s" />
<setBody>
<constant>
SELECT * FROM address
</constant>
</setBody>
<to uri="jdbc:dataSource" />
<split>
<simple>${body}</simple>
<process ref="SqlDecoder" />
<process ref="Validator" />
</split>
</route>
Thanks a lot!
I have a route where I want Camel to visit the following beans:
First, loggingBean
Second, an aggregator that waits for a certain number of messages to aggregate on it
Once the Aggregator's completionSize is reached (3), keep going to #4
Third, processorBean
And fourth/last, finalizerBean
Here is my route:
<route id="my-camel-route">
<from uri="direct:starter" />
<to uri="bean:loggingBean?method=shutdown" />
<aggregate strategyRef="myAggregationStrategy" completionSize="3">
<correlationExpression>
<simple>${header.id} == 1</simple>
</correlationExpression>
<to uri="bean:processorBean?method=process" />
</aggregate>
<to uri="bean:finalizerBean?method=shutdown" />
</route>
My question: do I need to place finalizerBean inside the <aggregate> element like so:
<aggregate strategyRef="myAggregationStrategy" completionSize="3">
<correlationExpression>
<simple>${header.id} == 1</simple>
</correlationExpression>
<to uri="bean:processorBean?method=process" />
<to uri="bean:finalizerBean?method=shutdown" />
</aggregate>
Basically, I'm wondering if the way I currently have things will prompt Camel to send the message to the aggregator, and then also send it on to finalizerBean (essentially, bypassing the aggregator). In my case, I want it to aggregate until completionSize is 3, and then send the aggregated exchange on to the processorBean and then finally finalizerBean.
Or have I configured this correctly? What's the difference between finalizerBean being inside the <aggregate> element vs being outside it?
The second example is correct.
<aggregate strategyRef="myAggregationStrategy" completionSize="3">
<correlationExpression>
<simple>${header.id} == 1</simple>
</correlationExpression>
<to uri="bean:processorBean?method=process" />
<to uri="bean:finalizerBean?method=shutdown" />
</aggregate>
If finalizerBean is "outside" the <aggregate>, it will get executed for every message that comes from direct:starter - which isn't what you want ;)
I have to stock my object (CcRequest) in a DeadLetterQueue with Json format.
How it is possible ?
Here my simple context :
<camelContext id="el1DMRCamelContext" autoStartup="true" xmlns="http://camel.apache.org/schema/blueprint" >
<template id="producerTemplate" />
<!-- Routes -->
<route id="createCcProcessorRoute" errorHandlerRef="createCcErrorHandler" >
<from uri="activemq:queue:createCc" />
<process ref="createCcProcessor" />
</route>
</camelContext>
<bean id="createCcErrorHandler" class="org.apache.camel.builder.DeadLetterChannelBuilder" >
<property name="deadLetterUri" value="activemq:queue:createCcDLQ" />
<property name="redeliveryPolicy" ref="redeliveryPolicy" />
</bean>
I would like my object (CcRequest stocked in the body of the exchange) in my activemq:queue:createCcDLQ to be in Json format instead of binaries.
Is it possible ?
I would not classify what you want to do as a dead letter handler. Dead letter is usually just moving/redelivery of the original message.
It is pretty simple you use the excetion clause instead.
<dataFormats>
<json id="jsonFormat" library="Jackson"/>
</dataFormats>
<onException>
<exception>java.lang.RuntimeException</exception>
<marshal ref="jsonFormat"/>
<to uri="activemq:queue:createCcDLQ"/>
</onException>
Camel Exception Clause docs
Camel JSON docs