I got a requirement that i have on main route and child route. In the main route will get the list of objects, there i need to make a call to child route for each individual elements in the list.
Then in Child route will make a call to web-service by appending that element as one of the parameter.
Code:
from("direct:SupplierRoute")
.choice()
.when(header(IS_SUPPLIER_AVAILABLE).isEqualTo(true))
.split(body())
.parallelProcessing()
.streaming()
.to("direct:SUPGetHotelAggregatorRatesRQ")
.bean(parallelProcessingRequestProcessor)
.end()
.end()
.end();
from("direct:SUPGetHotelAggregatorRatesRQ")
.process(startOperation(DISTRIBUTION, GET_HOTEL_AGGREGATOR_RATES_API_GENERATE_VM_REQUEST))
.to("velocity:velocity/GetHotelAggregatorRatesRQToGetHotelSupplierRatesRQ.vm")
.process(endOperation(DISTRIBUTION, GET_HOTEL_AGGREGATOR_RATES_API_GENERATE_VM_REQUEST))
.end();
I'm setting the value to the exchange body as below,
public static final List<HotelRefs.HotelRef> supplierHotelRefs = new ArrayList();
exchange.getIn().setBody(supplierHotelRefs);
But the above code sample was not working,If anyone let us know if there is any approach in camel to iterate over user defined collections at route level.
Thanks,
Raghavan
Put a log statement after the Splitter and run the code.
.split(body())
.log("How many log lines do you receive?")
If you get just one log line, the Splitter does not work as expected. You could then try to use another Camel version.
If you get as many log lines as your ArrayList contains elements, the iteration works fine and you've got another problem. You should then find out the real problem and ask a new question.
If you get no log line at all, your condition in when is not satisfied.
Related
I have a use case, i want to poll a directory, if the any .*xlsx file get pasted in that directory, i want to call a post rest API that will load the data.
I'm not able to find the my way, please suggest some way to do this.
I believe that you are looking for fully working sample and I doubt that is going to be one since your business task might not be the same what other people are doing.
Although we won't mind if you contribute back such a sample: https://github.com/spring-projects/spring-integration-samples.
So, to build a logic we need to provide an IntegrationFlow: https://docs.spring.io/spring-integration/docs/current/reference/html/dsl.html#java-dsl.
To read files from a dir we need to use a Files.inboundAdapter() with respective polling policy.
You may do some transformation (.transform()) about polled file content or so.
Call the REST service via Http.outboundGateway()
Do the post-process.
Something like this:
#Bean
public IntegrationFlow fileReadingFlow() {
return IntegrationFlows
.from(Files.inboundAdapter(new File("myDir"))
.patternFilter("*.xlsx"),
e -> e.poller(Pollers.fixedDelay(1000)))
.transform(...)
.handle(Http.outboundGateway("")
.expectedResponseType(String.class))
.transform(...)
.get();
}
(Haven't checked as working since I don't know what is your XSLT content and how you call the REST service.)
This sample does something with files reading to gather some ideas: https://github.com/spring-projects/spring-integration-samples/tree/main/applications/file-split-ftp
We have a system that does (almost) the same (just xml instead of xlsx) and we use Apache Camel https://camel.apache.org/
Integration is good with Spring Boot. You just need to define your route from("file:///<path").to("http:<host>:port/<path>) and it will do probably what you need.
Might need to tweek the line of code to get filtering and maybe add some transformation but it is a nice peace of software.
My main issue might be not understanding some conventions in the Camel documents.
https://camel.apache.org/components/latest/mongodb-component.html#_delete_operations
They have a camel route commented out, and two Java objects being defined, which are not commented out. What are they trying to indicate? Where are these objects at in a project?
Anyway, I'm subscribed to a JMS queue that I have another camel route publishing to. The message is a JSON string, which I save to a Mongo DB. But what I'd like to do is remove any current documents (based on criteria) and replace it with the new message.
from("jms:topic:orderbook.raw.feed")
.log("JMS Message: ${body}")
.choice()
.when().jsonpath("$.[?(#.type=='partial')]")
// Figure out how to delete the old orderbook from Mongo with a type=T1
.to("mongodb:mongo?database=k2_dev&collection=orderbooks&operation=save");
Does your orderbook have an ID? If so, you can enrich the JSON with an _id field (MongoDB default representation for identifiers) whose value would be that ID. Thus you'll be "upserting" that orderbook.
Obs.: Sure the Camel docs could be better.
But if you really feel you'd have to perform a remove operation before saving an orderbook, another option would be to extract its type from the current JSON string and use it as a filter when removing. Something like:
from("jms:topic:orderbook.raw.feed")
.log("JMS Message: ${body}")
.filter("$.[?(#.type=='partial')]")
.multicast().stopOnException()
.to("direct://orderbook-removal")
.to("direct://orderbook-save")
.end()
;
from("direct://orderbook-removal")
// extract type and set it as the body message. e.g. {"type":"T1"}
.to("mongodb:mongo?database=k2_dev&collection=orderbooks&operation=remove")
;
from("direct://orderbook-save")
.to("mongodb:mongo?database=k2_dev&collection=orderbooks&operation=save")
;
The multicast sends a copy of the message to each destination. So the content won't be affected.
I am storing the list of url's in the Apache camel header,below is the code,
List<String> supplierHotelRefs = new ArrayList();
supplierHotelRefs.add("a.com");
supplierHotelRefs.add("b.com");
supplierHotelRefs.add("c.com");
exchange.getIn().setBody(supplierHotelRefs);
Now i need to iterate this list present in the header and make a call to url's. This should be the parallel activity. I tried with split(..) which is working fine if we store the list in the body, But due to some constraints i can't store it in body. It will be helpful if i get code to iterate and parallely process the collection present in the Camel Header.
Regards,
Raghavan
You could set the list in the header and split on that header.
exchange.getIn().setHeader("supplierHotelRefs",supplierHotelRefs);
In your route definition you could split based on the header property and process them parallely.
from("").....
//split based on the header
split(header("supplierHotelRefs"))
//process every split exchange parallely
.parallelProcessing()
//end split block
.end()
//continue route definition after split
.log("completed split processing")
Note that the caller thread will still wait for all the split messages to be completed.
You can use Recipient List EIP, see http://camel.apache.org/recipient-list.html
You have to create a list with all recipients and store that list to a header. I observed in your code that you are not setting that list to a header, but as a body. You have to do something like
List<String> supplierHotelRefs = new ArrayList();
supplierHotelRefs.add("http4://a.com");
supplierHotelRefs.add("http4://b.com");
supplierHotelRefs.add("http4://c.com");
exchange.getIn().setHeader("yourHeaderName", supplierHotelRefs);
As you can notice each element of the list has a valid url of http4 Camel component.This component is used in order to make http calls. You can have anything you want into that list (other Camel's components).
Then you use the recipient list EIP, telling that all recipients are into the previous created header. parallelProcessing = true, call all items in the list in parallel. This is calling that in XML DSL:
<recipientList parallelProcessing="true">
<header>yourHeaderName</header>
</recipientList>
or in Java DSL:
from("...")
...
.recipientList(header("yourHeaderName"));
I am using Apache Camel in my Application. I am trying to use Composed Message Processor. I have exchange whose body contains some URLs to hit and by using split(body(), MyAggregationStrategy()), I am trying to get the data from urls and using Aggregation Strategy want to combine each data. But there is a problem where I am stuck. If there is some invalid url on the first line of the body then it happens that aggregation is working fine but it is not moving to the next processor and if invalid url is anywhere else except first line than it is working fine..
please help,
Here is the code for reference
onException(HttpOperationFailedException.class).handled(true)
.retryAttemptedLogLevel(LoggingLevel.DEBUG)
.maximumRedeliveries(5).redeliveryDelay(3000)
.process(new HttpExceptionProcessor(exceptions));
from("jms:queue:supplier")
.process(
new RequestParserProcessor(payloadDetailsMap,
metaDataDetailsPOJO, routesEndpointNamePOJO))
.choice().when(new AggregateStrategy(metaDataDetailsPOJO))
.to("direct:aggregate").otherwise().to("direct:single");
from("direct:aggregate").process(new SplitBodyProcessor())
.split(body(), new AggregatePayload(aggregatePayload))
.to("direct:aggregatepayloadData").end()
.to("direct:payloadDataAggregated").end();
from("direct:aggregatepayloadData").process(basicProcessor)
.recipientList(header(ApplicationConstants.URL));
from("direct:payloadDataAggregated")
.process(
new AggregateJsonGenerator(aggregatePayload,
payloadDetailsMap, metaDataDetailsPOJO)).
In this code AggregateJsonProcessor is never called if there some invalid url on the first hit..
You probably need to set continue(true) in your OnException code. See here:
http://camel.apache.org/exception-clause.html
I have a big file and I use splitter to process it. I use .split().tokenize("\n", 5).streaming(); to group lines.
How can I send every group to different endpoint?
This should do the trick for you.
.split().tokenize("\n", 250000).streaming()
.to(file://directory)
.end()
You can also use another endpoint instead of .to(file://).