I'm currently trying to import some data from Oracle to ElasticSearch (in JSON format) using Apache Camel. I'm totally new on this framework, so I was thinking that you might help with it !
The problem is that my routeBuilder take the data from my table Entreprise .setBody(constant("select * from entreprise")).to("jdbc:myDataSource") and I get something like that : {id=1231, test=hello}, so I put it in a custom processor which add the quotes. I think this is quite ugly to modify a String because of the integers, booleans, arrays that shouldn't be between quotes, so I'd rather use an Object like a HashMap.
I've tried the apache-sql example which use Spring to create his route :
<route id="processOrder-route">
<from uri="sql:{{sql.selectOrder}}?consumer.onConsume={{sql.markOrder}}"/>
<to uri="bean:orderBean?method=processOrder"/>
<log message="${body}"/>
</route>
and finally get the informations into a HashMap :
public String processOrder(Map<String, Object> data) {
return "Processed order id " + data.get("id") + " item " + data.get("item") + " of " + data.get("amount") + " copies of " + data.get("description");
}
So my main question his : what is the equivalent to this route in Java DSL which could return a HashMap or something like that ?
Here is my code :
public class MainApp {
public static void main(String[] args) throws Exception {
String url = "jdbc:oracle:thin:alexis/alexis#localhost:1521:xe";
System.out.println("Setting up data source.");
DataSource dataSource = setupDataSource(url);
System.out.println("Done.");
SimpleRegistry reg = new SimpleRegistry() ;
reg.put("myDataSource", dataSource);
CamelContext context = new DefaultCamelContext(reg);
ProducerTemplate template = context.createProducerTemplate();
context.addRoutes(new MainApp().new MyRouteBuilder());
context.start();
Thread.sleep(3000);
context.stop();
}
class MyRouteBuilder extends RouteBuilder {
public void configure() {
Processor proc = new ConvertToJSON();
String dst = "C:/Users/**/Desktop/Workspace_sts/democamelJava/data";
from("timer://foo?period=2s")
.setBody(constant("select * from entreprise"))
.to("jdbc:myDataSource").split(body()).process(proc)
.convertBodyTo(String.class)
.to("file://" + dst);
//.to("elasticsearch://localhost:9200?operation=INDEX&indexName=twitter&indexType=tweet");
}
}
Thank you for every answer !
Related
I'm trying to find all routes associated with a camel context?
I can't find the camel exchange getContext() data sheets so I know what methods can be called?
I have a dynamic route builder, drop a config file and the route gets created.
I need to create the routes in registry, as not started, and use a JGroups/Controlbus route that controls who is the active route. But I can't figure out how to get all routes associated with a camel context? if you can shed some light on this, I'd really be in your dept. thanks in advance.
This is what I have but I can't get to work, found on stacktrace.
#Override
public void process(Exchange exchange) throws Exception {
List<ProcessorDefinition<?>> outputProcessorDefs = exchange.getContext().getRouteDefinition("[routeId]").getOutputs();
for ( ProcessorDefinition rte : outputProcessorDefs ) {
log.info("ROUTES: " + rte);
}
}
thanks to a question answered by Claus How to find all Endpoints of route (Apache Camel, Java)
I was able to locate some info and found another simpler way to do this.
public void process(Exchange exchange) throws Exception {
List<Route> routeList = exchange.getContext().getRoutes();
for ( Route rte : routeList ) {
log.info("ROUTES: " + rte.getId());
}
This is my test for JGroups ControlBus management made possible by getRoutes() for dynamic route creation.
public class EndpointControlBusFileRouteBuilder extends RouteBuilder {
private static final Logger log = LoggerFactory.getLogger(EndpointControlBusFileRouteBuilder.class);
private String routeId;
private String ClusterId;
public EndpointControlBusFileRouteBuilder(String routeId) {
this.routeId = routeId;
}
#Override
public void configure() throws Exception {
log.info("*** JGroups routeCluster - RouteId : " + routeId + " ***");
ClusterId = routeId + ".JGroups";
from("jgroups:" + ClusterId + "?enableViewMessages=true&channelProperties=etc/jgroups.xml")
.autoStartup(true)
.routeId(ClusterId)
.filter(dropNonCoordinatorViews())
.threads().delay(delayIfContextNotStarted(SECONDS.toMillis(5))) // run in separated and delayed thread. Delay only if the context hasn't been started already.
.log("Starting JGroups JChannel Routes Consumer!!!!!!!!!!!!!!!!!!!!!")
.to("controlbus:route?routeId=" + routeId + "&action=start&async=true");
}
}
public class EndpointControlBusProcessor implements Processor {
private String routeId = "";
private static final Logger log = LoggerFactory.getLogger(EndpointControlBusProcessor.class);
#Override
public void process(Exchange exchange) throws Exception {
List<Route> routeList = exchange.getContext().getRoutes();
ProducerTemplate template = exchange.getContext().createProducerTemplate();
for ( Route rte : routeList ) {
routeId = rte.getId();
// log.info("ROUTES: " + routeId);
// ServiceStatus routeStatus = exchange.getContext().getRouteStatus(routeId);
// log.info("Route " + routeId + " Status: " + routeStatus);
String status = template.requestBody("controlbus:route?routeId=" + routeId + "&action=status", null, String.class);
log.info("Controlbus Route Status: " + status + " for route: " + routeId);
if ( (null == status) || status.equalsIgnoreCase("Stopped") ) {
exchange.getContext().addRoutes(new EndpointControlBusFileRouteBuilder(routeId));
// status = template.requestBody("controlbus:route?routeId=" + routeId + "&action=status", null, String.class);
// log.info("Controlbus Route Status: " + status + " for route: " + routeId);
} else {
log.info("Route " + routeId + " already started");
}
}
template.stop();
}
}
This seems like an incredibly simple problem but I've tried everything I can think of. Basically I have a timer route that sends its message to a bunch of different beans. Those beans set a property on the exchange (I've also tried a header on the message) and I want the exchange output from all of those beans to be directed to a filter (which checks for the property or header) and then optionally another endpoint. Something like this:
---> Bean A ---
/ \
timer --> multicast ------> Bean B ------> end --> filter --> endpoint
\ /
---> Bean C ---
Currently the route looks like this, and it works for multicasting to the beans:
from("timer://my-timer?fixedRate=true&period=20000&delay=0")
.multicast()
.to("bean:beanA", "bean:beanB", "bean:beanC");
Here are the some of the solutions I've tried:
Solution 1
from("timer://my-timer?fixedRate=true&period=20000&delay=0")
.multicast()
.to("bean:beanA", "bean:beanB", "bean:beanC")
.filter(new myPredicate())
.to("myOptionalEndpoint");
This puts the filter in parallel with the beans instead of after them.
Solution 2
from("timer://my-timer?fixedRate=true&period=20000&delay=0")
.multicast()
.to("bean:beanA", "bean:beanB", "bean:beanC")
.end()
.filter(new myPredicate())
.to("myOptionalEndpoint");
Does the beans in parallel and then does the filter. However, the properties/headers are not set. It seems like the exchange is fresh off the timer and is not the one that went through the beans...
Edit: I tried setting the body and in fact the message that arrives at the filter has no body. I can't imagine Camel would somehow shuck the payload of the message so I have to assume that this exchange is a new one from the timer, not one that went through the beans. However, it happens after the beans are done.
Solution 3
from("timer://my-timer?fixedRate=true&period=20000&delay=0")
.multicast()
.beanRef("beanA").to("direct:temp")
.beanRef("beanB").to("direct:temp")
.beanRef("beanC").to("direct:temp")
.end()
from("direct:temp")
.filter(new myPredicate())
.to("myOptionalEndpoint");
Messages reach the filter as expected but the properties/headers that I set are gone so no messages pass the filter.
Edit: The body is gone here too so clearly I am not getting the same exchange that is coming from the beans...
To clarify, I am looking for a solution where the a single exchange from the timer is multicasted to each bean (so now we have 3 exchanges) and each of these 3 is then sent to the filter.
Can anybody help me figure out how to build this route?
You need to use an aggregation strategy in order to aggregate all the results into one.
Below is a great example from http://javarticles.com/2015/05/apache-camel-multicast-examples.html (See the Multicast with a Custom Aggregation Strategy section)
public class CamelMulticastAggregationExample {
public static final void main(String[] args) throws Exception {
JndiContext jndiContext = new JndiContext();
jndiContext.bind("myBean", new MyBean());
CamelContext camelContext = new DefaultCamelContext(jndiContext);
try {
camelContext.addRoutes(new RouteBuilder() {
public void configure() {
from("direct:start")
.multicast()
.aggregationStrategy(new JoinReplyAggregationStrategy())
.to("direct:a", "direct:b", "direct:c")
.end()
.to("stream:out");
from("direct:a")
.to("bean:myBean?method=addFirst");
from("direct:b")
.to("bean:myBean?method=addSecond");
from("direct:c")
.to("bean:myBean?method=addThird");
}
});
ProducerTemplate template = camelContext.createProducerTemplate();
camelContext.start();
template.sendBody("direct:start", "Multicast");
} finally {
camelContext.stop();
}
}
}
where JoinReplyAggregationStrategy class looks as follows
public class JoinReplyAggregationStrategy implements AggregationStrategy {
public Exchange aggregate(Exchange exchange1, Exchange exchange2) {
if (exchange1 == null) {
return exchange2;
} else {
String body1 = exchange1.getIn().getBody(String.class);
String body2 = exchange2.getIn().getBody(String.class);
String merged = (body1 == null) ? body2 : body1 + "," + body2;
exchange1.getIn().setBody(merged);
return exchange1;
}
}
}
UPDATE In your case, your aggregation strategy might be to gather all of your exchanges together as follows:
public class ListAggregationStrategy implements AggregationStrategy {
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
Message newIn = newExchange.getIn();
Object newBody = newIn.getBody();
List list = null;
if (oldExchange == null) {
list = new ArrayList();
list.add(newBody);
newIn.setBody(list);
return newExchange;
} else {
Message in = oldExchange.getIn();
list = in.getBody(List.class);
list.add(newBody);
return oldExchange;
}
}
}
Use scatter gather EIP instead of multicast !
Here is the solution, inspired by Kalman's:
from("timer://my-timer?fixedRate=true&period=20000&delay=0")
.multicast()
.to("direct:a", "direct:b", "direct:c")
.end()
from("direct:a").beanRef("beanA").to("direct:temp")
from("direct:b").beanRef("beanB").to("direct:temp")
from("direct:c").beanRef("beanC").to("direct:temp")
from("direct:temp")
.filter(new myPredicate())
.to("myOptionalEndpoint");
This was a more complicated solution that I was expecting. There must be a more elegant way to achieve this but the above solution works. Obviously use different names than a, b, c and temp though...
I am using struts-2.3.16 and I have to suppress exceptions from Freemarker template globally in our application. This means that, instead of the yellow screen with the stacktrace from Freemarker, I have to forward to a global jsp that displays a generic message, so preventing the display of stacktraces to the user. For generic exceptions in struts we mapped a global-results in struts.xml, but it's not working for Freemarker exceptions.
So far I have implemented the solution from What are different ways to handle error in FreeMarker template?. So I created a CustomFreemarkerManager and a CustomTemplateExceptionHandler.
My CustomFreemarkerManager looks like this:
#Override
public void init(ServletContext servletContext) throws TemplateException {
super.config = super.createConfiguration(servletContext);
super.config.setTemplateExceptionHandler(new CustomTemplateExceptionHandler(servletContext));
super.contentType = "text/html";
super.wrapper = super.createObjectWrapper(servletContext);
if (LOG.isDebugEnabled()) {
LOG.debug("Using object wrapper of class " + super.wrapper.getClass().getName(), new String[0]);
}
super.config.setObjectWrapper(super.wrapper);
super.templatePath = servletContext.getInitParameter("TemplatePath");
if (super.templatePath == null) {
super.templatePath = servletContext.getInitParameter("templatePath");
}
super.configureTemplateLoader(super.createTemplateLoader(servletContext, super.templatePath));
super.loadSettings(servletContext);
}
#Override
protected Configuration createConfiguration(ServletContext servletContext) throws TemplateException {
Configuration configuration = new Configuration();
configuration.setTemplateExceptionHandler(new CustomTemplateExceptionHandler(servletContext));
if (super.mruMaxStrongSize > 0) {
configuration.setSetting("cache_storage", "strong:" + super.mruMaxStrongSize);
}
if (super.templateUpdateDelay != null) {
configuration.setSetting("template_update_delay", super.templateUpdateDelay);
}
if (super.encoding != null) {
configuration.setDefaultEncoding(super.encoding);
}
configuration.setLocalizedLookup(false);
configuration.setWhitespaceStripping(true);
return configuration;
}
From here I send the ServletContext to my CustomTemplateExceptionHandler so I can create a RequestDispatcher to forward to my exception.jsp. The problem is that in the exception handler I don't have the request and the response and I can't forward to my jsp.
The class CustomTemplateExceptionHandler looks like this so far:
private ServletContext servletContext;
public CustomTemplateExceptionHandler(ServletContext servletContext) {
this.servletContext = servletContext;
}
public void handleTemplateException(TemplateException te, Environment env, Writer out) throws TemplateException {
if (servletContext != null) {
RequestDispatcher requestDispatcher = servletContext.getRequestDispatcher("/resources/exception.jsp");
//HERE I have to forward to my jsp
}
}
Anybody knows how can I do that? I want the stacktrace to be logged only on the server, and in the UI to replace the stacktrace with a generic message.
Ok, so my solution for this problem was to print on the PrintWriter that comes in my CustomTemplateExceptionHandler a response similar with the standard HTML_DEBUG_HANDLER offered by Freemarker. Check out this link:
https://github.com/apache/incubator-freemarker/blob/2.3-gae/src/main/java/freemarker/template/TemplateExceptionHandler.java#L98
Here you can see how HTML_DEBUG_HANDLER is managed. I replaced printing the stacktrace with a general message. The Freemarker documentation advise you to use RETHROW_HANDLER and catch the exception later in your application after the call of Template.process(). See here:
http://freemarker.org/docs/app_faq.html#misc.faq.niceErrorPage
But because Struts2 is working with Freemarker backstage, and the Freemarker methods are executed after the action was executed, I couldn't figure it out how and where to catch the exception. I have managed to get the HttpServlet response and request in the method handleTemplateException() (see the question), but I could not forward to my exception.jsp because the response was already committed and so it was giving me an exception.
The final code looks like this:
Class CustomFreemarkerManager:
#Override
public void init(ServletContext servletContext) throws TemplateException {
super.config = super.createConfiguration(servletContext);
super.config.setTemplateExceptionHandler(new CustomTemplateExceptionHandler());
super.contentType = "text/html";
super.wrapper = super.createObjectWrapper(servletContext);
if (LOG.isDebugEnabled()) {
LOG.debug("Using object wrapper of class " + super.wrapper.getClass().getName(), new String[0]);
}
super.config.setObjectWrapper(super.wrapper);
super.templatePath = servletContext.getInitParameter("TemplatePath");
if (super.templatePath == null) {
super.templatePath = servletContext.getInitParameter("templatePath");
}
super.configureTemplateLoader(super.createTemplateLoader(servletContext, super.templatePath));
super.loadSettings(servletContext);
}
#Override
protected Configuration createConfiguration(ServletContext servletContext) throws TemplateException {
Configuration configuration = new Configuration();
configuration.setTemplateExceptionHandler(new CustomTemplateExceptionHandler());
if (super.mruMaxStrongSize > 0) {
configuration.setSetting("cache_storage", "strong:" + super.mruMaxStrongSize);
}
if (super.templateUpdateDelay != null) {
configuration.setSetting("template_update_delay", super.templateUpdateDelay);
}
if (super.encoding != null) {
configuration.setDefaultEncoding(super.encoding);
}
configuration.setLocalizedLookup(false);
configuration.setWhitespaceStripping(true);
return configuration;
}
Class CustomTemplateExceptionHandler:
public void handleTemplateException(TemplateException te, Environment env, Writer out) throws TemplateException {
boolean externalPw = out instanceof PrintWriter;
PrintWriter pw = externalPw ? (PrintWriter) out : new PrintWriter(out);
try {
pw.print("<!-- ERROR MESSAGE STARTS HERE -->"
+ "<!-- ]]> -->"
+ "</table></table></table>"
+ "<div align='left' style='"
+ "background-color:#FFFF7C; "
+ "display:block; "
+ "border-top:double; "
+ "padding:10px; "
+ "'>");
pw.print("<b style='"
+ "color: red; "
+ "font-size:14px; "
+ "font-style:normal; "
+ "font-weight:bold; "
+ "'>"
+ "Oops! We have encountered a problem. Please try again!"
+ "</b>");
pw.println("</div></html>");
pw.flush(); // To commit the HTTP response
} finally {
if (!externalPw) pw.close();
}
throw te;
}
If anyone finds a better response to this please post your answer!
It should be easier. If you don't want the yellow debug template error you have to switch the TemplateExceptionHandler from HTML_DEBUG_HANDLER to RETHROW_HANDLER (as the freemarker message on the top suggest: FreeMarker template error DEBUG mode; use RETHROW in production!)
Now, I prefer to do it programmatically (as you did) because I want to choose the TemplateExceptionHandler depending on the environment (PRO, TEST, DEVEL), my solution is to put environment information in the context.
public class CustomFreemarkerManager extends FreemarkerManager {
public CustomFreemarkerManager(){
super();
}
#Override
public void init(ServletContext servletContext) throws TemplateException {
//important!
super.init(servletContext);
//other stuff maybe you want to tune...
//Getting environmentInfo object from the context, it's a personal solution
EnvironmentInfo environmentInfo = (EnvironmentInfo)servletContext.getAttribute(EnvironmentInfo.CONTEXT_NAME);
if (environment.isPro()) {
config.setTemplateExceptionHandler(TemplateExceptionHandler.RETHROW_HANDLER);
}else{
config.setTemplateExceptionHandler(TemplateExceptionHandler.HTML_DEBUG_HANDLER);
}
}
}
and tell struts to use your manager in struts.properties
struts.freemarker.manager.classname=com.jobisjob.northpole.web.core.CustomFreemarkerManager
Hope this helps.
I have the following test:
public class MyTest extends CamelSpringTestSupport {
#Override
protected AbstractXmlApplicationContext createApplicationContext() {
return new ClassPathXmlApplicationContext(new String[] {
"classpath:my-config.xml",
});
}
#Test
public void testSomething() throws Exception {
Exchange exchange = new DefaultExchange(context);
exchange.getIn().setHeader("myId", "1234");
MessageContentsList parameters = new MessageContentsList();
parameters.add(0, "");
parameters.add(1, "1234");
exchange.getIn().setBody(parameters);
System.out.println("exchange before = " + System.identityHashCode(exchange)); //1657040467
template.send("direct:myRoute", exchange);
Object object = exchange.getOut().getBody();
System.out.println("result = " + object); //null
System.out.println("exchange after = " + System.identityHashCode(exchange)); //1657040467
assertFalse(exchange.isFailed());
}
}
Here is the last step inside the route "direct:myRoute":
public void doSomething(Exchange exchange)
{
System.out.println("exchange within = " + System.identityHashCode(exchange)); //1649140180
exchange.getOut().setBody(1);
}
My routes are defined by RouteBuilder classes and I load them by
<camelContext id="mainContext" xmlns="http://camel.apache.org/schema/spring">
<packageScan>
<package>com.my.route</package>
</packageScan>
</camelContext>
and also dynamically by code:
((CamelContext) applicationContext.getBean("mainContext")).addRoutes(routeBuilder);
As you can see, I cannot retrieve the output of the route when I call the route from the test, because within the test for some reason it is a different exchange object (however, the original exchange headers and properties are somehow copied to the exchange within the route).
This problem does not happen when I define the routes by XML.
Why is that and how could I retrieve the result of the route when I call it from a test?
I am not 100% sure but I don't you think you can write like this.
template.send("direct:myRoute", exchange);
Object object = exchange.getOut().getBody();
It should ideally be:
Exchange out=null;
out=template.requestBody("direct:myRoute", exchange);
String body=out.getOut().getBody();
For this to work you need to set the exchange pattern to InOut.
More info on the templates:
https://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/ProducerTemplate.html
I am pretty new with Camel. I have been trying to submit a data (Json from a file) to a webservice. This is my code:
public static void main(String args[]) throws Exception {
// create CamelContext
CamelContext context = new DefaultCamelContext();
// add our route to the CamelContext
context.addRoutes(new RouteBuilder() {
#Override
public void configure() {
from("file:data/inbox?noop=true")
.marshal()
.string()
.setHeader(Exchange.CONTENT_TYPE,constant("application/json"))
.to("http://www.a-service.com");
}
});
// start the route and let it do its work
context.start();
Thread.sleep(10000);
// stop the CamelContext
context.stop();
}
Then the webservice will response with Json which can be
{result:OK}
or
{result:FAIL}
Now, if a response has responseCode as 200, Camel will consider as success.
My question is, how can I have a validating process for responsed JSon so that if it is FAIL, Camel should not consider as success?
Solution Credit #Namphibian:
By adding processor and the end. This code has been tested:
from("file:data/inbox?noop=true")
.marshal()
.string("UTF-8")
.setHeader(Exchange.CONTENT_TYPE,constant("application/json"))
.to("http://monrif-test.userspike.com/monrif/rss/monrif_-all-global")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
Message in = exchange.getIn();
String msg = in.getBody(String.class);
System.out.println("Response: " + msg);
if(msg.contains("OK")){
// go to party
}else{
throw new Exception("test exception");
}
}
});
There are two broad strategies you can use to achieve this.
Processor Based:
Add a processor to the end of the route. In this processor do the check if the webservice then responds with a true or false value.
A processor would look something like this:
package com.example;
import java.util.Map;
import org.apache.camel.Body;
import org.apache.camel.Exchange;
import org.apache.camel.Handler;
import org.apache.camel.Headers;
import org.apache.camel.Message;
public class GeneralProcessor {
#Handler
public void PrepapreErrorImportReport
(
#Headers Map hdr
, Exchange exch
)
{
//todo: Get the message as a string;
Message in = exch.getIn();
String msg = (String)in.getBody();
// Now check if body contains failed or ok.
if(msg.contains("OK")){
//todo: go party the message was OK
}
else{
//todo: Oh Oh! Houston we have a problem
}
}
}
You can then modify your route to use this processor.
The Simple Expression Language
This is one way the other way is to use the simple expression language. See the example below on how to use this.
from("file:data/inbox?noop=true")
.marshal()
.string()
.setHeader(Exchange.CONTENT_TYPE,constant("application/json"))
.to("http://www.a-service.com")
.choice()
.when(simple("${body} contains 'OK'")).to("activemq:okqueue")
.otherwise().to("activemq:queue:other");
Notice the simple("${body} contains 'OK'") piece of code. That is the power of simple.
Both approaches have uses.
In the Process method , you can use below method and it will work
LOGGER.info("Response code " + message.getHeader(exchange.HTTP_RESPONSE_CODE, Integer.class));