Develop Reference

How to investigate data on Apache Camel Route?

My project is working on getting data from one system to another. We are using Apache Camel Routes to send the data between JBoss EAP v7 servers. My question is, is there a way to investigate what the content of the packages are as they come across different routes?
We have tried upping the logging but our files/console just get flooded. We have also tried to use Hawtio on the server to see the messages coming across the routes but have had no success identifying where our message is getting "stuck".
Any help is appreciated!

You can use unit tests to test your routes locally and then either log contents of the exchange at specific points using adviceWith and weave methods.
With unit tests you can easily debug your routes in your favourite IDE even if you're running camel in something like Karaf or Red Hat fuse.
package com.example;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.RoutesBuilder;
import org.apache.camel.builder.AdviceWithRouteBuilder;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.mock.MockEndpoint;
import org.apache.camel.model.dataformat.JsonLibrary;
import org.apache.camel.test.junit4.CamelTestSupport;
import org.junit.Test;
public class ExampleRouteTests extends CamelTestSupport {
#Test
public void exampleTest() throws Exception
{
ContractDetails testDetails = new ContractDetails(1512, 1215);
mockJDBCEndpoints();
context.getRouteDefinition("exampleRoute")
.adviceWith(context, new AdviceWithRouteBuilder(){
#Override
public void configure() throws Exception {
replaceFromWith("direct:start");
weaveByToUri("direct:getDetailsFromAPI")
.replace()
.to("log:testLogger?showAll=true")
.to("mock:api")
.setBody(constant(testDetails));
weaveByToUri("direct:saveToDatabase")
.replace()
.to("log:testLogger?showAll=true")
.to("mock:db");
}
});
MockEndpoint apiMockEndpoint = getMockEndpoint("mock:api");
apiMockEndpoint.expectedMessageCount(1);
MockEndpoint dbMockEndpoint = getMockEndpoint("mock:db");
dbMockEndpoint.expectedMessageCount(1);
context.start();
String body = "{\"name\":\"Bob\",\"age\":10}";
template.sendBody("direct:start", body);
apiMockEndpoint.assertIsSatisfied();
dbMockEndpoint.assertIsSatisfied();
}
#Override
protected RoutesBuilder createRouteBuilder() throws Exception {
return new RouteBuilder(){
#Override
public void configure() throws Exception {
from("amqp:queue:example")
.routeId("exampleRoute")
.unmarshal().json(JsonLibrary.Jackson,
Person.class)
.to("direct:getDetailsFromAPI")
.process(new SomeProcessor())
.to("direct:saveToDatabase");
from("direct:saveToDatabase")
.routeId("saveToDatabaseRoute")
.to("velocity:sql/insertQueryTemplate.vt")
.to("jdbc:exampleDatabase");
from("direct:getDetailsFromAPI")
.removeHeaders("*")
.toD("http4:someAPI?name=${body.getName()}")
.unmarshal().json(JsonLibrary.Jackson,
ContractDetails.class);
}
};
}
void mockJDBCEndpoints() throws Exception {
context.getRouteDefinition("saveToDatabaseRoute")
.adviceWith(context, new AdviceWithRouteBuilder(){
#Override
public void configure() throws Exception {
weaveByToUri("jdbc:*")
.replace()
.to("mock:db");
}
});
}
#Override
public boolean isUseAdviceWith() {
return true;
}
}
Now for troubleshooting problems that do not occur with unit tests you can configure generic or route specific exception handling with onException and use Dead letter channel to process and and store information about the failed exchange. Alternatively you can just use stream or file component to save information about the exception and failed exchange in to a separate file to avoid flooding logs.

Spring-batch : How to catch exception message with skip method in spring batch?

I am a novice of spring batch.
My question is how to catch exceptions with the skip method in spring-batch?
As I know, we can use a skip method to skip them when some exceptions happened in spring batch.
But how can I get the exception message with skip method?
somebody suggested me use SkipListener ,this class has three call back method like onSkipInProcess(),but it is no use for me.
And ItemProcessListener did not work either.
The code like below:(I use skip method to ignore the exception and two listeners to receive the exception info)
Step mainStep = stepBuilder.get("run")
.<ItemProcessing, ItemProcessing>chunk(5)
.faultTolerant()
.skip(IOException.class).skip(SocketTimeoutException.class)//skip IOException here
.skipLimit(2000)
.reader(reader)
.processor(processor)
.writer(writer)
.listener(stepExecListener)
.listener(new ItemProcessorListener()) //add process listener
.listener(new SkipExceptionListener()) //add skip exception listner
.build();
ItemProcessorListener like below:
//(this class implements ItemProcessListener )
{
#Override
public void beforeProcess(Object item) {
// TODO Auto-generated method stub
}
#Override
public void afterProcess(Object item, Object result) {
logger.info("invoke remote finished, item={},result={}",item,result);
}
#Override
public void onProcessError(Object item, Exception e) {
logger.error("invoke remote error, item={},exception={},{}",item,e.getMessage(),e);
}
}
SkipExceptionListener like below:
//(implements SkipListener<Object, Object>)
{
#Override
public void onSkipInRead(Throwable t) {
// TODO Auto-generated method stub
}
#Override
public void onSkipInWrite(Object item, Throwable t) {
// TODO Auto-generated method stub
}
#Override
public void onSkipInProcess(Object item, Throwable t) {
logger.info("invoke remote finished,item={},itemJsonStr={},errMsg={},e={}",
item,
JSONObject.toJSONString(item),
t.getMessage(),
t);
}
}
The issue is that all logger did not work. Actually the skip method does work well, I can get the skip count in table batch_step_execution. I am not sure these two listeners whether be callback. Who can tell me how can I do? Or is there anything else? Thanks a lot.

How to catch exception message with skip method in spring batch?
You can do that by implementing the SkipListener interface or extending the SkipListenerSupport class. All methods in the SkipListener interface have a Throwable parameter which is the exception thrown and that caused the item to be skipped. This is where you can get the exception message. Here is an example:
import java.util.Arrays;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.SkipListener;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.support.ListItemReader;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
#Configuration
#EnableBatchProcessing
public class MyJob {
#Autowired
private JobBuilderFactory jobs;
#Autowired
private StepBuilderFactory steps;
#Bean
public ItemReader<Integer> itemReader() {
return new ListItemReader<>(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9, 10));
}
#Bean
public ItemWriter<Integer> itemWriter() {
return items -> {
for (Integer item : items) {
System.out.println("item = " + item);
}
};
}
#Bean
public ItemProcessor<Integer, Integer> itemProcessor() {
return item -> {
if (item.equals(7)) {
throw new IllegalArgumentException("Sevens are not accepted!!");
}
return item;
};
}
#Bean
public Step step() {
return steps.get("step")
.<Integer, Integer>chunk(5)
.reader(itemReader())
.processor(itemProcessor())
.writer(itemWriter())
.faultTolerant()
.skip(IllegalArgumentException.class)
.skipLimit(3)
.listener(new MySkipListener())
.build();
}
#Bean
public Job job() {
return jobs.get("job")
.start(step())
.build();
}
public static class MySkipListener implements SkipListener<Integer, Integer> {
#Override
public void onSkipInRead(Throwable t) {
}
#Override
public void onSkipInWrite(Integer item, Throwable t) {
}
#Override
public void onSkipInProcess(Integer item, Throwable t) {
System.out.println("Item " + item + " was skipped due to: " + t.getMessage());
}
}
public static void main(String[] args) throws Exception {
ApplicationContext context = new AnnotationConfigApplicationContext(MyJob.class);
JobLauncher jobLauncher = context.getBean(JobLauncher.class);
Job job = context.getBean(Job.class);
jobLauncher.run(job, new JobParameters());
}
}
In this example, MySkipListener implements SkipListener and gets the message from the exception as you are trying to do. The example reads numbers from 1 to 10 and skips number 7. You can run the main method and should see the following output:
item = 1
item = 2
item = 3
item = 4
item = 5
item = 6
item = 8
item = 9
item = 10
Item 7 was skipped due to: Sevens are not accepted!!
I hope this helps.

I couldn't get it to work either with implementing SkipListener (would be nice to know why) but in the end I went with the annotation way which is only briefly mentioned in the docs. Also somebody had a similar issue here using the implementation method (question) and the guy in the answer uses this annotation method instead of implementing the interface.
Example bean:
#Component
public class CustomSkippedListener {
#OnSkipInRead
public void onSkipInRead(Throwable throwable) {
}
#OnSkipInWrite
public void onSkipInWrite(FooWritingDTO fooWritingDTO, Throwable throwable) {
LOGGER.info("balabla" + throwable.getMessage());
}
#OnSkipInProcess
public void onSkipInProcess(FooLoaderDTO fooLoaderDTO, Throwable throwable) {
LOGGER.info("blabla" + throwable.getMessage());
}
private static final Logger LOGGER = LoggerFactory.getLogger(CustomSkippedListener.class);
}
then autowire and include in the step chain as you did.

Adding REST route to an existing Jetty endpoint in Camel at runtime

I have been inventing a way how to work around the problem of adding consumers to a jetty endpoint (it does not allow multiple consumers). The way we do it in our company is to build our own router and a broadcasting endpoint which consumes from jetty and routes requests to underlying "subscriptions". Only one of them will eventually process the request. It kind of works but it's not completely ok, since recently when updating to latest Camel we have found our custom built component to leak memory and in general I consider using built-in functionality over custom hacks.
I started investigating the Camel REST API and found it very nice and pretty much replacing our home-grown component apart from one thing - you cannot re-configure it at runtime - you have to stop the context basically for this to work. Below I include my unit test with a happy path and the path that fails. Frankly I think is a bug, but if there is a legitimate way to achieve what I want, I'd like to hear sound advice:
package com.anydoby.camel;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.fail;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
import org.apache.commons.io.IOUtils;
import org.junit.Before;
import org.junit.Test;
/**
* Test tries to add/remove routes at runtime.
*/
public class RoutesTest {
private DefaultCamelContext ctx;
#Before
public void pre() throws Exception {
ctx = new DefaultCamelContext();
new RouteBuilder(ctx) {
#Override
public void configure() throws Exception {
restConfiguration("jetty").host("localhost").port(8080);
rest("/")
.get("/issues/{isin}").route().id("issues")
.process(e -> e.getOut().setBody("Here's your issue " + e.getIn().getHeader("isin"))).endRest()
.get("/listings").route().id("listings").process(e -> e.getOut().setBody("some listings"));
}
}.addRoutesToCamelContext(ctx);
ctx.start();
}
#Test
public void test() throws IOException {
{
InputStream stream = new URL("http://localhost:8080/issues/35").openStream();
assertEquals("Here's your issue 35", IOUtils.toString(stream));
}
{
InputStream stream = new URL("http://localhost:8080/listings").openStream();
assertEquals("some listings", IOUtils.toString(stream));
}
}
#Test
public void disableRoute() throws Exception {
ctx.stopRoute("issues");
ctx.removeRoute("issues");
try (InputStream stream = new URL("http://localhost:8080/issues/35").openStream()) {
fail();
} catch (Exception e) {
}
new RouteBuilder(ctx) {
#Override
public void configure() throws Exception {
rest().get("/issues/{isin}/{sedol}").route().id("issues")
.process(e -> e.getOut()
.setBody("Here's your issue " + e.getIn().getHeader("isin") + ":" + e.getIn().getHeader("sedol")))
.endRest();
}
}.addRoutesToCamelContext(ctx);
{
InputStream stream = new URL("http://localhost:8080/issues/35/65").openStream();
assertEquals("Here's your issue 35:65", IOUtils.toString(stream));
}
}
}
The disableRoute() test fails since I cannot add another consumer to an existing endpoint.
So my question is - "is there a way to add a new URL mapping to a restful camel-jetty endpoint"? If you do it during first configuration it works fine, but when later you want to reconfigure one of the routes the error is:
org.apache.camel.FailedToStartRouteException: Failed to start route because of Multiple consumers for the same endpoint is not allowed: jetty:http://localhost:8080/issues/%7Bisin%7D/%7Bsedol%7D?httpMethodRestrict=GET

Getting error: Route() in Route cannot be applied to String

I'm designing a Java based MongoDB app and I've ran into a snag when working with Spark.
package com.tengen;
import spark.Request;
import spark.Response;
import spark.Route;
import spark.Spark;
public class HelloWorldSparkStyle {
public static void main(String[] args) {
Spark.get(new Route("/") {
#Override
public Object handle(Request request, Response response) {
return "Hello World From Spark";
}
});
}
}
On new Route("/") I get the error Route() in route cannot be applied to java.lang.string.
I'm confused as to why this doesn't work as I've followed their code exactly.

This should probably be posted on the MongoDB class forum, but I ran into a similar issue. Looks like the get method changed from when the course material was produced. The get now requires a path and a Route
get(path, Route)
import spark.Request;
import spark.Response;
import spark.Route;
import spark.Spark;
public class HelloWorldSparkStyle {
public static void main(String[] args){
Spark.get("/", new Route() {
public Object handle(final Request request, final Response response){
return "Hello World from Spark";
}
});
}
}

Actually I used spark-core-1.1.1.jar it worked fine for me, may be the newer versions of Spark(version 2.0.0) support some different syntax.So if you are using the latest version then you can follow the example given by Mike or just add spark-core-1.1.1.jar to your classpath your example will work fine

That would work in Spark 1. In Spark 2 is recommended by Spark the following (source: http://sparkjava.com/news.html):
import static spark.Spark.*;
public class HelloWorld {
public static void main(String[] args) {
get("/", (req, res) -> "Hello World From Spark");
}
}

Currently, Implementation of Spark Java framework needs to use directory out of Route. So you have to correct your code as follow:
public static void main(String[] args) {
spark.Spark.port(PortNumber);
Spark.get("/", new Route() {
public Object handle(Request request, Response response) throws Exception {
return "This is a sample page";
}
});
}
actually, "/" is the resource to your program. And if you want to change the default spark.Spark.port(PortNumber).

Change the version of Spark in the POM file from the exercise files you download from handout. That fixed issue for me.

Play GlobalSettings Override not working

I've been using the onStart method for a while and it works no problem, but when I try to override onBadRequest, I get an error:
Here's the class:
import models.User;
import play.Application;
import play.GlobalSettings;
import play.Logger;
import play.mvc.Result;
import views.html.error_page;
import static play.mvc.Results.badRequest;
public class Global extends GlobalSettings {
#Override
public void onStart(Application app){
Logger.info("Application started!");
// Check if the database is empty
if(User.find.findRowCount()==0){
Ebean.save((List) Yaml.load("initial-data.yml"));
}
}
#Override
public void onStop(Application app){
Logger.info("Application stopped!");
}
#Override
public Result onBadRequest(String uri, String error) {
Logger.info("Bad Request");
return badRequest(error_page.render());
}
}
The first two work no problem, but the third one causes the error.
Here's the API entry:

You used the old API.
Here's the API for Play 2.1.1:
http://www.playframework.com/documentation/api/2.1.1/java/play/GlobalSettings.html
Called when an action has been found, but the request parsing has failed.
Result onBadRequest(Http.RequestHeader request, java.lang.String error)
Called when an exception occurred.
Result onError(Http.RequestHeader request, java.lang.Throwable t)

This works:
#Override
public Result onError(Http.RequestHeader request, Throwable t){
Logger.error("Error thrown: " + t.getMessage());
return internalServerError(error_page.render());
}
This page was useful. I'm still confused why the API is wrong.
I hope this helps out somebody with a similar problem.