I've got a fairly simple route:
PerfStubRouteBuilder.java
public class PerfStubRouteBuilder extends SpringRouteBuilder {
/* (non-Javadoc)
* #see org.apache.camel.builder.RouteBuilder#configure()
*/
#Override
public void configure() throws Exception {
from("direct:test-aces-input")
.log("Creating test Accident Number header")
.setHeader("AccidentNumber", simple("AB999999999"))
.log("Test Accident Number header created : ${header.AccidentNumber}")
.end();
}
}
and I'm testing it as follows:
PerfStubRouteBuilderTest.java
public class PerfStubRouteBuilderTest extends CamelTestSupport {
#Produce(uri = "direct:test-aces-input")
ProducerTemplate template;
PerfStubRouteBuilder route = new PerfStubRouteBuilder();
Exchange exch = new DefaultExchange(context);
#Override
protected RouteBuilder createRouteBuilder() {
return route;
}
#Test
public void test_PerfStubRouteBuilder_happyPath_addsAccidentNumberHeaderToExchange() throws Exception {
startCamelContext();
NotifyBuilder notify = new NotifyBuilder(context).from("direct:test-aces-input").whenDone(1).filter(header("AccidentNumber").isEqualTo("AB999999999")).create();
template.send(exch);
assertTrue(notify.matches(10, TimeUnit.SECONDS));
stopCamelContext();
}
}
assertTrue always come back as false, so I suspect I'm not using the NotifyBuilder correctly although at this point I can't be certain. Essentially I want to check that an exchange makes it through the route and the header specified in the actual route is added to the exchange. What I want to happen is for a match to occur if an exchange with that header value combo makes it to the end of the route, hence the filter step. I want to avoid adding an endpoint at the end of the route, for example, via AdviceWith given how small and simple the route itself is , it seems a bit heavyweight to start adding in mock endpoints for such a minor test
UPDATE:
Tried removing the filter portion from the expression, leaving the NotifyBuilder as NotifyBuilder notify = new NotifyBuilder(context).from("direct:test-aces-input").whenDone(1).create();
, the test still fails
Create the exchange from the endpoint in the test like
Endpoint endpoint = context.getEndpoint("direct:test-aces-input");
Exchange exchange = endpoint.createExchange();
template.send(exchange);
or just directly use sendBody
tempalte.sendBody("")
I have a messaging application built over redis, however, I noticed the spring data redis template's convertAndSend may be duplicating messages as the message listener receives duplicate messages one in every three trials.
As you can imagine this may not be good for certain applications, in my secondary storage is complaining about duplicate keys.
I register the message listener in a #Configuration annotated class as:
#Bean
RedisMessageListenerContainer container(JobsListener receiver, RedisConnectionFactory connectionFactory) {
MessageListenerAdapter jobsMessageListener = new MessageListenerAdapter(receiver);
RedisMessageListenerContainer container = new RedisMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.addMessageListener(jobsMessageListener, new PatternTopic(RedisCacheService.JOBS_KEY));
return container;
}
And in the JobsListener implementation, I use the onMessageReceived method.
#Override
public void onMessage(Message message, byte[] pattern) {
System.out.println(new String(message.getBody()));
Job job = cacheService.processNextJob();
if (job != null) {
logger.debug("Job id processed is " + job.getId() + " " + Thread.currentThread().getId());
update(job);
} else {
logger.debug("Job id processed is null");
}
}
However, if I add synchronized to the onMessageReceived method it seems to fix this.
Is there a reason why synchronized helps? Smells like some concurrency issue under the hood.
Functionally I wish to check a URL is active before I consume from a JMS (WMQ) endpoint.
If the URL cannot be reached or a server error, then I do not want to pick up from the queue. So I want to keep trying (with unlimited retries) the URL via a polling consumer. So as soon as it is available I can pick up from JMS.
I have a RouteBuilder that is set up with a direct endpoint, that is configured to run a Processor that will ping a service.
So:
public class PingRoute extends RouteBuilder {
#Override
public void configureCamel() {
from("direct:pingRoute").routeId(PingRoute.class.getSimpleName())
.process(new PingProcessor(url))
.to("log://PingRoute?showAll=true");
}
}
In another route I am setting up my timer:
#Override
public void configureCamel() {
from(timerEndpoint).beanRef(PollingConsumerBean.class.getSimpleName(), "checkPingRoute");
...
}
And with the PollingConsumerBean I am attempting to receive the body via a consumer:
public void checkPingRoute(){
// loop to check the consumer. Check we can carry on with the pick up from the JMS queue.
while(true){
Boolean pingAvailable = consumer.receiveBody("direct:pingRoute", Boolean.class);
...
}
I add the route to the context and use a producer to send:
context.addRoutes(new PingRoute());
context.start();
producer.sendBody(TimerPollingRoute.TIMER_POLLING_ROUTE_ENDPOINT, "a body");
And I get the following IllegalArgumentException:
Cannot add a 2nd consumer to the same endpoint. Endpoint Endpoint[direct://pingRoute] only allows one consumer.
Is there a way to setup the direct route as a polling consumer?
Business logic is not quite clear, unfortunately. As I understand it - you need to wait for a response from the service. IMHO you have to use Content Enricher EIP http://camel.apache.org/content-enricher.html . pollEnrich is what you need at timer route.
.pollEnrich("direct:waitForResponce", -1) or
.pollEnrich("seda:waitForResponce", -1)
public class PingRoute extends RouteBuilder {
#Override
public void configureCamel() {
from("direct:pingRoute").routeId(PingRoute.class.getSimpleName())
.process(new PingProcessor(url))
.choice().when(body())
.to("log://PingRoute?showAll=true")
.to("direct:waitForResponce")
.otherwise()
.to("direct:pingRoute")
.end();
}
};
timer:
#Override
public void configureCamel() {
from(timerEndpoint)
.inOnly("direct:pingRoute")
.pollEnrich("direct:waitForResponce", -1)
...
}
Based on the OP's clarification of their use case, they have several problems to solve:
Consume the message from the JMS queue if, and only if, the ping to the URL is positive.
If the URL is unresponsive, the JMS message should not disappear from the queue and a retry must take place until the URL becomes responsive again, in which case the message will be ultimately consumed.
The OP has not specified if the amount of retries is limited or unlimited.
Based on this problem scenario, I suggest a redesign of their solution that leverages ActiveMQ retries, broker-side redelivery and JMS transactions in Camel to:
Return the message to the queue if the URL ping failed (via a transaction rollback).
Ensure that the message is not lost (by using JMS persistence and broker-side redeliveries, AMQ will durably schedule the retry cycle).
Be able to specify a sophisticated retry cycle per message, e.g. with exponential backoffs, maximum retries, etc.
Optionally sending the message to a Dead Letter Queue if the retry cycle was exhausted without a positive result, so that some other (possibly manual) action can be planned.
Now, implementation-wise:
from("activemq:queue:abc?transacted=true") // (1)
.to("http4://host.endpoint.com/foo?method=GET") // (2) (3)
.process(new HandleSuccess()); // (4)
Comments:
Note the transacted flag.
If the HTTP invocation fails, the HTTP4 endpoint will raise an Exception.
Since there are no configured exception handlers, Camel will propagate the exception to the consumer endpoint (activemq) which will rollback the transaction.
If the invocation succeeded, the flow will continue and the exchange body will now contain the payload returned by the HTTP server and you can handle it in whichever way you wish. Here I'm using a processor.
Next, what's important is that you configure the redelivery policy in ActiveMQ, as well as enable broker-side redeliveries. You do that in your activemq.xml configuration file:
<plugins>
<redeliveryPlugin fallbackToDeadLetter="true" sendToDlqIfMaxRetriesExceeded="true">
<redeliveryPolicyMap>
<redeliveryPolicyMap>
<redeliveryPolicyEntries>
<redeliveryPolicy queue="my.queue"
initialRedeliveryDelay="30000"
maximumRedeliveries="17"
maximumRedeliveryDelay="259200000"
redeliveryDelay="30000"
useExponentialBackOff="true"
backOffMultiplier="2" />
</redeliveryPolicyEntries>
</redeliveryPolicyMap>
</redeliveryPolicyMap>
</redeliveryPlugin>
</plugins>
And make sure that the scheduler support is enabled in the top-level <broker /> element:
<broker xmlns="http://activemq.apache.org/schema/core"
brokerName="mybroker"
schedulerSupport="true">
...
</broker>
I hope that helps.
EDIT 1: OP is using IBM WebSphere MQ as a broker, I missed that. You could use a JMS QueueBrowser to peek at messages and try their corresponding URLs before actually consuming a message, but it is not possible to selectively consume an individual message – that's not what MOM (messaging-oriented middleware) is about.
So I insist that you should explore JMS transactions, but rather than leaving it up to the broker to redeliver the message, you can start the pinging cycle to the URL within the TX body itself. With regards to Camel, you could implement it as follows:
from("jms:queue:myqueue?transacted=true")
.bean(new UrlPinger());
UrlPinger.java:
public class UrlPinger {
#EndpointInject
private ProducerTemplate template;
private Pattern pattern = Pattern.compile("^(http(?:s)?)\\:");
#Handler
public void pingUrl(#Body String url, CamelContext context) throws InterruptedException {
// Replace http(s): with http(s)4: to use the Camel HTTP4 endpoint.
Matcher m = pattern.matcher(url);
if (m.matches()) {
url = m.replaceFirst(m.group(1) + "4:");
}
// Try forever until the status code is 200.
while (getStatusCode(url, context) != 200) {
Thread.sleep(5000);
}
}
private int getStatusCode(String url, CamelContext context) {
Exchange response = template.request(url + "?method=GET&throwExceptionOnFailure=false", new Processor() {
#Override public void process(Exchange exchange) throws Exception {
// No body since this is a GET request.
exchange.getIn().getBody(null);
}
});
return response.getIn().getHeader(Exchange.HTTP_RESPONSE_CODE, Integer.class);
}
}
Notes:
Note the throwExceptionOnFailure=false option. An Exception will not be raised, therefore the loop will execute until the condition is true.
Inside the bean, I'm looping forever until the HTTP status is 200. Of course, your logic will be different.
Between attempt and attempt, I'm sleeping 5000ms.
I'm assuming the URL to ping is in the body of the incoming JMS message. I'm replacing the leading http(s): with http(s)4: in order to use the Camel HTTP4 endpoint.
Performing the pinging inside the TX guarantees that the message will only be consumed once the ping condition is true (in this case HTTP status == 200).
You might want to introduce a desist condition (you don't want to keep trying forever). Maybe introduce some backoff to not overwhelm the other party.
If either Camel or the broker goes down within a retry cycle, the message will be automatically rolled back.
Take into account that JMS transactions are Session-bound, so if you want to start many concurrent consumers (concurrentConsumers JMS endpoint option), you'll need to set cacheLevelName=CACHE_NONE for each thread to use a different JMS Session.
I am having a bit of difficulty figuring out exactly what you want to do, but it appears to me that you want to consume data from an endpoint on an interval. For this the best pattern is a polling consumer: http://camel.apache.org/polling-consumer.html
The error you are currently receiving is because you have two consumers both trying to read from the "direct://pingRoute" If this was intended you could change the direct to a seda://pingRoute so its an in memory queue your data will be in.
All the answers here pointed me on the right direction but I finally came up with a solution that managed to fit our code base and framework.
Firstly, I discovered there isn't a need to have bean to act as a polling consumer but a processor could be used instead.
#Override
public void configureCamel() {
from("timer://fnzPoller?period=2000&delay=2000").processRef(UrlPingProcessor.class.getSimpleName())
.processRef(StopStartProcessor.class.getSimpleName()).to("log://TimerPollingRoute?showAll=true");
}
Then in the UrlPingProcessor there is CXF service to ping the url and can check the response :
#Override
public void process(Exchange exchange) {
try {
// CXF service
FnzPingServiceImpl fnzPingService = new FnzPingServiceImpl(url);
fnzPingService.getPing();
} catch (WebApplicationException e) {
int responseCode = e.getResponse().getStatus();
boolean isValidResponseCode = ResponseCodeUtil.isResponseCodeValid(responseCode);
if (!isValidResponseCode) {
// Sets a flag to stop for the StopStartProcessor
stopRoute(exchange);
}
}
}
Then in the StopStartProcessor it is using a ExecutorService to stop or start a route via new thread.:
#Override
public void process(final Exchange exchange) {
// routeBuilder is set on the constructor.
final String routeId = routeBuilder.getClass().getSimpleName();
Boolean stopRoute = ExchangeHeaderUtil.getHeader(exchange, Exchange.ROUTE_STOP, Boolean.class);
boolean stopRoutePrim = BooleanUtils.isTrue(stopRoute);
if (stopRoutePrim) {
StopRouteThread stopRouteThread = new StopRouteThread(exchange, routeId);
executorService.execute(stopRouteThread);
} else {
CamelContext context = exchange.getContext();
Route route = context.getRoute(routeId);
if (route == null) {
try {
context.addRoutes(routeBuilder);
} catch (Exception e) {
String msg = "Unable to add a route: " + routeBuilder;
LOGGER.warn(msg, e);
}
}
}
}
I have the following test:
public class MyTest extends CamelSpringTestSupport {
#Override
protected AbstractXmlApplicationContext createApplicationContext() {
return new ClassPathXmlApplicationContext(new String[] {
"classpath:my-config.xml",
});
}
#Test
public void testSomething() throws Exception {
Exchange exchange = new DefaultExchange(context);
exchange.getIn().setHeader("myId", "1234");
MessageContentsList parameters = new MessageContentsList();
parameters.add(0, "");
parameters.add(1, "1234");
exchange.getIn().setBody(parameters);
System.out.println("exchange before = " + System.identityHashCode(exchange)); //1657040467
template.send("direct:myRoute", exchange);
Object object = exchange.getOut().getBody();
System.out.println("result = " + object); //null
System.out.println("exchange after = " + System.identityHashCode(exchange)); //1657040467
assertFalse(exchange.isFailed());
}
}
Here is the last step inside the route "direct:myRoute":
public void doSomething(Exchange exchange)
{
System.out.println("exchange within = " + System.identityHashCode(exchange)); //1649140180
exchange.getOut().setBody(1);
}
My routes are defined by RouteBuilder classes and I load them by
<camelContext id="mainContext" xmlns="http://camel.apache.org/schema/spring">
<packageScan>
<package>com.my.route</package>
</packageScan>
</camelContext>
and also dynamically by code:
((CamelContext) applicationContext.getBean("mainContext")).addRoutes(routeBuilder);
As you can see, I cannot retrieve the output of the route when I call the route from the test, because within the test for some reason it is a different exchange object (however, the original exchange headers and properties are somehow copied to the exchange within the route).
This problem does not happen when I define the routes by XML.
Why is that and how could I retrieve the result of the route when I call it from a test?
I am not 100% sure but I don't you think you can write like this.
template.send("direct:myRoute", exchange);
Object object = exchange.getOut().getBody();
It should ideally be:
Exchange out=null;
out=template.requestBody("direct:myRoute", exchange);
String body=out.getOut().getBody();
For this to work you need to set the exchange pattern to InOut.
More info on the templates:
https://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/ProducerTemplate.html
Does anyone know how to test for different types of Collection in a route?
// This processor returns a Collection of 2 Sets
// 1. Set<GoodMessage>
// 2. Set<BadMessage>
.process(new MyGoodBadMessageProcessor())
// Split the result List
.split(body()).choice()
// How do you test for a Set<GoodMessage>??
.when(body().isInstanceOf(Set<GoodMessage>)
.to("direct:good")
.otherwise()
.to("direct:bad")
.endChoice()
Background: (In case someone can see a better way of doing this) I have a Processor that currently works as follows:
#Override
public void process(Exchange exchange) throws Exception {
Message message = exchange.getIn();
Set<UnknownMessage> unknownMessages = message.getBody(Set.class);
Set<GoodMessage> goodMessages = new HashSet<GoodMessage>();
for(UnknownMessage msg: unknownMessages) {
// Simplified logic here
if (msg.isGood()) {
goodMessages.add(msg.getGoodMessage());
}
}
message.setBody(goodMessages);
}
I'd like to update this as to now include the BadMessage(s) for reporting:
#Override
public void process(Exchange exchange) throws Exception {
Message message = exchange.getIn();
Set<UnknownMessage> unknownMessages = message.getBody(Set.class);
Set<GoodMessage> goodMessages = new HashSet<GoodMessage>();
Set<BadMessage> badMessages = new HashSet<BadMessage>();
List result = new ArrayList();
for(UnknownMessage msg: unknownMessages) {
// Simplified logic here
if (msg.isGood()) {
goodMessages.add(msg.getGoodMessage());
} else {
badMessages.add(msg.getBadMessage());
}
}
result.add(goodMessages)
result.add(badMessages)
message.setBody(result);
}
You cannot get the type of collection in this way (nothing to do with camel).
The way you've updated your process method does not need creating a different end point for bad messages.
One possible way to send this to a different end point based on message type is add a processor before the choice which inspects the type of the message and adds a header. Your choice statement can then work based on this header.
The following Predicate would work, although might give incorrect results when the Set is empty :/
Public class IsGoodMessage implements Predicate {
#Override
public boolean matches(Exchange exchange) {
Message message = exchange.getIn();
Set unknownSet = message.getBody(Set.class);
for (Object o : unknownSet) {
if (o instanceof GoodMessage) {
return true;
} else {
return false;
}
}
return false;
}
}
This helped:
How do I find out what type each object is in a ArrayList<Object>?
UPDATE: After some further reading, a better way to do this is to use a Header/Property to help distinguish the message type.
STEP 1: Update Processor to produce a Map that identifies different message types.
"GOOD_MSGS" -> List<GoodMessage>
"BAD_MSGS" -> List<BadMessage>
STEP 2: Create a splitter bean that splits this Map and then creates a header using the key of Map from the previous step.
(see "splitMessage" here http://camel.apache.org/splitter.html)
STEP 3: In the route use these headers to route the messages accordingly