I'm using multicast to send the incoming message to two different endpoints and aggregating the response.I would like to get the response,even if one of the endpoint response timed out.
which method i can use "timeout" or completionTimeout.?
I'm using parallel processing for processing message.
.multicast()
.to("direct:A","direct:B")
.parallelProcessing()
//.timeout(1000L)
.aggregationStrategy(new MyAggregationStrategy())
//.completionTimeout(2000L)
.end()
How can i use TimeoutAwareAggregationStrategy in this case.
public class MyAggregationStrategy implements TimeoutAwareAggregationStrategy {
#Override
public Exchange aggregate(Exchange newExchange, Exchange originalExchange) {
if(newExchange==null){
return originalExchange;
}
else {
ExchangeHelper.copyResults(originalExchange, newExchange);
return originalExchange;
}
}}
Use timeout, and see this unit test for an example: https://github.com/apache/camel/blob/master/camel-core/src/test/java/org/apache/camel/processor/MulticastParallelTimeoutAwareTest.java
Related
I'm trying to use request/response pattern for Spring Boot using AMQP and Spring-web. I have client service that has #RestController and AsyncRabbit configuration with Direct Exchange, Routing key etc. and server that has simple listener for request queue.
Client (something like rest gateway controller):
#RestController
public class ClientController {
#GetMapping("/test1")
public String getRequest() {
ListenableFuture<String> listenableFuture = asyncRabbitTemplate.convertAndReceiveAsType(
directExchange.getName(),
routingKey,
testDto,
new ParameterizedTypeReference<>() {}
);
return listenableFuture.get(); // Here I receive response from server services
}
#GetMapping("/test2")
public String getRequest2() {
ListenableFuture<String> listenableFuture = asyncRabbitTemplate.convertAndReceiveAsType(
/* Same properties but I use another DTO object for request */
);
return listenableFuture.get()
}
Server:
#RabbitListener(queues = "#{queue.name}", concurrency = "10")
#Component
public class Consumer {
#RabbitHandler
public String receive(TestDto testDto) {
...
}
#RabbitHandler
public String receive2(AnotherTestDto anotherTestDto) {
...
}
}
How should I implement Rabbit listener to process each REST request?
I found only two ways to do that:
Using #RabbitHandler as in the example above. But for each request method (GET, POST, etc.) I need unique DTO class to send message and process it in correct handler even if "request body" is almost same (number of request methods = number of DTO class to send). I'm not sure that is right way.
Leave one Consumer and call desired method to process that depends on message body (trivial if-else):
...
#RabbitListener(queues = "#{queue.name}")
public String receive(MessageDto messageDto) {
if (messageDto.requestType == "get.method1") {
return serverService.processThat(messageDto);
} else if (messageDto.requestType == "post.method2") {
return serverService.processAnother(messageDto);
} else if ...
...
}
...
But add new if-else branch every time is not very convenient so I really out of ideas.
You may consider to use different queues for different request types. All of them are going to be bound to the same direct exchange, but with their respective routing key.
What you would need on the consumer side is just to add a new #RabbitListener for respective queue. And bind that queue to the exchange with its routing key.
That's actually a beauty of the AMQP protol by itself: the producer always publish to the same exchange with respective routing key. The consumer registers its interest for routing keys and binds a queue. The rest of routing logic is done on the AMQP broker.
See more info in docs: https://www.rabbitmq.com/tutorials/tutorial-four-spring-amqp.html
I have some Spring applications that communicate between them using RabbitMQ as broker. I can send and receive messages asynchronously between them. But now, I need one application to send a message to another one and wait for the response. So, for this I am trying to implement the RPC pattern. It is working, but the problem is that I could only do it using temporary queues generated by Spring.
https://www.rabbitmq.com/tutorials/tutorial-six-spring-amqp.html
This is the code that sends the message and wait for the response.
public void send() {
....
Integer response = (Integer) template.convertSendAndReceive(exchange.getName(), "rpc", "message");
...
}
When I send the message, the execution is blocked until the response is received and a temporary queue is created by Spring for the response, as expected.
But what I need is to use a specific and fixed queue, defined by me, to receive the responses. I need responses to be sent to an exchange with a routing key pointing to the fixed response queue (doing this I'll be able to send the responses to another queue too, that will be logging all responses).
I tried setting the "setReplyTo" property to the message, but is not working.
What version are you using? With modern versions, direct reply_to is used by default, but you can revert to using a temporary queue by setting a property on the template.
https://docs.spring.io/spring-amqp/docs/current/reference/html/#direct-reply-to
To use a named reply queue, see the documentation about how to set up a reply container, with the template as the message listener:
https://docs.spring.io/spring-amqp/docs/current/reference/html/#message-correlation-with-a-reply-queue
and
https://docs.spring.io/spring-amqp/docs/current/reference/html/#reply-listener
EDIT
The template will block until the corresponding reply is passed into it by the reply container (or it times out).
#SpringBootApplication
public class So68986604Application {
public static void main(String[] args) {
SpringApplication.run(So68986604Application.class, args);
}
#RabbitListener(queues = "foo")
public String listen(String in) {
System.out.println(in);
return in.toUpperCase();
}
#Bean
Queue foo() {
return new Queue("foo");
}
#Bean
Queue replies() {
return new Queue("foo.replies");
}
#Bean
SimpleMessageListenerContainer replyContainer(ConnectionFactory cf, RabbitTemplate template) {
SimpleMessageListenerContainer replyer = new SimpleMessageListenerContainer(cf);
replyer.setQueueNames("foo.replies");
replyer.setMessageListener(template);
template.setReplyAddress("foo.replies");
return replyer;
}
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
return args -> {
System.out.println(template.convertSendAndReceive("", "foo", "test"));
};
}
}
test
TEST
I have an aggregation Strategy in my camel split() route.
from("direct:split")
.split()
.method(new SplitBean(), "splitMessage")
.aggregationStrategy(AggregationStrategies.groupedExchange())
.stopOnException()
.to("direct:destination")
.end();
The splitMessage method has split the data into 3 request data. So I am hitting the http destination endpoint 3 times.
Using the aggregation Strategy my http response got aggregated for the first 2 times.
Third time when the http call failed with an exception. The exchange returned to the caller does not contain the first two grouped exchanges.
How can I get the grouped exchanges with (success, exception) this case.
Please tell me if the question is not clear.
Change from .stopOnException() to .stopOnAggregateException()
create an AggregationStrategy strategy class and handle the exception from there
public void configure() throws Exception {
from("direct:split")
.split()
.method(new SplitBean(), "splitMessage")
.aggregationStrategy(new ErrorStrategy())
.stopOnAggregateException()
.to("direct:destination")
.end();
}
public class ErrorStrategy implements CompletionAwareAggregationStrategy {
#Override
public void onCompletion(Exchange exchange) {
}
#Override
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
if (newExchange.getException() != null) {
return oldExchange;
}
if (oldExchange == null) {
....
return newExchange;
}
.....
return oldExchange;
}
}
I have a Camel application and I'm trying to do some aggregation based on some outputs from different responses (REST Web services).
This is what I have so far (the Camel routes):
#Component
public final class AggregationRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
rest("/aggregation")
.get()
.to("direct:retrieve");
from("direct:retrieve")
.multicast(/*new BodyAggregationStrategy(), true*/)
.to("direct:foo")
.to("direct:foo1");
from("direct:foo")
.to("seda:http://graphical.weather.gov/xml/sample_products/browser_interface/ndfdXMLclient.php?sector=conus")
.to("direct:aggregate");
from("direct:foo1")
.to("seda:http://graphical.weather.gov/xml/sample_products/browser_interface/ndfdXMLclient.php?sector=conus")
.to("direct:aggregate");
from("direct:aggregate")
.aggregate(header("id"), new BodyAggregationStrategy())
.log(LoggingLevel.WARN, simple("${body}").getText());
}
}
...the "aggregation strategy"
public final class BodyAggregationStrategy implements AggregationStrategy {
#Override
public Exchange aggregate(final Exchange oldExchange, final Exchange newExchange) {
if (null == oldExchange) {
return newExchange;
}
String oldBody = oldExchange.getIn().getBody(String.class);
String newBody = newExchange.getIn().getBody(String.class);
oldExchange.getIn().setBody(oldBody + "+" + newBody);
return oldExchange;
}
}
...eventually, the Web services are going to be different, but I'm just trying now to see if I can solve this basic trouble first.
I defined a REST endpoint, when GET /aggregation is hit (on my side), I want to consult two or more REST Web services and aggregate the response from those; then "answer" back.
Any clues?
The multicast EIP has built-in aggregator, so configure the aggregation strategy on this pattern, instead of using a separate aggregator. This ensures the messages are mutlicasted and aggregate as part of the same unit of work and the result can be visible and send back to the calling REST client.
I have a dynamic route creator web application. According to flow design, I produce a camel route. Route may contain multicast, filter, aggregate, processor etc. After designing flow through the UI, my route has been created like this:
from("seda:start").routeId("idx")
.multicast()
.to("direct:a", "direct:b", "direct:c")
.parallelProcessing()
.end();
from("direct:a").transform(constant("A")).delay(1000).to("direct:merge");
from("direct:b").transform(constant("B")).delay(2000).to("direct:merge");
from("direct:c").transform(constant("C")).delay(3000).to("direct:merge");
from("direct:merge")
.aggregate(new MyAggregationStrategy()).constant(true).completionSize(3)
.to("mock:end");
I have an API to give result of this route to the users. When I execute this route with InOut MEP, response is 'C' but mock:end is satisfied with 'ABC':
MockEndpoint mock = getMockEndpoint("mock:end");
mock.expectedBodiesReceived("ABC"); //works as expected
String reply = template.requestBody("seda:start", "", String.class);
assertEquals("ABC", reply); //it returns 'C', but I expect 'ABC'
assertMockEndpointsSatisfied();
How can I change the code to get aggregated result with a synchronous call? Here is the code:
public class ResponseTest extends CamelTestSupport {
#Test
public void testAsyncInOut() throws Exception {
MockEndpoint mock = getMockEndpoint("mock:end");
mock.expectedBodiesReceived("ABC"); //works as expected
String reply = template.requestBody("seda:start", "", String.class);
assertEquals("ABC", reply); //it returns 'C', but I expect 'ABC'
assertMockEndpointsSatisfied();
}
#Override
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from("seda:start").routeId("idx")
.multicast()
.to("direct:a", "direct:b", "direct:c")
.parallelProcessing()
.end();
from("direct:a").transform(constant("A")).delay(1000).to("direct:merge");
from("direct:b").transform(constant("B")).delay(2000).to("direct:merge");
from("direct:c").transform(constant("C")).delay(3000).to("direct:merge");
from("direct:merge")
.aggregate(new MyAggregationStrategy()).constant(true).completionSize(3)
.to("mock:end");
}
};
}
class MyAggregationStrategy implements AggregationStrategy {
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
if (oldExchange == null) {
// this is the first time so no existing aggregated exchange
return newExchange;
}
// append the new word to the existing
String body = newExchange.getIn().getBody(String.class);
String existing = oldExchange.getIn().getBody(String.class);
oldExchange.getIn().setBody(existing + body);
return oldExchange;
}
}
}
EDIT
Multicasting message to 5 different endpoints doesn't mean that all the messages will be aggregated later. So I cannot use aggregate strategy in multicast definition. Some of them might be used for another kind of job. Flow definition might be like this:
After multicast message to 'a','b','c','d','e' endpoints, 'a' and 'b' can be aggregated together('direct:merge1'), 'c' and 'd' aggregated together('direct:merge2') and 'e' can be used for another thing.
And final aggregator will be aggregate 'direct:merge1' and 'direct:merge2'
to 'direct:merge3'. All these endpoints are created dynamically('direct:a','direct:b','direct:c','direct:d','direct:e','direct:merge1','direct:merge2','direct:merge3'). This scenario will be created like this:
from("seda:start").routeId("idx")
.multicast()
.to("direct:a", "direct:b", "direct:c", "direct:d", "direct:e")
.parallelProcessing()
.end();
from("direct:a").transform(constant("A")).delay(1000).to("direct:merge1");
from("direct:b").transform(constant("B")).delay(2000).to("direct:merge1");
from("direct:c").transform(constant("C")).delay(3000).to("direct:merge2");
from("direct:d").transform(constant("D")).delay(1000).to("direct:merge2");
from("direct:e").transform(constant("E")).delay(1000).to("mock:anywhere");
from("direct:merge1").aggregate(new MyAggregationStrategy()).constant(true).completionSize(2).to("direct:merge3");
from("direct:merge2").aggregate(new MyAggregationStrategy()).constant(true).completionSize(2).to("direct:merge3");
from("direct:merge3").aggregate(new MyAggregationStrategy()).constant(true).completionSize(2).to("mock:end");
When I send message to seda:start, I expect ABDC but I got 'E'. Is there a way to get final aggregated message('ABDC')? Here is the test method:
#Test
public void testAsyncInOut() throws Exception {
MockEndpoint mock = getMockEndpoint("mock:end");
mock.expectedBodiesReceived("ABDC"); //works as expected
String reply = template.requestBody("seda:start", "", String.class);
assertEquals("ABDC", reply); //it returns 'E' because of default multicast behavior, but I expect 'ABDC'
assertMockEndpointsSatisfied();
}
From the multicast documentation:
By default Camel will use the last reply as the outgoing message.
If you want to aggregate the results of the multicast into a single message, you state that in the multicast definition.
#Override
public void configure() throws Exception {
from("seda:start").routeId("idx")
.multicast(new MyAggregationStrategy()) //Put the Aggregation Strategy here!
.to("direct:a", "direct:b", "direct:c")
.parallelProcessing()
.end();
from("direct:a").transform(constant("A")).delay(1000).to("direct:merge");
from("direct:b").transform(constant("B")).delay(2000).to("direct:merge");
from("direct:c").transform(constant("C")).delay(3000).to("direct:merge");
from("direct:merge")
.to("mock:end");
}
Note that your mock endpoint will now be called 3 times, as the aggregation doesn't occur until later. You would need to modify your test accordingly.