I have a REST controller which calls a gateway annotated with #MessagingGateway(errorChannel = ERROR_CHANNEL)
This way, whatever error occurs downstream the integration flow initiated by the gateway will flow into an error channel which will be handled by another integration flow, this is working as expected.
Now, there is another scenario where an integration flow reads messages from Kafka, routes those messages to another channel, one more integration flow processes those messages and another flow sends an HTTP request to a remote service.
public IntegrationFlowBuilder attachmentEventTenantRouter(String tenantId) {
return attachmentEventBaseFlow(".*")
.filter(Message.class, m -> m.getHeaders().get(KafkaConstants.HEADER_PREFIX + MessageHeader.TENANT_ID_KEY) != null && m.getHeaders().get(KafkaConstants.HEADER_PREFIX + MessageHeader.TENANT_ID_KEY, String.class).equalsIgnoreCase(tenantId));
}
private IntegrationFlowBuilder attachmentEventBaseFlow(String eventRegex) {
return IntegrationFlows
.from(Kafka.messageDrivenChannelAdapter(kafkaListenerContainerFactory.createContainer(topic)).errorChannel(IntegrationContextUtils.ERROR_CHANNEL_BEAN_NAME))
.log(LoggingHandler.Level.DEBUG, "Inside Kafka Consumer")
.filter(Message.class, m -> filter(m, eventRegex))
.transform(messageToEventTransformer);
}
#Bean
public IntegrationFlow kafkaConsumerFlow() {
return fromKafkaFlowHelper.attachmentEventTenantRouter(TENANT_ID)
.route(Message.class, m -> m.getHeaders().get(KafkaConstants.HEADER_PREFIX + MessageHeader.EVENT_TYPE_KEY, String.class), p -> p
.resolutionRequired(false)
.channelMapping("eventType", "transformMessagesFromKafkaAndPublishAnotherEvent")
.defaultOutputChannel("nullChannel"))
.get();
}
#Bean
public IntegrationFlow transformMessagesFromKafkaAndPublishAnotherEvent() {
return flow -> flow
.transform(transformer)
.handle( getKafkaHandler() );
}
#Bean
public IntegrationFlow sendHttpRequestToRemoteServiceFromKafkaEvent() {
return flow -> flow
.transform(transformer)
.handle(gatewayCall, e -> e.advice(expressionAdvice()));
}
How can I do to handle the exceptions that might occur in the flows above?
As you can see, I am using a ExpressionEvaluatingRequestHandlerAdvice which does the work for the handle method, but not sure how to handle exceptions that might occur in the transformers?
The massage gateway with an error channel configured does the trick when the gateway is called by a rest controller, but when the flow is initiated by the Kafka consumer, I'm lost how to achieve this.
Thanks.
EDITED AFTER Artem's RESPONSE TO ADD CLARIFICATION:
This is the configuration of the integration flow that posts a request to a remote service and whose exceptions does not seem to be caught and routed to the errorChannel without a ExpressionEvaluatingRequestHandlerAdvice:
#Bean
public IntegrationFlow sendHttpRequestToRemoteServiceFromKafkaEvent() {
return flow -> flow
.transform(transformer)
.handle(getOAuth2Handler(HttpMethod.PUT, "remote url"), e -> e.advice(expressionEvaluatingRequestHandlerAdvice));
}
private OAuth2RequestHandler getOAuth2Handler(HttpMethod httpMethod, String url) {
return new OAuth2RequestHandler(oAuth2RestTemplate, httpMethod, url);
}
And class OAuth2RequestHandler which implements a MessageHandler
#Override
public void handleMessage(org.springframework.messaging.Message<?> message) throws MessagingException {
String requestBody = (String) message.getPayload();
ResponseEntity<String> response = oAuth2RestTemplate.exchange(url, httpMethod, new HttpEntity<>(requestBody), String.class);
}
I see you use already an errorChannel() on the Kafka message-driven channel adapter. So, what is the question?
This part of your flow is exact equivalent to mentioned #MesaagingGateway with its errorChannel configuration.
Related
Here I'm using scatter gather pattern. If I want to call another IntegrationFlow after aggregate() and before to(), how do I do that? can I use recipientFlow here so that I can make that flow conditional as well?
#Bean
public IntegrationFlow flow() {
return flow ->
flow.handle(validatorService, "validateRequest")
.split()
.channel(c -> c.executor(Executors.newCachedThreadPool()))
.scatterGather(
scatterer ->
scatterer
.applySequence(true)
.recipientFlow(flow1())
.recipientFlow(flow2())
.recipientFlow(flow3()),
gatherer ->
gatherer
.releaseLockBeforeSend(true)
.releaseStrategy(group -> group.size() == 2))
.aggregate(lionService.someMethod())
// here I want to call other Integration flows
.gateway(someFlow())
.to(someFlow2());
}
#Bean
public IntegrationFlow flow1() {
return flow ->
flow.channel(c -> c.executor(Executors.newCachedThreadPool()))
.enrichHeaders(h -> h.errorChannel("flow1ErrorChannel", true))
.handle(cdRequestService, "prepareCDRequestFromLoanRequest");
}
//same way I have flow2 and flow3, and I have set an custom error channel header for all the flows
#Bean
public IntegrationFlow someFlow() {
return flow ->
flow.filter("headers.sourceSystemCode.equals("001")").channel(c -> c.executor(Executors.newCachedThreadPool()))
.enrichHeaders(h -> h.errorChannel("someFlow1ErrorChannel", true))
.handle( Http.outboundGateway("http://localhost:4444/test2")
.httpMethod(HttpMethod.POST)
.expectedResponseType(String.class)).bridge();
}
Till now whenever any error occurred in any of the flow it goes through the custom error channels that have been assigned to them then I process the error but when I have used someFlow1() in .gateway(someFlow()) then the error occurring in that flow is not going to the assigned error channel. How to resolve that?
Inside errorhandler class I'm doing something like below --
//errorhandlerclass
#ServiceActivator(inputChannel = "flow1ErrorChannel")
public Message<?> processDBError(MessagingException payload) {
logger.atSevere().withStackTrace(StackSize.FULL).withCause(payload).log(
Objects.requireNonNull(payload.getFailedMessage()).toString());
MessageHeaders messageHeaders = Objects.requireNonNull(payload.getFailedMessage()).getHeaders();
return MessageBuilder.withPayload(
new LionException(ErrorCode.DATABASE_ERROR.getErrorData()))
.setHeader(MessageHeaders.REPLY_CHANNEL, messageHeaders.get("originalErrorChannel"))
.build();
}
#ServiceActivator(inputChannel = "someFlow1ErrorChannel")
public Message<?> processDBError(MessagingException payload) {
logger.atSevere().withStackTrace(StackSize.FULL).withCause(payload).log(
Objects.requireNonNull(payload.getFailedMessage()).toString());
MessageHeaders messageHeaders = Objects.requireNonNull(payload.getFailedMessage()).getHeaders();
return MessageBuilder.withPayload(
new LionException(ErrorCode.CUSTOM_ERROR.getErrorData()))
.setHeader(MessageHeaders.REPLY_CHANNEL, messageHeaders.get("originalErrorChannel"))
.build();
}
Again, if there's any error in someFlow() then error is shown but I want it to go to that method where I'm processing the error as per my requirement.
Also, you can see I've used filter in someFlow() so when the filter expression evaluates true then no problem but when it become false then it's throwing error but I want it to escape and go to next i.e.,.to(someFlow2()). I've used .bridge() by thinking that it'll return to previous context but that's not happening. I know there's some gap in my understanding. Kindly help with the above two problems.
To call another flow and come back to the main one you can use a gateway(). But that flow has to return in the end. There is no something like conditional flow: you may send to the channel (next endpoint in the flow) or not via filter() endpoint (or operator if you wish). The to() operator is terminal in the current flow, but you continue your logic in that destination flow whatever you want. Looks like you need to dedicate some of your time to understand what is a message channel and how it connects endpoints in Spring Integration. The IntegrationFlow is just logical container to express a business task - at runtime it is all endpoints and channels between them.
I have some Spring applications that communicate between them using RabbitMQ as broker. I can send and receive messages asynchronously between them. But now, I need one application to send a message to another one and wait for the response. So, for this I am trying to implement the RPC pattern. It is working, but the problem is that I could only do it using temporary queues generated by Spring.
https://www.rabbitmq.com/tutorials/tutorial-six-spring-amqp.html
This is the code that sends the message and wait for the response.
public void send() {
....
Integer response = (Integer) template.convertSendAndReceive(exchange.getName(), "rpc", "message");
...
}
When I send the message, the execution is blocked until the response is received and a temporary queue is created by Spring for the response, as expected.
But what I need is to use a specific and fixed queue, defined by me, to receive the responses. I need responses to be sent to an exchange with a routing key pointing to the fixed response queue (doing this I'll be able to send the responses to another queue too, that will be logging all responses).
I tried setting the "setReplyTo" property to the message, but is not working.
What version are you using? With modern versions, direct reply_to is used by default, but you can revert to using a temporary queue by setting a property on the template.
https://docs.spring.io/spring-amqp/docs/current/reference/html/#direct-reply-to
To use a named reply queue, see the documentation about how to set up a reply container, with the template as the message listener:
https://docs.spring.io/spring-amqp/docs/current/reference/html/#message-correlation-with-a-reply-queue
and
https://docs.spring.io/spring-amqp/docs/current/reference/html/#reply-listener
EDIT
The template will block until the corresponding reply is passed into it by the reply container (or it times out).
#SpringBootApplication
public class So68986604Application {
public static void main(String[] args) {
SpringApplication.run(So68986604Application.class, args);
}
#RabbitListener(queues = "foo")
public String listen(String in) {
System.out.println(in);
return in.toUpperCase();
}
#Bean
Queue foo() {
return new Queue("foo");
}
#Bean
Queue replies() {
return new Queue("foo.replies");
}
#Bean
SimpleMessageListenerContainer replyContainer(ConnectionFactory cf, RabbitTemplate template) {
SimpleMessageListenerContainer replyer = new SimpleMessageListenerContainer(cf);
replyer.setQueueNames("foo.replies");
replyer.setMessageListener(template);
template.setReplyAddress("foo.replies");
return replyer;
}
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
return args -> {
System.out.println(template.convertSendAndReceive("", "foo", "test"));
};
}
}
test
TEST
I am new to the Spring Integration project, now I need to create a flow with Java DSL and test it.
I came up with these flows. First one should run by cron and invoke second one, which invokes HTTP endpoint and translates XML response to POJO:
#Bean
IntegrationFlow pollerFlow() {
return IntegrationFlows
.from(() -> new GenericMessage<>(""),
e -> e.poller(p -> p.cron(this.cron)))
.channel("pollingChannel")
.get();
}
#Bean
IntegrationFlow flow(HttpMessageHandlerSpec bulkEndpoint) {
return IntegrationFlows
.from("pollingChannel")
.enrichHeaders(authorizationHeaderEnricher(user, password))
.handle(bulkEndpoint)
.transform(xmlTransformer())
.channel("httpResponseChannel")
.get();
}
#Bean
HttpMessageHandlerSpec bulkEndpoint() {
return Http
.outboundGateway(uri)
.httpMethod(HttpMethod.POST)
.expectedResponseType(String.class)
.errorHandler(new DefaultResponseErrorHandler());
}
Now I want to test flow and mock HTTP call, but struggling to mock HTTP handler, I tried to do it like that:
#ExtendWith(SpringExtension.class)
#SpringIntegrationTest(noAutoStartup = {"pollerFlow"})
#ContextConfiguration(classes = FlowConfiguration.class)
public class FlowTests {
#Autowired
private MockIntegrationContext mockIntegrationContext;
#Autowired
public DirectChannel httpResponseChannel;
#Autowired
public DirectChannel pollingChannel;
#Test
void test() {
final MockMessageHandler mockHandler = MockIntegration.mockMessageHandler()
.handleNextAndReply(message -> new GenericMessage<>(xml, message.getHeaders()));
mockIntegrationContext.substituteMessageHandlerFor("bulkEndpoint", mockHandler);
httpResponseChannel.subscribe(message -> {
assertThat(message.getPayload(), is(notNullValue()));
assertThat(message.getPayload(), instanceOf(PartsSalesOpenRootElement.class));
});
pollingChannel.send(new GenericMessage<>(""));
}
}
But I am always getting an error, that on line:
mockIntegrationContext.substituteMessageHandlerFor("bulkEndpoint", mockHandler);
org.springframework.beans.factory.BeanNotOfRequiredTypeException: Bean named 'bulkEndpoint' is expected to be of type 'org.springframework.integration.endpoint.IntegrationConsumer' but was actually of type 'org.springframework.integration.http.outbound.HttpRequestExecutingMessageHandler'
Am I doing something wrong here? I am assuming I have a problem with IntegrationFlow itself, or maybe my testing approach is a problem.
The error is correct. The bulkEndpoint is not an endpoint by itself. It is really a MessageHandler. The endpoint is created from the .handle(bulkEndpoint).
See docs: https://docs.spring.io/spring-integration/docs/current/reference/html/overview.html#finding-class-names-for-java-and-dsl-configuration and https://docs.spring.io/spring-integration/docs/current/reference/html/testing.html#testing-mocks.
So, to make it working you need to do something like this:
.handle(bulkEndpoint, e -> e.id("actualEndpoint"))
And then in the test:
mockIntegrationContext.substituteMessageHandlerFor("actualEndpoint", mockHandler);
You also probably need to think to not have that pollerFlow to be started when you test it sine you send the message into pollingChannel manually. So, there is no conflicts with what you'd like to test. For this reason you also add a id() into your e.poller(p -> p.cron(this.cron)) and use #SpringIntegrationTest(noAutoStartup) to have it stopped before your test. I see you try noAutoStartup = {"pollerFlow"}, but this is not going to help for static flows. You indeed need to have stopped an actual endpoint in this case.
I have two separate applications running either side of an ActiveMQ broker; application 1 sends synchronous requests to application 2 which returns the response back to application 1. At present the replies are via temporary queues and I am now trying to create a named reply destination to avoid the overhead of creating multiple temporary queues.
Application 1
#MessagingGateway
public interface OrderGateway {
#Gateway(requestChannel = "requestChannel", replyChannel = "responseChannel")
public OrderDto fetchOrder(OrderRequest orderRequest);
}
#Bean
public IntegrationFlow outgoingRequestFlow(ConnectionFactory connectionFactory) {
return IntegrationFlows.from("requestChannel")
.handle(Jms.outboundGateway(connectionFactory)
.requestDestination("request.queue")
.replyDestination("response.topic")
.correlationKey("JMSCorrelationID"))
.channel("responseChannel")
.get();
}
Application 2
#Bean
public IntegrationFlow incomingRequestFlow(ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Jms.inboundGateway(connectionFactory)
.destination("request.queue")
.correlationKey("JMSCorrelationID"))
.channel("requestChannel")
.handle("requestServiceActivator", "handleRequest")
.channel("responseChannel")
.get();
}
#Component
public class OrderServiceActivator {
#Autowired
OrderService orderService;
#ServiceActivator
public OrderDto fetchOrder(OrderRequest orderRequest) {
return orderService.getById(orderRequest.getId());
}
}
When I start both applications request.queue gets created and has one consumer (application 2). response.topic gets created but for some reason it has no consumers. Consequently when I send a request in to application 1 it reaches application 2, but after 5 seconds application 1 does not receive a reply and times out and the following errors are logged:
Application 2
org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'org.springframework.web.context.WebApplicationContext:/application-2.responseChannel'
Application 1
org.springframework.integration.MessageTimeoutException: failed to receive JMS response within timeout of: 5000ms
I presume I've made some simple configuration error, any help would be appreciated.
With your configuration, there is no long-lived consumer for the reply queue - a consumer is created for each request (with a message selector for the specific correlation id).
If you add .replyContainer() there will be a permanent consumer.
However, it should make no difference functionally.
I just ran tests similar to yours with and without replyContainer() and it all worked fine for me...
#Bean
public IntegrationFlow jmsOutboundGatewayFlow() {
return f -> f.handleWithAdapter(a ->
a.jmsGateway(this.jmsConnectionFactory)
// .replyContainer()
.replyDestination("pipereplies")
.correlationKey("JmsCorrelationID")
.requestDestination("jmsPipelineTest"));
}
I suggest you turn on debug logging to see if that sheds some light.
#Bean
public IntegrationFlow jmsInboundGatewayFlow() {
return IntegrationFlows.from((MessagingGateways g) ->
g.jms(this.jmsConnectionFactory)
.correlationKey("JmsCorrelationID")
.destination("jmsPipelineTest"))
.<String, String>transform(String::toUpperCase)
.get();
}
I'm failing to find how the #ServiceActivator is getting wired...
Usually it's something like:
#ServiceActivator(inputChannel = "requestChannel", outputChannel = "responseChannel")
public .....
Perhaps that is what you are missing.
What is the best way to do validation in Spring Integration.
For example if we have an inbound gateway, when a message is received we want to validate it. If it's not valid -> return the validation errors to the gateway, else -> proceed with the normal flow of the application(transform, handle ...).
I tried a filter:
#Bean
public IntegrationFlow flow() {
return IntegrationFlows.from(requestChannel())
.transform(new MapToObjectTransformer(Campaign.class))
.filter(Campaign.class,
c -> c.getId() > 10 ? true : false, //if id > 10 then it's valid
e -> e.discardChannel(validationError()))
.handle(new MyHandler())
.get();
}
#Bean
public IntegrationFlow validationErrorFlow() {
return IntegrationFlows.from(validationError())
.handle(new ValidationHandler())//construct a message with the validation errors
.get();
}
It works, but that way if I use a spring validator then i have to call it twice, in the filter and in the ValidationHandler (can be a transformer) to get the errors.
Any better way?
.handle(new ValidationHandler())
You don't really need to create a new handler for each error.
In your filter, if the validation fails, throw MyValidationException(errors).
In the error flow on the gateway's error channel, the ErrorMessage has a payload that is a MessagingException with the MyValidatationException as its cause, and the failedMessage.
Something like...
.handle(validationErrorHandler())
...
#Bean
public MessageHandler validationErrorHandler() {
return new AbstractReplyProducingMessageHandler() {
public Object handleRequestMessage(Message<?> error) {
MyValidationException myEx = (MyValidationException)
((ErrorMessage) error).getPayload.getCause();
Errors errors = myEx.getErrors();
...
}
}
}
Or you can use a POJO messageHandler
public Object handle(MessagingException e) {
...
}