I am new to the Spring Integration project, now I need to create a flow with Java DSL and test it.
I came up with these flows. First one should run by cron and invoke second one, which invokes HTTP endpoint and translates XML response to POJO:
#Bean
IntegrationFlow pollerFlow() {
return IntegrationFlows
.from(() -> new GenericMessage<>(""),
e -> e.poller(p -> p.cron(this.cron)))
.channel("pollingChannel")
.get();
}
#Bean
IntegrationFlow flow(HttpMessageHandlerSpec bulkEndpoint) {
return IntegrationFlows
.from("pollingChannel")
.enrichHeaders(authorizationHeaderEnricher(user, password))
.handle(bulkEndpoint)
.transform(xmlTransformer())
.channel("httpResponseChannel")
.get();
}
#Bean
HttpMessageHandlerSpec bulkEndpoint() {
return Http
.outboundGateway(uri)
.httpMethod(HttpMethod.POST)
.expectedResponseType(String.class)
.errorHandler(new DefaultResponseErrorHandler());
}
Now I want to test flow and mock HTTP call, but struggling to mock HTTP handler, I tried to do it like that:
#ExtendWith(SpringExtension.class)
#SpringIntegrationTest(noAutoStartup = {"pollerFlow"})
#ContextConfiguration(classes = FlowConfiguration.class)
public class FlowTests {
#Autowired
private MockIntegrationContext mockIntegrationContext;
#Autowired
public DirectChannel httpResponseChannel;
#Autowired
public DirectChannel pollingChannel;
#Test
void test() {
final MockMessageHandler mockHandler = MockIntegration.mockMessageHandler()
.handleNextAndReply(message -> new GenericMessage<>(xml, message.getHeaders()));
mockIntegrationContext.substituteMessageHandlerFor("bulkEndpoint", mockHandler);
httpResponseChannel.subscribe(message -> {
assertThat(message.getPayload(), is(notNullValue()));
assertThat(message.getPayload(), instanceOf(PartsSalesOpenRootElement.class));
});
pollingChannel.send(new GenericMessage<>(""));
}
}
But I am always getting an error, that on line:
mockIntegrationContext.substituteMessageHandlerFor("bulkEndpoint", mockHandler);
org.springframework.beans.factory.BeanNotOfRequiredTypeException: Bean named 'bulkEndpoint' is expected to be of type 'org.springframework.integration.endpoint.IntegrationConsumer' but was actually of type 'org.springframework.integration.http.outbound.HttpRequestExecutingMessageHandler'
Am I doing something wrong here? I am assuming I have a problem with IntegrationFlow itself, or maybe my testing approach is a problem.
The error is correct. The bulkEndpoint is not an endpoint by itself. It is really a MessageHandler. The endpoint is created from the .handle(bulkEndpoint).
See docs: https://docs.spring.io/spring-integration/docs/current/reference/html/overview.html#finding-class-names-for-java-and-dsl-configuration and https://docs.spring.io/spring-integration/docs/current/reference/html/testing.html#testing-mocks.
So, to make it working you need to do something like this:
.handle(bulkEndpoint, e -> e.id("actualEndpoint"))
And then in the test:
mockIntegrationContext.substituteMessageHandlerFor("actualEndpoint", mockHandler);
You also probably need to think to not have that pollerFlow to be started when you test it sine you send the message into pollingChannel manually. So, there is no conflicts with what you'd like to test. For this reason you also add a id() into your e.poller(p -> p.cron(this.cron)) and use #SpringIntegrationTest(noAutoStartup) to have it stopped before your test. I see you try noAutoStartup = {"pollerFlow"}, but this is not going to help for static flows. You indeed need to have stopped an actual endpoint in this case.
Related
Here I'm using scatter gather pattern. If I want to call another IntegrationFlow after aggregate() and before to(), how do I do that? can I use recipientFlow here so that I can make that flow conditional as well?
#Bean
public IntegrationFlow flow() {
return flow ->
flow.handle(validatorService, "validateRequest")
.split()
.channel(c -> c.executor(Executors.newCachedThreadPool()))
.scatterGather(
scatterer ->
scatterer
.applySequence(true)
.recipientFlow(flow1())
.recipientFlow(flow2())
.recipientFlow(flow3()),
gatherer ->
gatherer
.releaseLockBeforeSend(true)
.releaseStrategy(group -> group.size() == 2))
.aggregate(lionService.someMethod())
// here I want to call other Integration flows
.gateway(someFlow())
.to(someFlow2());
}
#Bean
public IntegrationFlow flow1() {
return flow ->
flow.channel(c -> c.executor(Executors.newCachedThreadPool()))
.enrichHeaders(h -> h.errorChannel("flow1ErrorChannel", true))
.handle(cdRequestService, "prepareCDRequestFromLoanRequest");
}
//same way I have flow2 and flow3, and I have set an custom error channel header for all the flows
#Bean
public IntegrationFlow someFlow() {
return flow ->
flow.filter("headers.sourceSystemCode.equals("001")").channel(c -> c.executor(Executors.newCachedThreadPool()))
.enrichHeaders(h -> h.errorChannel("someFlow1ErrorChannel", true))
.handle( Http.outboundGateway("http://localhost:4444/test2")
.httpMethod(HttpMethod.POST)
.expectedResponseType(String.class)).bridge();
}
Till now whenever any error occurred in any of the flow it goes through the custom error channels that have been assigned to them then I process the error but when I have used someFlow1() in .gateway(someFlow()) then the error occurring in that flow is not going to the assigned error channel. How to resolve that?
Inside errorhandler class I'm doing something like below --
//errorhandlerclass
#ServiceActivator(inputChannel = "flow1ErrorChannel")
public Message<?> processDBError(MessagingException payload) {
logger.atSevere().withStackTrace(StackSize.FULL).withCause(payload).log(
Objects.requireNonNull(payload.getFailedMessage()).toString());
MessageHeaders messageHeaders = Objects.requireNonNull(payload.getFailedMessage()).getHeaders();
return MessageBuilder.withPayload(
new LionException(ErrorCode.DATABASE_ERROR.getErrorData()))
.setHeader(MessageHeaders.REPLY_CHANNEL, messageHeaders.get("originalErrorChannel"))
.build();
}
#ServiceActivator(inputChannel = "someFlow1ErrorChannel")
public Message<?> processDBError(MessagingException payload) {
logger.atSevere().withStackTrace(StackSize.FULL).withCause(payload).log(
Objects.requireNonNull(payload.getFailedMessage()).toString());
MessageHeaders messageHeaders = Objects.requireNonNull(payload.getFailedMessage()).getHeaders();
return MessageBuilder.withPayload(
new LionException(ErrorCode.CUSTOM_ERROR.getErrorData()))
.setHeader(MessageHeaders.REPLY_CHANNEL, messageHeaders.get("originalErrorChannel"))
.build();
}
Again, if there's any error in someFlow() then error is shown but I want it to go to that method where I'm processing the error as per my requirement.
Also, you can see I've used filter in someFlow() so when the filter expression evaluates true then no problem but when it become false then it's throwing error but I want it to escape and go to next i.e.,.to(someFlow2()). I've used .bridge() by thinking that it'll return to previous context but that's not happening. I know there's some gap in my understanding. Kindly help with the above two problems.
To call another flow and come back to the main one you can use a gateway(). But that flow has to return in the end. There is no something like conditional flow: you may send to the channel (next endpoint in the flow) or not via filter() endpoint (or operator if you wish). The to() operator is terminal in the current flow, but you continue your logic in that destination flow whatever you want. Looks like you need to dedicate some of your time to understand what is a message channel and how it connects endpoints in Spring Integration. The IntegrationFlow is just logical container to express a business task - at runtime it is all endpoints and channels between them.
I have a REST controller which calls a gateway annotated with #MessagingGateway(errorChannel = ERROR_CHANNEL)
This way, whatever error occurs downstream the integration flow initiated by the gateway will flow into an error channel which will be handled by another integration flow, this is working as expected.
Now, there is another scenario where an integration flow reads messages from Kafka, routes those messages to another channel, one more integration flow processes those messages and another flow sends an HTTP request to a remote service.
public IntegrationFlowBuilder attachmentEventTenantRouter(String tenantId) {
return attachmentEventBaseFlow(".*")
.filter(Message.class, m -> m.getHeaders().get(KafkaConstants.HEADER_PREFIX + MessageHeader.TENANT_ID_KEY) != null && m.getHeaders().get(KafkaConstants.HEADER_PREFIX + MessageHeader.TENANT_ID_KEY, String.class).equalsIgnoreCase(tenantId));
}
private IntegrationFlowBuilder attachmentEventBaseFlow(String eventRegex) {
return IntegrationFlows
.from(Kafka.messageDrivenChannelAdapter(kafkaListenerContainerFactory.createContainer(topic)).errorChannel(IntegrationContextUtils.ERROR_CHANNEL_BEAN_NAME))
.log(LoggingHandler.Level.DEBUG, "Inside Kafka Consumer")
.filter(Message.class, m -> filter(m, eventRegex))
.transform(messageToEventTransformer);
}
#Bean
public IntegrationFlow kafkaConsumerFlow() {
return fromKafkaFlowHelper.attachmentEventTenantRouter(TENANT_ID)
.route(Message.class, m -> m.getHeaders().get(KafkaConstants.HEADER_PREFIX + MessageHeader.EVENT_TYPE_KEY, String.class), p -> p
.resolutionRequired(false)
.channelMapping("eventType", "transformMessagesFromKafkaAndPublishAnotherEvent")
.defaultOutputChannel("nullChannel"))
.get();
}
#Bean
public IntegrationFlow transformMessagesFromKafkaAndPublishAnotherEvent() {
return flow -> flow
.transform(transformer)
.handle( getKafkaHandler() );
}
#Bean
public IntegrationFlow sendHttpRequestToRemoteServiceFromKafkaEvent() {
return flow -> flow
.transform(transformer)
.handle(gatewayCall, e -> e.advice(expressionAdvice()));
}
How can I do to handle the exceptions that might occur in the flows above?
As you can see, I am using a ExpressionEvaluatingRequestHandlerAdvice which does the work for the handle method, but not sure how to handle exceptions that might occur in the transformers?
The massage gateway with an error channel configured does the trick when the gateway is called by a rest controller, but when the flow is initiated by the Kafka consumer, I'm lost how to achieve this.
Thanks.
EDITED AFTER Artem's RESPONSE TO ADD CLARIFICATION:
This is the configuration of the integration flow that posts a request to a remote service and whose exceptions does not seem to be caught and routed to the errorChannel without a ExpressionEvaluatingRequestHandlerAdvice:
#Bean
public IntegrationFlow sendHttpRequestToRemoteServiceFromKafkaEvent() {
return flow -> flow
.transform(transformer)
.handle(getOAuth2Handler(HttpMethod.PUT, "remote url"), e -> e.advice(expressionEvaluatingRequestHandlerAdvice));
}
private OAuth2RequestHandler getOAuth2Handler(HttpMethod httpMethod, String url) {
return new OAuth2RequestHandler(oAuth2RestTemplate, httpMethod, url);
}
And class OAuth2RequestHandler which implements a MessageHandler
#Override
public void handleMessage(org.springframework.messaging.Message<?> message) throws MessagingException {
String requestBody = (String) message.getPayload();
ResponseEntity<String> response = oAuth2RestTemplate.exchange(url, httpMethod, new HttpEntity<>(requestBody), String.class);
}
I see you use already an errorChannel() on the Kafka message-driven channel adapter. So, what is the question?
This part of your flow is exact equivalent to mentioned #MesaagingGateway with its errorChannel configuration.
I am using Spring Integration, Java DSL (release 1.1.3)
I have my org.springframework.integration.dsl.IntegrationFlow defined as follows
return IntegrationFlows.from(messageProducerSpec)
.handle(handler)
.handle(aggregator)
.handle(endpoint)
.get();
}
messageProducerSpec is instance of org.springframework.integration.dsl.amqp.AmqpBaseInboundChannelAdapterSpec
I would like my integration flow to consume messages from TWO separate messageProducerSpecs (two separate SimpleMessageListenerContainers, each using diffrent ConnectionFactory). How is it possible to construct integrationFlow from more than one messageProducerSpec? I see no integration component that is able to consume messages from multiple sources.
There is no reason to do that in Spring Integration.
You always can output different endpoints to the same MessageChannel.
Therefore you should have several simple IntegrationFlows for all those messageProducerSpec and finish them with the same channel, where also should be the main flow which will listen from that channel:
#Bean
public IntegrationFlow producer1() {
return IntegrationFlows.from(messageProducerSpec1)
.channel("input")
.get();
}
#Bean
public IntegrationFlow producer2() {
return IntegrationFlows.from(messageProducerSpec2)
.channel("input")
.get();
}
...
#Bean
public IntegrationFlow mainFlow() {
return IntegrationFlows.from("input")
.handle(handler)
.handle(aggregator)
.handle(endpoint)
.get();
}
I have two separate applications running either side of an ActiveMQ broker; application 1 sends synchronous requests to application 2 which returns the response back to application 1. At present the replies are via temporary queues and I am now trying to create a named reply destination to avoid the overhead of creating multiple temporary queues.
Application 1
#MessagingGateway
public interface OrderGateway {
#Gateway(requestChannel = "requestChannel", replyChannel = "responseChannel")
public OrderDto fetchOrder(OrderRequest orderRequest);
}
#Bean
public IntegrationFlow outgoingRequestFlow(ConnectionFactory connectionFactory) {
return IntegrationFlows.from("requestChannel")
.handle(Jms.outboundGateway(connectionFactory)
.requestDestination("request.queue")
.replyDestination("response.topic")
.correlationKey("JMSCorrelationID"))
.channel("responseChannel")
.get();
}
Application 2
#Bean
public IntegrationFlow incomingRequestFlow(ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Jms.inboundGateway(connectionFactory)
.destination("request.queue")
.correlationKey("JMSCorrelationID"))
.channel("requestChannel")
.handle("requestServiceActivator", "handleRequest")
.channel("responseChannel")
.get();
}
#Component
public class OrderServiceActivator {
#Autowired
OrderService orderService;
#ServiceActivator
public OrderDto fetchOrder(OrderRequest orderRequest) {
return orderService.getById(orderRequest.getId());
}
}
When I start both applications request.queue gets created and has one consumer (application 2). response.topic gets created but for some reason it has no consumers. Consequently when I send a request in to application 1 it reaches application 2, but after 5 seconds application 1 does not receive a reply and times out and the following errors are logged:
Application 2
org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'org.springframework.web.context.WebApplicationContext:/application-2.responseChannel'
Application 1
org.springframework.integration.MessageTimeoutException: failed to receive JMS response within timeout of: 5000ms
I presume I've made some simple configuration error, any help would be appreciated.
With your configuration, there is no long-lived consumer for the reply queue - a consumer is created for each request (with a message selector for the specific correlation id).
If you add .replyContainer() there will be a permanent consumer.
However, it should make no difference functionally.
I just ran tests similar to yours with and without replyContainer() and it all worked fine for me...
#Bean
public IntegrationFlow jmsOutboundGatewayFlow() {
return f -> f.handleWithAdapter(a ->
a.jmsGateway(this.jmsConnectionFactory)
// .replyContainer()
.replyDestination("pipereplies")
.correlationKey("JmsCorrelationID")
.requestDestination("jmsPipelineTest"));
}
I suggest you turn on debug logging to see if that sheds some light.
#Bean
public IntegrationFlow jmsInboundGatewayFlow() {
return IntegrationFlows.from((MessagingGateways g) ->
g.jms(this.jmsConnectionFactory)
.correlationKey("JmsCorrelationID")
.destination("jmsPipelineTest"))
.<String, String>transform(String::toUpperCase)
.get();
}
I'm failing to find how the #ServiceActivator is getting wired...
Usually it's something like:
#ServiceActivator(inputChannel = "requestChannel", outputChannel = "responseChannel")
public .....
Perhaps that is what you are missing.
I have a method which needs to execute multiple tasks async'ly.. I've managed to achieve that using the following IntegrationFlow:
#Bean
public IntegrationFlow startJobTask() {
return IntegrationFlows.from("TaskRoutingChannel")
.handle("jobService", "executeTasks")
.routeToRecipients(r -> r
.recipient("testTaskChannel")
.recipient("test2TaskChannel"))
.get();
}
#Bean ExecutorChannel testTaskChannel(){
return new ExecutorChannel(this.getAsyncExecutor());
}
#Bean ExecutorChannel test2TaskChannel(){
return new ExecutorChannel(this.getAsyncExecutor());
}
#Bean
public Executor getAsyncExecutor() {
SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor();
return executor;
}
#Bean
public IntegrationFlow testTaskFlow() {
return IntegrationFlows.from("testTaskChannel")
.handle("testTaskService", "executeAsync")
.get();
}
#Bean
public IntegrationFlow test2TaskFlow() {
return IntegrationFlows.from("test2TaskChannel")
.handle("test2TaskService", "executeAsync")
.get();
}
The above flow is basically as follows:
TaskRoutingChannel calls a serviceActivator method executeTasks
executeTasks returns something like Message<List<myTask>>
this Message is routed to channels testTaskChannel and test2TaskChannel
which calls their own async serviceActivator methods.
Now, the issue is that I don't want to hardcode the recipient channels.
I could avoid hardcoding with normal router by setting the destination channel as a header. However, recepientListRouters don't seem to have the capability to get recipient channel name using expressions.
Is there any way to set the recipient channels dynamically?
Sorry for delay first of all.
Actually the regular .route() can do that for you, because it has exactly has this option:
protected abstract Collection<MessageChannel> determineTargetChannels(Message<?> message);
So, if your header extraction returns the list of channels, the HeaderValueRouter will be able to the message to all of them.