What is the best way to do validation in Spring Integration.
For example if we have an inbound gateway, when a message is received we want to validate it. If it's not valid -> return the validation errors to the gateway, else -> proceed with the normal flow of the application(transform, handle ...).
I tried a filter:
#Bean
public IntegrationFlow flow() {
return IntegrationFlows.from(requestChannel())
.transform(new MapToObjectTransformer(Campaign.class))
.filter(Campaign.class,
c -> c.getId() > 10 ? true : false, //if id > 10 then it's valid
e -> e.discardChannel(validationError()))
.handle(new MyHandler())
.get();
}
#Bean
public IntegrationFlow validationErrorFlow() {
return IntegrationFlows.from(validationError())
.handle(new ValidationHandler())//construct a message with the validation errors
.get();
}
It works, but that way if I use a spring validator then i have to call it twice, in the filter and in the ValidationHandler (can be a transformer) to get the errors.
Any better way?
.handle(new ValidationHandler())
You don't really need to create a new handler for each error.
In your filter, if the validation fails, throw MyValidationException(errors).
In the error flow on the gateway's error channel, the ErrorMessage has a payload that is a MessagingException with the MyValidatationException as its cause, and the failedMessage.
Something like...
.handle(validationErrorHandler())
...
#Bean
public MessageHandler validationErrorHandler() {
return new AbstractReplyProducingMessageHandler() {
public Object handleRequestMessage(Message<?> error) {
MyValidationException myEx = (MyValidationException)
((ErrorMessage) error).getPayload.getCause();
Errors errors = myEx.getErrors();
...
}
}
}
Or you can use a POJO messageHandler
public Object handle(MessagingException e) {
...
}
Related
Here I'm using Spring integration's http outbound gateway to make a http call. I have also added timeout for the call. I have configured the timeout using restemplate. Here whenever it's taking more time then it's throwing an ResourceAccessException but I want to handle that exception and want to send proper msg to the user. Even though I'm using a custom error handler but the exception is not getting handled by that. Below the code where I'm using scatter gather pattern and flow2 is the http call where I want to handle the exception-
#Autowired private RestTemplateConfig resttemplateconfig.
#Bean
public IntegrationFlow mainFlow() {
return flow ->
flow.split()
.channel(c -> c.executor(Executors.newCachedThreadPool()))
.scatterGather(
scatterer ->
scatterer
.applySequence(true)
.recipientFlow(flow1())
.recipientFlow(flow2()),
gatherer ->
gatherer
.releaseLockBeforeSend(true)
.releaseStrategy(group -> group.size() == 1))
.aggregate()
.to(saveCDResponseToDB());
}
#Bean
public IntegrationFlow flow2() {
return flow ->
flow.channel(c -> c.executor(Executors.newCachedThreadPool()))
.handle(
Http.outboundGateway(
"http://localhost:4444/test", resttemplateconfig.restTemplate())
.extractPayload(true)
.httpMethod(HttpMethod.POST)
.expectedResponseType(String.class));
}
//RestTemplateConfig - The Resttemplate Config class where I'm setting the timeout and errorhandler.
#Configuration
public class RestTemplateConfig {
private final int TIMEOUT = (int) TimeUnit.SECONDS.toMillis(6);
#Autowired CustomErrorHandler errorHandler;
#Bean
public RestTemplate restTemplate() {
HttpComponentsClientHttpRequestFactory requestFactory =
new HttpComponentsClientHttpRequestFactory();
requestFactory.setConnectTimeout(TIMEOUT);
requestFactory.setReadTimeout(TIMEOUT);
RestTemplate restTemplate = new RestTemplate(requestFactory);
errorHandler.setMessageConverters(restTemplate.getMessageConverters());
restTemplate.setErrorHandler(errorHandler);
return restTemplate;
}
}
//custom error handler
#Component
public class CustomServiceErrorHandler implements ResponseErrorHandler {
private List<HttpMessageConverter<?>> messageConverters;
#Override
public boolean hasError(ClientHttpResponse response) throws IOException {
return hasError(response.getStatusCode());
}
protected boolean hasError(HttpStatus statusCode) {
return (statusCode.is4xxClientError() || statusCode.is5xxServerError());
}
#Override
public void handleError(ClientHttpResponse httpResponse) throws IOException {
if (httpResponse.getStatusCode().series() == SERVER_ERROR) {
// handle SERVER_ERROR
System.out.println("SERVER_ERROR");
} else if (httpResponse.getStatusCode().series() == CLIENT_ERROR) {
// handle CLIENT_ERROR
System.out.println("CLIENT_ERROR");
} else {
System.out.println("SOME_ERROR");
}
}
public void setMessageConverters(List<HttpMessageConverter<?>> messageConverters) {
this.messageConverters = messageConverters;
}
}
I'm getting below error that I want to handle -
Caused by: org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:4444/test": Read timed out; nested exception is java.net.SocketTimeoutException: Read timed out
If I'm adding .errorHandler(new CustomErrorHandler()) in Http.outboundgateway then I'm getting error saying -
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flow2' defined in class path resource [example/common/config/SpringIntegrationConfiguration.class]: Initialization of bean failed; nested exception is java.lang.RuntimeException: java.lang.IllegalArgumentException: the 'errorHandler' must be specified on the provided 'restTemplate'
the 'errorHandler' must be specified on the provided 'restTemplate'
It's probably clearly states that such an error handler must be configured on the externally provided RestTemplate.
When an IOException is thrown in the RestTemplate, that error handler is not invoked. See the source code of its doExecute() method:
...
response = request.execute();
handleResponse(url, method, response);
return (responseExtractor != null ? responseExtractor.extractData(response) : null);
}
catch (IOException ex) {
...
throw new ResourceAccessException("I/O error on " + method.name() +
" request for \"" + resource + "\": " + ex.getMessage(), ex);
This type of exceptions has to be handled around that HTTP Gateway call.
See if you can set a custom errorChannel header - enrichHeaders() and handle this error in the dedicated IntegrationFlow. Since you use an ExecutorChannel, the async error handling must have an effect.
Another way (and I recall as showed you before) is to use an ExpressionEvaluatingRequestHandlerAdvice on that Http.outboundGateway() endpoint.
See more in docs: https://docs.spring.io/spring-integration/reference/html/messaging-endpoints.html#message-handler-advice-chain
I am new to the Spring Integration project, now I need to create a flow with Java DSL and test it.
I came up with these flows. First one should run by cron and invoke second one, which invokes HTTP endpoint and translates XML response to POJO:
#Bean
IntegrationFlow pollerFlow() {
return IntegrationFlows
.from(() -> new GenericMessage<>(""),
e -> e.poller(p -> p.cron(this.cron)))
.channel("pollingChannel")
.get();
}
#Bean
IntegrationFlow flow(HttpMessageHandlerSpec bulkEndpoint) {
return IntegrationFlows
.from("pollingChannel")
.enrichHeaders(authorizationHeaderEnricher(user, password))
.handle(bulkEndpoint)
.transform(xmlTransformer())
.channel("httpResponseChannel")
.get();
}
#Bean
HttpMessageHandlerSpec bulkEndpoint() {
return Http
.outboundGateway(uri)
.httpMethod(HttpMethod.POST)
.expectedResponseType(String.class)
.errorHandler(new DefaultResponseErrorHandler());
}
Now I want to test flow and mock HTTP call, but struggling to mock HTTP handler, I tried to do it like that:
#ExtendWith(SpringExtension.class)
#SpringIntegrationTest(noAutoStartup = {"pollerFlow"})
#ContextConfiguration(classes = FlowConfiguration.class)
public class FlowTests {
#Autowired
private MockIntegrationContext mockIntegrationContext;
#Autowired
public DirectChannel httpResponseChannel;
#Autowired
public DirectChannel pollingChannel;
#Test
void test() {
final MockMessageHandler mockHandler = MockIntegration.mockMessageHandler()
.handleNextAndReply(message -> new GenericMessage<>(xml, message.getHeaders()));
mockIntegrationContext.substituteMessageHandlerFor("bulkEndpoint", mockHandler);
httpResponseChannel.subscribe(message -> {
assertThat(message.getPayload(), is(notNullValue()));
assertThat(message.getPayload(), instanceOf(PartsSalesOpenRootElement.class));
});
pollingChannel.send(new GenericMessage<>(""));
}
}
But I am always getting an error, that on line:
mockIntegrationContext.substituteMessageHandlerFor("bulkEndpoint", mockHandler);
org.springframework.beans.factory.BeanNotOfRequiredTypeException: Bean named 'bulkEndpoint' is expected to be of type 'org.springframework.integration.endpoint.IntegrationConsumer' but was actually of type 'org.springframework.integration.http.outbound.HttpRequestExecutingMessageHandler'
Am I doing something wrong here? I am assuming I have a problem with IntegrationFlow itself, or maybe my testing approach is a problem.
The error is correct. The bulkEndpoint is not an endpoint by itself. It is really a MessageHandler. The endpoint is created from the .handle(bulkEndpoint).
See docs: https://docs.spring.io/spring-integration/docs/current/reference/html/overview.html#finding-class-names-for-java-and-dsl-configuration and https://docs.spring.io/spring-integration/docs/current/reference/html/testing.html#testing-mocks.
So, to make it working you need to do something like this:
.handle(bulkEndpoint, e -> e.id("actualEndpoint"))
And then in the test:
mockIntegrationContext.substituteMessageHandlerFor("actualEndpoint", mockHandler);
You also probably need to think to not have that pollerFlow to be started when you test it sine you send the message into pollingChannel manually. So, there is no conflicts with what you'd like to test. For this reason you also add a id() into your e.poller(p -> p.cron(this.cron)) and use #SpringIntegrationTest(noAutoStartup) to have it stopped before your test. I see you try noAutoStartup = {"pollerFlow"}, but this is not going to help for static flows. You indeed need to have stopped an actual endpoint in this case.
I have a REST controller which calls a gateway annotated with #MessagingGateway(errorChannel = ERROR_CHANNEL)
This way, whatever error occurs downstream the integration flow initiated by the gateway will flow into an error channel which will be handled by another integration flow, this is working as expected.
Now, there is another scenario where an integration flow reads messages from Kafka, routes those messages to another channel, one more integration flow processes those messages and another flow sends an HTTP request to a remote service.
public IntegrationFlowBuilder attachmentEventTenantRouter(String tenantId) {
return attachmentEventBaseFlow(".*")
.filter(Message.class, m -> m.getHeaders().get(KafkaConstants.HEADER_PREFIX + MessageHeader.TENANT_ID_KEY) != null && m.getHeaders().get(KafkaConstants.HEADER_PREFIX + MessageHeader.TENANT_ID_KEY, String.class).equalsIgnoreCase(tenantId));
}
private IntegrationFlowBuilder attachmentEventBaseFlow(String eventRegex) {
return IntegrationFlows
.from(Kafka.messageDrivenChannelAdapter(kafkaListenerContainerFactory.createContainer(topic)).errorChannel(IntegrationContextUtils.ERROR_CHANNEL_BEAN_NAME))
.log(LoggingHandler.Level.DEBUG, "Inside Kafka Consumer")
.filter(Message.class, m -> filter(m, eventRegex))
.transform(messageToEventTransformer);
}
#Bean
public IntegrationFlow kafkaConsumerFlow() {
return fromKafkaFlowHelper.attachmentEventTenantRouter(TENANT_ID)
.route(Message.class, m -> m.getHeaders().get(KafkaConstants.HEADER_PREFIX + MessageHeader.EVENT_TYPE_KEY, String.class), p -> p
.resolutionRequired(false)
.channelMapping("eventType", "transformMessagesFromKafkaAndPublishAnotherEvent")
.defaultOutputChannel("nullChannel"))
.get();
}
#Bean
public IntegrationFlow transformMessagesFromKafkaAndPublishAnotherEvent() {
return flow -> flow
.transform(transformer)
.handle( getKafkaHandler() );
}
#Bean
public IntegrationFlow sendHttpRequestToRemoteServiceFromKafkaEvent() {
return flow -> flow
.transform(transformer)
.handle(gatewayCall, e -> e.advice(expressionAdvice()));
}
How can I do to handle the exceptions that might occur in the flows above?
As you can see, I am using a ExpressionEvaluatingRequestHandlerAdvice which does the work for the handle method, but not sure how to handle exceptions that might occur in the transformers?
The massage gateway with an error channel configured does the trick when the gateway is called by a rest controller, but when the flow is initiated by the Kafka consumer, I'm lost how to achieve this.
Thanks.
EDITED AFTER Artem's RESPONSE TO ADD CLARIFICATION:
This is the configuration of the integration flow that posts a request to a remote service and whose exceptions does not seem to be caught and routed to the errorChannel without a ExpressionEvaluatingRequestHandlerAdvice:
#Bean
public IntegrationFlow sendHttpRequestToRemoteServiceFromKafkaEvent() {
return flow -> flow
.transform(transformer)
.handle(getOAuth2Handler(HttpMethod.PUT, "remote url"), e -> e.advice(expressionEvaluatingRequestHandlerAdvice));
}
private OAuth2RequestHandler getOAuth2Handler(HttpMethod httpMethod, String url) {
return new OAuth2RequestHandler(oAuth2RestTemplate, httpMethod, url);
}
And class OAuth2RequestHandler which implements a MessageHandler
#Override
public void handleMessage(org.springframework.messaging.Message<?> message) throws MessagingException {
String requestBody = (String) message.getPayload();
ResponseEntity<String> response = oAuth2RestTemplate.exchange(url, httpMethod, new HttpEntity<>(requestBody), String.class);
}
I see you use already an errorChannel() on the Kafka message-driven channel adapter. So, what is the question?
This part of your flow is exact equivalent to mentioned #MesaagingGateway with its errorChannel configuration.
I use SpringIntegration-filter for validate my WS message. I implement Validators for validating and if WS message is valid, they returns true. But if WS messages are invalid, they throws a MyValidationException.
Is there a way for handle this exceptions with usage of SpringIntegration-filter? If I don't return false, filter don't work.
My code example is below. I want to use my validation exceptions in discard flow.
#Bean
public IntegrationFlow incomingRequest() {
return f -> f
.<IncomingRequest>filter(message ->
validatorFactory.validator(message.getName())
.validate(message),
filterEndpointSpec ->
filterEndpointSpec.discardChannel(discardChannel()))
.<IncomingRequest>handle((payload, headers) ->
applicationService.handle(payload));
}
#Bean
public IntegrationFlow discard() {
return IntegrationFlows.from(discardChannel())
.log("DISCARD FLOW")
.get();
}
#Bean(name = "discard.input")
public MessageChannel discardChannel() {
return MessageChannels.direct().get();
}
Given that the exception is comming from the validate when you check the WS request, you have to surround the call in a try catch. If an exception is thrown, it is catched and false is returned, indicating that the validation failed.
#Bean
public IntegrationFlow incomingRequest2() {
return f -> f
.filter(this::isValid, filterEndpointSpec ->
filterEndpointSpec.discardFlow(f2 -> f2.transform(this::getReason))
.discardChannel(discardChannel()))
.<IncomingRequest>handle((payload, headers) ->
applicationService.handle(payload));
}
And the helper methods.
public boolean isValid(IncomingRequest message) {
try {
return validatorFactory.validator(message.getName())
.validate(message);
} catch (Exception e) { // your exception
return false;
}
}
public String getReason(IncomingRequest message) { // return the object you need
try {
validatorFactory.validator(message.getName())
.validate(message);
return null;
} catch (Exception e) { // process exception as you want
return e.getMessage();
}
}
The discard channel just gets the rejected inbound message; there is no way to alter it in the filter.
You can do something like this...
.handle() // return an Exception on validation failure
.filter(...) // filter if payload is exception; the exceptions go to the discard channel
i.e. separate the validation and filter concerns.
I may just be missing something very simple here (or misusing something), but I was attempting to set up two direct channels such that one flow would pass some data to each sequentially. So using the Spring Integration JAVA DSL I had something like this (significantly simplified for this example):
public static final String TEST_CHANNEL = "testGateway";
public static final String TEST_UPPER_CHANNEL = "testChannelUpper";
public static final String TEST_LOWER_CHANNEL = "testChannelLower";
#Bean(name = TEST_CHANNEL)
public MessageChannel testGatewayChannel() {
return MessageChannels.direct(TEST_CHANNEL).get();
}
#Bean(name = TEST_UPPER_CHANNEL)
public MessageChannel testChannelUpperChannel() {
return MessageChannels.direct(TEST_UPPER_CHANNEL).get();
}
#Bean(name = TEST_LOWER_CHANNEL)
public MessageChannel testChannelLowerChannel() {
return MessageChannels.direct(TEST_LOWER_CHANNEL).get();
}
#Bean
public IntegrationFlow testFlow() {
return IntegrationFlows
.from(TEST_CHANNEL)
.channel(TEST_UPPER_CHANNEL)
.channel(TEST_LOWER_CHANNEL)
.get();
}
#Bean
public IntegrationFlow testUpperFlow() {
return IntegrationFlows
.from(TEST_UPPER_CHANNEL)
.<String, String>transform(String::toUpperCase)
.handle(System.out::println)
.get();
}
#Bean
public IntegrationFlow testLowerFlow() {
return IntegrationFlows
.from(TEST_LOWER_CHANNEL)
.<String, String>transform(String::toLowerCase)
.handle(System.out::println)
.get();
}
I'm using a REST endpoint to invoke the flow via a gateway, but when I do so it seems only one of the channels is invoked. The channel also seems to be random across invocations (sometimes going to the testChannelUpper and sometimes to the testChannelLower).
I basically end up with this across the executions :
(each time I am just hitting this endpoint http://localhost:9090/test?test=HellOoi)
Execution 1:
GenericMessage [payload=HELLOOI, headers={jobName=someActivity, history=someGateway,testGateway,testChannelUpper,testUpperFlow.channel#0, id=4aa7b075-23cc-6ab3-10a1-c7cb73bae49b, timestamp=1447686848477}]
Execution 2:
GenericMessage [payload=HELLOOI, headers={jobName=someActivity, history=someGateway,testGateway,testChannelUpper,testUpperFlow.channel#0, id=a18dcd01-da18-b00d-30c0-e1a03ce19104, timestamp=1447686853549}]
Execution 3:
GenericMessage [payload=hellooi, headers={jobName=someActivity, history=someGateway,testGateway,testChannelUpper,testLowerFlow.channel#0, id=5f0abcb9-378e-7a3c-9c93-a04ff6352927, timestamp=1447686857545}]
I believe that what I'm attempting here is also shown in the channelFlow example of the DSL wiki :
https://github.com/spring-projects/spring-integration-java-dsl/wiki/Spring-Integration-Java-DSL-Reference
Sooo the specs on what I'm using are :
Spring Boot v1.2.2.RELEASE
Spring v4.1.5.RELEASE
spring-integration-java-dsl 1.0.2.RELEASE
JDK 1.8.0_40-b25
So... has anyone else seen this kind of behavior? Am I just abusing the channel implementation? Any other ideas? Thanks in advance!
As Gary pointed out the best way to do this is to have a pub-sub and order the consumers on this :
#Bean(name = TEST_CHANNEL)
public MessageChannel testGatewayChannel() {
return MessageChannels.publishSubscribe(TEST_CHANNEL).get();
}
#Bean
public IntegrationFlow testUpperFlow() {
return IntegrationFlows
.from(TEST_CHANNEL)
.<String, String>transform(String::toUpperCase, e -> e.order(1))
.handle(System.out::println)
.get();
}
#Bean
public IntegrationFlow testLowerFlow() {
return IntegrationFlows
.from(TEST_CHANNEL)
.<String, String>transform(String::toLowerCase, e -> e.order(2))
.handle(System.out::println)
.get();
}
What is the purpose of this...
#Bean
public IntegrationFlow testFlow() {
return IntegrationFlows
.from(TEST_CHANNEL).fixedSubscriberChannel()
.channel(TEST_UPPER_CHANNEL)
.channel(TEST_LOWER_CHANNEL)
.get();
}
?
All that does is bridge the three channels together.
In fact, you end up with 2 consumers on TEST_UPPER_CHANNEL- the bridge in this flow and the transformer in your other flow.
By default, dispatching in direct channels uses round robin distribution. So the first message will go to the bridge, the next to the transformer, etc, etc.