We're using Spring integration in my application. I'd like put some objects into channel for asynchronous processing and error handling. So for this, I configured MessageGateway with error channel and PollableChannel for handling objects to be processed.
The problem
So I'm calling messageGateway.processMessage(message) to put message into channel. This works as expected - calling this method is non-blocking, messages get processed and are forwarded to next channel. But when processing method throws an exception, it is not redirected to error channel.
On the other hand when I change my processing channel from PollableChannel to SubscribableChannel, error channel works as expected, but calling the gateway is of course blocking. What am I missing? Can I have both non blocking call and error channel?
The code
Component doing the message processing:
#Component
public MessageProcessor {
#Transactional
#ServiceActivator(inputChannel = "msg.process", outputChannel = "msg.postprocess")
public void processMessage(MyMessage message) {
// Message processing that may throw exception
}
}
Channel definition:
#Configuration
public class IntegrationConfig {
#Bean(name = "msg.process")
private MessageChannel processChannel() {
return new RendezvousChannel();
}
#Bean(name = "msg.error")
private MessageChannel errorChannel() {
return new DirectChannel();
}
}
My gateway looks like this:
#MessagingGateway(errorChannel = "msg.error")
public interface MessagingGateway {
#Gateway(requestChannel = "msg.processing")
void processMessage(MyMessage message);
}
Error handler:
#Component
public ErrorHandlers {
#Transactional
#ServiceActivator(inputChannel = "msg.error")
public void processError(MessagingException me) {
// Error handling is here
}
}
But when processing method throws an exception, it is not redirected to error channel.
When a gateway method returns void, the calling thread is released immediately when it returns to the gateway.
The gateway does not add an error channel header in this case (in the next release - 5.0) we have changed that.
In the meantime, you can use a header enricher to set the errorChannel header to your error channel. You can also use the defaultHeaders property on the #MessagingGateway - see the comments on this answer.
Related
I have some Spring applications that communicate between them using RabbitMQ as broker. I can send and receive messages asynchronously between them. But now, I need one application to send a message to another one and wait for the response. So, for this I am trying to implement the RPC pattern. It is working, but the problem is that I could only do it using temporary queues generated by Spring.
https://www.rabbitmq.com/tutorials/tutorial-six-spring-amqp.html
This is the code that sends the message and wait for the response.
public void send() {
....
Integer response = (Integer) template.convertSendAndReceive(exchange.getName(), "rpc", "message");
...
}
When I send the message, the execution is blocked until the response is received and a temporary queue is created by Spring for the response, as expected.
But what I need is to use a specific and fixed queue, defined by me, to receive the responses. I need responses to be sent to an exchange with a routing key pointing to the fixed response queue (doing this I'll be able to send the responses to another queue too, that will be logging all responses).
I tried setting the "setReplyTo" property to the message, but is not working.
What version are you using? With modern versions, direct reply_to is used by default, but you can revert to using a temporary queue by setting a property on the template.
https://docs.spring.io/spring-amqp/docs/current/reference/html/#direct-reply-to
To use a named reply queue, see the documentation about how to set up a reply container, with the template as the message listener:
https://docs.spring.io/spring-amqp/docs/current/reference/html/#message-correlation-with-a-reply-queue
and
https://docs.spring.io/spring-amqp/docs/current/reference/html/#reply-listener
EDIT
The template will block until the corresponding reply is passed into it by the reply container (or it times out).
#SpringBootApplication
public class So68986604Application {
public static void main(String[] args) {
SpringApplication.run(So68986604Application.class, args);
}
#RabbitListener(queues = "foo")
public String listen(String in) {
System.out.println(in);
return in.toUpperCase();
}
#Bean
Queue foo() {
return new Queue("foo");
}
#Bean
Queue replies() {
return new Queue("foo.replies");
}
#Bean
SimpleMessageListenerContainer replyContainer(ConnectionFactory cf, RabbitTemplate template) {
SimpleMessageListenerContainer replyer = new SimpleMessageListenerContainer(cf);
replyer.setQueueNames("foo.replies");
replyer.setMessageListener(template);
template.setReplyAddress("foo.replies");
return replyer;
}
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
return args -> {
System.out.println(template.convertSendAndReceive("", "foo", "test"));
};
}
}
test
TEST
I currently have a TcpInboundGateway that takes in messages, does some processing on the message and then returns the appropriate response, all as a TcpInboundGateway should.
However, I am curious if this TcpInboundGateway can be configured in such a way that it will send an immediate response to the originating request but continue to process the request and send the post-processing response as well?
Think of this immediate response as an acknowledgement to the sender that the message was received.
Possible Solution:
After reviewing this post, I came up with what I believe to be a viable solution to this problem.
#Configuration
#EnableIntegration
public class Configuration {
#Bean
public AbstractServerConnectionFactory serverConnectionFactory() {
return new TcpNetServerConnectionFactory(2002);
}
#Bean
public TcpReceivingChannelAdapter inboundAdapter(AbstractServerConnectionFactory serverConnectionFactory) {
TcpReceivingChannelAdapter inboundAdapter = new TcpReceivingChannelAdapter();
inboundAdapter.setConnectionFactory(serverConnectionFactory);
inboundAdapter.setOutputChannelName("sendAcknowledgement");
return inboundAdapter;
}
#MessageEndpoint
public class InboundMessageHandler {
#Autowired
private OutboundMessageGateway gateway;
#ServiceActivator(inputChannel="sendAcknowledgement", outputChannel="doProcessing")
public Message<String> initialAck(Message<String> message) {
gateway.send("ACK", message.getHeaders().get(IpHeaders.CONNECTION_ID).toString());
return message;
}
#ServiceActivator(inputChannel="doProcessing", outputChannel="sendResponse")
public Message<String> mockDelay(Message<String> message) throws InterruptedException {
return message;
}
}
#MessagingGateway(defaultRequestChannel="sendResponse")
public interface OutboundMessageGateway {
void send(#Payload String message, #Header(IpHeaders.CONNECTION_ID) String connectionId);
}
#Bean
#ServiceActivator(inputChannel="sendResponse")
public TcpSendingMessageHandler outboundAdapter(AbstractServerConnectionFactory serverConnectionFactory) {
TcpSendingMessageHandler outboundAdapter = new TcpSendingMessageHandler();
outboundAdapter.setConnectionFactory(serverConnectionFactory);
return outboundAdapter;
}
}
For the use-case with the TcpInboundGateway and acking with later reply you need to use a PublishSubscribeChannel with an Executor injected to make a processing async.
The first subscriber should return some ack into the replyChannel header. This way your TcpInboundGateway will perform request-reply and return that ack into the socket connected.
At the same time as you want, the second subscriber can perform desired logic and build the real reply later. Only the point that we need to use the mention in the docs Collaborating Outbound and Inbound Channel Adapters (as you noticed already). So, since TcpInboundGateway populates an IpHeaders.CONNECTION_ID header into a request message, it is going to be available in your async process and subsequent TcpSendingMessageHandler will know where to send your processed reply:
private void handleMessageAsServer(Message<?> message) {
// We don't own the connection, we are asynchronously replying
String connectionId = message.getHeaders().get(IpHeaders.CONNECTION_ID, String.class);
TcpConnection connection = null;
if (connectionId != null) {
connection = this.connections.get(connectionId);
}
if (connection != null) {
try {
connection.send(message);
}
So, what you need is like this:
a PublishSubscribeChannel with an executor for your TcpInboundGateway
A simple handler to reply with an ack as a first subscriber
A sub-flow for processing a request
A TcpSendingMessageHandler to send a process response into the same TCP connection.
I have a simple Spring Boot service that listens to an AWS SQS queue using JMSTemplate. Everything works as expected when the message is properly handled.
I am using CLIENT_ACKNOWLEDGE so when an exception is thrown during processing, the message is received again. However the Default Visibility Timeout setting on the SQS queue is being ignored and the message is being received again immediately.
The SQS queue is configured with a 30 second Default Visibility Timeout and a re-drive policy of 20 receives before putting the message on a DLQ.
I have disabled the service and used the SQS Console to verify that the Default Visibility Timeout is properly set. I have also tried adding the JMS Message to the method signature and performing manual validation.
Here is code for the JMS Configuration:
#Configuration
#EnableJms
class JmsConfig
{
#Bean
#Conditional(AWSEnvironmentCondition.class)
public SQSConnectionFactory connectionFactory(#Value("${AWS_REGION}") String awsRegion)
{
return new SQSConnectionFactory(
new ProviderConfiguration(),
AmazonSQSClientBuilder.standard()
.withRegion(Regions.fromName(awsRegion))
.withCredentials(new DefaultAWSCredentialsProviderChain())
);
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(ConnectionFactory connectionFactory)
{
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setDestinationResolver(new DynamicDestinationResolver());
factory.setConcurrency("3-10");
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
factory.setErrorHandler(defaultErrorHandler());
return factory;
}
#Bean
public ErrorHandler defaultErrorHandler()
{
return new ErrorHandler()
{
#Override
public void handleError(Throwable throwable)
{
LOG.error("JMS message listener error: {}", throwable.getMessage());
}
};
}
#Bean
public JmsTemplate defaultJmsTemplate(ConnectionFactory connectionFactory)
{
return new JmsTemplate(connectionFactory);
}
}
And here is code for the Listener:
#Component
public class MessagingListener
{
#Autowired
private MessageService _messageService;
#Autowired
private Validator _validator;
#JmsListener(destination = "myqueue")
public void receiveMessage(String messageJson)
{
try
{
LOG.info("Received message");
// The following line throws an IOException is the message is not JSON.
MyMessage myMessage = MAPPER.readvalue(messageJson, MyMessage.class);
Set<ConstraintViolation<MyMessage>> _validator.validate(myMessage);
if (CollectionUtils.isNotEmpty(violations))
{
String errorMessage = violations.stream()
.map(v -> String.join(" : ", v.getPropertyPath().iterator().next().getName(),
v.getMessage()))
LOG.error("Exception occurred while validating the model, details: {}", errorMessage)
throw new ValidationException(errorMessage);
}
}
catch (IOException e)
{
LOG.error("Error parsing message", e);
throw new ValidationException("Error parsing message, details: " + e.getMessage());
}
}
}
When a message is placed on the SQS queue with either invalid JSON or JSON that that does not pass validation, the message is received 20 times very quickly and then ends up on the DLQ. What needs to be done so that the Default Visibility Timeout setting in SQS is respected?
In case of an exception, visibility timeout of the failed message is set to 0 via ChangeMessageVisibility so SQS will send this message immediately even though queue has a different visibilityTimeout setting.
How does that happen?
As you can see here, Spring JMS' AbstractMessageListenerContainer briefly does this:
try {
invokeListener(session, message); // This is your #JMSListener method
}
catch (JMSException | RuntimeException | Error ex) {
rollbackOnExceptionIfNecessary(session, ex);
throw ex;
}
commitIfNecessary(session, message);
On rollbackOnExceptionIfNecessary method, session.recover() will be invoked because:
session.getTransacted() will always be false since SQS does not support transactions. See here.
isClientAcknowledge(session) will return true because you're using CLIENT_ACKNOWLEDGE mode.
And lastly recover() of SQSSession negative acknowledges the message, which means setting visibilityTimeout of that specific message to 0, causes SQS to try sending that message immediately.
The easiest way to override this behavior would be implementing a CustomJmsListenerContainerFactory & CustomMessageListenerContainer instead of using DefaultJmsListenerContainerFactory & DefaultMessageListenerContainer.
public class CustomMessageListenerContainer extends DefaultMessageListenerContainer {
public CustomMessageListenerContainer() {
super();
}
#Override
protected void rollbackOnExceptionIfNecessary() {
// do nothing, so that "visibilityTimeout" will stay same
}
}
public class CustomJmsListenerContainerFactory {
#Override
protected DefaultMessageListenerContainer createContainerInstance() {
return new CustomMesageListenerContainer();
}
}
And make it a Spring bean either with #Component or just like you did in JmsConfig:
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(ConnectionFactory connectionFactory) {
DefaultJmsListenerContainerFactory factory = new CustomJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
// and set other stuff on factory
return factory;
}
NOTE:
If your application is consuming other type of data sources along SQS with JMS, make sure to use different Container and ContainerFactory for them so that rollbackOnExceptionIfNecessary behaves as expected.
I wrote a simple message flow with request and reply. I have to use two independent queues so i declare AmqpOutboundAdapter to send a message and AmqpInboundAdapter to receive a reply.
#Bean
#FindADUsers
public AmqpOutboundEndpoint newFindADUsersOutboundAdapter() {
return Amqp.outboundAdapter(amqpTemplate())
.routingKeyExpression("headers[" + ADUsersFindConfig.ROUTING_KEY_HEADER + "]")
.exchangeName(getExchange())
.headerMapper(amqpHeaderMapper())
.get();
}
#Bean
public AmqpInboundChannelAdapter newFindADUsersResponseInboundChannelAdapter(
ADUsersFindResponseConfig config) {
return Amqp.inboundAdapter(rabbitConnectionFactory(), findADUsersResponseQueue)
.headerMapper(amqpHeaderMapper())
.outputChannel(config.newADUsersFindResponseOutputChannel())
.get();
}
It should work with #MessagingGateway:
#MessagingGateway
public interface ADUsersFindService {
String FIND_AD_USERS_CHANNEL = "adUsersFindChannel";
String FIND_AD_USERS_REPLY_OUTPUT_CHANNEL = "adUsersFindReplyOutputChannel";
String FIND_AD_USERS_REPLY_CHANNEL = "adUsersFindReplyChannel";
String CORRELATION_ID_REQUEST_HEADER = "correlation_id";
String ROUTING_KEY_HEADER = "replyRoutingKey";
String OBJECT_TYPE_HEADER = "object.type";
#Gateway(requestChannel = FIND_AD_USERS_CHANNEL, replyChannel = FIND_AD_USERS_REPLY_CHANNEL)
ADResponse find(ADRequest adRequest, #Header(ROUTING_KEY_HEADER) String routingKey, #Header(OBJECT_TYPE_HEADER) String objectType);
}
And the ADUsersFindResponseConfig class looks like:
#Configuration
#Import(JsonConfig.class)
public class ADUsersFindResponseConfig {
#Autowired
public NullChannel nullChannel;
#Autowired
private JsonObjectMapper<?, ?> mapper;
/**
* #return The output channel for the flow
*/
#Bean(name = ADUsersFindService.FIND_AD_USERS_REPLY_OUTPUT_CHANNEL)
public MessageChannel newADUsersFindResponseOutputChannel() {
return MessageChannels.direct().get();
}
/**
* #return The output channel for gateway
*/
#Bean(name = ADUsersFindService.FIND_AD_USERS_REPLY_CHANNEL)
public MessageChannel newADUsersFindResponseChannel() {
return MessageChannels.direct().get();
}
#Bean
public IntegrationFlow findADUsersResponseFlow() {
return IntegrationFlows
.from(newADUsersFindResponseOutputChannel())
.transform(new JsonToObjectTransformer(ADResponse.class, mapper))
.channel(newADUsersFindResponseChannel())
.get();
}
}
Sending message works properly, but i have a problem with receiving message. I am expecting that received message will be passed to channel called FIND_AD_USERS_REPLY_OUTPUT_CHANNEL, then the message will be deserialized to ADResponse object using findADUsersResponseFlow , and next ADResponse object will be passed to gateway replyChannel - FIND_AD_USERS_REPLY_CHANNEL. Finally, 'find' method return this object. Unfortunately when org.springframework.integration.handler.BridgeHandler receive a message, i got exception:
org.springframework.messaging.MessagingException: ; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available
Message log looks like:
11:51:35.697 [SimpleAsyncTaskExecutor-1] INFO New message - GenericMessage [payload={...somepayload...}, headers={correlation_id=7cbd958e-4b09-4e4c-ba8e-5ba574f3309a, replyRoutingKey=findADUsersResponse.ad, amqp_consumerQueue=findADUsersResponseQueue, history=newFindADUsersResponseInboundChannelAdapter,adUsersFindReplyOutputChannel,adUsersFindReplyChannel,infoLog,infoLoggerChain.channel#0,infoLoggerChain.channel#1, id=37a4735d-6983-d1ad-e0a1-b37dc17e48ef, amqp_consumerTag=amq.ctag-8Qs5YEun1jXYRf85Hu1URA, object.type=USER, timestamp=1469094695697}]
So i'm pretty sure that message was passed to adUsersFindReplyChannel. Also (if it's important) both request message and reply message have 'replyTo' header set to null. What am I doing wrong?
The replyChannel header is a live object and can't be serialized over AMQP.
You can use an outbound gateway instead of the pair of adapters and the framework will take care of the headers.
If you must use adapters for some reason, you need to do 2 things:
Use the header channel registry to convert the channel object to a String which is registered with the registry.
Make sure that the header mapper is configured to send/receive the replyChannel header and that your receiving system returns the header in the reply.
I have two separate applications running either side of an ActiveMQ broker; application 1 sends synchronous requests to application 2 which returns the response back to application 1. At present the replies are via temporary queues and I am now trying to create a named reply destination to avoid the overhead of creating multiple temporary queues.
Application 1
#MessagingGateway
public interface OrderGateway {
#Gateway(requestChannel = "requestChannel", replyChannel = "responseChannel")
public OrderDto fetchOrder(OrderRequest orderRequest);
}
#Bean
public IntegrationFlow outgoingRequestFlow(ConnectionFactory connectionFactory) {
return IntegrationFlows.from("requestChannel")
.handle(Jms.outboundGateway(connectionFactory)
.requestDestination("request.queue")
.replyDestination("response.topic")
.correlationKey("JMSCorrelationID"))
.channel("responseChannel")
.get();
}
Application 2
#Bean
public IntegrationFlow incomingRequestFlow(ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Jms.inboundGateway(connectionFactory)
.destination("request.queue")
.correlationKey("JMSCorrelationID"))
.channel("requestChannel")
.handle("requestServiceActivator", "handleRequest")
.channel("responseChannel")
.get();
}
#Component
public class OrderServiceActivator {
#Autowired
OrderService orderService;
#ServiceActivator
public OrderDto fetchOrder(OrderRequest orderRequest) {
return orderService.getById(orderRequest.getId());
}
}
When I start both applications request.queue gets created and has one consumer (application 2). response.topic gets created but for some reason it has no consumers. Consequently when I send a request in to application 1 it reaches application 2, but after 5 seconds application 1 does not receive a reply and times out and the following errors are logged:
Application 2
org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'org.springframework.web.context.WebApplicationContext:/application-2.responseChannel'
Application 1
org.springframework.integration.MessageTimeoutException: failed to receive JMS response within timeout of: 5000ms
I presume I've made some simple configuration error, any help would be appreciated.
With your configuration, there is no long-lived consumer for the reply queue - a consumer is created for each request (with a message selector for the specific correlation id).
If you add .replyContainer() there will be a permanent consumer.
However, it should make no difference functionally.
I just ran tests similar to yours with and without replyContainer() and it all worked fine for me...
#Bean
public IntegrationFlow jmsOutboundGatewayFlow() {
return f -> f.handleWithAdapter(a ->
a.jmsGateway(this.jmsConnectionFactory)
// .replyContainer()
.replyDestination("pipereplies")
.correlationKey("JmsCorrelationID")
.requestDestination("jmsPipelineTest"));
}
I suggest you turn on debug logging to see if that sheds some light.
#Bean
public IntegrationFlow jmsInboundGatewayFlow() {
return IntegrationFlows.from((MessagingGateways g) ->
g.jms(this.jmsConnectionFactory)
.correlationKey("JmsCorrelationID")
.destination("jmsPipelineTest"))
.<String, String>transform(String::toUpperCase)
.get();
}
I'm failing to find how the #ServiceActivator is getting wired...
Usually it's something like:
#ServiceActivator(inputChannel = "requestChannel", outputChannel = "responseChannel")
public .....
Perhaps that is what you are missing.