Develop Reference

RabbitMQ Spring Java - RPC Pattern

I have some Spring applications that communicate between them using RabbitMQ as broker. I can send and receive messages asynchronously between them. But now, I need one application to send a message to another one and wait for the response. So, for this I am trying to implement the RPC pattern. It is working, but the problem is that I could only do it using temporary queues generated by Spring.
https://www.rabbitmq.com/tutorials/tutorial-six-spring-amqp.html
This is the code that sends the message and wait for the response.
public void send() {
....
Integer response = (Integer) template.convertSendAndReceive(exchange.getName(), "rpc", "message");
...
}
When I send the message, the execution is blocked until the response is received and a temporary queue is created by Spring for the response, as expected.
But what I need is to use a specific and fixed queue, defined by me, to receive the responses. I need responses to be sent to an exchange with a routing key pointing to the fixed response queue (doing this I'll be able to send the responses to another queue too, that will be logging all responses).
I tried setting the "setReplyTo" property to the message, but is not working.

What version are you using? With modern versions, direct reply_to is used by default, but you can revert to using a temporary queue by setting a property on the template.
https://docs.spring.io/spring-amqp/docs/current/reference/html/#direct-reply-to
To use a named reply queue, see the documentation about how to set up a reply container, with the template as the message listener:
https://docs.spring.io/spring-amqp/docs/current/reference/html/#message-correlation-with-a-reply-queue
and
https://docs.spring.io/spring-amqp/docs/current/reference/html/#reply-listener
EDIT
The template will block until the corresponding reply is passed into it by the reply container (or it times out).
#SpringBootApplication
public class So68986604Application {
public static void main(String[] args) {
SpringApplication.run(So68986604Application.class, args);
}
#RabbitListener(queues = "foo")
public String listen(String in) {
System.out.println(in);
return in.toUpperCase();
}
#Bean
Queue foo() {
return new Queue("foo");
}
#Bean
Queue replies() {
return new Queue("foo.replies");
}
#Bean
SimpleMessageListenerContainer replyContainer(ConnectionFactory cf, RabbitTemplate template) {
SimpleMessageListenerContainer replyer = new SimpleMessageListenerContainer(cf);
replyer.setQueueNames("foo.replies");
replyer.setMessageListener(template);
template.setReplyAddress("foo.replies");
return replyer;
}
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
return args -> {
System.out.println(template.convertSendAndReceive("", "foo", "test"));
};
}
}
test
TEST

Update: Spring Boot JMS static reply queue on IBM MQ Series

In my use case I need to do request-reply call to a remote system via managed queues. Using Spring Boot and IBM's MQ starter I have the problem that the application wants to create dynamic/temporary reply queues instead of using the already existing managed queue.
Configuration is set up here
#EnableJms
#Configuration
public class QueueConfiguration {
#Bean
public MQQueueConnectionFactory connectionFactory() throws JMSException {
MQQueueConnectionFactory factory = new MQQueueConnectionFactory();
factory.setTransportType(CT_WMQ); // is 1
factory.setHostName(queueProperties.getHost());
factory.setPort(queueProperties.getPort());
factory.setChannel(queueProperties.getChannel()); // combo of ${queueManager}%${channel}
return factory;
}
#Bean
public JmsMessagingTemplate messagingTemplate(ConnectionFactory connectionFactory) {
JmsMessagingTemplate jmt = new JmsMessagingTemplate(connectionFactory);
jmt.setDefaultDestinationName(queueProperties.getQueueName());
return jmt;
}
#Bean
public Jaxb2Marshaller marshaller() {
Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
marshaller.setPackagesToScan("com.foo.model");
return marshaller;
}
#Bean
public MessageConverter messageConverter(Jaxb2Marshaller marshaller) {
MarshallingMessageConverter converter = new MarshallingMessageConverter();
converter.setMarshaller(marshaller);
converter.setUnmarshaller(marshaller);
return converter;
}
}
Usage is pretty straight forward: Take the object convert and send it. Wait for response receive
and convert it.
#Component
public class ExampleSenderReceiver {
#Autowired
private JmsMessagingTemplate jmsMessagingTemplate;
#Override
#SneakyThrows
public ResponseExample sendAndReceive(RequestExample request, String correlationId) {
MessagePostProcessor mpp = message -> {
message = MessageBuilder.fromMessage(message)
.setHeader(JmsHeaders.CORRELATION_ID, correlationId)
// .setHeader(JmsHeaders.REPLY_TO, "DEV.QUEUE.3") this triggers queue creation
.build();
return message;
};
String destination = Objects.requireNonNull(jmsMessagingTemplate.getDefaultDestinationName());
return jmsMessagingTemplate.convertSendAndReceive(destination, request, ResponseExample.class, mpp);
}
I read already a lot of IBM documentation and think, I need to set the message type to "MQMT_REQUEST" but I do not find the right spot to do so.
Update
Added Spring Integration as Gary proposed and added a configuration for JmsOutboundGateway
#Bean
public MessageChannel requestChannel() {
return new DirectChannel();
}
#Bean
public QueueChannel responseChannel() {
return new QueueChannel();
}
#Bean
#ServiceActivator(inputChannel = "requestChannel" )
public JmsOutboundGateway jmsOutboundGateway( ConnectionFactory connectionFactory) {
JmsOutboundGateway gateway = new JmsOutboundGateway();
gateway.setConnectionFactory(connectionFactory);
gateway.setRequestDestinationName("REQUEST");
gateway.setReplyDestinationName("RESPONSE");
gateway.setReplyChannel(responseChannel());
gateway.setCorrelationKey("JMSCorrelationID*");
gateway.setIdleReplyContainerTimeout(2, TimeUnit.SECONDS);
return gateway;
}
And adapted my ExampleSenderReceiver class
#Autowired
#Qualifier("requestChannel")
private MessageChannel requestChannel;
#Autowired
#Qualifier("responseChannel")
private QueueChannel responseChannel;
#Override
#SneakyThrows
public ResponseExample sendAndReceive(RequestExample request, String correlationId) {
String xmlContent = "the marshalled request object";
Map<String, Object> header = new HashMap<>();
header.put(JmsHeaders.CORRELATION_ID, correlationId);
GenericMessage<String> message1 = new GenericMessage<>(xmlContent, header);
requestChannel.send(message1);
log.info("send done" );
Message<?> receive = responseChannel.receive(1500);
if(null != receive){
log.info("incoming: {}", receive.toString());
}
}
The important part is gateway.setCorrelationKey("JMSCorrelationID*");
Without that line the correlationId was not propagated correct.
Next step is re-adding MessageConverters and make it nice again.
Thank you.

The default JmsTemplate (used by the JmsMessagingTemplate) always uses a temporary reply queue. You can subclass it and override doSendAndReceive(Session session, Destination destination, MessageCreator messageCreator) to use your managed queue instead.
However, it will only work if you have one request outstanding at a time (e.g. all run on a single thread). You will also have to add code for discarding "late" arrivals by checking the correlation id.
You can use async sends instead and handle replies on a listener container and correlate the replies to the requests.
Consider using spring-integration-jms and its outbound gateway instead - it has much more flexibility in reply queue handling (and does all the correlation for you).
https://docs.spring.io/spring-integration/reference/html/jms.html#jms-outbound-gateway

You are missing the queue manager.
ibm:
mq:
queueManager: QM1
channel: chanel
connName: localhost(1414)
user: admin
password: admin

Configuring a TcpInboundGateway to send multiple responses to the same message

I currently have a TcpInboundGateway that takes in messages, does some processing on the message and then returns the appropriate response, all as a TcpInboundGateway should.
However, I am curious if this TcpInboundGateway can be configured in such a way that it will send an immediate response to the originating request but continue to process the request and send the post-processing response as well?
Think of this immediate response as an acknowledgement to the sender that the message was received.
Possible Solution:
After reviewing this post, I came up with what I believe to be a viable solution to this problem.
#Configuration
#EnableIntegration
public class Configuration {
#Bean
public AbstractServerConnectionFactory serverConnectionFactory() {
return new TcpNetServerConnectionFactory(2002);
}
#Bean
public TcpReceivingChannelAdapter inboundAdapter(AbstractServerConnectionFactory serverConnectionFactory) {
TcpReceivingChannelAdapter inboundAdapter = new TcpReceivingChannelAdapter();
inboundAdapter.setConnectionFactory(serverConnectionFactory);
inboundAdapter.setOutputChannelName("sendAcknowledgement");
return inboundAdapter;
}
#MessageEndpoint
public class InboundMessageHandler {
#Autowired
private OutboundMessageGateway gateway;
#ServiceActivator(inputChannel="sendAcknowledgement", outputChannel="doProcessing")
public Message<String> initialAck(Message<String> message) {
gateway.send("ACK", message.getHeaders().get(IpHeaders.CONNECTION_ID).toString());
return message;
}
#ServiceActivator(inputChannel="doProcessing", outputChannel="sendResponse")
public Message<String> mockDelay(Message<String> message) throws InterruptedException {
return message;
}
}
#MessagingGateway(defaultRequestChannel="sendResponse")
public interface OutboundMessageGateway {
void send(#Payload String message, #Header(IpHeaders.CONNECTION_ID) String connectionId);
}
#Bean
#ServiceActivator(inputChannel="sendResponse")
public TcpSendingMessageHandler outboundAdapter(AbstractServerConnectionFactory serverConnectionFactory) {
TcpSendingMessageHandler outboundAdapter = new TcpSendingMessageHandler();
outboundAdapter.setConnectionFactory(serverConnectionFactory);
return outboundAdapter;
}
}

For the use-case with the TcpInboundGateway and acking with later reply you need to use a PublishSubscribeChannel with an Executor injected to make a processing async.
The first subscriber should return some ack into the replyChannel header. This way your TcpInboundGateway will perform request-reply and return that ack into the socket connected.
At the same time as you want, the second subscriber can perform desired logic and build the real reply later. Only the point that we need to use the mention in the docs Collaborating Outbound and Inbound Channel Adapters (as you noticed already). So, since TcpInboundGateway populates an IpHeaders.CONNECTION_ID header into a request message, it is going to be available in your async process and subsequent TcpSendingMessageHandler will know where to send your processed reply:
private void handleMessageAsServer(Message<?> message) {
// We don't own the connection, we are asynchronously replying
String connectionId = message.getHeaders().get(IpHeaders.CONNECTION_ID, String.class);
TcpConnection connection = null;
if (connectionId != null) {
connection = this.connections.get(connectionId);
}
if (connection != null) {
try {
connection.send(message);
}
So, what you need is like this:
a PublishSubscribeChannel with an executor for your TcpInboundGateway
A simple handler to reply with an ack as a first subscriber
A sub-flow for processing a request
A TcpSendingMessageHandler to send a process response into the same TCP connection.

Spring Integration fetch paginated results from a REST service

I'm working on an integration with a REST service, the idea is that it's polled by an outbound gateway marketingCategoryOutboundGateway implemented by HttpRequestExecutingMessageHandler. The gateway makes a request to the REST service and pushes its response to the marketingCategory channel. The gateway itself is triggered by a message created by marketingCategoryPollerMessageSource using the makeTriggeringMessage factory method.
The problem is that the service returns paginated results. I something which would listen on the marketingCategory channel, apart from the service activator I already have, check if the response and push a new message with an incremented page number created by makeTriggeringMessage to the marketingCategoryPoller channel, so that the code would spin in a loop until it fetches all the pages from the REST service.
Does Spring Integration allow to make such filters which receive one message on the input channel, test it against a condition and push a new message to the output channel if the condition is true?
The code:
//Responses from the REST service go to this channel
#Bean("marketingCategory")
MessageChannel marketingCategory() { return new PublishSubscribeChannel();}
//This channel is used to trigger the outbound gateway which makes a request to the REST service
#Bean
MessageChannel marketingCategoryPoller() {return new DirectChannel();}
//An adapter creating triggering messages for the gateway
#Bean
#InboundChannelAdapter(channel = "marketingCategoryPoller", poller = #Poller(fixedDelay = "15000"))
public MessageSource<String> marketingCategoryPollerMessageSource() { return () -> makeTriggeringMessage(1);}
//A factory for producing messages which trigger the gateway
private Message<String> makeTriggeringMessage(int page) {
//make a message for triggering marketingCategoryOutboundGateway
return MessageBuilder.withPayload("")
.setHeader("Host", "eclinic")
.setHeader("page", page)
.build();
}
//An outbound gateway, makes a request to the REST service and returns the response to marketingCategory channel
#Bean
#ServiceActivator(inputChannel = "marketingCategoryPoller")
public MessageHandler marketingCategoryOutboundGateway(#Qualifier("marketingCategory") MessageChannel channel) {
//make a request to the REST service and push the response to the marketingCategory channel
}
//handler for REST service responses
#Bean
#ServiceActivator(inputChannel = "marketingCategory")
public MessageHandler marketingCategoryHandler() {
return (msg) -> {
//process the categories returned by marketingCategoryOutboundGateway
};
}

I've found a solution based on this posting Read and download from a paginated REST-Services with spring integration:
Trigger the outbound gateway which talks to the REST service and pushes the response to a channel using an inbound channel adapter with a poller.
The inbound channel adapter is a message source which originally generates a message with a header indicating the page number to fetch from the REST API.
The page message header is used by the outbound gateway to generate a url specifying the desired page
The channel to which the outbound gateway pushes REST service responses has 2 subscribers:
2.1. a service activator which does something with the fetched data
2.2. a filter, which checks if this is the last page and if not, it sends the message further to another channel used by a header enricher
Having received a message, the header enricher increments its page header and pushes the message further to the channel which triggers the outbound gateway,
the gateway read the incremented page header and fetches the next page from the REST service
The loop keeps spinning until the REST service returns the last page. The filter doesn't let this message to pass through to the header enricher breaking the loop.
Full code:
#Configuration
public class IntegrationConfiguration {
private final ApiGateConfig apiGateConfig;
IntegrationConfiguration(ApiGateConfig apiGateConfig) {
this.apiGateConfig = apiGateConfig;
}
#Bean("marketingCategory")
MessageChannel marketingCategory() {
return new PublishSubscribeChannel();
}
#Bean
MessageChannel marketingCategoryPoller() {
return new DirectChannel();
}
#Bean
MessageChannel marketingCategoryPollerNextPage() {
return new DirectChannel();
}
#Bean
#InboundChannelAdapter(channel = "marketingCategoryPoller", poller = #Poller(fixedDelay = "15000"))
public MessageSource<RestPageImpl<MarketingCategory>> marketingCategoryPollerMessageSource() {
return () -> makeTriggeringMessage(0);
}
/**
* Build a gateway triggering message
*/
private Message<RestPageImpl<MarketingCategory>> makeTriggeringMessage(int page) {
return MessageBuilder.withPayload(new RestPageImpl<MarketingCategory>())
.setHeader("Host", "eclinic")
.setHeader("page", page)
.build();
}
#Bean
#ServiceActivator(inputChannel = "marketingCategoryPoller")
public MessageHandler marketingCategoryOutboundGateway(#Qualifier("marketingCategory") MessageChannel channel) {
String uri = apiGateConfig.getUri() + "/marketingCategories?page={page}";
//the type of the payload
ParameterizedTypeReference<RestPageImpl<MarketingCategory>> type = new ParameterizedTypeReference<>() {
};
//page number comes from the message
SpelExpressionParser expressionParser = new SpelExpressionParser();
var uriVariables = new HashMap<String, Expression>();
uriVariables.put("page", expressionParser.parseExpression("headers.page"));
HttpRequestExecutingMessageHandler handler = new HttpRequestExecutingMessageHandler(uri);
handler.setHttpMethod(HttpMethod.GET);
handler.setExpectedResponseTypeExpression(new ValueExpression<>(type));
handler.setOutputChannel(channel);
handler.setUriVariableExpressions(uriVariables);
return handler;
}
#Bean
#ServiceActivator(inputChannel = "marketingCategory")
public MessageHandler marketingCategoryHandler() {
return (msg) -> {
var page = (RestPageImpl<MarketingCategory>) msg.getPayload();
System.out.println("Page #" + page.getNumber());
page.getContent().forEach(c -> System.out.println(c.getMarketingCategory()));
};
}
#Filter(inputChannel = "marketingCategory", outputChannel = "marketingCategoryPollerNextPage")
public boolean marketingCategoryPaginationFilter(RestPageImpl<MarketingCategory> page) {
return !page.isLast();
}
#Bean
#Transformer(inputChannel = "marketingCategoryPollerNextPage", outputChannel = "marketingCategoryPoller")
HeaderEnricher incrementPage() {
Map<String, HeaderValueMessageProcessor<?>> headersToAdd = new HashMap<>();
Expression expression = new SpelExpressionParser().parseExpression("headers.page+1");
var valueProcessor = new ExpressionEvaluatingHeaderValueMessageProcessor<>(expression, Integer.class);
valueProcessor.setOverwrite(true);
headersToAdd.put("page", valueProcessor);
return new HeaderEnricher(headersToAdd);
}
}

Spring Integration and RabbitMQ: no output-channel or replyChannel header available while receiving message

I wrote a simple message flow with request and reply. I have to use two independent queues so i declare AmqpOutboundAdapter to send a message and AmqpInboundAdapter to receive a reply.
#Bean
#FindADUsers
public AmqpOutboundEndpoint newFindADUsersOutboundAdapter() {
return Amqp.outboundAdapter(amqpTemplate())
.routingKeyExpression("headers[" + ADUsersFindConfig.ROUTING_KEY_HEADER + "]")
.exchangeName(getExchange())
.headerMapper(amqpHeaderMapper())
.get();
}
#Bean
public AmqpInboundChannelAdapter newFindADUsersResponseInboundChannelAdapter(
ADUsersFindResponseConfig config) {
return Amqp.inboundAdapter(rabbitConnectionFactory(), findADUsersResponseQueue)
.headerMapper(amqpHeaderMapper())
.outputChannel(config.newADUsersFindResponseOutputChannel())
.get();
}
It should work with #MessagingGateway:
#MessagingGateway
public interface ADUsersFindService {
String FIND_AD_USERS_CHANNEL = "adUsersFindChannel";
String FIND_AD_USERS_REPLY_OUTPUT_CHANNEL = "adUsersFindReplyOutputChannel";
String FIND_AD_USERS_REPLY_CHANNEL = "adUsersFindReplyChannel";
String CORRELATION_ID_REQUEST_HEADER = "correlation_id";
String ROUTING_KEY_HEADER = "replyRoutingKey";
String OBJECT_TYPE_HEADER = "object.type";
#Gateway(requestChannel = FIND_AD_USERS_CHANNEL, replyChannel = FIND_AD_USERS_REPLY_CHANNEL)
ADResponse find(ADRequest adRequest, #Header(ROUTING_KEY_HEADER) String routingKey, #Header(OBJECT_TYPE_HEADER) String objectType);
}
And the ADUsersFindResponseConfig class looks like:
#Configuration
#Import(JsonConfig.class)
public class ADUsersFindResponseConfig {
#Autowired
public NullChannel nullChannel;
#Autowired
private JsonObjectMapper<?, ?> mapper;
/**
* #return The output channel for the flow
*/
#Bean(name = ADUsersFindService.FIND_AD_USERS_REPLY_OUTPUT_CHANNEL)
public MessageChannel newADUsersFindResponseOutputChannel() {
return MessageChannels.direct().get();
}
/**
* #return The output channel for gateway
*/
#Bean(name = ADUsersFindService.FIND_AD_USERS_REPLY_CHANNEL)
public MessageChannel newADUsersFindResponseChannel() {
return MessageChannels.direct().get();
}
#Bean
public IntegrationFlow findADUsersResponseFlow() {
return IntegrationFlows
.from(newADUsersFindResponseOutputChannel())
.transform(new JsonToObjectTransformer(ADResponse.class, mapper))
.channel(newADUsersFindResponseChannel())
.get();
}
}
Sending message works properly, but i have a problem with receiving message. I am expecting that received message will be passed to channel called FIND_AD_USERS_REPLY_OUTPUT_CHANNEL, then the message will be deserialized to ADResponse object using findADUsersResponseFlow , and next ADResponse object will be passed to gateway replyChannel - FIND_AD_USERS_REPLY_CHANNEL. Finally, 'find' method return this object. Unfortunately when org.springframework.integration.handler.BridgeHandler receive a message, i got exception:
org.springframework.messaging.MessagingException: ; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available
Message log looks like:
11:51:35.697 [SimpleAsyncTaskExecutor-1] INFO New message - GenericMessage [payload={...somepayload...}, headers={correlation_id=7cbd958e-4b09-4e4c-ba8e-5ba574f3309a, replyRoutingKey=findADUsersResponse.ad, amqp_consumerQueue=findADUsersResponseQueue, history=newFindADUsersResponseInboundChannelAdapter,adUsersFindReplyOutputChannel,adUsersFindReplyChannel,infoLog,infoLoggerChain.channel#0,infoLoggerChain.channel#1, id=37a4735d-6983-d1ad-e0a1-b37dc17e48ef, amqp_consumerTag=amq.ctag-8Qs5YEun1jXYRf85Hu1URA, object.type=USER, timestamp=1469094695697}]
So i'm pretty sure that message was passed to adUsersFindReplyChannel. Also (if it's important) both request message and reply message have 'replyTo' header set to null. What am I doing wrong?

The replyChannel header is a live object and can't be serialized over AMQP.
You can use an outbound gateway instead of the pair of adapters and the framework will take care of the headers.
If you must use adapters for some reason, you need to do 2 things:
Use the header channel registry to convert the channel object to a String which is registered with the registry.
Make sure that the header mapper is configured to send/receive the replyChannel header and that your receiving system returns the header in the reply.