Develop Reference

RabbitMQ Spring Java - RPC Pattern

I have some Spring applications that communicate between them using RabbitMQ as broker. I can send and receive messages asynchronously between them. But now, I need one application to send a message to another one and wait for the response. So, for this I am trying to implement the RPC pattern. It is working, but the problem is that I could only do it using temporary queues generated by Spring.
https://www.rabbitmq.com/tutorials/tutorial-six-spring-amqp.html
This is the code that sends the message and wait for the response.
public void send() {
....
Integer response = (Integer) template.convertSendAndReceive(exchange.getName(), "rpc", "message");
...
}
When I send the message, the execution is blocked until the response is received and a temporary queue is created by Spring for the response, as expected.
But what I need is to use a specific and fixed queue, defined by me, to receive the responses. I need responses to be sent to an exchange with a routing key pointing to the fixed response queue (doing this I'll be able to send the responses to another queue too, that will be logging all responses).
I tried setting the "setReplyTo" property to the message, but is not working.

What version are you using? With modern versions, direct reply_to is used by default, but you can revert to using a temporary queue by setting a property on the template.
https://docs.spring.io/spring-amqp/docs/current/reference/html/#direct-reply-to
To use a named reply queue, see the documentation about how to set up a reply container, with the template as the message listener:
https://docs.spring.io/spring-amqp/docs/current/reference/html/#message-correlation-with-a-reply-queue
and
https://docs.spring.io/spring-amqp/docs/current/reference/html/#reply-listener
EDIT
The template will block until the corresponding reply is passed into it by the reply container (or it times out).
#SpringBootApplication
public class So68986604Application {
public static void main(String[] args) {
SpringApplication.run(So68986604Application.class, args);
}
#RabbitListener(queues = "foo")
public String listen(String in) {
System.out.println(in);
return in.toUpperCase();
}
#Bean
Queue foo() {
return new Queue("foo");
}
#Bean
Queue replies() {
return new Queue("foo.replies");
}
#Bean
SimpleMessageListenerContainer replyContainer(ConnectionFactory cf, RabbitTemplate template) {
SimpleMessageListenerContainer replyer = new SimpleMessageListenerContainer(cf);
replyer.setQueueNames("foo.replies");
replyer.setMessageListener(template);
template.setReplyAddress("foo.replies");
return replyer;
}
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
return args -> {
System.out.println(template.convertSendAndReceive("", "foo", "test"));
};
}
}
test
TEST

Handling exception that might occur in an integration flow

I have a REST controller which calls a gateway annotated with #MessagingGateway(errorChannel = ERROR_CHANNEL)
This way, whatever error occurs downstream the integration flow initiated by the gateway will flow into an error channel which will be handled by another integration flow, this is working as expected.
Now, there is another scenario where an integration flow reads messages from Kafka, routes those messages to another channel, one more integration flow processes those messages and another flow sends an HTTP request to a remote service.
public IntegrationFlowBuilder attachmentEventTenantRouter(String tenantId) {
return attachmentEventBaseFlow(".*")
.filter(Message.class, m -> m.getHeaders().get(KafkaConstants.HEADER_PREFIX + MessageHeader.TENANT_ID_KEY) != null && m.getHeaders().get(KafkaConstants.HEADER_PREFIX + MessageHeader.TENANT_ID_KEY, String.class).equalsIgnoreCase(tenantId));
}
private IntegrationFlowBuilder attachmentEventBaseFlow(String eventRegex) {
return IntegrationFlows
.from(Kafka.messageDrivenChannelAdapter(kafkaListenerContainerFactory.createContainer(topic)).errorChannel(IntegrationContextUtils.ERROR_CHANNEL_BEAN_NAME))
.log(LoggingHandler.Level.DEBUG, "Inside Kafka Consumer")
.filter(Message.class, m -> filter(m, eventRegex))
.transform(messageToEventTransformer);
}
#Bean
public IntegrationFlow kafkaConsumerFlow() {
return fromKafkaFlowHelper.attachmentEventTenantRouter(TENANT_ID)
.route(Message.class, m -> m.getHeaders().get(KafkaConstants.HEADER_PREFIX + MessageHeader.EVENT_TYPE_KEY, String.class), p -> p
.resolutionRequired(false)
.channelMapping("eventType", "transformMessagesFromKafkaAndPublishAnotherEvent")
.defaultOutputChannel("nullChannel"))
.get();
}
#Bean
public IntegrationFlow transformMessagesFromKafkaAndPublishAnotherEvent() {
return flow -> flow
.transform(transformer)
.handle( getKafkaHandler() );
}
#Bean
public IntegrationFlow sendHttpRequestToRemoteServiceFromKafkaEvent() {
return flow -> flow
.transform(transformer)
.handle(gatewayCall, e -> e.advice(expressionAdvice()));
}
How can I do to handle the exceptions that might occur in the flows above?
As you can see, I am using a ExpressionEvaluatingRequestHandlerAdvice which does the work for the handle method, but not sure how to handle exceptions that might occur in the transformers?
The massage gateway with an error channel configured does the trick when the gateway is called by a rest controller, but when the flow is initiated by the Kafka consumer, I'm lost how to achieve this.
Thanks.
EDITED AFTER Artem's RESPONSE TO ADD CLARIFICATION:
This is the configuration of the integration flow that posts a request to a remote service and whose exceptions does not seem to be caught and routed to the errorChannel without a ExpressionEvaluatingRequestHandlerAdvice:
#Bean
public IntegrationFlow sendHttpRequestToRemoteServiceFromKafkaEvent() {
return flow -> flow
.transform(transformer)
.handle(getOAuth2Handler(HttpMethod.PUT, "remote url"), e -> e.advice(expressionEvaluatingRequestHandlerAdvice));
}
private OAuth2RequestHandler getOAuth2Handler(HttpMethod httpMethod, String url) {
return new OAuth2RequestHandler(oAuth2RestTemplate, httpMethod, url);
}
And class OAuth2RequestHandler which implements a MessageHandler
#Override
public void handleMessage(org.springframework.messaging.Message<?> message) throws MessagingException {
String requestBody = (String) message.getPayload();
ResponseEntity<String> response = oAuth2RestTemplate.exchange(url, httpMethod, new HttpEntity<>(requestBody), String.class);
}

I see you use already an errorChannel() on the Kafka message-driven channel adapter. So, what is the question?
This part of your flow is exact equivalent to mentioned #MesaagingGateway with its errorChannel configuration.

How do I write a spring boot rabbitmq sender for a python receiver?

I want to write an application wherein I need a producer sending messages using spring boot rabbitmq and the receiver is written in python. The receiver part was easy-
receive.py
#!/usr/bin/env python
import pika
import sys
connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
channel = connection.channel()
channel.exchange_declare(exchange='topic_logs',
exchange_type='topic')
result = channel.queue_declare(exclusive=True)
queue_name = result.method.queue
binding_keys = sys.argv[1:]
if not binding_keys:
sys.stderr.write("Usage: %s [binding_key]...\n" % sys.argv[0])
sys.exit(1)
for binding_key in binding_keys:
channel.queue_bind(exchange='topic_logs',
queue=queue_name,
routing_key=binding_key)
print(' [*] Waiting for logs. To exit press CTRL+C')
def callback(ch, method, properties, body):
print(" [x] %r:%r" % (method.routing_key, body))
channel.basic_consume(callback,
queue=queue_name,
no_ack=True)
channel.start_consuming()
How to write a spring boot rabbitmq sender code for this? What are the necessary things to be declared there? Please help.

Spring Boot is Java application, first of all.
So, you need to make yourself familiar with that language.
And the choice is correct: you really can send to the RabbitMQ from Java and receive from the queue in any other client.
Spring Boot provides for you RabbitTemplate bean. So, if the story is about to send you just need to inject such a bean and use its API to send:
#Autowired
RabbitTemplate rabbitTemplate;
...
this.rabbitTemplate.convertAndSend("topic_logs", binding_key, data);
See Reference Manual for more info.

With spring integration, first you need to create a configuration class:
#Configuration
public class RabbitConfig {
#Autowired
private ConnectionFactory connectionFactory;
#Bean
DirectExchange dropfileExchange() {
return new DirectExchange("exchange_name", true, false);
}
#Bean
public Queue dropfileQueue() {
return new Queue("queue_name", true);
}
#Bean
Binding dropfileExchangeBinding(DirectExchange dropfileExchange, Queue dropfileQueue) {
return BindingBuilder.bind(dropfileQueue).to(dropfileExchange).with("key_name");
}
#Bean
public RabbitTemplate dropfileExchangeTemplate() {
RabbitTemplate rt = new RabbitTemplate(connectionFactory);
rt.setExchange("exchange_name");
rt.setRoutingKey("key_name");
rt.setConnectionFactory(connectionFactory);
return rt;
}
#Bean
public RabbitMessagingTemplate rabbitMessagingTemplate() {
return new RabbitMessagingTemplate(dropfileExchangeTemplate());
}
}
Then create a gateway service:
#MessagingGateway
public interface DropfileMessageGateway {
#Gateway(requestChannel = "channel_name")
void generate(String payload);
}
And then specify the producer flow using Java DSL as follows:
#Bean
public IntegrationFlow toOutboundQueueFlow() {
return IntegrationFlows.from("channel_name")
.transform(Transformers.toJson())
.handle(Amqp.outboundAdapter(rabbitConfig.dropfileExchangeTemplate())).get();
}
That will read a message from the channel, transform it to JSON and then dispatch it using an outbound adapter to the rabbit exchange.
Note that the messages go to the channel via the messaging gateway, so the channel name in both should be the same.
Do not forget to include the appropriate dependencies.

Spring integration error channel when using PollableChannel

We're using Spring integration in my application. I'd like put some objects into channel for asynchronous processing and error handling. So for this, I configured MessageGateway with error channel and PollableChannel for handling objects to be processed.
The problem
So I'm calling messageGateway.processMessage(message) to put message into channel. This works as expected - calling this method is non-blocking, messages get processed and are forwarded to next channel. But when processing method throws an exception, it is not redirected to error channel.
On the other hand when I change my processing channel from PollableChannel to SubscribableChannel, error channel works as expected, but calling the gateway is of course blocking. What am I missing? Can I have both non blocking call and error channel?
The code
Component doing the message processing:
#Component
public MessageProcessor {
#Transactional
#ServiceActivator(inputChannel = "msg.process", outputChannel = "msg.postprocess")
public void processMessage(MyMessage message) {
// Message processing that may throw exception
}
}
Channel definition:
#Configuration
public class IntegrationConfig {
#Bean(name = "msg.process")
private MessageChannel processChannel() {
return new RendezvousChannel();
}
#Bean(name = "msg.error")
private MessageChannel errorChannel() {
return new DirectChannel();
}
}
My gateway looks like this:
#MessagingGateway(errorChannel = "msg.error")
public interface MessagingGateway {
#Gateway(requestChannel = "msg.processing")
void processMessage(MyMessage message);
}
Error handler:
#Component
public ErrorHandlers {
#Transactional
#ServiceActivator(inputChannel = "msg.error")
public void processError(MessagingException me) {
// Error handling is here
}
}

But when processing method throws an exception, it is not redirected to error channel.
When a gateway method returns void, the calling thread is released immediately when it returns to the gateway.
The gateway does not add an error channel header in this case (in the next release - 5.0) we have changed that.
In the meantime, you can use a header enricher to set the errorChannel header to your error channel. You can also use the defaultHeaders property on the #MessagingGateway - see the comments on this answer.

Spring Integration Java DSL - Set RecepientListRouter recipients dynamically?

I have a method which needs to execute multiple tasks async'ly.. I've managed to achieve that using the following IntegrationFlow:
#Bean
public IntegrationFlow startJobTask() {
return IntegrationFlows.from("TaskRoutingChannel")
.handle("jobService", "executeTasks")
.routeToRecipients(r -> r
.recipient("testTaskChannel")
.recipient("test2TaskChannel"))
.get();
}
#Bean ExecutorChannel testTaskChannel(){
return new ExecutorChannel(this.getAsyncExecutor());
}
#Bean ExecutorChannel test2TaskChannel(){
return new ExecutorChannel(this.getAsyncExecutor());
}
#Bean
public Executor getAsyncExecutor() {
SimpleAsyncTaskExecutor executor = new SimpleAsyncTaskExecutor();
return executor;
}
#Bean
public IntegrationFlow testTaskFlow() {
return IntegrationFlows.from("testTaskChannel")
.handle("testTaskService", "executeAsync")
.get();
}
#Bean
public IntegrationFlow test2TaskFlow() {
return IntegrationFlows.from("test2TaskChannel")
.handle("test2TaskService", "executeAsync")
.get();
}
The above flow is basically as follows:
TaskRoutingChannel calls a serviceActivator method executeTasks
executeTasks returns something like Message<List<myTask>>
this Message is routed to channels testTaskChannel and test2TaskChannel
which calls their own async serviceActivator methods.
Now, the issue is that I don't want to hardcode the recipient channels.
I could avoid hardcoding with normal router by setting the destination channel as a header. However, recepientListRouters don't seem to have the capability to get recipient channel name using expressions.
Is there any way to set the recipient channels dynamically?

Sorry for delay first of all.
Actually the regular .route() can do that for you, because it has exactly has this option:
protected abstract Collection<MessageChannel> determineTargetChannels(Message<?> message);
So, if your header extraction returns the list of channels, the HeaderValueRouter will be able to the message to all of them.