Develop Reference

How can we have different handlers to subscribe messages from different topics using spring Integration?

I wrote my first spring integration application which uses the mqtt broker to subscribe messages from different topics which are coming from a device. The device is publishing the messages and the client(Code) is accessing those messages using same topics.
I added a handler for accessing the messages coming from the broker and use it further in classes. Now, in my case, I want to have different handlers for different topics so that they can all be mapped to different VO classes and use it further in business logic.
As I know, I want to create only one connection to the broker, one channel but different topics can come and they should be handled in different handlers for the same connection. How Can I achieve that?
#SpringBootApplication
public class MqttJavaApplication {
public static void main(String[] args) {
// SpringApplicationBuilder springApplicationBuilder = new SpringApplicationBuilder(MqttJavaApplication.class);
SpringApplication.run(MqttJavaApplication.class,args);
}
#Bean
public MessageChannel mqttInputChannel() {
return new DirectChannel();
}
#Bean
public MqttPahoMessageDrivenChannelAdapter inbound() {
String clientId = "uuid-" + UUID.randomUUID().toString();
MqttPahoMessageDrivenChannelAdapter adapter =
new MqttPahoMessageDrivenChannelAdapter("tcp://localhost:1883", clientId,"camera/status");
adapter.setCompletionTimeout(5000);
adapter.setConverter(new DefaultPahoMessageConverter());
adapter.setQos(1);
// adapter.setOutputChannelName("mqttInputChannel");
adapter.setOutputChannel(mqttInputChannel());
return adapter;
}
#Bean
public IntegrationFlow mqttInFlow() {
System.out.println(Arrays.stream(SubScribeMessages.class.getMethods()).findFirst());
return IntegrationFlows.from(inbound())
.transform(p -> p)
.handle("addTopics","handlHere")
.get();
}
#Component
public class MyService{
#Autowired
MqttPahoMessageDrivenChannelAdapter adapter;
#Bean
public String addTopics()
{
if(adapter.getTopic().length>0)
{
adapter.addTopic("camera/+/counts"); //new topic
adapter.addTopic("camera/+/live_counts"); //new topic
}
return "";
}
// topic "camera/+/counts" is handled here but other messages also come here, how do we handle other topics in separate handlers?
#ServiceActivator(inputChannel = "mqttInputChannel")
public void handleHere(#Payload Object mess) throws JsonProcessingException {
String[] topics = adapter.getTopic();
for(String topic:topics)
System.out.println(topic); // How can I get topic name which is using a wildcard?
ObjectMapper objectMapper = new ObjectMapper();
String json=mess.toString();
System.out.println(json);
CountVo countVo = objectMapper.readValue(json, CountVo.class);
if (!countVo.equals(null))
System.out.println(countVo.getIrisysCounts().get(0).getName());
}
}
}
Additional Question
How Can I get the full topic name when using a wildcard? The actual topic which was published but caught by wildcard.
Please help.

Add a router (.route(...)); you can route on the MqttHeaders.RECEIVED_TOPIC header (which contains the topic name) to different flows for each topic.
https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#messaging-routing-chapter
EDIT
The simplest router is to simply map the topic names to channel names. Here is an example:
#SpringBootApplication
public class So67391175Application {
public static void main(String[] args) {
SpringApplication.run(So67391175Application.class, args);
}
#Bean
public DefaultMqttPahoClientFactory pahoClientFactory() {
DefaultMqttPahoClientFactory pahoClientFactory = new DefaultMqttPahoClientFactory();
MqttConnectOptions connectionOptions = new MqttConnectOptions();
connectionOptions.setServerURIs(new String[] { "tcp://localhost:1883" });
pahoClientFactory.setConnectionOptions(connectionOptions);
return pahoClientFactory;
}
#Bean
public IntegrationFlow mqttInFlow(DefaultMqttPahoClientFactory pahoClientFactory) {
return IntegrationFlows.from(
new MqttPahoMessageDrivenChannelAdapter("testClient",
pahoClientFactory, "topic1", "topic2"))
.route("headers['" + MqttHeaders.RECEIVED_TOPIC + "']")
.get();
}
#Bean
public IntegrationFlow flow1() {
return IntegrationFlows.from("topic1")
.handle((payload, headers) -> {
System.out.println("message from topic1 " + payload + ": " + headers);
return null;
})
.get();
}
#Bean
public IntegrationFlow flow2() {
return IntegrationFlows.from("topic2")
.handle((payload, headers) -> {
System.out.println("message from topic2 " + payload + ": " + headers);
return null;
})
.get();
}
}
message from topic1 test: {mqtt_receivedRetained=false, mqtt_id=1, mqtt_duplicate=false, id=1d950bce-aa47-7e3b-1a0d-e4d01ed707de, mqtt_receivedTopic=topic1, mqtt_receivedQos=1, timestamp=1620250633090}
message from topic2 test: {mqtt_receivedRetained=false, mqtt_id=2, mqtt_duplicate=false, id=7e9c3f51-c148-2b18-3588-ed27e93dae19, mqtt_receivedTopic=topic2, mqtt_receivedQos=1, timestamp=1620250644602}

Thanks Gary! I think the answer you gave on routing can only take the defined topics, not on wildcards or for that matter any other regex. I could not understand how dynamic routing would help me.
Turns out, I can add wildcards when initializing bean and can handle using the service activator on the inputchannel using the adapter.
Like this:
#SpringBootApplication
public class MqttJavaApplication {
public static void main(String[] args) {
SpringApplication.run(MqttJavaApplication.class,args);
}
#Bean
public MessageChannel mqttInputChannel() {
return new DirectChannel();
}
#Bean
public MqttPahoMessageDrivenChannelAdapter inbound() {
String clientId = "uuid-" + UUID.randomUUID().toString();
MqttPahoMessageDrivenChannelAdapter adapter =
new MqttPahoMessageDrivenChannelAdapter("tcp://localhost:1883", clientId, "irisys/V4D-20230143/status" );
adapter.setCompletionTimeout(5000);
adapter.setConverter(new DefaultPahoMessageConverter());
adapter.setQos(1);
adapter.setOutputChannel(mqttInputChannel());
return adapter;
}
#Component
public class MyService{
#Autowired
MqttPahoMessageDrivenChannelAdapter adapter;
#Bean
public String addTopics()
{
if(adapter.getTopic().length>0)
{
adapter.addTopic("camera/+/counts");
}
return "";
}
#Bean
#ServiceActivator(inputChannel = "mqttInputChannel")
public MessageHandler handler() {
return new MessageHandler() {
#SneakyThrows
#Override
public void handleMessage(Message<?> message) throws MessagingException {
System.out.println(message.getPayload());
System.out.println(message.getHeaders());
if(message.getHeaders().get("mqtt_receivedTopic").toString().contains("counts"))
{
ObjectMapper objectMapper = new ObjectMapper();
String json=message.getPayload().toString();
System.out.println(json);
CountVo countVo = objectMapper.readValue(json, CountVo.class);
if (!countVo.equals(null))
System.out.println(countVo.getIrisysCounts().get(0).getName());
}
}
};
}
}
}
Do you think that there is better way than this? I couldn't think of anything other than this.

Content type different in RabbitListener Spring AMQP using ContentTypeDelegatingMessageConverter

Actually I'm using Spring AMQP to consume XML from Rabbit.
Here is my code for Listen the queue.
#RabbitListener(queues = DealerReceiverConfig.P8_QUEUE_NAME, id = Constants.P8_QUEUE_ID)
#SendTo("foo.bar")
public RequestDocument p8ContentReceiveMessage(RequestDocument request) {
System.out.println(request.getCorrelationId());
return request;
}
and my rabbit template configuration is:
#Override
#Bean(name = "dealerRabbit")
public RabbitTemplate rabbitTemplateSeguros(ConnectionFactory connectionFactory) {
RabbitTemplate template = new RabbitTemplate(connectionFactory);
template.setMessageConverter(producerMessageConverter());
return template;
}
#Override
#Bean
public MessageConverter producerMessageConverter() {
ContentTypeDelegatingMessageConverter messageConverter = new ContentTypeDelegatingMessageConverter();
Jackson2JsonMessageConverter jsonMessageConverter = new Jackson2JsonMessageConverter();
messageConverter.addDelegate("application/json", jsonMessageConverter);
MarshallingMessageConverter marshaller = new MarshallingMessageConverter();
marshaller.setMarshaller(oxmMarshaller());
marshaller.setUnmarshaller(oxmUnMarshaller());
messageConverter.addDelegate("application/xml", marshaller);
return messageConverter;
}
#Bean
public Marshaller oxmMarshaller() {
Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
marshaller.setPackagesToScan("foo.bar.model");
return marshaller;
}
#Bean
public Unmarshaller oxmUnMarshaller() {
Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
marshaller.setPackagesToScan("foo.bar.model");
return marshaller;
}
The problem is when I see the returned message in the #SentTo queue configured, I get the following message.
Returned message
Any suggestions?
Thanks for your help.

The template is not used to send replies from #RabbitListeners; you need to add the converter to the RabbitListenerContainerFactory (Simple... or Direct..., depending on which container type you are using).
EDIT
It's a bug; you should be able to do this...
#RabbitListener(queues = "foo")
#SendTo("bar")
public Message<String> listen(String in) {
System.out.println(in);
return MessageBuilder.withPayload(in.toUpperCase())
.setHeader(MessageHeaders.CONTENT_TYPE, "application/xml")
.build();
}
But the adapter still uses a raw MessageProperties which has application/x-java-serialized-object as the default CT.
https://github.com/spring-projects/spring-amqp/issues/1219

Handle the exceptions in RabbitMQ Spring Boot Application

I am using Spring Boot 1.4.1-RELEASE and RabbitMQ 3.2.3. My Application class looks like this -
#SpringBootApplication
#EnableAutoConfiguration
public class EventStoreMessageDeliveryApplication {
public final static String queueName = "customer.default.queue"; // spring-boot
#Bean
Queue queue() {
return new Queue(queueName, true);
}
#Bean
FanoutExchange exchange() {
return new FanoutExchange("customer.events.fanout.exchange", true, false); // spring-boot-exchange
}
#Bean
Binding binding() {
return new Binding(queueName, Binding.DestinationType.QUEUE, "customer.events.fanout.exchange", "*.*", null);
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory("localhost");
connectionFactory.setPublisherConfirms(true);
return connectionFactory;
}
#Bean
SimpleMessageListenerContainer container(MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory());
container.setQueueNames(queueName);
container.setMessageListener(listenerAdapter);
container.setRecoveryBackOff(new ExponentialBackOff(3000, 2));
return container;
}
#Bean
MessageListenerAdapter listenerAdapter(Receiver receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
public static void main(String[] args) throws InterruptedException {
SpringApplication.run(EventStoreMessageDeliveryApplication.class, args);
}
}
And my listener class looks like -
#Component
public class Receiver {
private CountDownLatch latch = new CountDownLatch(1);
public void receiveMessage(String message) {
System.out.println("Received <" + message + ">");
// do something
latch.countDown();
}
public CountDownLatch getLatch() {
return latch;
}
}
I want to handle the exceptions like connection refused which may come when the broker is down. How can I handle such exceptions? I am not sure where I can get the handle for the exceptions.

You can create a SimpleRabbitListenerContainerFactory. This is basically a listener for events from RabbitConnectionFactory.
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setErrorHandler(rabbitErrorHandler());
return factory;
}
rabbitErrorHandler() can return a bean of implementation of org.springframework.util.ErrorHandler.
Reference docs

I have a suggestion and it could work out. Since you want to have an exception of connection refused against the RabbitMQ broker, it is up to the client to catch the exception.
In your example, which looks like the one from SpringIO docs, I would assume you could make the exception handling in the main (not recommended though):
#Component
public class Runner implements CommandLineRunner {
private final RabbitTemplate rabbitTemplate;
private final Receiver receiver;
public Runner(Receiver receiver, RabbitTemplate rabbitTemplate) {
this.receiver = receiver;
this.rabbitTemplate = rabbitTemplate;
}
#Override
public void run(String... args) throws Exception {
System.out.println("Sending message...");
try {
rabbitTemplate.convertAndSend(Application.topicExchangeName, "foo.bar.baz", "Hello from RabbitMQ!");
receiver.getLatch().await(10000, TimeUnit.MILLISECONDS);
}catch(AmqpException the_exception) {
System.out.printl("Connection refused. Problem thrown when trying to connecto the RabbitMQ");
}
}
}
The AmqpException comes from the docs of the convertAndSend() method, which is being thrown if something went bad. Here you can capture your own custom message.
I hope this is what you are looking for or atleast guides you the correct destination.
/A

ERR Timed out after 1m0s: health check never passed

I am getting this exception after some 2-3 minutes of proper run of my app on cloud.I added timeout : 180 in manifest.yml file but the error still persists.
On STS console, I get the log as :
Execution failed for task ':cfPush'.> Application TestApp start timed out
Can anyone help me out in this please
Code:
#SpringBootApplication
public class SftpJavaApplication {
public static void main(String[] args) {
new SpringApplicationBuilder(SftpJavaApplication.class)
.web(false)
.run(args);
}
#Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost("ip");
factory.setPort(port);
factory.setUser("user");
factory.setPassword("pwd");
factory.setAllowUnknownKeys(true);
return new CachingSessionFactory<LsEntry>(factory);
}
#Bean
#Transformer(inputChannel = "stream",outputChannel="data")
public org.springframework.integration.transformer.Transformer transformer () {
return new org.springframework.integration.transformer.StreamTransformer("UTF-8");
}
#Bean
#InboundChannelAdapter(value = "stream", poller = #Poller(fixedDelay = "1000", maxMessagesPerPoll = "1"))
public MessageSource<InputStream> ftpMessageSource() {
SftpStreamingMessageSource messageSource = new SftpStreamingMessageSource(template(), null);
messageSource.setRemoteDirectory("/test1/test2/test3");
messageSource.setFilter(new SftpPersistentAcceptOnceFileListFilter(new SimpleMetadataStore(),
"streaming"));
return messageSource;
}
#Bean
public SftpRemoteFileTemplate template() {
return new SftpRemoteFileTemplate(sftpSessionFactory());
}
#Bean
#ServiceActivator(inputChannel = "data" )
public MessageHandler handler() {
return new MessageHandler() {
#Override
public void handleMessage(Message<?> message) throws MessagingException {
System.out.println(">>>>>>>>>>>>>"+message.getPayload()); //instead of a print the message is published to rabbit mq
}
};
}
}
There is another class which connects to a rabbit mq service on cloud and consumes the message

Spring and AMQP RabbitMQ topic exchange not working

I'm trying to set up topic exchange on my spring app.
Here's my context configuration:
#Configuration
public class IntegrationConfig {
public final static String queueName = "my-queue";
#Bean
AmqpAdmin amqpAdmin(ConnectionFactory connectionFactory) {
return new RabbitAdmin(connectionFactory);
}
#Bean
Queue queue() {
return new Queue(queueName);
}
#Bean
TopicExchange exchange() {
return new TopicExchange("my-exchange", false, true);
}
#Bean
Binding binding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with("ru.interosite.*");
}
#Bean
SimpleMessageListenerContainer container(ConnectionFactory connectionFactory, MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(queueName);
container.setMessageListener(listenerAdapter);
return container;
}
#Bean
ImageUploadReceiver receiver() {
return new ImageUploadReceiver();
}
#Bean
MessageListenerAdapter listenerAdapter(ImageUploadReceiver receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
}
This is receiver class:
public class ImageUploadReceiver {
private CountDownLatch latch = new CountDownLatch(1);
public void receiveMessage(String message) {
System.out.println("Received ");
latch.countDown();
}
public CountDownLatch getLatch() {
return latch;
}
}
This is sender code:
#RequestMapping("/sendmessage")
#ResponseBody
public String sendMessage() {
rabbitTemplate.convertAndSend("ru.interosite.1", "ttt1233");
try {
imageUploadReceiver.getLatch().await(3, TimeUnit.SECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
return "Msg received";
}
So I'm sending message to topic exchange using binding key "ru.interosite.1" to the queue that was bound with pattern "ru.interosite.*". I used these key and pattern when tried sample from https://www.rabbitmq.com/tutorials/tutorial-five-java.html and they worked fine.
But inside String AMQP it does not work, i.e. receiver never gets called. It called only if binding key and pattern are completely the same as if I were using DirectExchange.
Am I missing something here?

You don't show the config for the RabbitTemplate, but I guess it is with default options.
To send a message to the my-exchange you must specify it directly:
rabbitTemplate.convertAndSend("my-exchange", "ru.interosite.1", "ttt1233");

You can also set the exchange in the rabbit template like this:
#Configuration
public class IntegrationConfig {
// ... as above
#Bean
public RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
RabbitTemplate template = new RabbitTemplate(connectionFactory);
template.setExchange("my-exchange");
return template;
}
}
So you can send the message with:
public class MyController {
#Autowired
RabbitTemplate rabbitTemplate;
#RequestMapping("/sendmessage")
#ResponseBody
public String sendMessage() {
rabbitTemplate.convertAndSend("ru.interosite.1", "ttt1233");
// ... as above
}
}