I create the IntegrationFlows in DSL:
return IntegrationFlows
.from(yieldCurveConversionResultChannel)
.wireTap(notificationChannel, wt -> wt.selector(m -> (m.getPayload() instanceof Throwable)))
.get();
In the selector of wireTap, I tried to filter the message payload is instance of Exception.
There is also a Flow to handle notification service.
return IntegrationFlows
.from(notificationChannel)
.handle(commonNotificationService)
.get();
But it seems doesn't work. I could get the message which payload isn't instance of Exception.
Did anyone meet this issue before? Or I get something wrong?
Thank you so much.
Works well:
#SpringBootApplication
public class So66813194Application {
public static void main(String[] args) {
SpringApplication.run(So66813194Application.class, args);
}
#Bean
IntegrationFlow flow() {
return IntegrationFlows
.from("yieldCurveConversionResultChannel")
.wireTap(notificationChannel(), wt -> wt.selector(m -> (m.getPayload() instanceof Throwable)))
.get();
}
#Bean
MessageChannel notificationChannel() {
return new DirectChannel();
}
#Bean
IntegrationFlow notification() {
return IntegrationFlows
.from(notificationChannel())
.handle(System.out::println)
.get();
}
}
#SpringBootTest
class So66813194ApplicationTests {
#Autowired
MessageChannel yieldCurveConversionResultChannel;
#Test
void contextLoads() {
this.yieldCurveConversionResultChannel.send(new GenericMessage<>("foo"));
this.yieldCurveConversionResultChannel.send(new GenericMessage<>(new RuntimeException()));
}
}
And I see in the console only this:
2021-03-26 09:36:25.367 INFO 15164 --- [ main] o.s.i.s.s.So66813194ApplicationTests : Started So66813194ApplicationTests in 1.229 seconds (JVM running for 2.365)
GenericMessage [payload=java.lang.RuntimeException, headers={id=85633b3f-7a17-6b70-8a34-9cbc5aacbf5b, timestamp=1616765785796}]
2021-03-26 09:36:25.810 INFO 15164 --- [extShutdownHook] o.s.i.endpoint.EventDrivenConsumer : Removing {bridge} as a subscriber to the 'yieldCurveConversionResultChannel' channel
So, that foo message from my tests is really rejected by the selector.
Perhaps your problem that some process sends messages to this notificationChannel directly - not via the mentioned WireTap...
Related
I'm working on an apllication that sends message to as server, then given message is modified and sent back to the amq.rabbitmq.reply-to queue using Direct Reply-to . I've followed the the tutorial https://www.rabbitmq.com/direct-reply-to.html but I have some problems to implement it. In my case as I've understood I need to consume message from pseudo-queue amq.rabbitmq.reply-to in no-ack mode, Which in my case is MessageListenerContainer. Here's my config:
#Bean
public Jackson2JsonMessageConverter messageConverter() {
ObjectMapper mapper = new ObjectMapper();
return new Jackson2JsonMessageConverter(mapper);
}
#Bean
public RabbitAdmin rabbitAdmin(ConnectionFactory connectionFactory) {
return new RabbitAdmin(connectionFactory);
}
#Bean
public RabbitTemplate rabbitTemplate(final ConnectionFactory connectionFactory) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(messageConverter());
rabbitTemplate.setReplyAddress("amq.rabbitmq.reply-to");
return rabbitTemplate;
}
#Bean
MessageListenerContainer messageListenerContainer(ConnectionFactory connectionFactory ) {
DirectMessageListenerContainer directMessageListenerContainer = new DirectMessageListenerContainer();
directMessageListenerContainer.setConnectionFactory(connectionFactory);
directMessageListenerContainer.setAcknowledgeMode(AcknowledgeMode.NONE);
directMessageListenerContainer.setQueueNames("amq.rabbitmq.reply-to");
directMessageListenerContainer.setMessageListener(new PracticalMessageListener());
return directMessageListenerContainer;
}
Message is sent as JSON through the SEND frame on STOM protocol and converted. Then a new queue
is created dynamically and added to the MessageListenerContainer. So when the message arrives in the broker, I would like to modify it on the server side and send back to amq.rabbitmq.reply-to and original message to be sent to routing key messageTemp.getTo() which is subscribed on the SUBSCRIBE frame in STOMP.
#MessageMapping("/private")
public void send2(MessageTemplate messageTemp) throws Exception {
MessageTemplate privateMessage = new MessageTemplate(messageTemp.getPerson(),
messageTemp.getMessage(),
messageTemp.getTo());
AbstractMessageListenerContainer abstractMessageListenerContainer =
(AbstractMessageListenerContainer) mlc;
// here's the queue added to listener container
abstractMessageListenerContainer.addQueueNames(messageTemp.getTo());
MessageProperties mp = new MessageProperties();
mp.setReplyTo("amq.rabbitmq.reply-to");
mp.setCorrelationId("someId");
Jackson2JsonMessageConverter smc = new Jackson2JsonMessageConverter();
Message message = smc.toMessage(messageTemp, mp);
rabbitTemplate.sendAndReceive(
messageTemp.getTo() , message);
}
Message is modified onMessage method when message sent to messageTemp.getTo() routing key
#Component
public class PracticalMessageListener implements MessageListener {
#Autowired
RabbitTemplate rabbitTemplate;
#Override
public void onMessage(Message message) {
System.out.println(("message listener.."));
String body = "{ \"processing\": \"123456789\"}";
MessageProperties properties = new MessageProperties();
// some business logic on the message body
properties.setCorrelationId(message.getMessageProperties().getCorrelationId());
Message responseMessage = new Message(body.getBytes(), properties);
rabbitTemplate.convertAndSend("",
message.getMessageProperties().getReplyTo(), responseMessage);
}
I may misunderstand the concept of direct-reply and the documentation that says:
Consume from the pseudo-queue amq.rabbitmq.reply-to in no-ack mode. There is no need to declare this "queue" first, although the client can do so if it wants.
The question is where I need to consume from that queue? And how Can I access that modified message if I'm getting error:
2020-01-15 22:17:09.688 WARN 96222 --- [pool-1-thread-5] s.a.r.l.ConditionalRejectingErrorHandler : Execution of Rabbit message listener failed.
org.springframework.amqp.rabbit.support.ListenerExecutionFailedException: Listener threw exception
Caused by: java.lang.NullPointerException: null
at com.patrykmaryn.spring.second.PracticalMessageListener.onMessage(PracticalMessageListener.java:50) ~[classes/:na]
Which is coming from the place when I invoke rabbitTemplate.convertAndSend in PracticalMessageListener
EDIT
I got rid of setting amq.rabbitmq.reply-to in the DirectMessageListenerContainer and implemented DirectReplyToMessageListenerContainer:
#Bean
DirectReplyToMessageListenerContainer drtmlc (ConnectionFactory connectionFactory) {
DirectReplyToMessageListenerContainer drtmlc =
new DirectReplyToMessageListenerContainer(connectionFactory);
drtmlc.setConnectionFactory(connectionFactory);
drtmlc.setAcknowledgeMode(AcknowledgeMode.NONE);
drtmlc.setMessageListener(new DirectMessageListener());
return drtmlc;
}
The problem must be in onMessage method that doesn't allow to invoke any send method on rabbitTemplate, I've tried with different existing routing keys and exchanges. The listening is coming from queue defined with routing key messageTemp.getTo().
#Override
public void onMessage(Message message) {
System.out.println(("message listener.."));
String receivedRoutingKey = message.getMessageProperties()
.getReceivedRoutingKey();
System.out.println(" This is received routingkey: " +
receivedRoutingKey);
/// ..... rest of code goes here
rabbitTemplate.convertAndSend("",
message.getMessageProperties().getReplyTo(),
responseMessage);
Where messageTemp.getTo() is routing key defined at runtime, by selecting a receiver e.g if i select 'user1' it will print out 'user1'.
That's the first attempt to send message:
2020-01-16 02:22:20.213 DEBUG 28490 --- [nboundChannel-6] .WebSocketAnnotationMethodMessageHandler : Searching methods to handle SEND /app/private session=45yca5sy, lookupDestination='/private'
2020-01-16 02:22:20.214 DEBUG 28490 --- [nboundChannel-6] .WebSocketAnnotationMethodMessageHandler : Invoking PracticalTipSender#send2[1 args]
2020-01-16 02:22:20.239 INFO 28490 --- [nboundChannel-6] o.s.a.r.l.DirectMessageListenerContainer : SimpleConsumer [queue=user1, consumerTag=amq.ctag-Evyiweew4C-K1mXmy2XqUQ identity=57b19488] started
2020-01-16 02:22:20.268 INFO 28490 --- [nboundChannel-6] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService
2020-01-16 02:22:20.269 INFO 28490 --- [nboundChannel-6] .l.DirectReplyToMessageListenerContainer : Container initialized for queues: [amq.rabbitmq.reply-to]
2020-01-16 02:22:20.286 INFO 28490 --- [nboundChannel-6] .l.DirectReplyToMessageListenerContainer : SimpleConsumer [queue=amq.rabbitmq.reply-to, consumerTag=amq.ctag-IXWf-zEyI34xzQSSfbijzg identity=4bedbba5] started
And second that fails:
2020-01-16 02:23:20.247 DEBUG 28490 --- [nboundChannel-3] .WebSocketAnnotationMethodMessageHandler : Searching methods to handle SEND /app/private session=45yca5sy, lookupDestination='/private'
2020-01-16 02:23:20.248 DEBUG 28490 --- [nboundChannel-3] .WebSocketAnnotationMethodMessageHandler : Invoking PracticalTipSender#send2[1 args]
2020-01-16 02:23:20.248 WARN 28490 --- [nboundChannel-3] o.s.a.r.l.DirectMessageListenerContainer : Queue user1 is already configured for this container: org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer#3b152928, ignoring add
2020-01-16 02:23:20.250 WARN 28490 --- [nboundChannel-3] o.s.a.r.l.DirectMessageListenerContainer : Queue user1 is already configured for this container: org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer#3b152928, ignoring add
message listener..
This is received routingkey: user1
2020-01-16 02:23:20.271 WARN 28490 --- [pool-1-thread-5] s.a.r.l.ConditionalRejectingErrorHandler : Execution of Rabbit message listener failed.
org.springframework.amqp.rabbit.support.ListenerExecutionFailedException: Listener threw exception
EDIT
Putting DirectReplyToMessageListenerContainer in a separate class and setting its MessageListener as a #Bean and also
directMessageListenerContainer.setMessageListener(practicalMessageListener()); as #Bean seemed to get rid of NPE. But still even the reply goes to amq.rabbitmq.reply-to.g2dkABVyYWJ..... it doesn't seem to be listened in the DirectReplyToMessageListenerContainer drtmlc.
#Component
class DirectMessageListener implements MessageListener {
// This doesn't get invoked...
#Override
public void onMessage(Message message) {
System.out.println("direct reply message sent..");
}
}
#Component
class ReplyListener {
#Bean
public DirectMessageListener directMessageListener() {
return new DirectMessageListener();
}
#Bean
DirectReplyToMessageListenerContainer drtmlc (ConnectionFactory connectionFactory) {
DirectReplyToMessageListenerContainer drtmlc =
new DirectReplyToMessageListenerContainer(connectionFactory);
drtmlc.setConnectionFactory(connectionFactory);
drtmlc.setAcknowledgeMode(AcknowledgeMode.NONE);
drtmlc.setMessageListener(directMessageListener());
return drtmlc;
}
}
Yes, you have mis-understood the feature.
Each channel gets its own pseudo queue; you can only receive from that same channel so a general message listener container won't hack it.
directMessageListenerContainer.setQueueNames("amq.rabbitmq.reply-to");
You simply can't do that.
The framework already supports direct reply-to directly, internally in the RabbitTemplate. The RabbitTemplate has its own DirectReplyToMessageListenerContainer which maintains a pool of channels.
Each request checks out a channel and the reply is returned there and then the channel is returned to the pool for reuse by another request.
Use RabbitTemplate.convertSendAndReceive(); the default behavior (in recent versions) will automatically use direct reply-to.
EDIT
Why not let the framework do all the heavy lifting and you just concentrate on your business logic:
#SpringBootApplication
public class So59760805Application {
public static void main(String[] args) {
SpringApplication.run(So59760805Application.class, args);
}
#Bean
public SimpleMessageListenerContainer container(ConnectionFactory cf) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(cf);
container.setQueueNames("foo");
container.setMessageListener(new MessageListenerAdapter(new MyListener()));
return container;
}
#Bean
public MyExtendedTemplate template(ConnectionFactory cf) {
return new MyExtendedTemplate(cf);
}
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
return args -> System.out.println(template.convertSendAndReceive("", "foo", "test"));
}
}
class MyListener {
public String handleMessage(String in) {
return in.toUpperCase();
}
}
class MyExtendedTemplate extends RabbitTemplate {
MyExtendedTemplate(ConnectionFactory cf) {
super(cf);
}
#Override
public void onMessage(Message message) {
System.out.println("Response received (before conversion): " + message);
super.onMessage(message);
}
}
The rabbit template uses direct reply-to (internally) by default.
Response received (before conversion): (Body:'TEST' MessageProperties [headers={}, correlationId=1, ...receivedRoutingKey=amq.rabbitmq.reply-to.g2dkAA5yYWJiaXRAZ29sbHVtMgAAeE0AAADmAw==.RQ/uxjR79PX/hZF+7iAdWw==, ...
TEST
I wrote my first spring integration application which reads data from spring RSS and logs it into console:
#Configuration
#EnableIntegration
#IntegrationComponentScan
public class DslConfig {
#Bean
public IntegrationFlow feedFlow() throws MalformedURLException {
return IntegrationFlows.from(inBoundFeedDataAdapter(), configurer -> configurer.poller(Pollers.fixedDelay(1000)))
.channel(newsChannel())
.transform(source -> {
SyndEntry e = ((SyndEntry) source);
return e.getTitle() + " " + e.getLink();
})
.handle(messageHandler())
.get();
}
#Bean
public FeedEntryMessageSourceSpec inBoundFeedDataAdapter() throws MalformedURLException {
return Feed.inboundAdapter(new URL("https://spring.io/blog.atom"), "some_key");
}
#Bean
public MessageChannel newsChannel() {
return new DirectChannel();
}
#Bean
public MessageHandler messageHandler() {
return System.out::println;
}
}
But I have no idea how can I add one additional handler for writing result into file.
How can I achieve it ?
Additional questions:
What is the meaning of metadata key ?
There is a publishSubscribeChannel() to place in the flow and there you can add subscribe() for several sub-flows. Each of them is going to get the same message to process. If you also add an Executor to the configuration, the process is going to happen in parallel:
.publishSubscribeChannel(s -> s
.applySequence(true)
.subscribe(f -> f
.handle((p, h) -> "Hello"))
.subscribe(f -> f
.handle((p, h) -> "World!"))
);
See more info in Docs: https://docs.spring.io/spring-integration/docs/5.2.0.BUILD-SNAPSHOT/reference/html/dsl.html#java-dsl-subflows
I am new to spring boot and am trying to use the sample example from the spring integration in order to subscribe and publish using MQTT. I manage to integrate it with Thingsboard and the logger in the code below is able to receive the published message from Thingsboard.
public static void main(String[] args) {
SpringApplication.run(MqttTest.class);
}
#Bean
public MqttPahoClientFactory mqttClientFactory() {
DefaultMqttPahoClientFactory factory = new DefaultMqttPahoClientFactory();
MqttConnectOptions options = new MqttConnectOptions();
options.setServerURIs(new String[] { "URI HERE" });
options.setUserName("ACCESS TOKEN HERE");
factory.setConnectionOptions(options);
return factory;
}
// consumer
#Bean
public IntegrationFlow mqttInFlow() {
return IntegrationFlows.from(mqttInbound())
.transform(p -> p)
.handle(logger())
.get();
}
private LoggingHandler logger() {
LoggingHandler loggingHandler = new LoggingHandler("INFO");
loggingHandler.setLoggerName("LoggerBot");
return loggingHandler;
}
#Bean
public MessageProducerSupport mqttInbound() {
MqttPahoMessageDrivenChannelAdapter adapter = new MqttPahoMessageDrivenChannelAdapter("Consumer",
mqttClientFactory(), "v1/devices/me/rpc/request/+");
adapter.setCompletionTimeout(5000);
adapter.setConverter(new DefaultPahoMessageConverter());
adapter.setQos(1);
return adapter;
}
This is the console output. I am able to receive the published json message that was sent from the thingsboard dashboard. I am wondering if there is a call method to retrieve the json message string so that I can process it further. Thank you.
2019-02-01 14:06:23.590 INFO 13416 --- [ Call: Consumer] LoggerBot : {"method":"setValue","params":true}
2019-02-01 14:06:24.840 INFO 13416 --- [ Call: Consumer] LoggerBot : {"method":"setValue","params":false}
To handle the published messages, subscribe message handles to the flow to consume the messages.
MessageHandler
#Bean
public IntegrationFlow mqttInFlow() {
return IntegrationFlows.from(mqttInbound())
.transform(p -> p)
.handle( mess -> {
System.out.println("mess"+mess);
})
.get();
}
ServiceActivator
#Bean
public IntegrationFlow mqttInFlow() {
return IntegrationFlows.from(mqttInbound())
.transform(p -> p)
.handle("myService","handleHere")
.handle(logger())
.get();
}
#Component
public class MyService {
#ServiceActivator
public Object handleHere(#Payload Object mess) {
System.out.println("payload "+mess);
return mess;
}
}
Note: As we discussed, there are lot of different ways of achieving it.
This is just a sample for your understanding.
I have a spring integration IntegrationFlow that's defined like this:
IntegrationFlows.from(Amqp.inboundAdapter(connectionFactory, "queueName")
.id("id")
.autoStartup(autoStartup)
.concurrentConsumers(2)
.maxConcurrentConsumers(3)
.messageConverter(messageConverter()))
.aggregate(a -> ...)
.handle(serviceActivatorBean)
.get();
And serviceActivatorBean looks like this:
#Component
#Transactional
public class ServiceActivator {
#ServiceActivator
public void myMethod(Collection<MyEvent> events) {
....
}
}
If myMethod throws an exception it will be logged but no retry will happen. I've tried to change the IntegrationFlow to this:
RequestHandlerRetryAdvice advice = new RequestHandlerRetryAdvice();
RetryTemplate retryTemplate = new RetryTemplate();
SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy();
retryPolicy.setMaxAttempts(5);
retryTemplate.setRetryPolicy(retryPolicy);
advice.setRetryTemplate(retryTemplate);
IntegrationFlows.from(Amqp.inboundAdapter(connectionFactory, "queueName")
.id("id")
.autoStartup(autoStartup)
.adviceChain(advice)
.concurrentConsumers(2)
.maxConcurrentConsumers(3)
.messageConverter(messageConverter()))
.aggregate(a -> ...)
.handle(serviceActivatorBean)
.get();
But then I a log message like this (an retries won't happen):
2017-06-30 13:18:10.611 WARN 88706 --- [erContainer#1-2]
o.s.i.h.a.RequestHandlerRetryAdvice : This advice
org.springframework.integration.handler.advice.RequestHandlerRetryAdvice
can only be used for MessageHandlers; an attempt to advise method
'invokeListener' in
'org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$1'
is ignored
How can I configure this IntegrationFlow to behave the same way a RabbitListener would? I.e. let RabbitMQ publish the messages again.
Use a retry interceptor in the adapter's advice chain instead of the RequestHandlerRetryAdvice - that is for consuming endpoints, as the message says.
I may just be missing something very simple here (or misusing something), but I was attempting to set up two direct channels such that one flow would pass some data to each sequentially. So using the Spring Integration JAVA DSL I had something like this (significantly simplified for this example):
public static final String TEST_CHANNEL = "testGateway";
public static final String TEST_UPPER_CHANNEL = "testChannelUpper";
public static final String TEST_LOWER_CHANNEL = "testChannelLower";
#Bean(name = TEST_CHANNEL)
public MessageChannel testGatewayChannel() {
return MessageChannels.direct(TEST_CHANNEL).get();
}
#Bean(name = TEST_UPPER_CHANNEL)
public MessageChannel testChannelUpperChannel() {
return MessageChannels.direct(TEST_UPPER_CHANNEL).get();
}
#Bean(name = TEST_LOWER_CHANNEL)
public MessageChannel testChannelLowerChannel() {
return MessageChannels.direct(TEST_LOWER_CHANNEL).get();
}
#Bean
public IntegrationFlow testFlow() {
return IntegrationFlows
.from(TEST_CHANNEL)
.channel(TEST_UPPER_CHANNEL)
.channel(TEST_LOWER_CHANNEL)
.get();
}
#Bean
public IntegrationFlow testUpperFlow() {
return IntegrationFlows
.from(TEST_UPPER_CHANNEL)
.<String, String>transform(String::toUpperCase)
.handle(System.out::println)
.get();
}
#Bean
public IntegrationFlow testLowerFlow() {
return IntegrationFlows
.from(TEST_LOWER_CHANNEL)
.<String, String>transform(String::toLowerCase)
.handle(System.out::println)
.get();
}
I'm using a REST endpoint to invoke the flow via a gateway, but when I do so it seems only one of the channels is invoked. The channel also seems to be random across invocations (sometimes going to the testChannelUpper and sometimes to the testChannelLower).
I basically end up with this across the executions :
(each time I am just hitting this endpoint http://localhost:9090/test?test=HellOoi)
Execution 1:
GenericMessage [payload=HELLOOI, headers={jobName=someActivity, history=someGateway,testGateway,testChannelUpper,testUpperFlow.channel#0, id=4aa7b075-23cc-6ab3-10a1-c7cb73bae49b, timestamp=1447686848477}]
Execution 2:
GenericMessage [payload=HELLOOI, headers={jobName=someActivity, history=someGateway,testGateway,testChannelUpper,testUpperFlow.channel#0, id=a18dcd01-da18-b00d-30c0-e1a03ce19104, timestamp=1447686853549}]
Execution 3:
GenericMessage [payload=hellooi, headers={jobName=someActivity, history=someGateway,testGateway,testChannelUpper,testLowerFlow.channel#0, id=5f0abcb9-378e-7a3c-9c93-a04ff6352927, timestamp=1447686857545}]
I believe that what I'm attempting here is also shown in the channelFlow example of the DSL wiki :
https://github.com/spring-projects/spring-integration-java-dsl/wiki/Spring-Integration-Java-DSL-Reference
Sooo the specs on what I'm using are :
Spring Boot v1.2.2.RELEASE
Spring v4.1.5.RELEASE
spring-integration-java-dsl 1.0.2.RELEASE
JDK 1.8.0_40-b25
So... has anyone else seen this kind of behavior? Am I just abusing the channel implementation? Any other ideas? Thanks in advance!
As Gary pointed out the best way to do this is to have a pub-sub and order the consumers on this :
#Bean(name = TEST_CHANNEL)
public MessageChannel testGatewayChannel() {
return MessageChannels.publishSubscribe(TEST_CHANNEL).get();
}
#Bean
public IntegrationFlow testUpperFlow() {
return IntegrationFlows
.from(TEST_CHANNEL)
.<String, String>transform(String::toUpperCase, e -> e.order(1))
.handle(System.out::println)
.get();
}
#Bean
public IntegrationFlow testLowerFlow() {
return IntegrationFlows
.from(TEST_CHANNEL)
.<String, String>transform(String::toLowerCase, e -> e.order(2))
.handle(System.out::println)
.get();
}
What is the purpose of this...
#Bean
public IntegrationFlow testFlow() {
return IntegrationFlows
.from(TEST_CHANNEL).fixedSubscriberChannel()
.channel(TEST_UPPER_CHANNEL)
.channel(TEST_LOWER_CHANNEL)
.get();
}
?
All that does is bridge the three channels together.
In fact, you end up with 2 consumers on TEST_UPPER_CHANNEL- the bridge in this flow and the transformer in your other flow.
By default, dispatching in direct channels uses round robin distribution. So the first message will go to the bridge, the next to the transformer, etc, etc.