I am getting up too speed with Spring Integration DSL. I am playing with the below example.
#Bean
public IntegrationFlow flow() {
return IntegrationFlows.from(INBOX)
.transform(p -> "world")
.get();
}
I am looking for the ability from this one flow to subscribe to multiple channels. I cannot find anything about this.
For example something like the below, where this flow is subscribed to different channels.
#Bean
public IntegrationFlow flow() {
return IntegrationFlows.from(INBOX).flow(INBOX2)
.transform(p -> "world")
.get();
}
That isn't possible. There is just no any Endpoint with several inputChannels.
On the other hand we don't need such a complexity since we always can bridge from one channel to another:
#Bean
#BridgeTo(INBOX)
public MessageChannel INBOX2() {
return new DirectChannel();
}
Also you can consider to use some router to always evaluate to the desired channel for output.
The MessageChannel is complex per se in the Spring Integration design, that it won't sound good to mess the endpoint logic.
In this tutorial[Receive and send multiple JMS messages in one transaction with Spring Integration Java DSL], they have described like that
however this property is not yet available to the Java DSL. Another
way to solve this problem would be to replace the message driven
channel adapter with a transactional poller, this however is also not
possible in the current Java DSL. To fix this we replaced the
jmsFactory in the outbound adapter with a jmsTemplate with the session
transacted set to true. Resulting in:
IntegrationFlows
.from(subscribableChannel())
.handle(Jms.outboundAdapter(jmsTemplate).destination(QUEUE2))
.get();
Related
I try to convert the integration cafe demo to Java 8 DSL, there are some existing examples in the official Spring Integration Samples.
I want to combine all the good parts of these examples.
Java 8 DSL instead of XML
AMQP to slipt the workflow in smaller flows.
Using Jackson to handle the payload in JSON.
The codes(not work as expected, some issues) are here.
In the official XML samples, it is easy to set up a request/reply channel in Amqp gateway. But go to Java 8 DSL, the options are missing.
And when the application, it will complain "reply channel or output channel is required".
BTW, is there any better option to debug/unit testing the Spring integration application?
Update 1: confused when I was writing the flow of coldDrinks flow.
The XML is from the original cafe-amqp project.
<!-- To receive an AMQP Message from a Queue, and respond to its reply-to address, configure an inbound-gateway. -->
<int-amqp:inbound-gateway
id="coldDrinksBarista"
request-channel="coldJsonDrinks"
queue-names="cold-drinks"
connection-factory="rabbitConnectionFactory" />
<int:chain input-channel="coldJsonDrinks">
<int:json-to-object-transformer type="org.springframework.integration.samples.cafe.OrderItem"/>
<int:service-activator method="prepareColdDrink">
<bean class="org.springframework.integration.samples.cafe.xml.Barista"/>
</int:service-activator>
<int:object-to-json-transformer content-type="text/x-json"/>
</int:chain>
How to convert this into Java DSL effectively. I added my thoughts inline
#Bean
public IntegrationFlow coldDrinksFlow(AmqpTemplate amqpTemplate) {
return IntegrationFlows
.from("coldDrinks")
.handle(
Amqp.outboundGateway(amqpTemplate)
.exchangeName(TOPIC_EXCHANGE_CAFE_DRINKS)
.routingKey(ROUTING_KEY_COLD_DRINKS)
)
.log("coldDrinksFlow")
.channel(preparedDrinksChannel())
.get();
}
#Bean
public IntegrationFlow coldDrinksBaristaFlow(ConnectionFactory connectionFactory, Barista barista) {
return IntegrationFlows
.from(Amqp.inboundGateway(connectionFactory, QUEUE_COLD_DRINKS)
.configureContainer(
c -> c.receiveTimeout(10000)
)// If setup replyChannel the below `handle` is not worked as expected.
)
.handle(OrderItem.class, (payload, headers) -> (Drink) barista.prepareColdDrink(payload))
//If adding a channel here, the flow will NOT return back the `coldDrinksFlow` will cause another exception, "its requiresReply is set to true..."
.get();
}
In my before experience, I would like to break the whole flow by the protocols(HTTP, FTP, etc) as edges(inbound at the beginning and outbound as the end) in small flows. The inbound/outbound gateway is easy to set it to work without setting a reply channel etc, it should reply by default through the original route using its built-in protocols instead of channels. In my inboundGateway RSocket example, I don't set a reply channel there, but the message returns to the roscket routes and receive by the client side(outboudGateway).
Update: Finally it works, check here. An issue I encountered here when trying to use Amqp to send and receive object message, the are some class cast exception thrown, the TypeId in headers is NOT changed when using handle etc MessageHandler, have to transform bwteen json/object like the xml based cafe-amqp did to make it finally work. What is missing here?
There is already an official Java DSL sample in that repo: https://github.com/spring-projects/spring-integration-samples/tree/master/dsl/cafe-dsl.
Yes, there is no an AMQP variant, but if you follow an Amqp factory from spring-integration-amqp, that should not be too hard to convert that XML config to Java DSL.
Not sure what is that "bad option" for testing you use, but in the spring-integration-test we provide enough utils to simplify flows testing: https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/testing.html#testing
UPDATE
Your problem is here:
<int-amqp:outbound-gateway
id="coldDrinksBarista"
request-channel="coldDrinks"
reply-channel="preparedJsonDrinks"
exchange-name="cafe-drinks"
routing-key="drink.cold"
amqp-template="amqpTemplate" />
where your config:
#Bean
public IntegrationFlow coldDrinksFlow(AmqpTemplate amqpTemplate) {
return IntegrationFlows
.from("coldDrinks")
.handle(
Amqp.outboundGateway(amqpTemplate)
.exchangeName(TOPIC_EXCHANGE_CAFE_DRINKS)
.routingKey(ROUTING_KEY_COLD_DRINKS)
)
.get();
}
doesn't have a similar replyChannel part. That why you get that reply channel or output channel is required because you send from the route which does not expect any replies. Probably you just haven't finished to design your application for the whole logic...
My use case is notification service implementation in our project.
We have used spring with jms and its working fine with rest services. Able to send message to queue from one application and receive a message from queue in another application using JMSTemplate's convertAndSend(), convertAndReceive() functions.
Now I want to do this using spring integration's java dsl approach. We did a sample by using Jms's inboundGateway() and outboundGateway() in single application with ActiveMQ.
How to use spring integration's java dsl in different application for only sending/receiving messages?
Here is my code,
#Bean
public IntegrationFlow jmsInboundFlow() {
return IntegrationFlows
.from(Jms.inboundGateway(this.connectionFactory)
.destination("pan.outbound"))
.transform((String s) -> s.toUpperCase())
.channel("in-bound-request-channel")
.get();
}
#Bean
public IntegrationFlow jmsOutboundGatewayFlow() {
return IntegrationFlows.from("out-bound-request-channel")
.handle(Jms.outboundGateway(this.connectionFactory)
.requestDestination("pan.outbound"))
.log()
.bridge()
.get();
}
#MessagingGateway
interface EchoGateway {
#Gateway(requestChannel = "out-bound-request-channel")
String send(String message);
}
You have only to split configuration - JMS inbound adapter in one application and JMS outbound in second.
The way is exactly the same as in one application ? (Maybe I missed some information?)
Spring integration polls for the files in a directory , does some validation on it and then i want to move it to another directory. How to achieve the same?
#Bean
public IntegrationFlow inboundFileIntegration(#Value("${inbound.file.poller.fixed.delay}") long period,
#Value("${inbound.file.poller.max.messages.per.poll}") int maxMessagesPerPoll,
TaskExecutor taskExecutor,
MessageSource<File> fileReadingMessageSource
) {
return IntegrationFlows
.from(fileReadingMessageSource,
c -> c.poller(Pollers.fixedDelay(period)
.taskExecutor(taskExecutor)
.maxMessagesPerPoll(maxMessagesPerPoll)))
.channel(ApplicationConfiguration.INBOUND_CHANNEL)
.filter(validateDeal, "validate", e -> e.discardFlow(sf -> sf.handle(fileImproperEmailwriter,"sendEmail")))
.channel(ApplicationConfiguration.OUTBOUND_CHANNEL)
.get();
}
The FileReadingMessageSource populates these headers to the message:
return getMessageBuilderFactory()
.withPayload(file)
.setHeader(FileHeaders.RELATIVE_PATH,
file.getAbsolutePath()
.replaceFirst(Matcher.quoteReplacement(
this.directory.getAbsolutePath() + File.separator), ""))
.setHeader(FileHeaders.FILENAME, file.getName())
.setHeader(FileHeaders.ORIGINAL_FILE, file);
So, you always have access to the original file. To move it to different path you can use java.nio.file.Files.move(Path source, Path target, CopyOption... options) utility. And really use it somewhere in the flow.
Unfortunately, the flow you demonstrate doesn't have any clues how are you going to move file at all. On the other hand it isn't clear why you can't perform that on the ApplicationConfiguration.OUTBOUND_CHANNEL subscriber side for files passed validation and some other move in the discardFlow(). You always can use PublishSubscribeChannel to have several subscribers to perform some parallel work for the same message.
UPDATE
#Bean(name = ApplicationConfiguration.OUTBOUND_CHANNEL)
MessageChannel myOutboundChannel() {
return new PublishSubscribeChannel();
}
And since you use already .channel(ApplicationConfiguration.OUTBOUND_CHANNEL) this bean is going to be used instead. I guess you have some subscriber or IntegrationFlow to consume this channel. What you want to do should be expressed as one more subscriber or an appropriate IntegrationFlow for post-process work.
Due to the design of MQTT where you can only make a connection with a unique client id, is it possible to use the same connection to publish and subscribe in Spring Framework/Boot using Integration?
Taking this very simple example, it would connect to the MQTT broker to subscribe and get messages, but if you would want to publish a message, the first connection will disconnect and re-connect after the message is sent.
#Bean
public MqttPahoClientFactory mqttClientFactory() {
DefaultMqttPahoClientFactory factory = new DefaultMqttPahoClientFactory();
factory.setServerURIs("tcp://localhost:1883");
factory.setUserName("guest");
factory.setPassword("guest");
return factory;
}
// publisher
#Bean
public IntegrationFlow mqttOutFlow() {
return IntegrationFlows.from(CharacterStreamReadingMessageSource.stdin(),
e -> e.poller(Pollers.fixedDelay(1000)))
.transform(p -> p + " sent to MQTT")
.handle(mqttOutbound())
.get();
}
#Bean
public MessageHandler mqttOutbound() {
MqttPahoMessageHandler messageHandler = new MqttPahoMessageHandler("siSamplePublisher", mqttClientFactory());
messageHandler.setAsync(true);
messageHandler.setDefaultTopic("siSampleTopic");
return messageHandler;
}
// consumer
#Bean
public IntegrationFlow mqttInFlow() {
return IntegrationFlows.from(mqttInbound())
.transform(p -> p + ", received from MQTT")
.handle(logger())
.get();
}
private LoggingHandler logger() {
LoggingHandler loggingHandler = new LoggingHandler("INFO");
loggingHandler.setLoggerName("siSample");
return loggingHandler;
}
#Bean
public MessageProducerSupport mqttInbound() {
MqttPahoMessageDrivenChannelAdapter adapter = new MqttPahoMessageDrivenChannelAdapter("siSampleConsumer",
mqttClientFactory(), "siSampleTopic");
adapter.setCompletionTimeout(5000);
adapter.setConverter(new DefaultPahoMessageConverter());
adapter.setQos(1);
return adapter;
}
Working with 2 separate connections becomes difficult if you need to wait for an answer/result after publishing a message...
the first connection will disconnect and re-connect after the message is sent.
Not sure what you mean by that; both components will keep open a persistent connection.
Since the factory doesn't connect the client, the adapters do, it's not designed for using a shared client.
Using a single connection won't really help with coordination of requests/replies because the reply will still come back asynchronously on another thread.
If you have some data in the request/reply that you can use for correlation of replies to requests, you can use a BarrierMessageHandler to perform that task. See my answer here for an example; it uses the standard correlation id header, but that's not possible with MQTT, you need something in the message.
TL;DR
The answer is no, not with the current Spring Boot MQTT Integration implementation (and maybe not even with future ones).
Answer
I'm facing the same exact situation: I need an MQTT Client to be opened in both inbound and outbound, making the connection persistent and sharing the same configuration (client ID, credentials, etc.), using Spring Integration Flows as close to the design as possible.
In order to achieve this, I had to reimplement MqttPahoMessageDrivenChannelAdapter and MqttPahoMessageHandler and a Client Factory.
In both MqttPahoMessageDrivenChannelAdapter and MqttPahoMessageHandler I had to choose to use the Async one (IMqttAsyncClient) in order to fix which one to use. Then I had to review parts of code where the client instance is called/used in order to check if it was already instantiated by the other flow and checking the status (e.g. not trying to connect it if it was already connected).
Regarding the Client Factory, it was easier: I've reimplemented the getAsyncClientInstance(String url, String clientId) using the concatenation of url and clientId as hash as key to store the instance into a map that is used to retrieve the existing instance if the other flow requests it.
It somehow works, but it's just a test and I'm not even sure it's a good approach. (I've started another StackOverflow question in order to track my specific scenario).
Can you share how did you manage your situation?
I have a special IntegrationFlow configured like
#Bean
public IntegrationFlow setupRabbitFlow() {
return IntegrationFlows.from(myInputChannel)
.handle((p, h) -> rabbitPublisher.publishToRabbit(p, h))
.get();
}
and some other flow that processes incoming data from some XML files, e.g. as shown here Polling from file using Java DSL - compile error when adding Files.inboundAdapter. By the end of that flow I want to pass Message to the abovementioned rabbit-sending "sink". How do I declare this?
One of the first class citizen in Spring Integration is a MessageChannel abstraction.
Any interaction between Spring Integration components (endpoints) is really done through the message channels.
What you need from your second flow is just specify .channel(myInputChannel) in the end of that flow. And the result of the XML processing will be send to your first flow.