Develop Reference

Sending message to a changing number of endpoints using Camel http4

I have created the following piece of code and, instead of just one endpoint, I want to send the same JSON body to multiple endpoints using camel HTTP4. The number of endpoints is likely to change over time.
.choice()
.when(simple("${body} != null"))
.setHeader(Exchange.HTTP_METHOD, constant("POST"))
.setHeader(Exchange.HTTP_URI, simple(url))
.setProperty("ObjectBody", body())
.to("http4://someEndpoint?throwExceptionOnFailure=true&httpClient.socketTimeout=300000")
.otherwise()
.log(LoggingLevel.ERROR, "Incoming request has empty body")
.endChoice()
.end()
I'm trying to figure out the best way to configure and use multiple endpoints in this scenario.
I have looked at the Multicast EIP, but it looks like I would have to know the number of endpoints beforehand.
Any ideas are greatly appreciated!

The (dynamic in your case) recipientList is what you need !
See https://camel.apache.org/components/3.14.x/eips/recipientList-eip.html

Did you take a look at RouteBox?
THE NEED FOR A CAMEL ROUTEBOX ENDPOINT The routebox component is
designed to ease integration in complex environments needing a large
collection of routes and involving a wide set of endpoint technologies
needing integration in different ways
In such environments, it is often necessary to craft an integration
solution by creating a sense of layering among camel routes
effectively organizing them into
Coarse grained or higher level routes - aggregated collection of inner
or lower level routes exposed as Routebox endpoints that represent an
integration focus area. For example
https://camel.apache.org/components/2.x/routebox-component.html
Its usage:
from ("direct:sendToMapBasedRoutebox")
.setHeader("ROUTE_DISPATCH_KEY", constant("addToCatalog"))
.to("routebox:multipleRoutes?innerRegistry=#registry&routeBuilders=#routes&dispatchMap=#map")
.to("log:Routes operation performed?showAll=true");
Then in theory this should fan out the requests towards all (routes) encapsulated by a RouteBox.

Spring boot - Threads / Feign-Client / Messaging / Streamlistener

We struggle to find a solution for the following scenario:
Situation
Receive a message via Spring Cloud Streamlistener
Invoke a REST-Service via Feign-Client
We have configured several Feign-RequestInterceptor to enrich
request header data.
We want to avoid passing every request header on the method call and like the central configuration approach of the request interceptors.
Problem:
How to access data from a specific message, which contains informations, that need to be added to every request call via the Feign-RequestInterceptor.
We don't have a Request-Context, as we come from a message.
Can we be sure , that the message consumption and the REST call is happening on the same thread? If yes, we could use the NamedThreadLocal to store the information.

Yes, unless you hand off to another thread in your StreamListener, the rest call will be made on the same thread (assuming you are using RestTemplate and not the reactive web client).

Spring Cloud Stream with RabbitMQ binder, how to apply #Transactional?

I have a Spring Cloud Stream application that receives events from RabbitMQ using the Rabbit Binder. My application can be summarized as this:
#Transactional
#StreamListener(MySink.SINK_NAME)
public void processEvents(Flux<Event> events) {
// Transform events and store them in MongoDB using
// spring-boot-data-mongodb-reactive
...
}
The problem is that it doesn't seem that #Transactional works with Spring Cloud Stream (or at least that's my impression) since if there's an exception when writing to MongoDB the event seems to have already been ack:ed to RabbitMQ and the operation is not retried.
Given that I want to achieve basically the same functionality as when using the #Transactional around a function with spring-amqp:
Do I have to manually ACK the messages to RabbitMQ when using Spring
Cloud Stream with the Rabbit Binder?
If so, how can I achieve this?

There are several issues here.
Transactions are not required for acknowledging messages
Reactor-based #StreamListener methods are invoked exactly once, just to set up the Flux so #Transactional on that method is meaningless - messages then flow through the flux so anything pertaining to individual messages has to be done within the context of the flux.
Spring Transactions are bound to the thread - Reactor is non-blocking; the message will be acked at the first handoff.
Yes, you would need to use manual acks; presumably on the result of the mongodb store operation. You would probably need to use Flux<Message<Event>> so you would have access to the channel and delivery tag headers.

Triggering Spring Integration

I'm still new to Spring Integration and Spring framework as a whole so please bear with me. I've been looking at this example.
https://github.com/benjaminwootton/spring-integration-examples/blob/master/target/classes/direct-channel-example.xml
https://github.com/benjaminwootton/spring-integration-examples/tree/master/src/main/java/examples/components
I'm wondering how do I or Spring trigger the method exactly?
I'm trying to do a round-robin using direct channel for my REST services. My REST services consumes messages and processes it.
I understand that with direct channel, Spring will round-robin through the subscribers but I'm not sure how Spring actually triggers that method.
Thank you for any help or advice.

The first class citizen in the Spring Integration is MessageChannel, so to allow for message (HTTP request) travel in the Integration flow, we should place message to some <channel>.
Since you say that you are in the REST service, I assume you use:
<int-http:inbound-gateway path="/path1,/path2"
request-channel="myChannel"/>
Here the myChannel is a component to where the HTTP request will be sent after conversion to the Spring Integration Message.
Of course, the MessageChannel is a pipe, when we push a thing into one side and there really should be something on the other side to poll that thing. In case of DirectChannel it is some subscriber. And we involve here the second class citizen - MessageHandler.
And if you use there something like <service-activator input-channel="myChannel" ref="foo" method="service">, the call stack may look like:
DirectChannel#send -> UnicastingDispatcher#dispatch ->
ServiceActivatingHandler#handleMessage -> MethodInvokingMessageProcessor#processMessage ->
MessagingMethodInvokerHelper#process -> foo#service
HTH

How should I build my Messages in Spring Integration?

I have an application I coded which I am refactoring to make better use of Spring Integration. The application processes the contents of files.
The problem (as I see it) is that my current implementation passes Files instead of Messages, i.e. Spring Integration Messages.
In order to avoid further rolling my own code, which I then have to maintain later, I'm wondering if there is a recommended structure for constructing Messages in Spring Integration. What I wonder is if there is some recommended combination of channel with something like MessageBuilder that I should use.
Process/Code (eventually)
I don't yet have the code to configure it but I would like to end up with the following components/processes:
Receive a file, remove header and footer of the file, take each line and convert it into a Message<String> (This it seems will actually be a Splitter) which I send on to...
Channel/Endpoint sends message to Router
Router detects format String in Payload and routes to the appropriate channel similar to Order Router here...
Selected channel then builds appropriate type of Message, specifically typed messages. For example I have the following builder to build a Message...
public class ShippedBoxMessageBuilder implements CustomMessageBuilder {
#Override
public Message buildMessage(String input) {
ShippedBox shippedBox = (ShippedBox) ShippedBoxFactory.manufactureShippedFile(input);
return MessageBuilder.withPayload(shippedBox).build();
}
...
Message is routed by type to the appropriate processing channel
My intended solution does seem like I've complicated it. However, I've purposefully separated two tasks 1) Breaking a file into many lines of Messages<String> and 2) Converting Messages<String> into Messages<someType>. Because of that I think I need an additional router/Message builder for the second task.

Actually, there is MessageBuilder support in the Spring Integration.
The general purpose of such Frameworks is to help back-end developers to decouple their domain code from messaging infrastructure. Finally, to work with Spring Integration you need to follow the POJO and Method Invocation principles.
You write your own services, transformers and domain models. Then you just use some out of the box compoenents (e.g. <int-file:inbound-channel-adapter>) and just refer from there to your POJOs, but not vise versa.
I recommend you to read Spring Integration in Action book to have more pictures on the matter.
Can you explain the reason to get deal with Spring Integration components directly?
UPDATE
1) Breaking a file into many lines of Messages
The <splitter> is for you. You should write some POJO which returns List<String> - the lines from your file without header and footer. How to read lines from File isn't a task of Spring Integration. Especially, if the "line" is something logical, not the real file line.
2) Converting Messages into Messages
One more time: there is no reason to build Message object. It's just enough to build new payload in some transformer (again POJO) and framework wrap to its Message to send.
Payload Type Router speaks for itself: it checks a payload type, but not Message type.
Of course, payload can be Message too, and even any header can be as well.
Anyway your Builder snapshot shows exactly a creation of plain Spring Integration Message in the end. And as I said: it will be enough just to transform one payload to another and return it from some POJO, which you will use as a transformer reference.