My question is related to finding a best practice to include data persistence inside an integration flow while returning the Message object so that it can be further processed by the flow.
Let's consider the following flow:
#Bean
IntegrationFlow myFlow() {
return flowDefinition ->
flowDefinition
.filter(filterUnwantedMessages)
.transform(messageTransformer)
.wireTap(flow -> flow.trigger(messagePayloadPersister)) <--- here is the interesting part
.handle(terminalHandler);
}
The wide majority of cases, instead of the wireTap I have seen in some projects, a Transformer is used to persist data, which I do not particulary like, as
the name implies transformation of a message, and persistence is something else.
My wish is to find out alternatives to the wireTap, and a colleague of mine proposed using #ServiceActivator:
#Bean
IntegrationFlow myFlow() {
return flowDefinition ->
flowDefinition
.filter(filterUnwantedMessages)
.transform(messageTransformer)
.handle(messagePayloadPersister)
.handle(terminalHandler);
}
#Component
class MesssagePayloadPersister {
#ServiceActivator <--- interesting, but..
public Message handle(Message<?> msg) {
//persist the payload somewhere..
return message;
}
}
I like the flow, it looks clean now, but also I am not 100% happy with the solution, as I am mixing DSL with Spring.
Note: org.springframework.messaging.MessageHandler is not good because the handle method returns void so it is a terminal part to the flow. I need a method that returns Message object.
Is there any way to do this?
Need to understand what you are going to do with that persisted data in the future.
And what information from the message you are going to store (or the whole message at all).
See this parts of documentation - may be something will give you some ideas:
https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/system-management.html#message-store
https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/system-management.html#metadata-store
https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/message-transformation.html#claim-check
https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/core.html#persistent-queuechannel-configuration
https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/jdbc.html#jdbc-outbound-channel-adapter
With the last one you may need to consider to use a publishSubscribeChannel() of the Java DSL to be able to store in the DB and have a second subscriber to continue the flow.
Related
I have a Spring WebFlux application and am trying to write a WebGraphQlInterceptor to enforce authorization. The authorization requires access to HTTP headers and GraphQL variables, both of which are easily accessible from a WebGraphQlInterceptor. However, if the request fails authorization, I do not want to execute the controller handler and instead exit early with an error response. I have the custom error response working OK, but I cannot figure out how to bypass the controller -- it seems like I'm required to proceed down the original chain, execute the controller, and only then return the error response.
I'm hoping there's an easy solution I'm missing. The documentation seems sparse on this topic with very few examples online. The WebGraphQlInterceptor documentation lists some methods like apply() that sound like they might be helpful for altering the chain, but it's not clear how to use them. The interceptor interface requires a WebGraphQlResponse to be returned, and I can't find a way to return it without continuing down the original chain with chain.next(request) below:
#Override
public Mono<WebGraphQlResponse> intercept(WebGraphQlRequest request, Chain chain) {
return chain.next(request).map(response -> {
// Custom logic...
});
}
I also tried constructing my own custom WebGraphQlResponse from scratch to return from the interceptor, but this felt like a hack and a lot of unnecessary overhead.
I don't want the controller to be executed at all if authorization fails. Is this even possible with the WebGraphQlInterceptor? And what would the simplest implementation look like?
P.S. The WebFilter doesn't help me here because I need easy access to GraphQL variables, which isn't possible with WebFilters.
I have seen in many cases the .handle("someBean", "someMethod") EIP method, playing vast role in the integration flows. I can understand that it is just a Service Activator in the former XML Config, but I need some clarification on how to create this bean and what does the someMethod return. Also, in which cases do I have to use .handle(...)? Maybe a completed example using Java DSL should work for me.
As noticed correctly the .handle("someBean", "someMethod") is fully equal to the <int:service-activator ref="someBean" method="someMethod"/>: https://docs.spring.io/spring-integration/reference/html/messaging-endpoints-chapter.html#service-activator-namespace.
That means that you should have someBean definition for service invocation in the someMethod. For example you need to perform simple logic to convert payload of the incoming message to the upper case and return the result:
class MyService {
public String someMethod(String payload) {
return payload.toUpperCase();
}
}
The return of this method becomes as a payload of the outbound message to the next EIP endpoint in your IntegrationFlow definition.
Everything what you see in the Reference Manual is fully true for Java DSL. In particular all the rules for the <service-activator> or #ServiceActivator are applied for this .handle().
I've been looking at using RabbitMQ for cross-service messaging. I've been able to configure our Exchanges / Queues / DLX etc. using Spring annotations. Example (simple) queue listener:
#RabbitListener(queues = RabbitMessageType.QueueNames.SMS_NOTIFICATIONS)
public void receive1(Message message) throws Exception {
RabbitMessageDto messageDto = OBJECT_MAPPER.readValue(message.getBody(), RabbitMessageDto.class);
SmsNotificationDto payload = OBJECT_MAPPER.readValue(messageDto.getPayload(), SmsNotificationDto.class);
log.info(payload.getMessage());
}
I'm using spring-cloud-sleuth to generate correlationIds / traceIds, which are preserved when using HTTP requests to talk to other services, enabling us to trace the given ID throughout the logs of our various microservices.
While I can get the current traceId and insert that into my DTO:
#Autowired
private Tracer tracer;
private RabbitMessageDto createRabbitMessageWithPayload(String messageType,
String messageVersion,
Object payload) {
return new RabbitMessageDto.Builder()
.withTraceId(tracer.getCurrentSpan().getTraceId())
.withDtoName(messageType)
.withDtoVersion(messageVersion)
.withPayload(payload)
.build();
}
I cannot find a way to set the traceId in the receiving method.
Googling keeps bringing me to spring-cloud-stream and spring-cloud-stream-starter-rabbit; documentation seems to indicate that it's possible automatically insert / set traceIds, but I'm not familiar with spring-cloud-stream at all, and don't find the documentation particularly helpful.
So, I would love answers to the following:
Using SpanAdjuster or Tracer etc; can I set the traceId based on the value in my DTO?
Using spring-cloud-stream, can I automagically insert / retrieve the traceId, and where would I start?
So, incase someone comes across this looking to set the sleuth traceId context, we came up with the following solution:
#Autowired Tracer tracer;
private void someMethod(long traceId) {
Span span = Span.builder()
.traceId(traceId)
.spanId(new Random().nextLong())
.build();
tracer.continueSpan(span);
// do work
tracer.closeSpan(span);
}
It should be noted that all the documentation says that a span should be closed once you've finished with it. The do work section above should be wrapped with a try / catch / finally block to ensure this is closed.
Any methods called with the span still open will inherit the traceId.
EDIT
I should also say, it seems the better solution would be to replace the Spring AMQP library with spring-cloud-stream; from what I can tell, this should automatically include the traceId in rabbit messages (correlationId) and set it at the other end. I haven't had the opportunity to test this, however.
I was trying to understand the caching that happens at the client side.
Unfortunately I am unable to find any resources that can help me out.
I have employee model objects which are fairly small in size.
Once a use a GET request to obtain an employee object, I want it to be cached at the client side
Now when the request comes again to obtain the same employee, I want to see if the actual object has been modified, if not, then serve from the client cache else return the modified object also adding it to the cache.
I am using Spring boot to create a REST endpoint.
What I have been able to figure out is that cache-control would be used some how, but I am not sure how the objects would be added here in spring.
Any help here is much appreciated!!!!
Thanks,
Amar
HTTP caching is not an easy topic. There are different ways to do it, and you should probably start by familiarizing yourself with the mechanisms, this seems to be a good starting resource: HTTP caching
Then, you will probably identify some common usage patterns you will want to reuse. One way to do that is to create custom annotations and write an interceptor that reacts on them.
For example, you could write such an annotation:
#Inherited
#Retention(RUNTIME)
#Target({METHOD, TYPE})
public #interface CacheFor {
long amount();
TimeUnit unit() default TimeUnit.SECONDS;
}
and use it on controller methods like this:
#CacheFor(amount=10, unit = MINUTES)
#RequestMapping(bla bla)
public FooBar serveMyData(){
// code here
}
and in your interceptor, you will need to look at the handler method, check whether it has this annotation, and if it does, set the appropriate headers.
I have an application I coded which I am refactoring to make better use of Spring Integration. The application processes the contents of files.
The problem (as I see it) is that my current implementation passes Files instead of Messages, i.e. Spring Integration Messages.
In order to avoid further rolling my own code, which I then have to maintain later, I'm wondering if there is a recommended structure for constructing Messages in Spring Integration. What I wonder is if there is some recommended combination of channel with something like MessageBuilder that I should use.
Process/Code (eventually)
I don't yet have the code to configure it but I would like to end up with the following components/processes:
Receive a file, remove header and footer of the file, take each line and convert it into a Message<String> (This it seems will actually be a Splitter) which I send on to...
Channel/Endpoint sends message to Router
Router detects format String in Payload and routes to the appropriate channel similar to Order Router here...
Selected channel then builds appropriate type of Message, specifically typed messages. For example I have the following builder to build a Message...
public class ShippedBoxMessageBuilder implements CustomMessageBuilder {
#Override
public Message buildMessage(String input) {
ShippedBox shippedBox = (ShippedBox) ShippedBoxFactory.manufactureShippedFile(input);
return MessageBuilder.withPayload(shippedBox).build();
}
...
Message is routed by type to the appropriate processing channel
My intended solution does seem like I've complicated it. However, I've purposefully separated two tasks 1) Breaking a file into many lines of Messages<String> and 2) Converting Messages<String> into Messages<someType>. Because of that I think I need an additional router/Message builder for the second task.
Actually, there is MessageBuilder support in the Spring Integration.
The general purpose of such Frameworks is to help back-end developers to decouple their domain code from messaging infrastructure. Finally, to work with Spring Integration you need to follow the POJO and Method Invocation principles.
You write your own services, transformers and domain models. Then you just use some out of the box compoenents (e.g. <int-file:inbound-channel-adapter>) and just refer from there to your POJOs, but not vise versa.
I recommend you to read Spring Integration in Action book to have more pictures on the matter.
Can you explain the reason to get deal with Spring Integration components directly?
UPDATE
1) Breaking a file into many lines of Messages
The <splitter> is for you. You should write some POJO which returns List<String> - the lines from your file without header and footer. How to read lines from File isn't a task of Spring Integration. Especially, if the "line" is something logical, not the real file line.
2) Converting Messages into Messages
One more time: there is no reason to build Message object. It's just enough to build new payload in some transformer (again POJO) and framework wrap to its Message to send.
Payload Type Router speaks for itself: it checks a payload type, but not Message type.
Of course, payload can be Message too, and even any header can be as well.
Anyway your Builder snapshot shows exactly a creation of plain Spring Integration Message in the end. And as I said: it will be enough just to transform one payload to another and return it from some POJO, which you will use as a transformer reference.