I've been looking at using RabbitMQ for cross-service messaging. I've been able to configure our Exchanges / Queues / DLX etc. using Spring annotations. Example (simple) queue listener:
#RabbitListener(queues = RabbitMessageType.QueueNames.SMS_NOTIFICATIONS)
public void receive1(Message message) throws Exception {
RabbitMessageDto messageDto = OBJECT_MAPPER.readValue(message.getBody(), RabbitMessageDto.class);
SmsNotificationDto payload = OBJECT_MAPPER.readValue(messageDto.getPayload(), SmsNotificationDto.class);
log.info(payload.getMessage());
}
I'm using spring-cloud-sleuth to generate correlationIds / traceIds, which are preserved when using HTTP requests to talk to other services, enabling us to trace the given ID throughout the logs of our various microservices.
While I can get the current traceId and insert that into my DTO:
#Autowired
private Tracer tracer;
private RabbitMessageDto createRabbitMessageWithPayload(String messageType,
String messageVersion,
Object payload) {
return new RabbitMessageDto.Builder()
.withTraceId(tracer.getCurrentSpan().getTraceId())
.withDtoName(messageType)
.withDtoVersion(messageVersion)
.withPayload(payload)
.build();
}
I cannot find a way to set the traceId in the receiving method.
Googling keeps bringing me to spring-cloud-stream and spring-cloud-stream-starter-rabbit; documentation seems to indicate that it's possible automatically insert / set traceIds, but I'm not familiar with spring-cloud-stream at all, and don't find the documentation particularly helpful.
So, I would love answers to the following:
Using SpanAdjuster or Tracer etc; can I set the traceId based on the value in my DTO?
Using spring-cloud-stream, can I automagically insert / retrieve the traceId, and where would I start?
So, incase someone comes across this looking to set the sleuth traceId context, we came up with the following solution:
#Autowired Tracer tracer;
private void someMethod(long traceId) {
Span span = Span.builder()
.traceId(traceId)
.spanId(new Random().nextLong())
.build();
tracer.continueSpan(span);
// do work
tracer.closeSpan(span);
}
It should be noted that all the documentation says that a span should be closed once you've finished with it. The do work section above should be wrapped with a try / catch / finally block to ensure this is closed.
Any methods called with the span still open will inherit the traceId.
EDIT
I should also say, it seems the better solution would be to replace the Spring AMQP library with spring-cloud-stream; from what I can tell, this should automatically include the traceId in rabbit messages (correlationId) and set it at the other end. I haven't had the opportunity to test this, however.
Related
The below consumerListener() gets triggered for all messages being sent in the topic 'test_topic', but I would like it to only get triggered if the key of the message in the topic is 'Initiate'.
Currently, I am able to do it as shown below, but is there a better way?
#KafkaListener(topics = "test_topic", groupId = "group_id")
public void consumerListener(ConsumerRecord<String, MessageModel> message) throws IOException {
if(message.key().equals("Initiate"){
businessFunctionality(message.value());
}
}
See Filtering Messages.
The Spring for Apache Kafka project also provides some assistance by means of the FilteringMessageListenerAdapter class, which can wrap your MessageListener. This class takes an implementation of RecordFilterStrategy in which you implement the filter method to signal that a message is a duplicate and should be discarded. This has an additional property called ackDiscarded, which indicates whether the adapter should acknowledge the discarded record. It is false by default.
When you use #KafkaListener, set the RecordFilterStrategy (and optionally ackDiscarded) on the container factory so that the listener is wrapped in the appropriate filtering adapter.
In addition, a FilteringBatchMessageListenerAdapter is provided, for when you use a batch message listener.
Starting with version 2.8.4, you can override the listener container factory’s default RecordFilterStrategy (if any) by using the filter property on the listener annotations.
#KafkaListener(id = "filtered", topics = "topic", filter = "differentFilter")
public void listen(Thing thing) {
...
}
I have been agonizing over trying to get the Spring Cloud Sleuth traceId to carry over MQ for quite some time. As I understand based on various articles this is currently unsupported.
Yet when I attempted to add "spanId" as a header property on a JMS message I observed that on the MQ listener side the traceId was carried over automatically and I cannot explain why.
Can someone please help me understand why the below code works ?
Note: spring-cloud-starter-sleuth version = 3.0.0, Spring Boot version = 2.4.3
#AllArgsConstructor
public class MQMessagePublisher {
private final JMSTemplate;
private final Tracer tracer;
public void publishDummyMessage(){
TextMessage textMessage = new JMSTextMessage("some data");
Span span = tracer.currentSpan();
message.setStringProperty("spanId", span.context().spanId());
jmsTemplate.convertAndSend("SOME_Q_NAME", message);
}
}
public class MQListener {
#JmsListener(id="listener", destination = "SOME_Q_NAME", concurrency = "2-4")
public void listen(Message message) throws JMSException {
log.info("Span Id: {}", message.getStringProperty("spanId"));
}
}
Apologies for any syntax errors. I did not copy this code from my editor - just wrote it freehand just now but you should be able to replicate my observations with this.
The thing I cant understand is why does this work ? The log statement in the listener clearly carries across the same traceId without any additional work.
UPDATE :
Earlier the question was written incorrectly - its actually the traceId that gets carried over when the 'spanId' is added to the jms message.
I have updated the question above with the correct terms.
I have a requirement where the complete flow of different async requests is related with one trackingId across different services which communicate asynchronously with the help of messages, And I need to have a single traceId for all requests related to one flow or trackingId.
And I am facing issues with getting/setting TraceContext from/into Tracer, I wrote below code but it creates a new traceId for each request.
TraceContext traceContext = (TraceContext) contextMap.get(trackingId);
if (traceContext == null)
{
tracing.tracer().startScopedSpan(trackingId);
traceContext = tracing.currentTraceContext().get();
} else
{
tracing.tracer().startScopedSpanWithParent(trackingId, traceContext);
}
contextMap.put(trackingId, traceContext);
I'm not 100% sure what do you want to achieve here, as far as I can understand, what you described is the default behavior, please see the docs.
The tracing context is propagated out of the box, so all the related spans have the same traceId, please see the docs how you can set this up for messaging, there is also a sample project for messaging.
My question is related to finding a best practice to include data persistence inside an integration flow while returning the Message object so that it can be further processed by the flow.
Let's consider the following flow:
#Bean
IntegrationFlow myFlow() {
return flowDefinition ->
flowDefinition
.filter(filterUnwantedMessages)
.transform(messageTransformer)
.wireTap(flow -> flow.trigger(messagePayloadPersister)) <--- here is the interesting part
.handle(terminalHandler);
}
The wide majority of cases, instead of the wireTap I have seen in some projects, a Transformer is used to persist data, which I do not particulary like, as
the name implies transformation of a message, and persistence is something else.
My wish is to find out alternatives to the wireTap, and a colleague of mine proposed using #ServiceActivator:
#Bean
IntegrationFlow myFlow() {
return flowDefinition ->
flowDefinition
.filter(filterUnwantedMessages)
.transform(messageTransformer)
.handle(messagePayloadPersister)
.handle(terminalHandler);
}
#Component
class MesssagePayloadPersister {
#ServiceActivator <--- interesting, but..
public Message handle(Message<?> msg) {
//persist the payload somewhere..
return message;
}
}
I like the flow, it looks clean now, but also I am not 100% happy with the solution, as I am mixing DSL with Spring.
Note: org.springframework.messaging.MessageHandler is not good because the handle method returns void so it is a terminal part to the flow. I need a method that returns Message object.
Is there any way to do this?
Need to understand what you are going to do with that persisted data in the future.
And what information from the message you are going to store (or the whole message at all).
See this parts of documentation - may be something will give you some ideas:
https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/system-management.html#message-store
https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/system-management.html#metadata-store
https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/message-transformation.html#claim-check
https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/core.html#persistent-queuechannel-configuration
https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/jdbc.html#jdbc-outbound-channel-adapter
With the last one you may need to consider to use a publishSubscribeChannel() of the Java DSL to be able to store in the DB and have a second subscriber to continue the flow.
I working on an application which reads message from Azure service bus. This application was created using spring boot, Spring jms and Qpid jms client. I am able to read the message properly from Queue without any issues. PFB My code which I am using to read message.
#Service
public class QueueReceiver {
#JmsListener(destination = "testing")
public void onMessage(String message) {
if (null != message) {
System.out.println("Received message from Queue: " + message);
}
}}
Issue is we have different destinations for different environemnts, like testing for dev, testing-qa for qa and testing-prod for production, all these values are provided as azure.queueName in different application-(ENV).proerpties respectively. I want to pass these destinations dynamically to the destination in JmsListener Annotation. When i try using
#Value("${azure.queueName}")
private String dest;
and passing dest to annotation like #JmsListener(destination = dest)
I am getting The value for annotation attribute JmsListener.destination must be a constant expression Error. After googling with this Error i found that we cannot pass dynamic value to Annotation. Please help me how to resolve this issue or any other solution for this.
Use
destination="${azure.queueName}"
i.e. put the placeholder in the annotation directly.
You can use a dynamic name as defined in the application.properties file.For Example:
#JmsListener(destination = "${queue.name}")
Since you can't access any class variables here so this is the best option available.