Develop Reference

Apache Camel JMS with ActiveMQ: Sending Messages with Dynamic Priority

I am currently implementing a Java messaging system with Apache Camel and ActiveMQ. My goal is to dynamically set the priority of a message based on a few attributes the message has.
I already configured my ActiveMQ as explained here. Then I created the following method that sends a TextMessage:
public void send(BaseMessage baseMessage, int jmsPriority) throws JsonProcessingException {
Map<String, Object> messageHeaders = new HashMap<>();
messageHeaders.put(MESSAGING_HEADER_JMS_PRIORITY, jmsPriority);
messageHeaders.put(MESSAGING_HEADER_TYPE, baseMessage.getClass().getSimpleName());
String payload = objectMapper.writeValueAsString(baseMessage);
producerTemplate.sendBodyAndHeaders(payload, messageHeaders);
}
Sending the message perfectly works, and the dynamic type of BaseMessage is properly set to the header of each message. The priority is set as well, but is ignored. The order for the outcoming messages is still FIFO, as queues usually do.
Until now I did not achieve to set the priority of the message dynamically. I do not want to use Apache Camel's Resequencer since I would have to create several new queues only for "sorting". From my point of view ActiveMQ must be able to prioritize and reorder the messages itself.
Any tip is appreciated. Ask me for further details if required.

By default, ActiveMQ disables message priority. This is normal. When doing distributed messaging-- sending messages across servers, prioritization does not practically work out, since the broker can only scan so many messages in the queue for messages of a higher priority before it stats to slow down all traffic for that queue.
Prioritized messages can work well when embedding a broker and using it for task dispatch-- where queue depth generally doesn't exceed the low-thousands.
Updated:
Reminder-- the QOS settings in JMS must be set on the MessageProducer object, and not the message per JMS-spec.
Enable Prioritized Messages

Integration pattern : how to sync processing message received from multiple systems

I am building a system that will receive messages via a Message broker (Currently, JMS) from different systems. All the messages from all the senders systems have a deviceId and there is no order in the reception of the message.
For instance, system A can send a message with deviceId=1 and system b be can send a message with deviceId=2.
My goal is not to start processing of the messages concerning the same deviceId unless I got all the message from all the senders with the same deviceId.
For example, if I have 3 systems A, B and C sending messages to my system :
System A sends messageA1 with deviceId=1
System B sends messageB1 with deviceId=1
System C sends messageC1 with deviceId=3
System C sends messageC2 with deviceId=1 <--- here I should start processing of messageA1, messageB1 and messageC2 because they are having the same deviceID 1.
Should this problem be resolved by using some sync mechanism in my system , by the message broker or an integration framework like spring-integration/apache camel ?

A similar solution with the Aggregator (what #Artem Bilan mentioned) can also be implemented in Camel with a custom AggregationStrategy and with controlling the Aggregator completion by using the Exchange.AGGREGATION_COMPLETE_CURRENT_GROUP property.
The following might be a good starting point. (You can find the sample project with tests here)
Route:
from("direct:start")
.log(LoggingLevel.INFO, "Received ${headers.system}${headers.deviceId}")
.aggregate(header("deviceId"), new SignalAggregationStrategy(3))
.log(LoggingLevel.INFO, "Signaled body: ${body}")
.to("direct:result");
SignalAggregationStrategy.java
public class SignalAggregationStrategy extends GroupedExchangeAggregationStrategy implements Predicate {
private int numberOfSystems;
public SignalAggregationStrategy(int numberOfSystems) {
this.numberOfSystems = numberOfSystems;
}
#Override
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
Exchange exchange = super.aggregate(oldExchange, newExchange);
List<Exchange> aggregatedExchanges = exchange.getProperty("CamelGroupedExchange", List.class);
// Complete aggregation if we have "numberOfSystems" (currently 3) different messages (where "system" headers are different)
// https://github.com/apache/camel/blob/master/camel-core/src/main/docs/eips/aggregate-eip.adoc#completing-current-group-decided-from-the-aggregationstrategy
if (numberOfSystems == aggregatedExchanges.stream().map(e -> e.getIn().getHeader("system", String.class)).distinct().count()) {
exchange.setProperty(Exchange.AGGREGATION_COMPLETE_CURRENT_GROUP, true);
}
return exchange;
}
#Override
public boolean matches(Exchange exchange) {
// make it infinite (4th bullet point # https://github.com/apache/camel/blob/master/camel-core/src/main/docs/eips/aggregate-eip.adoc#about-completion)
return false;
}
}
Hope it helps!

You can do this in Apache Camel using a caching component. I think there is the EHCache component.
Essentially:
You receive a message with a given deviceId say deviceId1.
You look up in your cache to see which messages have been received for deviceId1.
As long as you have not received all three you add the current system/message to the cache.
Once all messages are there you process and clear the cache.
You could then off course route each incoming message to a specific deviceId based queue for temporary storage. This can be JMS, ActiveMQ or something similar.

Spring Integration provides component for exactly this kind of tasks - do not emit until the whole group is collected. And it's name an Aggregator. Your deviceId is definitely a correlationKey. The releaseStrategy really may be based on the number of systems - how much deviceId1 messages you are waiting before proceed to the next step.

javax.jms.IllegalStateException: The Session is closed

I have different problems with a Camel Producer that I tried to solved but I've fallen into other problems.
1) The first implementation I did was to create a producer template each time we needed to communicate with an ActiveMQ topic. That resulted with poor memory results leading to server crashing after sometime.
The solution for the memory problem was to stop() producer template after each request. That fix has corrected the memory issue but cause some latency problem.
2) I read somewhere that it's not necessary to create each time a producer template. So I decide to fix the latency problem and declared only one producer template in my class and use it for each request. It seem to work fine, no memory leak, fix the latency problem...
BUT, when we send multiple queries that take a lot of time (20 sec each), it looks like we hit a timeout and the component crash with something like «javax.jms.IllegalStateException: The Session is closed».
Is there a way to do multi threading? Is this cause by using InOut exchange pattern? How the MAXIMUM_CACHE_POOL_SIZE works? Is my implementation is right?
I've put a sample of the code of my component:
public void process(Exchange exchange) throws Exception
{
Message in = exchange.getIn();
if (producerTemplate == null) {
CamelContext camelContext = exchange.getContext();
//camelContext.getProperties().put(Exchange.MAXIMUM_CACHE_POOL_SIZE, "50");
producerTemplate = camelContext.createProducerTemplate();
}
...
result = producerTemplate.sendBody(String.format("activemq:%s", camelContext.resolvePropertyPlaceholders("{{channel1}}")), ExchangePattern.InOut, messageToSend).toString();
...
finalResult = producerTemplate.sendBody(String.format("activemq:%s", camelContext.resolvePropertyPlaceholders("{{channel2}}")), ExchangePattern.InOut, result).toString();
...
in.setBody(finalResult );
}

Yes it is because you use InOut pattern.
Your route expects a response to the specified reply queue, which is never received, and therefore results in the default 20 sec. timeout.
Change the Exchange pattern to InOnly to resolve your issue.
Apart from that, your posted code seems to be fine.
The MAXIMUM_CACHE_POOL_SIZE is used internally in Camel, and thus does not effect the ActiveMQ endpoint settings.

Dynamically added Camel routes not appearing in InflightExchange

I added a route dynamically to an existing CamelContext,
exchange.getContext().addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
from("sftp://...")
.routeId("test-dy-route")
...
.to("file://...");
}
});
When I try to get the number of inflight exchanges of this route, it returns zero. For the same route, the status is returned correctly, i.e., "STARTED"
ServiceStatus status = context.getRouteStatus("test-dy-route");
int inflightExchanges = context.getInflightRepository().size("test-dy-route");
Did a watch over the context object on runtime (using Eclipse), it didn't have the route "" in InflightRepository associated with it. Not sure, what could be the reason. Meanwhile, I can see an exchange being processed on this route in the logs. I need the no of inflight exchanges to stop the route as per requirements.
EDIT: It is not actually an issue, but expected behavior as unless the remote file from the ftp server has been downloaded (using localWorkDirectory option), the route won't proceed further and hence the resource is not yet considered as an inflight exchange.
So, the next problem is, how to detect if the component is downloading a file?

If your problem is to detect when the FTP component is downloading, you can try to enable the streamDownload option (see the component page). This way, as long as File component is reading the stream it should be considered an inflight exchange.

how to delay JMS message in spring on weblogic 10.3 server

I want to delay message delivery by certain time which will be different for each message.
I referred weblogic 10.3 WLMessage documentation which recommends using javax.jms.Message.getIntProperty("JMS_BEA_DeliveryTime").
However I'm unable to figure out how do I set JMS_BEA_DeliveryTime as INTEGER. I was expecting it to be long.
I'm unable to find out any documentation about this attribute JMS_BEA_DeliveryTime to undesratand how to set and use it. Can anybody give example about how to set the desired delivery time or link to documenation of use of this attribute.
My code looks like typical use of Spring JMS template:
jmsTemplate.send(destination, new MessageCreator() {
public Message createMessage(Session session) throws JMSException {
TextMessage message = session.createTextMessage(textMessage);
return message;
}
});
I was wondering if it is possible to set header for delivery time before returning message.

Good article with examples of delaying delivery for different JMS providers, include WL.

Well there is this page which states
JMS Delivery Time: The earliest absolute time at which
a message can be delivered to a consumer.
It should be trivial to set with Message.setIntProperty(...), there is not much mention as to what the time actually constitutes though, although I did find this article which suggests milliseconds from now.
Hope that helps.