Keep messages in ActiveMQ queue if ThreadPoolTaskExecutor has no free capacity - java

I have two Java processes, the first one of them produces messages and puts
them onto an ActiveMQ queue. The second process (consumer) uses Spring
Integration to get messages from the queue and processes them in threads.
I have two requirements:
The consumer should have 3 processing threads. If I have 10 messages
coming in through the queue, I want to have 3 threads processing the first 3
messages, and the other 7 messages should be buffered.
When the consumer stops while some messages are not yet processed, it
should continue processing the messages after a restart.
Here's my config:
<bean id="messageActiveMqQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="example.queue" />
</bean>
<int-jms:message-driven-channel-adapter
destination="messageActiveMqQueue" channel="incomingMessageChannel" />
<int:channel id="incomingMessageChannel">
<int:dispatcher task-executor="incomingMessageChannelExecutor" />
</int:channel>
<bean id="incomingMessageChannelExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="daemon" value="false" />
<property name="maxPoolSize" value="3" />
</bean>
<int:service-activator input-channel="incomingMessageChannel"
ref="myMessageProcessor" method="processMessage" />
The first requirement works as expected. I produce 10 messages and 3
myMessageProcessors start processing a message each. As soon as the 1st message has
finished, the 4th message is processed.
However, when I kill the consumer before all messages are processed, those
messages are lost. After a restart, the consumer does not get those messages
again.
I think in the above configuration that's because the threads generated by the
ThreadPoolTaskExecutor queue the messages. So the messages are already removed
from the incomingMessageChannel. Hence I tried setting the queue capacity of
the incomingMessageChannelExecutor:
<property name="queueCapacity" value="0" />
But now I get error messages when I have more than 3 messages:
2013-06-12 11:47:52,670 WARN [org.springframework.jms.listener.DefaultMessageListenerContainer] - Execution of JMS message listener failed, and no ErrorHandler has been set.
org.springframework.integration.MessageDeliveryException: failed to send Message to channel 'incomingMessageChannel'
I also tried changing the message-driven-channel-adapter to an inbound-gateway,
but this gives me the same error.
Do I have to set an error handler in the inbound-gateway, so that the errors go back to the ActiveMQ queue? How do I have to configure the queue so that the messages are kept in the queue if the ThreadPoolTaskExecutor doesn't have a free thread?
Thanks in advance,
Benedikt

No; instead of using an executor channel, you should be controlling the concurrency with the <message-driven-channel-adapter/>.
Remove the <dispatcher/> from the channel and set concurrent-consumers="3" on the adapter.

Related

consuming message from activemq queue using spring integration

So I have application that sends message to activemq queue with spring integration.
<int-feed:inbound-channel-adapter id="feedAdapter"
channel="feedChannel"
auto-startup="${auto.startup:true}"
url="https://stackoverflow.com/feeds/question/49479712">
<int:poller fixed-rate="10000"/>
</int-feed:inbound-channel-adapter>
<int:channel id="feedChannel"/>
<int:transformer id="transformer" input-channel="feedChannel"
expression="payload.title + payload.author + '#{systemProperties['line.separator']}'"
output-channel="feedOutputChannel"/>
<int:channel id="feedOutputChannel"/>
<jms:outbound-gateway id="jmsOutGateway"
request-destination="inputQueue"
request-channel="feedOutputChannel"
requires-reply="false"/>
But now I want to create different application which consumes message from that queue and just prints it out to console with spring integration. I have made this:
<jms:message-driven-channel-adapter id="JMSInboundAdapter" destination="inputQueue"/>
<bean id="inputQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="input.queue"/>
</bean>
It works when I run application that sends message to queue. But it doesnt when I run message consume application.
Error I get : Dispatcher has no subscribers for channel 'application.JMSInboundAdapter'.
How do I need to configure my message consumer application?
If there is no channel on the adapter, the id becomes the channel name.
You need something to subscribe to that channel (e.g. a <service-activator inputChannel="JMSInboundAdapter" ... />).

ActiveMQ Pending messages

I have a problem with ActiveMQ similar to this one:
http://activemq.2283324.n4.nabble.com/Messages-stuck-in-pending-td4617979.html
and already tried the solution posted here.
Some messages seem to get stuck on the queue and can sit there for literally days without being consumed. I have more than enough consumers that are free most of the time, so it's not an issue of "saturation" of consumers.
Upon restart of the ActiveMQ SOME of the pending messages are consumed right away. Just a moment ago I had situation where I had 25 free consumers avaiable for queue (they are visible in the admin panel) with 7 of those "stuck" messages. Four of them were consumed right away but other 3 are still stuck. The other strange thing is - new messages kept coming to queue and were consumed right away, while the 3 old ones were still stuck.
On the consumer side my config in spring looks as follows:
<jms:listener-container concurrency="${activemq.concurrent.consumers}" prefetch="1">
<jms:listener destination="queue.request" response-destination="queue.response" ref="requestConsumer" method="onRequest"/>
</jms:listener-container>
<bean id="prefetchPolicy" class="org.apache.activemq.ActiveMQPrefetchPolicy">
<property name="queuePrefetch" value="1" />
</bean>
<bean id="connectionFactory" class="org.apache.activemq.spring.ActiveMQConnectionFactory">
<property name="brokerURL" value="${activemq.broker.url}?initialReconnectDelay=100&maxReconnectDelay=10000&startupMaxReconnectAttempts=3"/>
<property name="prefetchPolicy" ref="prefetchPolicy"/>
</bean>
The "stuck" messages are probably considered as "in delivery", restarting the broker will close the connections and, as the message are yet not acknowledged, the broker considers them as not delivered and will deliver them again.
There may be several problem leading to such a situation, most common ones are a problem in transaction / acknowledgment configuration, bad error / acknowledgment management on consumer side (the message is consumed but never acknowledged) or consumer being stuck on an endless operation (for example a blocking call to a third party resource which doesn't respond and there is no timeout handling).

Disgraceful shutdown of Camel Route

How can I shutdown a Camel route context disgracefully?
As soon as I click the button, the Camel route should stop immediately. I don't want any delay.
Each time I do a camelroute.context.stop(), it takes some time to stop, and in that time since the route was active earlier queues and dequeues the messages are sent to the target queue.
I want to stop the route mid-way when I click the desired button.
Is there a way to handle it?
Have a look at the timeout property of the DefaultShutdownStrategy.
Try setting it to zero in your Camel Context:
<bean id="shutdownStrategy" class="org.apache.camel.impl.DefaultShutdownStrategy">
<property name="timeout" value="0"/>
</bean>
The value is in seconds by default.
Also, have a look at Graceful Shutdown in the Camel docs, if you haven't yet.
EDIT 1: The DefaultShutdownStrategy does not allow 0 timeouts. You could try setting it to 1 NANOSECOND which might help:
<bean id="shutdownStrategy" class="org.apache.camel.impl.DefaultShutdownStrategy">
<property name="timeout" value="1"/>
<property name="timeUnit" value="NANOSECONDS" /
</bean>
Alternatively, you can implement your own ShutdownStrategy if it's really important for you to guarantee absolute immediate shutdown.

Task Executor thread is not returned to pool in case of exception

when a nested chain fails and reaches the error channel flow, the task executor threads block and are not returned to pool. Is there any way to indicate that the end of flow has reached and they need to be returned to pool.
for example the splitter breaks the payload to 3 messages. The messages are served as -
message 1 - fileChannelTaskExecutor1
message 2 - fileChannelTaskExecutor2
if "nested-chain" gateway call is successful, the 3rd message is served to any of these executor thread which gets freed up earlier.
However if the "nested-chain" gateway call fails and reaches errChannel, both the above executor thread block and are not returned to pool. As a result of which successive messages(message 3) are not processed as no thread is available in pool.
<bean id="fileChannelTaskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="2"/>
<property name="daemon" value="false"/>
</bean>
<int:channel id="splitterResponseChannel">
<int:dispatcher task-executor="fileChannelTaskExecutor"/>
</int:channel>
<int:splitter input-channel="splitterRequestChannel" output-channel="splitterResponseChannel" >
<int:chain input-channel="splitterResponseChannel">
<int:gateway request-channel="nested-chain" error-channel="errChannel"/>
</int:chain>
<int:chain input-channel="errChannel" output-channel="nullChannel">
.....
</int:chain>
The problem here is around one-way nullChannel and the <int:gateway> request/reply nature.
Even if you send Exception to the error-channel you should should re-throw it to the call or return some compensation message from the error flow.
Other wise you end up with the hanging on the gateway which waits for the reply forever. By default, of course!
You can adjust reply-timeout on the matter and release your Thread to the pool over some time in case of that one-way error process.

Message to Websphere MQ doesn't receive reply (Fuse ESB 7.1)

I'm sending a "RequestSnapshot" to a Websphere-MQ queue which is define by the vendor as 'S' (synchronous) - so I am expecting to receive a reply via the Apache Camel inOut integration pattern, running in Fuse ESB.
from("direct:" + SEND_SUBSCRIPTION)
.routeId(getFinalRouteName(SEND_SUBSCRIPTION))
.log("Sending Request Snapshot request to webspheremq")
.bean(CreateSnapshotRequest.class)
.marshal(myDataFormat)
.convertBodyTo(String.class)
.inOut("webspheremq:queue:SNAPSHOT_REQUESTS.HT") // **1** works, but no Reply!!
.log("RequestSnapshot response: ${body}") // doesnt reach this line
I know that the snapshot request message is transmitted correctly, because after I send it I receive a data on a different MQ topic.
The problem is that I am not receiving a response, the execution stops on the line marked **1** - and after 20 seconds I get a timeout.
org.apache.camel.ExchangeTimedOutException: The OUT message was not received
within: 20000 millis due reply message with correlationID:
Camel-ID-XYZ012-54037-1385986757895-0-3 not received.
Normally, via java code, this would be done differently, but here we are using Fuse ESB and the inOut mechanism should send the message and get a reply on what is called the DYNAMIC REPLY-TO QUEUE.
My understanding of the "reply-to" queue is somewhat limited, I just understand that Fuse ESB should listen for a response on some sort of temporary mq, but that this should work transparently via inOut.
Here is my configuration bean from blueprint.xml:
<bean id="webspheremq" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory">
<bean class="com.ibm.mq.jms.MQConnectionFactory">
<property name="transportType" value="1"/>
<property name="hostName" value="1.2.3.4"/>
<property name="port" value="1417"/>
<property name="channel" value="SOME.CHANNEL"/>
<property name="temporaryModel" value="SOME_MODEL_QUEUE"/>
<property name="CCSID" value="789"/>
<property name="queueManager" value="SOMEGATE"/>
<property name="brokerSubQueue" value="SYSTEM.JMS.ND.MACHINE.USER.*"/>
<property name="brokerControlQueue" value="SOME_SUBSCRIPTION.HT"/>
<property name="brokerQueueManager" value="SOMEHUB"/>
<property name="tempQPrefix" value="MACHINE.USER.*"/>
</bean>
</property>
</bean>
The other question in my mind is firewall ports, we asked to open 1417 and 1499, but only 1417 seems to be open. Plus I do not know if the firewall ports are open in the opposite direction, from WebsphereMQ back to me.
I would be extremely grateful if anyone can offer any advice!
Edit 1:
I tried the replyTo=queue:XYZ suggestion from Claus, and the result was like this:
2013-12-03 14:38:04,636 | WARN | eplyManager[FOO] | entQueueMessageListenerContainer
| ? ? | 153 - org.springframework.jms -
3.0.7.RELEASE | Setup of JMS message listener invoker failed for destination 'FOO'
- trying to recover. Cause: MQJMS2008: failed to open MQ queue ; nested exception is
com.ibm.mq.MQException: MQJE001: Completion Code 2, Reason 2085
You can use named reply queues also. Maybe that works better with WMQ. So in the Camel endpoint uri, you just set replyTo=queue:foobar to use the queue named foobar as the reply queue.
That would set the JMSReplyTo header as foobar as the queue name, so the "other side" would use that for reply queue, instead of a temporary queue name.
See the Camel JMS documentation for more details about request/reply over JMS as there is many options you can configure and tweak: http://camel.apache.org/jms
Also in the past WMQ didn't link long correlation ids, and the option useMessageIDAsCorrelationID can be set to true, to use the message id that WMQ generated for the request message. Otherwise a long random id is generated by Camel.

Categories

Resources