I have a DefaultMessageListenerContainer, which is (in my opinion) not scaling up. The Container is defined to listen on a queue, where 100 messages are located in.
I would expect, that the Container is going to any lengths, that the messages would be consumed as fast as it is possible (by observing the maxConcurrentConsumers configuration). So i would assume, that there are 7 concurrentConsumers. (beginning by 2 concurrentConsumers at container-startup) Some logging-information:
activeConsumerCount: 5
concurrentConsumers: 2
scheduledConsumerCount: 5
idleConsumerLimit: 1
idleTaskExecLimit: 1
maxConcurrentConsumers: 7
My Spring-config (a part of it):
<bean id="abstractMessageListenerContainer" class="my.package.structure.LoggingListenerContainer" abstract="true">
<property name="connectionFactory" ref="jmscfCee" />
<property name="maxConcurrentConsumers" value="7"/>
<property name="receiveTimeout" value="100000" />
<property name="concurrentConsumers" value="2" />
</bean>
<bean class="my.package.structure.LoggingListenerContainer" parent="abstractMessageListenerContainer">
<property name="destinationName" value="MY.QUEUE" />
<property name="messageListener" ref="myMessageListener" />
</bean>
<bean id="myMessageListener" class="my.package.structure.ListenerClass"></bean>
My Logging Container
public class LoggingListenerContainer extends DefaultMessageListenerContainer{
private static final Logger logger = Logger
.getLogger(LoggingListenerContainer.class);
#Override
protected void doInvokeListener(MessageListener listener, Message message)
throws JMSException {
logger.info("activeConsumerCount: " + this.getActiveConsumerCount());
logger.info("concurrentConsumers: " + this.getConcurrentConsumers());
logger.info("scheduledConsumerCount: " + this.getScheduledConsumerCount());
logger.info("idleConsumerLimit: " + this.getIdleConsumerLimit());
logger.info("idleTaskExecLimit: " + this.getIdleTaskExecutionLimit());
logger.info("maxConcurrentConsumers: " + this.getMaxConcurrentConsumers());
super.doInvokeListener(listener, message);
}
My Listener Class:
public class ListenerClass implements MessageListener {
public void onMessage(Message msg) {
//Do some business function
}
}
Could someone be so kind to correct my configuration or give me some tipps concerning my configuration or explain me the approach of the Container? (if i had misunderstood something)
I'm locally testing with ActiveMQ (in Production with WebSphere MQ) - if it's relevant for scalability topics.
EDIT:
<bean id="jmscfCee" class="org.apache.activemq.spring.ActiveMQConnectionFactory">
<property name="brokerURL">
<value>${jmscfCee.hostName}</value>
</property>
</bean>
<bean id="jmscfCeeCachingConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory ">
<constructor-arg ref="jmscfCee" />
<property name="sessionCacheSize" value="10" />
</bean>
It depends. I had a similar issue with ActiveMQ a few years back, whereby its default behaviour is heavily opimized towards high volumes (many thousands) of small messages. By default each consumer will pre-fetch messages in batches of 1000, so if you have small numbers of messages you'll probably find they have all ended up in the pre-fetch buffer of one consumer, leaving the other consumers idle.
You can tune this behaviour using a prefetch policy, either on the connection URI or in the Spring configuration if that's how you're building your connection factory.
<amq:connectionFactory id="connectionFactory" brokerURL="vm://localhost">
<property name="prefetchPolicy">
<amq:prefetchPolicy all="1" />
</property>
</amq:connectionFactory>
The version of ActiveMQ I was using at the time didn't support a prefetch limit of 0 (i.e. don't prefetch, just go to the broker every time) but the documentation suggests that this is now allowed.
Related
We are implementing XA transaction between MQ and database and trying to create a connection factory as a service in karaf as per the below link.
https://access.redhat.com/documentation/fr-fr/red_hat_fuse/7.2/html/apache_karaf_transaction_guide/using-jms-connection-factories#manual-deployment-connection-factories
The MQ we are using is IBM and we are connecting to it through camel.
The karaf service is exposed from the same bundle that is going to use it. This is done through blueprint xml file present in the src/main/resources/OSGI-INF/blueprint folder.
When we use (through JNDI) the connection factory exposed as a service for setting the connection factory to be used by the JmsComponent of camel, we are able to get message from the queue but not able to put message into the queue. There is no error when the put operation fails and hence, the database gets updated with success. This happens specifically when using JmsPoolXAConnectionFactory as the pool connection factory. If we change it to JmsPoolConnectionFactory, the put operation works and the message is added to the queue.
Below are the sample routes for get and put to queue.
GET:
from("mq:queue:{{queueName}}")
.process(new CustomProcessor1())
.to("direct:call-sp")
.end();
from("direct:call-sp")
.to("sql-stored:call-sp")
.end();
PUT:
from("vm:send")
.process(new CustomProcessor2())
.to("mq:queue:{{queueName}}")
.to("sql-stored:update-sp")
.to("vm:nextroute")
.end();
Camel JmsComponent Configuration in camel-context.xml:
<reference id="ptm" interface="org.springframework.transaction.PlatformTransactionManager" />
<reference id="connectionFactory" interface="javax.jms.ConnectionFactory" filter="(osgi.jndi.service.name=jms/mq)" availability="optional" />
<bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
<property name="transacted" value="false" />
<property name="connectionFactory" ref="connectionFactory" />
<property name="transactionManager" ref="ptm" />
</bean>
<bean id="mq" class="org.apache.camel.component.jms.JmsComponent">
<property name="configuration" ref="jmsConfig" />
<property name="destinationResolver" ref="customDestinationResolver" />
</bean>
<bean id="customDestinationResolver" class="com.example.CustomDestinationResolver">
</bean>
Is there any put related specific configuration that we are missing?
To coordinate XA transactions, you need a transaction manager which implements the Java Transaction API (JTA).
Therefore, I think you need to use a JtaTransactionManager rather than a org.springframework.transaction.PlatformTransactionManager.
Check this out:
https://tomd.xyz/camel-xa-transactions-checklist/
Below is the config
<jms:outbound-channel-adapter id="someId" channel="inputChannel"
connection-factory="${connection.factory}" destination="queue">
<jms:request-handler-advice-chain>
<bean class="org.springframework.integration.handler.advice.ExpressionEvaluatingRequestHandlerAdvice">
<property name="onSuccessExpression" value="T(Boolean).TRUE"/>
<property name="successChannelName" value="afterSuccessDeliveryMessageChannel"/>
<property name="onFailureExpression" value="T(Boolean).FALSE"/>
<property name="failureChannelName" value="failureChannel"/>
</bean>
<bean id="retryWithBackoffAdviceSession"
class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice">
<property name="retryTemplate">
<bean class="org.springframework.retry.support.RetryTemplate">
<property name="retryPolicy">
<bean class="org.springframework.retry.policy.SimpleRetryPolicy">
<property name="maxAttempts" value="5"/>
</bean>
</property>
</bean>
</property>
<property name="recoveryCallback">
<bean class="org.springframework.integration.handler.advice.ErrorMessageSendingRecoverer">
<constructor-arg ref="failureChannel"/>
</bean>
</property>
</bean>
</jms:request-handler-advice-chain>
</jms:outbound-channel-adapter>
I am retrying message 5 times and then using recoveryCallBack logging message to some DB.
It works fine retry 5 times and call failureChannel channel but once it calls the failureChannel then again it pass to afterSuccessDeliveryMessageChannel.
I am not sure what I am doing wrong here?
I am expecting once it failed it should go failedChannel NOT again back to afterSuccessDeliveryMessageChannel.
Your problem is like this:
Your Retry advice sends to the failureChannel from the recoveryCallback and exits, successfully.
Then when we look into the ExpressionEvaluatingRequestHandlerAdvice code, we see this logic:
try {
Object result = callback.execute();
if (this.onSuccessExpression != null) {
evaluateSuccessExpression(message);
}
return result;
}
So, since there is no exception calling callback, it goes to the successChannel configured.
To make it fail and go to the failureChannel configured, you should consider to not use that recoveryCallback. Then RequestHandlerRetryAdvice will throw an exception which is going to be caught by the ExpressionEvaluatingRequestHandlerAdvice and sent to that failureChannel. There won't be onSuccessExpression evaluation since we will end up with an exception.
I have a standalone application using Spring to connect to TIBCO (queues). Occasionally, for various reasons, TIBCO connection is closed by the server. Most of the things are recovering from this. However, sometimes JmsTemplate is not able to send response because of the error below. I have a retry in place but the same error keeps coming (see trace below).
Details that may be important:
I am using DefaultMessageListenerContainer to get the request and send the response in that receiving thread. Also, I am using the same connection factory for both DefaultMessageListenerContainer and JmsTemplate.
Caused by: org.springframework.jms.IllegalStateException: Session is closed; nested exception is javax.jms.IllegalStateException: Session is closed
at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:279)
at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:169)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:487)
at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:559)
at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:682)
at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:670)
at org.springframework.integration.jms.JmsSendingMessageHandler.send(JmsSendingMessageHandler.java:149)
at org.springframework.integration.jms.JmsSendingMessageHandler.handleMessageInternal(JmsSendingMessageHandler.java:116)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
... 83 more
Caused by: javax.jms.IllegalStateException: Session is closed
at com.tibco.tibjms.TibjmsxSessionImp._createProducer(TibjmsxSessionImp.java:1067)
at com.tibco.tibjms.TibjmsxSessionImp.createProducer(TibjmsxSessionImp.java:5080)
at org.springframework.jms.core.JmsTemplate.doCreateProducer(JmsTemplate.java:1114)
at org.springframework.jms.core.JmsTemplate.createProducer(JmsTemplate.java:1095)
at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:591)
at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:562)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:484)
... 89 more
Communication with the TIBCO queue is done using Spring framework. Here is the configuration. A message is received by DefaultMessageListenerContainer, processed and JmsTemplate is used to send back the response. Connection factory is shared between receiver and sender (can this be an issue?).
<bean id="connectionFactory"
class="org.springframework.jms.connection.SingleConnectionFactory">
<constructor-arg ref="tibcoJNDI" />
<property name="targetConnectionFactory">
<bean class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiTemplate" ref="tibcoJNDI"/>
<property name="jndiName" value="${tibco.queueConnectionFactory}" />
</bean>
</property>
<property name="reconnectOnException" value="true"/>
</bean>
<bean id="client.req.msg.lstnr" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="autoStartup" value="false"/>
<property name="connectionFactory" ref="connectionFactory"/>
<property name="destination" ref="ext_client.request.queue"/>
<property name="sessionAcknowledgeMode" value="3"/>
<property name="concurrentConsumers" value="6"/>
<property name="receiveTimeout" value="60000"/>
</bean>
<jms:outbound-channel-adapter
jms-template="ext.outbound.jms.template"
channel="jms.to.ext.clnt.reply"/>
<bean id="ext.outbound.jms.template" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="defaultDestination" ref="ext_client.reply.queue"/>
<property name="timeToLive" value="10800000" />
<property name="explicitQosEnabled" value="true" />
</bean>
One more detail that might help. I just noticed that the first exception is actually different. There is a "Connection is closed" exception first followed by multiple "Session is closed" exception (on retry).
Caused by: javax.jms.JMSException: Connection is closed
at com.tibco.tibjms.TibjmsxLink.sendRequest(TibjmsxLink.java:322)
at com.tibco.tibjms.TibjmsxLink.sendRequest(TibjmsxLink.java:286)
at com.tibco.tibjms.TibjmsxLink.sendRequestMsg(TibjmsxLink.java:261)
at com.tibco.tibjms.TibjmsxSessionImp._createProducer(TibjmsxSessionImp.java:1075)
at com.tibco.tibjms.TibjmsxSessionImp.createProducer(TibjmsxSessionImp.java:5080)
at org.springframework.jms.core.JmsTemplate.doCreateProducer(JmsTemplate.java:1114)
at org.springframework.jms.core.JmsTemplate.createProducer(JmsTemplate.java:1095)
at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:591)
at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:562)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:484)
... 89 more
It seems that Spring does not handle properly this case. I solved this issue by overriding JmsTemplate (code below) and handling the exception myself (cleaning the session and the connection). I hope this helps.
public <T> T execute(SessionCallback<T> action, boolean startConnection) throws JmsException {
try {
return super.execute(action, startConnection);
} catch (JmsException jmse) {
logger.error("Exception while executing in JmsTemplate (will cleanup session & connection): ", jmse);
Object resourceHolder =
TransactionSynchronizationManager.getResource(getConnectionFactory());
if (resourceHolder != null && resourceHolder instanceof JmsResourceHolder) {
((JmsResourceHolder)resourceHolder).closeAll();
}
throw jmse;
}
}
This answer might help https://stackoverflow.com/a/24494739/208934 In particular:
When using JMS you shouldn't really cache the JMS Session (and
anything hanging of that such as a Producer). The reason being is that
the JMS Session is the unit of work within JMS and so should be a
short lived object. In the Java EE world that JMS Session might also
be enlisted with a global transaction for example and so needs to be
scoped correctly.
It sounds like you're reusing the same session for multiple operations.
I have seen this problem in some of my tests and telling Spring to use a pool of connections usually fixes it. If you were using active-mq then it can be done with the property spring.activemq.pooled=true. I'm not 100% how Tibco accomplishes the same thing.
You might get get same results by defining a PooledConnectionFactory bean that will override the default.
I currently have multiple Quartz pollers that check a multiple flags in the database and fire a service if flags are true and I also have one spring batch project reading files, then calling a service to update the information accordingly.
What i am wondering is if it is possible to get the spring batch application to only call a service very 5 minutes (dont check for any information just call a service which will handle all the business logic) so that i can remove the Quartz jobs from my application all my jobs are in one place (spring batch project)?
If so can any point me in the right direction on how to do this as i cannot find anything online to just set up a spring batch job just to fire a service?
You can simply schedule a Quartz job and call your service from executeInternal method of your schedular class. You will not be able to autowire your service directly in schedular class, so you need to set it using jobDataas Map.
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<list>
<bean id="cronTriggerCache"
class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="exampleJob" />
<property name="cronExpression" value="your cron expression" />
</bean>
</list>
</bean>
<bean name="exampleJob" class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="jobClass" value="example.ExampleJob"/>
<property name="jobDataAsMap">
<map>
<entry key="myService" ref="myService"/>
</map>
</property>
</bean>
package example;
public class ExampleJob extends QuartzJobBean {
private myService myService;
/**
* have getter setter for you service
*
*/
protected void executeInternal(JobExecutionContext ctx) throws JobExecutionException {
// do the actual work
//call your service method here
}
}
I have set up an spring http invoker example as described here http://static.springsource.org/spring/docs/3.1.x/spring-framework-reference/html/remoting.html in section 20.4
If i do several service calls in a row (see my for-loop), between the single calls is one second though the server processes the method in less than 4ms.
Any ideas.
Stefan
Here the config and call:
<!-- server side -->
<bean name="configurationServiceExporter"
class="org.springframework.remoting.httpinvoker.SimpleHttpInvokerServiceExporter">
<property name="service" ref="configurationService" />
<property name="serviceInterface"
value="remote.service.ConfigurationService" />
</bean>
<bean id="httpServer"
class="org.springframework.remoting.support.SimpleHttpServerFactoryBean">
<property name="contexts">
<util:map>
<entry key="/remoting/ConfigurationService" value-ref="configurationServiceExporter" />
</util:map>
</property>
<property name="port" value="${port.httpinvoker}" />
</bean>
<!-- client side -->
<bean id="configurationServiceProxy" class="org.springframework.remoting.httpinvoker.HttpInvokerProxyFactoryBean">
<property name="serviceUrl"
value="http://localhost:7777/remoting/ConfigurationService" />
<property name="serviceInterface"
value="remote.service.ConfigurationService" />
</bean>
/** here the service call*/
#Component
public class ServiceConsumer {
private ConfigurationService configurationService;
public void do(){
for (int i = 0; i < 10; i++)
this.configurationService.getConfigurationValue(SMTP_HOST);
}
I just encountered the very same problem:
Spring Remoting
Delays of "exactly" 1 second
Java 7
Unfortunately, I was unable to find a reason for this odd behavior, but there exists a workaround: use jetty instead of the SimpleHttpServerFactoryBean.
This boils down to changing the xml configuration a little, how exactly is described here.
The delays disappeared; firing requests even seems to have sped up compared to using SimpleHttpServerFactoryBean in Java 6.
Found the Problem. It was not connected to Spring HTTP Invoker. I updated to Java 7. When i ran my app with Java 6 it works as before the update (without waiting one second between the requests. If i knew anything more, i come back.