Develop Reference

ThreadPoolTaskExecutor creates new threadPool for each new request

We have a spring application deployed on weblogic server. We have a REST webservice in our project. It is a POST call. In the request body of this webservice, a name comes. When our webservice is being called, internally it calls other webservice using RestTemplate.
Now we need to pass more than 10 names in each request, and in turn our webservice should call the other web service 10 times. We need to make it a multi-threaded call.
So we are using ThreadPoolTaskExecutor. The following is the code for the same.
<bean id="threadPoolTaskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="3"></property>
<property name="maxPoolSize" value="4"></property>
<property name="WaitForTasksToCompleteOnShutdown" value="true">
</property>
</bean>
public class ApplicationDirectorImpl{
#Autowired
private ThreadPoolTaskExecutor threadPoolTaskExecutor;
....
....
public CustomResponseObject method(){
....
....
List<Future<FutureCustomResponse>> fList = new ArrayList<Future<FutureCustomResponse>>();
for(String name : nameList){
Future<FutureCustomResponse> fut = threadPoolTaskExecutor.submit(new Task(name));
fList.add(fut);
}
for(Future f: fList){
FutureCustomResponse fuResponse = f.get();
}
}
}
public class Task implements Callable<FutureCustomResponse>{
private String name;
public call() throws Exception{
System.out.println("name" + name + " performed by " + Thread.currentThread().getName());
return NameBuild.getPersonalInfo(this.name);
}
}
Now when a request comes with 5 names, it should be serviced by these 3 threads.
We have 2 servers, and so there will be 2 instances of our application. So the threadpool will be created for each server or common between 2 servers.
So in total how many threads will be created? 3 per server or 3 per application?

Error when upgarding Spring-AMQP from 1.2.0 to 1.3.0

Our application built and runs on :-
spring.version = 3.2.8.RELEASE
spring.amqp.version = 1.2.0.RELEASE
rabbitmq.version = 3.1.3 (client)
RabbitMQ Server version is 3.1.5
We wanted to upgrade the rabbitmq server from 3.1.5 to 3.3.5 and we did that successfully.
Now we wanted to upgrade the application to use latest version of spring-amqp, RabbitMQ java client, so we have upgrade the following components :-
spring.version = 3.2.8.RELEASE
spring.amqp.version = 1.3.0.RELEASE
rabbitmq.version = 3.2.4 (client)
RabbitMQ Server version is 3.3.5
However after upgrading to spring-amqp to 1.3.0, our application started to hung. Basically we start many listener containers during the application startup, and starting each listener container now takes exactly 60 secs to allow to the next step
after digging deep I found that, the program gets hanging in run() method in SimpleMessageListenerContainer class :-
org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer
public void run() {
boolean aborted = false;
int consecutiveIdles = 0;
int consecutiveMessages = 0;
try {
try {
SimpleMessageListenerContainer.this.redeclareElementsIfNecessary(); // Here is where the programing thread hangs.
this.consumer.start();
this.start.countDown();
}
As mentioned above code, the thread gets hung at redeclareElementsIfNecessary() method, and this method is introduced in this version of spring-rabbit only. I have no clue why its getting hanging there, whatever suggested parameter's that I pass to this SimpleMessageListenerContainer it doesn't seems to be working.
If I revert back to spring-amqp 1.2.0 release with the new RabbitMQ server 3.3.5 all seems to be working fine, but things are not working with new spring-amqp client.
I am kind of stuck here for couple of days now. Spring / Rabbitmq master's out there, can you please help me solving this problem?
Thanks for your quick response, however it seems the code is not reaching to that point and it just get hung just above the snippet that you have provide,
I have comment below on where the code gets hung exactly below
Set<String> queueNames = this.getQueueNamesAsSet();
Map<String, Queue> queueBeans = ((ListableBeanFactory) applicationContext).getBeansOfType(Queue.class); // The code started to hung here
for (Entry<String, Queue> entry : queueBeans.entrySet()) {
Queue queue = entry.getValue();
if (queueNames.contains(queue.getName()) && queue.isAutoDelete()
&& this.rabbitAdmin.getQueueProperties(queue.getName()) == null) {
if (logger.isDebugEnabled()) {
logger.debug("At least one auto-delete queue is missing: " + queue.getName()
+ "; redeclaring context exchanges, queues, bindings.");
}
this.rabbitAdmin.initialize();
break;
}
}
Actually we have upgraded to the latest version of spring-amqp only, that is
spring.version = 3.2.8.RELEASE
spring.amqp.version = 1.3.6.RELEASE
rabbitmq.version = 3.3.4 (client)
RabbitMQ Server version is 3.3.5
however we faced exactly the same issue, so just to find out from which version the issue started I ran down to the lower versions upto 1.3.0, seems the issue is starting in 1.3.0 version of spring-amqp itself. that's the reason.
I have attached the requested information including thread dumps which are based on spring-amqp 1.3.6 only.
Here is the configuration our listener container where the programs hangs, as you can see we have our own SimpleMessageLinstenerContainer which acts as a wrapper for the acutal spring's SimpleMessageListenerContainer,
I have also attached this custom wrapper file for your reference.
<bean id="tlogOutOfCycleMessageListenerPrototype" class="com.myorg.ips.cnccommon.support.amqp.SimpleMessageListenerContainer" scope="prototype">
<property name="channelTransacted" value="true" />
<property name="transactionManager" ref="transactionManager" />
<property name="concurrentConsumers" value="1" />
<property name="taskExecutor" ref="tlogOutOfCycleMessageListenerPool" />
<property name="messageListener" ref="tlogMLAOutOfCycle" />
<property name="errorHandler" ref="tlogOutOfCycleMessageHandler" />
<property name="autoStartup" value="false" />
<property name="instanceNameForLogging" value="site1TlogOutOfCycleMessageListener"/>
<!-- A dummy connection factory which will never be used -->
<property name="connectionFactory" ref="switchCompositeConnectionFactoryPrototype"/>
</bean>
Our wrapper class SimpleMessageListenerContainer.java
package com.myorg.ips.cnccommon.support.amqp;
import org.apache.commons.lang.builder.ToStringBuilder;
import org.slf4j.cal10n.LocLogger;
import org.springframework.util.ErrorHandler;
import com.myorg.ips.amqp.SwitchSiteSupport;
import com.myorg.ips.logging.LoggerFactory;
import com.myorg.ips.system.config.InitialisableSiteAware;
import static com.myorg.ips.logging.SystemWideLogMessages.ERROR_AMQP_FAILED_TO_START_LISTENER;
import static com.myorg.ips.logging.SystemWideLogMessages.INFO_AMQP_STOPPING_LISTENER;
/**
*
* Wrapper for the Spring SimpleMessageListenerContainer which simply allows us to delay (or prevent startup). Can also restart on command.
*
*/
public class SimpleMessageListenerContainer extends org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer implements InitialisableSiteAware {
private static final LocLogger logger = LoggerFactory.getLogger(SimpleMessageListenerContainer.class);
private boolean autoStart = true;
private ErrorHandler exposedErrorHandler;
private boolean springBeanInitialisationAttempted = false;
private boolean springBeanInitialised = false;
private String instanceNameForLogging;
#Override
public void initialize() {
// Do nothing -- we will instead perform the Spring bean initialisation later on via the factory bean, after the connection factory has been set
springBeanInitialisationAttempted = true;
springBeanInitialised = false;
}
#Override
public void initialise() {
SwitchSiteSupport.initialiseIfSiteAware(getMessageListener());
SwitchSiteSupport.initialiseIfSiteAware(getErrorHandler());
// If this object is a Spring bean, we should now complete the initialisation that the Spring framework attempted earlier
if (springBeanInitialisationAttempted && !springBeanInitialised) {
springBeanInitialised = true;
super.initialize();
if (isAutoStartup()) {
start();
}
}
}
#Override
public void configureForSite(final MultiHostConnectionFactory configuredConnectionFactory) {
setConnectionFactory(configuredConnectionFactory);
SwitchSiteSupport.configureIfSiteAware(getMessageListener(), configuredConnectionFactory);
SwitchSiteSupport.configureIfSiteAware(getErrorHandler(), configuredConnectionFactory);
setInstanceNameForLogging(SwitchSiteSupport.replaceWithSiteAlias(instanceNameForLogging, configuredConnectionFactory));
}
#Override
//CHECKSTYLE:OFF Unfortunately the parent springframework class throws and exception, so so do we
protected void doStart() throws Exception {
//CHECKSTYLE:ON
if (autoStart) {
logger.debug("Starting message listener " + instanceNameForLogging);
super.doStart();
logger.debug("Started message listener " + instanceNameForLogging);
}
}
/**
* Start this listener
*/
public void start() {
autoStart = true;
try {
doStart();
//CHECKSTYLE:OFF Unfortunately the parent springframework class throws and exception, so that is what we catch
} catch (Exception e) {
//CHECKSTYLE:ON
logger.error(ERROR_AMQP_FAILED_TO_START_LISTENER, e);
}
}
/**
* Stop listener
*/
public void stop() {
logger.info(INFO_AMQP_STOPPING_LISTENER, getBeanName());
autoStart = false;
doStop();
}
/**
* Stop and start this listener
*/
public void restart() {
stop();
start();
}
/**
* Store the errorHandler in a subclass-specific property so that we can retrieve it later
* #param errorHandler errorHandler
*/
#Override
public void setErrorHandler(final ErrorHandler errorHandler) {
this.exposedErrorHandler = errorHandler;
super.setErrorHandler(errorHandler);
}
/**
* Return the exposed errorHandler
* #return errorHandler
*/
public ErrorHandler getErrorHandler() {
return exposedErrorHandler;
}
public void setInstanceNameForLogging(final String instanceNameForLogging) {
this.instanceNameForLogging = instanceNameForLogging;
}
#Override
public String toString(){
return ToStringBuilder.reflectionToString(this);
}
}

Good catch anyway!
I'm just right now working with that code for Spring AMQP 1.4.
Would you mind sharing:
The configuration for the ListenerContainer on which you hang
A debug analiz for the code from that redeclareElementsIfNecessary()
Actually now that code looks like:
if (queueNames.contains(queue.getName()) && queue.isAutoDelete()
&& this.rabbitAdmin.getQueueProperties(queue.getName()) == null) {
if (logger.isDebugEnabled()) {
logger.debug("At least one auto-delete queue is missing: " + queue.getName()
+ "; redeclaring context exchanges, queues, bindings.");
}
this.rabbitAdmin.initialize();
break;
}
So, it may heppen only on the auto-delete Queue.
Or do you have another picture?..
UPDATE
According to your ThreadDump. This is illegal:
at com.vocalink.ips.amqp.AmqpMessageListenerManager.initialise(AmqpMessageListenerManager.java:106)
at com.vocalink.ips.amqp.SwitchSiteSupport.initialiseIfSiteAware(SwitchSiteSupport.java:29)
at com.vocalink.ips.system.config.AbstractSiteAwareComponentCachingFactory.createAndConfigureSiteAwareComponent(AbstractSiteAwareComponentCachingFactory.java:51)
You can't start component within initialization phase. Or leave it to the container, or just do start manually somewhere at runtime, when all of your beans are already created.
For example you can do that using ApplicationListener<ContextRefreshedEvent>.

wrap spring bean with another bean

I have some service bean which is accessible by identifier someSpecificService which I need to modify.
Beans are defined in different xml files and are collected together in runtime. So one big xml file is created where all these xmls are imported:
context.xml
....
<import path="spring1.xml" />
<import path="spring2.xml" />
...
So there is following configuration:
<-- definitions from spring1.xml -->
<alias name="defaultSomeSpecificService" alias="someSpecificService" />
<bean id="defaultSomeSpecificService" class="..."/>
....
<!-- definitions from spring2.xml -->
<alias name="myOwnSomeSpecificService" alias="someSpecificService" />
<bean id="myOwnSomeSpecificService" class="..." /> <!-- how to inject previously defined someSpecificService into this new bean? -->
I would like to override someSpecificService from spring1.xml in spring2.xml, however I do need to inject previously defined bean defaultSomeSpecificService and all I know is its alias name someSpecificService which I need to redefine to new bean myOwnSomeSpecificService.
Is it possible to implement?

One solution would be to avoid trying to override the definition, by creating a proxy for the service implementation to intercept all calls towards it.
1) For the sake of the example, suppose the service would be something like:
public interface Service {
public String run();
}
public class ExistingServiceImpl implements Service {
#Override
public String run() {
throw new IllegalStateException("Muahahahaha!");
}
}
2) Implement an interceptor instead of myOwnSomeSpecificService:
import org.aopalliance.intercept.MethodInterceptor;
import org.aopalliance.intercept.MethodInvocation;
public class SomeSpecificServiceInterceptor implements MethodInterceptor {
#Override
public Object invoke(MethodInvocation invocation) throws Throwable {
String status;
try {
// allow the original invocation to actually execute
status = String.valueOf(invocation.proceed());
} catch (IllegalStateException e) {
System.out.println("Existing service threw the following exception [" + e.getMessage() + "]");
status = "FAIL";
}
return status;
}
}
3) In spring2.xml define the proxy creator and the interceptor:
<bean id="serviceInterceptor" class="com.nsn.SomeSpecificServiceInterceptor" />
<bean id="proxyCreator" class="org.springframework.aop.framework.autoproxy.BeanNameAutoProxyCreator">
<property name="beanNames" value="someSpecificService"/>
<property name="interceptorNames">
<list>
<value>serviceInterceptor</value>
</list>
</property>
</bean>
4) Running a small example such as:
public class Main {
public static void main(String[] args) {
Service service = new ClassPathXmlApplicationContext("context.xml").getBean("someSpecificService", Service.class);
System.out.println("Service execution status [" + service.run() + "]");
}
}
... instead of the IllegalStateException stacktrace you'd normally expect, it will print:
Existing service threw the following exception [Muahahahaha!]
Service execution status [FAIL]
Please note that in this example the service instance is not injected in the interceptor as you asked because I had no user for it. However should you really need it, you can easily inject it via constructor/property/etc because the interceptor is a spring bean itself.

Property is not found from properties file using #Value

I use properties file in spring framework
root-context.xml
<context:property-placeholder location="classpath:config.properties" />
<util:properties id="config" location="classpath:config.properties" />
java code
#Value("#{config[ebookUseYN]}")
String EBOOKUSEYN;
when Using url call(#RequestMapping(value="/recommendbooks" , method=RequestMethod.GET, produces="application/json;charset=UTF-8")).. this work!
but, i use method call,
public void executeInternal(JobExecutionContext arg0) throws JobExecutionException {
IndexManageController indexManage = new IndexManageController();
CommonSearchDTO commonSearchDTO = new CommonSearchDTO();
try {
if("Y".equals(EBOOKUSEYN)){
indexManage.deleteLuceneDocEbook();
indexManage.initialBatchEbook(null, commonSearchDTO);
}
indexManage.deleteLuceneDoc(); <= this point
indexManage.deleteLuceneDocFacet();
indexManage.initialBatch(null, commonSearchDTO);
}catch (Exception e) {
e.printStackTrace();
}
}
when 'this point ' method call, changing controller, and don't read properties file field..
#Value("#{config[IndexBasePath]}")
String IndexBasePath;
#RequestMapping(value="/deleteLuceneDoc" , method=RequestMethod.GET, produces="application/json;charset=UTF-8")
public #ResponseBody ResultCodeMessageDTO deleteLuceneDoc()
throws Exception
{
long startTime = System.currentTimeMillis();
ResultCodeMessageDTO result = new ResultCodeMessageDTO();
System.out.println(IndexBasePath);
}
It doesn't read IndexBasePath

In your code you are creating a new instance of the IndexManageController, Spring doesn't know this instance and as such it will never be processed.
public void executeInternal(JobExecutionContext arg0) throws JobExecutionException {
IndexManageController indexManage = new IndexManageController();
Instead of creating a new instance inject the dependency for the IndexManageController so that it uses the pre-configured instance constructed and managed by Spring. (And remove the line which constructs a new instance of that class).
public class MyJob {
#Autowired
private IndexManageController indexManage;
}
Your configuration is also loading the properties twice
<context:property-placeholder location="classpath:config.properties" />
<util:properties id="config" location="classpath:config.properties" />
Both load the config.properties file. Simply wire the config to the property-placeholder element.
<context:property-placeholder properties-ref="config"/>
<util:properties id="config" location="classpath:config.properties" />
Saves you loading twice and saves you another bean.

Spring Batch ItemReader list processed only once

I'm trying to create a Spring Batch job using a ListItemReader<String>, ItemProcessor<String, String> and ItemWriter<String>.
The XML looks like the following,
<job id="sourceJob" xmlns="http://www.springframework.org/schema/batch">
<step id="step1" next="step2">
<tasklet>
<chunk reader="svnSourceItemReader"
processor="metadataItemProcessor"
writer="metadataItemWriter"
commit-interval="1" />
</tasklet>
</step>
<step id="step2">
<tasklet ref="lastRevisionLoggerTasklet"></tasklet>
</step>
</job>
<bean id="svnSourceItemReader"
class="com.example.repository.batch.SvnSourceItemReader"
scope="prototype">
<constructor-arg index="0">
<list>
<value>doc1.xkbml</value>
<value>doc2.xkbml</value>
<value>doc3.xkbml</value>
</list>
</constructor-arg>
</bean>
<bean id="metadataItemProcessor"
class="com.example.repository.batch.MetadataItemProcessor"
scope="prototype" />
<bean id="metadataItemWriter"
class="com.example.repository.batch.MetadataItemWriter"
scope="prototype" />
The reader, processor and writer are vanilla,
public class SvnSourceItemReader extends ListItemReader<String> {
public SvnSourceItemReader(List<String> list) {
super(list);
System.out.println("Reading data list " + list);
}
#Override
public String read() {
String out = (String) super.read();
System.out.println("Reading data " + out);
return out;
}
}
public class MetadataItemProcessor implements ItemProcessor<String, String> {
#Override
public String process(String i) throws Exception {
System.out.println("Processing " + i + " : documentId " + documentId);
return i;
}
}
public class MetadataItemWriter implements ItemWriter<String> {
#Override
public void write(List<? extends String> list) throws Exception {
System.out.println("Writing " + list);
}
}
The job is started like this, but on a schedule of every 10 seconds.
long nanoBits = System.nanoTime() % 1000000L;
if (nanoBits < 0) {
nanoBits *= -1;
}
String dateParam = new Date().toString() + System.currentTimeMillis()
+ "." + nanoBits;
param = new JobParametersBuilder().addString("date", dateParam)
.toJobParameters();
JobExecution execution = jobLauncher.run(job, param);
When the application starts, I see it read, process and write each of the three items in the list passed to the reader.
Reading data doc1.xkbml
Processing doc1.xkbml : documentId doc1
Writing [doc1.xkbml]
Reading data doc2.xkbml
Processing doc2.xkbml : documentId doc2
Writing [doc2.xkbml]
Reading data doc3.xkbml
Processing doc3.xkbml : documentId doc3
Writing [doc3.xkbml]
Because this sourceJob is on a scheduled timer, every 10 seconds I expected to see that list processed, but instead I see on all subsequent runs.
Reading data null
Does anyone know why this is happening? I'm new to Spring Batch and just can't get my hands around the issue.
Thanks /w

The problem is that you marked your reader as scope="prototype". It should be scope="step".
In Spring-batch there are only two scopes: singleton (the default) and step.
From the javadoc:
StepScope: Scope for step context. Objects in this scope use the
Spring container as an object factory, so there is only one instance
of such a bean per executing step. All objects in this scope are
(no need to decorate the bean definitions).
and
Using a scope of Step is required in order to use late binding since
the bean cannot actually be instantiated until the Step starts, which
allows the attributes to be found.
During the Spring context startup look at your log and you will see this line:
INFO: Done executing SQL script from class path resource
[org/springframework/batch/core/schema-hsqldb.sql] in 9 ms.
Reading data list [doc1.xkbml, doc2.xkbml, doc3.xkbml]
as you can see your reader has already been created and managed as a singleton; dynamic beans in spring-batch context should be managed with the special step scope so that Spring will create a fresh copy of the bean every time a step is executed.
In your reader, ListItemReader.read() is written as:
public T read() {
if (!list.isEmpty()) {
return list.remove(0);
}
return null;
}
In each read items are removed from original list! The reader is constructed once and, on second job execution, the list is empty!

Just an additional information: you can also use JavaConfig instead of the xml config file, and annotate the reader bean declaration with #StepConfig.
ex:
#Configuration
#EnableBatchProcessing
public class MyConfig {
...
#Bean
#StepScope
public ItemReader<HeadingBreakevenAssociation> readerHeadingBreakevenAssociationList(){
ItemReader<Person> itemReader = new ListItemReader<Person>(myList);
return itemReader;
}
}