Develop Reference

Mule / Spring transaction is not propagated

I have a problem with database transactions in mule flow. This is the flow that i have defined:
<flow name="createPortinCaseServiceFlow">
<vm:inbound-endpoint path="createPortinCase" exchange-pattern="request-response">
<custom-transaction action="ALWAYS_BEGIN" factory-ref="muleTransactionFactory"/>
</vm:inbound-endpoint>
<component>
<spring-object bean="checkIfExists"/>
</component>
<component>
<spring-object bean="createNewOne"/>
</component>
</flow>
The idea is that in checkIfExists we verify if some data exists (in the database) if it does we throw an exception. If it does not we go to createNewOne and create a new data.
The problem
is that if we run the flow concurrently new objects will be created multiple times in createNewOne and they should not be as we invoke checkIfExists just before it. This means that the transaction is not working properly.
More info:
both createNewOne and checkIfExists have following annotation:
#Transactional(propagation = Propagation.MANDATORY)
Definition of muleTransactionFactory looks as follows
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="dataSource" ref="teleportNpDataSource"/>
<property name="entityManagerFactory" ref="npEntityManagerFactory"/>
<property name="nestedTransactionAllowed" value="true"/>
<property name="defaultTimeout" value="${teleport.np.tm.transactionTimeout}"/>
</bean>
<bean id="muleTransactionFactory" class="org.mule.module.spring.transaction.SpringTransactionFactory">
<property name="manager" ref="transactionManager"/>
</bean>
I have set the TRACE log level (as #Shailendra suggested) and i have discovered that transaction is reused in all of the spring beans:
00:26:32.751 [pool-75-thread-1] DEBUG org.springframework.orm.jpa.JpaTransactionManager - Participating in existing transaction
In the logs transaction is commited at the same time which means that those transactions are created properly but there are executed concurrently which causes an issue.

Issue may be because multi-threading. When you post multiple objects to VM, they will be dispatched to multiple receiving threads and if multi-threading is not properly handled in your components then you might run into the issue you mentioned.
Test making these change - Add VM connector reference and turn off the dispatcher threading profile. That way, VM will process messages one at a time as there is just one dispatcher thread.
<vm:connector name="VM" validateConnections="true" doc:name="VM" >
<dispatcher-threading-profile doThreading="false"/>
</vm:connector>
<flow name="testFlow8">
<vm:inbound-endpoint exchange-pattern="one-way" doc:name="VM" connector-ref="VM">
<custom-transaction action="NONE"/>
</vm:inbound-endpoint>
</flow>
Be aware of the fact that, if the number of incoming messages on VM are very high and time taken to process each message is more, then you may run into SEDA-QUEUE errors due to no thread availability.
If without threading, your flow is behaving correctly then you may need to look how your components should behave in multi-threading.
Hope That helps!

Spring batch- parallel processing of two tasks but second task has dependency on first task

I am using spring batch for processing large data.
My problem is I have two different Tasklet that are executing one after another. I can not run it parallel as second tasklet has some dependency one first Tasklet. In order to save the time I would like to start next Tasklet immediately after the first few items has been processed by previous tasklet. As Spring Batch uses a Chunk Oriented processing style. Can ItemWriter of one Tasklet, pass chunk of already processed items to another Tasklet's ItemReader or ItemProcessor or ItemWriter for next processing? But while the another tasklet is working on the chunk, previous tasklet should NOT wait and it should continue iteration over next chunk.

Based on your description and additional information above, I would simply leverage a CompositeItemWriter to accomplish this.
The list of items will first be written by WriterOne and then be passed to and written by WriterTwo.
<bean id="compositeWriter"
class="org.springframework.batch.item.support.CompositeItemWriter">
<property name="delegates">
<list>
<ref bean="writerOne" />
<ref bean="writerTwo" />
</list>
</property>
</bean>
<bean id="writerOne" ... />
<bean id="writerTwo" ... />

Repeating a job from quartz (Spring + Quartz application) - Job Chaining

I have the following setup for my quartz job -
<bean id="jobDetail" class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="targetObject" ref="actualObject" /><br>
<property name="targetMethod" value="processData"/>
<property name="concurrent" value="false"/>
</bean>
<bean id="simpleTrigger" class="org.springframework.scheduling.quartz.SimpleTriggerFactoryBean">
<property name="jobDetail" ref="jobDetail" />
<property name="startDelay" value="10000" />
<property name="repeatInterval" value="1000" />
</bean>
And this works good for me.
What I would like to do is to call the processData agai,n as soon as it finishes.
I know the good old while(true) method is best suited, but I want to do this with quartz.

First of all you must explain why you "want to do this with quartz" because the "good old while(true)" is the right way to implement your use case (of course you need an extra thread, but Quartz needs that as well). It sounds like over-engineering so you'll better have a good reason.
That being said you have two choices:
reschedule the job to run immediately when you leave it. In principle:
public class HelloJob implements Job {
public HelloJob() {
}
public void execute(JobExecutionContext context) throws JobExecutionException
{
//do your job...
Trigger trigger = newTrigger().build();
JobDetail job = newJob(HelloJob.class).build();
context.getScheduler().scheduleJob(trigger, job);
}
}
You don't need your XML configuration but must schedule this job to run for the very first time somehow (e.g. in #PostConstruct using Scheduler directly). The momemnt your job finishes, it will run the same job again as it was just rescheduled.
JobChainingJobListener might work for you, see: Can Quartz Scheduler Run jobs serially?
Both solutions are very heavyweight compared to "good old while(true)".

Spring JMS - Draining Topic on a Timer

I'm fairly new to Spring JMS, and I've found lots of documentation and examples at the Spring site and elsewhere, but my use case doesn't seem to be described anywhere, or at least in a way I can understand. I hope you might be able to help.
I would like to create a publisher of a topic and several durable subscribers to that topic. I'm working on the first subscriber now, and it is intended to run hourly (on a timer) and drain the topic of messages and process them all at once (i.e. to send an email summarizing all messages).
I do not know how to configure this setup in Spring, although I feel like this should be easy. Advice would be tremendously helpful.
My plan, such as it is, was to have the timer invoke the "processBatch" method, which would call receiveAndConvert() in a loop until it timed out, building up its list of messages.
This doesn't seem to work, though, because the consumer isn't really subscribed to the topic. Certainly not before it's run, and potentially not afterward.
How can I configure this using Spring and/or direct ActiveMQ?
I'm not sure where I ended up with my XML is a useful place for this discussion to start, but I'll provide it in case it is helpful:
<beans>
<!-- some unrelated beans -->
<!-- my Active MQ connection factory -->
<bean id="mqConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" destroy-method="stop">
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="vm://broker"/>
</bean>
</property>
</bean>
<!-- my topic -->
<amq:topic id="completionsTopic" physicalName="completions.topic"/>
<!-- my subscriber -->
<bean id="emailer" class="com.j128.Emailer">
<property name="jmsTemplate">
<bean class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="mqConnectionFactory"/>
<property name="defaultDestination" ref="completionsTopic"/>
<property name="receiveTimeout" value="2000"/>
</bean>
</property>
</bean>
<!-- my scheduler and periodic call to the topic drainer -->
<task:scheduler id="taskScheduler" pool-size="10"/>
<task:scheduled-tasks>
<!-- send emails hourly -->
<task:scheduled ref="emailer" method="processBatch" cron="0 * * * *"/>
</task:scheduled-tasks>
</beans>
But I'm certain I fundamentally have the wrong strategy and that there's a simple way to configure this.
Thank you for your assistance.

Have a look at how-does-a-queue-compare-to-a-topic: "Only subscribers who had an active subscription at the time the broker receives the message will get a copy of the message."
And for a durable topic how-do-durable-queues-and-topics-work: "Durable topics however are different as they must logically persist an instance of each suitable message for every durable consumer - since each durable consumer gets their own copy of the message"
So with a non-durable topic, your plan wont work as the hourly job won't get any messages as it isn't running when the messages are published. If you set up a durable topic then it might work but it depends what you expect to happen when you say your subscriber will "drain the topic of messages". All it can do it read the messages published to it since it last run, it can't affect the messages going to other subscribers.
For a discussion around durable subscribers (I haven't used them on ActiveMQ) see this

Spring Batch: starting a job from within a Spring MVC contorller WITH A NEW THREAD

I have a Spring-Batch job that I launch from a Spring MVC controller. The controller gets an uploaded file from the user and the job is supposed to process the file:
#RequestMapping(value = "/upload")
public ModelAndView uploadInventory(UploadFile uploadFile, BindingResult bindingResult) {
// code for saving the uploaded file to disk goes here...
// now I want to launch the job of reading the file line by line and saving it to the database,
// but I want to launch this job in a new thread, not in the HTTP request thread,
// since I so not want the user to wait until the job ends.
jobLauncher.run(
jobRegistry.getJob(JOB_NAME),
new JobParametersBuilder().addString("targetDirectory", folderPath).addString("targetFile", fileName).toJobParameters()
);
return mav;
}
I've tried the following XML config:
<job id="writeProductsJob" xmlns="http://www.springframework.org/schema/batch">
<step id="readWrite">
<tasklet task-executor="taskExecutor">
<chunk reader="productItemReader" writer="productItemWriter" commit-interval="10" />
</tasklet>
</step>
</job>
<bean id="taskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="5" />
<property name="maxPoolSize" value="5" />
</bean>
...but it seems like the multithreading happens only within the job boundaries itself. I.e., the controller thread waits until the job ends, and the job execution is handled by multiple threads (which is good but not the main thing I wanted). The main thing I wanted is that the job will be launched on a separate thread (or threads) while the controller thread will continue its execution without waiting for the job threads to end.
Is there a way to achieve this with Spring-batch?

The official documentation describes your exact problem and a solution in 4.5.2. Running Jobs from within a Web Container:
[...] The controller launches a Job using a JobLauncher that has been configured to launch asynchronously, which immediately returns a JobExecution. The Job will likely still be running, however, this nonblocking behaviour allows the controller to return immediately, which is required when handling an HttpRequest.
Spring Batch http://static.springsource.org/spring-batch/reference/html-single/images/launch-from-request.png
So you were pretty close in trying to use TaskExecutor, however it needs to be passed to the JobLauncher instead:
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<property name="taskExecutor" ref="taskExecutor"/>
</bean>
Disclaimer: I have never used Spring Batch...

The jobLauncher.run() method can be called in a new Thread like so:
#RequestMapping(value = "/upload")
public ModelAndView uploadInventory(UploadFile uploadFile, BindingResult bindingResult) {
[...]
final SomeObject jobLauncher = [...]
Thread thread = new Thread(){
#Override
public void run(){
jobLauncher.run([...]);
}
};
thread.start();
return mav;
}
The thread.start() line will spawn a new thread, and then continue to execute the code below it.
Note that, if jobLauncher is a local variable, it must be declared final in order for it to be used inside of the anonymous Thread class.

If you don't need to show the processing errors to your client, you can start the spring batch job in a seperate thread.