Perhaps, I am doing something wrong, but I can't find a good way out for the following situation.
I would like to unit test a service that uses Spring Batch underneath to execute jobs. The jobs are executed via pre-configured AsyncTaskExecutor in separate threads. In my unit test I would like to:
Create few domain objects and persist them via DAO
Invoke the service method to launch the job
Wait until the job is completed
Use DAO to retrieve domain objects and check their state
Obviously, all above should be executed within one transaction, but unfortunately, transactions are not propagated to new threads (I understand the rationale behind this).
Ideas that came to my mind:
Commit the transaction#1 after step (1). Is not good, as the DB state should be rolled back after the unit test.
Use Isolation.READ_UNCOMMITTED in job configuration. But this requires two different configurations for test and for production.
I think the simplest solution would be configure the JobLauncher with a SyncTaskExecutor during test execution - this way the job is executed in the same thread as the test and shares the transaction.
The task executor configuration can be moved to a separate spring configuration xml file. Have two versions of it - one with SyncTaskExecutor which is used during testing and the other AsyncTaskExecutor that is used for production runs.
Although this is not a true solution to your question, I found it possible to start a new transaction inside a worker thread manually. In some cases this might be sufficient.
Source: Spring programmatic transactions.
Example:
#PersistenceContext
private EntityManager entityManager;
#Autowired
private PlatformTransactionManager txManager;
/* in a worker thread... */
public void run() {
TransactionStatus tx = txManager.getTransaction(new DefaultTransactionDefinition());
try {
entityManager.find(...)
...
entityManager.flush(...)
etc...
txManager.commit(tx);
} catch (RuntimeException e) {
txManager.rollback(tx);
}
}
If you do want separate configurations, I'd recommend templating the isolation policy in your configuration and getting its value out of a property file so that you don't wind up with a divergent set of Spring configs for testing and prod.
But I agree that using the same policy production uses is best. How vast is your fixture data, and how bad would it be to have a setUp() step that blew away and rebuilt your data (maybe from a snapshot, if it's a lot of data) so that you don't have to rely on rollbacks?
Related
I have the need to insert 2 different entities into 2 different tables, using the same transaction. If the second insert fails, the first one should be rolled back.
Is there any way of doing this nicely?
Pseudo code:
start tx
repo1.save(myEntity);
repo2.save(anotherEntity);
try commit
I know you can leverage #Transactioal but only on method level?
you need check that you don't have set autocommit = false.
wrap save operations into one service method and make it #Transactional. But if you use save() custom method check that save in not marked as #Transactional with propagation level required_new or nested. If you need you can use REQUIRES_NEW for saving service method to make this service method transaction independent of other transactions.
also you can wrap in with TransactionTemplate.
#Autowired
private TransactionTemplate transactionTemplate;
transactionTemplate.execute(new TransactionCallbackWithoutResult() {
#Override
public void doInTransactionWithoutResult(TransactionStatus transactionStatus) {
repo1.save(myEntity);
repo2.save(anotherEntity);
});
It is usually a wrong idea to have #Transactional declared around repository methods.
Repositories are only for you to access domain entities. Business logic normally involves multiple domain entities and collaborations between them.
In your architecture you should have a layer to compose business logic. This usually corresponds to a service exposed to external.
This is usually the place you should have your transaction boundary set on. Usually it is a Controller, or a Service method.
Steps to follow:
Ensure you use Interfaces for repo1 and repo2 methods as spring transactions works on proxy. ( If you use only classes, then you may need to add few other dependencies).
Annotate repo1.save(..) and repo2.save(..) with #Transactional(propagation=Propagation.REQUIRED) annotation.
Call repo1.save() from any method outside of class.
Unit test the code properly using special junit runner.
I have a question regarding #Transactional annotation.
Nothing special defined, so as I understand is PROPAGATION_REQUIRED
Let’s say I have a transactional annotation which on both service and dao layer.
Service
#Transactional
public long createStudentInDB(Student student) {
final long id = addStudentToDB (student);
addStudentToCourses (id, student.getCourseIds());
return id;
}
private long addStudentToDB (Student student) {
StudentEntity entity = new StudentEntity ();
convertToEntity(student, entity);
try {
final id = dao.create(entity);
} catch (Exception e){
//
}
return id;
}
private void addStudentToCourses (long studentId, List<String> coursesIds){
//add user to group
if(coursesIds!= null){
List<StudentCourseEntity> studentCourses = new ArrayList<>();
for(String coursesId: coursesIds){
StudentCourseEntity entity = new StudentCourseEntity ();
entity.setCourseId(coursesId);
entity.setStudentId(userId);
studentCourses.add(studentId);
}
anotherDao.saveAll(studentCourses);
}
}
DAO
#Transactional
public UUID create(StudentEntity entity) {
if ( entity == null ) { throw new Exception(//…); }
getCurrentSession().save(entity);
return entity.getId();
}
ANOTHER DAO:
#Transactional
public void saveAll(Collection< StudentCourseEntity > studentCourses) {
List< StudentCourseEntity > result = new ArrayList<>();
if(studentCourses!= null) {
for (StudentCourseEntity studentCourse : studentCourses) {
if (studentCourse!= null) {
save(studentCourse);
}
}
}
}
Despite the fact that’s not optimal, it seems it causing deadlocks.
Let’s say I have max 2 connections to the database.
And I am using 3 different threads to run the same code.
Thread-1 and thread-2 receive a connection, thread-3 is not getting any connection.
More than that, it seems that thread-1 become stuck when trying to get a connection in dao level, same for thread-2. Causing a deadlock.
I was sure that by using propagation_required this would not happen.
Am I missing something?
What’s the recommendation for something like that? Is there a way I can have #transactional on both layers? If not which is preferred?
Thanks
Fabrizio
As the dao.doSomeStuff is expected to be invoked from within other transactions I would suggest you to configure this method as:
#Transaction(propagation=REQUIRES_NEW)
Thanks to that the transaction which is invoking this method will halted until the one with REQUIRES_NEW will be finished.
Not sure if this is the fix for your particular deadlock case but your example fits this particular set-up.
You are right, Propagation.REQUIRED is the default. But that also means that the second (nested) invocation on dao joins / reuses the transaction created on service level. So there is no need to create another transaction for the nested call.
In general Spring (on high level usage) should manage resource handling by forwarding it to the underlying ORM layer:
The preferred approach is to use Spring's highest level template based
persistence integration APIs or to use native ORM APIs with
transaction- aware factory beans or proxies for managing the native
resource factories. These transaction-aware solutions internally
handle resource creation and reuse, cleanup, optional transaction
synchronization of the resources, and exception mapping. Thus user
data access code does not have to address these tasks, but can be
focused purely on non-boilerplate persistence logic.
Even if you handle it on your own (on low level API usage) the connections should be reused:
When you want the application code to deal directly with the resource
types of the native persistence APIs, you use these classes to ensure
that proper Spring Framework-managed instances are obtained,
transactions are (optionally) synchronized, and exceptions that occur
in the process are properly mapped to a consistent API.
...
If an existing transaction already has a connection synchronized
(linked) to it, that instance is returned. Otherwise, the method call
triggers the creation of a new connection, which is (optionally)
synchronized to any existing transaction, and made available for
subsequent reuse in that same transaction.
Source
Maybe you have to find out what is happening down there.
Each Session / Unit of Work will be bound to a thread and released (together with the assigned connection) after the transaction has ended. Of course when your thread gets stuck it won't release the connection.
Are you sure that this 'deadlock' is caused by this nesting? Maybe that has another reason. Do you have some test code for this example? Or a thread dump or something?
#Transactional works by keeping ThreadLocal state, which can be accessed by the (Spring managed) Proxy EntityManager. If you are using Propagation.REQUIRED (the default), and you have a non-transactional method which calls two different DAOs (or two Transactional methods on the same DAO), you will get two transactions, and two calls to acquire a pooled connection. You may get the same connection twice or two different connections, but you should only use one connection at the time.
If you call two DAOs from a #Transactional method, there will only be one transaction, as the DAO will find and join the existing transaction found in the ThreadLocal state, again you only need one connection from the pool.
If you get a deadlock then something is very wrong, and you may want to debug when your connections and transaction are created. A transaction is started by calling Connection.setAutoCommit(false), in Hibernate this happens in org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor#begin(). Connections are managed by a class extending org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor so these are some good places to put a break point, and trace the call-stack back to your code to see which lines created connections.
I'm looking for a lib that allow me to do
define a worker that will be invoked once on a specific time in the future (not need the re-schedule / cron like featrure) i.e. a Timer
The worker should accept a context which withe some parameters / inputs
all should be persistent in the DB (or file) the worker
worker should be managed by spring -- spring should instantiate the worker so it can be injected with dependencies
be able to create timers dynamically via API and not just statically via spring XML beans
nice to have:
support a cluster i.e. have several nodes that can host a worker. each store jobn in the DB will cause invokaction of ONE work on one of the nods
I've examined several alternatives none meets the requirements:
Quartz
when using org.springframework.scheduling.quartz.JobDetailBean makes quartz create your worker instance (and not by spring) so you can't get dependecy ijection, (which will lead me to use Service Locator which I want to avoid)
while using org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean you can't get a context. your Worker expose one public method that accepts no arguments.In addition when using MethodInvokingJobDetailFactoryBean you can't use persistence (form the Javadoc)
Note: JobDetails created via this FactoryBean are not serializable and thus not suitable for persistent job stores. You need to implement your own Quartz Job as a thin wrapper for each case where you want a persistent job to delegate to a specific service method.
Spring's Timer and simple JDK Timers does not support the persistence / cluster feature
I know I can impl thing myself using a DB and Spring (or even JDK) Timers but I prefer to use an a 3r party lib for that.
Any suggestions?
If you want to create the job details to generate triggers/job-details at runtime and still be able to use Spring DI on your beans you can refer to this blog post, it shows how to use SpringBeanJobFactory in conjunction with ObjectFactoryCreatingFactoryBean to create Quartz triggering objects at runtime with Spring injected beans.
For those interested in an alternative to Quartz, have a look at db-scheduler (https://github.com/kagkarlsson/db-scheduler). A persistent task/execution-schedule is kept in a single database table. It is guaranteed to be executed only once by a scheduler in the cluster.
Yes, see code example below.
Currently limited to a single string identifier for no format restriction. The scheduler will likely be extended in the future with better support for job-details/parameters.
The execution-time and context is persistent in the database. Binding a task-name to a worker is done when the Scheduler starts. The worker may be instantiated by Spring as long as it implements the ExecutionHandler interface.
See 3).
Yes, see code example below.
Code example:
private static void springWorkerExample(DataSource dataSource, MySpringWorker mySpringWorker) {
// instantiate and start the scheduler somewhere in your application
final Scheduler scheduler = Scheduler
.create(dataSource)
.threads(2)
.build();
scheduler.start();
// define a task and a handler that named task, MySpringWorker implements the ExecutionHandler interface
final OneTimeTask oneTimeTask = ComposableTask.onetimeTask("my-onetime-task", mySpringWorker);
// schedule a future execution for the task with a custom id (currently the only form for context supported)
scheduler.scheduleForExecution(LocalDateTime.now().plusDays(1), oneTimeTask.instance("1001"));
}
public static class MySpringWorker implements ExecutionHandler {
public MySpringWorker() {
// could be instantiated by Spring
}
#Override
public void execute(TaskInstance taskInstance, ExecutionContext executionContext) {
// called when the execution-time is reached
System.out.println("Executed task with id="+taskInstance.getId());
}
}
Your requirements 3 and 4 do not really make sense to me: how can you have the whole package (worker + work) serialized and have it wake up magically and do its work? Shouldn't something in your running system do this at the proper time? Shouldn't this be the worker in the first place?
My approach would be this: create a Timer that Spring can instantiate and inject dependencies to. This Timer would then load its work / tasks from persistent storage, schedule them for execution and execute them. Your class can be a wrapper around java.util.Timer and not deal with the scheduling stuff at all. You must implement the clustering-related logic yourself, so that only one Timer / Worker gets to execute the work / task.
I am using following code
TestDAO {
Session session = null;
public TestDAO() {
this.session = HibernateUtil.getSessionFactory().getCurrentSession();
}
//...more code create,update ...
//each method starts a transcation using "tx= session.beginTransaction();"
}
1)Now should i commit the transcation using tx.commit for a fetch operation too or only for save/update operation??
2)Should i create a seperate instance of TestDAO every time i need?Or should i create a singleton class that returns a single instance of DAO everytme?Will this have a problem?
You don't need tx.commit() for fetch operation. That is only needed for any save, update or delete. Close the session after data fetching.
If your application connect to only one database then use of single DAO is better. Spring framework encourages this. You will find more details about this on the following link
Don't repeat the DAO!
Transactions should not be the responsibility of the DAO, those really need to be controlled at a higher level. A DAO should be something that does queries and updates without being aware of the bigger picture, calls to DAOs can be grouped within an object like a Spring service or EJB session bean which is responsible for deciding what needs to go together in a transaction. This makes your DAO code more reusable since it doesn't have to know as much about the context in which it's operating.
Look at how Spring does it (in the sample applications like petstore that come with Spring), or better, look at the King/Bauer Hibernate-JPA book, which has a chapter on creating DAOs.
In our system we have multi-threaded processing engine. During processing each thread calls methods to retrieve data from the database. We determined that performance is greatly improved if methods called from the same thread use the same DB session (sessions are coming from the pool of course).
Is there any standard way in Spring to ensure such thing or we have to come up with our own custom solution?
UPDATE: Forgot to mention that same methods can be called in different context where they should use a standard way of getting the session from the pool
I did not see Spring anywhere in your question. So I assume you want a simple utility to do this.
class SessionUtil {
private ThreadLocal currentSession;
public Session getCurrentSession() {
if(currentSession.get() == null) {
Session s = //create new session
currentSession.set(s);
}
return (Session)currentSession.get();
}
}
The Thread local will ensure that within the same thread it is always the same session. If you are using Spring then the classes/utilities mentioned above (in other responses) should be perfect.
Spring has a class called TransactionSynchronizationManager. It stores the current Session in a ThreadLocal. The TransactionSynchronizationManager is not recommended for use by the developer, but you can try using it.
Session session = ((SessionHolder)
TransactionSynchronizationManager.getResource(sessionFactory)).getSession();
(if you are using EntityManager, simply replace "Session" with "EntityManager").
You can have the sessionFactory injected in your bean - it is per-application.
Take a look at this discussion.
Other options, which I think are preferable to manual thread-handling are:
Thread pooling
Spring batch
Spring-JMS integration
Spring 3.0 has a concept of thread-scoped beans (hovewer, this scope is not registered by default, see docs): 3.5 Bean scopes, 3.5.5.2 Using a custom scope
EDIT:
I say about this:
Thread-scoped beans As of Spring 3.0,
a thread scope is available, but is
not registered by default. For more
information, see the documentation for
SimpleThreadScope. For
instructions on how to register this
or any other custom scope, see
Section 3.5.5.2, “Using a custom
scope”.
Spring coordinates database sessions, connections and threads through it's Transaction Framework (actually, using its TransactionSynchronizationManager - see description here - but you really don't want to mess with that directly, it's fearsome). If you need to coordinate your threads, then this is by far the simplest way of doing it.
How you choose to use the framework, however, is up top you.