Develop Reference

JPA transaction handling

I have a Foo entity with fields Name, SecondaryName and Counter.
In the DB I have a unique constraint on (name, secondaryName, counter).
In the service layer I have the following method (where fooRepositry is a CrudRepository):
#Transactional(isolation = Isolation.SERIALIZABLE, propagation = Propagation.REQUIRES_NEW)
public void saveFoo(Foo foo) {
Optional<TestDto> fooWithHighestCounter= fooRepository.
findTopByNameAndSecondaryNameOrderByCounterDesc(foo.getName(), foo.getSecondaryName());
if (fooWithHighestCounter.isPresent()) {
foo.setCounter(fooWithHighestCounter.get().getCounter() + 1);
} else {
foo.setCounter(1);
}
Foo saved = fooRepository.save(foo);
}
With every call on saveFoo, a new record shall be created in the DB with already the existing highest counter + 1. Hence, the highest counter must be found, thus the #Transactional.
However, I constantly get ContraintViolationException when multiple threads call the saveFoo method as every thread finds the same highest counter value.
I assumed that every thread would create a new transaction and those transactions will run serially so no transaction would find the same counter value. (The #EnableTransactionManagement is put on the Application)
What else can I do to achieve the aforementioned behavior?

I think the fooRepository.save(foo) at last is saving the same values again and again in the database that is why it is giving ContrainViolationException. If you need to update the value to any existing Object just call the setCounter but dont call the .save() method instead call the update method of the repository (if you have any) else if it is a new entity which is not present in database yet then call the save method.
If it is done in hibernate refer the following link
Ref: http://www.objectdb.com/java/jpa/persistence/update

Hibernate: Insert objects with unique constraints from different threads

I have the following problem. I have 10 Threads which create objects that are inserted in the database. Each Thread has a ThreadLocal and its own session. All objects are inserted together, after they were created. These objects have a column which is marked as unique. However, I have the problem, that it can happen that two different threads create the same object. This behaviour is wanted but I don't know how I can insert them into my database.
Currently, each thread queries all objects that are inserted in the database, checks on the queried objects if they exist or not and inserts the non-existing objects into the database. However, as it can happen that the object did not exist on the query of all objects, I get a ConstraintViolationException when I insert the objects and they were already added by another Thread. However, doing a database (or cache) query for each object has to bad performance, as we are trying to add 1000 objects per thread and minute. If I try to flush the database after each single insert, then I get the following error: Deadlock found when trying to get lock; try restarting transaction
So my question is: How can I insert objects, that have a unique constraint from different threads simultanously.
//Edit: currently I'm using Hibernate with MYSQL InnoDB
//Edit2: Finally, the code which I use to write a single item.
public class ItemWriterRunnable implements Callable<Object> {
private final ThreadLocal<Session> session = new ThreadLocal<Session>();
private Item item;
public ItemWriterRunnable(Item item) {
super();
this.item= item;
}
protected Session currentSession() {
Session s = this.session.get();
// Open a new Session, if this thread has none yet
if (s == null || !s.isOpen()) {
s = HibernateUtils.getSessionFactory().openSession();
// Store it in the ThreadLocal variable
this.session.set(s);
}
return s;
}
#Override
public Object call() throws Exception {
Session currentSession = currentSession();
try {
currentSession.beginTransaction();
currentSession.save(this.item);
currentSession.getTransaction().commit();
} catch (ConstraintViolationException e) {
currentSession.getTransaction().rollback();
} catch (RuntimeException e) {
currentSession.getTransaction().rollback();
} finally {
currentSession.close();
currentSession = null;
this.session.remove();
}
return null;
}
}
Best regards,
André

If you write multiple objects in a thread, and one of them fails because it's a duplicate, then you'll have to work out which one was the duplicate, remove it from the set, and retry writing it to the DB (with a change of another failure). This takes a lot of time. Alternatively, you could read the DB to see if there are any duplicates before writing the set, and remove the duplicates before writing. This read/check/write pattern is flawed if it is not contained within a synchronised block, because other threads could write duplicates between the steps. The synchronisation needed to fix this will stall your server on every write, pausing all existing threads, potentially harming performance.
Instead, spawn a thread per object, and write the object within this thread (without the read/check). Most objects will write without issue, because most objects are not duplicated (an assumption, but it's probably right). Objects that are duplicates will fail with an exception, at which point you can terminate that thread because the relevant work is already done.

Difference between Hibernate update by Session.update and HibernateTemplate.merge

I saw to types of update operation:
First:
getHibernateTemplate().execute(new HibernateCallback() {
public Object doInHibernate(Session session) {
session.flush();
session.setCacheMode(CacheMode.IGNORE);
SomeObject ss = (SomeObject) session.get(SomeObject.class, id);
long next = ss.getAndIncrement();
session.update(ss);
session.flush();
return null;
}
});
and secondly
SomeObject ss = loadSomeObject();
long next = ss.getAndIncrement();
getHibernateTemplate.merge(ss);
These two method do the same. I want to know which one is better and safe and why. Thank you.

In the first operation the object ss is attached to the session. where as in the second operation its detached. So if you have an attached objects you can use update. If you have a detached objects then use merge which first attaches the object to the session then will do an update.
EDIT: For your information on attached(persistent) and detached objects :
Hibernate defines and supports the following object states:
Transient - an object is transient if it has just been instantiated using the new operator, and it is not associated with a Hibernate Session. It has no persistent representation in the database and no identifier value has been assigned. Transient instances will be destroyed by the garbage collector if the application does not hold a reference anymore. Use the Hibernate Session to make an object persistent (and let Hibernate take care of the SQL statements that need to be executed for this transition).
Persistent - a persistent instance has a representation in the database and an identifier value. It might just have been saved or loaded, however, it is by definition in the scope of a Session. Hibernate will detect any changes made to an object in persistent state and synchronize the state with the database when the unit of work completes. Developers do not execute manual UPDATE statements, or DELETE statements when an object should be made transient.
Detached - a detached instance is an object that has been persistent, but its Session has been closed. The reference to the object is still valid, of course, and the detached instance might even be modified in this state. A detached instance can be reattached to a new Session at a later point in time, making it (and all the modifications) persistent again. This feature enables a programming model for long running units of work that require user think-time. We call them application transactions, i.e., a unit of work from the point of view of the user.

Whats an API without any code examples?
SessionFactory sf = ctx.getBean("hibernateSessionFactory",SessionFactory.class);
Session session = sf.openSession();
Transaction t = session.beginTransaction();
try {
Session s2 = sf.openSession();
Organization org = (Organization)s2.get(Organization.class,100624l);//1
org.setOrgName("org");
s2.close();//2
Organization org1 = (Organization)session.get(Organization.class,100624l);//3
org.setOrgName("testOrg");
org1.setOrgName("org");//a
session.merge(org);//4
System.out.println(org == org1);//b
t.commit();
} catch (HibernateException e) {
t.rollback();
throw e;
}finally{
session.close();
}
1st instance is loaded and made persistent.
The instance is detached.
Another instance is loaded
At the point of executing this operation, there are 2 instances of the same object
in the session(org and org1) – org is detached and org1 is persistent
If we do an update() or saveOrUpdate() there, we get the below exception:
org.hibernate.NonUniqueObjectException: a different object with the same identifier value was already associated with the session: [com.spring.model.Organization#100624]
a. We do a merge here which:
I. Merges the state of the detached object + persistent object.
II. In case of conflict, the object which is merged wins, like in this case, the value saved will be : testOrg
III. Had we merged on org1, we would have got org.
b. This will always return false, meaning post merge, org was still in DETACHED state
I hope the diff. is clear now.
Summary :
saveOrUpdate() or update() will throw an exception if there are 2 instances of the same object in the session(one detached and one persistent)
merge() will not throw the exception, but will save the object while merging the changes.

Merge Does Following
Merge has intelligence. It has lot of pre-checks before it go actual merge(if required)
if Object is transient, It simply fires INSERT query makes object persistent(attached to session)
if Object is detached, fires select query to check whether data modified or not
if modified, fires UPDATE query otherwise just ignore merge task.
where as session.update
throws exception if object is transient.
if Object is detached, it simply fires UPDATE query irrespective of data changes to object.
session.merge is expensive than update

The basic difference is:
Merge() is not concerned about sessions whether persistent or detached...it will just update without considering the sessions.
In case of update() it will throw an exception like org.hibernate.NonUniqueObjectException: a different object with the same identifier value was already associated with the session.
It's explained here with a good example:
http://www.java4developer.com/difference-between-update-and-merge-in-hibernate/

Both update() and merge() methods in hibernate are used to convert the object which is in detached state into persistence state. But there is little difference. Let us see which method will be used in what situation.
Let Us Take An Example
SessionFactory factory = cfg.buildSessionFactory();
Session session1 = factory.openSession();
Employee s1 = null;
Object o = session1.get(Employee.class, new Integer(101));
s1 = (Student)o;
session1.close();
s1.setSSN(97);
Session session2 = factory.openSession();
Employee s2 = null;
Object o1 = session2.get(Employee.class, new Integer(101));
s2 = (Student)o1;
Transaction tx=session2.beginTransaction();
session2.merge(s1);
SessionFactory factory = cfg.buildSessionFactory();
Session session1 = factory.openSession();
Employee s1 = null;
Object o = session1.get(Employee.class, new Integer(101));
s1 = (Employee)o;
session1.close();
s1.setMarks(97);
Session session2 = factory.openSession();
Employee s2 = null;
Object o1 = session2.get(Employee.class, new Integer(101));
s2 = (Employee)o1;
Transaction tx=session2.beginTransaction();
session2.merge(s1);
Hope you are clear…, actually update and merge methods will come into picture when ever we loaded the same object again and again into the database, like above.

Find or insert based on unique key with Hibernate

I'm trying to write a method that will return a Hibernate object based on a unique but non-primary key. If the entity already exists in the database I want to return it, but if it doesn't I want to create a new instance and save it before returning.
UPDATE: Let me clarify that the application I'm writing this for is basically a batch processor of input files. The system needs to read a file line by line and insert records into the db. The file format is basically a denormalized view of several tables in our schema so what I have to do is parse out the parent record either insert it into the db so I can get a new synthetic key, or if it already exists select it. Then I can add additional associated records in other tables that have foreign keys back to that record.
The reason this gets tricky is that each file needs to be either totally imported or not imported at all, i.e. all inserts and updates done for a given file should be a part of one transaction. This is easy enough if there's only one process that's doing all the imports, but I'd like to break this up across multiple servers if possible. Because of these constraints I need to be able to stay inside one transaction, but handle the exceptions where a record already exists.
The mapped class for the parent records looks like this:
#Entity
public class Foo {
#Id
#GeneratedValue(strategy = IDENTITY)
private int id;
#Column(unique = true)
private String name;
...
}
My initial attempt at writting this method is as follows:
public Foo findOrCreate(String name) {
Foo foo = new Foo();
foo.setName(name);
try {
session.save(foo)
} catch(ConstraintViolationException e) {
foo = session.createCriteria(Foo.class).add(eq("name", name)).uniqueResult();
}
return foo;
}
The problem is when the name I'm looking for exists, an org.hibernate.AssertionFailure exception is thrown by the call to uniqueResult(). The full stack trace is below:
org.hibernate.AssertionFailure: null id in com.searchdex.linktracer.domain.LinkingPage entry (don't flush the Session after an exception occurs)
at org.hibernate.event.def.DefaultFlushEntityEventListener.checkId(DefaultFlushEntityEventListener.java:82) [hibernate-core-3.6.0.Final.jar:3.6.0.Final]
at org.hibernate.event.def.DefaultFlushEntityEventListener.getValues(DefaultFlushEntityEventListener.java:190) [hibernate-core-3.6.0.Final.jar:3.6.0.Final]
at org.hibernate.event.def.DefaultFlushEntityEventListener.onFlushEntity(DefaultFlushEntityEventListener.java:147) [hibernate-core-3.6.0.Final.jar:3.6.0.Final]
at org.hibernate.event.def.AbstractFlushingEventListener.flushEntities(AbstractFlushingEventListener.java:219) [hibernate-core-3.6.0.Final.jar:3.6.0.Final]
at org.hibernate.event.def.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:99) [hibernate-core-3.6.0.Final.jar:3.6.0.Final]
at org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:58) [hibernate-core-3.6.0.Final.jar:3.6.0.Final]
at org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:1185) [hibernate-core-3.6.0.Final.jar:3.6.0.Final]
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1709) [hibernate-core-3.6.0.Final.jar:3.6.0.Final]
at org.hibernate.impl.CriteriaImpl.list(CriteriaImpl.java:347) [hibernate-core-3.6.0.Final.jar:3.6.0.Final]
at org.hibernate.impl.CriteriaImpl.uniqueResult(CriteriaImpl.java:369) [hibernate-core-3.6.0.Final.jar:3.6.0.Final]
Does anyone know what is causing this exception to be thrown? Does hibernate support a better way of accomplishing this?
Let me also preemptively explain why I'm inserting first and then selecting if and when that fails. This needs to work in a distributed environment so I can't synchronize across the check to see if the record already exists and the insert. The easiest way to do this is to let the database handle this synchronization by checking for the constraint violation on every insert.

I had a similar batch processing requirement, with processes running on multiple JVMs. The approach I took for this was as follows. It is very much like jtahlborn's suggestion. However, as vbence pointed out, if you use a NESTED transaction, when you get the constraint violation exception, your session is invalidated. Instead, I use REQUIRES_NEW, which suspends the current transaction and creates a new, independent transaction. If the new transaction rolls back it will not affect the original transaction.
I am using Spring's TransactionTemplate but I'm sure you could easily translate it if you do not want a dependency on Spring.
public T findOrCreate(final T t) throws InvalidRecordException {
// 1) look for the record
T found = findUnique(t);
if (found != null)
return found;
// 2) if not found, start a new, independent transaction
TransactionTemplate tt = new TransactionTemplate((PlatformTransactionManager)
transactionManager);
tt.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
try {
found = (T)tt.execute(new TransactionCallback<T>() {
try {
// 3) store the record in this new transaction
return store(t);
} catch (ConstraintViolationException e) {
// another thread or process created this already, possibly
// between 1) and 2)
status.setRollbackOnly();
return null;
}
});
// 4) if we failed to create the record in the second transaction, found will
// still be null; however, this would happy only if another process
// created the record. let's see what they made for us!
if (found == null)
found = findUnique(t);
} catch (...) {
// handle exceptions
}
return found;
}

You need to use UPSERT or MERGE to achieve this goal.
However, Hibernate does not offer support for this construct, so you need to use jOOQ instead.
private PostDetailsRecord upsertPostDetails(
DSLContext sql, Long id, String owner, Timestamp timestamp) {
sql
.insertInto(POST_DETAILS)
.columns(POST_DETAILS.ID, POST_DETAILS.CREATED_BY, POST_DETAILS.CREATED_ON)
.values(id, owner, timestamp)
.onDuplicateKeyIgnore()
.execute();
return sql.selectFrom(POST_DETAILS)
.where(field(POST_DETAILS.ID).eq(id))
.fetchOne();
}
Calling this method on PostgreSQL:
PostDetailsRecord postDetailsRecord = upsertPostDetails(
sql,
1L,
"Alice",
Timestamp.from(LocalDateTime.now().toInstant(ZoneOffset.UTC))
);
Yields the following SQL statements:
INSERT INTO "post_details" ("id", "created_by", "created_on")
VALUES (1, 'Alice', CAST('2016-08-11 12:56:01.831' AS timestamp))
ON CONFLICT DO NOTHING;
SELECT "public"."post_details"."id",
"public"."post_details"."created_by",
"public"."post_details"."created_on",
"public"."post_details"."updated_by",
"public"."post_details"."updated_on"
FROM "public"."post_details"
WHERE "public"."post_details"."id" = 1
On Oracle and SQL Server, jOOQ will use MERGE while on MySQL it will use ON DUPLICATE KEY.
The concurrency mechanism is ensured by the row-level locking mechanism employed when inserting, updating, or deleting a record, which you can view in the following diagram:
Code avilable on GitHub.

Two solution come to mind:
That's what TABLE LOCKS are for
Hibernate does not support table locks, but this is the situation when they come handy. Fortunately you can use native SQL thru Session.createSQLQuery(). For example (on MySQL):
// no access to the table for any other clients
session.createSQLQuery("LOCK TABLES foo WRITE").executeUpdate();
// safe zone
Foo foo = session.createCriteria(Foo.class).add(eq("name", name)).uniqueResult();
if (foo == null) {
foo = new Foo();
foo.setName(name)
session.save(foo);
}
// releasing locks
session.createSQLQuery("UNLOCK TABLES").executeUpdate();
This way when a session (client connection) gets the lock, all the other connections are blocked until the operation ends and the locks are released. Read operations are also blocked for other connections, so needless to say use this only in case of atomic operations.
What about Hibernate's locks?
Hibernate uses row level locking. We can not use it directly, because we can not lock non-existent rows. But we can create a dummy table with a single record, map it to the ORM, then use SELECT ... FOR UPDATE style locks on that object to synchronize our clients. Basically we only need to be sure that no other clients (running the same software, with the same conventions) will do any conflicting operations while we are working.
// begin transaction
Transaction transaction = session.beginTransaction();
// blocks until any other client holds the lock
session.load("dummy", 1, LockOptions.UPGRADE);
// virtual safe zone
Foo foo = session.createCriteria(Foo.class).add(eq("name", name)).uniqueResult();
if (foo == null) {
foo = new Foo();
foo.setName(name)
session.save(foo);
}
// ends transaction (releasing locks)
transaction.commit();
Your database has to know the SELECT ... FOR UPDATE syntax (Hibernate is goig to use it), and of course this only works if all your clients has the same convention (they need to lock the same dummy entity).

The Hibernate documentation on transactions and exceptions states that all HibernateExceptions are unrecoverable and that the current transaction must be rolled back as soon as one is encountered. This explains why the code above does not work. Ultimately you should never catch a HibernateException without exiting the transaction and closing the session.
The only real way to accomplish this it would seem would be to manage the closing of the old session and reopening of a new one within the method itself. Implementing a findOrCreate method which can participate in an existing transaction and is safe within a distributed environment would seem to be impossible using Hibernate based on what I have found.

The solution is in fact really simple. First perform a select using your name value. If a result is found, return that. If not, create a new one. In case the creation fail (with an exception), this is because another client added this very same value between your select and your insert statement. This is then logical that you have an exception. Catch it, rollback your transaction and run the same code again. Because the row already exist, the select statement will find it and you'll return your object.
You can see here explanation of strategies for optimistic and pessimistic locking with hibernate here : http://docs.jboss.org/hibernate/core/3.3/reference/en/html/transactions.html

a couple people have mentioned different parts of the overall strategy. assuming that you generally expect to find an existing object more often than you create a new object:
search for existing object by name. if found, return
start nested (separate) transaction
try to insert new object
commit nested transaction
catch any failure from nested transaction, if anything but constraint violation, re-throw
otherwise search for existing object by name and return it
just to clarify, as pointed out in another answer, the "nested" transaction is actually a separate transaction (many databases don't even support true, nested transactions).

Well, here's one way to do it - but it's not appropriate for all situations.
In Foo, remove the "unique = true" attribute on name. Add a timestamp that gets updated on every insert.
In findOrCreate(), don't bother checking if the entity with the given name already exists - just insert a new one every time.
When looking up Foo instances by name, there may be 0 or more with a given name, so you just select the newest one.
The nice thing about this method is that it doesn't require any locking, so everything should run pretty fast. The downside is that your database will be littered with obsolete records, so you may have to do something somewhere else to deal with them. Also, if other tables refer to Foo by its id, then this will screw up those relations.

Maybe you should change your strategy:
First find the user with the name and only if the user thoes not exist, create it.

I would try the following strategy:
A. Start a main transaction (at time 1)
B. Start a sub-transaction (at time 2)
Now, any object created after time 1 will not be visible in the main transaction. So when you do
C. Create new race-condition object, commit sub-transaction
D. Handle conflict by starting a new sub-transaction (at time 3) and getting the object from a query (the sub-transaction from point B is now out-of-scope).
only return the object primary key and then use EntityManager.getReference(..) to obtain the object you will be using in the main transaction. Alternatively, start the main transaction after D; it is not totally clear to me in how many race conditions you will have within your main transaction, but the above should allow for n times B-C-D in a 'large' transaction.
Note that you might want to do multi-threading (one thread per CPU) and then you can probably reduce this issue considerably by using a shared static cache for these kind of conflicts - and point 2 can be kept 'optimistic', i.e. not doing a .find(..) first.
Edit: For a new transaction, you need an EJB interface method call annotated with transaction type REQUIRES_NEW.
Edit: Double check that the getReference(..) works as I think it does.

How to test whether lazy loaded JPA collection is initialized?

I have a service that gets a JPA entity from outside code. In this service I would like to iterate over a lazily loaded collection that is an attribute of this entity to see if the client has added something to it relative to the current version in the DB.
However, the client may have never touched the collection so it's still not initialized. This results in the well known
org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: com.example.SomeEntity.
Of course, if the client never touched the collection, my service doesn't have to check it for possible changes. The thing is that I can't seem to find a way to test whether the collection is initialized or not. I guess I could call size() on it and if it throws LazyInitializationException I would know, but I'm trying not to depend on such patterns.
Is there some isInitialized() method somewhere?

Are you using JPA2?
PersistenceUnitUtil has two methods that can be used to determine the load state of an entity.
e.g. there is a bidirectional OneToMany/ManyToOne relationship between Organization and User.
public void test() {
EntityManager em = entityManagerFactory.createEntityManager();
PersistenceUnitUtil unitUtil =
em.getEntityManagerFactory().getPersistenceUnitUtil();
em.getTransaction().begin();
Organization org = em.find(Organization.class, 1);
em.getTransaction().commit();
Assert.assertTrue(unitUtil.isLoaded(org));
// users is a field (Set of User) defined in Organization entity
Assert.assertFalse(unitUtil.isLoaded(org, "users"));
initializeCollection(org.getUsers());
Assert.assertTrue(unitUtil.isLoaded(org, "users"));
for(User user : org.getUsers()) {
Assert.assertTrue(unitUtil.isLoaded(user));
Assert.assertTrue(unitUtil.isLoaded(user.getOrganization()));
}
}
private void initializeCollection(Collection<?> collection) {
// works with Hibernate EM 3.6.1-SNAPSHOT
if(collection == null) {
return;
}
collection.iterator().hasNext();
}

org.hibernate.Hibernate.isInitialized(..)
There is no standard JPA solution to my knowledge. But if you want to actually initialize collections, you can create an utility method and iterate them (only one iteration is enough).

For eclipselink, users cast the collection you are trying to access to an org.eclipse.persistence.indirection.IndirectList, and then call its isInstantiated() method. The following link has more information:
http://www.eclipse.org/eclipselink/api/1.1/org/eclipse/persistence/indirection/IndirectList.html#isInstantiated.