Develop Reference

Conditional insert with Spring JPA / Hibernate

I'm working on a project that runs in a clustered environment, where there are many nodes and a single database. The project uses Spring-data-JPA (1.9.0) and Hibernate (5.0.1). I'm having trouble resolving how to prevent duplicate row issues.
For sake of example, here's a simple table
#Entity
#Table(name = "scheduled_updates")
public class ScheduledUpdateData {
public enum UpdateType {
TYPE_A,
TYPE_B
}
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "id")
private UUID id;
#Column(name = "type", nullable = false)
#Enumerated(EnumType.STRING)
private UpdateType type;
#Column(name = "source", nullable = false)
private UUID source;
}
The important part is that there is a UNIQUE(type, source) constraint.
And of course, matching example repository:
#Repository
public class ScheduledUpdateRepository implements JpaRepository<ScheduledUpdateData, UUID> {
ScheduledUpdateData findOneByTypeAndSource(final UpdateType type, final UUID source);
//...
}
The idea for this example is that parts of the system can insert rows to be schedule for something that runs periodically, any number of times between said runs. When whatever that something is actually runs, it doesn't have to worry about operating on the same thing twice.
How can I write a service method that would conditionally insert into this table? A few things I've tried that don't work are:
Find > Act - The service method would use the repository to see if a entry already exists, and then either update the found entry or save a new one as needed. This does not work.
Try insert > Update if fail - The service method would try to insert, catch the exception due to the unique constraint, and then do an update instead. This does not work since the transaction will already be in a rolled-back state and no further operations can be done in it.
Native query with "INSERT INTO ... WHERE NOT EXISTS ..."* - The repository has a new native query:
#Repository
public class ScheduledUpdateRepository implements JpaRepository<ScheduledUpdateData, UUID> {
// ...
#Modifying
#Query(nativeQuery = true, value = "INSERT INTO scheduled_updates (type, source)" +
" SELECT :type, :src" +
" WHERE NOT EXISTS (SELECT * FROM scheduled_updates WHERE type = :type AND source = :src)")
void insertUniquely(#Param("type") final String type, #Param("src") final UUID source);
}
This unfortunately also does not work, as Hibernate appears to perform the SELECT used by the WHERE clause on its own first - which means in the end multiple inserts are tried, causing a unique constraint violation.
I definitely don't know a lot of the finer points of JTA, JPA, or Hibernate. Any suggestions on how insert into tables with unique constraints (beyond just the primary key) across multiple JVMs?
Edit 2016-02-02
With Postgres (2.3) as a database, tried using Isolation level SERIALIZABLE - sadly by itself this still caused constraint violation exceptions.

You are trying to ensure that only 1 node can perform this operation at a time.
The best (or at least most DB-agnostic) way to do this is with a 'lock' table.
This table will have a single row, and will act as a semaphore to ensure serial access.
Make sure that this method is wrapped in a transaction
// this line will block if any other thread already has a lock
// until that thread's transaction commits
Lock lock = entityManager.find(Lock.class, Lock.ID, LockModeType.PESSIMISTIC_WRITE);
// just some change to the row, it doesn't matter what
lock.setDateUpdated(new Timestamp(System.currentTimeMillis()));
entityManager.merge(lock);
entityManager.flush();
// find your entity by unique constraint
// if it exists, update it
// if it doesn't, insert it

Hibernate and its query language offer support for an insert statement. So you can actually write that query with HQL. See here for more information. http://docs.jboss.org/hibernate/orm/5.0/userguide/html_single/Hibernate_User_Guide.html#_hql_syntax_for_insert

It sounds like an upsert case, that can be handled as suggested here.

Find > Act - The service method would use the repository to see if a entry already exists, and then either update the found entry or save a new one as needed. This does not work.
Why does this not work?
Have you considered "optimistic locking"?
These two posts may help:
https://www.baeldung.com/jpa-optimistic-locking
https://www.baeldung.com/java-jpa-transaction-locks

Hibernate - ConstraintViolation when persisting entity

I have an entity that contains a relationship to another entity in a manner I've never had to encounter before, and I'm getting an exception: "org.hibernate.exception.ConstraintViolationException: could not execute statement".
The parent entity is called "Post". A post can contain several Keyword entities. Keyword entities are unique by value, that is, if two posts contain the same keyword, both posts reference the same keyword entity.
My thought process was that there are many posts, each referencing many keywords, and any one keyword can be referenced by multiple posts, so it should be an #ManyToMany relationship. Obviously, it's not working. Inspecting the database shows that it is successfully persisting a few posts before it starts failing. As long as all the keywords are unique, it seems to be fine, but I'm thinking that it is dying whenever it's trying to persist a post with a keyword that is already being referenced by another post. Not sure how to fix this.
Here is what the classes look like (short version):
Post:
#Entity
public class Post implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "post_id_seq")
#SequenceGenerator(name = "post_id_seq", sequenceName = "post_id_seq", allocationSize = 1)
private Long id;
#ElementCollection(fetch = FetchType.EAGER)
#ManyToMany(cascade = CascadeType.ALL)
private Set<Keyword> keywords = new HashSet<>();
}
Keyword:
#Entity
public class Keyword implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "keyword_id_seq")
#SequenceGenerator(name = "keyword_id_seq", sequenceName = "keyword_id_seq", allocationSize = 1)
private Long id;
#Column(name = "KEYWORD_VALUE")
private String value;
private int count = 1;
}
UPDATE:
Here is the code I use in my service class to add a keyword to a post. Basically I have a Post object already that has Keywords filled in (request comes in via AJAX from a web front end and Spring unmarshals it automatically to a Post object). I have to loop through each keyword and see if an entity with the same value already exists in persistence. If so, increment the count for that keyword, merge it, then add that entity to the set that will end up replacing the Set that came in the request. If it doesn't already exist, I just use the Keyword that came in the request. Previously, I wasn't saving/merging the Keywords independently before adding them to the Post and persisting the post, but I started getting errors stating:
org.hibernate.TransientObjectException: object references an unsaved
transient instance - save the transient instance before flushing:
com.saic.jswe.clients.swtc.domain.social.Keyword
Anyway, here is my service code:
public void addPost(Post post){
Set<Keyword> keywords = new HashSet<>();
for (Keyword keyword : post.getKeywords()) {
Keyword persistedKeyword = keywordDao.findByValue(keyword.getValue());
if (persistedKeyword != null) {
persistedKeyword.setCount(persistedKeyword.getCount() + 1);
keywordDao.merge(persistedKeyword);
keywords.add(persistedKeyword);
} else {
keywordDao.persist(keyword);
keywords.add(keyword);
}
}
post.setKeywords(keywords);
postDao.persist(post);
}
Also, during my testing when I'm getting this error, it's just a single thread attempting to add test Post objects one at a time.
Checking the logs, here is the actual constraint violation:
rg.postgresql.util.PSQLException: ERROR: insert or update on table
"keyword" violates foreign key constraint
"fk_3tcnkw7v196mudsgmy3nriibl" Detail: Key (id)=(1) is not present
in table "post".
Hmmm... per the above code, it should only be adding a reference to a Keyword object with an ID if it did in fact find it in persistence. The keyword objects coming in with the Post object via the request should all have null IDs as they're not yet persisted.

I found where the issue came into play. A join table was being created called "post_keywords". It had 2 columns, one called "post" and one called "keyword". Each row represented the ID of a post and the ID of a keyword contained in that post. If there were multiple keywords in a post, there could be duplicate entries in the post column. However, as soon as a different post entity attempted to reference a keyword that was already used, it would complain about that ID already being present. Here's the visual:
post | keyword
-----+--------
1 | 1
1 | 2
1 | 4
2 | 3
2 | 4 <--- this would be a problem since keyword 4 is already related to post 1
So my knowledge/understanding of JPA is pretty weak, but I've only ever needed real basic relationships. Given that I understood where the problem was happening, I decided to quit playing and experimenting and start reading.
For a minute, I thought I found a solution just using a OneToMany relationship, because I didn't necessarily care or need the keyword entity to directly know which posts reference it. This was incorrect, however. I could get that code to execute without error, but I ended up with each keyword only being owned by one entity. As each post tried to reference that keyword, it would just override the previous ownership of the keyword. Anyway, I really did need a ManyToMany relationship.
I ended up finding examples (http://en.wikibooks.org/wiki/Java_Persistence/ManyToMany) showing tables where the multiple child entities reference the same parent entity, so I just implemented the same JPA attributes in my code and viola, it worked. Here is what the code looks like now:
#Entity
public class Post implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "post_id_seq")
#SequenceGenerator(name = "post_id_seq", sequenceName = "post_id_seq", allocationSize = 1)
#Column(name="POST_ID")
private Long id;
#ManyToMany(fetch = FetchType.EAGER)
#JoinTable(
name="POST_KEYWORD",
joinColumns={#JoinColumn(name="POST_ID", referencedColumnName="POST_ID")},
inverseJoinColumns={#JoinColumn(name="KEYWORD_ID", referencedColumnName="ID")})
private Set<Keyword> keywords = new HashSet<>();
}

How to tell Hibernate to conditionally ignore columns in CRUD operations

Is it possible to somehow tell Hibernate to conditionally ignore a missing column in a database table while doing the CRUD operations?
I've got a Java application using Hibernate as persistence layer. I'd like to be able to somehow tell Hibernate: If database version < 50, then ignore this column annotation (or set it transient).
This situation arises due to different database versions at different clients, but same entity code for all sites. For example, I've got a class, where the column description2 might miss in some databases.
#Entity
#Table(name = "MY_TABLE")
public class MyTable implements java.io.Serializable {
private Integer serialNo;
private String pickCode;
private String description1;
private String description2;
#Id
#Column(name = "Serial_No", nullable = false)
#GenericGenerator(name = "generator", strategy = "increment")
#GeneratedValue(generator = "generator")
public Integer getSerialNo() {
return this.serialNo;
}
#Column(name = "Pick_Code", length = 25)
public String getPickCode() {
return this.pickCode;
}
#Column(name = "Description1")
public String getDescription1() {
return this.description1;
}
#Column(name = "Description2") // <- this column might miss in some databases
//#TransientIf(...) <- something like this would be nice, or any other solution
public String getDescription2() {
return this.description2;
}
}
Background: I have a large application with a lot of customizations for different clients. Now it happens from time to time that one client (out of lets say 500) gets a new feature that requires a database structure update (e.g. a new field in a table). I release a new version for him, he runs a database schema update and can use the new feature. But all other clients won't do an incremental database update each time when any user gets a new feature. They just want to use the latest version, but are affected by the new feature (for that one client) they will never use.

I think it is only possible if you separate the mapping definition from the entities so that you can replace it. Thus you can not use annotation based mapping.
Instead I would suggest to use xml based mapping and create different xml mapping files for each client. Since you have about 500 clients you might want to create groups of clients who all share the same mapping file.
At least I think it will be very hard to maintain the different clients needs with one entity model and it will lead to a complex code structure. E.g. if you add properties to the enties that can be null for some clients than you will also add a lot more null checks to your code. One null check for each client specific property.

Best way to prevent unique constraint violations with JPA

I have an Keyword and a KeywordType as entities. There are lots of keywords of few types. When trying to persist the second keyword of a type, the unique constraint is violated and the transaction is rolled back. Searching SO i found several possibilies (some of them from different contexts, so I'm not sure of their validity here) - this post and this post advise catching the Exception which would be of no use to me as I end up where I started and still need to somehow persist the keyword.
Same applies to locking proposed for a different situaltion here Custom insert statements as proposed in this and this posts wouldn't work proper I guess, since I'm using Oracle and not MySQL and woulnd like to tie the implementation to Hibernate. A different workaround would be trying to retrieve the type first in the code generating the keywords, and set it on the keyword if found or create a new one if not.
So, what would be the best - most robust, portable (for different databases and persistence providers) and sane approach here?
Thank you.
The involved entities:
public class Keyword {
#Id
#GeneratedValue
private long id;
#Column(name = "VALUE")
private String value;
#ManyToOne
#JoinColumn(name = "TYPE_ID")
private KeywordType type;
...
}
and
#Entity
#Table(uniqueConstraints = {#UniqueConstraint(columnNames = { "TYPE" }) })
public class KeywordType {
#Id
#GeneratedValue
private long id;
#Column(name = "TYPE")
private String type;
...
}

Your last solution is the right one, IMO. Search for the keyword type, and if not found, create it.
Catching the exception is not a good option because
it's hard to know which exception to catch and make your code portable across JPA and DB engines
The JPA engine will be in an undetermined state after such an exception, and you should always rollback in this case.
Note however that with this technique, you might still have two transactions searching for the same type in parallel, and then try to insert it in parallel. One of the transaction will rollback, but it will be much less frequent.

If you're using EJB 3.1 and you don't mind serializing this operation, a singleton bean using container managed concurrency can solve the problem.
#Singleton
#ConcurrencyManagement(ConcurrencyManagementType.CONTAINER)
public class KeywordTypeManager
{
#Lock(LockType.WRITE)
public void upsert(KeywordType keywordType)
{
// Only one thread can execute this at a time.
// Your implementation here:
// ...
}
#Inject
private KeywordTypeDao keywordTypeDao;
}

I would go for this option:
A different workaround would be trying
to retrieve the type first in the code
generating the keywords, and set it on
the keyword if found or create a new
one if not.

Found shared references to a collection org.hibernate.HibernateException

I got this error message:
error: Found shared references to a collection: Person.relatedPersons
When I tried to execute addToRelatedPersons(anotherPerson):
person.addToRelatedPersons(anotherPerson);
anotherPerson.addToRelatedPersons(person);
anotherPerson.save();
person.save();
My domain:
Person {
static hasMany = [relatedPersons:Person];
}
any idea why this happens ?

Hibernate shows this error when you attempt to persist more than one entity instance sharing the same collection reference (i.e. the collection identity in contrast with collection equality).
Note that it means the same collection, not collection element - in other words relatedPersons on both person and anotherPerson must be the same. Perhaps you're resetting that collection after entities are loaded? Or you've initialized both references with the same collection instance?

I had the same problem. In my case, the issue was that someone used BeanUtils to copy the properties of one entity to another, so we ended up having two entities referencing the same collection.
Given that I spent some time investigating this issue, I would recommend the following checklist:
Look for scenarios like entity1.setCollection(entity2.getCollection()) and getCollection returns the internal reference to the collection (if getCollection() returns a new instance of the collection, then you don't need to worry).
Look if clone() has been implemented correctly.
Look for BeanUtils.copyProperties(entity1, entity2).

Explanation on practice. If you try to save your object, e.g.:
Set<Folder> folders = message.getFolders();
folders.remove(inputFolder);
folders.add(trashFolder);
message.setFiles(folders);
MESSAGESDAO.getMessageDAO().save(message);
you don't need to set updated object to a parent object:
message.setFiles(folders);
Simple save your parent object like:
Set<Folder> folders = message.getFolders();
folders.remove(inputFolder);
folders.add(trashFolder);
// Not set updated object here
MESSAGESDAO.getMessageDAO().save(message);

Reading online the cause of this error can be also an hibernate bug, as workaround that it seems to work, it is to put a:
session.clear()
You must to put the clear after getting data and before commit and close, see example:
//getting data
SrReq sr = (SrReq) crit.uniqueResult();
SrSalesDetailDTO dt=SrSalesDetailMapper.INSTANCE.map(sr);
//CLEAR
session.clear();
//close session
session.getTransaction().commit();
session.close();
return dt;
I use this solution for select to database, for update or insert i don't know if this solution can work or can cause problems.
My problem is equal at 100% of this: http://www.progtown.com/topic128073-hibernate-many-to-many-on-two-tables.html

I have experienced a great example of reproducing such a problem.
Maybe my experience will help someone one day.
Short version
Check that your #Embedded Id of container has no possible collisions.
Long version
When Hibernate instantiates collection wrapper, it searches for already instantiated collection by CollectionKey in internal Map.
For Entity with #Embedded id, CollectionKey wraps EmbeddedComponentType and uses #Embedded Id properties for equality checks and hashCode calculation.
So if you have two entities with equal #Embedded Ids, Hibernate will instantiate and put new collection by the first key and will find same collection for the second key.
So two entities with same #Embedded Id will be populated with same collection.
Example
Suppose you have Account entity which has lazy set of loans.
And Account has #Embedded Id consists of several parts(columns).
#Entity
#Table(schema = "SOME", name = "ACCOUNT")
public class Account {
#OneToMany(fetch = FetchType.LAZY, mappedBy = "account")
private Set<Loan> loans;
#Embedded
private AccountId accountId;
...
}
#Embeddable
public class AccountId {
#Column(name = "X")
private Long x;
#Column(name = "BRANCH")
private String branchId;
#Column(name = "Z")
private String z;
...
}
Then suppose that Account has additional property mapped by #Embedded Id but has relation to other entity Branch.
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "BRANCH")
#MapsId("accountId.branchId")
#NotFound(action = NotFoundAction.IGNORE)//Look at this!
private Branch branch;
It could happen that you have no FK for Account to Brunch relation id DB so Account.BRANCH column can have any value not presented in Branch table.
According to #NotFound(action = NotFoundAction.IGNORE) if value is not present in related table, Hibernate will load null value for the property.
If X and Y columns of two Accounts are same(which is fine), but BRANCH is different and not presented in Branch table, hibernate will load null for both and Embedded Ids will be equal.
So two CollectionKey objects will be equal and will have same hashCode for different Accounts.
result = {CollectionKey#34809} "CollectionKey[Account.loans#Account#43deab74]"
role = "Account.loans"
key = {Account#26451}
keyType = {EmbeddedComponentType#21355}
factory = {SessionFactoryImpl#21356}
hashCode = 1187125168
entityMode = {EntityMode#17415} "pojo"
result = {CollectionKey#35653} "CollectionKey[Account.loans#Account#33470aa]"
role = "Account.loans"
key = {Account#35225}
keyType = {EmbeddedComponentType#21355}
factory = {SessionFactoryImpl#21356}
hashCode = 1187125168
entityMode = {EntityMode#17415} "pojo"
Because of this, Hibernate will load same PesistentSet for two entities.

In my case, I was copying and pasting code from my other classes, so I did not notice that the getter code was bad written:
#OneToMany(fetch = FetchType.LAZY, mappedBy = "credito")
public Set getConceptoses() {
return this.letrases;
}
public void setConceptoses(Set conceptoses) {
this.conceptoses = conceptoses;
}
All references conceptoses but if you look at the get says letrases

I too got the same issue, someone used BeanUtils.copyProperties(source, target). Here both source and target, are using the same collection as attribute.
So i just used the deep copy as below..
How to Clone Collection in Java - Deep copy of ArrayList and HashSet

Consider an entity:
public class Foo{
private<user> user;
/* with getters and setters */
}
And consider an Business Logic class:
class Foo1{
List<User> user = new ArrayList<>();
user = foo.getUser();
}
Here the user and foo.getUser() share the same reference. But saving the two references creates a conflict.
The proper usage should be:
class Foo1 {
List<User> user = new ArrayList<>();
user.addAll(foo.getUser);
}
This avoids the conflict.

I faced similar exception in my application. After looking into the stacktrace it was clear that exception was thrown within a FlushEntityEventListener class.
In Hibernate 4.3.7 the MSLocalSessionFactory bean no longer supports the eventListeners property. Hence, one has to explicitly fetch the service registry from individual Hibernate session beans and then set the required custom event listeners.
In the process of adding custom event listeners we need to make sure the corresponding default event listeners are removed from the respective Hibernate session.
If the default event listener is not removed then the case arises of two event listeners registered against same event. In this case while iterating over these listeners, against first listeners any collections in the session will be flagged as reached and while processing the same collection against second listener would throw this Hibernate exception.
So, make sure that when registering custom listeners corresponding default listeners are removed from registry.

My problem was that I had setup an #ManyToOne relationship. Maybe if the answers above don't fix your problem you might want to check the relationship that was mentioned in the error message.

Posting here because it's taken me over 2 weeks to get to the bottom of this, and I still haven't fully resolved it.
There is a chance, that you're also just running into this bug which has been around since 2017 and hasn't been addressed.
I honestly have no clue how to get around this bug. I'm posting here for my sanity and hopefully to shave a couple weeks of your googling. I'd love any input anyone may have, but my particular "answer" to this problem was not listed in any of the above answers.

I had to replace the following collection initilization:
challenge.setGoals(memberChallenge.getGoals());
with
challenge.setGoals(memberChallenge.getGoals()
.stream()
.map(dmo -> {
final ChallengeGoal goal = new ChallengeGoalImpl();
goal.setMemberChallenge(challenge);
goal.setGoalDate(dmo.getGoalDate());
goal.setGoalValue(dmo.getGoalValue());
return goal;
})
.collect(Collectors.toList()));

I changed
#OneToMany( cascade= CascadeType.ALL)
#JoinColumn(
name = "some_id",
referencedColumnName = "some_id"
)
to
#OneToMany(mappedBy = "some_id", cascade= CascadeType.ALL)

You're using pointers(indirectly), so sometimes you're copying the memory address instead of the object/collection you want. Hibernate checks this and throw that error. Here's what can you do:
Don't copy the object/collection;
Initiate a new empty one;
Make a function to copy it's content and call it;
For example:
public Entity copyEntity(Entity e){
Entity copy = new Entity();
e.copy(name);
e.setCollection2(null);
e.setCollection3(copyCollection(e.getCollection3());
return copy;
}

In a one to many and many to one relationship this error will occur. If you attempt to devote same instance from many to one entity to more than one instance from one to many entity.
For example, each person can have many books but each of these books can be owned by only one person if you consider more than one owner for a book this issue is raised.