I have a simple use case:
I want to insert an entity that doesn't exist.
Here is the entity class, it has combined unique columns:
#Entity
#Table(uniqueConstraints = {
#UniqueConstraint(columnNames = {
"firstName", "lastName"
})
})
public class Customer {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
private String firstName;
private String lastName;
And here is the simple insertion logic:
Optional<Customer> customer = customerRepository.findByFirstNameAndLastName("John", "Doe");
if (!customer.isPresent()) {
customerRepository.save(new Customer("John", "Doe"));
}
when I call this via concurrent threads, I get this ConstraintViolationException and that's obvious
insert into customer (first_name, last_name, id) values (?, ?, ?) [23505-200]]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement] with root cause
org.h2.jdbc.JdbcSQLIntegrityConstraintViolationException: Unique index or primary key violation: "PUBLIC.UKKIYY7M3FWM4VO5NIL9IBP5846_INDEX_5 ON PUBLIC.CUSTOMER(FIRST_NAME, LAST_NAME) VALUES 8"; SQL statement:
insert into customer (first_name, last_name, id) values (?, ?, ?) [23505-200]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:459) ~[h2-1.4.200.jar:1.4.200]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:429) ~[h2-1.4.200.jar:1.4.200]
I can catch this exception, but ConstraintViolationException can occur in any scenario other than the unique index.
I also tried JPA's Optimistic locking but it seems to work only in the update and delete operations but not in insertion.
I don't want native query (like ON DUPLICATE KEY in mysql, ON CONFLICT in PostgresSQL) because I want to make operations to be handled in JPA itself.
Also, I cannot make it threadsafe because the application will be running on multiple JVMs.
Is there any way somehow I can handle this, like locking on insert or atleast update on same keys?
Related
I have simple Spring Boot app with in-memory H2 database, and I have two entities:
#Entity
#Data
#NoArgsConstructor
#AllArgsConstructor
public class DictEntity{
#Id
Long id;
#Column(unique = true)
String name;
}
and
#Entity
#Data
#AllArgsConstructor
#NoArgsConstructor
public class ComplexEntity {
#Id
Long id;
#Column
String name;
#Column
#OneToMany(cascade = CascadeType.ALL)
List<DictEntity> properties;
}
Hibernate generates me such DDL:
Hibernate: create table complex_entity (id bigint not null, name varchar(255), primary key (id))
Hibernate: create table complex_entity_entities (complex_entity_id bigint not null, entities_id bigint not null)
Hibernate: create table dict_entity (id bigint not null, name varchar(255), primary key (id))
Hibernate: alter table complex_entity_entities add constraint UK_nxr507qlfqwm2vk6gqes2b3lc unique (entities_id)
Hibernate: alter table dict_entity add constraint UK_m9sbjtymc6urpykp5ko3faep1 unique (name)
Hibernate: alter table complex_entity_entities add constraint FKn66943skv39n53nfkj6wwsnl2 foreign key (entities_id) references dict_entity
Hibernate: alter table complex_entity_entities add constraint FK6wqn8as4rkbi5caulfyex6sxm foreign key (complex_entity_id) references complex_entity
And then I have a code:
foo = anEntityRepository.save(new DictEntity(1L, "foo"));
bar = anEntityRepository.save(new DictEntity(2L, "bar"));
ComplexEntity m1 = complexEntityRepository.save(new ComplexEntity(1l, "m1", Collections.singletonList(foo)));
ComplexEntity m2 = complexEntityRepository.save(new ComplexEntity(2l, "m2", Collections.singletonList(foo)));
On last line i got an Exception:
Caused by: org.h2.jdbc.JdbcSQLIntegrityConstraintViolationException: Unique index or primary key violation: "PUBLIC.UK_NXR507QLFQWM2VK6GQES2B3LC_INDEX_9 ON PUBLIC.COMPLEX_ENTITY_ENTITIES(ENTITIES_ID) VALUES 1"; SQL statement:
insert into complex_entity_entities (complex_entity_id, entities_id) values (?, ?)
What went wrong?
Your error
Caused by: org.h2.jdbc.JdbcSQLIntegrityConstraintViolationException: Unique index or primary key violation: "PUBLIC.UK_NXR507QLFQWM2VK6GQES2B3LC_INDEX_9 ON PUBLIC.COMPLEX_ENTITY_ENTITIES(ENTITIES_ID) VALUES 1"; SQL statement:
insert into complex_entity_entities (complex_entity_id, entities_id) values (?, ?)
has nothing to do with what you suspect in the question as a cause, as you mention the
#Column
#OneToMany(cascade = CascadeType.ALL)
List<DictEntity> properties;
Probably the
#NoArgsConstructor
public class ComplexEntity {
...
has also another field which is used to create the mapping table complex_entity_entities and this is the one that is failing.
Update the question with all relevant code to find exactly the issue. I expect that you have a field named entities upon ComplexEntity.class.
I suspect though that the ComplexEntity has reference upon other ComplexEntity and this is the type of issue here.
I have a NOTIFICATION table who contains one ManyToMany association :
#Entity
#Table(name="NOTIFICATION")
#NamedQuery(name="Notification.findAll", query="SELECT f FROM Notification f")
public class Notification {
/** SOME COLUMN DEFINITION NOT IMPORTANT FOR MY CASE
COD, DATE, ID_THEME, ID_TYP, IC_ARCH, ID_CLIENT, INFOS, NAME, TITRE_NOT, ID_NOT
**/
#ManyToMany
#JoinTable(
name="PJ_PAR_NOTIF"
, joinColumns={
#JoinColumn(name="ID_NOTIF")
}
, inverseJoinColumns={
#JoinColumn(name="ID_PJ_GEN")
}
)
private List<PiecesJointesGen> piecesJointesGens;
}
So, I have an association table called PJ_PAR_NOTIF.
I try to persist a Notification entity. Here is piecesJointesGens initialisation, from a Value Object :
#PersistenceContext(unitName="pu/middle")
private EntityManager entityMgr;
FoaNotification lFoaNotification = new FoaNotification();
for(PieceJointeGenVO lPJGenVO : pNotificationVO.getPiecesJointes()){
PiecesJointesGen lPiecesJointesGen = new PiecesJointesGen();
lPiecesJointesGen.setLienPjGen(lPJGenVO.getLienPieceJointeGen());
lPiecesJointesGen.setIdPjGen(lPJGenVO.getIdPieceJointeGen());
lNotification.getFoaPiecesJointesGens().add(lFoaPiecesJointesGen);
}
entityMgr.persist(pNotification);
The persist doesn't work. JPA generate a first insert for my Notification object, that is ok :
insert
into
NOTIFICATION
(COD, DATE, ID_THEME, ID_TYP, IC_ARCH, ID_CLIENT, INFOS, NAME, TITRE_NOT, ID_NOT)
values
(?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
Then, JPA try to insert values in my association table, but piecesJointesGen doesn't exists for the moment :
insert
into
PJ_PAR_NOTIF
(ID_NOTIF, ID_PJ_GEN)
values
(?, ?)
So, I have this error :
GRAVE: EJB Exception: : java.lang.IllegalStateException: org.hibernate.TransientObjectException: object references an unsaved transient instance - save the transient instance before flushing: com.entities.PiecesJointesGen
Is there a way to tell JPA to insert piecesJointesGen before the PJ_PAR_NOTIF insert ?
Modify piecesJointesGens mapping to #ManyToMany(cascade = CascadeType.PERSIST).
I am using JPA's eclipseLink to perform CRUD operations on my entities. I am facing following problem:
I have two tables in DB:
CREATE TABLE User (
id INTEGER PRIMARY KEY AUTO_INCREMENT,
username VARCHAR(30) NOT NULL UNIQUE,
email VARCHAR(50) NOT NULL UNIQUE,
password VARCHAR(255) NOT NULL,
signUpDate timestamp NOT NULL DEFAULT NOW()
);
CREATE TABLE Friendship (
id INTEGER PRIMARY KEY AUTO_INCREMENT,
friendsSince timestamp NOT NULL DEFAULT NOW(),
user1_Id INTEGER NOT NULL REFERENCES User(id),
user2_Id INTEGER NOT NULL REFERENCES User(id)
);
The corresponding Entities
#Entity
public class User {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String username;
private String email;
private String password;
#Temporal(value = TemporalType.DATE)
private Date signUpDate;
// constructors & setters & getters ...
}
#Entity
public class Friendship {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#ManyToOne
#JoinColumn(name="user1_Id", referencedColumnName = "id")
private User user1;
#ManyToOne
#JoinColumn(name="user2_Id", referencedColumnName = "id")
private User user2;
#Temporal(value = TemporalType.DATE)
private Date friendsSince;
// constructors & setters & getters ...
}
If I want to retrieve a list of some entities, according to "WHERE" clause of a query I get this "unknown state or association field [user1_Id] of class [com.filip.xxx.Friendship]" error.
Specifically:
I try to build this query:
Query query = mgr.createQuery("select f.id ,f.friendsSince, f.user1_Id, f.user2_Id from Friendship f where f.user1_Id = :user1Id and f.user2_Id = :user2Id or f.user1_Id = :user11Id and f.user2_Id = :user12Id");
and recieve this exception:
java.lang.IllegalArgumentException: An exception occurred while creating a query in EntityManager:
Exception Description: Error compiling the query [select f.id ,f.friendsSince, f.user1_Id, f.user2_Id from Friendship f where f.user1_Id = :user1Id and f.user2_Id = :user2Id or f.user1_Id = :user11Id and f.user2_Id = :user12Id], line 1, column 31: unknown state or association field [user1_Id] of class [com.filip.xxx.Friendship].
It seems like there is a problem with mapping attributes back to the entities, because I have no problem with persisting these two entities.
And interesting is that, if I run this query:
Query query = mgr.createQuery("select f from Friendship f");
It returns me the correct list of all friendships entities.
Notice that the reference variable's name in friendship entity(user1, user2) are not the same as corresponding table's variables (user1_Id, user2_Id). Before I have used the same variable names in entity as in table, but recieved this error at persisting friendship entity:
javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.3.2.v20111125-r10461):
org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column 'USER1_ID' in 'field list'
Error Code: 1054
Call: INSERT INTO FRIENDSHIP (FRIENDSSINCE, USER1_ID, USER2_ID) VALUES (?, ?, ?)
bind => [3 parameters bound]
Basically I don't understand, why eclipse link renames the entity's reference variables (user1 -> USER1_ID, user2 -> USER2_ID) when creating sql query, when it has than problems to map it back to the entities.
I have already tried these solutions:
Build query and return user1_Id column as user1 and user2_Id as user2
select f.id ,f.friendsSince, f.user1_Id as user1, f.user2_Id as user2 from Friendship f where f.user1_Id = :user1Id and f.user2_Id = :user2Id or f.user1_Id = :user11Id and f.user2_Id = :user12Id
but recieved the same IllegalArgumentException as above.
Could you help me solve this problem ?
Thanks
The exception
unknown state or association field [user1_Id] of class [com.filip.xxx.Friendship]
is received because you are using user1_Id name which is a database column name.
From the other hand ElementManager.createQuery() method expects JPQL string which accepts the entity's field name user1. Try to replace your query string with:
select f.id, f.friendsSince, f.user1, f.user2
from Friendship f
where f.user1 = :user1Id and
f.user2 = :user2Id or
f.user1 = :user11Id and
f.user2 = :user12Id
I'm currently facing the well-known and common Hibernate insert batch problem.
I need to save batches 5 millions of rows long. I'm first trying with a much lighter payload. Since I have to insert entities of only 2 types (first all records of type A, then all records of type B, all pointing to common type C ManyToOne parent), I would like to take the most advantage from JDBC batch insert.
I have already read lots of documentation, but none that I have tried worked.
I know that in order to use batch inserts I must not use an entity generator. So I removed the AUTO_INCREMENT ID and I'm setting the ID with a trick: SELECT MAX(ID) FROM ENTITIES and increment every time.
I know that I must flush the session regularly. I'll post code ahead, but anyway I perform a transaction every 500 elements.
I know that I have to set hibernate.jdbc.batch_size consistent with my application's bulk size, so I set it in the LocalSessionFactoryBean (Spring ORM integration)
I know I have to enable rewriting batched statements in connection URL.
Here are my entities
Common parent entity. This gets inserted first in a single transaction. I don't care about auto increment column here. Only one record per batch job
#Entity
#Table(...)
#SequenceGenerator(...)
public class Deal
{
#Id
#Column(
name = "DEAL_ID",
nullable = false)
#GeneratedValue(
strategy = GenerationType.AUTO)
protected Long id;
................
}
One of the children (let's say 2.5M records per batch)
#Entity
#Table(
name = "TA_LOANS")
public class Loan
{
#Id
#Column(
name = "LOAN_ID",
nullable = false)
protected Long id;
#ManyToOne(
optional = false,
targetEntity = Deal.class,
fetch = FetchType.LAZY)
#JoinColumn(
name = "DEAL_ID",
nullable = false)
protected Deal deal;
.............
}
The other children type. Let's say the other 2.5M records
#Entity
#Table(
name = "TA_BONDS")
public class Bond
{
#Id
#Column(
name = "BOND_ID")
#ManyToOne(
fetch = FetchType.LAZY,
optional = false,
targetEntity = Deal.class)
#JoinColumn(
name = "DEAL_ID",
nullable = false,
updatable = false)
protected Deal deal;
}
Simplified code that inserts records
long loanIdCounter = loanDao.getMaxId(), bondIdCounter = bondDao.getMaxId(); //Perform SELECT MAX(ID)
Deal deal = null;
List<Bond> bondList = new ArrayList<Bond>(COMMIT_BATCH_SIZE); //500 constant value
List<Loan> loanList = new ArrayList<Loan>(COMMIT_BATCH_SIZE);
for (String msg: inputStreamReader)
{
log.debug(msg.toString());
if (this is a deal)
{
Deal deal = parseDeal(msg.getMessage());
deal = dealManager.persist(holder.deal); //Called in a separate transaction using Spring annotation #Transaction(REQUIRES_NEW)
}
else if (this is a loan)
{
Loan loan = parseLoan(msg.getMessage());
loan.setId(++loanIdCounter);
loan.setDeal(deal);
loanList.add(loan);
if (loanList.size() == COMMIT_BATCH_SIZE)
{
loanManager.bulkInsert(loanList); //Perform a bulk insert in a single transaction, not annotated but handled manually this time
loanList.clear();
}
}
else if (this is a bond)
{
Bond bond = parseBond(msg.getMessage());
bond.setId(++bondIdCounter);
bond.setDeal(deal);
bondList.add(bond);
if (bondList.size() == COMMIT_BATCH_SIZE) //As above
{
bondManager.bulkInsert(bondList);
bondList.clear();
}
}
}
if (!bondList.isEmpty())
bondManager.bulkInsert(bondList);
if (!loanList.isEmpty())
loanManager.bulkInsert(loanList);
//Flush remaining items, not important
Implementation of bulkInsert:
#Override
public void bulkInsert(Collection<Bond> bonds)
{
// StatelessSession session = sessionFactory.openStatelessSession();
Session session = sessionFactory.openSession();
try
{
Transaction t = session.beginTransaction();
try
{
for (Bond bond : bonds)
// session.persist(bond);
// session.insert(bond);
session.save(bond);
}
catch (RuntimeException ex)
{
t.rollback();
}
finally
{
t.commit();
}
}
finally
{
session.close();
}
}
As you can see from comments, I have tried several combinations of stateful/stateless session. None worked.
My dataSource is a ComboPooledDataSource with following URL
<b:property name="jdbcUrl" value="jdbc:mysql://server:3306/db?autoReconnect=true&rewriteBatchedStatements=true" />
My SessionFactory
<b:bean id="sessionFactory" class="class.that.extends.org.springframework.orm.hibernate3.LocalSessionFactoryBean" lazy-init="false" depends-on="dataSource">
<b:property name="dataSource" ref="phoenixDataSource" />
<b:property name="hibernateProperties">
<b:props>
<b:prop key="hibernate.dialect">${hibernate.dialect}</b:prop> <!-- MySQL5InnoDb-->
<b:prop key="hibernate.show_sql">${hibernate.showSQL}</b:prop>
<b:prop key="hibernate.jdbc.batch_size">500</b:prop>
<b:prop key="hibernate.jdbc.use_scrollable_resultset">false</b:prop>
<b:prop key="hibernate.cache.use_second_level_cache">false</b:prop>
<b:prop key="hibernate.cache.provider_class">org.hibernate.cache.EhCacheProvider</b:prop>
<b:prop key="hibernate.cache.use_query_cache">false</b:prop>
<b:prop key="hibernate.validator.apply_to_ddl">false</b:prop>
<b:prop key="hibernate.validator.autoregister_listeners">false</b:prop>
<b:prop key="hibernate.order_inserts">true</b:prop>
<b:prop key="hibernate.order_updates">true</b:prop>
</b:props>
</b:property>
</b:bean>
Even if my project-wide class extends LocalSessionFactoryBean, it does not override its methods (only adds few project-wide methods)
I'm getting mad since a few days. I read a few articles and none helped me enable batch inserts. I run all of my code from JUnit tests instrumented with Spring context (so I can #Autowire my classes). All of my attempts only produce a lots of separate INSERT statements
https://stackoverflow.com/questions/12011343/how-do-you-enable-batch-inserts-in-hibernate
https://stackoverflow.com/questions/3469364/faster-way-to-batch-saves-with-hibernate
https://forum.hibernate.org/viewtopic.php?p=2374413
https://stackoverflow.com/questions/3026968/high-performance-hibernate-insert
What am I missing?
It's likely your queries are being rewritten but you wouldn't know if by looking at the Hibernate SQL logs. Hibernate does not rewrite the insert statements - the MySQL driver rewrites them. In other words, Hibernate will send multiple insert statements to the driver, and then the driver will rewrite them. So the Hibernate logs only show you what SQL Hibernate sent to the driver, not what SQL the driver sent to the database.
You can verify this by enabling MySQL's profileSQL parameter in connection url:
<b:property name="jdbcUrl" value="jdbc:mysql://server:3306/db?autoReconnect=true&rewriteBatchedStatements=true&profileSQL=true" />
Using an example similar to yours, this is what my output looks like:
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
insert into Person (firstName, lastName, id) values (?, ?, ?)
Wed Feb 05 13:29:52 MST 2014 INFO: Profiler Event: [QUERY] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) duration: 1 ms, connection-id: 81, statement-id: 33, resultset-id: 0, message: insert into Person (firstName, lastName, id) values ('person1', 'Name', 1),('person2', 'Name', 2),('person3', 'Name', 3),('person4', 'Name', 4),('person5', 'Name', 5),('person6', 'Name', 6),('person7', 'Name', 7),('person8', 'Name', 8),('person9', 'Name', 9),('person10', 'Name', 10)
The first 10 lines are being logged by Hibernate though this not what is actually being sent to MySQL database. The last line is coming from MySQL driver and it clearly shows a single batch insert with multiple values and that is what is actually being sent to the MySQL database.
I've mapped my class as follow (omitted other fields as only ID matters):
#Entity
#Table(name = "MODEL_GROUP")
#Cache(usage = CacheConcurrencyStrategy.TRANSACTIONAL)
public class SettlementModelGroup implements Serializable
{
#Id
#GeneratedValue(generator = "MODEL_GROUP_SEQ", strategy = GenerationType.SEQUENCE)
#GenericGenerator(name = "MODEL_GROUP_SEQ",
strategy = "sequence",
parameters = #Parameter(name = "sequence", value = "SEQ_MODEL_GROUP_MODEL_GROUP_ID"))
#Column(name = "MODEL_GROUP_ID", nullable = false)
private Integer modelId;
}
when I'm saving new object:
Integer modelGroupId = sessionFactory.getCurrentSession().save( modelGroup );
System.out.println( modelGroupId );
ID is set as for example 23, but when I look at the database it is actually 24. This is leading to many problems, as I'm using this ID later on. Any idea why it is making this gap?
SQL logs show that everything is fine (I thinks so):
Hibernate:
select
SEQ_MODEL_GROUP_MODEL_GROUP_ID.nextval
from
dual
Hibernate:
insert
into
MODEL_GROUP
(DOMAIN_ID, DESCRIPTION, NAME, PERIOD_TYPE_ID, MODEL_GROUP_TYPE_ID, STATUS_ID, OWNER_ID, MODEL_GROUP_ID)
values
(?, ?, ?, ?, ?, ?, ?, ?)
Trigger and Sequence:
CREATE SEQUENCE "SEQ_MODEL_GROUP_MODEL_GROUP_ID"
INCREMENT BY 1
START WITH 1
NOMAXVALUE
MINVALUE 1
NOCYCLE
NOCACHE
NOORDER
;
CREATE OR REPLACE TRIGGER "TRG_MODEL_GROUP_MODEL_GROUP_ID"
BEFORE INSERT
ON "MODEL_GROUP"
FOR EACH ROW
WHEN (NEW."MODEL_GROUP_ID" is NULL)
BEGIN
SELECT "SEQ_MODEL_GROUP_MODEL_GROUP_ID".NEXTVAL
INTO :NEW."MODEL_GROUP_ID"
FROM DUAL;
END;
Apparently, when Hibernate ask your database for nextValue of ID, it fires also Trigger. So when I ask for ID, I've got number 23 but when actually saving to database by commiting transaction, it is increased again so I've got 24. Solution is described here:
HIbernate issue with Oracle Trigger for generating id from a sequence
To make it work correctly, I changed Trigger:
CREATE OR REPLACE TRIGGER "TRG_MODEL_GROUP_MODEL_GROUP_ID"
BEFORE INSERT
ON "MODEL_GROUP"
FOR EACH ROW
WHEN (NEW."MODEL_GROUP_ID" is NULL)
BEGIN
SELECT "SEQ_MODEL_GROUP_MODEL_GROUP_ID".NEXTVAL
INTO :NEW."MODEL_GROUP_ID"
FROM DUAL;
END;