Well, i'm trying to making a batch insert in JPA but i think this don't work.
My method is this :
public void saveBatch(List<? extends AbstractBean> beans) {
try {
begin();
logger.info("Salvando em batch " + beans.size() + " bean(s)");
for (int i = 0; i < beans.size(); i++) {
if (i % 50 == 0) {
entityManager.flush();
entityManager.clear();
}
entityManager.merge(beans.get(i));
}
commit();
} catch (Exception e) {
logger.error("Ocorreu um erro ao tentar salvar batch. MSG ORIGINAL: "
+ e.getMessage());
rollback();
throw new DAOException("Ocorreu um erro ao tentar salvar batch");
}
}
My ideia is that each 50 rows the hibernate will make:
insert into tableA values (),(),()...
But watching the log i see one INSERT for each merge() command link this:
insert into tableA values ()
insert into tableA values ()
insert into tableA values ()
insert into tableA values ()
What is wrong ? This is correct ?
Hibernate does not enable batching by default. You will want to consider the following settings (I think at least batch_size is required to get any batch inserts/updates to work):
hibernate.jdbc.batch_size
A non-zero value enables use of JDBC2 batch updates by Hibernate. e.g.
recommended values between 5 and 30
hibernate.jdbc.batch_versioned_data
Set this property to true if your JDBC driver returns correct row
counts from executeBatch(). It is usually safe to turn this option on.
Hibernate will then use batched DML for automatically versioned data.
Defaults to false. e.g. true | false
hibernate.order_updates (similarly, hibernate.order_inserts)
Forces Hibernate to order SQL updates by the primary key value of the
items being updated. This will result in fewer transaction deadlocks
in highly concurrent systems. e.g. true | false
Related
I would like to improve my insert performance on hibernate with my Java EJB Application deployed on Wildfly 10.0
I would like to perform batch inser on hibernate but with my code the insert is more slowly than without batch insert.
Here is my Method which performs the insert. It gets a List of customers and should persist it. The If-clause makes the difference.
public List<Customer> insertCustomerList(List<Customer> cusList)
{
try {
int batchSize = 25;
for ( int i = 0; i < cusList.size(); ++i ) {
em.persist( cusList.get(i) );
if ( i > 0 && i % batchSize == 0 ) {
System.out.println("FLUSHFG");
//flush a batch of inserts and release memory
em.flush();
em.clear();
}
}
} catch (RuntimeException e) {
throw e;
}
return cusList;
}
In my opinion it should be faster with the flush and clear but it is way more slowly than without!
In my Wildfly container I can not activate a new session or a new transaction because I get an error.
Can you tell me how I can manage the batch insert with wildfly so that the insert of large and many entity will be more fast and not more slowly?
In my persistence xml I have this property:
<property name="hibernate.jdbc.batch_size" value="25"/>
Thanks!
Hi want to delete the millions of rows from the table in batch to avoid locking. I am trying below code but its deleting all the rows.
Session session;
try {
session = dao.getHibernateTemplate().getSessionFactory().getCurrentSession();
} catch (HibernateException e) {
session = dao.getHibernateTemplate().getSessionFactory().openSession();
}
String sql = "delete from "+clazz.getSimpleName();
session.createQuery(sql).setFetchSize(limit).executeUpdate();
dao.getHibernateTemplate().flush();
Is there any better way of doing it
I am considering "clazz.getSimpleName();" is returning a table name.
If this is the case than your query is - "delete from 'tablename'" here you are not specifying any condition which restrict the delete statement, that's why it is deleting all the rows from the table.
As you are using setFetchSize - setFetchSize(int value) is a 'hint' to the driver, telling it how many rows it should fetch.
I think this method is not require in case of delete query.
I am using following method to update data in database.
String hql = "UPDATE EmployeeSalary set salary = :sl,"
+ "monthYr=:dt "
+ "WHERE id =:id and client.id=:cid";
for (EmployeeSalary e : eList) {
Query query = session.createQuery(hql);
query.setParameter("sl", e.getSalary());
query.setParameter("dt", e.getMonthYr());
query.setParameter("id", e.getId());
query.setParameter("cid", e.getClient().getId());
int result = query.executeUpdate();
System.out.println("result is " + result);
if (eAttList.size() % 20 == 0) {
session.flush();
session.clear();
}
}
Is there any problem with code.
On execution it is showing
result is 0
How to resolve above problem.
The documentation about update queries says:
No "Forms of join syntax", either implicit or explicit, can be specified in a bulk HQL query. Sub-queries can be used in the where-clause, where the subqueries themselves may contain joins.
Your query seems to violate this rule: client.id=:cid is an implicit join to the client entity.
Note that you're making your life difficult. You could simply get the entity by ID from the session (using Session.get()), and update it. Update queries are useful to update many rows at once.
I have a code looking like this:
Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
try {
for ( Customer customer: customers ) {
i++;
session.update(customer);
if ( i % 200 == 0 ) { //200, same as the JDBC batch size
//flush a batch of inserts and release memory:
session.flush();
session.clear();
}
}
} catch (Exc e) {
//TODO want to know customer id here!
}
tx.commit();
session.close();
Say, at some point session.flush() raises an DataException, because one of the fields did not map into the database column size, one of those batch of 200 customers. Nothing wrong with it, data can be corrupted, it's ok in this case. BUT, I really need to know the customer id which failed. Database returns meaningless error message, not stating what was the params of the statement, etc. Catched exception also does not contain which customer did fail, only the sql statement text, looking like 'update Customer set name=?'
Can I somehow determine it using the hibernate session? Does it store somewhere the information about last entity it tried to save down?
I'm using iBatis/Java and Postgres 8.3.
When I do an insert in ibatis i need the id returned.
I use the following table for describing my question:
CREATE TABLE sometable ( id serial NOT NULL, somefield VARCHAR(10) );
The Sequence sometable_id_seq gets autogenerated by running the create statement.
At the moment i use the following sql map:
<insert id="insertValue" parameterClass="string" >
INSERT INTO sometable ( somefield ) VALUES ( #value# );
<selectKey keyProperty="id" resultClass="int">
SELECT last_value AS id FROM sometable_id_seq
</selectKey>
</insert>
It seems this is the ibatis way of retrieving the newly inserted id. Ibatis first runs a INSERT statement and afterwards it asks the sequence for the last id.
I have doubts that this will work with many concurrent inserts.
Could this cause problems? Like returning the id of the wrong insert?
( See also my related question about how to get ibatis to use the INSERT .. RETURING .. statements )
This is definitely wrong. Use:
select currval('sometable_id_seq')
or better yet:
INSERT INTO sometable ( somefield ) VALUES ( #value# ) returning id
which will return you inserted id.
Here is simple example:
<statement id="addObject"
parameterClass="test.Object"
resultClass="int">
INSERT INTO objects(expression, meta, title,
usersid)
VALUES (#expression#, #meta#, #title#, #usersId#)
RETURNING id
</statement>
And in Java code:
Integer id = (Integer) executor.queryForObject("addObject", object);
object.setId(id);
I have another thought. ibatis invokes the insert method delegate the Class: com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate,with the code:
try {
trans = autoStartTransaction(sessionScope, autoStart, trans);
SelectKeyStatement selectKeyStatement = null;
if (ms instanceof InsertStatement) {
selectKeyStatement = ((InsertStatement) ms).getSelectKeyStatement();
}
// Here we get the old value for the key property. We'll want it later if for some reason the
// insert fails.
Object oldKeyValue = null;
String keyProperty = null;
boolean resetKeyValueOnFailure = false;
if (selectKeyStatement != null && !selectKeyStatement.isRunAfterSQL()) {
keyProperty = selectKeyStatement.getKeyProperty();
oldKeyValue = PROBE.getObject(param, keyProperty);
generatedKey = executeSelectKey(sessionScope, trans, ms, param);
resetKeyValueOnFailure = true;
}
StatementScope statementScope = beginStatementScope(sessionScope, ms);
try {
ms.executeUpdate(statementScope, trans, param);
}catch (SQLException e){
// uh-oh, the insert failed, so if we set the reset flag earlier, we'll put the old value
// back...
if(resetKeyValueOnFailure) PROBE.setObject(param, keyProperty, oldKeyValue);
// ...and still throw the exception.
throw e;
} finally {
endStatementScope(statementScope);
}
if (selectKeyStatement != null && selectKeyStatement.isRunAfterSQL()) {
generatedKey = executeSelectKey(sessionScope, trans, ms, param);
}
autoCommitTransaction(sessionScope, autoStart);
} finally {
autoEndTransaction(sessionScope, autoStart);
}
You can see that the insert and select operator are in a Transaction. So I think there is no concureency problem with the insert method.