Is there any way to do batch updates ?
i create a simple entity
#Entity
#Getter
#Setter
public class A {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "A_ID_GENERATOR")
#SequenceGenerator(name = "A_ID_GENERATOR", sequenceName = "a_id_seq")
private Long id;
private String name;
}
next step i generated 10000 objects of class A and put them to db
next step i get list of A from db ,set new name and save them again
#PutMapping
#Transactional
public String updateAllTest(){
var list=aRepository.findAll();
for (int i = 0; i <list.size() ; i++) {
list.get(i).setName("AA"+i);
}
return "OK";
}
what did i expect- i expect that hibernate will do batch update
and hibernate did it -hibernate statistics says - it execute 200 batches ( batch size=500)
next i go to db log files and what i see there- there are no batches- only single updates -10 000 rows
it looks like same with batch insert without adding reWriteBatchedInserts=true to JDBC driver
so is there any way to do batch updates in postgres with hibernate or no?
The key thing to understand is that reusing the same server side statement handle for multiple executions is batching. The log is just telling you about every execution, but that doesn't mean it is slow. It's doing exactly what it should do.
Related
I am trying to insert multiple records in a table using loop and getting sequence number for that using below method. It is getting sequence number for very first time alone and during next iteration below exception is coming.Please help in resolving this
14:03:51.928 [http-nio-8080-exec-5] ERROR org.hibernate.AssertionFailure - HHH000099: an assertion failure occured (this may indicate a bug in Hibernate, but is more likely due to unsafe use of the session): org.hibernate.AssertionFailure: possible non-threadsafe access to session
14:03:51.938 [http-nio-8080-exec-5] ERROR u.s.m.e.p.o.b.c.ORBudgetController - 2020/08/26_14:03:51.938|1|pa23690|bearer 6d7417d8-6835-485e-956d-c362cb7bce2b|createRecord|possible non-threadsafe access to session
#Override
public int getNextSequenceNumber(String seqName) {
int nextValue = 0;
String strQuery = "SELECT " + seqName + ".NEXTVAL FROM DUAL";
Query q = entityManager.createNativeQuery(strQuery);
BigDecimal bd = (BigDecimal) q.getSingleResult();
nextValue = bd.intValue();
return nextValue;
}
You need to generate the sequence automatically, do it manually is a bad practice and can bring you problems in the future. There are several JPA strategies to automatically generate the sequence, this, for example, is The Sequence Strategy
#Entity
// Define a sequence - might also be in another class:
#SequenceGenerator(name="seq", initialValue=1, allocationSize=100)
public class EntityWithSequenceId {
// Use the sequence that is defined above:
#GeneratedValue(strategy=GenerationType.SEQUENCE, generator="seq")
#Id long id;
}
You can also use The Auto Strategy
#Entity
public class EntityWithAutoId1 {
#Id #GeneratedValue(strategy=GenerationType.AUTO) long id;
}
I am having trouble using Hibernate with MSSQL Server 2012. No matter what I do when I try to insert a value in a certain table using Hibernate I get generated id=0.
Here is the model.
#Entity
#Table(name = "tbl_ClientInfo")
public class ClientInfo {
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
#Column (name = "auto_Client_ID", unique=true, nullable=false)
private int auto_Client_ID;
...
Here is the write.
public boolean addNewClient(Client client) {
// there is a class that wraps SessionFactory as singleton
Session session = getSessionFactory().openSession();
Transaction tx = null;
Integer clientFamId; //client family info id
Integer clientId; // actual client id
try {
// create fam info first with some data - need id for ClientInfo
tx = session.beginTransaction();
ClientFam clientFam = new ClientFam();
clientFamId = (Integer) session.save(clientFam);
clientFamId = (Integer) session.getIdentifier(clientFam); // this returns the right id
session.flush();
ClientInfo clientInfo = new ClientInfo();
clientInfo.setABunchOfFields(withStuff); //multiple methods
session.save(clientInfo);
clientInfoId = (Integer) session.getIdentifier(clientInfo); // this is always 0
session.flush();
tx.commit();
} catch (HibernateException e) {
if (tx!=null) tx.rollback();
e.printStackTrace();
return false;
} finally {
session.close();
}
return true;
}
In the database the PK auto_Client_ID is clustered, set to IDENTITY(1,1). Both ClientInfo and ClientFam records are created in the db, but hibernate returns 0. I also tried catching the value from save, but it's also 0.
I don't want to commit in-between separate insert: the transaction is when all inserts are fine (there are more after this, but I can't get to them because of this id issue yet).
The model for ClientFam is almost the same: the id field is #GeneratedValue(strategy=GenerationType.IDENTITY) as well.
I also tried specifying this for ClientInfo
#GeneratedValue(generator="increment", strategy=GenerationType.IDENTITY)
#GenericGenerator(name = "increment", strategy = "increment")
The first time I ran it it returned the correct value. However, the second time I ran it I got an error:
Cannot insert explicit value for identity column in table 'Report' when IDENTITY_INSERT is set to OFF
And that was the end of trying that. Everywhere I looked the recommendation is to use GenerationType.IDENTITY for auto incremented field in the db. That's supposed to return the right values. What might I be doing wrong?
I also tried getting the id from the ClientInfo object itself (I thought it should get written into it) after the right, but it's was also 0. Makes me think something is wrong with my ClientInfo model and/or annotations in it.
I found the problem with my situation - has nothing to do with Hibernate. There is a instead of insert trigger that wasn't returning id and hence messing up what save() returns.
This is just an educated guess, but you might want to remove the "unique=true" clause from the #Column definition. Hibernate may be handling the column as a unique constraint as opposed to a primary key.
Entity with id autogenerated from oracle trigger sequence.
#Entity
#Table(name = "REPORT", schema = "WEBPORTAL")
public class Report {
private Integer id;
....
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator="report_sequence")
#SequenceGenerator(name="report_sequence", sequenceName = "report_id_seq")
#Column(name="REPORT_ID", unique = true, nullable = false)
public Integer getId() {
return id;
}
....
}
Service
#Service("reportService")
public class ReportServiceImpl implements ReportService {
....
#Transactional(readOnly=false)
public void saveOrUpdate(Report report) {
reportDAO.saveOrUpdate(report);
}
}
DAO
#Repository
public class ReportDAOImpl implements ReportDAO {
....
#Override
public Report save(Report report) {
try {
Session session = sessionFactory.getCurrentSession();
session.save(report);
} catch (Exception e) {
logger.error("error", e);
}
return report;
}
}
And When I call service's saveOrUpdate and then try to reach id of entity I get different value than persisted in database. Values on database with autogeneration all is ok. Any suggestions?
reportService.saveOrUpdate(report);
System.out.println(report.getId());
prints: 4150
but saved id in database is: 84
NOTE: My purpose to get Id comes from that I wanted to save childs with cascade. But foreign key on child was different in database(the values of id that I get with getId()).
And Id generated in database is incremented by 2. EX: 80, 82, 84.
UPDATE:
Oracle trigger for sequence generation
CREATE OR REPLACE TRIGGER REPORT_ID_TRIG
BEFORE INSERT ON WEBPORTAL.REPORT
FOR EACH ROW
BEGIN
SELECT report_id_seq.NEXTVAL
INTO :new.report_id
FROM dual;
END;
ANSWER: Trigger should check if id is null
CREATE OR REPLACE TRIGGER REPORT_ID_TRIG
BEFORE INSERT ON WEBPORTAL.REPORT
FOR EACH ROW
WHEN (new.report_id is null)
BEGIN
SELECT report_id_seq.NEXTVAL
INTO :new.report_id
FROM dual;
END;
DESCRIPTION:
#GeneratedValue is not just a sequence generator. It's bit of HiLo algorithm.When it first requests id from database it multiplies it with 50(it can differ) and next 50 new entities will be given ids consequently, and than next request to database. This is to decrease request to database.
The numbers that I get from java was right numbers that should be saved on report.
Without id null value check Hibernate firstly requested for id from database and sequence.nextval called. When hibernate was persisting it(completing transaction) the database called sequence.next second time and set that value to database. So on ReportDetails there was true id value of report and on the Report id it was id set from database.
The problem is that two separate mechanisms are in place to generate the key:
one at Hibernate level which is to call a sequence and use the value to populate an Id column and send it to the database as the insert key
and another mechanism at the database that Hibernate does not know about: the column is incremented via a trigger.
Hibernate thinks that the insert was made with the value of the sequence, but in the database something else occurred. The simplest solution would probably be to remove the trigger mechanism, and let Hibernate populate the key based on the sequence only.
Another Solution:
check your trigger definition that should be in this format
(WHEN (new.report_id is null) ) is important.
CREATE OR REPLACE TRIGGER TRIGGER_NAME
BEFORE INSERT ON TABLE_NAME
FOR EACH ROW
WHEN (new.id is null)
BEGIN
SELECT SEQUENCE_NAME.NEXTVAL
INTO :new.id
FROM dual;
END
I'm using Websphere Application Server 7 with buildin OpenJPA 1.2.3 and an Oracle database. I have the following entity:
#NamedNativeQuery(name=Contract.GIVE_ALL_CONTRACTS,
query="SELECT number, name \n" +
"FROM contracts \n" +
"WHERE startdate <= ?1 \n" +
"AND enddate > ?1",
resultSetMapping = Contract.GIVE_ALL_CONTRACTS_MAPPING)
#SqlResultSetMapping(name = Contract.GIVE_ALL_CONTRACTS_MAPPING,
entities = { #EntityResult(entityClass = Contract.class, fields = {
#FieldResult(name = "number", column = "number"),
#FieldResult(name = "name", column = "name")
})
})
#Entity
public class Contract {
public static final String GIVE_ALL_CONTRACTS = "Contract.giveAllContracts";
public static final String GIVE_ALL_CONTRACTS_MAPPING = "Contract.giveAllContractsMapping";
#Id
private Integer number;
private String name;
public Integer getNumber() {
return number;
}
public String getName() {
return name;
}
}
And the following code to retrieve the contracts:
Query query = entityManager.createNamedQuery(Contract.GIVE_ALL_CONTRACTS);
query.setParameter(1, referenceDate);
List contracts = query.getResultList();
entityManager.clear();
return contracts;
The retrieved contracts are passed to a webservice.
Executing this query in Oracle developer takes around 0,35 seconds for 3608 records.
The call to query.getResultList() takes around 4 seconds.
With a logger in the constuctor of the entity, it logs that there are about 10-20 entities created with the same timestamp. Then 0,015 seconds it does something else. I guess OpenJPA stuff.
Is there a way to speed up OpenJPA? Or is the only solution caching?
Object creation may have its fair share in the performance hit. While running your code in the server, you're not only querying the database but also you allocate memory and create a new Contract object for each row. An expanding heap or garbage collection cycle may count for idle periods that you observed.
I'd suggest you skim through OpenJPA documentation on how to process large results sets.
I suggest you downloading VisualVM and set up a profiling for the packages involved. VisualVM can show the time spent in different methods that will sum up to 0.35sec in your case theoretically. You will be able to analyze the distribution of the total time between your code, OpenJPA and the network IO. This will help you to identify the bottleneck.
I want the first to be generated:
#Id
#Column(name = "PRODUCT_ID", unique = true, nullable = false, precision = 12,
scale = 0)
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "PROD_GEN")
#BusinessKey
public Long getAId() {
return this.aId;
}
I want the bId to be initially exactly as the aId. One approach is to insert the entity, then get the aId generated by the DB (2nd query) and then update the entity, setting the bId to be equal to aId (3rd query). Is there a way to get the bId to get the same generated value as aId?
Note that afterwards, I want to be able to update bId from my gui.
If the solution is JPA, even better.
Choose your poison:
Option #1
you could annotate bId as org.hibernate.annotations.Generated and use a database trigger on insert (I'm assuming the nextval has already been assigned to AID so we'll assign the curval to BID):
CREATE OR REPLACE TRIGGER "MY_TRIGGER"
before insert on "MYENTITY"
for each row
begin
select "MYENTITY_SEQ".curval into :NEW.BID from dual;
end;
I'm not a big fan of triggers and things that happen behind the scene but this seems to be the easiest option (not the best one for portability though).
Option #2
Create a new entity, persist it, flush the entity manager to get the id assigned, set the aId on bId, merge the entity.
em.getTransaction().begin();
MyEntity e = new MyEntity();
...
em.persist(e);
em.flush();
e.setBId(e.getAId());
em.merge(e);
...
em.getTransaction().commit();
Ugly, but it works.
Option #3
Use callback annotations to set the bId in-memory (until it gets written to the database):
#PostPersist
#PostLoad
public void initialiazeBId() {
if (this.bId == null) {
this.bId = aId;
}
}
This should work if you don't need the id to be written on insert (but in that case, see Option #4).
Option #4
You could actually add some logic in the getter of bId instead of using callbacks:
public Long getBId() {
if (this.bId == null) {
return this.aId;
}
return this.bId;
}
Again, this will work if you don't need the id to be persisted in the database on insert.
If you use JPA, after inserting the new A the id should be set to the generated value, i tought (maybe it depends on which jpa provider you use), so no 2nd query needed. then set bld to ald value in your DAO?