VERSION
Red Hat Build of Quarkus
<properties>
<compiler-plugin.version>3.8.1</compiler-plugin.version>
<maven.compiler.parameters>true</maven.compiler.parameters>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<quarkus-plugin.version>1.7.5.Final</quarkus-plugin.version>
<quarkus.platform.artifact-id>quarkus-universe-bom</quarkus.platform.artifact-id>
<quarkus.platform.group-id>io.quarkus</quarkus.platform.group-id>
<quarkus.platform.version>1.7.5.Final</quarkus.platform.version>
<surefire-plugin.version>3.0.0-M5</surefire-plugin.version>
</properties>
I am totally confused about this inconsistency and despite logging set to the maximum, all I see is that my code is not behaving the same in this project as compared to other projects.
Everything is configured the same for versions and settings.
I have three quarkus jax-rs services with hibernate-orm pushing to PostgreSQL.
2 of the services all behave as expected with approximately 25 identically configured PUT endpoints with a merge backing manager (code below). In the two working services, the statement in the logs shows where an UPDATE statement is required. However, the third (new) service simply ignores the change. It responds with 200 and even returns the updated data, but the database is never updated.
simple entity
#Entity
#Table(name = "applicants")
public class Applicant implements Serializable {
private static final long serialVersionUID = 6796452320678418766L;
#Column(name = "applicant_id")
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "applicant_seq")
public long applicant_id;
#Column(name = "email", nullable = false, unique = true)
public String email;
}
#PUT Resource
#PUT
#Path("{id}")
public Response update(#PathParam("id") Long id, #Valid Applicant applicant) {
try {
Applicant applicantToUpdate = this.manager.findById(id);
if (applicantToUpdate != null) {
applicantToUpdate = this.manager.save(applicant);
if (applicantToUpdate != null) {
return Response.ok(applicantToUpdate).build();
} else {
...
}
}
return Response.status(Response.Status.NOT_FOUND).build();
} catch (Exception ex) {
...
}
}
Backing Manager for persistance
#Transactional
public Applicant save(Applicant applicant) {
return this.em.merge(applicant);
}
Both the ApplicantResource and the ApplicantManager classes are #RequestScoped.
The logs for the failing service
2020-11-16 13:51:26,122 DEBUG [org.hib.res.jdb.int.LogicalConnectionManagedImpl] (executor-thread-198) `hibernate.connection.provider_disables_autocommit` was enabled. This setting should only be enabled when you are certain that the Connections given to Hibernate by the ConnectionProvider have auto-commit disabled. Enabling this setting when the Connections do not have auto-commit disabled will lead to Hibernate executing SQL operations outside of any JDBC/SQL transaction.
2020-11-16 13:51:26,123 DEBUG [org.hib.SQL] (executor-thread-198)
select
applicant0_.applicant_id as applican1_0_0_,
applicant0_.email as email2_0_0_
from
applicants applicant0_
where
applicant0_.applicant_id=?
Hibernate:
select
applicant0_.applicant_id as applican1_0_0_,
applicant0_.email as email2_0_0_
from
applicants applicant0_
where
applicant0_.applicant_id=?
2020-11-16 13:51:26,156 DEBUG [org.hib.loa.pla.exe.pro.int.EntityReferenceInitializerImpl] (executor-thread-198) On call to EntityIdentifierReaderImpl#resolve, EntityKey was already known; should only happen on root returns with an optional identifier specified
2020-11-16 13:51:26,156 DEBUG [org.hib.eng.int.TwoPhaseLoad] (executor-thread-198) Resolving attributes for [org.protechskillsinstitute.applicant.entity.Applicant#1]
2020-11-16 13:51:26,157 DEBUG [org.hib.eng.int.TwoPhaseLoad] (executor-thread-198) Processing attribute `email` : value = original#email.com
2020-11-16 13:51:26,157 DEBUG [org.hib.eng.int.TwoPhaseLoad] (executor-thread-198) Attribute (`email`) - enhanced for lazy-loading? - false
2020-11-16 13:51:26,157 DEBUG [org.hib.eng.int.TwoPhaseLoad] (executor-thread-198) Done materializing entity [org.protechskillsinstitute.applicant.entity.Applicant#1]
2020-11-16 13:51:26,159 DEBUG [org.hib.res.jdb.int.LogicalConnectionManagedImpl] (executor-thread-198) Initiating JDBC connection release from afterStatement
2020-11-16 13:51:26,159 DEBUG [org.hib.loa.ent.pla.AbstractLoadPlanBasedEntityLoader] (executor-thread-198) Done entity load : org.protechskillsinstitute.applicant.entity.Applicant#1
2020-11-16 13:51:26,159 DEBUG [org.hib.res.jdb.int.LogicalConnectionManagedImpl] (executor-thread-198) Initiating JDBC connection release from afterTransaction
2020-11-16 13:51:26,160 DEBUG [org.hib.res.jdb.int.LogicalConnectionManagedImpl] (executor-thread-198) `hibernate.connection.provider_disables_autocommit` was enabled. This setting should only be enabled when you are certain that the Connections given to Hibernate by the ConnectionProvider have auto-commit disabled. Enabling this setting when the Connections do not have auto-commit disabled will lead to Hibernate executing SQL operations outside of any JDBC/SQL transaction.
2020-11-16 13:51:26,160 DEBUG [org.hib.res.tra.bac.jta.int.JtaTransactionCoordinatorImpl] (executor-thread-198) Hibernate RegisteredSynchronization successfully registered with JTA platform
2020-11-16 13:51:26,161 DEBUG [org.hib.res.tra.bac.jta.int.JtaTransactionCoordinatorImpl] (executor-thread-198) JTA transaction was already joined (RegisteredSynchronization already registered)
2020-11-16 13:51:26,161 DEBUG [org.hib.eve.int.EntityCopyObserverFactoryInitiator] (executor-thread-198) Configured EntityCopyObserver strategy: disallow
2020-11-16 13:51:26,161 DEBUG [org.hib.loa.Loader] (executor-thread-198) Loading entity: [org.protechskillsinstitute.applicant.entity.Applicant#1]
2020-11-16 13:51:26,212 DEBUG [org.hib.SQL] (executor-thread-198)
select
applicant0_.applicant_id as applican1_0_0_,
applicant0_.email as email2_0_0_
from
applicants applicant0_
where
applicant0_.applicant_id=?
Hibernate:
select
applicant0_.applicant_id as applican1_0_0_,
applicant0_.email as email2_0_0_
from
applicants applicant0_
where
applicant0_.applicant_id=?
2020-11-16 13:51:26,216 DEBUG [org.hib.loa.Loader] (executor-thread-198) Result set row: 0
2020-11-16 13:51:26,216 DEBUG [org.hib.loa.Loader] (executor-thread-198) Result row: EntityKey[org.protechskillsinstitute.applicant.entity.Applicant#1]
2020-11-16 13:51:26,217 DEBUG [org.hib.eng.int.TwoPhaseLoad] (executor-thread-198) Resolving attributes for [org.protechskillsinstitute.applicant.entity.Applicant#1]
2020-11-16 13:51:26,217 DEBUG [org.hib.eng.int.TwoPhaseLoad] (executor-thread-198) Processing attribute `email` : value = original#email.com
2020-11-16 13:51:26,217 DEBUG [org.hib.eng.int.TwoPhaseLoad] (executor-thread-198) Attribute (`email`) - enhanced for lazy-loading? - false
2020-11-16 13:51:26,217 DEBUG [org.hib.eng.int.TwoPhaseLoad] (executor-thread-198) Done materializing entity [org.protechskillsinstitute.applicant.entity.Applicant#1]
2020-11-16 13:51:26,218 DEBUG [org.hib.res.jdb.int.LogicalConnectionManagedImpl] (executor-thread-198) Initiating JDBC connection release from afterStatement
2020-11-16 13:51:26,218 DEBUG [org.hib.loa.Loader] (executor-thread-198) Done entity load
2020-11-16 13:51:26,219 DEBUG [org.hib.eve.int.AbstractFlushingEventListener] (executor-thread-198) Processing flush-time cascades
2020-11-16 13:51:26,219 DEBUG [org.hib.eve.int.AbstractFlushingEventListener] (executor-thread-198) Dirty checking collections
2020-11-16 13:51:26,219 DEBUG [org.hib.eve.int.AbstractFlushingEventListener] (executor-thread-198) Flushed: 0 insertions, 0 updates, 0 deletions to 1 objects
2020-11-16 13:51:26,220 DEBUG [org.hib.eve.int.AbstractFlushingEventListener] (executor-thread-198) Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
2020-11-16 13:51:26,220 DEBUG [org.hib.int.uti.EntityPrinter] (executor-thread-198) Listing entities:
2020-11-16 13:51:26,220 DEBUG [org.hib.int.uti.EntityPrinter] (executor-thread-198) org.protechskillsinstitute.applicant.entity.Applicant{applicant_id=1, email=stephen.w.boyd#gmail.com}
2020-11-16 13:51:26,220 DEBUG [org.hib.res.jdb.int.LogicalConnectionManagedImpl] (executor-thread-198) Initiating JDBC connection release from afterStatement
2020-11-16 13:51:26,221 DEBUG [org.hib.eng.tra.int.TransactionImpl] (executor-thread-198) On TransactionImpl creation, JpaCompliance#isJpaTransactionComplianceEnabled == false
2020-11-16 13:51:26,221 DEBUG [org.hib.eve.int.AbstractFlushingEventListener] (executor-thread-198) Processing flush-time cascades
2020-11-16 13:51:26,221 DEBUG [org.hib.eve.int.AbstractFlushingEventListener] (executor-thread-198) Dirty checking collections
2020-11-16 13:51:26,221 DEBUG [org.hib.eve.int.AbstractFlushingEventListener] (executor-thread-198) Flushed: 0 insertions, 0 updates, 0 deletions to 1 objects
2020-11-16 13:51:26,222 DEBUG [org.hib.eve.int.AbstractFlushingEventListener] (executor-thread-198) Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
2020-11-16 13:51:26,222 DEBUG [org.hib.int.uti.EntityPrinter] (executor-thread-198) Listing entities:
2020-11-16 13:51:26,222 DEBUG [org.hib.int.uti.EntityPrinter] (executor-thread-198) org.protechskillsinstitute.applicant.entity.Applicant{applicant_id=1, email=edited#email.com}
2020-11-16 13:51:26,222 DEBUG [org.hib.res.jdb.int.LogicalConnectionManagedImpl] (executor-thread-198) Initiating JDBC connection release from afterStatement
2020-11-16 13:51:26,223 INFO [org.hib.eng.int.StatisticalLoggingSessionEventListener] (executor-thread-198) Session Metrics {
112700 nanoseconds spent acquiring 1 JDBC connections;
9700 nanoseconds spent releasing 1 JDBC connections;
147400 nanoseconds spent preparing 1 JDBC statements;
340500 nanoseconds spent executing 1 JDBC statements;
0 nanoseconds spent executing 0 JDBC batches;
0 nanoseconds spent performing 0 L2C puts;
0 nanoseconds spent performing 0 L2C hits;
0 nanoseconds spent performing 0 L2C misses;
3546700 nanoseconds spent executing 2 flushes (flushing a total of 2 entities and 0 collections);
0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0 entities and 0 collections)
}
2020-11-16 13:51:26,227 DEBUG [org.hib.res.jdb.int.LogicalConnectionManagedImpl] (executor-thread-198) Initiating JDBC connection release from afterTransaction
2020-11-16 13:51:26,230 INFO [org.hib.eng.int.StatisticalLoggingSessionEventListener] (executor-thread-198) Session Metrics {
34900 nanoseconds spent acquiring 1 JDBC connections;
18700 nanoseconds spent releasing 1 JDBC connections;
77500 nanoseconds spent preparing 1 JDBC statements;
430200 nanoseconds spent executing 1 JDBC statements;
0 nanoseconds spent executing 0 JDBC batches;
0 nanoseconds spent performing 0 L2C puts;
0 nanoseconds spent performing 0 L2C hits;
0 nanoseconds spent performing 0 L2C misses;
0 nanoseconds spent executing 0 flushes (flushing a total of 0 entities and 0 collections);
0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0 entities and 0 collections)
}
Everything in the code and logs are identical for the working and non-working services with one exception. The log shows the Update is required for the working services and does not for the failing service.
Thoughts? As always, thank you for your assistance!
Drilled in logs
2020-11-16 13:51:26,156 DEBUG [org.hib.loa.pla.exe.pro.int.EntityReferenceInitializerImpl] (executor-thread-198) On call to EntityIdentifierReaderImpl#resolve, EntityKey was already known; should only happen on root returns with an optional identifier specified
2020-11-16 13:51:26,156 DEBUG [org.hib.eng.int.TwoPhaseLoad] (executor-thread-198) Resolving attributes for [org.protechskillsinstitute.applicant.entity.Applicant#1]
2020-11-16 13:51:26,157 DEBUG [org.hib.eng.int.TwoPhaseLoad] (executor-thread-198) Processing attribute `email` : value = original#email.com
2020-11-16 13:51:26,157 DEBUG [org.hib.eng.int.TwoPhaseLoad] (executor-thread-198) Attribute (`email`) - enhanced for lazy-loading? - false
and the check
11-16 13:51:26,220 DEBUG [org.hib.int.uti.EntityPrinter] (executor-thread-198) Listing entities:
2020-11-16 13:51:26,220 DEBUG [org.hib.int.uti.EntityPrinter] (executor-thread-198) org.protechskillsinstitute.applicant.entity.Applicant{applicant_id=1, email=edited#email.com}
2020-11-16 13:51:26,220 DEBUG [org.hib.res.jdb.int.LogicalConnectionManagedImpl] (executor-thread-198) Initiating JDBC connection release from afterStatement
2020-11-16 13:51:26,221 DEBUG [org.hib.eng.tra.int.TransactionImpl] (executor-thread-198) On TransactionImpl creation, JpaCompliance#isJpaTransactionComplianceEnabled == false
2020-11-16 13:51:26,221 DEBUG [org.hib.eve.int.AbstractFlushingEventListener] (executor-thread-198) Processing flush-time cascades
2020-11-16 13:51:26,221 DEBUG [org.hib.eve.int.AbstractFlushingEventListener] (executor-thread-198) Dirty checking collections
2020-11-16 13:51:26,221 DEBUG [org.hib.eve.int.AbstractFlushingEventListener] (executor-thread-198) Flushed: 0 insertions, 0 updates, 0 deletions to 1 objects
2020-11-16 13:51:26,222 DEBUG [org.hib.eve.int.AbstractFlushingEventListener] (executor-thread-198) Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
2020-11-16 13:51:26,222 DEBUG [org.hib.int.uti.EntityPrinter] (executor-thread-198) Listing entities:
2020-11-16 13:51:26,222 DEBUG [org.hib.int.uti.EntityPrinter] (executor-thread-198) org.protechskillsinstitute.applicant.entity.Applicant{applicant_id=1, email=edited#email.com}
Are you sure the email actually changed? Hibernate will obviously not execute an update query if the data didn't change.
Have you tried flush and commit after merge? did you check if the flush mode is manual or auto?
Based on the Resolved Dependencies it looks like Red Hat is using:
hibernate-core: 5.4.21.Final-redhat-00001
hibarenate-commons-annotations: 5.1.0.Final-redhat-00005
hibernate-graalvm: 5.4.21.Final-redhat-00001
Per the comments below, I have opened a ticket with Red Hat.
Case 02805123
Related
I'm trying to add a batch update to my spring boot project. But multiple queries are still executed when I check SQL logs and Hibernate stats.
Hibernate stats
290850400 nanoseconds spent acquiring 1 JDBC connections;
0 nanoseconds spent releasing 0 JDBC connections;
3347700 nanoseconds spent preparing 19 JDBC statements;
5919028800 nanoseconds spent executing 19 JDBC statements;
0 nanoseconds spent executing 0 JDBC batches;
0 nanoseconds spent performing 0 L2C puts;
0 nanoseconds spent performing 0 L2C hits;
0 nanoseconds spent performing 0 L2C misses;
2635900 nanoseconds spent executing 1 flushes (flushing a total of 1 entities and 0 collections);
19447300 nanoseconds spent executing 19 partial-flushes (flushing a total of 18 entities and 18 collections)
Versions
Spring Boot v2.7.1
Spring v5.3.21
Java 17.0.3.1
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
application.yml
spring:
datasource:
url: jdbc:oracle:thin:#localhost:1521/db
username: user
password: password
driver-class-name: oracle.jdbc.OracleDriver
jpa:
database-platform: org.hibernate.dialect.Oracle12cDialect
hibernate:
naming:
physical-strategy: org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl
ddl-auto: update
show-sql: true
properties:
hibernate:
dialect: org.hibernate.dialect.Oracle12cDialect
format_sql: false
jdbc:
fetch_size: 100
batch_size: 5
order_updates: true
batch_versioned_data: true
generate_statistics: true
Snapshot entity
#Entity
#Table(name = "SNAPSHOT", schema = "SYSTEM", catalog = "")
public class Snapshot {
#GeneratedValue(strategy = GenerationType.SEQUENCE)
#Id
#Column(name = "ID")
private long id;
#Basic
#Column(name = "CREATED_ON")
private String createdOn;
...
}
SnapshotRepository
#Repository
public interface SnapshotRepository extends JpaRepository<Snapshot, Long> {
#Modifying
#Query("UPDATE Snapshot s SET s.fieldValue =?1,s.createdOn=?2 where s.id = ?3 and s.fieldName = ?4")
int updateSnapshot(String fieldValue, String createdOn, String id, String fieldName);
}
And this repository method is called from the service class.
for (Map.Entry<String, String> entry : res.getValues().entrySet()) {
snapshotRepository.updateSnapshot(entry.getValue(), createdOn, id, entry.getKey());
}
pom.xml
<dependency>
<groupId>com.oracle.database.jdbc</groupId>
<artifactId>ojdbc11</artifactId>
<version>21.7.0.0</version>
</dependency>
In the application.yml, I think I'm configuring all required properties to activate batch update but still no luck.
Please let me know what I'm doing incorrectly.
Batching only works when Hibernate does the flushing of entities. If you are executing manual queries, Hibernate can't do batching.
The way you are implementing this, Hibernate will reuse the same prepared statement on the database side though, but no JDBC batching.
I am using Hibernate 4.13 and Spring 4
If I try to store timestamp values on Oracle, it fails. But surprisingly, it works on OracleXE. Only difference in schema is, On Oracle I have partition on that table. On OracleXE, I dont.
On Oracle,
While printing entity it shows value correctly.
2018-03-19 11:25:59.456 DEBUG
[org.hibernate.internal.util.EntityPrinter : qtp771935287-17] -
net.jigarshah.domain.MyEntity{myId=123123,
myUuid=ef5c5159-1e91-4a8e-a3bb-8f583fb3cce2,
receivedOn=2018-03-19T11:25:59.277+01:00[Arctic/Longyearbyen], errorCode=null,
lastUpdatedOn=2018-03-19T11:25:59.274+01:00[Arctic/Longyearbyen], id=42001, status=RECEIVED}
In Insert it just removed those values
2018-03-19 11:25:59.470 TRACE [org.hibernate.type.descriptor.sql.BasicBinder : qtp771935287-17] -
binding parameter [3] as [TIMESTAMP] - [null]
2018-03-19 11:25:59.470 TRACE [org.hibernate.type.descriptor.sql.BasicBinder : qtp771935287-17] -
binding parameter [4] as [TIMESTAMP] - [null]
Complete logs
2018-03-19 11:25:59.278 DEBUG [org.springframework.transaction.support.AbstractPlatformTransactionManager : qtp771935287-17] - Initiating transaction commit
2018-03-19 11:25:59.278 DEBUG [org.springframework.orm.hibernate4.HibernateTransactionManager : qtp771935287-17] - Committing Hibernate transaction on Session [SessionImpl(PersistenceContext[entityKeys=[EntityKey[net.jigarshah.domain.MyEntity#42001]],collectionKeys=[]];ActionQueue[insertions=org.hibernate.engine.spi.ExecutableList#425f14f6 updates=org.hibernate.engine.spi.ExecutableList#390223c9 deletions=org.hibernate.engine.spi.ExecutableList#611117d9 orphanRemovals=org.hibernate.engine.spi.ExecutableList#1bb9208a collectionCreations=org.hibernate.engine.spi.ExecutableList#21fb6faf collectionRemovals=org.hibernate.engine.spi.ExecutableList#94f1c32 collectionUpdates=org.hibernate.engine.spi.ExecutableList#60897d1d collectionQueuedOps=org.hibernate.engine.spi.ExecutableList#3266acf6 unresolvedInsertDependencies=UnresolvedEntityInsertActions[]])]
2018-03-19 11:25:59.278 DEBUG [org.hibernate.engine.transaction.spi.AbstractTransactionImpl : qtp771935287-17] - committing
2018-03-19 11:25:59.278 DEBUG [org.hibernate.event.internal.AbstractFlushingEventListener : qtp771935287-17] - Processing flush-time cascades
2018-03-19 11:25:59.278 DEBUG [org.hibernate.event.internal.AbstractFlushingEventListener : qtp771935287-17] - Dirty checking collections
2018-03-19 11:25:59.455 DEBUG [org.hibernate.event.internal.AbstractFlushingEventListener : qtp771935287-17] - Flushed: 1 insertions, 1 updates, 0 deletions to 1 objects
2018-03-19 11:25:59.455 DEBUG [org.hibernate.event.internal.AbstractFlushingEventListener : qtp771935287-17] - Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
2018-03-19 11:25:59.455 DEBUG [org.hibernate.internal.util.EntityPrinter : qtp771935287-17] - Listing entities:
2018-03-19 11:25:59.456 DEBUG [org.hibernate.internal.util.EntityPrinter : qtp771935287-17] - net.jigarshah.domain.MyEntity{myId=123123, myUuid=ef5c5159-1e91-4a8e-a3bb-8f583fb3cce2, receivedOn=2018-03-19T11:25:59.277+01:00[Arctic/Longyearbyen], errorCode=null, lastUpdatedOn=2018-03-19T11:25:59.274+01:00[Arctic/Longyearbyen], id=42001, status=RECEIVED}
2018-03-19 11:25:59.468 DEBUG [org.hibernate.engine.jdbc.spi.SqlStatementLogger : qtp771935287-17] -
insert
into
my_entity
(my_id, error_code, last_updated_on, received_on, my_uuid, status, id)
values
(?, ?, ?, ?, ?, ?, ?)
2018-03-19 11:25:59.469 TRACE [org.hibernate.type.descriptor.sql.BasicBinder : qtp771935287-17] - binding parameter [1] as [VARCHAR] - [123123]
2018-03-19 11:25:59.469 TRACE [org.hibernate.type.descriptor.sql.BasicBinder : qtp771935287-17] - binding parameter [2] as [VARCHAR] - [null]
2018-03-19 11:25:59.470 TRACE [org.hibernate.type.descriptor.sql.BasicBinder : qtp771935287-17] - binding parameter [3] as [TIMESTAMP] - [null]
2018-03-19 11:25:59.470 TRACE [org.hibernate.type.descriptor.sql.BasicBinder : qtp771935287-17] - binding parameter [4] as [TIMESTAMP] - [null]
2018-03-19 11:25:59.470 TRACE [org.hibernate.type.descriptor.sql.BasicBinder : qtp771935287-17] - binding parameter [5] as [BINARY] - [ef5c5159-1e91-4a8e-a3bb-8f583fb3cce2]
2018-03-19 11:25:59.471 TRACE [org.hibernate.type.EnumType$EnumValueMapperSupport : qtp771935287-17] - Binding [RECEIVED] to parameter: [6]
2018-03-19 11:25:59.472 TRACE [org.hibernate.type.descriptor.sql.BasicBinder : qtp771935287-17] - binding parameter [7] as [BIGINT] - [42001]
2018-03-19 11:25:59.472 DEBUG [org.hibernate.engine.jdbc.batch.internal.BatchingBatch : qtp771935287-17] - Executing batch size: 1
Type conversion
LocalSessionFactoryBuilder builder = new LocalSessionFactoryBuilder(dataSource());
builder.setNamingStrategy(ImprovedNamingStrategy.INSTANCE);
builder.scanPackages(packages);
builder.addProperties(createHibernateProperties());
new **StandardTypeOverrideApplier**(builder).apply();
SessionFactory sessionFactory = builder.buildSessionFactory();
return sessionFactory;
Type converter
private Configuration configuration;
public StandardTypeOverrideApplier(Configuration configuration) {
this.configuration = configuration;
}
public void apply() {
configuration.registerTypeOverride(jvm(new org.jadira.usertype.dateandtime.joda.PersistentDateTime()), new String[]{"jodaDateTime", DateTime.class.getName()});
configuration.registerTypeOverride(new org.jadira.usertype.dateandtime.joda.PersistentLocalDate(), new String[]{"jodaLocalDate", org.joda.time.LocalDate.class.getName()});
**configuration.registerTypeOverride(new PersistentZonedDateTime(), new String[]{"zonedDateTime", ZonedDateTime.class.getName()});**
configuration.registerTypeOverride(new PersistentLocalDate(), new String[]{"localDate", LocalDate.class.getName()});
}
This had nothing to do with Type conversion. Issue was with SavOrUpdateEvent Listner.
I was updating receivedOn value in saveOrUpdate event Listner. Unfortunately, hibernate first creates insert query which does not have timestamp. and Then update query to update receivedOn. And since I had index on receivedOn, if never created update query. It was always failing on insert itself. I would really appriciate if anyone knows alter native to #PreCreate and #PreUpdate in case of hibernate saveOrUpdate. (unfortunately we cant get rid of saveOrUpdate due to some historical reasons in code base)
I am trying to persist data to an embedded Derby DB using hibernate. But data not getting persisted at all.
hibernate configuration
<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE hibernate-configuration PUBLIC
"-//Hibernate/Hibernate Configuration DTD//EN"
"http://www.hibernate.org/dtd/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
<session-factory>
<property name="hibernate.connection.url">
jdbc:derby:wso2FlightRecorder
</property>
<property name="hibernate.connection.driver_class">
org.apache.derby.jdbc.EmbeddedDriver
</property>
<property name="hibernate.dialect">
org.hibernate.dialect.DerbyTenSevenDialect
</property>
<property name="connection.username"/>
<property name="connection.password"/>
<!-- DB schema will be updated if needed -->
<property name="hibernate.hbm2ddl.auto">create</property>
<mapping class="org.wso2.esbMonitor.network.PassThruHTTPBean">
</mapping>
</session-factory>
</hibernate-configuration>
This is the DAO class
package org.wso2.esbMonitor.network;
import javax.annotation.Generated;
import javax.persistence.*;
import java.util.Date;
#Entity
#Table(name = "HTTP_LOG")
public class PassThruHTTPBean {
#Id
#GeneratedValue
#Column(name = "id")
private int id;
#Column(name = "activeThreadCount")
private int activeThreadCount;
#Column(name = "avgSizeRecieved")
private double avgSizeRecieved;
#Column(name ="avgSizeSent")
private double avgSizeSent;
#Column(name ="faultsRecieving")
private long faultsRecieving;
#Column(name ="faultSending")
private long faultSending;
#Column(name ="messagesRecieved")
private long messagesRecieved;
#Column(name ="messageSent")
private long messageSent;
#Column(name ="queueSize")
private int queueSize;
#Column(name = "time")
private Date date;
}
This is the method use to commit to DB
public synchronized static void addNetworkTrafficDetailsToDB(){
try {
if (scheduledList.size() > 0){
logger.info("Started persisting");
Session session = HibernateSessionCreator.getSession();
for(PassThruHTTPBean passThruHTTPBean : scheduledList){
Transaction tx;
tx = session.beginTransaction();
session.save(passThruHTTPBean);
tx.commit();
}
session.flush();
session.close();
scheduledList.clear();
}
Stack trace
2016-06-01 09:31:08 DEBUG AbstractTransactionImpl:158 - begin
2016-06-01 09:31:08 DEBUG LogicalConnectionImpl:212 - Obtaining JDBC
connection
2016-06-01 09:31:08 TRACE DriverManagerConnectionProviderImpl:175 - Total
checked-out connections: 0
2016-06-01 09:31:08 TRACE DriverManagerConnectionProviderImpl:181 - Using
pooled JDBC connection, pool size: 0
2016-06-01 09:31:08 DEBUG LogicalConnectionImpl:218 - Obtained JDBC
connection
2016-06-01 09:31:08 DEBUG JdbcTransaction:69 - initial autocommit status: false
2016-06-01 09:31:08 TRACE AbstractServiceRegistryImpl:146 - Initializing service [role=org.hibernate.event.service.spi.EventListenerRegistry]
2016-06-01 09:31:08 TRACE DefaultSaveOrUpdateEventListener:177 - Saving transient instance
2016-06-01 09:31:08 TRACE AbstractSaveEventListener:167 - Saving [org.wso2.esbMonitor.network.PassThruHTTPBean#<null>]
2016-06-01 09:31:08 TRACE ActionQueue:177 - Adding an EntityIdentityInsertAction for [org.wso2.esbMonitor.network.PassThruHTTPBean] object
2016-06-01 09:31:08 TRACE ActionQueue:185 - Executing inserts before finding non-nullable transient entities for early insert: [EntityIdentityInsertAction[org.wso2.esbMonitor.network.PassThruHTTPBean#<null>]]
2016-06-01 09:31:08 TRACE ActionQueue:193 - Adding insert with no non-nullable, transient entities: [EntityIdentityInsertAction[org.wso2.esbMonitor.network.PassThruHTTPBean#<null>]]
2016-06-01 09:31:08 TRACE ActionQueue:211 - Executing insertions before resolved early-insert
2016-06-01 09:31:08 DEBUG ActionQueue:213 - Executing identity-insert immediately
2016-06-01 09:31:08 TRACE AbstractEntityPersister:2960 - Inserting entity: org.wso2.esbMonitor.network.PassThruHTTPBean (native id)
2016-06-01 09:31:08 DEBUG SQL:104 - insert into HTTP_LOG (id, activeThreadCount, avgSizeRecieved, avgSizeSent, time, faultSending, faultsRecieving, messageSent, messagesRecieved, queueSize) values (default, ?, ?, ?, ?, ?, ?, ?, ?, ?)
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:319 - Registering statement [c065801d-0155-0a1f-282f-000004235ae8]
2016-06-01 09:31:08 TRACE AbstractEntityPersister:2780 - Dehydrating entity: [org.wso2.esbMonitor.network.PassThruHTTPBean#<null>]
2016-06-01 09:31:08 TRACE BasicBinder:84 - binding parameter [1] as [INTEGER] - 0
2016-06-01 09:31:08 TRACE BasicBinder:84 - binding parameter [2] as [DOUBLE] - 0.0
2016-06-01 09:31:08 TRACE BasicBinder:84 - binding parameter [3] as [DOUBLE] - 0.0
2016-06-01 09:31:08 TRACE BasicBinder:72 - binding parameter [4] as [TIMESTAMP] - <null>
2016-06-01 09:31:08 TRACE BasicBinder:84 - binding parameter [5] as [BIGINT] - 0
2016-06-01 09:31:08 TRACE BasicBinder:84 - binding parameter [6] as [BIGINT] - 0
2016-06-01 09:31:08 TRACE BasicBinder:84 - binding parameter [7] as [BIGINT] - 0
2016-06-01 09:31:08 TRACE BasicBinder:84 - binding parameter [8] as [BIGINT] - 0
2016-06-01 09:31:08 TRACE BasicBinder:84 - binding parameter [9] as [INTEGER] - 0
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:358 - Releasing statement [c065801d-0155-0a1f-282f-000004235ae8]
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:472 - Closing prepared statement [c065801d-0155-0a1f-282f-000004235ae8]
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:249 - Starting after statement execution processing [ON_CLOSE]
2016-06-01 09:31:08 DEBUG SQL:104 - values identity_val_local()
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:319 - Registering statement [787c0020-0155-0a1f-282f-000004235ae8]
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:374 - Registering result set [org.apache.derby.impl.jdbc.EmbedResultSet42#1718dbaa]
2016-06-01 09:31:08 DEBUG IdentifierGeneratorHelper:93 - Natively generated identity: 1
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:401 - Releasing result set [org.apache.derby.impl.jdbc.EmbedResultSet42#1718dbaa]
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:515 - Closing result set [org.apache.derby.impl.jdbc.EmbedResultSet42#1718dbaa]
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:358 - Releasing statement [787c0020-0155-0a1f-282f-000004235ae8]
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:472 - Closing prepared statement [787c0020-0155-0a1f-282f-000004235ae8]
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:249 - Starting after statement execution processing [ON_CLOSE]
2016-06-01 09:31:08 TRACE UnresolvedEntityInsertActions:214 - No unresolved entity inserts that depended on [[org.wso2.esbMonitor.network.PassThruHTTPBean#1]]
2016-06-01 09:31:08 TRACE UnresolvedEntityInsertActions:121 - No entity insert actions have non-nullable, transient entity dependencies.
2016-06-01 09:31:08 DEBUG AbstractTransactionImpl:173 - committing
2016-06-01 09:31:08 TRACE SessionImpl:403 - Automatically flushing session
2016-06-01 09:31:08 TRACE AbstractFlushingEventListener:82 - Flushing session
2016-06-01 09:31:08 DEBUG AbstractFlushingEventListener:144 - Processing flush-time cascades
2016-06-01 09:31:08 DEBUG AbstractFlushingEventListener:185 - Dirty checking collections
2016-06-01 09:31:08 TRACE AbstractFlushingEventListener:200 - Flushing entities and processing referenced collections
2016-06-01 09:31:08 TRACE AbstractFlushingEventListener:242 - Processing unreferenced collections
2016-06-01 09:31:08 TRACE AbstractFlushingEventListener:254 - Scheduling collection removes/(re)creates/updates
2016-06-01 09:31:08 DEBUG AbstractFlushingEventListener:118 - Flushed: 0 insertions, 0 updates, 0 deletions to 1 objects
2016-06-01 09:31:08 DEBUG AbstractFlushingEventListener:125 - Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
2016-06-01 09:31:08 DEBUG EntityPrinter:114 - Listing entities:
2016-06-01 09:31:08 DEBUG EntityPrinter:121 - org.wso2.esbMonitor.network.PassThruHTTPBean{date=null, faultsRecieving=0, queueSize=0, activeThreadCount=0, avgSizeRecieved=0.0, messagesRecieved=0, id=1, avgSizeSent=0.0, faultSending=0, messageSent=0}
2016-06-01 09:31:08 TRACE AbstractFlushingEventListener:327 - Executing flush
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:249 - Starting after statement execution processing [ON_CLOSE]
2016-06-01 09:31:08 TRACE AbstractFlushingEventListener:359 - Post flush
2016-06-01 09:31:08 TRACE SessionImpl:612 - before transaction completion
2016-06-01 09:31:08 DEBUG JdbcTransaction:113 - committed JDBC Connection
2016-06-01 09:31:08 TRACE TransactionCoordinatorImpl:136 - after transaction completion
2016-06-01 09:31:08 TRACE SessionImpl:624 - after transaction completion
2016-06-01 09:31:08 TRACE AbstractFlushingEventListener:82 - Flushing session
2016-06-01 09:31:08 DEBUG AbstractFlushingEventListener:144 - Processing flush-time cascades
2016-06-01 09:31:08 DEBUG AbstractFlushingEventListener:185 - Dirty checking collections
2016-06-01 09:31:08 TRACE AbstractFlushingEventListener:200 - Flushing entities and processing referenced collections
2016-06-01 09:31:08 TRACE AbstractFlushingEventListener:242 - Processing unreferenced collections
2016-06-01 09:31:08 TRACE AbstractFlushingEventListener:254 - Scheduling collection removes/(re)creates/updates
2016-06-01 09:31:08 DEBUG AbstractFlushingEventListener:118 - Flushed: 0 insertions, 0 updates, 0 deletions to 1 objects
2016-06-01 09:31:08 DEBUG AbstractFlushingEventListener:125 - Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
2016-06-01 09:31:08 DEBUG EntityPrinter:114 - Listing entities:
2016-06-01 09:31:08 DEBUG EntityPrinter:121 - org.wso2.esbMonitor.network.PassThruHTTPBean{date=null, faultsRecieving=0, queueSize=0, activeThreadCount=0, avgSizeRecieved=0.0, messagesRecieved=0, id=1, avgSizeSent=0.0, faultSending=0, messageSent=0}
2016-06-01 09:31:08 TRACE AbstractFlushingEventListener:327 - Executing flush
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:249 - Starting after statement execution processing [ON_CLOSE]
2016-06-01 09:31:08 TRACE AbstractFlushingEventListener:359 - Post flush
2016-06-01 09:31:08 TRACE SessionImpl:342 - Closing session
2016-06-01 09:31:08 TRACE JdbcCoordinatorImpl:171 - Closing JDBC container [org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl#4e6454a]
2016-06-01 09:31:08 TRACE LogicalConnectionImpl:164 - Closing logical connection
2016-06-01 09:31:08 DEBUG LogicalConnectionImpl:232 - Releasing JDBC connection
2016-06-01 09:31:08 TRACE DriverManagerConnectionProviderImpl:233 - Returning connection to pool, pool size: 1
2016-06-01 09:31:08 DEBUG LogicalConnectionImpl:250 - Released JDBC connection
2016-06-01 09:31:08 TRACE LogicalConnectionImpl:176 - Logical connection closed
Session factory
public class HibernateSessionCreator {
private static SessionFactory ourSessionFactory;
private static ServiceRegistry serviceRegistry;
public static void init() {
try {
Configuration configuration = new Configuration();
configuration.configure();
serviceRegistry = new ServiceRegistryBuilder().applySettings(configuration.getProperties()).buildServiceRegistry();
ourSessionFactory = new AnnotationConfiguration()
.addAnnotatedClass(PassThruHTTPBean.class)
.configure()
.buildSessionFactory(serviceRegistry);
} catch (Throwable ex) {
throw new ExceptionInInitializerError(ex);
}
}
public static Session getSession() throws HibernateException {
return ourSessionFactory.openSession();
}
Please help me solve the problem :) Thank you in advance
I'm working on an Integrator for Hibernate (background on Integrators: https://docs.jboss.org/hibernate/orm/4.3/manual/en-US/html/ch14.html#objectstate-decl-security) that by using listeners is supposed to take my data from how it's stored in the DB and convert it into a different form for processing at runtime. This works great when saving the data using .persist() however there's an odd behavior involving transactions. The following code is from Hibernate's own quickstart tutorial code:
// now lets pull events from the database and list them
entityManager = entityManagerFactory.createEntityManager();
entityManager.getTransaction().begin();
List<Event> result = entityManager.createQuery( "from Event", Event.class ).getResultList();
for ( Event event : result ) {
System.out.println( "Event (" + event.getDate() + ") : " + event.getTitle() );
}
entityManager.getTransaction().commit();
entityManager.close();
Notice the unusual transaction begin/commit wrapping the query to select the data. Running this gives the following output after the query completes:
01:01:59.111 [main] DEBUG org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(175) - committing
01:01:59.112 [main] DEBUG org.hibernate.event.internal.AbstractFlushingEventListener.prepareEntityFlushes(149) - Processing flush-time cascades
01:01:59.112 [main] DEBUG org.hibernate.event.internal.AbstractFlushingEventListener.prepareCollectionFlushes(189) - Dirty checking collections
01:01:59.114 [main] DEBUG org.hibernate.event.internal.AbstractFlushingEventListener.logFlushResults(123) - Flushed: 0 insertions, 2 updates, 0 deletions to 2 objects
01:01:59.114 [main] DEBUG org.hibernate.event.internal.AbstractFlushingEventListener.logFlushResults(130) - Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
01:01:59.114 [main] DEBUG org.hibernate.internal.util.EntityPrinter.toString(114) - Listing entities:
01:01:59.114 [main] DEBUG org.hibernate.internal.util.EntityPrinter.toString(121) - org.hibernate.tutorial.em.Event{date=2015-07-28 01:01:57.776, id=1, title=Our very first event!}
01:01:59.114 [main] DEBUG org.hibernate.internal.util.EntityPrinter.toString(121) - org.hibernate.tutorial.em.Event{date=2015-07-28 01:01:58.746, id=2, title=A follow up event}
01:01:59.115 [main] DEBUG org.hibernate.SQL.logStatement(109) - update EVENTS set EVENT_DATE=?, title=? where id=?
Hibernate: update EVENTS set EVENT_DATE=?, title=? where id=?
01:01:59.119 [main] DEBUG org.hibernate.SQL.logStatement(109) - update EVENTS set EVENT_DATE=?, title=? where id=?
Hibernate: update EVENTS set EVENT_DATE=?, title=? where id=?
01:01:59.120 [main] DEBUG org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.doCommit(113) - committed JDBC Connection
01:01:59.120 [main] DEBUG org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.close(201) - HHH000420: Closing un-released batch
01:01:59.121 [main] DEBUG org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.releaseConnection(246) - Releasing JDBC connection
01:01:59.121 [main] DEBUG org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.releaseConnection(264) - Released JDBC connection
01:01:59.121 [main] DEBUG org.hibernate.internal.SessionFactoryImpl.close(1339) - HHH000031: Closing
It appears that since the Integrator does a modification on the entity in question it gets marked as "dirty" and upon committing this odd transaction, it bypasses my event listeners and writes the value back in the wrong format! I did some digging in the code and it turns out that org.hibernate.event.internal.AbstractFlushingEventListener.flushEntities(FlushEvent, PersistenceContext) gets called above and tries to get listeners for EventType.FLUSH_ENTITY. Unfortunately a listener added for this EventType is never called in my Integrator. How can I write my Integrator to behave correctly in this case so that I can "undo" the conversion that has happened with my entities at runtime and not flush the wrong value out?
Ultimately the problem was due to the EventTypes of the event listeners added with the EventListenerRegistry. What worked was using EventType.POST_LOAD for all the read operations combined with EventType.PRE_UPDATE and EventType.PRE_INSERT for writes that call a helper method for handling both the same way.
To prevent unneeded writes after making your entity updates it's a good idea to reset the data used for tracking if the entity is dirty in EntityEntry called loadedState. This is a private field in Hibernate 4 so you'll need to use Reflection, however in Hibernate 5 it's available via the getLoadedState() method. One more gotcha is you need to update values of the "state" used when actually flushing the values to the database by the PreInsertEvent and PreUpdateEvent which can be retrieved from the getState() method defined in each.
This program does tens of thousands of consecutive inserts one after the other. I've never used Hibernate before. I'm getting extremely slow performance (if I just connect and execute the SQL manually I am 10-12x quicker. My batch_size is set to 50 as per many hibernate tutorials.
Here is a log from a single insert - perhaps you could help me understand exactly what is happening:
START INSERT
11:02:56.121 [main] DEBUG org.hibernate.impl.SessionImpl - opened session at timestamp: 13106053761
11:02:56.121 [main] DEBUG o.h.transaction.JDBCTransaction - begin
11:02:56.121 [main] DEBUG org.hibernate.jdbc.ConnectionManager - opening JDBC connection
11:02:56.121 [main] TRACE o.h.c.DriverManagerConnectionProvider - total checked-out connections: 0
11:02:56.121 [main] TRACE o.h.c.DriverManagerConnectionProvider - using pooled JDBC connection, pool size: 0
11:02:56.121 [main] DEBUG o.h.transaction.JDBCTransaction - current autocommit status: false
11:02:56.121 [main] TRACE org.hibernate.jdbc.JDBCContext - after transaction begin
11:02:56.121 [main] TRACE org.hibernate.impl.SessionImpl - setting flush mode to: MANUAL
11:02:56.121 [main] TRACE o.h.e.def.DefaultLoadEventListener - loading entity: [com.xyzcompany.foo.edoi.ejb.msw000.MSW000Rec#component[keyW000]{keyW000=F000 ADSUFC}]
11:02:56.121 [main] TRACE o.h.e.def.DefaultLoadEventListener - creating new proxy for entity
11:02:56.122 [main] TRACE o.h.e.d.DefaultSaveOrUpdateEventListener - saving transient instance
11:02:56.122 [main] DEBUG o.h.e.def.AbstractSaveEventListener - generated identifier: component[keyW000]{keyW000=F000 ADSUFC}, using strategy: org.hibernate.id.CompositeNestedGeneratedValueGenerator
11:02:56.122 [main] TRACE o.h.e.def.AbstractSaveEventListener - saving [com.xyzcompany.foo.edoi.ejb.msw000.MSW000Rec#component[keyW000]{keyW000=F000 ADSUFC}]
11:02:56.123 [main] TRACE o.h.e.d.AbstractFlushingEventListener - flushing session
11:02:56.123 [main] DEBUG o.h.e.d.AbstractFlushingEventListener - processing flush-time cascades
11:02:56.123 [main] DEBUG o.h.e.d.AbstractFlushingEventListener - dirty checking collections
11:02:56.123 [main] TRACE o.h.e.d.AbstractFlushingEventListener - Flushing entities and processing referenced collections
11:02:56.125 [main] TRACE o.h.e.d.AbstractFlushingEventListener - Processing unreferenced collections
11:02:56.125 [main] TRACE o.h.e.d.AbstractFlushingEventListener - Scheduling collection removes/(re)creates/updates
11:02:56.126 [main] DEBUG o.h.e.d.AbstractFlushingEventListener - Flushed: 1 insertions, 0 updates, 0 deletions to 62 objects
11:02:56.126 [main] DEBUG o.h.e.d.AbstractFlushingEventListener - Flushed: 0 (re)creations, 0 updates, 0 removals to 0 collections
11:02:56.132 [main] TRACE o.h.e.d.AbstractFlushingEventListener - executing flush
11:02:56.132 [main] TRACE org.hibernate.jdbc.ConnectionManager - registering flush begin
11:02:56.132 [main] TRACE o.h.p.entity.AbstractEntityPersister - Inserting entity: [com.xyzcompany.foo.edoi.ejb.msw000.MSW000Rec#component[keyW000]{keyW000=F000 ADSUFC}]
11:02:56.132 [main] DEBUG org.hibernate.jdbc.AbstractBatcher - about to open PreparedStatement (open PreparedStatements: 0, globally: 0)
11:02:56.132 [main] DEBUG org.hibernate.SQL - insert into MSW000 (W000_DATA_REC, W000_FILE_FLAGS, KEY_W000) values (?, ?, ?)
11:02:56.132 [main] TRACE org.hibernate.jdbc.AbstractBatcher - preparing statement
11:02:56.132 [main] TRACE o.h.p.entity.AbstractEntityPersister - Dehydrating entity: [com.xyzcompany.foo.edoi.ejb.msw000.MSW000Rec#component[keyW000]{keyW000=F000 ADSUFC}]
11:02:56.132 [main] TRACE org.hibernate.type.StringType - binding ' ADSUFCA ' to parameter: 1
11:02:56.132 [main] TRACE org.hibernate.type.StringType - binding ' ' to parameter: 2
11:02:56.132 [main] TRACE org.hibernate.type.StringType - binding 'F000 ADSUFC' to parameter: 3
11:02:56.132 [main] DEBUG org.hibernate.jdbc.AbstractBatcher - Executing batch size: 1
11:02:56.133 [main] DEBUG org.hibernate.jdbc.AbstractBatcher - about to close PreparedStatement (open PreparedStatements: 1, globally: 1)
11:02:56.133 [main] TRACE org.hibernate.jdbc.AbstractBatcher - closing statement
11:02:56.133 [main] TRACE org.hibernate.jdbc.ConnectionManager - registering flush end
11:02:56.133 [main] TRACE o.h.e.d.AbstractFlushingEventListener - post flush
11:02:56.133 [main] DEBUG o.h.transaction.JDBCTransaction - commit
11:02:56.133 [main] TRACE org.hibernate.impl.SessionImpl - automatically flushing session
11:02:56.133 [main] TRACE org.hibernate.jdbc.JDBCContext - before transaction completion
11:02:56.133 [main] TRACE org.hibernate.impl.SessionImpl - before transaction completion
11:02:56.133 [main] DEBUG o.h.transaction.JDBCTransaction - committed JDBC Connection
11:02:56.133 [main] TRACE org.hibernate.jdbc.JDBCContext - after transaction completion
11:02:56.133 [main] DEBUG org.hibernate.jdbc.ConnectionManager - transaction completed on session with on_close connection release mode; be sure to close the session to release JDBC resources!
11:02:56.133 [main] TRACE org.hibernate.impl.SessionImpl - after transaction completion
11:02:56.133 [main] TRACE org.hibernate.impl.SessionImpl - closing session
11:02:56.133 [main] TRACE org.hibernate.jdbc.ConnectionManager - performing cleanup
11:02:56.133 [main] DEBUG org.hibernate.jdbc.ConnectionManager - releasing JDBC connection [ (open PreparedStatements: 0, globally: 0) (open ResultSets: 0, globally: 0)]
11:02:56.133 [main] TRACE o.h.c.DriverManagerConnectionProvider - returning connection to pool, pool size: 1
11:02:56.133 [main] TRACE org.hibernate.jdbc.JDBCContext - after transaction completion
11:02:56.133 [main] DEBUG org.hibernate.jdbc.ConnectionManager - transaction completed on session with on_close connection release mode; be sure to close the session to release JDBC resources!
11:02:56.134 [main] TRACE org.hibernate.impl.SessionImpl - after transaction completion
FINISH INSERT
When you call session.save(), hibernate will generate an INSERT SQL. This INSERT SQL will be appended to be issued to the DB during flushing (i.e session.flush()) .
During flushing, if hibernate.jdbc.batch_size is set to some non-zero value, Hibernate will use the batching feature introduced in the JDBC2 API to issue the batch insert SQL to the DB .
For example , if you save() 100 records and your hibernate.jdbc.batch_size is set to 50. During flushing, instead of issue the following SQL 100 times :
insert into TableA (id , fields) values (1, 'val1');
insert into TableA (id , fields) values (2, 'val2');
insert into TableA (id , fields) values (3, 'val3');
.........................
insert into TableA (id , fields) values (100, 'val100');
Hiberate will group them in batches of 50 , and only issue 2 SQL to the DB, like this:
insert into TableA (id , fields) values (1, 'val1') , (2, 'val2') ,(3, 'val3') ,(4, 'val4') ,......,(50, 'val50')
insert into TableA (id , fields) values (51, 'val51') , (52, 'val52') ,(53, 'val53') ,(54, 'val54'),...... ,(100, 'val100')
Please note that Hibernate would disable insert batching at the JDBC level transparently if the primary key of the inserting table isGenerationType.Identity.
From your log: you save() only one record and then flush(), so there is only one appending INSERT SQL to be processed for every flush. That's why Hibernate cannot help you to batch inserting as there is only one INSERT SQL to be processed. You should save() up to the certain amount of records before calling flush() instead of calling flush() for every save().
The best practise of batch inserting is something like this:
Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
for ( int i=0; i<888888; i++ ) {
TableA record = new TableA();
record.setXXXX();
session.save(record)
if ( i % 50 == 0 ) { //50, same as the JDBC batch size
//flush a batch of inserts and release memory:
session.flush();
session.clear();
}
}
tx.commit();
session.close();
You save and flush the records batch by batch. In the end of each batch you should clear the persistence context to release some memory to prevent memory exhaustion as every persistent object is placed into the first level cache (your JVM's memory). You could also disable the second-level cache to reduce the unnecessary overhead.
Reference:
Official Hibernate Documentation : Chapter 14. Batch processing
Hibernate Batch Processing – Why you may not be using it. (Even if you think you are)
If you must use hibernate for huge batch jobs StatelessSession is the way to go. It strips things down to the most basic converting-objects-to-SQL-statements mapping and eliminates all of the overhead of the ORM features you're not using when just cramming rows into the DB wholesale.
It would also be much easier to make suggestions on your actual code than the log :)
11:02:56.133 [main] DEBUG o.h.transaction.JDBCTransaction - commit
This is saying that the database is committing after every insert. Ensure you are not committing your transaction / closing your session inside the insert loop. Do this once at the end instead.