I'm inserting, updating and deleting many detached objects with hibernate and a c3p0 connectionpool.
The problem is that hibernate does not batch the statements but instead does a
select ##session.tx_read_only
between every session.persist/insert/update/delete(object). Profiling the sql-connection it looks like this:
select ##session.tx_read_only
insert...
select ##session.tx_read_only
insert...
select ##session.tx_read_only
insert...
select ##session.tx_read_only
insert...
select ##session.tx_read_only
insert...
select ##session.tx_read_only
with select ##session.tx_rad_only always returning "0" (of course). It doesn't matter whether i use a stateless or stateful session. The resulting performance is not acceptable and far of any expectation.
My Hibernate Konfiguration:
<property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>
<property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="hibernate.connection.url">jdbc:mysql://127.0.0.1:4040/xy?zeroDateTimeBehavior=convertToNull</property>
<property name="hibernate.connection.username">xy</property>
<property name="hibernate.connection.password">xy</property>
<property name="hibernate.connection.autocommit">false</property>
<property name="hibernate.show_sql">false</property>
<property name="hibernate.format_sql">false</property>
<property name="hibernate.use_sql_comments">false</property>
<property name="hibernate.query.factory_class">org.hibernate.hql.internal.classic.ClassicQueryTranslatorFactory</property>
<property name="hibernate.connection.provider_class">org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider</property>
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">20</property>
<property name="hibernate.c3p0.timeout">300</property>
<property name="hibernate.c3p0.max_statements">250</property>
<property name="hibernate.c3p0.idle_test_period">3000</property>
<property name="hibernate.jdbc.batch_size">250</property>
<property name="hibernate.connection.release_mode">auto</property>
<property name="hibernate.order_inserts">true</property>
<property name="hibernate.order_updates">true</property>
<property name="hibernate.cache.use_second_level_cache">false</property>
<property name="hibernate.cache.region.factory_class">org.hibernate.cache.ehcache.EhCacheRegionFactory</property>
<property name="hibernate.cache.provider_class">org.hibernate.cache.EhCacheProvider</property>
<property name="hibernate.cache.use_query_cache">true</property>
<property name="net.sf.ehcache.configurationResourceName">hibernate_ehcache.xml</property>
I'm using:
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-c3p0</artifactId>
<version>4.3.5.Final</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.31</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>4.3.4.Final</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>ejb3-persistence</artifactId>
<version>1.0.2.GA</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-ehcache</artifactId>
<version>4.3.4.Final</version>
</dependency>
I have nearly no expirience with hibernate and it is a good guess i made a huge mistake so please feel free to suggest anything.
I switched to hibernate because the ORM functionality and am coming from plain jdbc prepared statements with a blazing fast performance. The MYSQL-Server is in a good configuration-state.
edit1:
i'm aware of:
Unnecessary queries in Hibernate - MySql
I've got no transactional annotations in my entities nor a defined isolationlevel anywhere
edit2:
i changed my connectionpool to bonecp - and the problem continues. Seems to be clearly a hibernate copnfiguration issue.
edit3:
tried many different things and found maybe a trace of a hint:
If I manualy session.flush() every 5 inserts (=size of batch e.g.)[tried the batch-example AGAIN from hibernate], the select ##session.tx_read_only query appears double - every 5 queries. I therefore assume that select ##session.tx_read_only is related to flushing. are there any ways to prevent hibernate from flushing every single insert/update?
I tried so far: session.setFlushMode(FlushMode.COMMIT/NEVER/etc) without any change in the behaviour. Maybe I misconfigured anything... what does hibernate trigger to flush every insert? Unique constraint at the tables? hibernate validation framework? complex object-graphs? difficult concurrency? maybe a locking issue (Hibernate isn't sure if someone else locked the tables and doesn't batch but checks every single insert if the table is read only?)?
i found nothing related to this extrem (I assume) flushing bahaviour.
We solved this issue by just setting useLocalSessionState=true in the connection string.
The below link explain the details of ReadOnly related changes happened from Mysql5.6 and the java connector 5.1.23.
http://dev.mysql.com/doc/relnotes/connector-j/en/news-5-1-23.html
You need to include all those CRUD operations in a single Transaction so all statements are executed in the same DB connection.
You can also enable the following Hibernate configurations:
<property name="hibernate.order_inserts" value="true"/>
<property name="hibernate.order_updates" value="true"/>
<property name="hibernate.jdbc.batch_size" value="50"/>
Those queries don't mean you don't have batching. It's a MySQL thing
Some drivers require special batching reordering directives:
<property name="hibernate.connection.url">jdbc:mysql://host:port/db?rewriteBatchedStatements=true</property
I found the flaw in my configuration.
I had to change the mysql-connectionsstring from
<property name="hibernate.connection.url">jdbc:mysql://127.0.0.1:4040/xy?zeroDateTimeBehavior=convertToNull</property>
to
<property name="hibernate.connection.url">jdbc:mysql://127.0.0.1:4040/xy?rewriteBatchedStatements=true</property>
this solved my problems.
Related
I am trying to configure C3P0 connection pool for my hibernate application.
I am using below dependencies.
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>5.5.6.Final</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-c3p0</artifactId>
<version>5.5.6.Final</version>
</dependency>
I added below configs in my hibernate.cft.xml
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">20</property>
<property name="hibernate.c3p0.acquire_increment">5</property>
<property name="hibernate.c3p0.timeout">1800</property>
But I get the warning below:
WARN: HHH000022: c3p0 properties were encountered, but the c3p0 provider class was not found on the classpath; these properties are going to be ignored
If I explicitly specify the provider class like given below, it works.
<property name="hibernate.connection.provider_class">org.hibernate.c3p0.internal.C3P0ConnectionProvider</property>
But the documentation of the above class says it should be picked by default.
A connection provider that uses a C3P0 connection pool. Hibernate will
use this by default if the hibernate.c3p0.* properties are set.
Why is this not class picked up by default? Is it correct to explicitly specify org.hibernate.c3p0.internal.C3P0ConnectionProvider? It looks like org.hibernate.connection.C3P0ConnectionProvider is the class which is picked up by default, and most of the references found in the web are regarding it, but it is not available in the above mentioned maven dependencies.
Remove hibernate. from the property name, it should be:
<property name="connection.provider_class">org.hibernate.connection.C3P0ConnectionProvider</property>
In my application I'm using Eclipselink as ORM for OracleDB and I encountered performance problem.
I'm executing code like this:
entityManager
.createNamedQuery(RoleToPermissionEntity.FIND_BY_APPLICATION_ROLE, RoleToPermissionEntity.class)
.setParameter(RoleToPermissionEntity.APPLICATION_ROLES_QUERY_PARAM, applicationRoles)
.getResultList();
with named query:
SELECT mapping
FROM RoleToPermissionEntity mapping
WHERE mapping.applicationRole IN :applicationRoles
ORDER BY mapping.id
Entity manager is set by #PersistenceContext.
For 3 given application roles application gets 123 rows (from 393), 9 column each (2 Timestamps with time zone, 3 numbers, 4 short varchars).
I checked time of execution as difference between System.nanoTime() before and after execution of given code. It's about 550 ms, no matter if it's executed 1st time or 10th in a row. And my assumption is that it should be much faster.
My first guess was problem with query, so I checked Eclipselink logs. Executed query is:
SELECT *all_columns*
FROM *table_name*
WHERE (APPLICATION_ROLE IN (?,?,?)) ORDER BY ID
bind => [3_application_roles]
Looks ok for me. I tried to execute it as native query, but result is the same. I tried also other queries like SELECT * FROM table_name, but time still is about 500-600 ms.
I wanted to have some comparison for this time so I created database connection manually and executed query like:
Class.forName("oracle.jdbc.driver.OracleDriver");
connection = DriverManager.getConnection(database_args);
Statement statement = connection.createStatement();
statement.executeQuery(query);
I executed it for several times, first (when connection was established) took quite a long time, but next took like 50-60 ms.
My second guess was problem with connection pool. I tried to find something in Eclipselink docs and I noticed only that parameters:
<property name="eclipselink.connection-pool.default.initial" value="1"/>
<property name="eclipselink.connection-pool.default.min" value="16"/>
<property name="eclipselink.connection-pool.default.max" value="16"/>
should be set. They are, but the problem still exists.
Content of my persistence.xml:
<persistence>
<persistence-unit name=unit transaction-type="JTA">
<jta-data-source>datasource</jta-data-source>
<exclude-unlisted-classes>false</exclude-unlisted-classes>
<!-- cache needs to be deactivated for multiple pods -->
<!-- https://wiki.eclipse.org/EclipseLink/Examples/JPA/Caching -->
<shared-cache-mode>NONE</shared-cache-mode>
<properties>
<property name="eclipselink.logging.level" value="FINE"/>
<property name="eclipselink.logging.level.sql" value="FINE"/>
<property name="eclipselink.logging.parameters" value="true"/>
<!--<property name="eclipselink.ddl-generation" value="create-or-extend-tables"/>-->
<property name="eclipselink.weaving" value="false"/>
<property name="eclipselink.target-database"
value="org.eclipse.persistence.platform.database.oracle.Oracle12Platform"/>
<property name="eclipselink.connection-pool.default.initial" value="1"/>
<property name="eclipselink.connection-pool.default.min" value="16"/>
<property name="eclipselink.connection-pool.default.max" value="16"/>
</properties>
</persistence-unit>
</persistence>
What can I do to fix this behavior?
After few next hours I found the problem. Default fetch size of OJDBC is 10, so with increasing number of rows to fetch time increases very fast.
What is strange: this was my first idea, so I tried to set <property name="eclipselink.jdbc.fetch-size" value="100"/> in persistence.xml. It didn't work, so I jumped to other solutions. Today I set it on single query by query.setHint("eclipselink.jdbc.fetch-size", 100) and it works.
I am trying to use HikariCP with Hibernate in a maven java ee web app in Netbeans. This is my first time using Hibernate, and I am not using Spring yet as I am not familiar with it but may consider it in the future (please feel free to offer reason why I should adapt it).
In the HikariCP wiki, it mentions that:
As of Hibernate 4.3.6 there is an official ConnectionProvider class
from Hibernate, which should be used instead of the HikariCP
implementation. The class is called
org.hibernate.hikaricp.internal.HikariCPConnectionProvider.
It then goes on to describe the configuration for Hibernate 4.x. It talks about the old ConnectionProvider class there however so I am assuming this section is outdated?
I have added the following dependency to my POM.XML file:
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-hikaricp</artifactId>
<version>4.3.8.Final</version>
</dependency>
and with the following properties declared within the tag in hibernate.cfg.xml:
<property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>
<property name="hibernate.connection.provider_class">org.hibernate.hikaricp.internal.HikariCPConnectionProvider</property>
<property name="hibernate.hikari.dataSource.url">jdbc:mysql://localhost/testdb?zeroDateTimeBehavior=convertToNull</property>
<property name="hibernate.hikari.dataSource.user">testuser</property>
<property name="hibernate.hikari.dataSource.password">password</property>
<property name="hibernate.hikari.dataSourceClassName">com.mysql.jdbc.jdbc2.optional.MysqlDataSource</property>
<property name="hibernate.hikari.dataSource.cachePrepStmts">true</property>
<property name="hibernate.hikari.dataSource.prepStmtCacheSize">250</property>
<property name="hibernate.hikari.dataSource.prepStmtCacheSqlLimit">2048</property>
<property name="hibernate.hikari.dataSource.useServerPrepStmts">true</property>
<property name="hibernate.current_session_context_class">thread</property>
I am able to interact with the database as I was before, using the default Hibernate connection pool. Is this all I need to do? Is it now using the HikariCP connection pool? If not what other configuration is required?
If I were to later integrate Spring, how would this affect the HikariCP configuration, or location of the configuration?
I am using Hibernate tools 3.3 in Eclipse Indigo.
Is there any way to view the Sql equivalent query for the criteria that I created?
There is one Hibernate Dynamic SQL View which shows Sql preview for Hql editor.
But I haven't find any preview for criteria.
With Hibernate Criteria API the only way to view the SQL output is to run the query. No preview. In order to view the generated SQL you must configure your datasource to log your sql statements. Here is a persistence.xml example for Hibernate MS SQLServer dialect. The ="true" elements are all instructions to be verbose when running Hibernate queries. This has a big impact on performance, needless to say.
<persistence-unit name="projectPU" transaction-type="RESOURCE_LOCAL">
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<jta-data-source>jdbc/projectDS</jta-data-source>
<properties>
<property name="hibernate.dialect"
value="org.hibernate.dialect.SQLServerDialect" />
<property name="hibernate.show_sql" value="true" />
<property name="hibernate.format_sql" value="true" />
<property name="hibernate.use_sql_comments" value="true" />
</properties>
</persistence-unit>
The sql will only be generated when you run criteria.list()
Criteria crit = session.createCriteria(Foo.class);
// create aliases and projections etc. whose effects are not visible yet
List<Foo> fooList = crit.list(); // only now can you see errors!
See Logging hibernate SQL using log4j
I have set up Tapestry 5 project and all went fine, until I deployed Hibernate. I have created hibernate.xml file and
<hibernate-configuration>
<session-factory>
<property name="dialect">org.hibernate.dialect.MySQLDialect</property>
<property name="connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="connection.url">jdbc:mysql://localhost/project</property>
<property name="connection.username">root</property>
<property name="connection.password">password12</property>
<property name="connection.pool_size">5</property>
<!-- Print SQL to stdout. -->
<property name="show_sql">true</property>
<property name="format_sql">true</property>
<property name="use_sql_comments">true</property>
<property name="generate_statistics">true</property>
<property name="hibernate.archive.autodetection">class, hbm</property>
<property name="hibernate.transaction.flush_before_completion">true</property>
<!-- Mapping files TODO: Classify those mappings in exact order and define the relations between them in entities some time later on.-->
<mapping class="rs.project.com.entities.Fruit"/>
<mapping class="rs.project.com.entities.Article"/>
</session-factory>
and it's OK as far as the implementation of it is concerned. However when I deploy the app it defines me some other config, which can be seen on my trace log, and uses some other xml file, based on the mappings it shows me on the log, and it's about some completely different project I used a while ago. The thing is I can't see what's causing such a behavior, and I am really frustrated. I am using Tomcat Apache Catalina and MySQL for Hibernate. Also, I did some research and found out that persistance.xml file is being used in my project.properties which is kinda strange.
persistence.xml.dir=${conf.dir}
Driver for connecting my app to MySQL is jdbc.mysql.driver.So my goal is to possibly define the matter that causes such behavior here with you, and to solve it.
Thanks in advance for your answers.
If your tomcat log is referring to a different project, maybe your context declaration is not right?
Check your contexts directory (for me it's $Tomcat_home\conf\Catalina\localhost) or the Server.xml (if that's what you're using). Make sure that the context file in the contexts directory is pointing to the right directory/project. This error has happened to me before when a previous project had the same context-name as my current one.