I have the following hibernate query:
Query query = session.createQuery("from MyHibernateClass");
List<MyHibernateClass> result = query.list();// executes in 7000ms
When logging the sql being executed in MySQL I see
select
myhibernat0_.myFirstColumn as myfirstcolumn92_,
myhibernat0_.mySecondColumn as mysecondcolumn92_,
myhibernat0_.mythirdcolumn as mythirdcolumn92_,
myhibernat0_.myFourthColumn as myfourthcolumn92_
from MyHibernateClass myhibernat0_
where (1=1);
When measurering the java code in the jvm on a small dataset of 3500 rows in MyHibernateClass database table this takes about 7000ms.
If I on the otherhand uses direct jdbc as follows:
Statement statement = session.connection().createStatement();
ResultSet rs = statement.executeQuery("select * from MyHibernateClass");// 7ms
List<MyHibernateClass> result = convert(rs);// executes in 20ms
I see the same sql going into the database but now the time spend in the java code in the jvm is 7ms.
The MyHibernateClass is a simple java bean class with getters and setters, I use no special resulttransformers as can be seen in the example. I only need a read-only instance of the class, and it doesn't need to be attached to the hibernate session.
I would rather like to use the hibernate version but cannot accept the execution times.
Added information:
After adding hibernate logging I see
[2011-07-07 14:26:26,643]DEBUG [main] [logid: ] -
org.hibernate.jdbc.AbstractBatcher.logOpenResults(AbstractBatcher.java:426) -
about to open ResultSet (open ResultSets: 0, globally: 0)
followed by 3500 of the following log statements
[2011-07-07 14:26:26,649]DEBUG [main] [logid: ] -
org.hibernate.loader.Loader.getRow(Loader.java:1197) -
result row: EntityKey[com.mycom.MyHibernateClass#1]
followed by 3500 log statements like
[2011-07-07 14:27:06,789]DEBUG [main] [logid: ] -
org.hibernate.engine.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:130) -
resolving associations for [com.mycom.MyHibernateClass#1]
[2011-07-07 14:27:06,792]DEBUG [main] [logid: ] -
org.hibernate.engine.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:226) -
done materializing entity [com.mycom.MyHibernateClass#1]
What does this mean?
What is Hibernate doing in the first implementation, how can I find out?
Adding a constructor with all attributes of the class did the trick, now the execution times are 70ms for the hibernate query. Previously the class only had a default constructor without arguments and a constructor with the entity id argument.
Based on the new information I felt I should provide another answer. The difference looks like that you have a one-to-many association specified for a List or Set property in your bean.
You are probably specifying that lazy=false which will turn off lazy loading. With lazy loading turned off it will fetch every associated record for every MyHibernateClass entity and this is why it is taking so long to execute.
Try setting lazy=true and this will perform much faster and then only retrieve the associated entities when explicitly requesting them from the entity.
If you utilize Log4j in your application you can set a variety of different logging options specific to Hibernate to get a better picture of what is going on behind the scenes in Hibernate.
http://docs.jboss.org/hibernate/core/3.3/reference/en/html/session-configuration.html#configuration-logging
My guess is that this is the typical initial load time that occurs when first calling an HQL query in an application. Subsequent HQL queries should be noticeably and considerably faster after this first one.
I know this thread is old, but to update I ran into the same problem but with SQL Server and it turns out to be that SQL being printed by Hibernate and SQL Sent using the driver is different. Using MSSQL Driver by default sends the queries as stored procedures as RPC calls it's because the driver tries to optimize the query plan for MSSQL Standards , so it sends the queries something like
Hibernate Query:
select c.col1,c.col2 from customer c where c.name like #param1 and c.country like #param2
Actual Driver Sent Query:
#param1=somevalue, #param2=somevalue
declar sp ....
select c.col1,c.col2 from customer c where c.name like #param1 and c.country like #param2
go
Note: This Query I got through SQL Profiler Tool directly listening on DB
It turns out to be that sp_exec optimizations on the MSSQL tend to produce good Query plans that's get cached, but this would result in 'parameter sniffing' to know more about this problem read here...
So to overcome this I had following options:
Change my HQL to native Queries and add OPTION RECOMPILE FOR SOME PARAM
Use Direct query values instead of prepared statements so there will be no translation for param values and queries will not be modified as Stored Procedures by the Driver
Change the driver settings to not send the stored procedures (this is still bad because now the query plans in MSSQL server will be specific to this query, this is same as Option:2 but outside the code)
I didn't want to use OPTION 1 & 2 since that eliminates the whole purpose of using ORM Frameworks and I end up using OPTION 3 for now
So I changed the JDBC URL to send option prepareStatement=false
After setting this I had one more problem the query being sent like
Select * from customer c where c.name like **N**'somename' and c.country=**N**'somevalue'
Here there is a prefix before the values which states that to convert the encoding scheme , so I disable the JDBC url to sendUnicode = false
This all I did in JTDS driver options.. As far as I am concerned now the application is up and running fast. I have also introduced second level caches to cache it for some time..
Hope this helps for someone, if you have any good suggestion please let me know.
I had an incident where my application was always using every row in the result set of a query. I found a 40-fold increase in speed by setting my fetch size using the setFetchSize method below. (The performance improvement includes the addition of the count query.)
Long count = getStoreCount(customerId);
Query query = session.getNamedQuery("hqlGetStoresByCustomerId")
.setString("i_customerid",customerId)
.setFetchSize(count.intValue());
Be careful while doing this; my data set had about 100 rows, and it was scoped to a the life of a web request. If you have larger data sets, you will be eating Java Heap for the duration of the existence of that data, prior to returning it to the Java Heap.
I know this is an old question but here is what fixed it for me...
In your hibernate.cfg.xml make sure you have the correct !DOCTYPE... it should be as follows:
<!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate Configuration DTD 3.0//EN" "http://www.hibernate.org/dtd/hibernate-configuration-3.0.dtd">
Anyone else who is facing a similar issue with SQL Server can use sendStringParametersAsUnicode=false in the JDBC Query String as shown in this answer:
JPA (Hibernate) Native Query for Prepared Statement SLOW
If you're not using Unicode for your prepared statement parameters and want to utilize the index on the varchar field which you're using as a parameter for the prepared statement, this can help.
It took me 10 seconds to execute a simple select all query before I found out that DOCTYPE tag is written wrongly in hibernate.cfg.xml and *mapping object*.hbm.class
Make sure that hibernate.cfg.xml start with
<!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate Configuration DTD 3.0//EN" "http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
And mapping xml.class with
<!DOCTYPE hibernate-mapping PUBLIC "-//Hibernate/Hibernate Mapping DTD 3.0//EN" "http://www.hibernate.org/dtd/hibernate-mapping-3.0.dtd">
Now it took me 1-2 seconds to execute any queries.
Related
For the purpose of optimizing big insert processes in our system, we intend to batch in our insert sql statements. But no matter what I configure, so far, I'm not able to accomplish that.
Based on the answer on this SO question:
Hibernate batch size confusion
It should be possible.
I already configured the hibernate.jdbc.batch_size property but still was not able to batch the insert statements? I can confirm this by enabling sql logs on both hibernate side and the postgres server side.
Is it possible to merge multiple insert statements to 1 insert statement in hibernate with postgres db?
Sample entity I used below:
class GenericMessage {
String name
//other fields
}
I'm using GORM, so it's on groovy (These are done in a transaction):
GenericMessage message1 = new GenericMessage(name: 'name1').save()
GenericMessage message2 = new GenericMessage(name: 'name2').save()
session.flush()
session.clear()
I have a java application that does a SQL query against an Oracle database, that for some reason gives way less values when executed from the SQL Developer and from the application itself.
Now to the technicalities. The application produces a connection to the db using a wrapper library that employs c3p0. The c3p0 config has been checked, so we know that this things can't be:
-Pointing to wrong database/schema
-Restricted user
Then there's the query:
select to_char(AGEPINDW.TRANSACTION.TS_TRANSACTION,'yyyy-mm') as Time,result, count(*) as TOTAL, sum(face_value) as TOTAL_AMOUNT
from AGEPINDW.TRANSACTION
where (ts_transaction >= to_timestamp(to_char(add_months(sysdate,-1),'yyyy-mm'),'yyyy-mm')
and ts_transaction < to_timestamp(to_char(sysdate,'yyyy-mm'),'yyyy-mm')) and service_id in (2,23)
group by to_char(AGEPINDW.TRANSACTION.TS_TRANSACTION,'yyyy-mm'), result;
It doesn't have any parameter and is executed via your standard PreparedStatement. Yet the return from the app is wrong and I don't know what may be. Any suggestions?
storing filedata via a hibernate (using postgresql or oracle 10g/11g) property mapped as
<property name="fileData" type="binary">
<column name="fileData" length="104857600" />
</property>
from a java application i need to access the length of the field from the db (i do not want to load whole the object, for performance reasons). i did not find any hibernate (hql) solution to query that information, so i decided to search for features of the different databases that were used. for postgresql i found:
select BIT_LENGTH(filedata) from table
which works perfectly (and astonishing fast). now i need something similar for the use with oracle. i already tried
select utl_raw.length(filedata) from table
and
select DBMS_LOB.GETLENGTH(filedata) from table
which both results in the error msg:
"Error: ORA-00997: illegal use of LONG datatype"
is there any possibilty to query the length of that hibernate property on an oracle db without selecting the object itself?
(hibernate creates a "long raw" field on oracle, and a "bytea" field on postgresql)
thx in advance
(addition: statement will be used in a migration-context, newly created filedata entities will get a filesize property programatically)
Well, looks like i found an answer here:
https://community.oracle.com/thread/2137593
create table temp_deleteme as select to_lob(<long raw field>) obj from <tablename>;
select dbms_lob.getlength(obj) from temp_deleteme;
after creating the tempTable i can select the filedatalength from it, write it to my entities, and delete the table again. still not really good performance, but that let the db do the work and i will not have to transfer entities to my server for only calculating the needed information.
I am working on a project that requires JDBC Calls to an Oracle Database. I have set up UCP pooling to work with SpringJDBC. I have a fairly simple query that I am executing like the following...
SELECT * FROM TABLE WHERE ID IN (:ids)
my java code to set this query up looks like the following...
NamedParameterJdbcTemplate template = new NamedParameterJdbcTemplate(datasource);
Map<String,Object> paramMap = new HashMap<String,Object>();
paramMap.put("ids", Arrays.asList(idArray));
List<Result> results = template.query("SELECT * FROM TABLE WHERE ID IN (:ids)",
paramMap, new ResultRowMapper());
This all performs fine as long as there is only 1 id in the array. When I add a 2nd ID the query takes nearly 5 minutes to run. If I take the exact query and execute it in SQLDeveloper, it takes .093 seconds.
Something must be going terribly wrong with my code or configuration... Does anyone have any ideas?
EDIT:
I stripped out the usage of the Spring NamedParameterJdbcTemplate and went with just straight Jdbc and everything seems to perform great. What is it that NamedParameterJdbcTemplate is doing differently?
Well, I found the difference between my straight jdbc solution and my spring-jdbc solution in this situation... It appears that as #Annjawn below explained, it is a bind variable issue not a spring-jdbc issue. My spring-jdbc issue was trying to bind a variable to an index (that doesn't exist) thus doing a table scan...
My straight JDBC solution ends up just doing a string replacement and executing as is thus no table scan...
The below link explains the difference.
http://bytes.com/topic/oracle/answers/65559-jdbc-oracle-beware-bind-variables
Something of an novice with HSQL and Hibernate...
em.getTransaction().begin();
for (Activity theActivity : activities) {
em.persist(theActivity);
}
em.getTransaction().commit();
em.close();
followed by...
EntityManager em = emf.createEntityManager();
em.getTransaction().begin();
System.out.println("QUERY:: "
+ em.createQuery("SELECT COUNT(*) FROM " + Activity.class.getName()).getSingleResult()
.toString());
em.getTransaction().commit();
Prints 25000 (the number of Activity objects in activities). But when I run this test again, the number of objects in the count(*) doesn't increase (and is 0 at the beginning of the program). So the objects aren't getting durably written.
This is my hsqldb connection string:
name="hibernate.connection.url" value="jdbc:hsqldb:file:data/cmon"
so it's not an in-memory database as far as I know...
Does anyone have any ideas why the objects aren't getting persisted beyond a single JVM session? Happy to supply more information but there's so much state associated with Hibernate / JPA / HSQL that it's not clear exactly what is pertinent.
Does anyone have any ideas why the objects aren't getting persisted beyond a single JVM session?
HSQLDB doesn't write changes immediately to disk after a commit (see "WRITE DELAY"), HSQLDB is not Durable by default (that's from where "performances" are coming from).
Either try to set the connection property shutdown=true in the connection string to get the changes written when the last connection will end.
jdbc:hsqldb:file:data/cmon;shutdown=true
If it doesn't help, try to set the WRITE DELAY to 0 (or false). If you're using HSQLDB 1.8.x, use the SQL command:
SET WRITE_DELAY 0
If you're using HSQLDB 2.0.x, you can now also use a connection property hsqldb.write_delay:
jdbc:hsqldb:file:data/cmon;hsqldb.write_delay=false
The solution is :
<property name="dialect">org.hibernate.dialect.HSQLDialect</property>
in hibernate.cfg.xml
This is rest of my configuration:
Libs:
HsqlDb 2.0.0
Hibernate 3.5.6
Url:
<property name="connection.url">jdbc:hsqldb:file:data/mydb;shutdown=true;hsqldb.write_delay=false;</property>
Did you set hibernate.hbm2ddl.auto to create-drop in your persistence.xml? This drops your tables and re-creates them on every startup.
You can set it to update instead, or if you want to manage the schema yourself, then set it to validate.
I was using HSQL DB version 2.2.5.
I tried above approaches i.e. setting shutdown=true and hsqldb.write_delay=false
It did not work. As suggested in some blog, I added statement
org.hsqldb.DatabaseManager.closeDatabases(0);
after transaction commit. But it did not work.
HSQL DB version 2.2.9 seems better than this. With one workaround it solves this problem. To handle above problem take following steps :-
1) hsqldb.jar from lib of HSQL DB version 2.2.9
2) In hibernate config xml just specify URL
I am using HSQL file-based database.
<property name="hibernate.connection.url">jdbc:hsqldb:file:D:\JavaProj\TmpDBLocation\myKauDB</property>
3) In your program at the end write statement
org.hsqldb.DatabaseManager.closeDatabases(0);
Now run the hibernate program that commits the data to DB.
Check HSQL DB by opening it in standalone mode and with URL
jdbc:hsqldb:file:D:\JavaProj\TmpDBLocation\myKauDB
You should see your changes persisted in DB.
Simply close your EntityManagerFactory with HSQL in filemode, after the commit to really persist datas...
Closing sessionfactory worked for me.