is there any limit of statement i can use where in(1,2,3,4,5,6....) in createSQLQuery in spring hibernatetemplate() ?
As mentioned, Oracle has a limit. One workaround is to execute multiple queries, each with a subset of the IN parameters. The other is to use an OR in the clause: where column in (:firstsubset) or column in (:secondsubset) or ... Some databases also have a limit on the total SQL string length. I've never run into that with Oracle, but wouldn'be surprised.
Yes, but said limit is not caused by Hibernate or Spring but rather underlying database.
For example, Oracle AFAIK allows no more than 1000 items in IN() clause. There are usually also limits on the maximum statement length.
I believe it depends on the underlying database you are using. For example, Oracle used to limit you to 1000 items in an IN clause and I believe it still does, but I'm not certain. What is your database?
Related
I am new to hibernate. I am confused with criteria's setFirstResult method.
From the documentation it seems hibernate returns rows from the the given number.
Since SQL query does not guarantee the ordering of rows without order by clause,
how setFirstQuery works in this case(without orderBy clause)?
Does hibernate read index information from the database?
If I execute same SQL query multiple times, ordering might change, in this case how setFirstResult work?
Hibernate can not do something by its own unless its supported by underlying databases. Because Hibernate queries finally get transformed to Sql only.
Having said that it uses underlying databases capabilities like for PostgresSQL and MySQL it will generate query like limit ? offset ? .
You can add custom order using addOrder
.addOrder( Order.asc("name") )
It is your task to add an order by, the function as you noticed won't guarantee same results if executed several times over the same set of data.
setFirstResult is typically used in pagination.
I have a table that has a well defined index, what I understand from
org.hiber....table(appliesTo="tableName", indexes={#Index(name=" ",columnNames=" "})})
is that it creates an index, now will doing this and mentioning the column names used in the actual oracle DB index give me the optimal results or is the Index never used ? How do i use the index explicitly in HQL ? Also how do i ascertain the index is being used ?
It depends on what DBMS you are using. For example, in Oracle, you cannot control which index to use through the SQL if you are using cost-based optimizer. So it is even less likely you can do anything related to this through HQL.
To assure if indices are used, it is also depends on the DBMS. Normally I will get the actual SQL issued to DBMS, by dumping it in log through Hibernate or other JDBC logging tools (e.g. JdbcDsLog), and view the execution plan of the SQL
I am trying to write a database independant application with JDBC. I now need a way to fetch the top N entries out of some table. I saw there is a setMaxRows method in JDBC, but I don't feel comfortable using it, because I am scared the database will push out all results, and only the JDBC driver will reduce the result. If I need the top 5 results in a table with a billion rows this will break my neck (the table has an usable index).
Writing special SQL-statements for every kind of database isn't very nice, but will let the database do clever query planning and stop fetching more results than necessary.
Can I rely on setMaxRows to tell the database to not work to much?
I guess in the worst case I can't rely on this working in the hoped way. I'm mostly interested in Postgres 9.1 and Oracle 11.2, so if someone has experience with these databases, please step forward.
will let the database do clever query planning and stop fetching more
results than necessary.
If you use
PostgreSQL:
SELECT * FROM tbl ORDER BY col1 LIMIT 10; -- slow without index
Or:
SELECT * FROM tbl LIMIT 10; -- fast even without index
Oracle:
SELECT *
FROM (SELECT * FROM tbl ORDER BY col1 DESC)
WHERE ROWNUM < 10;
.. then only 10 rows will be returned. But if you sort your rows before picking top 10, all basically qualifying rows will be read before they can be sorted.
Matching indexes can prevent this overhead!
If you are unsure, what JDBC actually send to the database server, run a test and have the database engine log the statements received. In PostgreSQL you can set in postgresql.conf:
log_statement = all
(and reload) to log all statements sent to the server. Be sure to reset that setting after the test or your log files may grow huge.
The thing which could/may kill you with billion(s) of rows is the (highly likely) ORDER BY clause in your query. If this order cannot be established using an index then . . . it'll break your neck :)
I would not depend on the jdbc driver here. As a previous comment suggests it's unclear what it really does (looking at different rdbms).
If you are concerned regarding speed of your query you can use a LIMIT clause as well. If you use LIMIT you can at least be sure that it's passed on to the DB server.
Edit: Sorry, I was not aware that Oracle doesn't support LIMIT.
In direct answer to your question regarding PostgreSQL 9.1: Yes, the JDBC driver will tell the server to stop generating rows beyond what you set.
As others have pointed out, depending on indexes and the plan chosen, the server might scan a very large number of rows to find the five you want. Proper server configuration can help accurately model the costs to prevent this, but if value distribution is unusual you may need to introduce and optimization barrier (like with a CTE) to coerce the planner to produce a good plan.
Hibernate Criteria support provides a setMaxResults() method to limit the results returned from the db.
I can't find any answer to this in their documentation - how is this implemented? Is it querying for the entire result set and then returning only the request number? Or is it truly limiting the query on the database end (think LIMIT keyword as in mySql).
This is important because if a query could potentially return many many results, I really need to know if the setMaxResults() will still query for all the rows in the database (which would be bad).
Also - if its truly limiting the number of rows on the database end, how is it achieving this cross-db (since I don't think every rdbms supports a LIMIT functionality like mySql does).
Hibernate asks the database to limit the results returned by the query. It does this via the dialect, which uses whatever database-specific mechanism there is to do this (so for SQL Server it will do somthing like "select top n * from table", Oracle will do "select * from table where rownum < n", MySQL will do "select * from table limit n" etc). Then it just returns what the database returns.
The class org.hibernate.dialect.Dialect contains a method called supportsLimit(). If dialect subclasses override this method, they can implement row limit handling in a fashion native to their database flavor. You can see where this code is called from in the class org.hibernate.loader.Loader which has a method titled prepareQueryStatement, just search for the word limit.
However, if the dialect does not support this feature, there is a hard check in place against the ResultSet iterator that ensures Java object (entity) results will stop being constructed when the limit is reached. This code is also located in Loader as well.
I use both Hibernate and Hibernate Search and without looking at the underlying implementation I can tell you that they definitely do not return all results. I have implemented the same query returning all results and then changed it to set the first result and max results (to implement pagination) and the performance gains were massive.
They likely use dialect specific SQL for this, e.g. LIMIT in MySQL, ROWNUM in Oracle. Your entity manager is aware of the dialect that you are using so this is simple.
Lastly if you really want to check what SQL Hibernate is producing for this query, just set the "show_sql" property to true when you create your entity manager / factory and it spits out all the SQL it is running to the console.
HQL does not suppport a limitation inside a query like in SQL, only the setMaxResults() which you also found.
To find out if it transform the setMaxResults() into a LIMIT query, you can turn on your SQL logging.
I know Question is bit old. But yes setMaxResults() is truly limiting the number of rows on the database end.
If you really look into your Hibernate SQL output, you can find the following SQL statement has been appended to your query.
limit ?
I am new to MySql database. I've large table(ID,...). I select ID frequently with java code and.And that make a heavy load on transaction
select from tableName where ID=someID
notes:
1.Database could be 100,000 records
2.I can't cache result
3.ID is a primary key
4.I try to optimize time needed to return result from query.
Any ideas for optimization ?
thanks in advance
I fail to see the need to optimize. This is a simple query against a very tiny table in database terms and the item inthe where clause is a PK and thus indexed. This should run very fast.
Have you considere partitioning? Improving Database Performance with Partitioning.
If you change the query to use a parameter, it might be a bit more efficient. The server would not have to parse and semantic check the statement each time.
select * from tableName where ID = #someID
Then assign the parameter value for each execution. Here is an explanation of using prepared statements.