I have a table that has a well defined index, what I understand from
org.hiber....table(appliesTo="tableName", indexes={#Index(name=" ",columnNames=" "})})
is that it creates an index, now will doing this and mentioning the column names used in the actual oracle DB index give me the optimal results or is the Index never used ? How do i use the index explicitly in HQL ? Also how do i ascertain the index is being used ?
It depends on what DBMS you are using. For example, in Oracle, you cannot control which index to use through the SQL if you are using cost-based optimizer. So it is even less likely you can do anything related to this through HQL.
To assure if indices are used, it is also depends on the DBMS. Normally I will get the actual SQL issued to DBMS, by dumping it in log through Hibernate or other JDBC logging tools (e.g. JdbcDsLog), and view the execution plan of the SQL
Related
In the Oracle database, the select statement, select * from tablename, does not give output in the order of insertion. In few articles, we have found that the Oracle database stores the row information based on Rowid.
We are using Oracle in a web application based on Java and there is a requirement to display the data in order of insertion in each module. So, applying an order by clause on each table is not feasible and can degrade the application's performance.
Is there any other way that the select statement returns data in insertion order?
Oracle version used is "Oracle Database 19c Standard Edition 2 Release 19.0.0.0.0 - Production Version 19.3.0.0.0"
Oracle is a relational database. In it, rows don't have any particular order which means that select statement might return result in different order when you run it several times. Usually it doesn't, but - if there are a lot of inserts/deletes - sooner or later you'll notice such a behavior. Therefore, the only certain way to return rows in desired order is to use - ta-daaa! - order by clause.
Also, you'll have to maintain your own order of insertion. A simple way to do that is to use a column whose source is a sequence.
I would check here first: previous-post
I would recommend not relying on any ordering unless you specify order by.
But what is the deficiency of adding something like ORDER BY ROWNUM ASC; to your queries? You can trim your result sets ( paginate ) or even only do it to the entities you want to 'maintain insertion order'.
Are you using anything for entity management? Hibernate has some defaults you could use as well. - Post some code examples and can provide additional help.
In our application using a H2 database (version 1.4.196) we have a search on a varchar field doing either "contains" (column like '%searchterm%) or "begins with" (column like 'searchterm%) searches.
The table is quite large (approx. 400,000 entries) and the search turns out to be slow (varying between 3 seconds on my local development computer to 6 to 12 seconds on our customer's machines).
I found out that the column in question was not indexed and added an index. It turned out that the search time did not improve, even when I added an index hint explicitly to the query. explain revealed to me that no index was used in both cases.
From my experience with other database systems (e.g. MSSQL) I know that, at least for "begins with" queries, indexes can be used to improve search peformances.
As I did not find any related documentation for the H2 database my question:
Is it possible to use indexes in like queries in H2?
Most DB systems WILL use the index if you have a LIKE query... but only up to the first % in it. There is pretty much no DB system in existence where the mere act of having an index on the column would have any effect on the speed of, say: SELECT * FROM my_table WHERE my_column LIKE '%findme%';.
What you are presumably looking for is an index specifically designed to aid in searching. For example, the tsquery system that postgres has (see postgres tsquery documentation).
Another way to go is to add a dependency to your software that does this. The obvious choice there is Apache Lucene.
H2 has some support for this. Please refer to H2 documentation on full text search.
I am trying to write a database independant application with JDBC. I now need a way to fetch the top N entries out of some table. I saw there is a setMaxRows method in JDBC, but I don't feel comfortable using it, because I am scared the database will push out all results, and only the JDBC driver will reduce the result. If I need the top 5 results in a table with a billion rows this will break my neck (the table has an usable index).
Writing special SQL-statements for every kind of database isn't very nice, but will let the database do clever query planning and stop fetching more results than necessary.
Can I rely on setMaxRows to tell the database to not work to much?
I guess in the worst case I can't rely on this working in the hoped way. I'm mostly interested in Postgres 9.1 and Oracle 11.2, so if someone has experience with these databases, please step forward.
will let the database do clever query planning and stop fetching more
results than necessary.
If you use
PostgreSQL:
SELECT * FROM tbl ORDER BY col1 LIMIT 10; -- slow without index
Or:
SELECT * FROM tbl LIMIT 10; -- fast even without index
Oracle:
SELECT *
FROM (SELECT * FROM tbl ORDER BY col1 DESC)
WHERE ROWNUM < 10;
.. then only 10 rows will be returned. But if you sort your rows before picking top 10, all basically qualifying rows will be read before they can be sorted.
Matching indexes can prevent this overhead!
If you are unsure, what JDBC actually send to the database server, run a test and have the database engine log the statements received. In PostgreSQL you can set in postgresql.conf:
log_statement = all
(and reload) to log all statements sent to the server. Be sure to reset that setting after the test or your log files may grow huge.
The thing which could/may kill you with billion(s) of rows is the (highly likely) ORDER BY clause in your query. If this order cannot be established using an index then . . . it'll break your neck :)
I would not depend on the jdbc driver here. As a previous comment suggests it's unclear what it really does (looking at different rdbms).
If you are concerned regarding speed of your query you can use a LIMIT clause as well. If you use LIMIT you can at least be sure that it's passed on to the DB server.
Edit: Sorry, I was not aware that Oracle doesn't support LIMIT.
In direct answer to your question regarding PostgreSQL 9.1: Yes, the JDBC driver will tell the server to stop generating rows beyond what you set.
As others have pointed out, depending on indexes and the plan chosen, the server might scan a very large number of rows to find the five you want. Proper server configuration can help accurately model the costs to prevent this, but if value distribution is unusual you may need to introduce and optimization barrier (like with a CTE) to coerce the planner to produce a good plan.
Hibernate Criteria support provides a setMaxResults() method to limit the results returned from the db.
I can't find any answer to this in their documentation - how is this implemented? Is it querying for the entire result set and then returning only the request number? Or is it truly limiting the query on the database end (think LIMIT keyword as in mySql).
This is important because if a query could potentially return many many results, I really need to know if the setMaxResults() will still query for all the rows in the database (which would be bad).
Also - if its truly limiting the number of rows on the database end, how is it achieving this cross-db (since I don't think every rdbms supports a LIMIT functionality like mySql does).
Hibernate asks the database to limit the results returned by the query. It does this via the dialect, which uses whatever database-specific mechanism there is to do this (so for SQL Server it will do somthing like "select top n * from table", Oracle will do "select * from table where rownum < n", MySQL will do "select * from table limit n" etc). Then it just returns what the database returns.
The class org.hibernate.dialect.Dialect contains a method called supportsLimit(). If dialect subclasses override this method, they can implement row limit handling in a fashion native to their database flavor. You can see where this code is called from in the class org.hibernate.loader.Loader which has a method titled prepareQueryStatement, just search for the word limit.
However, if the dialect does not support this feature, there is a hard check in place against the ResultSet iterator that ensures Java object (entity) results will stop being constructed when the limit is reached. This code is also located in Loader as well.
I use both Hibernate and Hibernate Search and without looking at the underlying implementation I can tell you that they definitely do not return all results. I have implemented the same query returning all results and then changed it to set the first result and max results (to implement pagination) and the performance gains were massive.
They likely use dialect specific SQL for this, e.g. LIMIT in MySQL, ROWNUM in Oracle. Your entity manager is aware of the dialect that you are using so this is simple.
Lastly if you really want to check what SQL Hibernate is producing for this query, just set the "show_sql" property to true when you create your entity manager / factory and it spits out all the SQL it is running to the console.
HQL does not suppport a limitation inside a query like in SQL, only the setMaxResults() which you also found.
To find out if it transform the setMaxResults() into a LIMIT query, you can turn on your SQL logging.
I know Question is bit old. But yes setMaxResults() is truly limiting the number of rows on the database end.
If you really look into your Hibernate SQL output, you can find the following SQL statement has been appended to your query.
limit ?
is there any limit of statement i can use where in(1,2,3,4,5,6....) in createSQLQuery in spring hibernatetemplate() ?
As mentioned, Oracle has a limit. One workaround is to execute multiple queries, each with a subset of the IN parameters. The other is to use an OR in the clause: where column in (:firstsubset) or column in (:secondsubset) or ... Some databases also have a limit on the total SQL string length. I've never run into that with Oracle, but wouldn'be surprised.
Yes, but said limit is not caused by Hibernate or Spring but rather underlying database.
For example, Oracle AFAIK allows no more than 1000 items in IN() clause. There are usually also limits on the maximum statement length.
I believe it depends on the underlying database you are using. For example, Oracle used to limit you to 1000 items in an IN clause and I believe it still does, but I'm not certain. What is your database?