We are using a Java EE application and we are right now using Informix DB.Our code hits the DB with queries like
"select first 10 * from test"
Now as far as I know Oracle does not support 'first 10 *' kind of statements.We have more than 1000 queries like this.Should we manually change this or can have some manual customization?
This is a good reason for either only using standard SQL as much as possible, or for isolating those dependencies into stored procedures (yes, I know that doesn't help you in this specific case, I just thought I'd mention it for future reference).
I suspect you'll have to change each one individually although a simple search over you source code for "select " or "first " will be a good start.
Then you can decide how you want to change them, since you may also still want it to work on Informix.
For what it's worth, I think you get the same effect with Oracle's
select * from ( select * from mytable ) where rownum <= 10
I would be farming the job of dynamically constructing a query (based on a template) out to another layer which can return a different query based on which database you have configured. Then, when you also want to support DB2 (for example), it's a simple matter of changing just that layer.
For example, have a call like:
gimmeRowLimitedSqlQuery ("* from test",10);
which would give you either of:
select first 10 * from test
select * from test where rownum <= 10
I should also mention, although I realise your query is just an example, that SQL can return rows in any order if you don't specify order by so
select first 10 * from test
makes little sense, especially if you may be running it in different DBMS'.
You could write an extension to the JDBC driver to modify the queries on the fly but probably that is an overkill so a careful search and replace on the source code to modify all queries would be more appropriate.
Oracle has the concept of ROWNUM for limiting results. You will have to update your queries for this.
TOP-n and Pagination queries are a little bit more complex than just using ROWNUM. For example, you might be surprised that you don't get the expected results when using ROWNUM with ORDER BY in the same query.
Check http://www.oracle.com/technology/oramag/oracle/07-jan/o17asktom.html for more info on those type of queries in Oracle.
Related
I have a question, would like to get some help with.
I have the query running from Java.
SELECT DISTINCT field1, field1
from tblTableA WITH (NOLOCK)
WHERE criteriaField='CONSTANT TEXT'
I run it with jpa
Query qry = entMgr.createNativeQuery(myQry) ;
List sqlResult = qry.getResultList() ;
Now, that qry.getResultList() takes too much time to run - 75 or more seconds. Yes, it returns close to 700 000 records, but the same query ran on Weblogic 10, using ejb2 runs in less than 5 seconds time
Can anyone help resolving this issue, seems like there maybe a configuration I am missing, or a technique I am not following.
There is something on account of using
jbosscmp-jdbc.xml.
I don't have that in my set up, but found out that there is a lazy-loading feature that we can configure. Now, I am not sure how make the query I am running be configured in xml file.
Also, can this be used with annotations instead of xml file ?
I would try to run this query inside of a non-transactional method:
#TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED)
List getResults(..){
Query qry = entMgr.createNativeQuery(myQry) ;
return qry.getResultList() ;
}
This is sometimes not allowed depending on the environment and is mainly used for the optimization of queries expecting to have large results sets and which would later be managed by the PersistenceContext (so basically when you would use HQL instead of native)
But i would give it a try.
You are performing this select query within a transaction scope. I found an old JIRA ticket on Jboos's site. As the ticket suggests, there is a potential around the flush. If you perform a query with EJB3, a flush is performed or attempted automatically for all the objects you retrieve with your native query. The idea seems to be avoid getting stale objects from the database. But in your case, it is not applicable. Set the flush mode to COMMITand see if the performance improves.
query.setFlushMode( FlushModeType.COMMIT );
Also turn off the Hibernate logging and see if that makes any difference.
I have a query which has 2 'in' Clauses. First in clause takes around 125 values and second in clause of query takes around 21000 values. Its implemented using JPA CriteriaBuilder.
Query itself executes very fast and return results within seconds. Only problem is entityManager.createQuery(CriteriaQuery) takes around 12-13 minutes to return.
I search all over SO, all the threads are related to performance of Query.getResultList. None of them discuss about performance of entityManager.createQuery(CriteriaQuery). If you have seen such behavior earlier, please let me know, how to resolve it.
My JDK version is 1.7. Dependency version of javaee-api is 6.0. Application is deployed on JBOSS EAP 6.4. But that's not the concern as of now, as I am testing my code using junit using EntityManager connected to actual Oracle database. If you require more information, kindly let me know.
A hybrid approach is to dynamically create a query and then save it as a named query in the entity manager factory.
At that point it becomes just like any other named query that may have been declared statically in metadata. While this may seem like a good compromise, it turns out to be useful in only a few specific cases. The main advantage it offers is if there are queries that are not known until runtime, but then reissued repeatedly. Once the dynamic query becomes a named query it will only bear the cost of processing once.
It is implementation-specific whether that cost is paid when the query is registered as a named query, or deferred until the first time it is executed.
A dynamic query can be turned into a named query by using the
EntityManagerFactory addNamedQuery()
Keep us informed by the result and good luck
I observed that, having single query with 21 IN clauses (each with 1000 expressions) and all combined with OR clauses, made query run slower. I tried another approach of executing every IN Clause as a part of separate query. So these 21 individual queries performed better overall.
Another issue I observed was that Query with CriteriaBuilder was slow when result set is huge (something like 20K rows in result set). I solved this issue by adding query hint to my typed query:
TypedQuery.setHint("org.hibernate.fetchSize", 5000);
Hope it will help others.
Code in Hibernate is not expected to be used for binding lots of params:
for ( ImplicitParameterBinding implicitParameterBinding : parameterMetadata.implicitParameterBindings() ) {
implicitParameterBinding.bind( jpaqlQuery );
}
Unfortunately you need to find different approach if you want to do something similar.
I was writing test cases for query that uses connect by hierarchical clause.
It seems that there is no support for this clause in HSQL Db.
Are there any alternatives for testing the query or writing a different query that does the same thing.
The query is simple
SELECT seq.nextval
FROM DUAL
CONNECT BY level <= ?
Thanks.
You don't need a recursive query for that.
To generate a sequence of numbers you can use sequence_array
select *
from unnest(sequence_array(1, ?, 1))
More details are in the manual:
http://hsqldb.org/doc/2.0/guide/builtinfunctions-chapt.html#N14088
If you need that to advance a sequence a specific number of entries, you can use something like this:
select NEXT VALUE FOR seq
from unnest(sequence_array(1, 20, 1));
If you need that to set the sequence to a new value, this is much easier in HSQLDB:
ALTER SEQUENCE seq restart with 42;
If you are looking for a recursive query, then HSQLDB supports the ANSI SQL standard for that: recursive common table expressions, which are documented in the manual:
http://hsqldb.org/doc/2.0/guide/dataaccess-chapt.html#dac_with_clause
According to this 2-year-old ticket, only Oracle and a database called CUBRID have CONNECT BY capability. If you really want it, maybe you could vote on the ticket. However, as far as I have been able to tell, there are only two people working on the project, so don't hold your breath.
I'm new to Documentum and have a simple problem, I am trying to retrieve all the record according to last modified.
Basically I have a datatable with 1000 records.
current we use
Select * from docfolder enabled (FETCH_ALL_RESULTS 1000)
The problem with the above statement is sometimes a newly created report or modified report will out of the 1000 range and our users will complain report not found * valid complain *
actually the last modified record does not even need to be the first on the list, it just need to appear.
I tried using
Select * from docfolder order by r_modify_date enabled (FETCH_ALL_RESULTS 1000)
but this takes too long(never complete). I try replacing * with a,b,c,d (fields) but it does not work too.
May I know if there is other solutions to my issue?
I am considering documentum "ENABLE (RETURN_TOP 10)" hint but I doubt it work for Oracle 11g and how does documentum define top 1000?
UPDATE: It seems that using data link via toad is faster than using DQL, but I need a DQL solution due to legacy issues.
Documentum 6.0 and Oracle 11g.
What version of Documentum are you using?
Ensure that there are indexes on the r_object_id. You may also want to add an index to the r_modify_date.
Further, when adding fields a,b,c,d - ensure that these fields are "non-repeating". In this way, Documentum will not need to join the _r table making the overall query faster.
Further, in DA, if you do the query, you can actually see the SQL query passed to Oracle. Take this query and run it in Toad and look for optimizations. You may also register the _s table so that your can DQL query the _s table directly.
I manage to solve this problem by querying the under lining table in oracle database.
The reason for slow performance was because of the table begin joint behind to obtain the result.
In future if you have exhausted all ways to optimize your DQL, just fall back to querying the oracle database.
I have recommended for all table view and search to query via oracle.
Only individual report are retrieved via documentum, sometimes I question the purpose of having documentum.
I am trying to write a database independant application with JDBC. I now need a way to fetch the top N entries out of some table. I saw there is a setMaxRows method in JDBC, but I don't feel comfortable using it, because I am scared the database will push out all results, and only the JDBC driver will reduce the result. If I need the top 5 results in a table with a billion rows this will break my neck (the table has an usable index).
Writing special SQL-statements for every kind of database isn't very nice, but will let the database do clever query planning and stop fetching more results than necessary.
Can I rely on setMaxRows to tell the database to not work to much?
I guess in the worst case I can't rely on this working in the hoped way. I'm mostly interested in Postgres 9.1 and Oracle 11.2, so if someone has experience with these databases, please step forward.
will let the database do clever query planning and stop fetching more
results than necessary.
If you use
PostgreSQL:
SELECT * FROM tbl ORDER BY col1 LIMIT 10; -- slow without index
Or:
SELECT * FROM tbl LIMIT 10; -- fast even without index
Oracle:
SELECT *
FROM (SELECT * FROM tbl ORDER BY col1 DESC)
WHERE ROWNUM < 10;
.. then only 10 rows will be returned. But if you sort your rows before picking top 10, all basically qualifying rows will be read before they can be sorted.
Matching indexes can prevent this overhead!
If you are unsure, what JDBC actually send to the database server, run a test and have the database engine log the statements received. In PostgreSQL you can set in postgresql.conf:
log_statement = all
(and reload) to log all statements sent to the server. Be sure to reset that setting after the test or your log files may grow huge.
The thing which could/may kill you with billion(s) of rows is the (highly likely) ORDER BY clause in your query. If this order cannot be established using an index then . . . it'll break your neck :)
I would not depend on the jdbc driver here. As a previous comment suggests it's unclear what it really does (looking at different rdbms).
If you are concerned regarding speed of your query you can use a LIMIT clause as well. If you use LIMIT you can at least be sure that it's passed on to the DB server.
Edit: Sorry, I was not aware that Oracle doesn't support LIMIT.
In direct answer to your question regarding PostgreSQL 9.1: Yes, the JDBC driver will tell the server to stop generating rows beyond what you set.
As others have pointed out, depending on indexes and the plan chosen, the server might scan a very large number of rows to find the five you want. Proper server configuration can help accurately model the costs to prevent this, but if value distribution is unusual you may need to introduce and optimization barrier (like with a CTE) to coerce the planner to produce a good plan.