Develop Reference

Oracle: Result set in insertion order

In the Oracle database, the select statement, select * from tablename, does not give output in the order of insertion. In few articles, we have found that the Oracle database stores the row information based on Rowid.
We are using Oracle in a web application based on Java and there is a requirement to display the data in order of insertion in each module. So, applying an order by clause on each table is not feasible and can degrade the application's performance.
Is there any other way that the select statement returns data in insertion order?
Oracle version used is "Oracle Database 19c Standard Edition 2 Release 19.0.0.0.0 - Production Version 19.3.0.0.0"

Oracle is a relational database. In it, rows don't have any particular order which means that select statement might return result in different order when you run it several times. Usually it doesn't, but - if there are a lot of inserts/deletes - sooner or later you'll notice such a behavior. Therefore, the only certain way to return rows in desired order is to use - ta-daaa! - order by clause.
Also, you'll have to maintain your own order of insertion. A simple way to do that is to use a column whose source is a sequence.

I would check here first: previous-post
I would recommend not relying on any ordering unless you specify order by.
But what is the deficiency of adding something like ORDER BY ROWNUM ASC; to your queries? You can trim your result sets ( paginate ) or even only do it to the entities you want to 'maintain insertion order'.
Are you using anything for entity management? Hibernate has some defaults you could use as well. - Post some code examples and can provide additional help.

Is there a way within hibernate to retrieve fast non-blocking row counts?

The following query generated by hibernate takes 13+ seconds and locks the table:
SELECT COUNT(auditentit0_.audit_id) AS col_0_0_ FROM Audit auditentit0_ WHERE 1=1;
The growing Microsoft SQL server database table contains 90+ million rows.
For Microsoft SQL server, I have found an accurate meta data way of getting the same information very quickly.
However, I would rather not write custom code for Microsoft sql server and oracle (the next database) if hibernate has a way of getting this information.
Here is an example meta data query for Microsoft sql server that is accurate and almost instant:
SELECT SUM (row_count) FROM sys.dm_db_partition_stats WHERE object_id=OBJECT_ID('huge_audit_table') AND (index_id=0 or index_id=1);
Is there a way to have hibernate issue a similar query for a table row count?
One posted answer has indicated that a view could be of use. I'm investigating this post to see if it can solve the issue:
https://vladmihalcea.com/map-jpa-entity-to-view-or-sql-query-with-hibernate/

In hibernate you should use projections like in the link you provided in order to guarantee that it works on multiple dbms:
protected Long countByCriteria(DetachedCriteria criteria) {
Criteria crit = criteria.getExecutableCriteria(getSession());
crit.setProjection(Projections.rowCount());
return (Long)crit.uniqueResult();
}
What engine are you using in mysql? I never had a blocking problem with row count in MySql or Oracle. Maybe the following link will help you: Any way to select without causing locking in MySQL?
Also, after some quick reading i see that Sql Server does indeed block on count.
Maybe you could use a stored procedure or some other mechanism to pass the problem to the dbms.
Edit:
Projections in Hibernate are used to select the columns to fetch, the columns to group elements by, and to use built-in aggregate functions (sum, count, avg, max, min, countDistinct).
It helps you keep your application database-agnotic. Remember that hibernate supports around 30 databases.
In your case you have an specific problem with mssql as the count blocks the table prioritizing accuracy. And using the system views is really quick as you get an estimate but isn´t standard.
You could encapsulate the problem into a view or stored procedure dbms dependant. Or maybe you could try with a NOLOCK hint or READ UNCOMMITED in hibernate (in a count of an audit table it should be acceptable).

To solve this particular problem we stepped back and changed how the UI functions. Through a collaborative effort between UIX and UI developers we agreed that unfiltered queries will NOT ask for total counts. The initial screen load will show only a page full of data. No page 1 of 60,000 controls will exists. Only when the user enters specific criteria will the total count come into play. Those queries should be very fast. Now... it is possible for the user to still setup a query that will be just as bad as the original problem. It should be the exception versus the norm.
So there really is not a solid answer for the OP. If you are faced with this type of problem, if you have control of the UI and API, then it is time to rethink the solution. Think of how google handles paging from a UI perspective. The days of showing a "page 1 of (XX)" are gone IMHO.

How to bulk delete records using temporary tables in Hibernate?

I have a question. Where did these methods go?
Dialect.supportsTemporaryTables();
Dialect.generateTemporaryTableName();
Dialect.dropTemporaryTableAfterUse();
Dialect.getDropTemporaryTableString();
I've tried to browse git history for Dialect.java, but no luck. I found that something like
MultiTableBulkIdStrategy was created but I couldn't find any example of how to use it.
To the point...I have legacy code (using hibernate 4.3.11) which is doing batch delete from
multiple tables using temporary table. In those tables there may be 1000 rows, but also there may
be 10 milion rows. So just to make sure I don't kill DB with some crazy delete I create temp table where I put (using select query with some condition) 1000 ids at once
and then use this temp table to delete data from 4 tables. It's running in while cycle until all data based on some condition is not deleted.
Transaction is commited after each cycle.
To make it more complicated this code has to run on top of: mysql, mariadb, oracle, postgresql, sqlserver and h2.
It was done using native SQL, with methods mentioned above. But not I can't find a way how
to refactor it.
My first try was to create query using nested select like this:
delete from TABLE where id in (select id from TABLE where CONDITION limit 1000) but this is way slower as I have to run select query multiple times for each delete and limit is not supported in nested select in HQL.
Any ideas or pointers?
Thanks.

The methods were present in version 4.3.11 but removed in version 5.0.0. It seems a bit unusual that they were removed rather than deprecated - the background is on this Jira ticket.
To quote from this:
Long term, I think the best approach is to remove the Dialect method
intended to support table tabled in a piecemeal fashion and to make
MultiTableBulkIdStrategy be a fully self-contained contract.
The methods were removed in this commit.
So it seems that getDefaultMultiTableBulkIdStrategy() is the intended replacement for these methods - but I'm not entirely clear on how, as it currently has no Javadoc. Guess you could try to work it out from the source code ...or if all else fails, perhaps try to contact Steve Ebersole, who implemented the change?

Replicate Oracle into HsqlDB (and knowing what the change was)

I am interested in taking an Oracle DB and "replicating" it into hsqldb - very fast, close to real time. And hopefully, also be aware of what fields were changed. (I need this in order to boost queries duration - and saw that HSQLDB in embedded in memory mode is much faster than even cached Oracle. However, since oracle grants me persistency, failover etc. I still want to use it).
So, I thought about a few possible approaches:
Use trigger on every possible table in my oracle db. The trigger will write the change to an auxiliary table. Very bad performance & practice, in my opion.
periodically select each table for all the latest updates (select * from T where ora_rowscn > ?). ?=latest maximal row scn. This has the disadvantage of not knowing about deletes (even though we can figure some other way for deletes). This also has the disadvantage of having to diff the previous record with the new record to understand the change. The table may be of 100 fields and the change on only one.
Use Oracle notifications, available since 11/10g - using a simple JDBC link - though this has some limitations, like: number of fields you can get that have changed.
Use "2" approach along with quering the sql_text table, in order to see which fields were affected in latest updates, and to diff only those from the last 1 minute. This will actually also help with figuring out deletes.
Use timesten instead of HSQLDB, but that costs money.
What do you think? What is the best way?
Thank you

You should explore the existing tools, notably SymmetricDS (http://www.symmetricds.org) and see if they can be configured or modified to support this.
An alternative approach is to write the triggers in HSQLDB to update the Oracle backend when there is a data change.

Migrating to Oracle

We are using a Java EE application and we are right now using Informix DB.Our code hits the DB with queries like
"select first 10 * from test"
Now as far as I know Oracle does not support 'first 10 *' kind of statements.We have more than 1000 queries like this.Should we manually change this or can have some manual customization?

This is a good reason for either only using standard SQL as much as possible, or for isolating those dependencies into stored procedures (yes, I know that doesn't help you in this specific case, I just thought I'd mention it for future reference).
I suspect you'll have to change each one individually although a simple search over you source code for "select " or "first " will be a good start.
Then you can decide how you want to change them, since you may also still want it to work on Informix.
For what it's worth, I think you get the same effect with Oracle's
select * from ( select * from mytable ) where rownum <= 10
I would be farming the job of dynamically constructing a query (based on a template) out to another layer which can return a different query based on which database you have configured. Then, when you also want to support DB2 (for example), it's a simple matter of changing just that layer.
For example, have a call like:
gimmeRowLimitedSqlQuery ("* from test",10);
which would give you either of:
select first 10 * from test
select * from test where rownum <= 10
I should also mention, although I realise your query is just an example, that SQL can return rows in any order if you don't specify order by so
select first 10 * from test
makes little sense, especially if you may be running it in different DBMS'.

You could write an extension to the JDBC driver to modify the queries on the fly but probably that is an overkill so a careful search and replace on the source code to modify all queries would be more appropriate.

Oracle has the concept of ROWNUM for limiting results. You will have to update your queries for this.

TOP-n and Pagination queries are a little bit more complex than just using ROWNUM. For example, you might be surprised that you don't get the expected results when using ROWNUM with ORDER BY in the same query.
Check http://www.oracle.com/technology/oramag/oracle/07-jan/o17asktom.html for more info on those type of queries in Oracle.