Optimizing a combination of update and select query using JAVA - java

I have one update query which I am updating using JAVA. It is connected to a Oracle SQL database. It is a combination of an update and select query.
Query: "Update table1 set col1 = 'true', col2= current_timestamp where table1.col1 <> 'true' and table1.col3 in (select col3 from table1 inner join table2 on table1.col3 = table2.col3 where table2.trdate is null or table2.trdate >= current_timestamp group by table1.col3 )
(select col3 from table1 inner join table2 on table1.col3 = table2.col3 where table2.trdate is null or table2.trdate >= current_timestamp group by table1.col3 ) -> Output of this query will be more 10 million rcords.
select table1.col3 where table1.col1 <> 'true' and table1.col3 in (select table1.col3 from table1 inner join table2 on table1.col3 = table2.col3 where table2.trdate is null or table2.trdate >= current_timestamp group by table1.col3 ) -> Output of this query will be around 30k rcords.
trdate is a timestamp.
col3 is a varchar and is connecting the two tables using join.
The above update query is taking lot of time (more than 6-7 hours) and is blocking the database.
Can someone suggest any better approach to optimize the query?
I am thinking to split the process into batches but since there are more than 10 million records, they will be using my space. Can someone help?

Related

Oracle SQL query working in console but not working in HQL

I'm not sur why my SQL query works fine on Oracle console, but not with HQL.
Here's the query as I run it in my Oracle console :
select distinct table1.*
from TABLE1 table1, TABLE2 table2
where table1.ID in (
select max(table2.RID_TABLE2) from TABLE2 table2 group by RID_OBJECT
);
This query returns 5 rows, and for some reason, the exact same query in HQL returns 6 rows.
Here is the HQL query:
#Query(name = "select distinct table1.*
from TABLE1 table1, TABLE2 table2
where table1.ID in (
select max(table2.RID_TABLE2) from TABLE2 table2 group by RID_OBJECT
)")
Set<Object> findTable1ByTable2();
From what I've seen here, it might be related to the fact that I use a subquery in my where clause, and it seems like HQL queries do not allow it.
Basically, it seems like the following part of my 'where' my query is simply ignored with HQL table1.ID in (select max(table2.RID_TABLE2) from TABLE2 table2 group by RID_OBJECT).
My goal is to have both my SQL and HQL queries return 5 rows.
Is there a solution to this ?

Oracle SQL Developer - JOIN on 2 queries with a one-to-many relationship

I have two queries that I'm trying to join together.
In first_query TABLE2.PROCESS_ID, every PROCESS_ID is unique in that table. In second_query though there are several PROCESS_ID's with the same number in TABLE3, so I think I have to do a one-to-many join. The join_query I have is giving me an error ORA-00933: SQL command not properly ended which I'm assuming has something to do with the one-to-many relationship with the JOIN.
I'm not really sure how to resolve this. Any help would be much appreciated!
first_query = """
SELECT TABLE1.RULE_ID, TABLE2.STATUS, TABLE2.ERROR_MESSAGE, TABLE2.PROCESS_ID
FROM TABLE2 LEFT JOIN
TABLE1
ON TABLE1.RULE_ID = TABLE2.RULE_ID
WHERE TABLE1.RULE_NAME IN ('TEST1', 'TEST2')
"""
second_query = """
SELECT RECORDS_PROCESSED, PROCESS_ID, STATUS
FROM TABLE3
"""
join_query = """
SELECT RULE_ID, STATUS, ERROR_MESSAGE, PROCESS_ID
FROM (first_query) as query_1
INNER JOIN (second_query) as query_2
ON query_1.PROCESS_ID = query_2.PROCESS_ID
GROUP BY PROCESS_ID desc
"""
You can not select 4 columns and group by only one of them unles you include selected columns as part of aggregation fucntion(like max(), sum(),...). One of the options is this:
SELECT query_1.RULE_ID --1
, query_2.STATUS --2
, query_1.ERROR_MESSAGE --3
, query_1.PROCESS_ID --4
FROM (SELECT TABLE1.RULE_ID
, TABLE2.STATUS
, TABLE2.ERROR_MESSAGE
, TABLE2.PROCESS_ID
FROM TABLE2
LEFT JOIN TABLE1
ON TABLE1.RULE_ID = TABLE2.RULE_ID
WHERE TABLE1.RULE_NAME IN ('TEST1', 'TEST2')) query_1
INNER JOIN (SELECT RECORDS_PROCESSED
, PROCESS_ID
, STATUS
FROM TABLE3) query_2
ON query_1.PROCESS_ID = query_2.PROCESS_ID
GROUP BY query_1.RULE_ID
, query_2.STATUS
, query_1.ERROR_MESSAGE
, query_1.PROCESS_ID
Also please do consider using aliases like this(in your first query):
SELECT T1.RULE_ID
, T2.STATUS
, T2.ERROR_MESSAGE
, T2.PROCESS_ID
FROM TABLE2 T2
LEFT JOIN TABLE1 T1 ON T1.RULE_ID = T2.RULE_ID
WHERE T1.RULE_NAME IN ('TEST1', 'TEST2')
Also, apply the same logic with aliases on your final query or else you will have a different kind of error : "ORA-00918: column ambiguously defined"
Here is a small demo
CTE (i.e. the WITH factoring clause) might help.
WITH first_query
AS (SELECT table1.rule_id,
table2.status,
table2.error_message,
table2.process_id
FROM table2 LEFT JOIN table1 ON table1.rule_id = table2.rule_id
WHERE table1.rule_name IN ('TEST1', 'TEST2')),
second_query
AS (SELECT records_processed, process_id, status FROM table3)
SELECT a.rule_id,
a.status,
a.error_message,
a.process_id
FROM first_query a INNER JOIN second_query b ON a.process_id = b.process_id
GROUP BY you used is invalid; you can't group results by only one column. If results have to be unique, use select distinct. If you have to use group by, specify all columns returned by select (which leads you back to what distinct does), or see whether some column(s) have to be aggregates - in that case, group by makes sense.
Also, you should always use table aliases. Without them, query is invalid as database engine doesn't know which table those columns (if they share the same name) belong to.

Hibernate HQL LEFT JOIN not performing same as equivalent SQL

Cannot work out why these two statements return a different number of results. They should be the same. The SQL version (correctly) returns 2 results and the HQL version returns (incorrectly) 3 results.
The output of the SQL returns 3 results of tb2.user with values 1, null and null. The where clause means this filters down to 2 results, removing the result with a tb2.user value of 1. However, the HQL version returns 3 results. I would like the HQL to return 2 results.
My SQL
SELECT * FROM table1 as tb1 LEFT JOIN table2 as tb2 ON tb1.user = tb2.blocked WHERE tb2.user <> 1 OR tb2.user is null;
My HQL
SELECT r FROM table1 tb1 LEFT JOIN table2 tb2 ON tb1.user.id = tb2.user.id WHERE tb2.user.id <> :userId OR tb2.user.id is null GROUP BY tb1
Any help on this is much appreciated!
You should not use left joined table's column in where condition .. this work as an inner join
you should move these condition in the related ON clause
and in the second query ( My HQL) you have an improper group by without aggregation function (so the query are not equivalent)
(when you need distinct result .. use Distinct clause )
SELECT r
FROM table1 tb1
LEFT JOIN table2 tb2 ON tb1.user.id = tb2.user.id
AND ( tb2.user.id <> :userId OR tb2.user.id is null )

Using JPA Criteria API, how can I execute this type of Delete Query

DELETE FROM table_name
WHERE NOT IN (
SELECT id
FROM table_name
WHERE user_id = 1
ORDER BY date DESC
LIMIT 50
)
I've tried creating a subquery, but it doesn't seem to allow you to set an ORDER BY or LIMIT value.

jOOQ - Concisely Representing Both Columns and Aggregate/Window Functions in a Query

This post comes as a result of a comment I left on a similar question: https://stackoverflow.com/a/19860271/2308858
I'm using PostgreSQL and jOOQ 3.4 and trying to represent the following SQL query in jOOQ:
SELECT *, COUNT(*) OVER()
FROM table1 t1
JOIN table2 t2 ON (t1.id = t2.id)
JOIN table3 t3 ON (t1.otherId = t3.otherId)
I like how Postgres lets me concisely represent "all columns plus the count column" with nothing more than SELECT *, COUNT(*) OVER(). But when I try to represent this same query in jOOQ, the most concise way I can do is:
create.select( TABLE1.fields() ).select( TABLE2.fields() ).select( TABLE3.fields() ).select( count().over() )
.from( TABLE1 )
.join( TABLE2 ).on( TABLE1.ID.equal( TABLE2.ID ))
.join( TABLE3 ).on( TABLE1.OTHER_ID.equal( TABLE3.OTHER_ID ))
Ideally, I'd write this instead:
create.select().select( count().over() )
.from( TABLE1 )
.join( TABLE2 ).on( TABLE1.ID.equal( TABLE2.ID ))
.join( TABLE3 ).on( TABLE1.OTHER_ID.equal( TABLE3.OTHER_ID ))
But this doesn't seem to work. Any thoughts on how I can do this?
This solution, which you've found yourself, is indeed the way to go with the jOOQ API:
create.select( TABLE1.fields() )
.select( TABLE2.fields() )
.select( TABLE3.fields() )
.select( count().over() )
...
It conceptually corresponds to this valid SQL query:
SELECT table1.*, table2.*, table3.*, COUNT(*) OVER()
...
Manipulating the jOOQ "model API":
But if this is annoying to you, you can also work around this issue with this little trick:
// Get access to the "model API" from a statement without any SELECT fields
SelectQuery<?> select =
create.select()
.from( TABLE1 )
.join( TABLE2 ).on( TABLE1.ID.equal( TABLE2.ID ))
.join( TABLE3 ).on( TABLE1.OTHER_ID.equal( TABLE3.OTHER_ID ))
.getQuery();
// Copy all fields from the SELECT statement:
List<Field<?>> fields = new ArrayList<>(select.getSelect());
// And explicitly add them:
select.addSelect(fields);
select.addSelect(count().over());
This is equally verbose as your original attempt, but might be a bit simpler to use, generically.
Using a derived table
Of course, you could also simply write the following, equivalent SQL query, which would be more standard SQL:
SELECT t.*, COUNT(*) OVER()
FROM (
SELECT *
FROM table1 t1
JOIN table2 t2 ON (t1.id = t2.id)
JOIN table3 t3 ON (t1.otherId = t3.otherId)
) t
With jOOQ, this would translate to:
Table<?> t = select()
.from( TABLE1 )
.join( TABLE2 ).on( TABLE1.ID.equal( TABLE2.ID ))
.join( TABLE3 ).on( TABLE1.OTHER_ID.equal( TABLE3.OTHER_ID ))
.asTable("t");
create.select(t.fields(), count().over())
.from(t);
Support for the asterisk
In a future version of jOOQ, the actual asterisk (*) might be supported explicitly through the jOOQ API. At this point, it is a bit unclear how that can be achieved syntactically, though.
On a side-note:
I have always found it very curious that PostgreSQL allows this syntax here:
SELECT *, COUNT(*) OVER()
...
It is hardly ever supported by SQL engines and a bit "unpredictable". Neither does the SQL standard allow for a "standalone asterisk" to be combined with other column expressions.

Categories

Resources