I need to scrub an SQL Server table on a regular basis, but my solution is taking ridiculously long (about 12 minutes for 73,000 records).
My table has 4 fields:
id1
id2
val1
val2
For every group of records with the same "id1", I need to keep the first (lowest id2) and last (highest id2) and delete everything in between UNLESS val1 or val2 has changed from the previous (next lowest "id2") record.
If you're following me so far, what would a more efficient algorithm be? Here is my java code:
boolean bDEL=false;
qps = conn.prepareStatement("SELECT id1, id2, val1, val2 from STATUS_DATA ORDER BY id1, id2");
qrs = qps.executeQuery();
//KEEP FIRST & LAST, DISCARD EVERYTHING ELSE *EXCEPT* WHERE CHANGE IN val1 or val2
while (qrs.next()) {
thisID1 = qrs.getInt("id1");
thisID2 = qrs.getInt("id2");
thisVAL1= qrs.getInt("val1");
thisVAL2= qrs.getDouble("val2");
if (thisID1==lastID1) {
if (bDEL) { //Ensures this is not the last record
qps2 = conn2.prepareStatement("DELETE FROM STATUS_DATA where id1="+lastID1+" and id2="+lastID2);
qps2.executeUpdate();
qps2.close();
bDEL = false;
}
if (thisVAL1==lastVAL1 && thisVAL2==lastVAL2) {
bDEL = true;
}
} else if (bDEL) bDEL=false;
lastID1 = thisID1;
lastID2 = thisID2;
lastVAL1= thisVAL1;
lastVAL2= thisVAL2;
}
UPDATE 4/20/2015 # 11:10 AM
OK so here is my final solution - for every record, the Java code enters an XML record into a string which is written to file every 10,000 records and then java calls a stored procedure on SQL Server and passes the file name to read. The stored procedure can only use the file name as a variable if dynamic SQL is used to execute the openrowset. I will play around with the interval of procedure execution but so far my performance results are as follows:
BEFORE (1 record delete at a time):
73,000 records processed, 101 records per second
AFTER (bulk XML import):
1.4 Million records processed, 5800 records per second
JAVA SNIPPET:
String ts, sXML = "<DataRecords>\n";
boolean bDEL=false;
qps = conn.prepareStatement("SELECT id1, id2, val1, val2 from STATUS_DATA ORDER BY id1, id2");
qrs = qps.executeQuery();
//KEEP FIRST & LAST, DISCARD EVERYTHING ELSE *EXCEPT* WHERE CHANGE IN val1 or val2
while (qrs.next()) {
thisID1 = qrs.getInt("id1");
thisID2 = qrs.getInt("id2");
thisVAL1= qrs.getInt("val1");
thisVAL2= qrs.getDouble("val2");
if (bDEL && thisID1==lastID1) { //Ensures this is not the first or last record
sXML += "<nxtrec id1=\""+lastID1+"\" id2=\""+lastID2+"\"/>\n";
if ((i + 1) % 10000 == 0) { //Execute every 10000 records
sXML += "</DataRecords>\n"; //Close off Parent Tag
ts = String.valueOf((new java.util.Date()).getTime()); //Each XML File Uniquely Named
writeFile(sDir, "ds"+ts+".xml", sXML); //Write XML to file
conn2=dataSource.getConnection();
cs = conn2.prepareCall("EXEC SCRUB_DATA ?");
cs.setString(1, sdir + "ds"+ts+".xml");
cs.executeUpdate(); //Execute Stored Procedure
cs.close(); conn2.close();
deleteFile(SHMdirdata, "ds"+ts+".xml"); //Delete File
sXML = "<DataRecords>\n";
}
bDEL = false;
}
if (thisID1==lastID1 && thisVAL1==lastVAL1 && thisVAL2==lastVAL2) {
bDEL = true;
} else if (bDEL) bDEL=false;
} else if (bDEL) bDEL=false;
lastID1 = thisID1;
lastID2 = thisID2;
lastVAL1= thisVAL1;
lastVAL2= thisVAL2;
i++;
}
qrs.close(); qps.close(); conn.close();
sXML += "</DataRecords>\n";
ts = String.valueOf((new java.util.Date()).getTime());
writeFile(sdir, "ds"+ts+".xml", sXML);
conn2=dataSource.getConnection();
cs = conn2.prepareCall("EXEC SCRUB_DATA ?");
cs.setString(1, sdir + "ds"+ts+".xml");
cs.executeUpdate();
cs.close(); conn2.close();
deleteFile(SHMdirdata, "ds"+ts+".xml");
XML FILE OUTPUT:
<DataRecords>
<nxtrec id1="100" id2="1112"/>
<nxtrec id1="100" id2="1113"/>
<nxtrec id1="100" id2="1117"/>
<nxtrec id1="102" id2="1114"/>
...
<nxtrec id1="838" id2="1112"/>
</DataRecords>
SQL SERVER STORED PROCEDURE:
PROCEDURE [dbo].[SCRUB_DATA] #floc varchar(100) -- File Location (dir + filename) as only parameter
BEGIN
SET NOCOUNT ON;
DECLARE #sql as varchar(max);
SET #sql = '
DECLARE #XmlFile XML
SELECT #XmlFile = BulkColumn
FROM OPENROWSET(BULK ''' + #floc + ''', SINGLE_BLOB) x;
CREATE TABLE #TEMP_TABLE (id1 INT, id2 INT);
INSERT INTO #TEMP_TABLE (id1, id2)
SELECT
id1 = DataTab.value(''#id1'', ''int''),
id2 = DataTab.value(''#id2'', ''int'')
FROM
#XmlFile.nodes(''/DataRecords/nxtrec'') AS XTbl(DataTab);
delete from D
from STATUS_DATA D
inner join #TEMP_TABLE T on ( (T.id1 = D.id1) and (T.id2 = D.id2) );
';
EXEC (#sql);
END
It is almost for certain that your performance issues are not in your algorithm, but rather in the implementation. Say for example your cleanup step has to remove 10,000 records, this means you will have 10000 round trips to your database server.
Instead of doing that, write each of the id pairs to be deleted to an XML file, and send that XML file to SQL server stored proc that shreds the XML into a corresponding temp or temp_var table. Then use a single delete from (or equivalent) to delete all 10K rows.
If you don't know how to shred xml in TSQL, it is well worth the time to learn. Take a look at a simple example to get you started, out just check out a couple of search results for "tsql shred xml" to get going.
ADDED
Pulling 10K records to client should be < 1 second. Your Java code likewise. If you don't have the time to learn use XML as suggested, you could write a quick an dirty stored proc that accepts 10 (20, 50?) pairs of ids and delete the corresponding records from within the stored proc. I use the XML approach regularly to "batch" stuff from the client. If your batches are "large", you might take a look at using the BULK INSERT command on SQL Server -- but the XML is easy and a bit more flexible as it can contain nested data structures. E.g., master/detail relationships.
ADDED
I just did this locally
create table #tmp
(
id int not null
primary key(id)
)
GO
insert #tmp (id)
select 4
union
select 5
GO
-- now has two rows #tmp
delete from L
from TaskList L
inner join #tmp T on (T.id = L.taskID)
(2 row(s) affected)
-- and they are no longer in TaskList
i.e., this should not be a problem unless you are doing it wrong somehow. Are you creating the temp table and then attempting to use it in different databases connections/sessions. If the sessions are different, the temp table won't be seen in the 2nd session.
Hard to think of another way for this to be wrong off the top of my head.
Have you considered doing something that pushes more of the calculating to SQL instead of java?
This is ugly and doesn't take into account your "value changing" part, but it could be a lot faster:
(This deletes everything except the highest and lowest id2 for each id1)
select * into #temp
FROM (SELECT ROW_NUMBER() OVER (PARTITION BY id1 ORDER BY id2) AS 'RowNo',
* from myTable)x
delete from myTable i
left outer join
(select t.* from #temp t
left outer join (select id1, max(rowNo) rowNo from #temp group by id1) x
on x.id1 = t.id1 and x.rowNo = t.RowNo
where t.RowNo != 1 and x.rowNo is null)z
on z.id2 = i.id2 and z.id1 = i.id1
where z.id1 is not null
Never underestimate the power of SQL =)
Although I understand this seems more 'straightforward' to implement in a row-by-row fashion, doing it 'set-based' will make it fly.
Some code to create test-data:
SET NOCOUNT ON
IF OBJECT_ID('mySTATUS_DATA') IS NOT NULL DROP TABLE mySTATUS_DATA
GO
CREATE TABLE mySTATUS_DATA (id1 int NOT NULL,
id2 int NOT NULL PRIMARY KEY (id1, id2),
val1 varchar(100) NOT NULL,
val2 varchar(100) NOT NULL)
GO
DECLARE #counter int,
#id1 int,
#id2 int,
#val1 varchar(100),
#val2 varchar(100)
SELECT #counter = 100000,
#id1 = 1,
#id2 = 1,
#val1 = 'abc',
#val2 = '123456'
BEGIN TRANSACTION
WHILE #counter > 0
BEGIN
INSERT mySTATUS_DATA (id1, id2, val1, val2)
VALUES (#id1, #id2, #val1, #val2)
SELECT #counter = #counter - 1
SELECT #id2 = #id2 + 1
SELECT #id1 = #id1 + 1, #id2 = 1 WHERE Rand() > 0.8
SELECT #val1 = SubString(convert(varchar(100), NewID()), 0, 9) WHERE Rand() > 0.90
SELECT #val2 = SubString(convert(varchar(100), NewID()), 0, 9) WHERE Rand() > 0.90
if #counter % 1000 = 0
BEGIN
COMMIT TRANSACTION
BEGIN TRANSACTION
END
END
COMMIT TRANSACTION
SELECT top 1000 * FROM mySTATUS_DATA
SELECT COUNT(*) FROM mySTATUS_DATA
And here the code to do the actual scrubbing. Mind that the why column is there merely for educational purposes. If you're going to put this in production I'd advice to put it into comments as it only slows down the operations. Also, you could combine the checks on val1 and val2 in 1 single update... in fact, with a bit of effort you probably can combine everything into 1 single DELETE statement. However, I very much doubt it would make things much faster... but it surely would make things a lot less readable.
Anyway, when I run this on my laptop for 100k records it takes a only 5 seconds so I doubt performance is going to be an issue.
IF OBJECT_ID('tempdb..#working') IS NOT NULL DROP TABLE #working
GO
-- create copy of table
SELECT id1, id2, id2_seqnr = ROW_NUMBER() OVER (PARTITION BY id1 ORDER BY id2),
val1, val2,
keep_this_record = Convert(bit, 0),
why = Convert(varchar(500), NULL)
INTO #working
FROM STATUS_DATA
WHERE 1 = 2
-- load records
INSERT #working (id1, id2, id2_seqnr, val1, val2, keep_this_record, why)
SELECT id1, id2, id2_seqnr = ROW_NUMBER() OVER (PARTITION BY id1 ORDER BY id2),
val1, val2,
keep_this_record = Convert(bit, 0),
why = ''
FROM STATUS_DATA
-- index
CREATE UNIQUE CLUSTERED INDEX uq0 ON #working (id1, id2_seqnr)
-- make sure we keep the first record of each id1
UPDATE upd
SET keep_this_record = 1,
why = upd.why + 'first id2 for id1 = ' + Convert(varchar, id1) + ','
FROM #working upd
WHERE id2_seqnr = 1 -- first in sequence
-- make sure we keep the last record of each id1
UPDATE #working
SET keep_this_record = 1,
why = upd.why + 'last id2 for id1 = ' + Convert(varchar, upd.id1) + ','
FROM #working upd
JOIN (SELECT id1, max_seqnr = MAX(id2_seqnr)
FROM #working
GROUP BY id1) mx
ON upd.id1 = mx.id1
AND upd.id2_seqnr = mx.max_seqnr
-- check if val1 has changed versus the previous record
UPDATE upd
SET keep_this_record = 1,
why = upd.why + 'val1 for ' + Convert(varchar, upd.id1) + '/' + Convert(varchar, upd.id2) + ' differs from val1 for ' + Convert(varchar, prev.id1) + '/' + Convert(varchar, prev.id2) + ','
FROM #working upd
JOIN #working prev
ON prev.id1 = upd.id1
AND prev.id2_seqnr = upd.id2_seqnr - 1
AND prev.val1 <> upd.val1
-- check if val1 has changed versus the previous record
UPDATE upd
SET keep_this_record = 1,
why = upd.why + 'val2 for ' + Convert(varchar, upd.id1) + '/' + Convert(varchar, upd.id2) + ' differs from val2 for ' + Convert(varchar, prev.id1) + '/' + Convert(varchar, prev.id2) + ','
FROM #working upd
JOIN #working prev
ON prev.id1 = upd.id1
AND prev.id2_seqnr = upd.id2_seqnr - 1
AND prev.val2 <> upd.val2
-- delete those records we do not want to keep
DELETE del
FROM STATUS_DATA del
JOIN #working w
ON w.id1 = del.id1
AND w.id2 = del.id2
AND w.keep_this_record = 0
-- some info
SELECT TOP 500 * FROM #working ORDER BY id1, id2
SELECT TOP 500 * FROM STATUS_DATA ORDER BY id1, id2
Related
In Android/Java, I am trying to compute and store the difference between 2 values in sqlite when one value is entered.
My table looks like this:
When weights are added/stored in the table in the column 'Weight', the column 'Diff_Weight' shall receive "Weight(N) - Weight(N-1)". Example: the last cell of Diff_Weight (row 6) = 88.0 - 55.2 = 32.8. // Row 5 shall get '-0.7' etc. Their type is REAL (col Weight & col Diff_Weight).
This 32.8 should be calculated and added at the same time when 88.0 is added to the table.
So far, I have read lots of tutorials and can't figure how to proceed. (My code to create and insert in the DB is fine, but reading is somehow more complex).
My code to read the entry is very bad because I don't see how to set it up:
public Bouble getData() {
String selectQuery= "SELECT * FROM " + TABLE_NAME2 + " ORDER BY COL_4 DESC LIMIT 1";
SQLiteDatabase db = this.getWritableDatabase();
Cursor cursor = db.rawQuery(TABLE_NAME2, null);
result2 = Double.valueOf(cursor.getString(cursor.getColumnIndex("Weight")));
result1 = Double.valueOf(cursor.getString(cursor.getColumnIndex("Weight")-1));
insertdata(result2-result1); //insert in row 6 of Diff_Weight
return
}
Can anybody help there?
If that is unclear, I was needing some help for the sqlite command AND the java to get the difference result.
Simplistically you can get the data by joining to the same table
SELECT a.id, a.weight, b.weight, (b.weight - a.weight) FROM TABLE_NAME2 a
join TABLE_NAME2 b on (b.id = a.id + 1);
One way is to use the lag() window function to get the value of the previous row (As ordered by id; using timestamps would be better but between splitting up the date and time into different columns and not using a date format that can be meaningfully sorted, this is easier.):
SELECT id, weight,
round(coalesce(weight - lag(weight, 1) OVER (ORDER BY id), weight), 1) AS diff_weight
FROM example
ORDER BY id
which gives
id weight diff_weight
---------- ---------- -----------
1 22.0 22.0
2 22.2 0.2
3 55.0 32.8
4 55.9 0.9
5 55.2 -0.7
6 88.0 32.8
You can make a view of this query use that like a normal table if you like. Generating the differences dynamically like this has the advantage that if an existing weight value changes, everything that depends on it doesn't have to be updated.
ok, after a long search, here is a possible result (sqlite + java):
first, you need to query the last row of the table...
...and handle the case if there is no row in your table (blank or new table)
then you must query the 'Weight' value from the known column 'Weight' (Y) and the row with ID you already have (X)
and when you have your value (last_weight), you need to write the difference (weight-last_weight) in the column 'Diff_Weight'.
Here is the code:
SQLiteDatabase db = this.getWritableDatabase();
//query the last row of the table
Cursor cursor = db.rawQuery("SELECT ID FROM TABLE_NAME2 ORDER BY ID DESC LIMIT 1", null);
cursor.moveToLast();
int lastID;
//handle the case if there is no row in your table
try {
lastID = cursor.getInt(0);
} catch (Exception e) {
lastID = 0;
}
Double lastWeight = 0.0;
//query the 'Weight' value
if (lastID >= 1) {
Cursor cursor2 = db.rawQuery("SELECT Weight FROM TABLE_NAME2 WHERE ID=" + lastID, null);
if (cursor2.moveToFirst()) { //this is boundary otherwise 'lastWeight' doesn't get the value
lastWeight= cursor2.getDouble(0);
}
} else {
lastWeight = 0.0;
}
//write the difference in the Diff_Weight column (=COL_5)
ContentValues cValues2 = new ContentValues();
//add your data in COL_1 to COL_4 here...
cValues2.put(COL_5, weight - lastWeight);
long id2 = db.insert(TABLE_NAME2, null, cValues2);
... And so you get the red figures in the column Diff_Weight from the table photo in the question.
I am trying to implement sqlparser and using gsqlparser from here. The source of the jar is in Java but I am implementing the same in Scala.
Below is my query which contains a join condition.
SELECT e.last_name AS name, e.commission_pct comm, e.salary * 12 "Annual Salary" FROM scott.employees AS e right join scott.companies as c on c.orgid = e.orgid and c.orgname = e.orgn WHERE e.salary > 1000 ORDER BY e.first_name, e.last_name
I was able to parse the query to read names & aliases of columns, where conditions, table names (checking the table names directly inside the query) as below.
val sqlParser = new TGSqlParser(EDbVendor.dbvsnowflake)
sqlParser.sqltext = "SELECT e.last_name AS name, e.commission_pct comm, e.salary * 12 \"Annual Salary\" FROM scott.employees AS e right join scott.companies as c on c.orgid = e.orgid and c.orgname = e.orgn WHERE e.salary > 1000 ORDER BY e.first_name, e.last_name"
val selectStmnt = sqlParser.sqltext
println("Columns List:")
for(i <- 0 until selectStmnt.getResultColumnList.size()) {
val resCol = selectStmnt.getResultColumnList.getResultColumn(i)
println("Column: " + resCol.getExpr.toString + " Alias: " + resCol
.getAliasClause().toString)
}
Output:
Columns List:
Column: e.last_name Alias: name
Column: e.commission_pct Alias: comm
Column: e.salary * 12 Alias: "Annual Salary"
I am trying to parse the join condition and get the details inside it
for(j <- 0 until selectStmnt.getJoins.size()) {
println(selectStmnt.getJoins.getJoin(j).getTable)
}
The problem here is there is only one join condition in the query, so the size returned is 1.
Hence the output is scott.employees.
If I do it a bit different as below using getJoinItems
println("Parsing Join items")
for(j <- 0 until selectStmnt.getJoins.size()) {
println(selectStmnt.getJoins.getJoin(j).getJoinItems)
}
I get the output by cutting off the first table from the join condition as below:
scott.companies as c on c.orgid = e.orgid and c.orgname = e.orgn
The method: getJoinItems() returns a list: TJoinItemList which I thought of traversing through. But even its size is 1.
println(selectStmnt.getJoins.getJoin(j).getJoinItems.size()) -> 1
I am out of ideas now. Could anyone let me know how can I parse the query's join condition and get the table names inside the join ?
I don't have access to Snowflake dialect in GSP but I mimicked this scenario with Teradata dialect using the following query and created a sql parser.
SELECT e.last_name as name
FROM department d
RIGHT JOIN
trimmed_employee e
ON d.dept_id = e.emp_id
WHERE e.salary > 1000
ORDER BY e.first_name
Here is the Groovy code of getting both the tables department, trimmed_employee. It boils down to iterating over each join and while doing so collect the current join's items (joinItems) using curJoin.joinItems only if it is not null.
stmt.joins.asList().collect { curJoin ->
[curJoin.table] + (curJoin?.joinItems?.asList()?.collect { joinItems -> joinItems.table } ?: [])
}.flatten()
Result:
department
trimmed_employee
For this simple sql that you mentioned in my case, the following code also works.
stmt.tables.asList()
Right now I have created a sequence and a trigger to auto increment the ID value like this:
CREATE OR REPLACE TRIGGER "RTH"."TBL_USER_TRIGGER"
BEFORE INSERT ON TBL_USER
FOR EACH ROW
BEGIN
SELECT TBL_USER_SEQ.nextval
INTO :new.USR_ID
FROM dual;
END;
ALTER TRIGGER "RTH"."TBL_USER_TRIGGER" ENABLE;
Let say I have three rows:
User ID FIRSTNAME LASTNAME
====================================
1 John smith
2 James smith
3 Pat smith
When I delete the first row(1) I want the ID values to auto correct itself to correct numbers so that second row ID values becomes 1 and third row ID values becomes 2
Is it possible in oracle? or do I have do it through code as I am using Java to insert records into table when user submits a form.
Java DAO class:
public class RegistrationDAO {
public void insert(User user) {
try {
Connection con = DBConnection.getConnection();
String query = "insert into TBL_USER(USR_FIRST_NAME,USR_LST_NAME,USR_PRIMARY_EMAIL,USR_PASSWORD) values(?,?,?,?)";
PreparedStatement pst = con.prepareStatement(query);
pst.setString(1, user.getFirstName());
pst.setString(2, user.getLastName());
pst.setString(3, user.getEmail());
pst.setString(4, user.getPassword());
pst.executeUpdate();
} catch (Exception e) {
System.out.println("####Record insertion error in Registration DAO####");
System.out.println(e);
}
}
}
The simple answer is: "No, you don't want to do that." The purpose of an id is to uniquely identify each row. A sequential id also has the feature that it provides insertion order. It is not intended to change over time. Row 1 is Row 1 is Row 1.
If you want ordering, then declare the id to be the primary key and use a query such as this:
select t.*, row_number() over (order by usr_id) as seqnum
from t;
Ideally, you should not be changing USER ID values. But still if there is requirement. Then below are the steps.
Create a procedure to reset the sequence TBL_USER_SEQ. This is needed as you are resetting the USER ID values. So after resetting, whenever you insert new record, Sequence should start with proper value ( i.e. MAX USER_ID value of the table). So that there will be no gap in USER_ID
Write Delete Trigger on table which will adjust USER_ID values.
Procedure to reset sequence
CREATE OR REPLACE PROCEDURE reset_sequence_p (p_seq IN VARCHAR2, p_new_seq NUMBER)
AS
l_value NUMBER;
BEGIN
-- Select the next value of the sequence
EXECUTE IMMEDIATE 'SELECT ' || p_seq || '.NEXTVAL FROM DUAL' INTO l_value;
-- Alter the sequnce to increment by the difference
EXECUTE IMMEDIATE 'ALTER SEQUENCE ' || p_seq || ' INCREMENT BY ' || ( p_new_seq - l_value ) ;
-- Increment to next value after diference
EXECUTE IMMEDIATE 'SELECT ' || p_seq || '.NEXTVAL FROM DUAL' INTO l_value;
-- Set the increment back to 1
EXECUTE IMMEDIATE 'ALTER SEQUENCE ' || p_seq || ' INCREMENT BY ' || 1 ;
END reset_sequence_p;
Delete Trigger
CREATE OR REPLACE TRIGGER "RTH"."TBL_USER_TRIG_DEL"
BEFORE DELETE
ON TBL_USER
FOR EACH ROW
DECLARE
v_new_user_id_seq NUMBER;
PRAGMA AUTONOMOUS_TRANSACTION;
BEGIN
-- Update USER_ID value for all remaining records greater than USER_ID that got deleted.
UPDATE TBL_USER
SET USER_ID = USER_ID - 1
WHERE USER_ID > :old.USER_ID;
-- Retrieve max USER_ID available in table.
SELECT MAX (user_id) INTO v_new_user_id_seq FROM TBL_USER;
-- Call procedure to reset sequence
reset_sequence_p ('TBL_USER_SEQ', v_new_user_id_seq);
COMMIT;
END;
NOTE : Make sure that Primary key on table is disabled before execution of Delete statement.
I hope this helps.
I have big SQL script with creating temp tables and inserting data in them from selecting data from another tables, many inserts, like
SET #EndDate = DATEADD(dd, -1, #EndDate);
DECLARE #EndDatePlusOneDay SMALLDATETIME
SET #EndDatePlusOneDay = DATEADD(dd, 1, #EndDate);
CREATE TABLE #cntrs
(
ContractID DSIDENTIFIER,
ContractDateFrom dsdatetime,
ContractNumber dsvarfullname40,
ContractClientID DSIDENTIFIER,
ContractClientName VARCHAR(500),
CreditDateTo dsdatetime,
Amount dsmoney,
LoanDept dsmoney,
PledgeRate dsmoney,
CollatVal dsmoney,
WarrantyType dsfullname,
WarrantyNumber VARCHAR(20),
WarrantyDate dsoperday,
WarrantyQty dsmoney,
)
INSERT INTO #cntrs
SELECT c.ContractID,
cc.CreditDateFrom,
c.Number,
ti.InstitutionID,
(
CASE
WHEN ti.PropDealPart = 0 THEN ti.Name + ' ' + ti.Name1 + ' ' +
ti.Name2 + ' '
ELSE ti.Name
END
) AS ContractClientName,
cc.CreditDateTo,
c.Amount,
0 AS LoanDept,
70 AS PledgeRate,
0 AS CollatVal,
'' AS WarrantyType,
'' AS WarrantyNumber,
'19000101' AS WarrantyDate,
0 AS WarrantyQty
FROM dataTable1 c(NOLOCK)
INNER JOIN dataTable2 cc(NOLOCK)
ON c.ContractID = cc.ContractCreditID
INNER JOIN dataTable3 o(NOLOCK)
ON c.ContractID = o.ID
AND o.ObjectTypeID = 105
INNER JOIN dataTable4 p(NOLOCK)
ON o.CurrProtocolID = p.ProtocolID
INNER JOIN dataTable5 t(NOLOCK)
ON p.TransitionID = t.TransitionID
INNER JOIN dataTable6 n(NOLOCK)
ON t.TargetStateID = n.NodeID
INNER JOIN dataTable7 ti WITH(NOLOCK)
ON ti.InstitutionID = c.InstitutionID
WHERE 1 = 1
-- AND #BranchID IN (0, ISNULL(NULLIF(c.BranchExtId, 0), c.BranchID))
AND n.Brief IN ('Предоставл', 'НеОплВовр', 'Завершен')
AND cc.CreditDateFrom BETWEEN #StartDate AND #endDate
ORDER BY
cc.CreditDateFrom
IF OBJECT_ID('tempdb..#AccInner') IS NOT NULL
DROP TABLE #AccInner
CREATE TABLE #AccInner
(
ContractID NUMERIC(15, 0),
ResourceID NUMERIC(15, 0)
)
CREATE UNIQUE INDEX x1 ON #AccInner(ContractID)
DECLARE #DepParentID DSIDENTIFIER, -- Субконто КатегорияСредств
#DepRepaymentID DSIDENTIFIER, -- Субконто ТипОперВУ - Упл/Погаш
#DepAccrualID DSIDENTIFIER -- Субконто ТипОперВУ - Выд/Нач
SELECT #DepParentID = d.DepartmentID
FROM tDepartment d(NOLOCK INDEX = XAK3tDepartment)
WHERE d.Brief = 'КатСрдств'
UPDATE c
SET c.CollatVal = c.LoanDept * (c.PledgeRate / 100)
FROM #cntrs c
SELECT *
FROM #cntrs
ORDER BY
ContractDateFrom
I need execute this T-SQL code in one batch query. I plan to read T-SQL code from the file to String and execute this String at once.
How I can do this with JDBC for MS SQL Server?
You can always save this T-SQL code as MSSQL stored procedure and execute it using JDBC CallableStatement.
I 'm using Hibernate connecting to mysql as database layer, the weird thing is for some reason
the result sets executed from Hibernate api java code are different from those executed directly from mysql. They are not a part of each other, they just look like no relationship.
Here's the Java code at Dao layer:
int totalPage = 0;
int reminder = total % pagination.getPageSize();
if(total == 0){
totalPage = 1;
pagination.setTotalPage(totalPage);
}else{
totalPage = reminder == 0 ? total / pagination.getPageSize() : (total - reminder) / pagination.getPageSize() + 1;
pagination.setTotalPage(totalPage);
}
pagination.setTotalRows(total);
pagination.setBeginAndEndPage(total);
final StringBuilder queryString = new StringBuilder();
queryString.append(" select a.target_id as itemId, a.isFree as isFree, ");
queryString.append(" ni.fullName as item, nc.fullName as category, a.downloadCount as downloadCounts from( ");
queryString.append(" select dldc.current_category_id, dldc.target_id, dldc.isFree, ");
queryString.append(" sum(dldc.counts) as downloadCount from download_log_day_count dldc ");
queryString.append(" group by dldc.current_category_id, dldc.target_id) as a ");
queryString.append(" left join item i on a.target_id = i.objectId ");
queryString.append(" left join name ni on i.nameId = ni.objectId ");
queryString.append(" left join category c on a.current_category_id = c.objectId ");
queryString.append(" left join name nc on c.nameId = nc.objectId ");
queryString.append(" order by downloadCounts desc");
List<ItemReportVO> reportList = (List<ItemReportVO>)getJpaTemplate().execute(new JpaCallback(){
#Override
public Object doInJpa(EntityManager em)
throws PersistenceException {
SQLQuery query = ((Session)em.getDelegate()).createSQLQuery(queryString.toString());
query.addScalar("itemId", Hibernate.LONG);
query.addScalar("isFree", Hibernate.BOOLEAN);
query.addScalar("item", Hibernate.STRING);
query.addScalar("category", Hibernate.STRING);
query.addScalar("downloadCounts", Hibernate.LONG);
query.setResultTransformer(Transformers.aliasToBean(ItemReportVO.class));
int firstResult = (pagination.getCurrentPage()-1) * pagination.getPageSize();
query.setFirstResult(firstResult);
if(token != null && "all".equals(token)){
int maxResult = pagination.getPageSize();
query.setMaxResults( maxResult );
}else{
int maxResult = pagination.getPageSize() > Integer.valueOf(token) ? Integer.valueOf(token) : pagination.getPageSize();
query.setMaxResults( maxResult );
}
return query.list();
}
});
When I set show_sql=true in Hibernate's config file, Hibernate prints the sql query below when Java code runs "return query.list();":
select a.target_id as itemId, a.isFree as isFree, ni.fullName as item, nc.fullName as category, a.downloadCount as downloadCounts
from(
select dldc.current_category_id, dldc.target_id, dldc.isFree, sum(dldc.counts) as downloadCount
from download_log_day_count dldc
group by dldc.current_category_id, dldc.target_id) as a
left join item i on a.target_id = i.objectId
left join name ni on i.nameId = ni.objectId
left join category c on a.current_category_id = c.objectId
left join name nc on c.nameId = nc.objectId
order by downloadCounts desc
limit ?
I always consider they should be the same, but it seems they are not, anybody can help me to adjust the java codes?
Hibernate will alter your query to apply limit that you coded via API setMaxResults(). It should also add rownum> for setFirstResult() but, in your case, it may be very first query.
You don't have to change anything as Hibernate applies those in database specific and correct way.
Is there any other difference that concerns you?