What I have are 7 tables, one of which is a master list and the other 6 are archives. Also, the archives are on another database. I need to go through the archive tables and delete rows, by checking 3 different column IDs (top, middle, bottom) that are not listed in the master table, as they are not relevant anymore. My SQL statement is below. I don't have a way to test it yet, but since I am not very familiar with SQL I was hoping people could give a few tips.
String[] tables;
tables = new String[]{"archive1", "archive2", "archive3",
"archive4", "archive5", "archive6"};
String query;
Statement stmt;
String objs = "TOP AND MIDDLE AND BOTTOM";
while(i<tables.length){
//TODO: CONFIRM THE QUERY IS CORRECT
query = "DELETE FROM "+ tables[i] + " WHERE "+ objs +
"NOT IN(SELECT " + objs + " FROM DB.masterTable WHERE " +
objs + " IS NOT NULL)";
//IS NOT NULL may not be necessary
try{
//TODO: VERIFY CONNECTION IS CORRECT
stmt = this.DB2.createStatement();
stmt.executeUpdate(query);
}catch(SQLException x){
System.out.println("Failure in loop queries!");
}
i++;
}
}
In the tables there are many columns, but I am concerned with comparing the top, middle, and bottom IDs (together they are unique to each row, but e.g. top might have many of the same values). So like I said, if there isn't a row with the same TOP, MIDDLE, BOTTOM in the master table, OBJECTS, then that row can be deleted from the archive that has it listed. I tried to put everything in one query but maybe I need multiple?
My main questions are:
A) Is my query correct in any sense?
B) Since the tables are on 2 different databases how should I handle that?
Solution found:
query = "DELETE FROM "+ tables[i] +
" WHERE (TOP, MIDDLE, BOTTOM) NOT IN "+
"(SELECT TOP, MIDDLE, BOTTOM FROM DB1.DB.masterTable)";
Main problem became figuring out how to compare all 3 fields of each row at a time and accessing the DB tables
A) No. There are some fundamental errors in your query. If you expand out your query, where objs = "OBJ_ID_TOP AND OBJ_ID_MIDDLE AND OBJ_ID_BOTTOM"
query = "DELETE FROM "+ tables[i]
+ " WHERE OBJ_ID_TOP AND OBJ_ID_MIDDLE AND OBJ_ID_BOTTOM
NOT IN(SELECT OBJ_ID_TOP AND OBJ_ID_MIDDLE AND OBJ_ID_BOTTOM
FROM db1Connection.OBJECTS
WHERE OBJ_ID_TOP AND OBJ_ID_MIDDLE AND OBJ_ID_BOTTOM IS NOT NULL)";
This is not proper SQL... If we take a glance at the wiki page on the WHERE clause,
The proper syntax for writing SQL Where clause is
SELECT <> FROM table WHERE column operatorvalue
Use AND and OR to string up multiple column conditions in your WHERE clause.
Also, AND is not valid in a SELECT statement. If you want to combine multiple result sets, use UNION.
Something of the form (see below) is closer to what you need. (Note: it is not optimized by any means... just a demonstration)
EDIT Think I originally misunderstood what you were trying to do... but I think you want to delete from some_table, not master_table.
--DELETE --can swap out SELECT for DELETE when the selected results look right
SELECT s.*
FROM some_table s --this is table[i]
LEFT OUTER JOIN master_table mt --db1Connection.OBJECTS
on s.ID_TOP = mt.ID_TOP
AND s.ID_MIDDLE = mt.ID_MIDDLE
AND s.ID_BOTTOM = mt.ID_BOTTOM
WHERE mt.ID_TOP IS NULL
AND mt.ID_MIDDLE IS NULL
AND mt.ID_BOTTOM IS NULL
B) I can't help you with this question... someone with more DB2(?) chops can help you
You need to get the list of data to be deleted from the first DB.
Then go for deletion in the second DB
And, you just can't compare saying obj1 and obj2 and obj3 not in (some list), it should be like obj1 not in (somelist) and obj2 not in (somelist) and obj3 not is(whatever).
In this case, it would be a good idea to create a temp table in DB2 where the archive tables are, with data from the master table in DB1. Then, run queries like
delete from archive1 where col1 not in (select col_master from temp_table);
Related
I am trying to delete the first & last row with Java HyperAPI which i used to optimise big data into .hyper file. in tableau server. Im not sure if this possible but based on the documentation, some of the use cases i read , its possible
based on normal sql query , we can delete the first row with
delete top(1) from tablename
Script shown below is working fine which is able to remove row based on where condition
long rowCountDeletedInCustomersTable = connection.executeCommand(
"DELETE FROM " + escapeName("Customer") +
"WHERE " + escapeName("Customer Name") + "=" + escapeStringLiteral("Dennis Kane")
).getAsLong();
however when i tried to delete the first with
long rowCountDeletedInCustomersTable = connection.executeCommand(
DELETE TOP(1) from " + escapeName("Customer")).getAsLong()
or
long rowCountDeletedInCustomersTable = connection.executeCommand(
DELETE FROM table WHERE ROWNUM = 1).getAsLong()
this will throw error, im not sure on which approach should i use
I am not really sure why you want to delete the "first" or "last" row from a Hyper file. Note that Hyper has no notion of "row order", so which row is considered to be the first/last row is somewhat random...
Hyper's SQL syntax follows Postgres' conventions instead of Microsoft's SQL dialect. DELETE TOP(1) is a Microsoft-specific syntax.
Hyper doesn't have a corresponding syntax. But assuming you have a column your_row_id which forms a key for your relationship, you can use
DELETE FROM tablename
WHERE your_row_id IN (SELECT your_row_id FROM tablename LIMIT 1)
I have a program in JAVA which creates a table in a database and then I insert rows in this table.
The table is created as below:
String sql = "CREATE TABLE IF NOT EXISTS weather (\n"
+ " city string,\n"
+ " temp real,\n"
+ " feels_like real,\n"
+ " temp_min real,\n"
+ " temp_max real,\n"
+ " pressure integer,\n"
+ " humidity integer\n"
+ ");";
When I add rows, I don't want to have duplicates for the field named "city".
So, for example, if I already have London with its data, I don't want to add it again, even though that all its data may have changed. I want to have it only once in my table.
I have this query for insertion :
String sql = "INSERT INTO weather VALUES(?,?,?,?,?,?,?);";
and I want to modify it so I don't insert city duplicates.
Can anyone help me please? Thanks!
If the version of SQLite you use is 3.24.0+ and there is a unique constraint for the column city, you can use upsert which gives you an option to do NOTHING or UPDATE the table if a unique constraint violation occurs.
In this case:
String sql =
"INSERT INTO weather VALUES(?,?,?,?,?,?,?) " +
"ON CONFLICT DO NOTHING";
if you try to insert a row with an existing city, the statement will fail without an error.
But if the new row contains up to date data for the other columns and you want the row updated, you can do this:
String sql =
"INSERT INTO weather VALUES(?,?,?,?,?,?,?) " +
"ON CONFLICT(city) DO UPDATE SET "
"temp = excluded.temp, " +
"feels_like = excluded.feels_like, " +
"temp_min = excluded.temp_min, " +
"temp_max = excluded.temp_max, " +
"pressure = excluded.pressure, " +
"humidity = excluded.humidity";
and the other 6 columns will be overwritten by the new values you supplied.
If there isn't a unique constraint defined for city and you don't want to or can't define one, then you can avoid inserting the same city twice with NOT EXISTS like this:
String sql =
"INSERT INTO weather SELECT ?,?,?,?,?,?,? " +
"WHERE NOT EXISTS (SELECT 1 FROM weather WHERE city = ?);
In this case you will have to pass in your Java code as an additional 8th parameter the value of the city again.
There is no way to do this in 'effectively standard SQL'*.
But, each individual DB engine does usually have some way of accomplishing this goal.
The concept is called merging or upserting - those are terms you can search the web for. Just search for 'how to postgres upsert' for example. It's called upsert because the more general application is: If some subset of the values I am inserting doesn't exist yet in the DB, INSERT a new row with this data. Otherwise, find the row with the same values for this subset, and then update all the other values with it. For example: "Find the student with ID 12345, and then change the name to 'Joe Bloggs'. If there is no row with that, then make it".
make ALL the values 'the key' and you've reduced your 'insert, ignore if it is already there' to a standard UPSERT.
In psql, you can do ON CONFLICT. searching for 'mysql upsert' gets you to blog posts that give you varying tactics depending on your exact needs, from using INSERT IGNORE (I advice against this, this ignores any and all errors, vs only ignoring the 'already in here' error only), ON DUPLICATE KEY UPDATE (better idea, this), or using REPLACE.
Similar blog posts will be found for any db engine you are using here.
*) Defined not as 'as per some version of the SQL standard', but defined as: 'works in the majority of existing DB engines'.
You can now use on conflict. This requires that you have a unique index/constraint on city but it is already defined as the primary key. Check.
insert into weather ( . . . )
values ( . . . )
on conflict (city) ignore;
SQLite also allows this shorthand:
insert or ignore into weather ( . . . )
values ( . . . );
I've this query with WrappedBean :
buffer.append("SELECT new myPackage.WrappedBean(E.debitant.id, E.dateCalcul, E.verse, E.incidenceReprise, E.soldeSoumission) ");
buffer.append("FROM " + getEntityClassName() + " E ");
buffer.append("LEFT JOIN E.debitant DT ");
buffer.append("WHERE E.debitant.id = :idDebitant ");
buffer.append("AND YEAR(E.dateCalcul) = :year ");
buffer.append("GROUP BY E.debitant.id");
hqlQuery = session.createQuery(buffer.toString());
hqlQuery.setInteger("idDebitant", idDebitant);
hqlQuery.setInteger("year", year);
I've created WrappedBean for returning somme columns and for using Group BY.
When i try to execute it, i obtain this error :
org.postgresql.util.PSQLException: ERREUR: The column « complement0_.date_calcul » must appear in GROUP BY clause or must be used in Select (i translate the error from french)
My POSTGRES query doesnt contain date_calcul in Group BY.
Another problem, in my query i've also this :
SUM(CASE WHEN YEAR(dateCalcul)=#PECAnnee AND verse>0 THEN verse ELSE 0 END)
I know in HQL, we cant do case when in select, for this reason, i dont add SUM to column verse
What i've forgot ?
My POSTGRES query doesnt contain date_calcul in Group BY
That's the problem and what Postgres is complaining about. Why isn't it in the SQL query? Because it isn't in the HQL query. Any column that is selected without the use of some aggregate method like sum(), min(), max() etc. needs to be part of the GROUP BY clause since otherwise the DB doesn't know how to handle multiple values/conflicts.
As an example, what value of E.dateCalcul should be passed to WrappedBean if there are multiple debitors (debitants) (which is most probably the case since otherwise there wouldn't be any need for the GROUP BY clause)?
So to fix this either use
GROUP BY E.debitant.id, E.dateCalcul, E.verse, E.incidenceReprise, E.soldeSoumission
or use aggregate functions, e.g.
WrappedBean(E.debitant.id, max(E.dateCalcul), min(E.verse), max(E.incidenceReprise), sum(E.soldeSoumission))
I'm salvaging Java code left for me from the previous employee, I was only left with the source code itself, I've had to recreate DB structure by myself. So far I fully restored it, but one line always throws exceptions at me and I can't understand why.
connBeztirazh.execSQLU("declare #ki table (ID_ki int); " +
"declare #bil table (ID_bilet int); " +
"insert into ki (i1,i2,i3,i4,i5,i6) output inserted.ID_ki into #ki values ("+i1+","+i2+","+i3+","+i4+","+i5+","+i6+"); " +
"insert into bilet(ID_ki,data) output inserted.ID_bilet into #bil values ((select ID_ki from #ki),(select SYSDATETIME())); " +
"insert into rezultat (ID_players, ID_bilet) output inserted.ID_bilet values ((select ID_players from players where telnomer='"+number+"'),(select ID_bilet from #bil));", false);
This line throws out the following exception:
com.microsoft.sqlserver.jdbc.SQLServerException: Invalid object name 'ki'.
I triple-checked everything, I recreated tables bilet, players and rezultat, they are used elsewhere in the code and everything's alright. As far as I understand SQL, ki and bil are temporary tables and since they are created and accessed in the same query there could not be any access/session problems.
In this case, I overlooked that in fact there are two tables: constant table ki and variable table ki. They are referenced by ki and #ki respectively, which is a really bad way to name your tables. After adding table ki to the database, everything started working as intended, as far as I can tell.
Thanks to everyone who commented on the post.
I am going to be given a staging table, which I run a process to load the data into a bigger table, and remove data from the staging table. Some rows in the staging table, denote deletions of rows in the main table though.
Right now the algorithms is as follows:
ResultSet dataToLoad = select * from ...;
ResultSet mainTable = select * from ...;
while (dataToLoad.next())
{
if(insert)
//insert this row into main table, and remove row from this table
else if(delete)
//Find the row that matches in the main table and delete it. remove row from this table
}
My question is for the delete block. Would it be best to make a new ResultSet to find the row using a where and just delete the single row in the set, or to put the cursor to the beginning and loop through the cursor checking conditions? Essentially like implementing a where clause in java code.
How much does JDBC cache when you load in a ResultSet? is it going to do a full network round trip for each cursor jump?
Lastly, I read up briefly on the Halloween Problem. Am I at risk of that here?
Why dont you try to load a List of inserts and deletes, for example
List deletesId = .....
Whie(...){
if(delete) {
deletesId.add(idDataForDeleting);
}
}
And then you can do something like:
String sql = "DELETE FROM TABLE WHERE ID IN (" + separateByComma(deletesId) + ")";
executeDelete(sql);
Of course, try to use PreparedStamment, I mean with this you will the delete operatabion in one step against the database, is better than one by one.