Develop Reference

Huge Update or Inserts into DB2 using JAVA and JPA 1.0

I have a JAVA requirement where i have 1500 records that I have to update or insert into the database.
If a record exists with userId, then update it.
If a record does not exist with userId, then Insert it.
And, if there is an error in lets say, 10th record,,,I need to get
the error code for that record.
It looks like I have 2 options using JPA 1.0
A) Fire a select to check if record exists. If yes, then fire update. If not, fire insert.
B) Fire an insert always,,,but i get an uniqe record exception, only then fire an update query..
Are there any other more efficient ways ? how can this be done with as few queries and as quick as possible ?
ENV- JAVA, JPA 1.0, DB2

You did not specify which version of DB2 you use and on which system. Anyway, check if MERGE statement is available on your DB:
LUW from 9.5.0: http://www.ibm.com/support/knowledgecenter/SSEPGG_9.5.0/com.ibm.db2.luw.sql.ref.doc/doc/r0010873.html
Z/OS from 10.0.0: http://www.ibm.com/support/knowledgecenter/SSEPEK_10.0.0/sqlref/src/tpc/db2z_sql_merge.html
Another way is to do delete + insert on every record (poor performance).
Third option is to create dynamic one delete statement with listed ID/KEY in where clause from data you are going to update, fire delete and then insert all data.
Performance of every option will depend on table specification, indexes etc.

you can write query in mysql as below
//suppose a as pk
INSERT INTO table (a,b,c) VALUES (1,2,3)
ON DUPLICATE KEY UPDATE c=c+1,b=b+1;
here update will run when record with pk as a=1 is already present
refer below link http://dev.mysql.com/doc/refman/5.7/en/insert-on-duplicate.html

Increasing INSERT speed

I'm currently using the following query to insert into a table only if the record does not already exist, presumably this leads to a table scan. It inserts 28000 records in 10 minutes:
INSERT INTO tblExample(column)
(SELECT ? FROM tblExample WHERE column=? HAVING COUNT(*)=0)
If I change the query to the following, I can insert 98000 records in 10 minutes:
INSERT INTO tblExample(column) VALUES (?)
But it will not be checking whether the record already exists.
Could anyone suggest another way of querying such that my insert speed is faster?

One simple solution (but not recommended) could be to simply have insert statement, catch duplicate key exception and log them. Assuming that the table has unique key constraint.

Make sure that you have an index on the column[s] you're checking. In general, have a look at the query execution plan that the database is using - this should tell you where the time is going, and so what to do about it.
For Derby db this is how you get a plan and how to read it.
Derby also has a merge command, which can act as insert-if-not-there. I've not used it myself, so you'd need to test it to see if it's faster for your circumstances.

Refreshing PrimaryID to start from one after a deleted Row

Im programming a program in java and i have a database in a JTable just like the ones below. I wanted to know if it is possible to refresh the primaryID location from 1 on the GUI interface form one when a row is deleted? for example below the LoactionID is deleted for London and added again with an id 4. Is this possible?
Im using SQL in java

To answer your question, yes it is possible.
There is no good reason for you to do this though, and I highly recommend you don't do this.
The only reason to do this would be for cosmetic ones - the database doesn't care if records are sequential, only that they relate to one another consistently. There's no need to "correct" the values for the database's sake.
If you use these Id's for some kind of numbering on the UI (cosmetic reason):
Do not use your identity for this. Separate the visual row number, order or anything else from the internal database key.
If you REALLY want to do this,
Google "reseeding or resetting auto increment primary ID's" for your sql product.
Be aware for some solutions if you reset the identity seed below values that you currently have in the table, that you will violate the indentity column's uniqueness constraint as soon as the values start to overlap
Thanks Andriy for mentioning my blindly pasting a mysql solution :)
Some examples:
ALTER TABLE table_name ALTER COLUMN auto_increment_column_name RESTART WITH 8 Java DB
DBCC CHECKIDENT (mytable, RESEED, 0)
Altering the sequence

Possible in JDBC / MySQL to retrieve updated rows?

preparedStatement.executeUpdate()
Returns the number of rows updated. To my research so far it's not possible to do an update-query in which you would retrieve the updated rows, but this seems like such a basic feature that I'm clearly missing something. How to accomplish this?

Per first comment on question this is simply not possible in MySQL. PostgreSQL supports UPDATE...RETURNING as this feature.

If you use executeQuery instead of executeUpdate, you get a resultset back.
Then, change your stored procedure to be a function, and return the changed rows in a select at the end of the function. AFAIK, you cannot return data from a procedure in MySQL (as opposed to e.g. Microsoft SQL server).
EDIT: The suggestion struck out above is not possible. The JDBC specification does not allow updates in query statements (see the answer for this one: http://bugs.mysql.com/bug.php?id=692).
BUT, if you know the WHERE clause of the rows you are about to update, you can always select them first, to get the primary keys, perform the update, and then perform a select on them afterwards. Then you get the changed rows.

when you fire preparedStatement.executeUpdate() you already have the row identifiers using which you can uniquely identify the rows you want updated- you need to use the same identifiers to do a query and fetch the updated rows. you can not accomplish update and retrieval in one shot using JDBC apis.

database audit table

I have an existing application that I am working w/ and the customer has defined the table structure they would like for an audit log. It has the following columns:
storeNo
timeChanged
user
tableChanged
fieldChanged
BeforeValue
AfterValue
Usually I just have simple audit columns on each table that provide a userChanged, and timeChanged value. The application that will be writing to these tables is a java application, and the calls are made via jdbc, on an oracle database. The question I have is what is the best way to get the before/after values. I hate to compare objects to see what changes were made to populate this table, this is not going to be efficient. If several columns change in one update, then this new table will have several entries. Or is there a way to do this in oracle? What have others done in the past to track not only changes but changed values?

This traditionally what oracle triggers are for. Each insert or update triggers a stored procedure which has access to the "before and after" data, which you can do with as you please, such as logging the old values to an audit table. It's transparent to the application.
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:59412348055

If you use Oracle 10g or later, you can use built in auditing functions. You paid good money for the license, might as well use it.
Read more at http://www.oracle.com/technology/pub/articles/10gdba/week10_10gdba.html

"the customer has defined the table structure they would like for an audit log"
Dread words.
Here is how you would implement such a thing:
create or replace trigger emp_bur before insert on emp for each row
begin
if :new.ename = :old.ename then
insert_audit_record('EMP', 'ENAME', :old.ename, :new.ename);
end if;
if :new.sal = :old.sal then
insert_audit_record('EMP', 'SAL', :old.sal, :new.sal);
end if;
if :new.deptno = :old.deptno then
insert_audit_record('EMP', 'DEPTNO', :old.deptno, :new.deptno);
end if;
end;
/
As you can see, it involves a lot of repetition, but that is easy enough to handle, with a code generator built over the data dictionary. But there are more serious problems with this approach.
It has a sizeable overhead: an
single update which touches ten
field will generate ten insert
statements.
The BeforeValue and AfterValue
columns become problematic when we
have to handle different datatypes -
even dates and timestamps become
interesting, let alone CLOBs.
It is hard to reconstruct the state
of a record at a point in time. We
need to start with the earliest
version of the record and apply the
subsequent changes incrementally.
It is not immediately obvious how
this approach would handle INSERT
and DELETE statements.
Now, none of those objections are a problem if the customer's underlying requirement is to monitor changes to a handful of sensitive columns: EMPLOYEES.SALARY, CREDIT_CARDS.LIMIT, etc. But if the requirement is to monitor changes to every table, a "whole record" approach is better: just insert a single audit record for each row affected by the DML.

I'll ditto on triggers.
If you have to do it at the application level, I don't see how it would be possible without going through these steps:
start a transaction
SELECT FOR UPDATE of the record to be changed
for each field to be changed, pick up the old value from the record and the new value from the program logic
for each field to be changed, write an audit record
update the record
end the transaction
If there's a lot of this, I think I would be creating an update-record function to do the compares, either at a generic level or a separate function for each table.