I have a table named CUSTOMERS with the following columns :
CUSTOMER_ID (NUMBER), DAY(DATE), REGISTERED_TO(NUMBER)
There are more columns in the table but it is irrelevant to my question as only the above columns are defined together as the primary key
In our application we do a large amount of inserts into this table so we do not use MERGE but use the following statement :
INSERT INTO CUSTOMERS (CUSTOMER_ID , DAY, REGISTERED_TO)
SELECT ?, ?, ?
FROM DUAL WHERE NOT EXISTS
(SELECT NULL
FROM CUSTOMERS
WHERE CUSTOMER_ID = ?
AND DAY = ?
AND REGISTERED_TO = ?
)";
We use a PreparedStatement object using the batch feature to insert a large number of records collected through the flow of the application per customer.
Problem is that sometimes I get the following error :
ORA-00001: unique constraint (CUSTOMERS_PK)
violated
Strange thing is that when I do NOT use batch inserts and insert each record one by one (by simply executing pstmt.execute()) there are no errors.
Is it something wrong with the insert statement ? the jdbc driver ? Am I not using the batch mechanism correctly ?
Here is a semi-pseudo-code of my insertion loop :
pstmt = conn.prepareStatement(statement);
pstmt.setQueryTimeout(90);
for each customer :
- pstmt.setObject(1, customer id);
- pstmt.setObject(2, current day);
- pstmt.setObject(3, registered to);
- pstmt.addBatch();
end for
pstmt.executeBatch();
It is all enclosed in a try/catch/finally block making sure the statement and connection are closed at the end of this process.
I guess you are using several threads or processes in parallel, each doing inserts. In this case, Oracle's transaction isolation feature defeats your attempt to do the merge, because sometimes the following is bound to happen:
session A runs your statement, inserts a row (x,y,z)
session B runs the same statement, tries to insert row (x,y,z), gets a lock and waits
session A commits
session B receives the "unique constraint violated" error
That's because until session A commits, session B doesn't see the new row, so it tries to insert the same.
Related
I am inserting data into a teradata table using executeBatch method. Currently if one insert in the batch fails all the other inserts in the batch also fails and no records end up being inserted. How can I change this behaviour to let the other inserts in the batch succeed if any inserts fails and the some ability to track the rejected records.
PS: I have ensured that TMODE is set to TERA and autocommit enabled.
UPDATE:
target table definition.
CREATE SET TABLE mydb.mytable ,NO FALLBACK ,
NO BEFORE JOURNAL,
NO AFTER JOURNAL,
CHECKSUM = DEFAULT,
DEFAULT MERGEBLOCKRATIO
(
col1 INTEGER,
col2 VARCHAR(10) CHARACTER SET LATIN NOT CASESPECIFIC NOT NULL)
PRIMARY INDEX ( col1 );
Below is the sample scala code. As you can see, this batch contains 5 insert statements. The First insert is set to fail because it is trying to insert null into an not null field (col2). The other 4 inserts dont have any issues and should succeed. But as you can see from below all 5 inserts in the batch failed. Is there any way we can make other inserts succeed?. As stated above tmode is tera and autocommit is enabled. if there is no way other than re-submitting all failed queries individually then we would have to reduce the batch size and settle for lower throughput.
Class.forName("com.teradata.jdbc.TeraDriver");
val conn = DriverManager.getConnection("jdbc:teradata://teradata-server/mydb,tmode=TERA","username","password")
val insertSQL = "INSERT INTO mydb.mytable VALUES (?,?)"
val stmt = conn.prepareStatement(insertSQL)
stmt.setInt(1,1)
stmt.setNull(2,Types.VARCHAR) // Inserting Null here. This insert will fail
stmt.addBatch()
stmt.setInt(1,2)
stmt.setString(2,"XXX")
stmt.addBatch()
stmt.setInt(1,3)
stmt.setString(2,"YYY")
stmt.addBatch()
stmt.setInt(1,4)
stmt.setString(2,"ZZZ")
stmt.addBatch()
stmt.setInt(1,5)
stmt.setString(2,"ABC")
stmt.addBatch()
try {
val res = stmt.executeBatch()
println(res.mkString(","))
}
catch {
case th: BatchUpdateException => {
println(th.getUpdateCounts().mkString(","))
}
}
Result
-3,-3,-3,-3,-3
This is from Teradata's JDBC manual:
Beginning with Teradata Database 13.10 and Teradata JDBC Driver
13.00.00.16, PreparedStatement batch execution can return individual success and error conditions for each parameter set.
An application using the PreparedStatement executeBatch method must
have a catch-block for BatchUpdateException and the application must
examine the error code returned by the BatchUpdateException
getErrorCode method.
PreparedStatement BatchUpdateException Handling
Execute a multi-statement request using a PreparedStatement batch request and demonstrates the handling of the PreparedStatement BatchUpdateException
I have two table Order1 (OrderNO is PK in Order1)and order2 (OrderNO is FK in order2 ) in MS Access , i have to insert data in this two table using JDBC. so please can any one tell me the sol. i try it but data is inserting in only 1 st table and gives error INSERT INTO is wrong
You can use batch update facility of JDBC 2.0 in order to insert into multiple tables as a batch or as a single unit. In this case your application will hit underlying database(MS access in your case) only once so performance will be increased as compared to one by one insertion.
You can add below given code in your own..
or simply can get idea that how to implement.
Statement stmt = con.createStatement();
con.setAutoCommit(false);
stmt.addBatch("INSERT INTO Order1 VALUES (OrderNO , ..., ...)");
stmt.addBatch("INSERT INTO Order2 VALUES (OrderNO , ...)");
int [] updateCounts = stmt.executeBatch();
Here AutoCommit() is set to false so it will free your application to decide whether to commit or not if any one of command in batch fails to execute or in case of any other error.
I'm asking myself if it is possible to SELECT with LAST_INSERT_ID() in WHERE Clause after an batch of INSERTs without getting corrupt data in the tables? I'm thinking of the scenario that multiple users doing the same stuff at the same time. I develop an JSF Application in which this scenario can be possible.
In hard Code my SELECT after INSERTs looks like this:
preparedstatement.addBatch(
"INSERT INTO table1(all the FIELDS)"
+ "VALUES(null, ...);"
);
preparedstatement.addBatch(
"INSERT INTO table2(all the FIELDS)"
+ "VALUES(null, LAST_INSERT_ID(), ...);"
);
preparedstatement = connect.prepareStatement(
"SELECT id FROM table3 WHERE id = LAST_INSERT_ID();"
);
preparedstatement.executeBatch();
resultSet = preparedstatement.executeQuery();
Get I problems with this implementation or is there an better way?
Best Regards
You should be fine, quoting MySQL's documentation:
The ID that was generated is maintained in the server on a
per-connection basis. This means that the value returned by the
function to a given client is the first AUTO_INCREMENT value generated
for most recent statement affecting an AUTO_INCREMENT column by that
client. This value cannot be affected by other clients, even if they
generate AUTO_INCREMENT values of their own. This behavior ensures
that each client can retrieve its own ID without concern for the
activity of other clients, and without the need for locks or
transactions.
MySQL Last_insert_id
I have a store program in java and a database made in access. I already have 2 tables in my database which are the customers table and the products table.
I want to add an orders table wherein it's primary key is an autonumber and an order_line table to complete this app. I want to have tables like this..
customer(cust_id, name, ....)
orders(order_no, cust_id, date_purchased,...)
order_line(order_no, product_id, ...)
products(product_id, product_name, price,....)
When the customer purchased the products, i could insert new values to the orders table. The thing that is not clear to me is how could i insert also in the order_line table, because the order_no I created in access is of type autonumber.
Would I make a select statement first to get the order_no value to put it to the order_no in order_line's table? Or I need to put this in one query only.
Anyone with experience to this? Any advice is appreciated.
The insertion into orders and order_line table should happen in a single transaction. While doing so, if you are using plain JDBC to insert record into orders table, you can register the order_no as an OUT parameter in your CallableStatement and get the value after the statement is executed and use to set the order_no attribute on the order_line records.
// begin transaction
connection.setAutoCommit(false);
CallableStatement cs = connection.prepareCall(INSERT_STMT_INTO_ORDERS_TABLE);
cs.registerOutParameter(1, Types.INT);
int updateCount = cs.execute();
// Check the update count.
long orderNo = cs.getInt(1);
// CallableStatement csLine for inserting into order_line table
// for (OrderLine line: orderLines) {
// Set the orderNo in line.
// set paramters on csLine.
// csLine.addBatch();
// }
// run the batch and verify update counts
connection.commit();
// connection.rollback() on error.
The JDBC-way (if you like database-independence), is to use the getGeneratedKeys() method of statement.
Use setAutoCommit(false), then execute the first query with the option Statement.RETURN_GENERATED_KEYS (eg for PreparedStatement).
Then use the getGeneratedKeys() method to retrieve the key (note: reference by column name, as the exact implementation and number of returned columns depends on the driver implementation.
And execute the second statement with that retrieved key.
Finally, commit().
I have Oracle 10gR2 database with IOT table within:
create table countries (
id number primary key,
name varchar2(30) not null enable
) organization index;
I try to update table values with this Java (version 1.6) code:
Statement stmt = conn.createStatement(ResultSet.TYPE_FORWARD_ONLY,
ResultSet.CONCUR_UPDATABLE);
ResultSet src = stmt.executeQuery("select id, name from countries");
src.next();
src.updateString("name", "__test__");
src.updateRow();
But updateRow throws SQLException (ORA-01410: invalid ROWID). If I try to update a heap (ordinary) table - all works.
I have use this code with different versions of oracle drivers (from here http://www.oracle.com/technology/software/tech/java/sqlj_jdbc/htdocs/jdbc_10201.html)
After some research I have detected that IOT and HEAP table have different format of rowids:
IOT example *BAJzKgwCwRb+
HEAP example AAAbgVAAJAAMyr8AAA
But I still don't know how to solve this problem. Are you have any ideas?
Can you get the results of an extended SQL trace of your query to see what JDBC's doing under the covers? I suspect it's attempting to do
UPDATE COUNTRIES SET NAME = '__TEST__' WHERE ROWID = :rowid_fetched
and ROWID means something totally different in Oracle IOT's; it's not the immutable address of a row, but a guess as to the path to the row.
My recommendation as to how to do this is to propagate a system-generated timestamp field onto all of your tables, and use that for concurrency control rather than declaring an updatable recordset -- which will take and hold locks for every record in the recordset.
Then your application would fetch the rowset as normal, but issue statements like:
UPDATE COUNTRIES SET NAME = '__TEST__' WHERE MOD_TS = :mod_ts_fetched
to give stateless optimistic locking.
Looks like your table does not really need to be IOT. I would suggest you recreate it as a normal table and add index on both ID and name. Same performance, same logic, no ROWID problem.