I have this scenario ,on a airline website (using Java) two separate customers send two requests at same time to book a same seat in same airline
from New York to Chicago. I am using the oracle database and isolation level is read committed.My question here is that does oracle database provide
any solution to deal with this kind of concurrent scenario? what I know is when first transaction DML statement is fired it will get a lock on affected
rows and will release when transaction completes i.e on issuing rollback or commit.But as soon as commit is done and second request will proceed as soon as
first is completed and will override the first one.So it does not help?
Yes in Java I can deal with making my db class as singleton and using synchronized keyword on method which is doing update. But want to know is there
anyway we can this kind of issue at database level itself?Probably isolation level as serializable can help. But not sure?
It will only over write if you allow it. You can try something like
UPDATE seatTable
SET seatTaken = true
WHERE .. find the seat, flight etc.. AND seatTaken = false
This will return 1 row updated the first time and 0 rows updated after that.
As you mention, transanction settings will help you achieving one operation. The best way to enforce this kind of restrictions it to ensure that your relational model is constrained not to accept the 2nd operation once the 1st one succeeds.
Instead of having to do an update on a row, say update .... seat = "taken", create a reservation table (customer, flight, seat) which has a constrain (column:seat = unique) (lookup ora docs to learn the syntax for that on table creation). That way your reservation process becomes an insert in the reservation table and you can rely on the RDBMS to enforce your relational constrains to keep your business model valid.
e.g. Let t1 be the earlier operation time, you'll have:
t1=> insert into reservations(customer1,flight-x,seat-y) // succeeds. Customer 1 reserved the seat-y
t2=> insert into reservations(customer2,flight-x,seat-y) // fails with RDBMS unique constrain violated.
The only way to reserve seat-y again is to first remove the previous reservation, which is probably what your business process wants to achieve.
To handle concurrency in a web site a common practice it to have a column on each record that allows you to check it has not been updated since you got it. Either last update date or a sequential version number (auto incremented by a trigger).
Typically you will read the data (plus the concurrency column)
SELECT seat,etc,version_no
FROM t1
WHERE column = a_value
Then when the user eventually gets round to booking the seat the update will work unless there has been an update.
(the version number or update date will change after every update)
BEGIN
UPDATE t1
SET seatTaken = true
WHERE seatid = .....
AND version_no = p_version
RETURNING version_no INTO p_version;
EXCEPTION WHEN NOT_FOUND THEN
--Generate a custom exception
--concurrency viloation the record has been updated already
END;
the trigger to auto update the version number would look a little like this
CREATE OR REPLACE TRIGGER t1_version
AFTER INSERT OR UPDATE ON t1
FOR EACH ROW
BEGIN
IF :new.version_no IS NULL THEN
:new.version_no := 0;
ELSE
:new.version_no := :old.version_no + 1;
END IF;
END;
Aside from doing everything in a single UPDATE by carefully crafting WHERE clause, you can do this:
Transaction 1:
SELECT ... FOR UPDATE exclusively locks the row for the duration of the transaction.
Check if the returned status of the row is "booked" and exit (or retry another row) if it is.
UPDATE the row and set its "status" to "booked" - it is guaranteed nobody else updated it in the meantime.
Commit. This removes the exclusive lock.
Transaction 2:
SELECT ... FOR UPDATE blocks until Transaction 1 finishes, then exclusively locks the row.
The returned status of the row is "booked" (since Transaction 1 marked it that way), so exit (or possibly retry another row).
Related
I am trying to learn JPA with hibernate and use MySQL as the db.
From my understanding,
LockModeType.OPTIMISTIC: The entity version is checked towards the end
of the currently running transaction.
REPEATABLE READ: All consistent reads within the same transaction read
the snapshot established by the first such read in that transaction
Is it true that LockModeType.OPTIMISTIC in hibernate does not work with MySQL's default isolation level?
Say I have the following code:
tx.begin();
EntityManager em = JPA.createEntityManager();
Item item = em.find(Item.class, 1, LockModeType.OPTIMISTIC);
// Assume the item here has version = 0
// Read the item fields etc, during that another transaction commits and made item version increased to version = 1
tx.commit(); // Here Hibernate should execute SELECT during flushing to check version,
// i.e SELECT version FROM Item WHERE id = 1
em.close();
What I would expect is that, during flushing, Hibernate would throw OptimisticLockException because the version of the item is no longer 0. However, due to the isolation level, in the same transaction Hibernate would still see the item in version = 0 and not triggering OptimisitcLockExcpetion.
I tried to search but seems no one raised such question before, hopefully someone can help clear my confusion on OptimisticLock.
If your question is actually is there a flaw in HBN implementation (or JPA specification) related to the following statement:
If transaction T1 calls for a lock of type LockModeType.OPTIMISTIC on
a versioned object, the entity manager must ensure that neither of the
following phenomena can occur:
P1 (Dirty read): Transaction T1 modifies a row. Another transaction T2 then reads that row and obtains the modified value, before T1 has
committed or rolled back. Transaction T2 eventually commits
successfully; it does not matter whether T1 commits or rolls back and
whether it does so before or after T2 commits.
P2 (Non-repeatable read): Transaction T1 reads a row. Another transaction T2 then modifies or deletes that row, before T1 has
committed. Both transactions eventually commit successfully.
Lock modes must always prevent the phenomena P1 and P2.
then the answer is yes, you are correct: in case when you are performing computations based on some entity state, but you are not modifying those entity state, HBN just issues select version from ... where id = ... at the end of transaction and hence it do not see changes from other transactions due to RR isolation level. However I would not say that RC isolation level performs much better for this particular case: it's behaviour more correct from technical perspective but it is completely unreliable from business perspective because it depends on timings, so just do not rely on LockModeType.OPTIMISTIC - it is unreliable by design and use another techniques like:
store data from different domains in different entities
take advantage of #OptimisticLock annotation to prevent incrementing of version when it is not required (actually this will poison you domain model by HBN annotations)
mark some properties as updatable=false and update them via JPQL update in order to prevent version increment
UPD.
Taking the P2 as example, if I really need T1 (only read row) to fail if T2 (modify/delete row) commits first, the only workaround I can think of is to use LockModeType.OPTIMISTIC_FORCE_INCREMENT. So when T1 commits it will try to update the version and fail. Can you elaborate more on how your provided 3 points at the end can help with this situation if we keep using RR isolation level?
The short story:
LockModeType.OPTIMISTIC_FORCE_INCREMENT does not seem to be a good workaround, cause it turns reader into writer, so incrementing version will fail both writers and other readers. However in your case it might be acceptable to issue LockModeType.PESSIMISTIC_READ which for some DBs translates into select ... from ... for share/lock in share mode, which in turn blocks only writer and blocks (or fails) current reader, so you will avoid the phenomenon we are talking about.
The long story:
When we have started thinking about some "business consistency" the JPA specification is not our friend anymore, the problem is they define consistency in terms of "denied phenomena" and "someone must fail", but does not give us any clues and APIs how to control the behaviour in the correct way from business perspective. Let's consider the following example:
class User {
#Id
long id;
#Version
long version;
boolean locked;
int failedAuthAttempts;
}
our goal is to lock user account when failedAuthAttempts exceeds some threshold value. The pure SQL solution for our problem is very simple and straightforward:
update user
set failed_auth_attempts = failed_auth_attempts + 1,
locked = case failed_auth_attempts + 1 >= :threshold_value then 1 else 0 end
where id = :user_id
but JPA complicates everything... at first glance our naive implementation should look like:
void onAuthFailure(long userId) {
User user = em.find(User.class, userId);
int failedAuthAttempts = user.failedAuthAttempts + 1;
user.failedAuthAttempts = failedAuthAttempts;
if (failedAuthAttempts >= thresholdValue) {
user.locked = true;
}
em.save(user);
}
but that implementation has obvious flaw: if someone actively bruteforces user account not all failed auth attempts get recorded due to concurrency (here I'm not paying attention that it might be acceptable because sooner or later we will lock user account). How to resolve such issue? May we write something like:
void onAuthFailure(long userId) {
User user = em.find(User.class, userId, LockModeType.PESSIMISTIC_WRITE);
int failedAuthAttempts = user.failedAuthAttempts + 1;
user.failedAuthAttempts = failedAuthAttempts;
if (failedAuthAttempts >= thresholdValue) {
user.locked = true;
}
em.save(user);
}
? Actually no. The problem is for entities which are not present in persistence context (i.e. "unknown entities") hibernate issues select ... from ... where id=:id for update, but for known entities it issues select ... from ... where id=:id and version=:version for update and obviously fails due to version mismatch. So we have following tricky options to make our code to work "correctly":
spawn another transaction (I believe in most cases it is not a good option)
lock entity via select query, i.e. smth. like em.createQuery("select id from user where id=:id").setLockMode(LockModeType.PESSIMISTIC_WRITE).getFirstResult() (I believe that may not work in RR mode, moreover following refresh call looses data)
mark properties as non-updatable and update them via JPQL update (pure SQL solution)
Now let's pretend we need to add another business data into our User entity, say "SO reputation", how are we supposed to update new field keeping in mind that someone might bruteforce our user? The options are following:
continue to write "tricky code" (actually that might lead us to the counterintuitive idea that we always need to lock entity before updating it)
split data from different domains across different entities (sounds counterintuitive too)
use mixed techniques
I do believe this UPD will not help you much, however it's purpose was to demonstrate that it does not worth to discuss consistency in JPA domain without knowledge about target model.
To understand this, let's have a quick look on how hibernates optimistic locking works:
1: begin a new transaction
2: find an entity by ID (hibernate issues a SELECT ... WHERE id=xxx;), which e.g. could have a version count of 1
3: modify the entity
4: flush the changes to the DB (e.g. triggered automatically before committing a transaction):
4.1: hibernate issues an UPDATE ... SET ..., version=2 WHERE id=xxx AND version=1 which returns the number of updated rows
4.2: hibernate checks whether there was one row actually updated, throwing a StaleStateException if not
5: commit the transaction / rollback in case of the exception
With the repeatable_read isolation level, the first SELECT establishes the state (snapshot) which subsequent SELECTs of the same transaction read. However, the key here is that the UPDATE does not operate on the established snapshot, but on the committed state of the row (which might have been changed by other committed transactions in the meantime).
Therefore the update does not actually update any rows in case the version counter was already updated by another committed transaction in the meantime, and hibernate can detect this.
Also see:
https://dev.mysql.com/doc/refman/8.0/en/innodb-consistent-read.html
Repeatable Read isolation level SELECT vs UPDATE...WHERE
I have an Java(Spring) REST API endpoint where I get 3 data inputs and I need to Insert in the oracle database based on some unique ID using JDBCTemplate. But just to be sure something doesn't break, I want have a check first if I need to insert or just update.
1st Approach
Make a database call with a simple query like
SELECT COUNT(*) FROM TABLENAME WHERE ID='ABC' AND ROWNUM=1
And based on the value of count, make a separate Database call for Insert or Update. (count would never exceed 1)
2nd Approach
Make one single MERGE query hit using jdbctemplate.update() that would look like
MERGE INTO TABLENAME
USING DUAL ON ID='ABC'
WHEN MATCHED THEN UPDATE
SET COL1='A', COL2='B'
WHERE ID='ABC'
WHEN NOT MATCHED THEN
INSERT (ID, COL1, COL2) VALUES ('ABC','A','B')
Based on what I read on different sites, using MERGE is a bit more costly in terms of CPU reads based on an experiment on this site. But they have done it for purely for DB script use where they do it with 2 tables and my context of use is via API call and using DUAL.
I also read on this question that MERGE could result in ORA-0001: unique constraint and some concurrency issue.
I want to do this on a table on which some other operation is possible at the same time for a different row and a very very small chance for the same row value. So I want to know which approach to follow for such use case and I know this might be a common one but I could not find answer to what I'm looking for anywhere. I want to know the performance/reliability of both approach.
Looking at the code running in concurrent sessions environment, after each atomic statement we need to ask "what if another session have just broken our assumption?" and make adjustments according to that.
Option 1. Count and decide INSERT or UPDATE
declare
v_count int;
begin
SELECT count(1) INTO v_count FROM my_table WHERE ...;
IF v_count = 0 THEN
-- what if another session inserted the same row just before this point?
-- this statement will fail
INSERT INTO my_table ...;
ELSE
UPDATE my_table ...;
END IF;
end;
Option 2. UPDATE, if nothing is updated - INSERT
begin
UPDATE my_table WHERE ...;
IF SQL%COUNT = 0 THEN
-- what if another session inserted the same row just before this point?
-- this statement will fail
INSERT INTO my_table ...;
END IF;
end;
Option 3. INSERT, if failed - UPDATE
begin
INSERT INTO my_table ...;
exception when DUP_VAL_ON_INDEX then
-- what if another session updated the same row just before this point?
-- this statement will override previous changes
-- what if another session deleted this row?
-- this statement will do nothing silently - is it satisfactory?
-- what if another session locked this row for update?
-- this statement will fail
UPDATE my_table WHERE ...;
end;
Option 4. use MERGE
MERGE INTO my_table
WHEN MATCHED THEN UPDATE ...
WHEN NOT MATCHED THEN INSERT ...
-- We have no place to put our "what if" question,
-- but unfortunately MERGE is not atomic,
-- it is just a syntactic sugar for the option #1
Option 5. use interface for DML on my_table
-- Create single point of modifications for my_table and prevent direct DML.
-- For instance, if client has no direct access to my_table,
-- use locks to guarantee that only one session at a time
-- can INSERT/UPDATE/DELETE a particular table row.
-- This could be achieved with a stored procedure or a view "INSTEAD OF" trigger.
-- Client has access to the interface only (view and procedures),
-- but the table is hidden.
my_table_v -- VIEW AS SELECT * FROM my_table
my_table_ins_or_upd_proc -- PROCEDURE (...) BEGIN ...DML on my_table ... END;
PROCEDURE my_table_ins_or_upd_proc(pi_row my_table%ROWTYPE) is
l_lock_handle CONSTANT VARCHAR2(100) := 'my_table_' || pi_row.id;
-- independent lock handle for each id allows
-- operating on different ids in parallel
begin
begin
request_lock(l_lock_handle);
-->> this code is exactly as in option #2
UPDATE my_table WHERE ...;
IF SQL%COUNT = 0 THEN
-- what if another session inserted the same row just before this point?
-- NOPE it cannot happen: another session is waiting for a lock on the line # request_lock(...)
INSERT INTO my_table ...;
END IF;
--<<
exception when others then
release_lock(l_lock_handle);
raise;
end;
release_lock(l_lock_handle);
end;
Not going too deep into low level details here, see this article to find out how to use locks in Oracle DBMS.
Thus, we see that options 1,2,3,4 have potential problems that cannot be avoided in a general case. But they could be applied if the safety is guaranteed by domain rules or a particular design conventions.
Option 5 is bulletproof and fast as it is relies on the DBMS contracts.
Nevertheless, this will be a prize for clean design, and it cannot be implemented if my_table is barenaked and clients rely on straightforward DML on this table.
I believe that performance is less important than data integrity, but let's mention that for completeness.
After proper consideration it is easy to see that the options order according to the "theoretical" average performance is:
2 -> 5 -> (1,4) -> 3
Of course, the step of performance measuring goes after obtaining at least two properly working solutions, and should be done exclusively for a particular application under a given workload profile. And that is another story. At this moment no need to bother about theoretical nanoseconds in some synthetic benchmarks.
I guess currently we see that there will be no magic. Somewhere in the application it is required to ensure that every id inserted into my_table is unique.
If id values do not matter (95% of cases) - just go for using a SEQUENCE.
Otherwise, create a single point of manipulation on my_table (either in Java or in DBMS schema PL/SQL) and control the uniqueness there. If the application can guarantee that at most a single session at a time manipulates data in my_table, then it is possible to just apply the option #2.
I have a user table with field lastusedecnumber.
I need to access and increment lastusedecnumber.
During that accessing time I need to lock that particular user row (not the entire table).
How do I do this?
The table type is MyISAM.
MySQL uses only table-level locking from MyISAM tables. If you can, switch to InnoDB for row-level locking.
Here's a link to the MySQL site describing Locks set by SQL Statements for InnoDB tables.
http://dev.mysql.com/doc/refman/5.0/en/innodb-locks-set.html
Kind of late, but hope it will help someone:
UPDATE user SET lastusedecnumber = LAST_INSERT_ID(lastusedecnumber + 1);
SELECT LAST_INSERT_ID();
Will give you atomic increment of lastusedecnumber and ability to read new value of lastusedecnumber field (after increment) using SELECT LAST_INSERT_ID().
As a workaround you could add a column to your table, like locked TINYINT(1) - whenever you want the row to be locked you set it to 1. When you now try to access this row, the first thing you do is check if the locked fields is set.
To unlock the row, simply set it to 0 again. Not nice but a really simple workaround.
I didn't feel like converting my whole database from myisam. So I simply try to create a new table named based on the id of the record I want to lock. If create table is successful, do my work and delete the table at the end. If create table not successful, stop.
A better workaround is to create a column containting a timestamp. Whenever you want to lock the row you update it to the current time. To unlock update to a time at least x minutes in the past. Then to check if its locked check that the time stamp is at least x minutes old.
This way if the process crashes (or the user never completes their operation) the lock effectively expires after x minutes.
We need to generate sequential numbers for our transactions. We encountered sqlcode=-911, sqlstate=40001, sqlerrmc=2 (deadlock) when concurrent users are trying to book transactions at the same time. Currently deadlock occurs because it is reading and updating to the same record. How can we design this so that deadlock can be prevented?
Create a "seed" table that contains a single data row.
This "seed" table row holds the "Next Sequential" value.
When you wish to insert a new business data row using the "Next Sequential" value. perform the following steps.
1). Open a cursor for UPDATE on the "seed" table and fetch the current row. This gives you exclusive control over the seed value.
2). You will employ this fetched row as the "Next Value"... However before doing so
3). Increment the fetched "Next Value" and commit the update. This commit closes your cursor and releases the seed row with the new "Next Value".
you are now free to employ your "Next Value".
There are a number of ways around this issue, some less performant than others.
Deadlocks can be prevented if all objects are locked in the same hierarchical sequence. [https://en.wikipedia.org/wiki/Deadlock#Prevention]
However, solutions to the Dining Philosophers Problem [https://en.wikipedia.org/wiki/Dining_philosophers_problem] which completely prevent deadlocks are often less performant than simply rolling back the transaction and retrying. You'll need to test your solution.
If you're looking for a data-side solution, an old-fashioned (and potentially under-performant) approach is to force the acquisition of new transaction IDs to be atomic by establishing a rigorous lock sequence.
A quick-ish solution (test this under load before releasing to production!) could be to use TRANSACTION boundaries and have a control row acting as a gatekeeper. Here's a stupid example which demonstrates the basic technique.
It has no error checking, and the code to reclaim ghost IDs is outside the scope of this example:
DECLARE #NewID INTEGER;
BEGIN TRANSACTION;
UPDATE [TABLE] SET [LOCKFLAG] = CURRENT_TIMESTAMP WHERE [ROW_ID] = 0;
SELECT #NewID = MAX([ROW_ID])+1 FROM [TABLE];
INSERT INTO [TABLE] ([ROW_ID]) VALUES (#NewID);
UPDATE [TABLE] SET [LOCKFLAG] = NULL WHERE [ROW_ID] = 0;
COMMIT TRANSACTION;
The idea is to make this atomic, single-threaded, serialized operation very, very short in duration -- do only what is needed to safely reserve the ID and get out of the way.
By making the first step to update row 0, if all ID requests comply with this standard, then competing users will simply queue up behind that first step.
Once you have your ID reserved, go off and do what you like, and you can use a new transaction to update the row you've created.
You'd need to cover cases where the later steps decide to ROLLBACK, as there would now be a ghost row in the table. You'd want a way to reclaim those; a variety of simple solutions can be used.
The problem is this:
I have multiple competing threads (100+) that need to access one database table
Each thread will pass a String name - where that name exists in the table, the database should return the id for the row, where the name doesn't already exist, the name should be inserted and the id returned.
There can only ever be one instance of name in the database - ie. name must be unique
How do I ensure that thread one doesn't insert name1 at the same time as thread two also tries to insert name1? In other words, how do I guarantee the uniqueness of name in a concurrent environment? This also needs to be as efficient as possible - this has the potential to be a serious bottleneck.
I am using MySQL and Java.
Thanks
Assuming there is a unique constraint on the name column, each insert will acquire a lock. Any thread that attempts to insert it a second time concurrently will wait until the 1st insert either succeeds or fails (tx commit or rolls back).
If the 1st transaction succeeds, 2nd transaction will fail with with a unique key violation. Then you know it exists already.
If there is one insert per transaction, it'ok. If there are more than 1 insert per transaction, you may deadlock.
Each thread will pass a String name -
where that name exists in the table,
the database should return the id for
the row, where the name doesn't
already exist, the name should be
inserted and the id returned.
So all in all, the algo is like this:
1 read row with name
2.1 if found, return row id
2.2 if not found, attempt to insert
2.2.1 if insert succeeds, return new row id
2.2.2 if insert fails with unique constraint violation
2.2.2.1 read row with name
2.2.2.2 read should succeed this time, so return row id
Because there can be a high contention on the unique index, the insert may block for some time. In which case the transaction may time out. Make some stress test, and tune the configuration until it works correctly with your load.
Also, you should check if you get a unique constraint violation exception or some other exception.
And again, this works only if there is one insert per transaction, otherwise it may deadlock.
Also, you can try to read the row at step 1 with "select * for update". In this case, it waits until a concurrent insert either commits or succeeds. This can slightly reduce the amount of error at step 2.2.2 due to the contention on the index.
Create a unique constraint on name column in database.
Add a unique constraint for the name column.