Row lock contention issue with large transactions - java

I have a situation where we are acquiring lock on an object from database using SELECT FOR UPDATE. This is necessary for us to insert and delete records from multiple tables in an orderly fashion. The functionality works something like this.
Login -> Acquire lock on unique lock object and insert records to multiple tables and release lock -> Logout -> Acquire lock on same unique lock object and delete records from multiple tables and release lock.
We have synchronization enabled to track users have logged in before logging him out. It is taken care in Java code. However we obtain another lock at database level to make sure the database transactions are synchronized when large number of users are logging in.
Problem: The whole system works perfectly in multi-clustered servers and singleton servers. However, when the number of concurrent users reaches 4000+, we are facing row lock contention (Mode 6) in the database. And few users are not able to login.
Objective: To fix the locking mechanism to enable users to login and logout successfully.
Things tried so far: Added NOWAIT and SKIP LOCKED to SELECT FOR UPDATE query. This doesn't solve my problem because the first one simply throws an error and the second one basically skips the lock which would affect synchronization.
Need suggestions and opinions from Database experts to resolve this issue. TIA.
UPDATE: Just adding one more information. We do not update or do anything with the locked row. It is just used as a mechanism to synchronize other database tasks we do.

Instead of relying on pessimistic locking(your current approach)- use optimistic locking possibly using some ORM.

Related

How to lock a database row for a fixed time interval?

I have two databases. I have to commit my transaction only when both the entries are available in those two databases. For that, I planned to use a 2 phase similar algorithm. But the implementation is getting somewhat tricky.
I thought about implementation as follows:
I query both the databases for the respective entries and if available, I will lock them so that no other user can access them and return success.
If i get success from both, I commit them. or else i will remove the lock, if any of the entry is locked.
I also have to handle faults, so the lock should be released after a timeout if an explicit release query hasn't been sent.
My question is, how to explicitly lock the rows for no concurrent access to multiple clients and is there any better implementation that I should think about? Thanks for the help in advance.

Concurrent update and delete on one table

Using Java, Hibernate and Oracle database.
I have two concurrent processes:
Process1 removes some entities from table1. (multiple: delete from table1 where id =...) Done by native hibernate query.
Process2 updates SAME/other entities in table1. (multiple: update table1 set name=... where id=...) Done by jpa repository delete method.
Currently sometimes exception
CannotAcquireLockException is thrown,
(SQL Error: 60, SQLState: 61000..
ORA-00060: deadlock detected while waiting for resource)
So, the question is: what is going on and how I can avoid exception? Any workaround?
IMPORTANT: In case of collisions I would be satisfied if delete succeeds and update won't do anything.
Session A waits for B, B waits for A - this is what a deadlock basically is.
Nothing to wait for any more, Oracle kills either of the sessions.
Option 1
Create semaphore to effectively serialize concurrent processes.
create table my_semaphore(dummy char(1));
Session 1:
LOCK TABLE my_semaphore in exclusive mode;
UPDATE <your update here>;
COMMIT;
Session 2:
LOCK TABLE my_semaphore in exclusive mode;
DELETE <your delete here>;
COMMIT;
Option 2
Try processing rows with both statements in the same order, say by rowid or whatever.
So that session B never returns to rows held by A, if A is stuck in behind by rows locked by B. This more tricky and resource-intesive.
"locking tables doesnt look attractive at all -what the point then of having severaal processes working with database"
Obviously we want to enable concurrent processes. The trick is to design processes which can run concurrently without interfering with each other. Your architecture is failing to address this point. It should not be possible for Process B to update records which are being deleted by Process A.
This is an unfortunate side-effect of the whole web paradigm which is stateless and favours an optimistic locking strategy. Getting locks at the last possible moment "scales" but incurs the risk of deadlock.
The alternative is a pessimistic locking strategy, in which a session locks the rows it wants upfront. In Oracle we can do this with SELECT .. FOR UPDATE. This locks a subset of rows (the set defined by the WHERE clause) and not the whole table. Find out more.
So it doesn't hinder concurrent processes which operate on different subsets of data but it will prevent a second session grabbing records which are already being processed. This still results in an exception for the second session but at least that happens before the session has done any work, and provides information to re-evaluate the task (hmmm, do we want to delete these records if they're being updated?).
Hibernate supports the SELECT FOR UPDATE syntax. This StackOverflow thread discusses it.

Transaction in PostgreSql

I'd like to realize following scenario in PosgreSql from java:
User selects data
User starts transaction: inserts, updates, deletes data
User commits transaction
I'd like data not be available for other users during the transaction. It would be enough if I'd get an exception when other user tries to update the table.
I've tried to use select for update or select for share, but it locks data for reading also. I've tried to use lock command, but I'm not able to get a lock (ERROR: could not obtain lock on relation "fppo10") or another transaction gets lock when trying to commit transaction, not when updating the data.
Does it exist a way to lock data in a moment of transaction start to prevent any other call of update, insert or delete statement?
I have this scenario working successfully for a couple of years on DB2 database. Now I need the same application to work also for PostgreSql.
Finally, I think I get what you're going for.
This isn't a "transaction" problem per se (and depending on the number of tables to deal with and the required statements, you may not even need one), it's an application design problem. You have two general ways to deal with this; optimistic and pessimistic locking.
Pessimistic locking is explicitly taking and holding a lock. It's best used when you can guarantee that you will be changing the row plus stuff related to it, and when your transactions will be short. You would use it in situations like updating "current balance" when adding sales to an account, once a purchase has been made (update will happen, short transaction duration time because no further choices to be made at that point). Pessimistic locking becomes frustrating if a user reads a row and then goes to lunch (or on vacation...).
Optimistic locking is reading a row (or set of), and not taking any sort of db-layer lock. It's best used if you're just reading through rows, without any immediate plan to update any of them. Usually, row data will include a "version" value (incremented counter or last updated timestamp). If your application goes to update the row, it compares the original value(s) of the data to make sure it hasn't been changed by something else first, and alerts the user if the data changed. Most applications interfacing with users should use optimistic locking. It does, however, require that users notice and pay attention to updated values.
Note that, because a lock is rarely (and for a short period) taken in optimistic locking, it usually will not conflict with a separate process that takes a pessimistic lock. A pessimistic locking app would prevent an optimistic one from updating locked rows, but not reading them.
Also note that this doesn't usually apply to bulk updates, which will have almost no user interaction (if any).
tl;dr
Don't lock your rows on read. Just compare the old value(s) with what the app last read, and reject the update if they don't match (and alert the user). Train your users to respond appropriately.
Instead of select for update try a "row exclusive" table lock:
LOCK TABLE YourTable IN ROW EXCLUSIVE MODE;
According to the documentation, this lock:
The commands UPDATE, DELETE, and INSERT acquire this lock mode on the
target table (in addition to ACCESS SHARE locks on any other
referenced tables). In general, this lock mode will be acquired by any
command that modifies data in a table.
Note that the name of the lock is confusing, but it does lock the entire table:
Remember that all of these lock modes are table-level locks, even if
the name contains the word "row"; the names of the lock modes are
historical

Sybase/JDBC: how to detect reorgs or exclusive locks?

We use Sybase ASE (15.5) server as our DB and are having strange, intermittent SPID blocking issues that I am trying to detect and mitigate programmatically at the application-layer.
Sybase allows you to schedule so-called "reorgs" which from what I can tell are periodic re-indexes/table compactions, cleanups, etc. Scheduled DB maintenance, basically.
Every once in a while, we get all the planets coming into alignment with each other, where:
A query is executed (creating a SPID in Sybase) and hangs for some reason. This places a (blocking) shared lock on, say, the widgets table; then
The scheduled reorg kicks off, and wants to cleanup the widgets table. The reorg places an exclusive lock request on widgets, but can't obtain the lock because widgets is already locked and blocked by the hanging SPID/query; then
Subsequent queries are executed, each requesting shared locks on widgets; such that
The whole system is now tied up: the reorg can't start until it obtains an exclusive lock on widgets, but widgets is tied up in a blocking shared lock by a hung SPID. And because the reorg has placed an exclusive lock on widgets, all other queries wanting shared locks on widgets have to wait until the reorg is complete (because a newly requested exclusive lock trumps a newly requested shared lock).
I think my ideal strategy here would be to:
Timeout DB queries after say, 2 minutes, which will prevent SPIDs from hanging and thus preventing the reorgs from running; and then
If a query attempts to hit a table that has an exclusive lock on it, detect this and hadle it specially (like schedule the query to run again 1hr later, when hopefully the reorg is complete, etc.)
My questions:
How do I timeout a query to release a shared lock after, say, 2mins?
Is there a way to programmatically (most likely through the Sybase JDBC driver, but perhaps via Sybase command-line, HTTP calls, etc.) determine if a reorg is running? Or, that an exclusive lock exists on a table? That way I could detect the exclusive lock and handle it in a special way.
Thanks in advance!
You can get the commands that are running in the database with the following query:
select cmd from sysprocesses
To find locking information you can join master..syslocks and your_db..sysobjects to find out what locks exist on the object you are trying to access. syslocks.type indicates the kind of lock that is in place, and those possible values can be found here:
select object_name(id), db_name(dbid), type from master..syslocks
where dbid = db_id("your_db")
http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc36274.1550/html/tables/X16427.htm
Hopefully that helps.
To find if the lock is tied to a reorg, I think you should be able to join syslocks.spid to sysprocesses.spid where cmd = "REORG" or something like:
select p.cmd, p.spid, l.type from master..sysprocesses p, master..syslocks l where CMD = "REORG"

Check MySQL table's ROW LOCK STATUS via Java

I have a Java frontend and a MySQL backend scenario, I used a 'LOCK IN SHARE MODE' for SELECT. If I request the same row from another process, it gives the data.. However it does not allow me to update. What I would like to do is inform the user they will only have a READ only copy, so if they wish to see the information they can else they can request it later.. How could I check the status of the ROW so that the user will be informed about this situation?? If I use 'FOR UPDATE', It just waits for until the first user saves the data. I find it less user friendly, if they just have a blank screen or when they click button it does nothing. Any help will be greatly appreciated. Using MySQL 5.5, Java 7.
The short answer is "You can't"!
You may want to take a look at this discussion.
[EDIT]
The answer to that post states:
You can't (check lock's state) for non-named locks!!!! More info:
http://forums.mysql.com/read.php?21,222363,223774#msg-223774
Row-level locks are not meant for application level locks. They are just means to implement consistent reads and writes. That means you have to release them as soon as possible. You need to implement your own application level lock and it's not that much hard. Perhaps a simple user_id field will do. If it is null then there's no lock. But if it's not null, the id indicates who is holding the record. In this case you'll need row-level locking to update the user_id field. And as I said before, you'll have to release MySQL lock as soon as you are done locking / unlocking the record.
The question's entire premise lies in the rather liberal use of RDBMS' row-level locking (which is usually used for short-lived concurrency control) directly for interactive UI control.
But putting that aside and answering the question, one can set the session's innodb_lock_wait_timeout to a very short value, minimum being 1, and catching the resulting Lock wait timeout exceeded; try restarting transaction when unable to lock.
The exception class was com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException when I just tried with mysql-connector-java 5.1.38, but other exception classes has changed over releases so this too may be different in older version of MySQL Connector/J.
The "attempt and fail" method of acquiring locks is the standard way of tackling these types of concurrency situations, as the alternate method of "check before attempting" is an anti-pattern that creates a race-condition between checking and the actual attempt to lock.

Categories

Resources