Develop Reference

LockModeType.OPTIMISTIC and Mysql's default isolation level REPEATABLE READ don't work together?

I am trying to learn JPA with hibernate and use MySQL as the db.
From my understanding,
LockModeType.OPTIMISTIC: The entity version is checked towards the end
of the currently running transaction.
REPEATABLE READ: All consistent reads within the same transaction read
the snapshot established by the first such read in that transaction
Is it true that LockModeType.OPTIMISTIC in hibernate does not work with MySQL's default isolation level?
Say I have the following code:
tx.begin();
EntityManager em = JPA.createEntityManager();
Item item = em.find(Item.class, 1, LockModeType.OPTIMISTIC);
// Assume the item here has version = 0
// Read the item fields etc, during that another transaction commits and made item version increased to version = 1
tx.commit(); // Here Hibernate should execute SELECT during flushing to check version,
// i.e SELECT version FROM Item WHERE id = 1
em.close();
What I would expect is that, during flushing, Hibernate would throw OptimisticLockException because the version of the item is no longer 0. However, due to the isolation level, in the same transaction Hibernate would still see the item in version = 0 and not triggering OptimisitcLockExcpetion.
I tried to search but seems no one raised such question before, hopefully someone can help clear my confusion on OptimisticLock.

If your question is actually is there a flaw in HBN implementation (or JPA specification) related to the following statement:
If transaction T1 calls for a lock of type LockModeType.OPTIMISTIC on
a versioned object, the entity manager must ensure that neither of the
following phenomena can occur:
P1 (Dirty read): Transaction T1 modifies a row. Another transaction T2 then reads that row and obtains the modified value, before T1 has
committed or rolled back. Transaction T2 eventually commits
successfully; it does not matter whether T1 commits or rolls back and
whether it does so before or after T2 commits.
P2 (Non-repeatable read): Transaction T1 reads a row. Another transaction T2 then modifies or deletes that row, before T1 has
committed. Both transactions eventually commit successfully.
Lock modes must always prevent the phenomena P1 and P2.
then the answer is yes, you are correct: in case when you are performing computations based on some entity state, but you are not modifying those entity state, HBN just issues select version from ... where id = ... at the end of transaction and hence it do not see changes from other transactions due to RR isolation level. However I would not say that RC isolation level performs much better for this particular case: it's behaviour more correct from technical perspective but it is completely unreliable from business perspective because it depends on timings, so just do not rely on LockModeType.OPTIMISTIC - it is unreliable by design and use another techniques like:
store data from different domains in different entities
take advantage of #OptimisticLock annotation to prevent incrementing of version when it is not required (actually this will poison you domain model by HBN annotations)
mark some properties as updatable=false and update them via JPQL update in order to prevent version increment
UPD.
Taking the P2 as example, if I really need T1 (only read row) to fail if T2 (modify/delete row) commits first, the only workaround I can think of is to use LockModeType.OPTIMISTIC_FORCE_INCREMENT. So when T1 commits it will try to update the version and fail. Can you elaborate more on how your provided 3 points at the end can help with this situation if we keep using RR isolation level?
The short story:
LockModeType.OPTIMISTIC_FORCE_INCREMENT does not seem to be a good workaround, cause it turns reader into writer, so incrementing version will fail both writers and other readers. However in your case it might be acceptable to issue LockModeType.PESSIMISTIC_READ which for some DBs translates into select ... from ... for share/lock in share mode, which in turn blocks only writer and blocks (or fails) current reader, so you will avoid the phenomenon we are talking about.
The long story:
When we have started thinking about some "business consistency" the JPA specification is not our friend anymore, the problem is they define consistency in terms of "denied phenomena" and "someone must fail", but does not give us any clues and APIs how to control the behaviour in the correct way from business perspective. Let's consider the following example:
class User {
#Id
long id;
#Version
long version;
boolean locked;
int failedAuthAttempts;
}
our goal is to lock user account when failedAuthAttempts exceeds some threshold value. The pure SQL solution for our problem is very simple and straightforward:
update user
set failed_auth_attempts = failed_auth_attempts + 1,
locked = case failed_auth_attempts + 1 >= :threshold_value then 1 else 0 end
where id = :user_id
but JPA complicates everything... at first glance our naive implementation should look like:
void onAuthFailure(long userId) {
User user = em.find(User.class, userId);
int failedAuthAttempts = user.failedAuthAttempts + 1;
user.failedAuthAttempts = failedAuthAttempts;
if (failedAuthAttempts >= thresholdValue) {
user.locked = true;
}
em.save(user);
}
but that implementation has obvious flaw: if someone actively bruteforces user account not all failed auth attempts get recorded due to concurrency (here I'm not paying attention that it might be acceptable because sooner or later we will lock user account). How to resolve such issue? May we write something like:
void onAuthFailure(long userId) {
User user = em.find(User.class, userId, LockModeType.PESSIMISTIC_WRITE);
int failedAuthAttempts = user.failedAuthAttempts + 1;
user.failedAuthAttempts = failedAuthAttempts;
if (failedAuthAttempts >= thresholdValue) {
user.locked = true;
}
em.save(user);
}
? Actually no. The problem is for entities which are not present in persistence context (i.e. "unknown entities") hibernate issues select ... from ... where id=:id for update, but for known entities it issues select ... from ... where id=:id and version=:version for update and obviously fails due to version mismatch. So we have following tricky options to make our code to work "correctly":
spawn another transaction (I believe in most cases it is not a good option)
lock entity via select query, i.e. smth. like em.createQuery("select id from user where id=:id").setLockMode(LockModeType.PESSIMISTIC_WRITE).getFirstResult() (I believe that may not work in RR mode, moreover following refresh call looses data)
mark properties as non-updatable and update them via JPQL update (pure SQL solution)
Now let's pretend we need to add another business data into our User entity, say "SO reputation", how are we supposed to update new field keeping in mind that someone might bruteforce our user? The options are following:
continue to write "tricky code" (actually that might lead us to the counterintuitive idea that we always need to lock entity before updating it)
split data from different domains across different entities (sounds counterintuitive too)
use mixed techniques
I do believe this UPD will not help you much, however it's purpose was to demonstrate that it does not worth to discuss consistency in JPA domain without knowledge about target model.

To understand this, let's have a quick look on how hibernates optimistic locking works:
1: begin a new transaction
2: find an entity by ID (hibernate issues a SELECT ... WHERE id=xxx;), which e.g. could have a version count of 1
3: modify the entity
4: flush the changes to the DB (e.g. triggered automatically before committing a transaction):
4.1: hibernate issues an UPDATE ... SET ..., version=2 WHERE id=xxx AND version=1 which returns the number of updated rows
4.2: hibernate checks whether there was one row actually updated, throwing a StaleStateException if not
5: commit the transaction / rollback in case of the exception
With the repeatable_read isolation level, the first SELECT establishes the state (snapshot) which subsequent SELECTs of the same transaction read. However, the key here is that the UPDATE does not operate on the established snapshot, but on the committed state of the row (which might have been changed by other committed transactions in the meantime).
Therefore the update does not actually update any rows in case the version counter was already updated by another committed transaction in the meantime, and hibernate can detect this.
Also see:
https://dev.mysql.com/doc/refman/8.0/en/innodb-consistent-read.html
Repeatable Read isolation level SELECT vs UPDATE...WHERE

Different ways for solving Race condition in distributed environment in Rest Services in java

Here is a description of a problem that I am facing right now
I want to give user some promotion based on whether he has done a transaction in which he has been given a promotional discount
Promotional discount is some percent off on the transaction
So the condition is, before processing transactions, it will be checked whether he as done a transaction using a promotion or not, and based on that amount will be calculated.
Problem is if two request comes at the same time and reads that a transaction has not been done, and on both the transaction promotion is applied.
Found a solution, https://dzone.com/articles/synchronized-by-the-value-of-the-object-in-java, but not valid for distributed environment.
What are can be different ways to solve such a program. Was just very curious on this problem >

Distributed environments require distributed locks. And there are a lot of options from here, we have used zookeeper for this, some other team of mine used redis. I also know (theoretically) that hazelcast has such a principle also; I bet there are many more.
There are some things you need to consider in a distributed environment that slightly complicate things. What if the service that "provides" the locking mechanism dies, what if the client that acquired a lock never releases it (it might die or block internally forever), etc. There are fault tolerant policies for the first and there are mechanisms for the second (a timeout for release or an "auto-disconnect" of the lock).
Some time ago I worked for a company that did this "by hand" against a MSSQL database. What they did (I was only a consumer of that service), is create a java-agent that would instrument byte-code and, at some points of execution would connect to a database and try to "CAS" (compare-and-swap) a certain row in a table. I don't know the faith of that project now, but I still love the idea to this day.
Bottom line is that there are many options, really. It highly depends on your infrastructure and team that you are involved in.

There are different ways this can be achieved. Accessing the table using serializable transaction isolation level is one way. However, since you mentioned the process involves potentially long running external calls as well, then you can do a two-phase approach like this:
Start a transaction
Use "select for update" to search and lock the record that entitles the user with a promotion.
Check if the promotion is used already. If yes, commit the transaction, promotion is not available. If no, commit the transaction. Promotion is available.
Do your external calls
Start a transaction and do the promotion check again, with "select for update", but this time, insert the record for using the promotion as well

How to achieve row level locking in cassandra

I have a Cassandra cluster setup like Node N1, Node N2 and Node N3
I have a user table, I need to write create a row level locking for it across the nodes in the cluster, So could you please guide me by answering the following questions?
1) What is the maximum level of locking possible in Cassandra?
2) What is lightweight transaction? How much it is possible to achieve row level locking?
3) Is there an alternate way to achieve the row level locking in Cassandra?

None, but you might stretch it to say column level.
It uses paxos for consensus and can perform conditional updates. It doesnt do locking. It will either succeed or not if another update occurred, if it doesnt succeed you can try again. If it does succeed everything in "transaction" (poor naming) will apply. However, theres still no isolation within it and if multiple columns in row are updated you may read between them being applied. details here
Design data model so you dont need locking.

There is no transactions in cassandra, there is no locking. There is however light weight transactions. They're not great, the performance is worse and there are alot of tradeoffs.
Depending on what the use case is for this lock you could do:
INSERT INTO User (userID, email)
VALUES (‘MyGuid’, ‘user#example.com’)
IF NOT EXISTS;
If the query returns an error/failure you would have to handle that, it won't just fail if someone inserts before you. A failure also might mean that 1 of your nodes did get the write but not all of them. LWT don't roll back.

How to code optimistic and pessimistic locking from java code

I know what optimistic and pessimistic locking is, but when you write a java code how do you do it? Suppose I am using Oracle with Java, do I have any methods in JDBC that will help me do that? How will I configure this thing? Any pointers will be appreciated.

You can implement optimistic locks in your DB table in this way (this is how optimistic locking is done in Hibernate):
Add integer "version" column to your table.
Increase the value of this column with each update of corresponding row.
To obtain lock, just read "version" value of the row.
Add "version = obtained_version" condition to where clause of
your update statement. Verify number of affected rows after update.
If no rows were affected - someone has already modified your entry.
Your update should look like
UPDATE mytable SET name = 'Andy', version = 3 WHERE id = 1 and version = 2
Of course, this mechanism works only if all parties follow it, contrary to DBMS-provided locks that require no special handling.
Hope this helps.

Suppose I am using Oracle with Java, do I have any methods in JDBC that will help me do that?
This Oracle paper should provide you with some tips on how to do this.
There are no specific JDBC methods. Rather, you achieve optimistic locking by the way that you design your SQL queries / updates and where you put the transaction boundaries.

MySQL InnoDB hangs on waiting for table-level locks

I have a big production web-application (Glassfish 3.1 + MySQL 5.5). All tables are InnoDB. Once per several days application totally hangs.
SHOW FULL PROCESSLIST shows many simple insert or update queries on different tables but all having status
Waiting for table level lock
Examples:
update user<br>
set user.hasnewmessages = NAME_CONST('in_flag',_binary'\0' COLLATE 'binary')
where user.id = NAME_CONST('in_uid',66381)
insert into exchanges_itempacks
set packid = NAME_CONST('in_packId',332149), type = NAME_CONST('in_type',1), itemid = NAME_CONST('in_itemId',23710872)
Queries with the longest 'Time' are waiting for the table-level lock too.
Please help to figure out why MySQL tries to get level lock and what can be locking all these tables. All articles about the InnoDB locking say this engine uses no table locking if you don't force it to do so.
My my.cnf has this:
innodb_flush_log_at_trx_commit = 0
innodb_support_xa = 0
innodb_locks_unsafe_for_binlog = 1
innodb_autoinc_lock_mode=2
Binary log is off. I have no "LOCK TABLES" or other explicit locking commands at all. Transactions are READ_UNCOMMITED.
SHOW ENGINE INNODB STATUS output:
http://avatar-studio.ru:8080/ph/imonout.txt

Are you using MSQLDump to backup your database while it is still being accessed by your application? This could cause that behaviour.

I think there are some situations when MySQL does a full table lock (i.e. using auto-inc).
I found a link which may help you: http://mysqldatabaseadministration.blogspot.com/2007/06/innodb-table-locks.html
Also review java persistence code having all con's commited/rollbacked and closed. (Closing always in finally block.)
Try setting innodb_table_locks=0 in MySQL configuration.
http://dev.mysql.com/doc/refman/5.0/en/innodb-parameters.html#sysvar_innodb_table_locks
Just a few ideas ...

I see you havily use NAME_CONST in your code. Just try not to use it. You know, mysql can be sometimes buggy (I also found several bugs), so I recommend don't rely on features which are not so common / well tested. It is related to column names, so maybe it locks something? Well it should't if it affects only the result, but who knows? This is suspicious. Moreover, this is marked as a function for internal use only.

This may seem simple, but you don't have a long-running select statement that is possibly locking out updates and inserts? There's no query that's actually running and not locked?

Have you considered using MyISAM instead of InnoDB?
If you are not utilizing any transactional features, MyISAM might make more sense.
Its simpler, easier to optimize, and since it doesn't have sophisticated transactional capabilities, easier to configure in your my.cnf.
Also, depending on the type of db load your app creates, MyISAM might be more appropriate. I prefer MyISAM for read-heavy applications, again, it's easier to configure and understand.
Other suggestions:
It might be a good idea to find a way to not use NAME_CONST in your SQL.
"This function was added in MySQL 5.0.12. It is for internal use only."
When the documentation of an open source product says this, its probably a good idea to heed it's advise.
By default, MySQL stores all InnoDB tables & schemas data in 1 enormous file, there could be some kind of OS level locking on that particular file that propogates to MySQL that prevents all table access. By using the innodb_file_per_table option , you may eliminate that potential issue. This also makes MySQL more space efficient.

in this case you have to create several different database table with same column each other and do not inset more then 3000 row per table, in this case if you want to enter more data into table you have to create another dynamic table(generate table using code) and insert new data into this table and access data from that table. in your condition if more and more table will have to generate then you have to create new database.
i think this tip will help you to design your database more carefully and solve error.