Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
What could be the best solution for multithreaded Java application to ensure that all threads access db synchronously? For example, each thread represents separate transaction, and first checks db for value and then depending on answer has to insert or update some fields in database (note between check, insert and commit application is doing other processing). But the problem is that another thread might be doing just the same thing on same table.
More specific example. Thread T1 starts transaction, then checks table ENTITY_TABLE for entry with code '111'. If found updates its date, if not found inserts new entry, then commits transaction. Now imagine thread T2 does exactly same thing. Now there are few problems:
T1 and T2 checks db and find nothing and both insert same entry.
T1 checks db, find entry with old date, but on commit T2 already has updated entry to more recent date.
If we use cache and synchronize access to cache we have a problem: T1 acquires lock checks db and cache if not found add to cache, release lock, commit. T2 does the same, finds entry in cache going to commit. But T1 transaction fails and is roll backed. Now T2 is in bad shape, because it should insert to ENTITY_TABLE but doesn't know that.
more?
I'm working on creating simple custom cache with syncronization and solving problem 3. But maybe there is some more simple solution?
This should be dealt with primarily within the DB, by configuring the desired transaction isolation level. Then on top of this, you need to select your locking strategy (optimistic or pessimistic).
Without transaction isolation, you will have a hard time trying to ensure transaction integrity solely in the Java domain. Especially taking into consideration that even if the DB is currently accessed only from your Java app, this may change in the future.
Now as to which isolation level to choose, from your description it might seem that you need the highest isolation level, serializable. However, in practice this tends
to be a real performance hog due to extensive locking. So you may want to reevaluate your requirements to find the best balance of isolation and performance for your specific situation.
If you want to SQL SELECT a row from a database, and then later UPDATE the same row, you have 2 choices as a Java developer.
SELECT with ROWLOCK, or whatever the
row lock syntax is for your
particular data base.
SELECT the row, do your processing,
and just before you're ready to
update, SELECT the row again with ROWLOCK to see
if any other thread made changes.
If the two SELECTS return the same
values, UPDATE. If not, throw an
error or do your processing again.
The problem you are facing is transaction isolation.
Seems like you need to have each thread lock the row concerned in the where clause, which requires serializable isolation.
I tumbled into this problem when working with a multi-threaded Java program that was using a Sqllite database. It uses file locking so I had to make sure that only one thread was doing work at the same time.
I basically ended up with using synchronized. When the ConnectionFactory returns a db connection, it also returns a lock object that one should lock when using the connection. So you could do synchronization lock manually, or create a subclass of the class below which does it for you:
/**
* Subclass this class and implement the persistInTransaction method to perform
* an update to the database.
*/
public abstract class DBOperationInTransaction {
protected Logger logger = Logger.getLogger(DBOperationInTransaction.class.getName());
public DBOperationInTransaction(ConnectionFactory connectionFactory) {
DBConnection con = null;
try {
con = connectionFactory.getConnection();
if(con == null) {
logger.log(Level.SEVERE, "Could not get db connection");
throw new RuntimException("Could not get db connection");
}
synchronized (con.activityLock) {
con.connection.setAutoCommit(false);
persistInTransaction(con.connection);
con.connection.commit();
}
} catch (Exception e) {
logger.log(Level.SEVERE, "Failed to persist data:", e);
throw new RuntimeException(e);
} finally {
if(con != null) {
//Close con.connection silently.
}
}
}
/**
* Method for persisting data within a transaction. If any SQLExceptions
* occur they are logged and the transaction is rolled back.
*
* In the scope of the method there is a logger object available that any
* errors/warnings besides sqlException that you want to log.
*
* #param con
* Connection ready for use, do not do any transaction handling
* on this object.
* #throws SQLException
* Any SQL exception that your code might throw. These errors
* are logged. Any exception will rollback the transaction.
*
*/
abstract protected void persistInTransaction(Connection con) throws SQLException;
}
And the DBConnection struct:
final public class DBConnection {
public final Connection connection;
public final String activityLock;
public DBConnection(Connection connection, String activityLock) {
this.connection = connection;
this.activityLock = activityLock;
}
}
Offhand, I think you would have to lock the table before you query it. This will force sequential operation of your threads. Your threads should then be prepared for the fact that they will have to wait for the lock and of course, the lock acquisition might time out. This could introduce quite a bottleneck into your application as well as your threads will all have to queue up for database resources.
Related
I have a method in servlet that inserts tutoring bookings in database. This method has a business rule that checks if the tutor of this session is already busy in that date and hour. The code looks something like this :
class BookingService {
public void insert(Booking t) {
if(available(t.getTutor(), t.getDate(), t.getTime())) {
bookingDao.insert(t);
} else {
// reject
}
}
}
The problem is that multiple users may simultaneously try to book the same tutor on the same date and time, and there is nothing that prevents them both to pass the test and insert their bookings. I've tried making insert() synchronized and using locks, but it doesn't work. How can I prevent concurrent access to this method?
Using synchronized is an inadequate way to try to solve this problem:
First, you will have coded your application so that only one instance can be deployed at a time. This isn’t just about scaling in the cloud. It is normal for an IT department to want to stand up more than one instance of an application so that it is not a single point of failure (so that in case the box hosting one instance goes down the application is still available). Using static synchronized means that the lock doesn’t extend beyond one application classloader so multiple instances can still interleave their work in an error prone way.
If you should leave the project at some point, later maintainers may not be aware of this issue and may try to deploy the application in a way you did not intend. Using synchronized means you will have left a land mine for them to stumble into.
Second, using the synchronized block is impeding the concurrency of your application since only one thread can progress at a time.
So you have introduced a bottleneck, and at the same time cut off operations’ ability to work around the bottleneck by deploying a second instance. Not a good solution.
Since the posted code shows no signs of where transactions are, I’m guessing either each DAO creates its own transaction, or you’re connecting in autocommit mode. Databases provide transactions to help with this problem, and since the functionality is implemented in the database, it will work regardless of how many application instances are running.
An easy way to fix the problem which would avoid the above drawbacks would be to put the transaction at the service layer so that all the DAO calls would execute within the same transaction. You could have the service layer retrieve the database connection from a pool, start the transaction, pass the connection to each DAO method call, commit the transaction, then return the connection to the pool.
One way you could solve the problem is by using a synchronized block. There are many things you could choose as your locking object - for the moment this should be fine:
class BookingService {
public void insert(Booking t) {
synchronized(this) {
if(available(t.getTutor(), t.getDate(), t.getTime())) {
bookingDao.insert(t);
} else {
// reject
}
}
}
}
If you have more than one instance of the servlet, then you should use a static object as a lock.
I am working on a program that allows multiple users to access a db (MySQL), and at various times I'm getting a SQLException: Lock wait timeout exceeded .
The connection is created using:
conn = DriverManager.getConnection(connString, username, password);
conn.setAutoCommit(false);
and the calls all go through this bit of code:
try {
saveRecordInternal(record);
conn.commit();
} catch (Exception ex) {
conn.rollback();
throw ex;
}
Where saveRecordInternal has some internal logic, saving the given record. Somewhere along the way is the method which I suspect is the problem:
private long getNextIndex() throws Exception {
String query = "SELECT max(IDX) FROM MyTable FOR UPDATE";
PreparedStatement stmt = conn.prepareStatement(query);
ResultSet rs = stmt.executeQuery();
if (rs.next()) {
return (rs.getLong("IDX") + 1);
} else {
return 1;
}
}
This method is called by saveRecordInternal somewhere along it's operation, if needed.
For reasons that are currently beyond my control I cannot use auto-increment index, and anyway the index to-be-inserted is needed for some internal-program logic.
I would assume having either conn.commit() or conn.rollback() called would suffice to release the lock, but apparently it's not. So my question is - Should I use stmt.close() or rs.close() inside getNextIndex? Would that release the lock before the transaction is either committed or rolled back, or would it simply ensure the lock is indeed released when calling conn.commit() or conn.rollback()?
Is there anything else I'm missing / doing entirely wrong?
Edit: At the time the lock occurs all connected clients seem to be responsive, with no queries currently under-way, but closing all connected clients does resolve the issue. This leads me to think the lock is somehow preserved even though the transaction (supposedly?) ends, either by committing or rolling back.
Even though not closing a Statement or ResultSet is a bad idea but that function doesn't seem responsible for error that you are receiving. Function , getNextIndex() is creating local Statement andResultSet but not closing it. Close those right there or create those Statement and ResultSetobjects in saveRecordInternal() and pass as parameters or better if created in your starting point and reused again and again. Finally, close these when not needed anymore ( in following order - ResultSet, Statement, Connection ).
Error simply means that a lock was present on some DB object ( Connection, Table ,Row etc ) while another thread / process needed it at the same time but had to wait ( already locked ) but wait timed out due to longer than expected wait.
Refer , How to Avoid Lock wait timeout exceeded exception.? to know more about this issue.
All in all this is your environment specific issue and need to be debugged on your machine with extensive logging turned on.
Hope it helps !!
From the statements above I don't see any locks that remain open!
In general MySql should release the locks whenever a commit or rollback is called, or when the connection is closed.
In your case
SELECT max(IDX) FROM MyTable FOR UPDATE
would result in locking the whole table, but I assume that this is the expected logic! You lock the table until the new row is inserted and then release it to let the others insert.
I would test with:
SELECT IDX FROM MyTable FOR UPDATE Order by IDX Desc LIMIT 1
to make sure that the lock remains open even when locking a single row.
If this is not the case, I might be a lock timeout due to a very large table.
So, what I think happen here is: you query is trying to executed on some table but that table is locked with some another process. So, till the time the older lock will not get released from the table, your query will wait to get executed and after some time if the lock will no release you will get the lock time out exception.
You can also take a look on table level locks and row level locks.
In brief: table level locks lock the whole table and till the lock is there you want be able to execute any other query on the same table.
while
row level lock will put a lock on a specific row, so apart from that row you can execute queries on the table.
you can also check if there is any query running on the table from long time and due to that you are not able to execute another query and getting exception. How, to check this will vary upon database, but you can google it for your specific database to find out query to get the open connections or long running queries on database.
I write a java application where different threads (each thread has an own connection object using a connection pool c3p0) call a method like that.
Pseudo code:
void example(Connection connection) {
connection.update("LOCK TABLES Test WRITE");
resultSet = connection.query("SELECT * FROM Test WHERE Id = '5'");
if (resultSet.next()) {
connection.update("UPDATE Test SET Amount = '10' WHERE Id = '5'");
} else {
connection.update("INSERT INTO Test (Id, Amount) VALUES ('5', '10')");
}
connection.update("UNLOCK TABLES");
connection.commit();
}
There are a few other similar methods which lock a table, select/update/insert something and then unlock the table. The aim is to prevent race conditions and deadlocks.
Is it possible to cause MySQL deadlocks when I call such a method from different threads? If yes, can you give me an example how that happens (timing of two transactions which cause a deadlock)? I am a noob with deadlocks and I want to get into this.
Edit: Make clear that the connection that should be used in the method is passed from the thread that calls the method.
Edit: Replace READ with WRITE
It cannot here. As there is no complex logic and the code immediately commits after update, there must be always one thread which goes through. Even in more complex scenarios it would probably require highest serialization level (repeatable reads) which I believe MySql does not support.
This would possibly create a deadlock. Actually I'm not sure if it'll even execute, because you need to acquire a "WRITE" lock, not a "READ".
I am getting difficulties when trying to understand how can version-based optimistic locking prevent "last-commit-wins" issue and appropriate overriding.
To make the question more concrete, let's consider the following pseudo-code that uses JDBC:
connection.setAutoCommit(false);
Account account = select(id);
if (account.getBalance() >= amount) {
account.setBalance(account.getBalance() - amount);
}
int rowsUpdated = update(account); // version=:oldVer+1 WHERE version=:oldVer
if (rowsUpdated == 0) throw new OptimisticLockException();
connection.commit();
Here what if other transaction would commit its change right between the update and the commit ? If the transactions are concurrent, then the update made by the first transaction is not yet committed and so not visible to the second transaction (with proper isolation levels) and so the first transaction commit will override the changes of the second transaction without any notification or error.
Is this the case that optimistic locking just decrease the probability of the issue while not preventing it in general ?
the idea of a database "transaction" is that it is supposed to provide a guarantee of "consistency" across multiple conceptual operations. the database is in charge of enforcing this. so, when the transaction commits, the database should only allow the transaction to complete if it can ensure that everything that happened during the transaction is still valid.
In practice, a database will typically handle this such that once one of the updates succeeds, the relevant row will be write locked until the relevant transaction completes. Thus, one of the updates is guaranteed to fail.
Note: this also requires an appropriate isolation level in your jdbc connection. the isolation level ensures that the test for the current value done before the update is still applicable at the time of the write.
Please correct me if I am wrong somewhere.
I am having an issue where my transaction are not being saved to the data base and some sort of racing is occurring which screws up the data. The app is hit in parallel by multiple instances. I have used #Transactional, which I know is to do a transaction with database and the transaction is committed when the method returns.
The question is, does hitting it through multiple instance still maintain this one transaction per hit thing, or it does not handle the situation and data will screw up because of racing?
Can a solution be suggested for the given condition?
The #Transactional is not related to synchronization. It just makes sure that your flow either succeeds or fails. Each hit has its own flow and its own success or failure.
I guess what you're experiencing is due to the use of shared data.
For example. If you have a class Foo that looks like this:
public class Foo {
private static boolean flag = true;
#Transactional
public void doSomething() {
flag = false;
}
}
In this case it doesn't matter that you have many Foo instances because they all use the same flag.
Another scenario would be if you have one instance of Foo (very common if you use something like Spring) and you have data that is changed for this instance. You can look at the same Foo example and just remove the static from flag:
public class Foo {
private boolean flag = true;
#Transactional
public void doSomething() {
flag = false;
}
}
In either of those cases you need to synchronize the data changes somehow. It has nothing to do with #Transactional.
That transactions are database transactions and behavior is database engine dependant but it usually works this way:
A thread enter the method.
A thread enter the same or any other transactional method. It does not block as #Transactional is not about synchronization.
One thread execute any query that blocks a database resource. Eg. SELECT * FROM MYTABLE FOR UPDATE;.
Another thread try to execute anything that needs the same database resource. Eg. UPDATE MYTABLE SET A = A + 1; And it blocks.
The thread that acquired the lock on step 3 completes the transactional method successfully making an implicit commit or fails making an implicit rollback.
The blocked thread wakes up and continues as it can now get the resource that was locked.