TokuMX MVCC Transactions in Java - java

How are highly contended records saved when running concurrent transactions on the same document(s)?
It appears that this is happening:
MVCC Transaction A begins.
MVCC Transaction B begins.
Transaction A updates docA and docB.
Transaction A commits.
Transaction B updates docA and docC - locks are gained as Transaction A has committed and holds no locks.
Transaction B commits overwriting the work transition A has done on docA.
Here is the example code:
mongoClient = new MongoClient( "localhost" , 27017 );
db = mongoClient.getDB("test");
collection = db.getCollection("testData");
//Create usable Mongo key from key String (i.e {_id:ObjectId("53b4477d44aef43e83c18922")})
String key = "53b4477d44aef43e83c18922";
String key2 = "53bfff9e44aedb6d98a5c578";
ObjectId keyObj = new ObjectId(key);
ObjectId keyObj2 = new ObjectId(key2);
//Set up the transaction
BasicDBObject transaction = new BasicDBObject();
transaction.append("beginTransaction", 1);
transaction.append("isolation", "mvcc");
db.command(transaction);
//Create search query
BasicDBObject query = new BasicDBObject().append("_id",keyObj);
BasicDBObject query2 = new BasicDBObject().append("_id",keyObj2);
//Create set
BasicDBObject set = new BasicDBObject();
set.append("$inc", new BasicDBObject().append("balance",50));
//Run command
collection.update(query, set);
collection.update(query2, set);
//Commit the transactions
BasicDBObject commitTransaction = new BasicDBObject();
commitTransaction.append("commitTransaction", 1);
db.command(commitTransaction);
Is there a check I can do to decide whether or not to commit the transaction? Or is this intended behaviour of TokuMX (or am I doing something wrong)?

TokuMX multi-statement transactions are very similar to MySQL multi-statement transactions. In your example, a document-level lock will be held while the update is happening, so updates to that record will be serialized.
If there is a conflict because two transactions are updating the same document at the same time, the update method will return an error that says there is a lock conflict.
To help you understand what happens, have two threads run this, but have neither commit. You will see one thread wait and eventually time out with a lock timeout error.
Also, if your transaction is a single update, you can just run it, you don't need to wrap it in a transaction. If you want to use a multi-statement transaction, you may want "serializable" isolation rather than MVCC, if you'll be doing reads as part of the transaction: http://docs.tokutek.com/tokumx/tokumx-transactions.html#tokumx-transactions-isolation-serializable Also, you will need to reserve a connection for the transaction, or the connection pool can make your transactions behave improperly: http://docs.tokutek.com/tokumx/tokumx-transactions.html#tokumx-transactions-multi-statement-drivers

Related

JPA : insert and delete object in same transaction

I want to insert an object into database in a transaction and after that object is saved in the database, I'd like to delete that it once a specific operation is done. Can I restart the transaction again and perform deletion and then commit? Is this a correct way of doing it?
Example :
Employee employee = new Employee();
String name = "Ronnie";
entityManager.getTransaction.begin();
employee.setName(name);
entityManager.persist(employee);
entityManager.getTransaction.commit();
//After few steps
entityManager.getTransaction.begin();
entityManager.remove(employee);
entityManager.getTransaction.commit();
SHORT ANSWER: Yes, you can do that whithout problems.
LONG ANSWER: Yes, you can.
Every transaction is independent of any other transaction. So, if you do some operations, commit them (remember, committing a transaction execs the operations in the DB, and closes it), and then reopen it lately, it is independent of the last transaction.
You can even be in the same transaction, whithout closing it, by flushing changes to the DB:
Employee employee = new Employee();
String name = "Ronnie";
entityManager.getTransaction.begin();
employee.setName(name);
entityManager.persist(employee);
entityManager.flush();
//After few steps, the transaction is still the same
entityManager.remove(employee);
entityManager.getTransaction.commit();
The transaction isolate database state from other transactions. So you can insert and delete in the same transaction. no need to commit it.

Java with Hibernate Database

I have code which updates a column in a database which looks like this:
logger.info("Entering Update Method");
Query query =session.createQuery("update CardMaster cm set cm.otpAmount = :otpAmount" + " where cm.cardNumber = :cardnumber");
double otpAmount= cardMaster.getOtpAmount();
String cardNumber=cardMaster.getCardNumber();
query.setParameter("otpAmount",otpAmount);
query.setParameter("cardnumber",cardNumber);
query.executeUpdate();
logger.info("cardMasterUpdated successfully");
In this I am getting otpamount ,cardnumber and it is giving result of executeupdate as 1 but it is not reflecting in Database .. I am opening the session and committing correctly outside.
Instead of using this, if I use update() of hibernate it is happening correctly.
Can you help me out of this?
You have to commit the transaction.
Since you do not commit, nothing is visible for other processes, like the tool you are using to view your DB.
You can acquire your transaction with session.getTransaction(). However, normally you start your transaction manually like this:
Transaction tx = session.beginTransaction();
// Do your stuff
session.flush();
tx.commit()

Lock a set of DB operations in Java

I have a requirement in my Java application to execute a set of DB statements in an atomic & isolated fashion. For example, the application needs to read data rows from one table and update a data row in another table.
QueryRunner queryRunner = new QueryRunner(); // DBUtils query runner
Object[] params = new Object[] { param };
Connection conn = null;
try {
conn = ...; // get connection
conn.setAutoCommit(false);
result = queryRunner.query(conn, "select x, y, z from table1 where column1 = ?", new BeanHandler<SomeBean>(SomeBean.class), params);
// logic to get value for update
queryRunner.update(conn, "update table2 set p = ? where q = ?", some_value, some_id);
conn.commit();
} catch (SQLException e) {
//
} finally {
DBUtils.closeQuietly(conn);
}
The transaction management is achieved by setting auto commit to false for the connection and explicitly commit later on, as shown above. But the above code can also be executed in a multi-thread environment and I also want the two DB statements (select & update) to be run as a whole mutual exclusively.
I have some idea to use a shared Java Lock object in that method, depicted below.
In the class,
private Lock lock = new ReentrantLock(); // member variable
In the method,
lock.lock();
try {
conn = ...; // get connection
conn.setAutoCommit(false);
result = queryRunner.query(conn, "select x, y, z from table1 where column1 = ?", new BeanHandler<SomeBean>(SomeBean.class), params);
// logic to get value for update
queryRunner.update(conn, "update table2 set p = ? where q = ?", some_value, some_id);
conn.commit();
} finally {
DBUtils.closeQuietly(conn);
lock.unlock();
}
It seems sort of capable of solving the issue. However, I am wondering if this is the best practice and is there any better alternative (such as frameworks) for this?
My suggestion is to have the database manage those locks for you instead of your application. This handles the case where there are multiple JVM's running the code. The locking mechanims you mentioned can only be effective in a single JVM.
The way to accomplish this is to do a SELECT ... FOR UPDATE. This will place a lock on the selected rows and the lock will be released when your transaction is committed or rolled back. This is better than a table level lock because those rows can still be read by other transactions that just want to read the current value but not update them. If another transaction tries to obtain a FOR UPDATE lock, then it will block until the first one finishes.
The only way you will achieve the atomicy you are requiring is to use a stored procedure in the database to isolate the data and lock it all at once. The locking at the Java level can't do what the locking in the database can as easily.
Another way you can handle problems like this is to use the serializable transaction isolation level for all of your database transactions. This causes any set of transactions to behave as though they were run one at a time, without actually making them run one at a time. If you do this you should be using a framework which catches serialization failures (SQLState 40001) and retries the transactions. The big up-side is that you don't need to worry about particular interactions among transactions -- if a transaction does the right thing when it is the only thing running, it will do the right thing in any transaction mix.
Note that all transactions must be serializable for this to work so simply.
From my understanding, do you just want to make that block of code containing select and update statements as Thread safe? That's what synchronized keyword is used for. Even though this question is asked long back i just want to make a note of it here. put those lines of code under synchronized block.

using Hibernate to loading 20K products, modifying the entity and updating to db

I am using hibernate to update 20K products in my database.
As of now I am pulling in the 20K products, looping through them and modifying some properties and then updating the database.
so:
load products
foreach products
session begintransaction
productDao.MakePersistant(p);
session commit();
As of now things are pretty slow compared to your standard jdbc, what can I do to speed things up?
I am sure I am doing something wrong here.
The right place to look at in the documentation for this kind of treatment is the whole Chapter 13. Batch processing.
Here, there are several obvious mistakes in your current approach:
you should not start/commit the transaction for each update.
you should enable JDBC batching and set it to a reasonable number (10-50):
hibernate.jdbc.batch_size 20
you should flush() and then clear() the session at regular intervals (every n records where n is equal to the hibernate.jdbc.batch_size parameter) or it will keep growing and may explode (with an OutOfMemoryException) at some point.
Below, the example given in the section 13.2. Batch updates illustrating this:
Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
ScrollableResults customers = session.getNamedQuery("GetCustomers")
.setCacheMode(CacheMode.IGNORE)
.scroll(ScrollMode.FORWARD_ONLY);
int count=0;
while ( customers.next() ) {
Customer customer = (Customer) customers.get(0);
customer.updateStuff(...);
if ( ++count % 20 == 0 ) {
//flush a batch of updates and release memory:
session.flush();
session.clear();
}
}
tx.commit();
session.close();
You may also consider using the StatelessSession.
Another option would be to use DML-style operations (in HQL!): UPDATE FROM? EntityName (WHERE where_conditions)?. This the HQL UPDATE example:
Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
String hqlUpdate = "update Customer c set c.name = :newName where c.name = :oldName";
// or String hqlUpdate = "update Customer set name = :newName where name = :oldName";
int updatedEntities = s.createQuery( hqlUpdate )
.setString( "newName", newName )
.setString( "oldName", oldName )
.executeUpdate();
tx.commit();
session.close();
Again, refer to the documentation for the details (especially how to deal with the version or timestamp property values using the VERSIONED keyword).
If this is pseudo-code, I'd recommend moving the transaction outside the loop, or at least have a double loop if having all 20K products in a single transaction is too much:
load products
foreach (batch)
{
try
{
session beginTransaction()
foreach (product in batch)
{
product.saveOrUpdate()
}
session commit()
}
catch (Exception e)
{
e.printStackTrace()
session.rollback()
}
}
Also, I'd recommend that you batch your UPDATEs instead of sending each one individually to the database. There's too much network traffic that way. Bundle each chunk into a single batch and send them all at once.
I agree with the answer above about looking at the chapter on batch processing.
I also wanted to add that you should make sure that you only load what is neccessary for the changes that you need to make for the product.
What I mean is, if the product eagerly loads a large number of other objects that are not important for this transaction, you should consider not loading the joined objects - it will speed up the loading of products and depending on their persistance strategy, may also save you time when making the product persistent again.
The fastest possible way to do a batch update would be to convert it to a single SQL statement and execute it as raw sql on the session. Something like
update TABLE set (x=y) where w=z;
Failing that you can try to make less transactions and do updates in batches:
start session
start transaction
products = session.getNamedQuery("GetProducs")
.setCacheMode(CacheMode.IGNORE)
.scroll(ScrollMode.FORWARD_ONLY);
count=0;
foreach product
update product
if ( ++count % 20 == 0 ) {
session.flush();
session.clear();
}
}
commit transaction
close session
For more information look at the Hibernate Community Docs

Hibernate transaction duplicate problem

I want update some of my table in database and want all of these work do in 1 transaction,
first of all I delete some entry in branchbuildin(Table) and Insert new one after this action
The problem occurred when I insert and entry with same buildingname and branch_fk (be cause I have this constraint on this table ( uniqueConstraints={#UniqueConstraint(columnNames={"buildingname","branch_fk"})})) but when I don't use hibernate session and use normal JDBC transaction I don't have these problem.
List<Integer> allBranchBuilding = branchBuildingDao.getAllBranchBuildingID(pkId, sess);
for (Integer integer : allBranchBuilding) {
branchBuildingDao.delete(integer, sess); // delete kardane tamame BranchBuilding ha va tel haie aanha
}
Address myAdr = new Address();
setAddress(myAdr, centralFlag, city, latit, longit, mainstreet, remainAdr, state);
BranchBuildingEntity bbe = new BranchBuildingEntity();
setBranchBuildingEntity(bbe, be, myAdr, city, centralFlag, latit, longit, mainstreet, buildingName, remainAdr, state, des);
branchBuildingDao.save(bbe, sess);//Exception Occurred
I get my session at the first of Method:
Session sess = null;
sess = HibernateUtil.getSession();
Transaction tx = sess.beginTransaction();
You're right, everything occurs in the same transaction, and the same Hibernate Session.
The Session keeps track of every entity it manages. Even though you asked to delete it in the database, the corresponding object is still memorised in the Session until the Session is terminated.
In general, it is possible that
Hibernate reorders your operations
when sending them to the database, for
efficiency reasons.
What you could do is flush (ie. send to the database) your transaction because the save (if needed, you could also clear - ie empty the entities memorized by the Session - it after flushing):
sess.flush();
// sess.clear(); // if needed or convenient for you
branchBuildingDao.save(bbe, sess);
Note also that while your entities are memorized by the session, modifying them will trigger an automatic update when closing the session.
In our project, we have a method that deletes efficiently a collection (and another for an array, declared using the convenient ... parameter syntax) of entities (it works for all entities, it doesn't have to be done for each entity), removing them out of the session at the same time, and taking care of the flushing before :
Loop on all entities, delete it (using sess.delete(e)) and add it to a 'deleteds' list.
Every 50 entities (corresponding to the batch size we configured for efficiency reasons) (and at the end) :
flush the Session to force Hibernate to send immediately the changes to the database,
loop on 'deleteds' list, clear each entity from the Session (using sess.evict(e)).
empty the 'deleteds'list.
Don't worry, flush only sends the SQL to the database. It is still subject to commit or rollback.

Categories

Resources