I need to have a atomic counter with ObjectDB but the following code doesn't work as I expected:
final EntityManagerFactory emf = Persistence.createEntityManagerFactory("test.odb");
EntityManager em = emf.createEntityManager();
Point p = new Point(0, 0);
em.getTransaction().begin();
em.persist(p);
em.getTransaction().commit();
em.close();
final CountDownLatch l = new CountDownLatch(100);
for (int i = 0; i < 100; i++) {
Thread t = new Thread(new Runnable() {
#Override
public void run() {
EntityManager em = emf.createEntityManager();
em.getTransaction().begin();
//Query q = em.createQuery("UPDATE Point SET x = x + 1");
Query query = em.createQuery("UPDATE Point SET x = x + 1");
query.executeUpdate();
em.getTransaction().commit();
em.close();
l.countDown();
}
});
t.start();
}
l.await();
em = emf.createEntityManager();
TypedQuery<Point> myquery = em.createQuery("SELECT p from Point p", Point.class);
List<Point> results = myquery.getResultList();
System.out.println("X coordiate is: " + results.get(0).getX());
em.close();
It should have printed out X coordinate is 100. But in reality, it doesn't.
What is wrong with my code?
You can fix your code by one of the following ways:
Synchronize your update queries, so they will be executed sequently, rather than concurrently:
synchronized (lock) {
em.createQuery("UPDATE Point SET x = x + 1").executeUpdate();
}
Your lock object must be one object that is shared by all the threads.
Or, use ObjectDB / JPA locking, by setting a pessimistic locking timeout, e.g. by:
Map<String, Integer> properties =
Collections.singletonMap("javax.persistence.lock.timeout", 1000);
EntityManagerFactory emf =
Persistence.createEntityManagerFactory(
"objectdb:$objectdb/db/test.tmp;drop", properties);
and then replacing the UPDATE query with a retrieval with a lock and an update:
Point point = em.find(Point.class, 1, LockModeType.PESSIMISTIC_WRITE);
point.setX(point.getX() + 1);
Better create a DAO class for Point and wrap the persist(), merge() ... functions with synchronized functions instead.
Alternatively an AtomicInteger with getAndIncrement() should solve your problem without the need for synchronization and locks.
Related
I have 2 bugs rarely happening during last 3 years.
If I have 100 orders during a day 1-2 orders have alerts when manual db check says that number was not incremented, !! but when i check db manually it is really incremented.
If I have 3000 orders during a month 3-5 orders have alerts when lock is not released from order after order completion. But when I check db manually it is not null when it should be null
I am using only jdbcTemplate and transactional template(select, update, read). I am using JPA only when insert a model to mysql.
Everything is done with lock by 1 thread.
Code snippet to show the issue:
public synchronized void test() {
long payment = 999;
long bought_times_before = jdbcTemplate.queryForObject("select bought_times from user where id = ?", new Object[]{1}, Long.class);
TransactionTemplate tmpl = new TransactionTemplate(txManager);
tmpl.setTimeout(300);
tmpl.setName("p:" + payment);
tmpl.executeWithoutResult(status -> {
jdbcTemplate.update("update orders set attempts_to_verify = attempts_to_verify + 1, transaction_value = null where id = ?", payment);
jdbcTemplate.update("update orders set locked = null where id = ?", payment);
jdbcTemplate.update("update user set bought_times = bought_times + 1 where id = 1");
});
long bought_times_after = jdbcTemplate.queryForObject("select bought_times from user where id = ?", new Object[]{1}, Long.class);
if (bought_times_after <= bought_times_before) log.error("bought_times_after <= bought_times_before");
}
I upgraded mysql and implemented redis distributed lock to allow only 1 thread run code with select and transaction and select.
UPDATE:
default isolation level is read comited
i tried serializable but it still has the same bug
UPDATE 2:
re: lock != null after transaction it is somehow related to high load on mysql, since it is never occur when low load.
UPDATE 3:
i checked mysql logs - nothing, no errors
also i tried to use REQUIRED_NEW + SERIALIZABLE but received dead locks
UPDATE 4:
i wrote a test and cannot reproduce the issue - but on production there are more than 1 transaction as well as more updates and reads but i guess it is hardware issue or mysql bug
#PostConstruct
public void test(){
jdbcTemplate.execute("CREATE TEMPORARY TABLE IF NOT EXISTS TEST ( id int, name int, locked boolean )");
jdbcTemplate.execute("insert into TEST values(1, 1, 1);");
for(int i = 0; i < 100000; i++) {
long prev = jdbcTemplate.queryForObject("select name from TEST where id = 1", Long.class);
TransactionTemplate tmpl = new TransactionTemplate(txManager);
jdbcTemplate.update("update TEST set locked = true where id = 1;");
tmpl.execute(new TransactionCallbackWithoutResult() {
#SneakyThrows
#Override
protected void doInTransactionWithoutResult(org.springframework.transaction.TransactionStatus status) {
jdbcTemplate.update("update TEST set name = name + 1 where id = 1;");
jdbcTemplate.update("update TEST set locked = false where id = 1;");
}
});
long curr = jdbcTemplate.queryForObject("select name from TEST where id = 1", Long.class);
boolean lock = jdbcTemplate.queryForObject("select locked from TEST where id = 1", Boolean.class);
if(curr <= prev){
log.error("curr <= prev");
}
if(lock){
log.error("lock = true");
}
}
}
UPDATE 5: WAS ABLE TO REPRODUCE IT!!!!
#PostConstruct
public void test(){
jdbcTemplate.execute("CREATE TEMPORARY TABLE IF NOT EXISTS TEST ( id int, name int, locked boolean )");
jdbcTemplate.execute("insert into TEST values(1, 1, 1);");
ExecutorService executorService = Executors.newFixedThreadPool(100);
for(int i = 0; i < 100000; i++) {
executorService.submit(() -> {
RLock rLock = redissonClient.getFairLock("lock");
try {
rLock.lock(120, TimeUnit.SECONDS);
long prev = jdbcTemplate.queryForObject("select name from TEST where id = 1", Long.class);
TransactionTemplate tmpl = new TransactionTemplate(txManager);
jdbcTemplate.update("update TEST set locked = true where id = 1;");
tmpl.execute(new TransactionCallbackWithoutResult() {
#SneakyThrows
#Override
protected void doInTransactionWithoutResult(org.springframework.transaction.TransactionStatus status) {
jdbcTemplate.update("update TEST set name = name + 1 where id = 1;");
jdbcTemplate.update("update TEST set locked = false where id = 1;");
}
});
long curr = jdbcTemplate.queryForObject("select name from TEST where id = 1", Long.class);
boolean lock = jdbcTemplate.queryForObject("select locked from TEST where id = 1", Boolean.class);
if (curr <= prev) {
log.error("curr <= prev");
}
if (lock) {
log.error("lock = true");
}
} finally {
rLock.unlock();
}
});
}
}
UPDATE 7: after the second and third run i cannot reproduce it again neither with Lock nor with FairLock ..
UPDATE 8: on prod i am using 3 redis lock with 120 sec timeouts so i think there is timeout occurs rarely on 1 of 3 lock thus code might be executed by 2 threads without lock
SOLUTION: increase lock timeout as well as transaction timeout up to 500 seconds
UPDATE 9: looks like the issue has been resolved but i need to monitor it during couple of weeks before close the issue on stack overflow
Session session = sessionFactory.openSession();
Transaction transaction = session.beginTransaction();
Query updateQuery = session.createQuery("UPDATE Lot l SET l.currentRate = l.currentRate + 100, l.lastOwner = :lastowner WHERE l.id = :lotid", null);
updateQuery.setLockMode(LockModeType.PESSIMISTIC_WRITE);
So, here is my code fragment. I got an exception: java.lang.IllegalStateException: Expecting a SELECT query : UPDATE Lot l SET l.currentRate = l.currentRate + 100, l.lastOwner = :lastowner WHERE l.id = :lotid when I try to call setLockMode.
Why? Am I doing something wrong?
See the Java doc of the method you are trying to call: https://docs.oracle.com/javaee/6/api/javax/persistence/Query.html#setLockMode(javax.persistence.LockModeType)
It explicitly says, that an IllegalStateException will be thrown if you execute this on a non-SELECT statement.
I'm currently using Hibernate 6 and H2. I want to safely increment count field of Entity class but using more then 1 thread per time just to make sure that transaction is actually locking my entity. But when I ran this code, result count column in H2 wasn't 10, but instead some random number under 10. What am I missing about pessimistic locking?
for (int a = 0; a < 5; a++) {
executorService.execute(() -> {
Session innerSession = sessionFactory.openSession();
Transaction innerTransaction = innerSession.beginTransaction();
Entity entity = innerSession.get(Entity.class, id, LockMode.PESSIMISTIC_WRITE);
entity.setCount(entity.getCount() + 1);
innerSession.flush();
innerTransaction.commit();
innerSession.close();
});
executorService.execute(() -> {
Session innerSession = sessionFactory.openSession();
Transaction innerTransaction = innerSession.beginTransaction();
Entity entity = innerSession.get(Entity.class, id, LockMode.PESSIMISTIC_WRITE);
entity.setCount(entity.getCount() + 1);
innerSession.flush();
innerTransaction.commit();
innerSession.close();
});
}
Entire method:
Long id;
SessionFactory sessionFactory;
Session session;
Transaction transaction;
ExecutorService executorService = Executors.newFixedThreadPool(4);
Properties properties = new Properties();
Configuration configuration = new Configuration();
properties.put(AvailableSettings.URL, "jdbc:h2:tcp://localhost/~/test");
properties.put(AvailableSettings.USER, "root");
properties.put(AvailableSettings.PASS, "root");
properties.put(AvailableSettings.DIALECT, H2Dialect.class.getName());
properties.put(AvailableSettings.SHOW_SQL, true);
properties.put(AvailableSettings.HBM2DDL_AUTO, Action.CREATE.getExternalHbm2ddlName());
// classes are provided by another library
entityClasses.forEach(configuration::addAnnotatedClass);
sessionFactory = configuration.buildSessionFactory(new StandardServiceRegistryBuilder().applySettings(properties).build());
session = sessionFactory.openSession();
transaction = session.beginTransaction();
// initial value of count field is 0
id = (Long) session.save(new Entity());
transaction.commit();
for (int a = 0; a < 5; a++) {
executorService.execute(() -> {
Session innerSession = sessionFactory.openSession();
Transaction innerTransaction = innerSession.beginTransaction();
Entity entity = innerSession.get(Entity.class, id, LockMode.PESSIMISTIC_WRITE);
entity.setCount(entity.getCount() + 1);
innerSession.flush();
innerTransaction.commit();
innerSession.close();
});
executorService.execute(() -> {
Session innerSession = sessionFactory.openSession();
Transaction innerTransaction = innerSession.beginTransaction();
Entity entity = innerSession.get(Entity.class, id, LockMode.PESSIMISTIC_WRITE);
entity.setCount(entity.getCount() + 1);
innerSession.flush();
innerTransaction.commit();
innerSession.close();
});
}
executorService.shutdown();
executorService.awaitTermination(5, TimeUnit.SECONDS);
session.clear(); // prevent reading from cache
System.out.println(session.get(Entity.class, id).getCount()); // printed result doesn't match 10, same for reading from H2 browser interface
session.close();
Answer was simple, I need just to upgrade version of hibernate to 6.0.0.Alpha9. Higher versions requires 11 java to compile (I'm using 8). Seems like it was a bug in 6.0.0.Alpha6, which I used previously. There was no problem with H2 1.4.200. From hibernate sql logs I understood that the main problem in 6.0.0.Alpha6 was incorrect select query for transaction with pessimistic lock, it was just regular select, but in 6.0.0.Alpha9 already used select for update, which prevents other transactions from reading this row.
i am struggling with hibernate prepared statement count.
I am using the following JPA criteria query:
int count = 30
EntityManager manager = ...
CriteriaBuilder builder = manager.getCriteriaBuilder();
CriteriaQuery<String> select = builder.createQuery(String.class);
Root<AdministrationParameter> root = select.from(AdministrationParameter.class);
select.select(root.get(AdministrationParameter_.value));
ParameterExpression<String> peF1 = builder.parameter(AdministrationParameter_.context.getBindableJavaType(), "f1");
ParameterExpression<String> peF2 = builder.parameter(AdministrationParameter_.parameter.getBindableJavaType(), "f2");
Predicate p1 = builder.equal(root.get(AdministrationParameter_.context), peF1);
Predicate p2 = builder.equal(root.get(AdministrationParameter_.parameter), peF2);
select.where(p1, p2);
List<String> results = Collections.emptyList();
TypedQuery<String> query = manager.createQuery(select);
for (int i = 0; i < count; i++) {
query.setParameter(peF1, administrationParameterTypeInterface.getContext());
query.setParameter(peF2, administrationParameterTypeInterface.getParameter());
query.getResultList();
}
The count variable is to execute the query n times, e.g. to run a db trace in background (the query is executed against a db2 database).
Assume count = 30
The db2 trace says, there are "30 prepares" and "30 describes", the "statement found count = 30".
Hibernate give me the same values:
EntityManagerFactory factory = ...;
SessionFactory sessionFactory = factory.unwrap(SessionFactory.class);
statistics = sessionFactory.getStatistics();
Statistics statistics = statistics.setStatisticsEnabled(true);
...running the query above...
System.out.println("prepared statement count: " + statistics.getPrepareStatementCount());//is 30
System.out.println("query cache hit count: " + statistics.getQueryCacheHitCount());//0
System.out.println("query cache miss count: " + statistics.getQueryCacheMissCount());//0
System.out.println("query execution count: " + statistics.getQueryExecutionCount());//30
According to the javadoc https://docs.jboss.org/hibernate/orm/3.2/api/org/hibernate/stat/Statistics.html the statistics.getPrepareStatementCount() is "The number of prepared statements that were acquired".
Shouldn't it be 1?
Which Hibernate version are you using? This might be a bug that has already been fixed in newer versions. If updating doesn't help please create an issue in the issue tracker(https://hibernate.atlassian.net) with a test case(https://github.com/hibernate/hibernate-test-case-templates/blob/master/orm/hibernate-orm-5/src/test/java/org/hibernate/bugs/JPAUnitTestCase.java) that reproduces the issue.
I use OpenJPA 2.2.0 on WebSphere Application Server 8 with a MySQL 5.0 DB.
I have a list of objects which I want to merge into the DB.
it's like:
for (Object ob : list) {
Long start = Calendar.getInstance().getTimeInMillis();
em = factory.createEntityManager();
em.getTransaction().begin();
em.merge(ob);
em.getTransaction().commit();
em.close();
Long end = Calendar.getInstance().getTimeInMillis();
Long diff = end - start;
LOGGER.info("Time: " + diff);
}
When I run this loop I need about 300-600 Milliseconds to merge one object. When I delete the line "em.merge(ob);" then I need "0" Milliseconds to iterate over 1 List Object.
So my question is: What can I do to improve the time to merge one object?
Thanks!
You can try starting the transaction before iteration & then commiting it afterwards within a single transaction. So, basically you are creating a batch which would be merged/persisted on commit.
Also, you can limit the number of objects in a batch to be processed at a time & can explicitly flush the changes into database.
Here, you are initiating a transaction & commiting it in each iteration and also creating/closing entity manager each time, will affect performance for numerous data.
It will be something like below code.
em = factory.createEntityManager();
em.getTransaction().begin();
int i = 0;
for (Object ob : list) {
Long start = Calendar.getInstance().getTimeInMillis();
em.merge(ob);
Long end = Calendar.getInstance().getTimeInMillis();
Long diff = end - start;
LOGGER.info("Time: " + diff);
/*BATCH_SIZE is the number of entities
that will be persisted/merged at once */
if(i%BATCH_SIZE == 0){
em.flush();
em.clear();
}
i++;
}
em.getTransaction().commit();
em.close();
Here, you can also rollback the whole transaction if any of the object fails to persist/merge.