I was curious whether spring jparepository methods are thread safe and then I read the stackflow article (Is a Spring Data (JPA) Repository thread-safe? (aka is SimpleJpaRepository thread safe)). From there, I understood that repository methods are thread safe and then I made one POC to test the thread safety. I made one repository say FormRepository to do CRUD operations for 'form' entity, that is extending the JpaRepository. From DAO, I simply invoked 100 threads making the form object and manually setting its id and then saving the 'form' object.
Below is the code for reference:-
#Repository
public interface FormRepository extends JpaRepository<Tbldynamicform, Long> {
Tbldynamicform save(Tbldynamicform tblform);
#Query("SELECT max(tblform.formid) FROM Tbldynamicform tblform")
Optional<Integer> findMaxId();
}
......End of Repository above and start of DAO below...
#Component
public class DynamicFormDAO implements DynamicFormDAO {
#Inject
private FormRepository formRepository;
public void testThreadSafety() throws Exception {
List<Callable<Integer>> tasks = new ArrayList<>(100);
for (int i = 0; i < 100; i++) {
tasks.add(() -> {
try {
Tbldynamicform tbldynamicform = new Tbldynamicform();//Set all the required fields for form
if (tbldynamicform.getFormid() == null)
tbldynamicform.setFormid(findFormID());
Tbldynamicform form = formRepository.save(tbldynamicform);
return form.getFormid();
} catch (Exception e) {
e.printStackTrace();
}
return null;
});
}
ExecutorService executor = Executors.newFixedThreadPool(100);
executor.invokeAll(tasks);
}
private int findFormID() throws Exception {
Optional<Integer> id = formRepository.findMaxId();
if (id != null && id.isPresent() && id.get() != null) {
int generatedId = id.get().intValue();
return ++generatedId;
}
return 0;
}
}
When I do this, I was assuming that things have to work fine because the form repository methods are thread safe but somehow I am getting the sql dataintegrityviolationexception several times in logs making the insertion of several records failure. Below error for reference:-
org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint ["PRIMARY KEY ON PUBLIC.TBLDYNAMICFORM(FORMID)"; SQL statement:
insert into Tbldynamicform (clientid, copyfromexisting, creationdate, formdesc, formmode, formname, formtemplate, formtitle, procutype, status, formid) values (?, ?, ?, ?,...
This has made me to think whether this is the problem of thread safety or some other problem? In my understanding, all the 'tbldynamicform' objects I created in my dao will remain on thread stack. Only the formRepository will be on heap storage and if the formrepository methods are thread safe, 100 records has to be inserted in database without any problem.
If I do the setId and save in synchronized block, everything works ok but that's not my intention and not required if the repository methods are thread safe.
Experts, any help please?
Your saving task is not atomic - two threads might fetch the same maximum id before one of them saved the new entity.
And then, even if the save method of the repository is thread - safe, it wont help.
The maxId is thread safe, the save is thread safe, but your method inside the runnable of each thread is not thread safe.
Simply put, yes, it is threadsafe, but your database is also stateful (obviously) and for integrity to be maintained you may need things like a locking strategy (hold locks to make things synchronous, or use an optimistic strategy and retry where required). As someone has noted in another answer, if you simply used a different method of generating an ID (check out SUID) you code would work fine.
The problem comes from how you retrieve the last ID with findFormID(), it doesn't work in a concurrent context.
What if two threads ask an ID at the same time ? They will retrieve the same ID and create two objects with the same ID. here is your problem.
Some integrated solutions for generated IDs already exist and you should not try to implement your own unless you know what you do.
Related
Given the following code:
public class OrderService {
#PersistanceContext
private EntityManager entityManager;
#Transactional
public void updateOrder(long orderId, OrderDTO updatedOrder) {
Order order = entityManager.find(Order.class, orderId);
if (order != null) {
order.setName(updated.getName());
} else {
throw new EntityNotFoundException(Order.class, orderId);
}
}
}
I was asked to point out all the queries that are executed when the updateOrder method is called including transactional sentences.
My answer was 1 query, the one that retrieves the order by calling entityManager.find(Order.class, orderId) however it seems that is not correct. How is that even possible? I do see the setName method is called on the order but there is not a call to save that order back to the database.
Is there any documentation that explains how this works or any way to see all the sentences executed in that transaction?
When you call find() method,your object becames in persistent state. Hibernate will detect any changes made to an object in persistent state and synchronize the state with the database when the unit of work completes. You can read about object states : https://docs.jboss.org/hibernate/orm/3.3/reference/en/html/objectstate.html
The answer is it depends, the first one for sure is entityManager.find(...) which does a select. And if it finds a record, you are setting a new name(setName(...)) for which hibernated detects the object as dirty. So that it will flush the new data to db. Hence, as a second call save(...) will be triggered. Check here
I have a table products. In this table I need is_active with constraint - only one row with the same type can be true.
I have service for saving new Product with checking:
#Service
public class ProductServiceImpl implements ProductService {
private final ProductRepository productRepository;
public ProductServiceImpl(ProductRepository productRepository) {
this.productRepository = productRepository;
}
#Override
public void save(Product product) {
Product productInDb = productRepository.findOneByTypeAndIsActive(product.getType());
if (productInDb != null)
throw new AlreadyActiveException();
product.setActive(true);
productRepository.saveAndFlush(product);
}
}
When I call save method in a few threads and try to check active product - in both threads findOneByTypeAndIsActive methods return productInDb is null because I haven't active products in the table.
In each thread I set product.setActive(true); and try to save in DB.
If I don't have the constraint in DB - I save both products in is_active = true state and this checking not performed:
if (productInDb != null)
throw new AlreadyActiveException();
My question - Can I fix this without adding of constraint in DB?
And checking above is useless?
From my point of view this is not the best db tables design to have is_active flag in record structure in pair with restriction, that only one record in table can be is_active at same time.
You have to use database schema constraints or you have to lock whole table with all records. How to lock whole table for modification is database specific. I don't think JPA natively supports such locks.
But you wrote:
Can I fix this without adding of constraint in DB?
No, it is not possible with strict guarantee for all clients.
But if you have only one application that uses this table - you can use local, application specific locks, for example you can create Read/Write java locks on #Service level.
Your operation consists of 2 actions:
Get an entity from DB
Save a new entity if it doesn't exist
Your problem is that few threads can start this operation at the same time and don't see changes of each other. That's definitely not what you need. Your operation which consists of a few actions must be atomic.
If I understand correct you have a rule to keep only 1 active product of the same type in datastore. It sounds like a data consistency requirement, which should be solved on application level.
The most naive option to solve your problem is to acquire a lock before performing your operation. It may be solved either with synchronised or explicit lock:
#Override
public synchronised void save(Product product) {
Product productInDb = productRepository.findOneByTypeAndIsActive(product.getType());
if (productInDb != null)
throw new AlreadyActiveException();
product.setActive(true);
productRepository.saveAndFlush(product);
}
Hi I'm struggling with Optimistick Lock on JPA2 and I have no more ideas why is it occurring.
My case is that I'm running multiple threads but there is one entity in the DB which stores progress. Which means that different threads are trying to update this entity during execution to make possible to see the progress by user.
I have a methods addAllItems and addDone. Both of methods are used to update the entity by several threads and I'm displaying the result by showing (done/allItems)*100.
Methods were simple at the beginning
#Transactional(propagation=Propagation.REQUIRES_NEW)
public void addAllItems(Long id, Integer items){
Job job = jobDao.findById(id);
job.setAll(job.getAll() + items);
jobDao.merge(job);
}
#Transactional(propagation=Propagation.REQUIRES_NEW)
public void addDone(Long id, Integer done){
Job job = jobDao.findById(id);
job.setDone(job.getDone() + done);
jobDao.merge(job);
}
When I realized that Optimistic Lock is occurring I changed both methods by adding synchronized to the signature. It has no effect so I added refresh (from entity manager) to make sure that I'm having current version. It also made no difference. I also added manual flush at the end, but still nothing better...
Here is final version of method (addAllItems is pretty much the same, only difference is in setter):
#Transactional(propagation=Propagation.REQUIRES_NEW)
public synchronized void addDone(Long id, Integer done){
Job job = jobDao.findById(id);
job = jobDao.refresh(job);
job.setDone(job.getDone() + done);
jobDao.merge(job);
jobDao.flush();
}
Where the jobDao.refresh method is just calling refresh on entityManager.
I'm using eclipselink 2.40.
What else can I check?
I'm out of ideas at the moment...
As you are sure that you are using a proxy (I assume you correctly configured the PlatformTransactionManager), you could try to use an explicit pessimistic lock inside the transaction - normally you should not need to do so, but if it fixes the problem ...
I suppose that in your dao, you have something like :
Job job = EntityManager.find(Job.class, jobId);
To force a pessimistic lock, just change it to :
Job job = EntityManager.find(Job.class, jobId, LockModeType.PESSIMISTIC_WRITE);
As you just do load, modify, store and commit, it might be a correct use case for pessimistic locking.
I have the following problem. I have 10 Threads which create objects that are inserted in the database. Each Thread has a ThreadLocal and its own session. All objects are inserted together, after they were created. These objects have a column which is marked as unique. However, I have the problem, that it can happen that two different threads create the same object. This behaviour is wanted but I don't know how I can insert them into my database.
Currently, each thread queries all objects that are inserted in the database, checks on the queried objects if they exist or not and inserts the non-existing objects into the database. However, as it can happen that the object did not exist on the query of all objects, I get a ConstraintViolationException when I insert the objects and they were already added by another Thread. However, doing a database (or cache) query for each object has to bad performance, as we are trying to add 1000 objects per thread and minute. If I try to flush the database after each single insert, then I get the following error: Deadlock found when trying to get lock; try restarting transaction
So my question is: How can I insert objects, that have a unique constraint from different threads simultanously.
//Edit: currently I'm using Hibernate with MYSQL InnoDB
//Edit2: Finally, the code which I use to write a single item.
public class ItemWriterRunnable implements Callable<Object> {
private final ThreadLocal<Session> session = new ThreadLocal<Session>();
private Item item;
public ItemWriterRunnable(Item item) {
super();
this.item= item;
}
protected Session currentSession() {
Session s = this.session.get();
// Open a new Session, if this thread has none yet
if (s == null || !s.isOpen()) {
s = HibernateUtils.getSessionFactory().openSession();
// Store it in the ThreadLocal variable
this.session.set(s);
}
return s;
}
#Override
public Object call() throws Exception {
Session currentSession = currentSession();
try {
currentSession.beginTransaction();
currentSession.save(this.item);
currentSession.getTransaction().commit();
} catch (ConstraintViolationException e) {
currentSession.getTransaction().rollback();
} catch (RuntimeException e) {
currentSession.getTransaction().rollback();
} finally {
currentSession.close();
currentSession = null;
this.session.remove();
}
return null;
}
}
Best regards,
André
If you write multiple objects in a thread, and one of them fails because it's a duplicate, then you'll have to work out which one was the duplicate, remove it from the set, and retry writing it to the DB (with a change of another failure). This takes a lot of time. Alternatively, you could read the DB to see if there are any duplicates before writing the set, and remove the duplicates before writing. This read/check/write pattern is flawed if it is not contained within a synchronised block, because other threads could write duplicates between the steps. The synchronisation needed to fix this will stall your server on every write, pausing all existing threads, potentially harming performance.
Instead, spawn a thread per object, and write the object within this thread (without the read/check). Most objects will write without issue, because most objects are not duplicated (an assumption, but it's probably right). Objects that are duplicates will fail with an exception, at which point you can terminate that thread because the relevant work is already done.
I am in the midst of changing my JPA code around to make use of threads. I have a separate entity manager and transaction for each thread.
What I used to have (for the single threaded environment) was code like:
// get object from the entity manager
X x = getObjectX(jpaQuery);
if(x == null)
{
x = new X();
x.setVariable(foo);
entityManager.persist(x);
}
With that code in the multi threaded environment I am getting duplicate keys since, I assume, getObjectX returns null for a thread, then that thread is swapped out, the next thread calls getObjextX, also getting null, and then both threads will create and persist a new X().
Short of adding in synchronization, is there an atomic way to get/save-if-doesn't-exist a value with JPA or should I rethink my approach
EDIT:
I am using the latest Eclipselink and MySql 5.1
EDIT 2:
I added synchronization... MASSIVE performance hit (to the point that it cannot be used). Going to gather all of the data up back on the main thread and then do the creations on that thread.
Short sad answer is no the JPA API cannot do that for you. The whole API is more or less built around the optimistic principle of assuming things are going to work and throwing exceptions in the event of concurrent modification.
If this is happening often, there's likely some other component (whatever generates foo?) that could benefit from being made threadsafe, as perhaps an alternative to synchronizing around the query+create.
Some "hack" to consider:
implement hashCode() and equals() based on the business key of the objects (not the generated id)
synchronize on:
(obj.getClass().getName() + String.valueOf(hashCode())).intern()
Thus you will get locks only in the relevant cases.
I think you will need to add a unique constraint on the fields that are used in "jpaQuery" so that the database can not create duplicate rows with the same criteria used in the contraints for that query. The calling code will need to trap the resulting exception that occurs as a result of the constraint violation (ideally it will be an EntityExistsException, but the spec is not clear on this case).
Are you sure you need multiple entitymanagers? In a similar situation, I just use one entitymanager and simple per-method lock objects:
private Object customerLock = new Object[0];
public Customer createCustomer(){
Customer customer = new Customer();
synchronized(customerLock){
entityManager.persist(customer);
}
return customer;
}
Edit: OK, can't do much about performance except saying that it performs ok in my apps, but for uniqueness use something like this:
public Customer getOrCreateCustomer(String firstName, String lastName){
synchronized(customerLock){
List<Customer> customers =
entityManager.createQuery(
"select c from Customer c where c.firstName = :firstName"
+ " and c.lastName = :lastName"
)
.setParam("firstName", firstName)
.setParam("lastName", lastName)
.setMaxResults(1)
.getResultList();
if(customers.isEmpty()){
Customer customer = new Customer(firstName, lastName);
entityManager.persist(customer);
}else{
customer = customers.get(0);
}
}
return customer;
}