Develop Reference

Multiple entityManager in Spring application. Persistence of duplicate objects issue

My Spring component gets a request from a client, asks a web-service about some data and saves received objects to a database.
I identify all objects and save only new ones.
The issue occurs when the client makes two or more same requests in the same time (or due to even different user requests I receive same objects from web-service).
To describe the issue with persistence here some details. For each client request my component starts execution in a separate thread, I get a new entityManager, begin a transaction, receive a data from web-service, then I identify objects and persist new ones using given entityManager in a current transaction.
If in separate transactions I receive the same objects from web-service and if they are new ones that are not yet in database I am not able to identify them in not-commited transactions and so they are persisted in all transactions. Then all duplicate objects will be commited and saved to database.
What could be good solutions in this case? Is there any way to identify new objects properly even in different transactions? Or what approaches can be applied?
May be Spring provides some approaches to manage transactions or entityManagers so that it can help with this issue...
Note. Of course I can use database instruments to avoid saving duplicate objects but in this case it is not a very good solution.

Check if objects are present in a database before saving.
Use #UniqueConstraint or #Column(unique = true) to prevent duplicate rows, handle exceptions appropriately.
Use #Version to manage concurrent modification for existing entities. More about optimistic and pesimistic locking: Chapter 5. Locking. Related discussions: Hibernate Automatic Versioning and When to use #Version and #Audited in Hibernate?
You may use thread locks / synchronization mechanisms to ensure that requests for the same user will happen in order. However, this won't work if your service in running on more than 1 node.

So the solution in my case is the following:
Make transactions pretty small and commit every object separately.
Make unique constraints in database to prevent duplicating of
objects. This point will not help us a lot but needed for point 3.
Every commit() method we insert in try-catch block. If we try to
commit duplicate object in parallel transactions then we will receive an exception and in catch block we can check the database, select the object that is already there and work with it futher.
The example:
boolean reidentifyNeed = false;
try {
DofinService.getEntityManagerThreadLocal().getTransaction().begin();
DofinService.getEntityManagerThreadLocal().persist(entity);
try {
DofinService.getEntityManagerThreadLocal().getTransaction().commit();
//if commit is successfull
entityIdInDB = (long) entity.getId();
DofinService.getEntityManagerThreadLocal().clear();
} catch (Exception ex) {
logger.error("Error committing " + entity.getClass().getSimpleName() + " in DB. Possibly duplicate object. Will try to re-identify object. Error: " + ex.toString());
reidentifyNeed = true;
}
if(reidentifyNeed){
//need clear entityManager, because if duplicated object was persisted then during *select* an object flush() method will be executed and it will thrown ConstrainViolationException
DofinService.getEntityManagerThreadLocal().clear();
CheckSimilarObject checkSimilarObject = new CheckSimilarObject();
long objectId = checkSimilarObject.checkObject(dofinObject);
logger.warn("Re-identifying was done. EntityId = " + objectId);
entityIdInDB = objectId;
}
} catch (Exception ex) {
logger.error("Error persisting and commiting object: " + ex.toString());
}

Java - Oracle data use

Does anyone know how to use data from Oracle database in Java Netbeans?
I have successfully connected the database to Netbeans but I do not know how to actually use this data. All I have done is to create Entity classes from database and JPA Controller classes from Entity Classes which contains methods for CRUD functionality but I do not know how to use them.
Here is an example of a "Insert" button:
Staff s1 = new Staff();
s1.setStaffId(txtId.getText());
EntityManagerFactory emf = Persistence.createEntityManagerFactory("PCR4B");
StaffJpaController ajc = new StaffJpaController(emf);
try {
ajc.create(s1);
} catch (Exception ex) {
Logger.getLogger(StaffGUI.class.getName()).log(Level.SEVERE, null, ex);
}
Is there anything wrong with this part of code? For some reason I get many errors but I am not sure if they come from this part of code or not.
Thanks in advance.

It's too many errors to display them here. The build is successful though.
I was wondering if there is a way to store the data of a table in an ArrayList and then use the ArrayList to make changes to elements and then store them back to the database?
The JPA controller class provides the CRUD functionality but I do not know how to make use of the functions. I do not understand what the EntityManager is.

JPA managed entities vs JavaFX properties

My current project is done using JavaFX. I use properties to bind (bidirectionnal) view fields to bean (with BeanPathAdapter of JFXtras).
I choose to use JPA with ObjectDB as model.
This is the first time I use JPA in a standalone project and here I'm facing the problem of managed entities.
Actually, I bind managed entities to view fields and when the value of a view field changes, the entities is updated... and the database also.
I'm trying to find a way to manually persist/merge an entity so I can ask the user if he wants to save or not.
Here's the code i use to get list :
EntityManagerFactory emf = Persistence.createEntityManagerFactory("$objectdb/data/db.odb");
EntityManager em = emf.createEntityManager();
List<XXX> entities = em.createQuery("SELECT x FROM XXX x").getResultList();
So when i do
entity.setName("test");
the entity is updated in the database.
What i'm looking for is that the entity doesn't update automatically.
I tried (just after the getResultList)
em.clear();
or
em.detach(entity);
but it looses the relations instances even with CascadeType.DETACH.
I also tried
em.setFlushMode(FlushModeType.COMMIT);
but it still updates automatically...
I also tried to clone the object. But when i want to merge it, it gives me an exception :
Attempt to reuse an existing primary key value
I thought an alternative solution : use a variable as 'buffer' and fill the managed bean with buffer if the user saves. But BeanPathAdapter looses its sense. It's the same as filling view fields manually and filling bean fields manually after saving.
Could you help me to find a solution ?
Thanks,
Smoky
EDIT:
I answer to my own question :p
After 3 hours of research, I found a solution.
The 'cloning' solution was the 'best' of each I quoted but I don't think it's the best one.
The cause of the exception was the code I used to persist/merge my entity. I was persisting an entity non-managed with an already existing id. I thought I was merging...
I did a generic method not to fail again
public <T extends IEntity> T persist(T object) {
em.getTransaction().begin();
if (object.getId() == null) {
em.persist(object);
em.flush();
em.getTransaction().commit();
em.refresh(object);
}
else {
object = em.merge(object);
em.getTransaction().commit();
}
return object;
}
So the solution : When I have to bind the entity to the view, I use entity.clone() so I can use the entity as non-managed and merge when I want.
But if you have a proper solution, i'm interested :)
Thanks again

In addition to the solution above, standard solutions are:
Use detached objects in the model and then merge them into the EntityManager.
Use managed objects in the model, keeping the EntityManager open (with no detach/merge).

How to refresh JPA entities when backend database changes asynchronously?

I have a PostgreSQL 8.4 database with some tables and views which are essentially joins on some of the tables. I used NetBeans 7.2 (as described here) to create REST based services derived from those views and tables and deployed those to a Glassfish 3.1.2.2 server.
There is another process which asynchronously updates contents in some of tables used to build the views. I can directly query the views and tables and see these changes have occured correctly. However, when pulled from the REST based services, the values are not the same as those in the database. I am assuming this is because JPA has cached local copies of the database contents on the Glassfish server and JPA needs to refresh the associated entities.
I have tried adding a couple of methods to the AbstractFacade class NetBeans generates:
public abstract class AbstractFacade<T> {
private Class<T> entityClass;
private String entityName;
private static boolean _refresh = true;
public static void refresh() { _refresh = true; }
public AbstractFacade(Class<T> entityClass) {
this.entityClass = entityClass;
this.entityName = entityClass.getSimpleName();
}
private void doRefresh() {
if (_refresh) {
EntityManager em = getEntityManager();
em.flush();
for (EntityType<?> entity : em.getMetamodel().getEntities()) {
if (entity.getName().contains(entityName)) {
try {
em.refresh(entity);
// log success
}
catch (IllegalArgumentException e) {
// log failure ... typically complains entity is not managed
}
}
}
_refresh = false;
}
}
...
}
I then call doRefresh() from each of the find methods NetBeans generates. What normally happens is the IllegalArgumentsException is thrown stating somethng like Can not refresh not managed object: EntityTypeImpl#28524907:MyView [ javaType: class org.my.rest.MyView descriptor: RelationalDescriptor(org.my.rest.MyView --> [DatabaseTable(my_view)]), mappings: 12].
So I'm looking for some suggestions on how to correctly refresh the entities associated with the views so it is up to date.
UPDATE: Turns out my understanding of the underlying problem was not correct. It is somewhat related to another question I posted earlier, namely the view had no single field which could be used as a unique identifier. NetBeans required I select an ID field, so I just chose one part of what should have been a multi-part key. This exhibited the behavior that all records with a particular ID field were identical, even though the database had records with the same ID field but the rest of it was different. JPA didn't go any further than looking at what I told it was the unique identifier and simply pulled the first record it found.
I resolved this by adding a unique identifier field (never was able to get the multipart key to work properly).

I recommend adding an #Startup #Singleton class that establishes a JDBC connection to the PostgreSQL database and uses LISTEN and NOTIFY to handle cache invalidation.
Update: Here's another interesting approach, using pgq and a collection of workers for invalidation.
Invalidation signalling
Add a trigger on the table that's being updated that sends a NOTIFY whenever an entity is updated. On PostgreSQL 9.0 and above this NOTIFY can contain a payload, usually a row ID, so you don't have to invalidate your entire cache, just the entity that has changed. On older versions where a payload isn't supported you can either add the invalidated entries to a timestamped log table that your helper class queries when it gets a NOTIFY, or just invalidate the whole cache.
Your helper class now LISTENs on the NOTIFY events the trigger sends. When it gets a NOTIFY event, it can invalidate individual cache entries (see below), or flush the entire cache. You can listen for notifications from the database with PgJDBC's listen/notify support. You will need to unwrap any connection pooler managed java.sql.Connection to get to the underlying PostgreSQL implementation so you can cast it to org.postgresql.PGConnection and call getNotifications() on it.
An an alternative to LISTEN and NOTIFY, you could poll a change log table on a timer, and have a trigger on the problem table append changed row IDs and change timestamps to the change log table. This approach will be portable except for the need for a different trigger for each DB type, but it's inefficient and less timely. It'll require frequent inefficient polling, and still have a time delay that the listen/notify approach does not. In PostgreSQL you can use an UNLOGGED table to reduce the costs of this approach a little bit.
Cache levels
EclipseLink/JPA has a couple of levels of caching.
The 1st level cache is at the EntityManager level. If an entity is attached to an EntityManager by persist(...), merge(...), find(...), etc, then the EntityManager is required to return the same instance of that entity when it is accessed again within the same session, whether or not your application still has references to it. This attached instance won't be up-to-date if your database contents have since changed.
The 2nd level cache, which is optional, is at the EntityManagerFactory level and is a more traditional cache. It isn't clear whether you have the 2nd level cache enabled. Check your EclipseLink logs and your persistence.xml. You can get access to the 2nd level cache with EntityManagerFactory.getCache(); see Cache.
#thedayofcondor showed how to flush the 2nd level cache with:
em.getEntityManagerFactory().getCache().evictAll();
but you can also evict individual objects with the evict(java.lang.Class cls, java.lang.Object primaryKey) call:
em.getEntityManagerFactory().getCache().evict(theClass, thePrimaryKey);
which you can use from your #Startup #Singleton NOTIFY listener to invalidate only those entries that have changed.
The 1st level cache isn't so easy, because it's part of your application logic. You'll want to learn about how the EntityManager, attached and detached entities, etc work. One option is to always use detached entities for the table in question, where you use a new EntityManager whenever you fetch the entity. This question:
Invalidating JPA EntityManager session
has a useful discussion of handling invalidation of the entity manager's cache. However, it's unlikely that an EntityManager cache is your problem, because a RESTful web service is usually implemented using short EntityManager sessions. This is only likely to be an issue if you're using extended persistence contexts, or if you're creating and managing your own EntityManager sessions rather than using container-managed persistence.

You can either disable caching entirely (see: http://wiki.eclipse.org/EclipseLink/FAQ/How_to_disable_the_shared_cache%3F ) but be preparedto a fairly large performance loss.
Otherwise, you can perform a clear cache programmatically with
em.getEntityManagerFactory().getCache().evictAll();
You can map it to a servlet so you can call it externally - this is better if your database is modify externally very seldom and you just want to be sure JPS will pick up the new version

Just a thought, but how do you receive your EntityManager/Session/whatever?
If you queried the entity in one session, it will be detached in the next one and you will have to merge it back into the persistence context to get it managed again.
Trying to work with detached entities may result in those not-managed exceptions, you should re-query the entity or you could try it with merge (or similar methods).

JPA doesn't do any caching by default. You have to explicitly configure it. I believe its the side effect of the architectural style you have chosen: REST. I think caching is happening at the web servers, proxy servers etc. I suggest you read this and debug more.

Persisting an object in Hibernate while having a known primary key.

My problem is with detached objects...
I am currently using Spring with Hibernate.
I have a mapped object that has a primary key as a String (I know it sucks... but refactoring the code would take months), and I wish to persist it. (I have simplified the object with just two attributes)
#Id
private String id;
private String pattern;
So for example I want to add something like:
["id":"myFirstPattern","pattern":".*"]
Notice that my primary key is already set. The problem with that is that whenever I try to persist, Hibernate will try to link this object with any object within the context (because of the primary key) and will fail to do so, since there are none. Throwing a detached object error.
I've done some research and came to the conclusion that merge() would suffice my needs, since it persists and updates even if the object is not available. However I found this a rather dirty workaround and wanted to check if there are any other solutions to this problem.
Take into account that we have a Helper layer, so Services layer will not work directly with the HibernateDao layer. So I can "mask" this by adding 'persist' and 'update' methods that will invoke the same merge DAO method.
Thanks,
Flavio.

Have you tried saveOrUpdate?
Session sess = factory.openSession();
Transaction tx;
try {
tx = sess.beginTransaction();
session.saveOrUpdate( yourObjectHere );
tx.commit();
}
catch (Exception e) {
if (tx!=null) tx.rollback();
throw e;
}
finally {
sess.close();
}

I tried using some different approach after I tried the idea Mauricio gave to me. Since SaveOrUpdate was using the cached entities to verify if it should update or save an object I thought about making a clear just before saving my object.
so, here is my piece of code:
try {
getHibernateTemplate().clear();
getHibernateTemplate().save(entity);
} catch (DataAccessException e) {
if (e.getCause() instanceof ConstraintViolationException)
throw new HibernateDaoException("Entity could not be persisted. Constraint violation.");
throw new HibernateDaoException(e);
}
For now, it is working as expected, even though it seems that it will kill my cached database reason to exist... However this update feature will be used sparingly, as the main reason for the component is returning info, matching patterns and returning the best results.
Anyway I will return soon if I find any flaws.
Any comments, please feel free to post them :)

I'm not certain, and I can't try it myself right now, but doesn't setting the #Id property to use the "assigned" generator do exactly that?