I am using Hibernate implementation of JPA. Let's say I have a list of objects which I have to persist in a table called Event. All these objects have the same zip code.
public class Event {
String id;
String zipCode;
String locationCode;
String eventName;
String eventDesc;
}
Here id is the primary key and zipCode and locationCode together make a unique key (UK_zipCode_locationCode). The table might already have objects with the given zip code. So, instead of finding which ones should be added, deleted or updated, what I do is delete all the objects in the table with the given zip code first and then insert all the given objects.
// getEventsToAdd method returns the list of events to be added for the zipCode 1234
// getEventsFromTheDB method returns all the events in the db with the zipCode 1234
List<Event> eventsToAdd = getEventsToAdd("1234");
List<Event> oldEvents = getEventsFromTheDB("1234");
for (Event e : oldEvents) {
entityManager.remove(e);
}
for (Event e : eventsToAdd) {
entityManager.persist(e);
}
entityManager.flush();
// ...
This works when the oldEvents list is empty or when all objects in the oldEvents are also in eventsToAdd list (by this I mean the event objects with the same id and same zip code).
However, if there are some event objects in oldEvents which have different id, i.e., does not match with the id of any object in eventsToAdd list, then it throws an exception
Duplicate Entry found for key UK_zipCode_locationCode
The error is as if the old events were not deleted from the table and now inserting the events with the same values of zipCode and locationCode is causing org.hibernate.exception.ConstraintViolationException.
However, if I call entityManager.flush() after deleting the old events, it works -
// This works!
for (Event e : oldEvents) {
entityManager.remove(customizedProviderAttribute);
}
// flush after removing all the old events
entityManager.flush();
for (Event e : eventsToAdd) {
entityManager.persist(e);
}
So, why does flushing at the end does not work but flushing after removing the old entities work?
By default the EntityManager does all SQL commands at the point when transaction is committed. However it can decide in which order it does the SQL commands and in your case the inserts were done before delete, which caused the ConstraintViolationException. flush() causes all SQL to be done immediately, so you achieve deletion before insertion. World is not perfect, neither is Hibernate.
The entity manager does not necessarily issue delete and insert statements when you call remove and persist, it waits and generates the SQL later, typically when you flush explicitly or implicitly. That means the order of the statements will be different, so some inserts may be performed before some deletes, thus triggering the constraint violation. Your workaround with the intermediate flush is common practice in cases like this.
In your second working example when you flush after deletion ,Hibernate will change the state of those entities to REMOVED,to stay synchronized with the database AS if the deletion was physically done,and in your logs you'll see a delete sql query issued,that's why when you persist those same entities ,it'll work ,as for the first example not working,because those entities are still in MANAGED state and you're trying to persist them again,which causes duplicate entries,like #Michal said ,the insertions where issued before the deletion , because the order is not guaranteed.
Related
I have a somewhat strange situation which I need to deal with, but can't seem to find a solution.
I need to solve a potential race condition on a customer insertion. We receive the customers through a topic, so they come with an id(we keep it because it's the same id we have in a different database for a different microservice). So, if by some chance, after the same customer is committed to the database before the flush operation is actioned, we should update the record in the database with the one that arrived through the topic, if the last activity field on that one is after the last activity field on the db entry.
The problem we encounter is that, while the flush option is recognizes the newly committed consumer and throws the ConstraintViolationException, when it gets to the find line it returns the customer we try to persist above, not the customer in the database
The code breaks down like this.
try{
entityManager.persist(customer);
//at this point, I insert a new customer in the database with the same id as the one I've persisted
entityManager.flush();
}catch(PersistenceException e){
if(e.getCause() instanceof ConstraintViolationException) {
dbCustomer = Optional.of(entityManager.find(Customer.class,
customer.getId()));
//update DB Customer with data from persisted customer if the last update date on the persisted customer is after the one on the db customer
}
}
I tried different options of transaction propagation, with no success, however, and to use the detach(customer) method before trying to find the db customer, however, in this case, the find function returns Null
Thanks
As soon as a flush fails, the persistence context is essentially broken. If you need to do something with the result of this code block that needs flushing, you need to do that in a new transaction in case of a constraint violation.
I have a One to many mapping in JPA as follows:
In my blockchain class I have the #OneToMany annotation over ArrayList (which is the "chain" property) for which I have a Block class.
I have a method for replacing the chain of a blockchain instance with another when a new chain is broadcast or new block is broadcast on the wire (e.g. pubsub, triggred by async listener). The method replaces the chain if it is found to be valid and of sufficient length.
The chain is reflected in the database as a join table between blockchain and block. When a new chain comes in, it will be mostly identical. In other words there will be at least 1 collision with primary key, if only its the genesis block. More likely all but one or a few blocks at the tip will collide, so I want the ability to handle this without incident. Ideally JPA would have figured out how to do it without me with the following code, but that's not the case.
#Override
public boolean replaceChain(String name, ArrayList<Block> new_chain) throws NoSuchAlgorithmException, ChainTooShortException, GenesisBlockInvalidException, BlocksInChainInvalidException {
this.connect();
em.getTransaction().begin();
Query query = em.createQuery("select b from Blockchain b where b.instance_name = :name");
query.setParameter("name", name);
Blockchain blockchain = (Blockchain) query.getSingleResult();
blockchain.replace_chain(new_chain);
em.getTransaction().commit();
this.disconnect();
return true;
}
From there I tried many permutations and tricks I could think of. I tried manually deleting each block that is a duplicate from the block entity but then it had a problem with the join table and stack overflow said apparently JPA is not set up to manage that manually. It's not the OOP way. I'm fine with that, but then my question is what is the OOP way. Just to overwrite a one to many relationship. The new incoming OneToMany should overwrite everything, displace everything else and that's it, but it tries to duplicate. I read other SO posts but I either didn't understand them well enough or they didn't see to help.
I'm running this through a DAO service, wired up to work through a pubnub listener callback. I have two servers and in fact codebases running- this Main "node on the network" that is dealing with the database (port 8080) and an "in memory" one on 9080 that starts with only the genesis block and if it gets a 200 GET request to 8080 will clone that and replace that chain. Replace chain works- just not to write to database. I call the second node on the network the PEER instance. It has the ability to mine blocks and when it does, it broadcasts to pubsub which triggers the main node. That's my setup. Everything seems to be working beautifully except the JPA part of it. I'm using Eclipselink and Tomcat.
From my understanding, when you start a transaction with entitymanager, it basically watches what you do and takes notes and records the results and does its magic, kind of like a scribe or observer. You set it up to watch, and then you do your business and then you tell it to commit, it still has limits and constraints to deal with or exceptions will be thrown but that's my understanding and that's the route I initially went.
Here is my error log for just this code above, not the trying to manually delete the blocks of a given chain. I could do that but I couldn't get to the join table and I know that's not the ideal way
Error Code: 1062
Call: INSERT INTO block (TIMESTAMP, DATA, DIFFICULTY, HASH, LASTHASH, NONCE) VALUES (?, ?, ?, ?, ?, ?)
bind => [1617166967254, [B#5ebbe21e, 9, 002057d17e0de9c5f97f6a0f3e7534c0599036ae307ece2ee3f645025c153f80, 007e833b320c58bcf29096e22ced52a5c90c915e23830eeae0a7093290af4080, 246]
Query: InsertObjectQuery(privblock.gerald.ryan.entity.Block#d6f817c0)
at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.commit(EntityTransactionImpl.java:157)
at privblock.gerald.ryan.dao.BlockchainDao.replaceChain(BlockchainDao.java:97)
at privblock.gerald.ryan.service.BlockchainService.replaceChainService(BlockchainService.java:38)
at pubsub.PubNubSubCallback.message(PubNubSubCallback.java:132)
at com.pubnub.api.managers.ListenerManager.announce(ListenerManager.java:61)
at com.pubnub.api.workers.SubscribeMessageWorker.processIncomingPayload(SubscribeMessageWorker.java:228)
at com.pubnub.api.workers.SubscribeMessageWorker.takeMessage(SubscribeMessageWorker.java:83)
at com.pubnub.api.workers.SubscribeMessageWorker.run(SubscribeMessageWorker.java:74)
at java.base/java.lang.Thread.run(Thread.java:832)
Any help or insight is appreciated!
I figured it out. It seems to work with one extra annotation property orphanRemoval=true as below
#OneToMany(targetEntity = Block.class, cascade = CascadeType.PERSIST, orphanRemoval=true)
#JoinTable(name = "BlocksByChain")
List<Block> chain; // The chain itself
I knew it had to be simple and some feature that already existed. It was just not the framework default
EDIT: Not quite. Not perfectly. I still have to have code that flushes it out inside the DAO, and that is clunky and not optimal. Also I get a console output about a deadlock or something. I didn't notice that before as my application works as expected, but I know there has to be a better way.
This code also has to exist for it to work:
#Override
public boolean replaceChain(String name, ArrayList<Block> new_chain) throws NoSuchAlgorithmException,
ChainTooShortException, GenesisBlockInvalidException, BlocksInChainInvalidException {
this.connect();
em.getTransaction().begin();
Query query = em.createQuery("select b from Blockchain b where b.instance_name = :name");
query.setParameter("name", name);
Blockchain blockchain = (Blockchain) query.getSingleResult();
// blockchain.replace_chain(new_chain);
// em.merge(blockchain);
System.out.println("GOING TO REPLACE CHAIN AS SERVICE");
// THIS LONG BLOCK IS BECAUSE I COULDN'T FIND A MORE NATURAL WAY. I KEEP GETTING
// ERRORS.
// I JUST WANT TO OVERWRITE THE CHAIN OR DO A SMART MERGE
// INSTEAD IT TRIES TO APPEND. I HAVE TO WRITE AN EMPTY SET TO DB AND COMMIT IT
// AND THEN REPOPULATE IT. ALTERNATELY I COULD MAYBE DO A NATIVE QUERY AND
// TRUNCATE
// REGARDLESS IT DOESN'T SEEM TO SMARTLY MERGE THE TWO CHAINS
// -- IT SHOULD BE EASY WHEN THE NEW CHAIN IS AN EXTENSION, VS A FORK
// -- HANDLING THE "FORK" POTENTIAL OF BLOCKCHAIN ADDS TO THE COMPLEXITY IN
// WHICH CASE EASIEST TO TRUNCATE AND START FRESH
// Try Flush
if (blockchain.willReplace(new_chain)) {
blockchain.setChain(null);
em.getTransaction().commit();
em.getTransaction().begin();
// em.flush();
Query query2 = em.createQuery("select b from Blockchain b where b.instance_name = :name");
query.setParameter("name", name);
Blockchain blockchain2 = (Blockchain) query.getSingleResult();
blockchain2.setChain(new_chain);
em.getTransaction().commit();
this.disconnect();
return true;
}
em.getTransaction().commit();
this.disconnect();
return true;
}
Sorry in advance for the long post. I'm working with a Java WebApplication which uses Spring (2.0, I know...) and Jpa with Hibernateimplementation (using hibernate 4.1 and hibernate-jpa-2.0.jar). I'm having problems retrieving the value of a column from a DB Table (MySql 5) after i update it. This is my situation (simplified, but that's the core of it):
Table KcUser:
Id:Long (primary key)
Name:String
.
.
.
Contract_Id: Long (foreign key, references KcContract.Id)
Table KcContract:
Id: Long (primary Key)
ColA
.
.
ColX
In my server I have something like this:
MyController {
myService.doSomething();
}
MyService {
private EntityManager myEntityManager;
#Transactional(readOnly=true)
public void doSomething() {
List<Long> IDs = firstFetch(); // retrieves some users IDs querying the KcContract table
doUpdate(IDs); // updates a column on KcUser rows that matches the IDs retrieved by the previous query
secondFecth(IDs); // finally retrieves KcUser rows <-- here the returned rows contains the old value and not the new one i updated in the previous method
}
#Transactional(readOnly=true)
private List<Long> firstFetch() {
List<Long> userIDs = myEntityManager.createQuery("select c.id from KcContract c" ).getResultList(); // this is not the actual query, there are some conditions in the where clause but you get the idea
return userIDs;
}
#Transactional(readOnly=false, propagation=Propagation.REQUIRES_NEW)
private void doUpdate(List<Long> IDs) {
Query hql = myEntityManager().createQuery("update KcUser t set t.name='newValue' WHERE t.contract.id IN (:list)").setParameter("list", IDs);
int howMany = hql.executeUpdate();
System.out.println("HOW MANY: "+howMany); // howMany is correct, with the number of updated rows in DB
Query select = getEntityManager().createQuery("select t from KcUser t WHERE t.contract.id IN (:list)" ).setParameter("list", activeContractIDs);
List<KcUser> users = select.getResultList();
System.out.println("users: "+users.get(0).getName()); //correct, newValue!
}
private void secondFetch(List<Long> IDs) {
List<KcUser> users = myEntityManager.createQuery("from KcUser t WHERE t.contract.id IN (:list)").setParameter("list", IDs).getResultList()
for(KcUser u : users) {
myEntityManager.refresh(u);
String name = u.getName(); // still oldValue!
}
}
}
The strange thing is that if i comment the call to the first method (myService.firstFetch()) and call the other two methods with a constant list of IDs, i get the correct new KcUser.name value in secondFetch() method.
Im not very expert with Jpa and Hibernate, but I thought it might be a cache problem, so i've tried:
using myEntityManager.flush() after the update
clearing the cache with myEntityManager.clear() and myEntityManager.getEntityManagerFactory().evictAll();
clearing the cache with hibernate Session.clear()
using myEntityManager.refresh on KcUser entities
using native queries (myEntityManager.createNativeQuery("")), which to my understanding should not involve any cache
Nothing of that worked and I always got returned the old KcUser.name value in secondFetch() method.
The only things that worked so far are:
making the firstFetch() method public and moving its call outside of myService.doSomething(), so doing something like this in MyController:
List<Long> IDs = myService.firstFetch();
myService.doSomething(IDs);
using a new EntityManager in secondFetch(), so doing something like this:
EntityManager newEntityManager = myEntityManager.getEntityManagerFactory().createEntityManager();
and using it to execute the subsequent query to fetch users from DB
Using either of the last two methods, the second select works fine and i get users with the updated value in "name" column.
But I'd like to know what's actually happening and why noone of the other things worked: if it's actually a cache problem a simply .clear() or .refresh() should have worked i think. Or maybe i'm totally wrong and it's not related to the cache at all, but then i'm bit lost to what might actually be happening.
I fear there might be something wrong in the way we are using hibernate / jpa which might bite us in the future.
Any idea please? Tell me if you need more details and thanks for your help.
Actions are performed in following order:
Read-only transaction A opens.
First fetch (transaction A)
Not-read-only transaction B opens
Update (transaction B)
Transaction B closes
Second fetch (transaction A)
Transaction A closes
Transaction A is read-only. All subsequent queries in that transaction see only changes that were committed before the transaction began - your update was performed after it.
I've been struggling for few hours with this one and could do with some help.
A client sends an object that contains a list;
One of the objects in the list has been modified on the client;
In some cases I don't want that modified entity to be persisted to the database, I want to keep the original database values.
I have tried the following and various attempts to clear(), refresh() and flush() the session:
List<Integer> notToModifyIds = dao.getDoNotModifyIds(parentEntity.getId());
MyEntityFromList entityFromClient, entityFromDb;
for(Integer notToModifyId : notToModifyIds){
ListIterator iterator = parentEntity.getEntities().listIterator();
while(iterator.hasNext()){
entityFromClient = (MyEntity) iterator.next();
if(Objects.equals(entityFromClient.getId(), notToModifyId)){
dao.evict(entityFromClient);
entityFromDb = (MyEntity) dao.get(MyEntity.class, notToModifyId);
iterator.remove(entityFromClient);
iterator.add(entityFromDb);
}
}
}
However, no matter what I try I always get the values from the client persisted to the database. When I add a breakpoint after iterator.add() I can check that the database value has not been updated at that point, hence I know that if I could load the entity from the DB then I would have the value I want.
I'm feeling a little suppid!
I don't know if I got the whole scenario here. Are those modified "entitiesFromClient" attached to the Hibernate session? If they are, the changes were probably automatically flushed to the database before you "evicted" them.
Setting a MANUAL flush mode would help you avoid the automatic behaviour.
First of all, I would enable the Hibernate SQL logging to see more precisely what is happening. See Enable Hibernate logging.
Checking the database in another session (while stopped in the breakpoint) will not help if this code is running within a transaction. Even if the change was already flushed in the database you wouldn't see it until the transaction is commited.
I am using spring, hibernate and postgreSQL.
Let's say I have a table looking like this:
CREATE TABLE test
(
id integer NOT NULL
name character(10)
CONSTRAINT test_unique UNIQUE (id)
)
So always when I am inserting record the attribute id should be unique
I would like to know what is better way to insert new record (in my spring java app):
1) Check if record with given id exists and if it doesn't insert record, something like this:
if(testDao.find(id) == null) {
Test test = new Test(Integer id, String name);
testeDao.create(test);
}
2) Call straight create method and wait if it will throw DataAccessException...
Test test = new Test(Integer id, String name);
try{
testeDao.create(test);
}
catch(DataAccessException e){
System.out.println("Error inserting record");
}
I consider the 1st way appropriate but it means more processing for DB. What is your opinion?
Thank you in advance for any advice.
Option (2) is subject to a race condition, where a concurrent session could create the record between checking for it and inserting it. This window is longer than you might expect because the record might be already inserted by another transaction, but not yet committed.
Option (1) is better, but will result in a lot of noise in the PostgreSQL error logs.
The best way is to use PostgreSQL 9.5's INSERT ... ON CONFLICT ... support to do a reliable, race-condition-free insert-if-not-exists operation.
On older versions you can use a loop in plpgsql.
Both those options require use of native queries, of course.
Depends on the source of your ID. If you generate it yourself you can assert uniqueness and rely on catching an exception, e.g. http://docs.oracle.com/javase/1.5.0/docs/api/java/util/UUID.html
Another way would be to let Postgres generate the ID using the SERIAL data type
http://www.postgresql.org/docs/8.1/interactive/datatype.html#DATATYPE-SERIAL
If you have to take over from an untrusted source, do the prior check.