I am looking for a way to save or update records, according to the table's unique key which is composed of several columns).
I want to achieve the same functionality used by INSERT ... ON DUPLICATE KEY UPDATE - meaning to blindly save a record, and have the DB/Hibernate insert a new one, or update the existing one if the unique key already exists.
I know I can use #SQLInsert( sql="INSERT INTO .. ON DUPLICATE KEY UPDATE"), but I was hoping not to write my own SQLs and let Hibernate do the job. (I am assuming it will do a better job - otherwise why use Hibernate?)
Hibernate may throw a ConstraintViolationException when you attempt to insert a row that breaks a constraint (including a unique constraint). If you don't get that exception, you may get some other general Hibernate exception - it depends on the version of Hibernate and the ability of Hibernate to map the MySQL exception to a Hibernate exception in the version and type of database you are using (I haven't tested it on everything).
You will only get the exception after calling flush(), so you should make sure this is also in your try-catch block.
I would be careful of implementing solutions where you check that the row exists first. If multiple sessions are updating the table concurrently you could get a race condition. Two processes read the row at nearly-the-same time to see if it exists; they both detect that it is not there, and then they both try to create a new row. One will fail depending on who wins the race.
A better solution is to attempt the insert first and if it fails, assume it was there already. However, once you have an exception you will have to roll back, so that will limit how you can use this approach.
This doesn't really sound like a clean approach to me. It would be better to first see if an entity with given key(s) exists. If so, update it and save it, if not create a new one.
EDIT
Or maybe consider if merge() is what you're looking for:
if there is a persistent instance with the same identifier currently associated with the session, copy the state of the given object onto the persistent instance
if there is no persistent instance currently associated with the session, try to load it from the database, or create a new persistent instance
the persistent instance is returned
the given instance does not become associated with the session, it remains detached
< http://docs.jboss.org/hibernate/core/3.3/reference/en/html/objectstate.html
You could use saveOrUpdate() from Session class.
Related
I am using spring-hibernate-postgres. I have a table say 'some_entity'. It already contains some records.
I have a program that tries to create new SomeEntity object. I am populating that object with appropriate properties, and afterwards I am calling persist() method on it.
In the log, I see that hibernate is trying to get nextVal() from the table sequences. But, nextval that my postgres returns is same as id of 2nd row of some_entity table. So, my hibernate tries to create row with that id. And hence my persist() fails with hibernate constraint violation exception.
May be I am not phrasing the question correctly. I hope someone has already encountered this problem and has resolution for it.
Thanks
I had this problem. I solved it through execution of sql, that updates sequence at application launch
ALTER SEQUENCE names_id_seq RESTART WITH currentId;
,where currentId I get from
SELECT currval('names_id_seq');
I'm working on an AppEngine project and I'm using JDO on top of the AppEngine datastore for persistence. I have an entity that uses an encoded string as the key and also uses an application generated keyname (also a string). I did this because my app would frequently scoop data (potentially scooping the same thing) from the wild and attempt to persist them. In an attempt to avoid persisting several entities which essentially contain the same data, I decided to hash some properties about these data so as to get a consistent keyname (not manipulating keys directly because of entity relationships).
The problem now is that whenever I calculate my hash (keyname) and attempt to store the entity, if it already exists in the datastore, the datastore (or JDO or whoever the culprit is) silently overwrites the properties of the entity in the datastore without raising any exception. This has serious effects on the app because it overrides the timeStamps (a field) of the entities (which we use for ordering).
How best can I get around this?
You need to do get-before-set (Check and set or CAS).
CAS is a fundamental tenant of concurrency, and it's a necessary evil of parallel computing.
Gets are much cheaper than sets anyway, so it may actually save you money.
Instead of blind writing to datastore, first retrieve; if the entity doesn't exist, catch the exception and just put the entity. If it does exist, do a deep compare before you save. If nothing has changed, don't persist it (and save that cost). If it has changed, choose your merge strategy however you please. One (slightly ugly) way to maintain dated revisions is to store the previous entity as a field in the updated entity (may not work for many revisions).
But, in this case, you have to get before set. If you don't expect many duplicates and want to be really chintzy, you can do an exists query first... Which is to do a keys-only count query on the key you want to use (costs 7x less than a full get). If (count() == 0) then put() else getAndMaybePut() fi
The count query syntax might look slow, but from my benchmarks, it's the fastest (and cheapest) possible way to tell if an entity exists:
public boolean exists(Key key){
Query q;
if (key.getParent() == null)
q = new Query(key.getKind());
else
q = new Query(key.getKind(), key.getParent());
q.setKeysOnly();
q.setFilter(new FilterPredicate(
Entity.KEY_RESERVED_PROPERTY, FilterOperator.EQUAL, key));
return 1 == DatastoreServiceFactory.getDatastoreService().prepare(q)
.countEntities(FetchOptions.Builder.withLimit(1));
}
You must do a get() to see if an entity with the same key exists before you put() the new entity. There is no way around doing this.
You can use memcache and local "in-memory" caching to speed up your get() operation. This may only help if you are likely to read the same information multiple times. If not, the memcache query may actually slow down your process.
To ensure that two requests do not overwrite each other you should use a transaction (not possible with a query as suggested by Ajax unless you put all items in a single entity group which may limit your updates to 1 per second)
In pseudo code:
Create Key from hashing data
Check in-memory cache for key (use a ConcurrentHashSet of keys), return if found
Check MemcacheService for key, return if found
Start transaction
Get entity from datastore, return if found
Create entity in datastore
Commit transaction, return if fails due to concurrent update
Put Key in cache (in-memory and memcache)
Step 7 will fail if another request (thread) has already written the same key at the same time.
What I suggest you is that instead of saving the ID as a string either use a Long ID for your entity or you may use Key datatype, which is auto generated by appengine.
#PersistenceCapable
public class Test{
#PrimaryKey
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long ID;
// getter and setter
}
This will return a unique value to you everytime.
I have more of theoretical question:
When data gets inserted into a database? is it after persist or after commit is called? Because I have a problem with unique keys (manually generated) - they get duplicate. I'm thinking this is due multiple users inserting data simultaneously into a same table.
UPDATE 1:
I generate keys in my application. Keys example: '123456789123','123456789124','123456789125'...
Key field is varchar type, because there are lot of old keys (I can't delete or change them) like 'VP123456','VP15S3456'. Another problem, that after inserting them into one database, these keys have to be inserted in another database. And I don't know what are DB sequences and Atomic objects..
UPDATE 2:
These keys are used in finance documents and not as database keys. So they must be unique, but they are not used anywhere in programming as object keys.
I would suggest you create a Singleton that takes care of generating your keys. Make sure you can only get a new id once the singleton has initialized with the latest value from the database.
To safeguard you from incomplete inserts into the two databases I would suggest you try to use XA transactions. This will allow you to have all-or-nothing inserts and updates. So if any of the operations on any of the databases fails, everything will be rolled back. Of course there is a downside of XA transactions; they are quite slow and not all databases and database drivers support it.
How do you generate these keys? Have you tried using sequences in DB or atomic objects?
I'm asking because it is normal to populate DB concurrently.
EDIT1:
You can write a method that returns new keys based on atomic counter, this way you'll know that anytime you request a new key you receive a unique key. This strategy may and will lead to some keys being discarded but it is a small price to pay, unless it is a requirement that keys in the database are sequential.
private AtomicLong counter; //initialized somewhere else.
public String getKey(){
return "VP" + counter.incrementAndGet();
}
And here's some help on DB Sequences in Oracle, MySql, etc.
What is the best/fastest way to check if an Entity exists in a google-app-engine datastore? For now I'm trying to get the entity by key and checking if the get() returns an error.
I don't know the process of getting an Entity on the datastore. Is there a faster way for doing only this check?
What you proposed would indeed be the fastest way to know if your entity exists. The only thing slowing you down is the time it takes to fetch and deserialize your entity. If your entity is large, this can slow you down.
IF this action (checking for existence) is a major bottleneck for you and you have large entities, you may want to roll your own system of checking by using two entities - first you would have your existing entity with data, and a second entity that either stores the reference to the real entity, or perhaps an empty entity where the key is just a variation on the original entity key that you can compute. You can check for existence quickly using the 2nd entity, and then fetch the first entity only if the data is necessary.
The better way I think would just be to design your keys such they you know there would not be duplicates, or that your operations are idempotent, so that even if an old entity was overwritten, it wouldn't matter.
com.google.appengine.api has been deprecated in favor of the App Engine GCS client.
Have you considered using a query? Guess-and-check is not a scalable way to find out of an entity exists in a data store. A query can be created to retrieve entities from the datastore that meet a specified set of conditions:
https://developers.google.com/appengine/docs/java/datastore/queries
EDIT:
What about the key-only query? Key-only queries run faster than queries that return complete entities. To return only the keys, use the Query.setKeysOnly() method.
new Query("Kind").addFilter(Entity.KEY_RESERVED_PROPERTY, FilterOperator.EQUAL, key).setKeysOnly();
Source: [1]: http://groups.google.com/group/google-appengine-java/browse_thread/thread/b1d1bb69f0635d46/0e2ba938fad3a543?pli=1
You could fetch using a List<Key> containing only one Key, that method returns a Map<Key, Entity> which you can check if it contains an actual value or null, for example:
Entity e = datastoreService.get(Arrays.asList(key)).get(key);
In general though I think it'd be easier to wrap the get() in a try/catch that returns null if the EntityNotFoundException is caught.
Consider am using java , struts, hibernate and oracle. How can i prevent duplicate entries stored in database. One way is to make field as Unique . For example i am entering country "USA" in jsp page,USA is already available means how can i prevent it. Please let me know.
Regards,
sara
You should always indeed put a unique constraint on fields which must stay unique. This will, however, lead to a cryptic exception at commit time. If you want to be more user-friendly, you should check if the entry already exists (using a query) before inserting it, and display a useful and readable error message to the user if the entry already exists.
This still allows two concurrent users to check at the same time, then insert at the same time, but it greatly reduces the probability, and the unique constraint makes sure that one of the commits will fail, leaving your database in a consistent state.
Query your database whether it already contains USA or not. If it does, then don't store it. If not, then do.
Add a unique index to your database table on the country column.
Additionally you can annotate the country attribute of your hibernate object with #Column(unique=true).