Develop Reference

Use CascadeType.MERGE and CascadeType.PERSIST depending on case

I am wondering if the following problem can be solved solely through the use of JPA Annotations and Spring Data's CrudRepository.
The following entity is newly created with every request:
#Entity
#Table(name = "my_entity")
public class MyEntityDAO {
....
#ManyToOne(fetch = FetchType.EAGER, cascade = CascadeType.PERSIST)
#JoinColumn(name = "my_referenced_entity_id")
#JsonBackReference
private MyReferencedEntityDAO myReferencedEntity;
....
}
The CrudRepository is storing the entity plus its referenced element using.
myEntityRepository.save(myEntityDAO);
The problem is that this only works for newly created referenced (MyReferencedEntity) entities. In some cases this is desired, but sometimes this referenced entity will already exist causing a: detached entity passed to persist
When I set
CascadeType.MERGE
it works for the case that the entity exists, but fails when a new one needs to be created.
Is there a way to make this work without handling this operation programmatically using the .persist() and .merge() methods? I have already tried adding both CascadeTypes through annotations but it did result in the same errors.

Spring Data JPA: Batch insert for nested entities

I have a test case where I need to persist 100'000 entity instances into the database. The code I'm currently using does this, but it takes up to 40 seconds until all the data is persisted in the database. The data is read from a JSON file which is about 15 MB in size.
Now I had already implemented a batch insert method in a custom repository before for another project. However, in that case I had a lot of top level entities to persist, with only a few nested entities.
In my current case I have 5 Job entities that contain a List of about ~30 JobDetail entities. One JobDetail contains between 850 and 1100 JobEnvelope entities.
When writing to the database I commit the List of Job entities with the default save(Iterable<Job> jobs) interface method. All nested entities have the CascadeType PERSIST. Each entity has it's own table.
The usual way to enable batch inserts would be to implement a custom method like saveBatch that flushes every once in a while. But my problem in this case are the JobEnvelope entities. I don't persist them with a JobEnvelope repository, instead I let the repository of the Jobentity handle it. I'm using MariaDB as database server.
So my question boils down to the following: How can I make the JobRepository insert it's nested entities in batches?
These are my 3 entites in question:
Job
#Entity
public class Job {
#Id
#GeneratedValue
private int jobId;
#OneToMany(fetch = FetchType.EAGER, cascade = CascadeType.PERSIST, mappedBy = "job")
#JsonManagedReference
private Collection<JobDetail> jobDetails;
}
JobDetail
#Entity
public class JobDetail {
#Id
#GeneratedValue
private int jobDetailId;
#ManyToOne(fetch = FetchType.EAGER, cascade = CascadeType.PERSIST)
#JoinColumn(name = "jobId")
#JsonBackReference
private Job job;
#OneToMany(fetch = FetchType.EAGER, cascade = CascadeType.PERSIST, mappedBy = "jobDetail")
#JsonManagedReference
private List<JobEnvelope> jobEnvelopes;
}
JobEnvelope
#Entity
public class JobEnvelope {
#Id
#GeneratedValue
private int jobEnvelopeId;
#ManyToOne(fetch = FetchType.EAGER, cascade = CascadeType.PERSIST)
#JoinColumn(name = "jobDetailId")
private JobDetail jobDetail;
private double weight;
}

Make sure to configure Hibernate batch-related properties properly:
<property name="hibernate.jdbc.batch_size">100</property>
<property name="hibernate.order_inserts">true</property>
<property name="hibernate.order_updates">true</property>
The point is that successive statements can be batched if they manipulate the same table. If there comes the statement doing insert to another table, the previous batch construction must be interrupted and executed before that statement. With the hibernate.order_inserts property you are giving permission to Hibernate to reorder inserts before constructing batch statements (hibernate.order_updates has the same effect for update statements).
jdbc.batch_size is the maximum batch size that Hibernate will use. Try and analyze different values and pick one that shows best performance in your use cases.
Note that batching of insert statements is disabled if IDENTITY id generator is used.
Specific to MySQL, you have to specify rewriteBatchedStatements=true as part of the connection URL. To make sure that batching is working as expected, add profileSQL=true to inspect the SQL the driver sends to the database. More details here.
If your entities are versioned (for optimistic locking purposes), then in order to utilize batch updates (doesn't impact inserts) you will have to turn on also:
<property name="hibernate.jdbc.batch_versioned_data">true</property>
With this property you tell Hibernate that the JDBC driver is capable to return the correct count of affected rows when executing batch update (needed to perform the version check). You have to check whether this works properly for your database/jdbc driver. For example, it does not work in Oracle 11 and older Oracle versions.
You may also want to flush and clear the persistence context after each batch to release memory, otherwise all of the managed objects remain in the persistence context until it is closed.
Also, you may find this blog useful as it nicely explains the details of Hibernate batching mechanism.

To complete the previous answer of Dragan Bozanovic. Hibernate sometimes silently deactivates the order of execution of the batches if for example it encounters cyclic relations between the entities when it builds the graph of dependencies between the batches (see InsertActionSorter.sort(..) method). It would have been interesting for hibernate to trace this behavior when this happens.

EntityNotFoundException when removing from OneToMany collection

I have two entities connected with a bidirectional OneToMany/ManyToOne relationship.
#Entity
class One {
#OneToMany(mappedBy = "one", orphanRemoval = true, fetch = FetchType.EAGER)
private Set<Many> manies;
// ...
}
#Entity
class Many {
#ManyToOne
#JoinColumn(name = "one_id", nullable = false)
private One one;
// ...
}
When I want to remove a Many instance, I remove it from its One's manies Set and delete it from the database. If I take the same One instance and save it again (because I changed anything, it doesn't have to be related to the relationship), I get an exception:
javax.persistence.EntityNotFoundException: Unable to find test.Many with id 12345
The ID (in this example 12345) is the ID of the just removed entity. Note that removal of the entity 12345 succeeded: The transaction commits successfully and the row is removed from the database.
Adding and removing instances is done with Spring Data repositories. Removing looks more or less like this:
one.getManies().remove(manyToRemove);
oneDao.save(one);
manyDao.delete(manyToRemove);
I debugged a little and found out that the Set is a Hibernate PersistentSet which contains a field storedSnapshot. This in turn is another Set that still references the removed entity. I have no idea why this reference is not removed from the snapshot but I suspect this is the problem: I think Hibernate tries to remove the entity a second time because it's in the snapshot but not in the actual collection. I searched for quite a while but I didn't encounter others with a similar problem.
I use Hibernate 4.2.2 and Spring Data 1.6.0.
Am I doing something inherently wrong? How can I fix this?

I'm having the same problem.
A workaround is to replace the entire collection.
Set<Many> maniesBkp = one.getManies(); // Instance of PersistentSet
maniesBkp.remove(manyToRemove);
Set<Many> manies = new HashSet<Many>();
manies.addAll(maniesBkp);
one.setManies(manies);
...
manyDao.delete(manyToRemove);
...
oneDao.save(one);

As your #OneToMany relation has orphanRemoval = true, you don't have to explicitly remove the child element from the database - just remove it from the collection and save the parent element.

Try add CascadeType.REMOVE and orphanRemoval
#OneToMany(mappedBy = "one", orphanRemoval = true, fetch = FetchType.EAGER, cascade = CascadeType.REMOVE, orphanRemoval = true)
Than perform delete following
one.getManies().remove(manyToRemove);
oneDao.save(one);
Edited:
I have created POC, look on the UserRepositoryIntegrationTest.departmentTestCase (code)

Delete cascade hangs in JPA when large number of objects

I have a JPA Entities like this:
#Entity
class MyEntity{
#JsonIgnore
#OneToMany(mappedBy = "application", cascade = ALL, fetch = LAZY)
private List<MyChildEnity> myChildEntities;
}
...
#Entity
class MyChildEnity {
#ManyToOne(optional = false, fetch = FetchType.LAZY, cascade = { REFRESH,
DETACH })
#JoinColumn(name = "APPLICATION_ID")
private MyEntity application;
}
I access this entity from a REST call. When the number of elements is very large, and I try to delete the MyEntity Object the REST call hangs and then timeout. For small number of elements in MyChildEnity table it works fine. When I debugged, I saw that JPA fetches one record at a time and deletes it. This is too slow and too much work done.
Is this an expected behavior? Shouldn't JPA be intelligent to convert this to a single DELETE call on the MyChildEnity table.
I'm using OpenJPA with Derby and DB2 database.

The reason why you get one delete statement for each element probably has something to do with the fact that JPA let you do something pre- and post removal. If you write a JPQL with a deletestatement you are able to bypass the callback mechanism and delete everything in a single request.
Documentation for entity listeners and callbacks. (This is JPA functionality).

JPA / EclipseLink lazy collections not loading

I have a User entity with this mapping:
#OneToMany(mappedBy = "supervisor", fetch = LAZY, cascade = [CascadeType.REFRESH])
List<Group> supervisedGroups = new ArrayList<Group>()
and a Group entity with this mapping:
#ManyToOne(fetch = LAZY, cascade = [CascadeType.REFRESH])
#JoinColumn(name = "supervisor")
User supervisor
I fetch a user thanks to a repository
User user = userRepository.findById(id)
The findById method is wrapped by a transaction (method is intercepted by a transaction manager advice), and the JPA unit of work lasts as long as the request last (session per view).
When I get the user, I do a
user.getSupervisedGroups()
This returns me an empty list. The collection type is the Eclipselinks's IndirectList and even if i call the size() method it does nothing more.
But if I execute
entitymanager.refresh(user)
user.getSupervisedGroups()
Then I have 2 items in my list... Why ? Does it means EclipseLink does not support at all LAZY fetching on collections ?

My guess is you have corrupted that objects in the shared L2 cache.
When you inserted this object, did you add the correct objects to the collection? My guess is you have a bidirectional OneToMany/ManyToOne relationship and when inserting the One side and not adding to the Many side. You must maintain your object's relationships correctly.
See,
http://en.wikibooks.org/wiki/Java_Persistence/Relationships#Object_corruption.2C_one_side_of_the_relationship_is_not_updated_after_updating_the_other_side

The answer by James might be the origin of the problem. Another solution is to disable EclipseLinks L2 cache:
Persistence Unit property:
<property name="eclipselink.cache.shared.default" value="false"/>
Or JPA 2.0 persistence unit element:
<shared-cache-mode>NONE</shared-cache-mode>
Source: https://wiki.eclipse.org/EclipseLink/FAQ/How_to_disable_the_shared_cache%3F

Change your fetch attributes fetch = LAZY to fetch=FetchType.LAZY from both entities likes this;
#OneToMany(mappedBy = "supervisor", fetch=FetchType.LAZY, cascade = CascadeType.REFRESH)