I'm observing what I think is an unexpected behaviour in JPA 2 when fetching a list attribute with a criteria query.
My query is as follows (an extract of it):
CriteriaBuilder b = em.getCriteriaBuilder();
CriteriaQuery<MainObject> c = b.createQuery(MainObject.class);
Root<MainObject> root = c.from(MainObject.class);
Join<MainObject, FirstFetch> firstFetch = (Join<MainObject, FirstFetch>) root.fetch(MainObject_.firstFetch);
firstFetch.fetch(FirstFetch_.secondFetch); //secondFetch is a list
c.select(root).distinct(true);
(So let's say I'm fetching a list as a property of the property of an object.)
The thing is when the query returns multiple results, secondFetch values are duplicated as many times as rows are returned. Each firstFetch should have just one secondFetch but has n instead.
The only particularity i see in this case is all MainObjects happen to have the same FirstFetch instance.
So my guess is the join is being crossed, which is normal, but then JPA fails to assign its secondFetch object to each one of the firstFetchs.
Mappings shouldn't be too special, the're more or less like this
#Entity
#Table(name="mainobject")
public class MainObject{
//...
private FirstFetch firstFetch;
#ManyToOne(fetch=FetchType.LAZY)
#JoinColumn(name="mainObject_column")
public FirstFetch getFirstFetch() {
return firstFetch;
}
}
and
#Entity
#Table(name="firstFetch")
public class FirstFetch{
//...
private List<SecondFetch> secondFetch;
#OneToMany(mappedBy="secondFetch")
public List<SecondFetch> getSecondFetch() {
return secondFetch;
}
}
& finally
#Entity
#Table(name="secondFetch")
public class SecondFetch {
//....
private FirstFetch firstFetch; //bidirectional
#ManyToOne
#JoinColumn(name="column")
public FirstFetch getFirstFetch() {
return firstFetch;
}
}
I've been looking for some sort of distinct sentence to apply to the fetch but there's none (would have been a 'patch' anyway...)
If i change
List<SecondFetch>
for
Set<SecondFetch>
i'll get the expected result thanks to Sets' Keys, so I do feel this is kind of a misbehaviour in JPA's lists.
I'm not an expert, though, so i could perfectlly be making some mistake in the mappings or query.
Any feeback is very welcome to help clear this out.
Thanks.
I had the exact same problem though I was using JPA criteria API to do the query.
After some research I found a solution which you already mentioned (but was not available, since your not using criteria API): Using distinct.
With JPA criteria it would look like this:
CriteriaQuery<FirstFetch> query = cb.createQuery(FirstFetch.class);
Root<AbschnittC> root = query.from(FirstFetch.class);
root.fetch(FirstFetch_.secondFetch, JoinType.LEFT);
query.distinct(true);
Without using query.distinct(true); the resultset was multiplied with the amount of objects in the secondFetch list.
Hibernate does have something like DISTINCT_ROOT_ENTITY which sound more adequate than just setting a query distinct. But I have not further investigated this. I am also using Hibernate as the JPA provider. Maybe setting the query distinct in JPA ends up using the same code as Hibernates DISTINCT_ROOT_ENTITY would?
Related
Background
I have seen multiple answers and questions here on SO and in many popular blogs about the necessity of the distinct keyword in JPQL JOIN FETCH queries and about the PASS_DISTINCT_THROUGH query hint.
For example, see these two questions
How does DISTINCT work when using JPA and Hibernate
Select DISTINCT on JPA
and these blog posts
The best way to use the JPQL DISTINCT keyword with JPA and Hibernate
The DISTINCT pass-through Hibernate Query Hint
Hibernate Tips: How To Apply DISTINCT to Your JPQL But Not Your SQL Query
What I am missing
Now my problem is that I cannot fully understand when exactly the distinct keyword must be included in the JPQL query. More specifically, if it depends on which method is used to execute the query (getResultList or getSingleResult).
The following is an example to clarify what I mean.
Everything I am writing from now on was tested on Ubuntu Linux 18.04, with Java 8, Hibernate 5.4.13 and an in-memory H2 database (version 1.4.200).
Suppose I have a Department entity which has a lazy bidirectional one to many relationship with a DepartmentDirector entity:
// Department.java
#Entity
public class Department {
// ...
private Set<DepartmentDirector> directors;
#OneToMany(mappedBy = "department", fetch = FetchType.LAZY)
public Set<DepartmentDirector> getDirectors() {
return directors;
}
// ...
}
// DepartmentDirector.java
#Entity
public class DepartmentDirector {
// ...
private Department department
#ManyToOne
#JoinColumn(name = "department_fk")
public Department getDepartment() {
return department;
}
// ...
}
Suppose that my database currently contains one department (department1) and two directors associated with it.
Now I want to retrieve the department by its uuid (the primary key), along with all its directors. This can be done with the following JOIN FETCH JPQL query:
String query = "select department from Department department left join fetch "
+ "department.directors where department.uuid = :uuid";
As the preceding query uses a join fetch with a child collection, I expected it to return two duplicated departments when issued: however this only happens when using the query with the getResultList method and not when using the getSingleResult method. This is somehow reasonable, but I have found that the Hibernate implementation of getSingleResult uses getResultList behind the curtains so I expected a NonUniqueResultException to be thrown.
I also briefly went through JPA 2.2 specification but no distinction in treating the duplicates between the two methods is mentioned, and every code sample concerning this issue uses the getResultList method.
Conclusion
In my example I found out that JOIN FETCH queries executed with getSingleResult do not suffer the duplicated entities problem explained in the resources I linked in the section Background.
If the above claim would be correct, it would mean that the same JOIN FETCH query would need distinct if executed with getResultList, but would not need it when executed with getSingleResult.
I need someone to explain me if this is expected or if I misunderstood something.
Appendix
Results of the two queries:
Query ran with the getResultList method. I get two duplicated departments as expected (this was done just to test the behaviour of the query, getSingleResult should be used instead for this):
List<Department> resultList = entityManager.createQuery(query, Department.class)
.setParameter("uuid", department1.getUuid())
.getResultList();
assertThat(resultList).containsExactly(department1, department1); // passes
Query ran with the getSingleResult method. I would expect the same duplicated departments to be retrieved, and thus a NonUniqueResultException to be thrown. Instead, a single department is retrieved and everything works nice:
Department singleResult = entityManager.createQuery(query, Department.class)
.setParameter("uuid", department1.getUuid())
.getSingleResult();
assertThat(singleResult).isEqualTo(department1); // passes
Interesting question.
First of all let me point out that getSingleResult() was meant for queries that due to their nature always return a single result (meaning: mostly aggregate queries like SELECT SUM(e.id) FROM Entity e). A query that you think, based on some business domain-specific rule, should return a single result, does not really qualify.
That being said, the JPA Spec states that getSingleResult() should throw NonUniqueResultException when the query returns more than one result:
The NonUniqueResultException is thrown by the persistence provider when Query.getSingleResult or TypedQuery.getSingleResult is invoked and there is more than one result from the query. This exception will not cause the current transaction, if one is active, to be marked for rollback.
However, looking at the Hibernate implementation:
#Override
public R getSingleResult() {
try {
final List<R> list = list();
if ( list.size() == 0 ) {
throw new NoResultException( "No entity found for query" );
}
return uniqueElement( list );
}
catch ( HibernateException e ) {
if ( getProducer().getFactory().getSessionFactoryOptions().isJpaBootstrap() ) {
throw getExceptionConverter().convert( e );
}
else {
throw e;
}
}
}
public static <R> R uniqueElement(List<R> list) throws NonUniqueResultException {
int size = list.size();
if ( size == 0 ) {
return null;
}
R first = list.get( 0 );
for ( int i = 1; i < size; i++ ) {
if ( list.get( i ) != first ) {
throw new NonUniqueResultException( list.size() );
}
}
return first;
}
it turns out Hibernate's interpretation of 'more than one result' seems to be 'more than one unique result'.
In fact, I tested your scenario with all JPA providers, and it turns out that:
Hibernate does indeed return duplicates from getResultList(), but does not throw the exception due to the peculiar way getSingleResult() is implemented
EclipseLink is the only one that does not suffer from the duplicate result bug in getResultList() and consequently, getSingleResult() does not throw an exception, either (to me, this behaviour is only logical, but as it turns out, it is all a matter of interpretation)
OpenJPA and DataNucleus both return duplicate results from getResultList() and throw an exception from getSingleResult()
Tl;DR
I need someone to explain me if this is expected or if I misunderstood something.
It really boils down to how you interpret the specification
We have a project where we need to lazily load collections of an entity, but in some cases we need them loaded eagerly. We have added a #NamedEntityGraph annotation to our entity. In our repository methods we add a "javax.persistence.loadgraph" hint to eagerly load 4 of attributes defined in said annotation. When we invoke that query, Hibernate throws org.hibernate.loader.MultipleBagFetchException: cannot simultaneously fetch multiple bags.
Funnily, when I redefine all of those collection as eagerly fetched Hibernate does fetch them eagerly with no MultipleBagFetchException.
Here is the distilled code.
Entity:
#Entity
#NamedEntityGraph(name = "Post.Full", attributeNodes = {
#NamedAttributeNode("comments"),
#NamedAttributeNode("plusoners"),
#NamedAttributeNode("sharedWith")
}
)
public class Post {
#OneToMany(cascade = CascadeType.ALL, mappedBy = "postId")
private List<Comment> comments;
#ElementCollection
#CollectionTable(name="post_plusoners")
private List<PostRelatedPerson> plusoners;
#ElementCollection
#CollectionTable(name="post_shared_with")
private List<PostRelatedPerson> sharedWith;
}
Query method (all cramped together to make it postable):
#Override
public Page<Post> findFullPosts(Specification<Post> spec, Pageable pageable) {
CriteriaBuilder builder = entityManager.getCriteriaBuilder();
CriteriaQuery<Post> query = builder.createQuery(Post.class);
Root<Post> post = query.from(Post.class);
Predicate postsPredicate = spec.toPredicate(post, query, builder);
query.where(postsPredicate);
EntityGraph<?> entityGraph = entityManager.createEntityGraph("PlusPost.Full");
TypedQuery<GooglePlusFullPost> typedQuery = entityManager.createQuery(query);
typedQuery.setHint("javax.persistence.loadgraph", entityGraph);
query.setFirstResult(pageable.getOffset());
query.setMaxResults(pageable.getPageSize());
Long total = QueryUtils.executeCountQuery(getPostCountQuery(specification));
List<P> resultList = total > pageable.getOffset() ? query.getResultList() : Collections.<P>emptyList();
return new PageImpl<P>(resultList, pageable, total);
}
Any hints on why is this working with eager fetches on entity level, but not with dynamic entity graphs?
I'm betting the eager fetches you think were working, were actually working incorrectly.
When you eager fetch more than one "bag" (an unorder collection allowing duplicates), the sql used to perform the eager fetch (left outer join) will return multiple results for the joined associations as explained by this SO answer. So while hibernate does not throw the org.hibernate.loader.MultipleBagFetchException when you have more than one List eagerly fetched it would not return accurate results for the reason given above.
However, when you give the query the entity graph hint, hibernate will (rightly) complain. Hibernate developer, Emmanuel Bernard, addresses the reasons for this exception to be thrown:
eager fetching is not the problem per se, using multiple joins in one SQL query is. It's not limited to the static fetching strategy; it has never been supported (property), because it's conceptually not possible.
Emmanuel goes on to say in a different JIRA comment that,
most uses of "non-indexed" List or raw Collection are erroneous and should semantically be Sets.
So bottom line, in order to get the multiple eager fetching to work as you desire:
use a Set rather than a List
persist the List index using JPA 2's #OrderColumn annotation,
if all else fails, fallback to Hibernate specific fetch annotations (FetchMode.SELECT or FetchMode.SUBSELECT)
EDIT
related:
https://stackoverflow.com/a/17567590/225217
https://stackoverflow.com/a/24676806/225217
Currently, I have a UserDao class which creates a query using CriteriaQuery, for example to retrieve an instance of User from the database using its ID (let's call that method findById). So far, it's pretty straightforward and that works fine.
However, I also have a GenericDao, which is extended by UserDao. Before userDao.findById returns its results, it passes the criteriaQuery to the GenericDao so I can add some restrictions to the criteriaQuery.
Example:
public class GenericDao
{
private EntityManager entityManager;
protected Object executeCriteriaQuery(CriteriaQuery criteriaQuery)
{
return entityManager.createQuery(prepareQuery(criteriaQuery)).getSingleResult();
}
private CriteriaQuery prepareQuery(CriteriaQuery criteriaQuery)
{
// add restrictions to criteriaQuery
Predicate predicate = ... // some predicate which will be appended to the already formed criteriaQuery
criteriaQuery.where(criteriaBuilder.and(criteriaQuery.getRestriction(), predicate));
return criteriaQuery;
}
}
public class UserDao extends GenericDao
{
public User findById(String id)
{
CriteriaQuery query = criteriaBuilder.createQuery(entityClass);
Root<P> entity = query.from(entityClass);
query.select(entity);
query.where(criteriaBuilder.equal(entity.get("id"), id.toUpperCase()));
return executeCriteriaQuery(query);
}
}
The thing is: the query that is built in the DAOs that extend my GenericDao may contain a JOIN. I want to add a predicate to a JOIN that is added to the criteriaQuery.
From what I can see, I can retrieve the JOINs in the query by using From.getJoins(). However, according to the Javadoc for that method:
Return the joins that have been made from this bound type. Returns empty set if no joins have been made from this bound type. Modifications to the set do not affect the query.
Returns: joins made from this type
How can I add a predicate to JOIN clauses in my criteriaQuery?
After some time, I decided I'm going to implement a different approach to this particular problem. Since I couldn't find any options on how to do this, I attempted to solve it on a JPA implementation level. Since I use Hibernate, I used Hibernate filters. For Eclipselink users: you can use Eclipselink's Multi tenancy feature.
Unfortunately I haven't been able to find a solution on a higher level, so I still have to write a new implementation for this if I were to switch implementations. On the other hand, I probably won't switch to new implementations on a regular basis so that probably won't be an issue.
I am having problem with building a query with JPA Criteria API.
Entities and significant properties:
#Entity
public class Post {
#Id int id;
#OneToMany (mappedBy = "post") Set<Comment> comments;
//...
}
#Entity
public class Comment {
#Id int id;
#ManyToOne Post post;
//...
}
I need a query that will return all posts from db ordered by number of comments (OneToMany in Post).
At first I thought this can be implemented with JPQL like:
SELECT p
FROM Post p
ORDER BY SIZE(p.comments) DESC
But function SIZE(...) can not be used to be ordered by it in JPQL.
So, I found about JPA Criteria API, and tried following:
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<Post> cq = cb.createQuery(Post.class);
Root<Post> p = cq.from(Post.class);
cq.select(p);
cq.orderBy(cb.desc(p.get("comments")));
List<Post> resultList = em.createQuery(cq).getResultList();
With this query I am not getting proper results. I am aware that I am missing getting size of the set 'comments', but don't know how to add that part. I am not really familiar with JPA Criteria API. How should this query look to get all posts ordered by size of its comments field(set)?
CriteriaBuilder.size(Expression) returns an Expression<Integer> that you may use in the ORDER BY clause. This line of code:
p.get("comments")
..returns a Path<X> which extends Expression<X> so it is safe to use the returned value as an argument to Collection.size().
However, the previously quoted line of code is using a particular overloaded version of Path.get() which will make it impossible for the compiler to infer type parameter X. Instead, the type argument will be assumed to be Object. But Collection.size() has declared his Expression-parameter to be a "parameterized type" with an "upper bound" of Collection (this is not reflected accurately in the first reference to CriteriaBuilder.size() in my answer, StackOverflow insist on erasing the type from the method signature. Please see the JavaDocs instead!). So we must provide the type argument explicitly.
Try this:
cq.orderBy(cb.desc(cb.size(p.<Collection>get("comments"))));
What exactly does JPA's fetch strategy control? I can't detect any difference between eager and lazy. In both cases JPA/Hibernate does not automatically join many-to-one relationships.
Example: Person has a single address. An address can belong to many people. The JPA annotated entity classes look like:
#Entity
public class Person {
#Id
public Integer id;
public String name;
#ManyToOne(fetch=FetchType.LAZY or EAGER)
public Address address;
}
#Entity
public class Address {
#Id
public Integer id;
public String name;
}
If I use the JPA query:
select p from Person p where ...
JPA/Hibernate generates one SQL query to select from Person table, and then a distinct address query for each person:
select ... from Person where ...
select ... from Address where id=1
select ... from Address where id=2
select ... from Address where id=3
This is very bad for large result sets. If there are 1000 people it generates 1001 queries (1 from Person and 1000 distinct from Address). I know this because I'm looking at MySQL's query log. It was my understanding that setting address's fetch type to eager will cause JPA/Hibernate to automatically query with a join. However, regardless of the fetch type, it still generates distinct queries for relationships.
Only when I explicitly tell it to join does it actually join:
select p, a from Person p left join p.address a where ...
Am I missing something here? I now have to hand code every query so that it left joins the many-to-one relationships. I'm using Hibernate's JPA implementation with MySQL.
Edit: It appears (see Hibernate FAQ here and here) that FetchType does not impact JPA queries. So in my case I have explicitly tell it to join.
JPA doesn't provide any specification on mapping annotations to select fetch strategy. In general, related entities can be fetched in any one of the ways given below
SELECT => one query for root entities + one query for related mapped entity/collection of each root entity = (n+1) queries
SUBSELECT => one query for root entities + second query for related mapped entity/collection of all root entities retrieved in first query = 2 queries
JOIN => one query to fetch both root entities and all of their mapped entity/collection = 1 query
So SELECT and JOIN are two extremes and SUBSELECT falls in between. One can choose suitable strategy based on her/his domain model.
By default SELECT is used by both JPA/EclipseLink and Hibernate. This can be overridden by using:
#Fetch(FetchMode.JOIN)
#Fetch(FetchMode.SUBSELECT)
in Hibernate. It also allows to set SELECT mode explicitly using #Fetch(FetchMode.SELECT) which can be tuned by using batch size e.g. #BatchSize(size=10).
Corresponding annotations in EclipseLink are:
#JoinFetch
#BatchFetch
"mxc" is right. fetchType just specifies when the relation should be resolved.
To optimize eager loading by using an outer join you have to add
#Fetch(FetchMode.JOIN)
to your field. This is a hibernate specific annotation.
The fetchType attribute controls whether the annotated field is fetched immediately when the primary entity is fetched. It does not necessarily dictate how the fetch statement is constructed, the actual sql implementation depends on the provider you are using toplink/hibernate etc.
If you set fetchType=EAGER This means that the annotated field is populated with its values at the same time as the other fields in the entity. So if you open an entitymanager retrieve your person objects and then close the entitymanager, subsequently doing a person.address will not result in a lazy load exception being thrown.
If you set fetchType=LAZY the field is only populated when it is accessed. If you have closed the entitymanager by then a lazy load exception will be thrown if you do a person.address. To load the field you need to put the entity back into an entitymangers context with em.merge(), then do the field access and then close the entitymanager.
You might want lazy loading when constructing a customer class with a collection for customer orders. If you retrieved every order for a customer when you wanted to get a customer list this may be a expensive database operation when you only looking for customer name and contact details. Best to leave the db access till later.
For the second part of the question - how to get hibernate to generate optimised SQL?
Hibernate should allow you to provide hints as to how to construct the most efficient query but I suspect there is something wrong with your table construction. Is the relationship established in the tables? Hibernate may have decided that a simple query will be quicker than a join especially if indexes etc are missing.
Try with:
select p from Person p left join FETCH p.address a where...
It works for me in a similar with JPA2/EclipseLink, but it seems this feature is present in JPA1 too:
If you use EclipseLink instead of Hibernate you can optimize your queries by "query hints". See this article from the Eclipse Wiki: EclipseLink/Examples/JPA/QueryOptimization.
There is a chapter about "Joined Reading".
to join you can do multiple things (using eclipselink)
in jpql you can do left join fetch
in named query you can specify query hint
in TypedQuery you can say something like
query.setHint("eclipselink.join-fetch", "e.projects.milestones");
there is also batch fetch hint
query.setHint("eclipselink.batch", "e.address");
see
http://java-persistence-performance.blogspot.com/2010/08/batch-fetching-optimizing-object-graph.html
I had exactly this problem with the exception that the Person class had a embedded key class.
My own solution was to join them in the query AND remove
#Fetch(FetchMode.JOIN)
My embedded id class:
#Embeddable
public class MessageRecipientId implements Serializable {
#ManyToOne(targetEntity = Message.class, fetch = FetchType.LAZY)
#JoinColumn(name="messageId")
private Message message;
private String governmentId;
public MessageRecipientId() {
}
public Message getMessage() {
return message;
}
public void setMessage(Message message) {
this.message = message;
}
public String getGovernmentId() {
return governmentId;
}
public void setGovernmentId(String governmentId) {
this.governmentId = governmentId;
}
public MessageRecipientId(Message message, GovernmentId governmentId) {
this.message = message;
this.governmentId = governmentId.getValue();
}
}
Two things occur to me.
First, are you sure you mean ManyToOne for address? That means multiple people will have the same address. If it's edited for one of them, it'll be edited for all of them. Is that your intent? 99% of the time addresses are "private" (in the sense that they belong to only one person).
Secondly, do you have any other eager relationships on the Person entity? If I recall correctly, Hibernate can only handle one eager relationship on an entity but that is possibly outdated information.
I say that because your understanding of how this should work is essentially correct from where I'm sitting.