Develop Reference

How to fetch two-level deep #OneToMany association in Spring Data and avoid MultipleBagFetchException

I'm still a JPA beginner and would like to know how to best use Spring powerful features to fetch the following simple structure (associations are lazy by default but I have a use case where the whole structure should be loaded without proxies, possibly with the lowest number of SQL queries generated). Simplified entities concerned:
#Entity
public class Bundle {
#Id
private Long id;
#OneToMany(mappedBy = "bundle", cascade = CascadeType.ALL, orphanRemoval = true)
private List<Group> groups = new ArrayList<>();
}
...
#Entity
public class Group {
#Id
private Long id;
#ManyToOne()
#JoinColumn(name = "BUNDLE_ID")
private Bundle bundle;
#OneToMany(mappedBy = "group", cascade = CascadeType.ALL, orphanRemoval = true)
private List<Element> elements = new ArrayList<>();
...
public class Element {
#Id
private Long id;
#ManyToOne
#JoinColumn(name = "GROUP_ID")
private Group group;
}
My attempt to find all groups and elements under a given bundle (in order to efficiently process them and convert to a DTO later before returning from an endpoint) was to fetch inside #Query
public interface BundleRepository extends JpaRepository<Bundle, Long> {
#Query("SELECT bundle FROM Bundle bundle "
+ "JOIN FETCH bundle.groups groups "
+ "JOIN FETCH groups.elements "
+ "WHERE bundle.id = :id")
Optional<Bundle> fetchBundle(#Param("id") long id);
}
However, this results in org.hibernate.loader.MultipleBagFetchException: cannot simultaneously fetch multiple bags. I did some reading on the subject and found that changing Lists to Sets may do the trick, on the other hand some sources discourage from this.
This double #OneToMany structure seems very ordinary and multiple JOINs also nothing uncommon but nevertherless I'd like to ask you to point out the right approach. Maybe fetching groups for a bundle separately and then for every group fetching its elements? This would be 1 + number of groups queries, not a bit wasteful? Please let me know if considering this as a trade-off in this manner is a step in the good direction.

Assuming that a. duplicates aren't allowed in the collections, and b. you don't care about the iteration order of the entities in the entity object collections for groups and elements (you could still impose ordering when transforming to DTOs using some property of them), then the headline answer is:
Yes, change from List to a Set which communicates both to callers, and to the JPA implementation that duplicates aren't allowed and iteration order can be undefined (which would also solve the bag exception - but at the cost of a gigantic cross-join).
To avoid the gigantic cross join that might result from loading both sets at the same time, do the load of the graph in 2 queries (as long as both participate in the same transaction):
#Query("SELECT bundle FROM Bundle bundle "
+ "JOIN FETCH bundle.groups groups "
+ "WHERE bundle.id = :id")
Optional<Bundle> fetchBundle(#Param("id") long id);
}
and then a second call to populate the elements on the groups:
#Query("SELECT groups FROM Group groups "
+ "JOIN FETCH groups.elements "
+ "WHERE groups.bundle.id = :id")
Collection<Group> fetchBundleGroups(#Param("id") long id);
}
Note that you can in fact ignore the response value of the second call and just do ...
Optional<Bundle> maybeBundle = repository.fetchBundle(id);
repository.fetchBundleGroups(id);
// now, maybeBundle.get().groups.elements will all be populated
due to a useful feature of the JPA spec, which requires that, within a given JPA EntityManager persistence context scope (usually the same as transaction scope), the same object instance is returned for an entity with the same ID, and thus effects from the second query (to populate the elements on the Group objects) will use the same instances returned from the first query.
Worth noting that the same technique also works if there were to be multiple collections on one of the entities - e.g. Bundle.authors - you would just add a 3rd query to populate the authors collection on the Bundle and call that as well to avoid the cross-join between authors and groups.

ManyToMany N+1 problem - All right side not being returned

I have written code that has N+1 problem.
It is a BI-directional ManyToMany relationship.
I'm attempting to solve it using LEFT JOIN FETCH
The issue i am having is that when i look at the results, only 1 single record from the right side is being joined. There should be many more for each left record.
The left records come back correct, but the Set attribute only ever contains 1 instance from the right.
value = "SELECT cat from CatEntity cat "
+ "INNER JOIN cat.owner o "
+ "LEFT JOIN FETCH cat.toys "
+ "WHERE (o.name = :something)
I get all 69 cat records, but the cats that have toys only have 1 toy.
If i remove the LEFT JOIN FETCH. I get all cats and all the complete toys... and of course the N+1 problem.
When i add in LEFT JOIN FETCH.. it solves N+1 problem but doesnt return full toy set.
public class CatEntity implements Serializable {
#JoinTable(
name = "cat_toy",
joinColumns = {#JoinColumn(name = "cat_id")},
inverseJoinColumns = {#JoinColumn(name = "toy_id")})
#Fetch(FetchMode.JOIN)
#OrderBy(value = "toy_index desc")
private Set<ToyEntity> toys;
}
}
public class ToyEntity implements Serializable {
#ManyToMany(mappedBy = "toys")
Set<CatEntity> cats;
}
How do i avoid N+1 problem but fully populate the Set?
//Company frowns upon publishing code so i use cat/toy as example replacing class names.

I found the solution to my problem. I will post here hoping it helps others.
I was using
#Fetch(FetchMode.JOIN)
in my code. I needed to use
#Fetch(FetchMode.SUBSELECT)
This is specifically designed for collections. It populates the right side of a ManyToMany bidirection relationship using 2 queries rather than N+1.
It first gets all the cats. It then gets all the toys. Only 2 queries are executed.
You should NOT put LEFT JOIN FETCH in your query itself. Just add the fetch mode to the entity mapping.
hope this helps others.

Hibernate JPA recursive query

I have the following query, I'm using hibernate a JPA provider:
entityManager().createQuery(
"SELECT page FROM ProjectPage page"
+" left join fetch page.categorySet as category"
+ " where page.id = :id "
+ " and category.parentCategory is null "
+ " and (category.status is null or category.status != :status_val) "
,ProjectPage.class).setParameter("id", id).setParameter("status_val", Status.DELETED).getSingleResult();
and below are the entities of ProjectPage and Category respectively:
#Entity
#Table(name="project_page")
#Configurable
public class ProjectPage {
#OneToMany( mappedBy = "parentPage")
private Set<Category> categorySet = new HashSet<Category>();
}
#Configurable
#Table(name="category")
#Entity
public class Category{
#OneToMany(cascade = CascadeType.ALL, mappedBy = "parentCategory",fetch=FetchType.EAGER)
private Set<com.se.dataadminbl.model.Category> categorySet = new HashSet<com.se.dataadminbl.model.Category>();
}
in the above query, i'm trying to fetch a ProjectPage along with its categorySet and as shown above class Category contains a set of its type, so every ProjectPage object will contain a set of Category and each object inside this set will contains a set of Category, now the problem is that when i retrieve the ProjectPage object the conditions in the where clause applied only on the first level set of Category not on each set inside each Category, i want to make a recursive query so that i can apply the where condition to the nth level instead of doing that with code, i tried to use interceptor but doesn't work, any idea of how to do that?

The WHERE condition will always filter out nulls when you reference a LEFT JOIN column in your WHERE clause. So the end result is an INNER JOIN.
Neither JPA nor Hibernate support recursive queries, because there's no one and only one standard implementation amongst all databases Hibernate supports.
In case you use PostgreSQL, you can use a Common Table Expression.

JPA / EclipseLink: Joined WHERE query does not give expected result

Now that's very confusing... I have a JPA entity Order that references an entity User. The User can be either buyer or seller of the Order.
Because both buyer and seller can enter additional information for an order, I moved that to an extra entity OrderUserData. There might or might not be a corresponding OrderUserData object, but IF one exists, the user should only be able to see the entry they created (based on USER_ID) and not the one of the other party.
The entities look like this:
#Entity
#Table(name = "T_ORDER")
public class Order {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#ManyToOne
#JoinColumn(name = "SELLER_ID")
private User seller;
#ManyToOne
#JoinColumn(name = "BUYER_ID")
private User buyer;
#OneToMany(mappedBy = "order", fetch = FetchType.EAGER)
private List<OrderUserData> userData = new ArrayList<>();
//..
}
--
#Entity
#Table(name = "T_ORDER_USERDATA")
public class OrderUserData {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#ManyToOne
#JoinColumn(name = "ORDER_ID")
private Order order;
#ManyToOne
#JoinColumn(name = "USER_ID")
private User user;
private String comment;
//...
}
( User is not very exciting, just ID and basic name fields )
Now when I'm trying to select the appropriate data to display in the website, I have a problem:
String qry = "SELECT o FROM Order o LEFT JOIN o.userData ud "
+ " WHERE (o.seller.id = :userId OR o.buyer.id = :userId)"
+ " AND ( ud.user IS NULL OR ud.user.id = :userId )";
TypedQuery<Order> query = em.createQuery(qry, Order.class);
query.setParameter("userId", userId);
Let's say I execute this, setting userId to 2:
My Database looks like this:
ORDER
=====
ID SELLER_ID BUYER_ID
1 1 2
2 2 3
3 3 1
ORDER_USERDATA
===============
ID ORDER_ID USER_ID COMMENT
1 1 1 Comment that only user 1 should see
2 1 2 Comment that only user 2 should see
But unlike you would expect, when executing the above query, both records are included in the userData list! It seems like JPA is executing two queries (despite the EAGER fetch) and ignoring the WHERE on the second one. Why is that? And what other solution than to loop through the userData list on Java level and kick out the entry that the appropriate user should not see?

There is no way to load OrderUserData objects inside an Order object using a query. Maybe you're confusing the ORM functionality, mapping rows in the database to Java objects, with the query functionality.
Mapping means 1-1 correspondence between rows and objects, hence Order objects always contain all OrderUserData objects for each OrderUserData row related to Order rows.
The fetch type is just a loading strategy, determining at which time are the related objects fetched, as soon as the containing object is loaded (EAGER) or as soon as the contained objects are accessed (LAZY).
You can obtain your list issuing a query on OrderUserData objects with the proper filters and getting Order objects from each of them, i.e.
SELECT ud FROM OrderUserData ud WHERE (ud.order.seller.id = :userId
OR ud.order.buyer.id = :userId) AND ( ud.user IS NULL OR ud.user.id =
:userId )

your query seems to work well as it selects properly Order entity. Then JPA fetch all the OrderUserData child of the selected Order : that's because oneToMany join is not filtered.
I don't think it is possible to modelize pre-filtered oneToMany with eclipseLink (like Hibernate #FILTER), so you should remove it and map orderUserDataId field only. Then you can fetch your entities in 1 query, but they will not be linked
SELECT o, ud FROM Order o, o.userData ud WHERE (o.seller.id = :userId OR o.buyer.id = :userId) AND ( ud.orderUserDataId = o.id and (ud.user IS NULL OR ud.user.id = :userId) )";
On the other hand, if the oneToMany is required by other use cases, then you can create 2 different Order entities :
1 "OrderLight" without the oneToMany
1 "OrderFull" with the oneToMany, derived from OrderLight.

While user3580357 and remigio have already given the correct answer as to why this doesn't work, might I suggest that you create a view on database level.
Something like (might need to be adapted for your needs or RDBMS):
CREATE OR REPLACE VIEW
ORDER_WITH_USERDATA
AS
SELECT o.*, oud.*
FROM ORDER o
LEFT JOIN ORDER_USERDATA oud
ON o.id = oud.order_id
This will essentially give you two different "logical" records for every order. You can then create an additional JPA entity that works on this view and do your SELECT/WHERE... without needing to (LEFT)JOIN at all.

Hibernate Criteria Join problem

I have a 2 classes that share a UUID and are uni-directionally mapped. I use the UUID to group related rows, and this group shares many details (this is just an example):
#Entity #Table
class Something {
#Id #Column("something_id")
private Long id;
private String uuid = UUID.randomUUID().toString();
#OneToMany
#JoinColumn("uuid")
private List<Detail> details = new LinkedList<Detail>();
}
#Entity #Table
class Detail {
#Id #Column("detail_id")
private Long id;
private String value;
private String uuid;
}
I'm attempting to use Criteria:
Criteria c = getSession().createCriteria(Something.class).createAlias("details", "detail").add(Restrictions.eq("detail.value", someValue));
This is all fine and dandy, but I'm not getting results because of the join:
inner join DETAIL d1_ on this_.SOMETHING_ID=d1_.UUID
Is it possible to specify:
inner join DETAIL d1 on this_.UUID=d1.UUID
I would have expected the join to use the #JoinColumn annotaiton to find the column to join on. I see that I can specify a join type, but I don't see a way to specify the actual column.

I would have expected the join to use the #JoinColumn annotation to find the column to join on. I see that I can specify a join type, but I don't see a way to specify the actual column.
The join is using the JoinColumn annotation since it's joining on d1_.UUID. However, because you didn't specify the referencedColumnName element, the foreign key is assumed to refer to the primary key of the referenced table (this_.SOMETHING_ID), hence the obtained result.
In other words, try this:
#OneToMany
#JoinColumn(name="uuid", referencedColumnName="uuid")
private List<Detail> details = new LinkedList<Detail>();
I'm not sure to understand the benefit but let's say it's another story.