I have major performance issues when I try to map an entity into a response.
This is the entity:
#Entity
#Table(name = "MyEntity")
public class MyEntity extends BaseEntity {
#Column(name = "someOtherId", nullable = false)
private String someOtherId;
#ElementCollection
#CollectionTable(name = "Phones", joinColumns = #JoinColumn(name = "myEntityId"))
#Column(name = "phone")
private List<String> phones; // <------- we care about this
#ElementCollection
#CollectionTable(name = "Websites", joinColumns = #JoinColumn(name = "myEntityId"))
#Column(name = "websites")
private List<String> websites; // <------- we care about this
#Fetch(FetchMode.SUBSELECT)
#OneToMany(cascade = CascadeType.ALL, fetch = FetchType.LAZY, mappedBy = "myEntity")
private List<ContactEntity> bbb;
#Fetch(FetchMode.SUBSELECT)
#OneToMany(cascade = CascadeType.ALL, fetch = FetchType.LAZY, mappedBy = "myEntity")
private List<AddressEntity> ccc;
}
This is how I use the DAL to fetch it:
List<MyEntity> findByTenantIdAndIdIn(String someOtherId, Set<String> MyEntityIds);
Now when I iterate over List<MyEntity> to map it, and call myEntity.getPhones(), I see that a DB call is being made, which is what causes the slowdown, a 70 seconds slowdown on 1000 entities.
So what can I do to force it to join in the first query it did when I called findByTenantIdAndIdIn?
Notes:
Phones is a simple table with columns: [myEntityId, phone]
The same problem happens with Websites
This has nothing to do with it being an #ElementCollection. Like you figured out, you can use subselect fetching or could also use a batch size for select fetching(the default strategy). Another possibility is to use a fetch join in the query, but be careful when fetch joining multiple collections as that might create a cartesian product which leads to a performance problem cause by too many rows being transfered. A fetch join example HQL query looks like this:
SELECT e FROM MyEntity e LEFT JOIN FETCH e.phones LEFT JOIN FETCH e.websites
I solved it right after I posted this.
I annotated phones and websites with #Fetch(FetchMode.SUBSELECT) which creates a parallel subquery.
Another way to solve this is simply to not use #ElementCollection because it has bad performance, use an Entity instead as they recommend in the video here: https://thorben-janssen.com/hibernate-tips-query-elementcollection/
Related
I know for a fact that with clause on fetch join are not allowed by hibernate
I am using spring data jpa and postgres.
Here is how my entity is designed
public class Organisation {
#Id
private Long id;
#OneToMany(mappedBy = "organisation", cascade = CascadeType.ALL)
#LazyCollection(LazyCollectionOption.EXTRA)
private Set<Assignment> assignments = new HashSet<>();
#OneToMany(mappedBy = "organisation", cascade = CascadeType.ALL)
private List<Event> events;
}
public class Event {
#Id
private Long id;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "organisations_id", nullable = false)
private Organisation organisation;
#OneToMany(mappedBy = "event", cascade = CascadeType.ALL, fetch = FetchType.LAZY)
private Set<EventValue> eventValues = new HashSet<>();
}
public class EventValue {
#Id
private Long id;
#ManyToOne(fetch = FetchType.LAZY, optional = false)
#JoinColumn(name = "event_id")
private Event Event;
#ManyToOne(fetch = FetchType.LAZY, optional = false)
#JoinColumn(name = "assignment_id")
private Assignment assignment;
}
public class Assignment {
#Id
private Long id;
#OneToMany(mappedBy = "assignment", cascade = CascadeType.ALL, fetch = FetchType.LAZY)
private Set<EventValue> eventValues = new HashSet<>();
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "organisation_id", nullable = false)
private Organisation organisation;
}
Kind of a three way mapping. What the above entity design says is:
one organisation can have many events
one events can have many event values
one organisation can have many assignments
one assignment can be mapped to only one organisation and whithin the event of this organisation it is supposed to have only one event value (but as per entity design above it can have set of values which is not directly mapped to assignment)
So, I tried to query something like this.
#Query("select assignment from Assignment left join fetch assignment.organisation org
left join fetch org.event event left join fetch event.eventValues eventValue
with eventValue.assignment.id=?1 where assignment.id=?1)
Assignment getByAssignmentId(Long id);
What am I trying to achive with the query ?
To get assignment with given (id) -> organisation -> list of activities with HashSet containing only ONE activity value mapped to assignment.
The query is obviously going to fail because of using with clause on fetch join. I somehow feel the entity has 3 way dependency so it might be wrong.
I do not want to generic jdbcTemplate solution or SqlResultMapping solution where we need to do some kind of projection and set values manually. Is there a ORM solution to solve this problem ?
The reason why a WITH or ON clause is disallowed for join fetches is pretty simple. Hibernate works on managed entities, which means, once the entities are managed by the current persistence context, changes done to these objects will be flushed back to the database at the end of the transaction.
Now, if you were allowed to use the WITH or ON clause in a join fetch, the querying itself could alter the managed state of a collection, which would lead to UPDATE/DELETE statements to flush the collection changes back. Since this is completely unexpected, but a necessary side effect, it is disallowed.
Having said that, this is a perfect use case for Blaze-Persistence Entity Views.
Blaze-Persistence is a query builder on top of JPA which supports many of the advanced DBMS features on top of the JPA model. I created Entity Views on top of it to allow easy mapping between JPA models and custom interface defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure the way you like and map attributes(getters) via JPQL expressions to the entity model. Since the attribute name is used as default mapping, you mostly don't need explicit mappings as 80% of the use cases is to have DTOs that are a subset of the entity model.
A DTO mapping for your model could look as simple as the following
#EntityView(Assignment.class)
interface AssignmentDto {
Long getId();
OrganisationDto getOrganisation();
}
#EntityView(Organisation.class)
interface OrganisationDto {
Long getId();
List<EventDto> getEvents();
}
#EntityView(Event.class)
interface EventDto {
Long getId();
#Mapping("eventValues[assignment.id = VIEW_ROOT(id)]")
EventValueDto getEventValue();
}
#EntityView(EventValue.class)
interface EventValueDto {
Long getId();
// Other stuff
}
The JOIN condition is modeled in the mapping expression eventValues[assignment.id = VIEW_ROOT(id)] which translates to what you would expect.
Querying is a matter of applying the entity view to a query, the simplest being just a query by id.
AssignmentDto dto = entityViewManager.find(entityManager, AssignmentDto.class, id);
But the Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
It will only fetch the mappings that you tell it to fetch.
We are using Spring Data repositories with Hibernate 5.x
We have a entity graph with a deep hierarchy.
The mapping looks like this:
#Entity
public class FooBar {
#OneToMany(mappedBy = "fooBar", cascade = CascadeType.ALL, orphanRemoval = true)
private Set<Foo> chassis = new HashSet<>(0);
...
}
#Entity
public class Foo {
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "foobar_id")
private FooBar fooBar;
#OneToMany(mappedBy = "foo", cascade = CascadeType.ALL, orphanRemoval = true)
private Set<Bar> chassis = new HashSet<>(0);
...
}
#Entity
public class Bar {
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "foo_id")
private FooBar foo;
...
}
As you can see the FooBar entity has a set of Foo entities. Each Foo entity contains more Bar entities and so on.
We use the Fetchgraph feature to load the FooBar entity with the relations we need during runtime to avoid n+1 query issue when fetching lazy associations.
After the service call to load the entity graph the transaction has ended and the entity is detached.
When calling save on the FooBar entity at a later time, this causes multiple select statements. Each fetching one of the child entities.
I know that this comes from the entitymanager merge() call which fetches the object graph from the db before copying state changes from the detached objects.
I have two questions:
Why is hibernate not able to join these statements to one big select like what happens when using the fetchgraph?
When i remove all cascade options from the relations it still causes multiple selects but only attributes of the top, FooBar entity, will be updated. Why is hibernate still fetching all loaded child entites during merge even with no cascade merge?
Thanks
You can use session.update instead of merge to overcome this issue.
Session session = entityManager.unwrap(Session.class);
for (Post post: posts) {
session.update(post);
}
I have similar issue with your case, and the reason is the setting of cascading CascadeType.ALL on the #OneToMany association. Updating and merging the parent entity cause a lot of select on the child association.
#Entity
public class FooBar {
#OneToMany(mappedBy = "fooBar", cascade = CascadeType.ALL, orphanRemoval = true)
private Set<Foo> chassis = new HashSet<>(0);
...
}
I fix my case by reducing the scope of cascading, only PERSIST and REMOVE is sufficient
#OneToMany(mappedBy = "fooBar", cascade = {CascadeType.PERSIST, CascadeType.REMOVE}, orphanRemoval = true)
private Set<Foo> chassis = new HashSet<>(0);
What is difference between codes?
#LazyCollection(LazyCollectionOption.FALSE)
#OneToMany(mappedBy = "siteesTypeSite", cascade = CascadeType.ALL,fetch = FetchType.LAZY)
public Set<Sites> getSitees() {
return sitees;
}
and
#OneToMany(mappedBy = "siteesTypeSite", cascade = CascadeType.ALL,fetch = FetchType.EAGER)
public Set<Sites> getSitees() {
return sitees;
}
As for me both of them makes similar result, but second case is more cleaner. If i mistook correct me please.
The main difference between the annotations is that #OneToMany is a pure JPA annotation. Whereas #LazyCollection is Hibernate specific.
So if you want your code to be portable across various JPA providers you should use JPA annotations.
Update
To explain between those two annotation, consider the OneToMany relationship between Department -> Employee
Case 1:
#OneToMany(cascade = CascadeType.ALL, fetch = FetchType.EAGER, orphanRemoval = true)
#JoinColumn(name = "DEPARTMENT_ID")
private List<Employee> employees = new ArrayList<>();
if you fetch a Department object from the db using :
entityManager.find(Department.class, 1L);
following query gets fired to fetch the data
SELECT department0_.DEPARTMENT_ID AS DEPARTMENT_ID1_0_0_,
department0_.DEPARTMENT_NAME AS DEPARTMENT_NAME2_0_0_,
department0_.LOCATION AS LOCATION3_0_0_,
employees1_.DEPARTMENT_ID AS DEPARTMENT_ID3_1_1_,
employees1_.EMPLOYEE_ID AS EMPLOYEE_ID1_1_1_,
employees1_.EMPLOYEE_ID AS EMPLOYEE_ID1_1_2_,
employees1_.DEPARTMENT_ID AS DEPARTMENT_ID3_1_2_,
employees1_.EMPLOYEE_NAME AS EMPLOYEE_NAME2_1_2_
FROM DEPARTMENT department0_
LEFT OUTER JOIN EMPLOYEE employees1_
ON department0_.DEPARTMENT_ID =employees1_.DEPARTMENT_ID
WHERE department0_.DEPARTMENT_ID=?
so it means it will fetch all the data in a single query at once.
Case 2:
#LazyCollection(LazyCollectionOption.FALSE)
#OneToMany(cascade = CascadeType.ALL,fetch = FetchType.LAZY, orphanRemoval = true)
#JoinColumn(name = "DEPARTMENT_ID")
private List<Employee> employees = new ArrayList<>();
similary if you fetch a Department object from the db using :
entityManager.find(Department.class, 1L);
following queries gets fired to fetch the data :
SELECT department0_.DEPARTMENT_ID AS DEPARTMENT_ID1_0_0_,
department0_.DEPARTMENT_NAME AS DEPARTMENT_NAME2_0_0_,
department0_.LOCATION AS LOCATION3_0_0_
FROM DEPARTMENT department0_
WHERE department0_.DEPARTMENT_ID=?
SELECT employees0_.DEPARTMENT_ID AS DEPARTMENT_ID3_1_0_,
employees0_.EMPLOYEE_ID AS EMPLOYEE_ID1_1_0_,
employees0_.EMPLOYEE_ID AS EMPLOYEE_ID1_1_1_,
employees0_.DEPARTMENT_ID AS DEPARTMENT_ID3_1_1_,
employees0_.EMPLOYEE_NAME AS EMPLOYEE_NAME2_1_1_
FROM EMPLOYEE employees0_
WHERE employees0_.DEPARTMENT_ID=?
So to summarize, in first case since the FetchType is EAGER Employees are fetched eagerly along with Department in a single JOIN query.
And,
In second case, Employees are fetched with Department but since the FetchType is LAZY a seperate query will be fired to fetch Employees. And if you remove #LazyCollection(LazyCollectionOption.FALSE) Employees wont be fetched at all until you access Employees on Department instance.
I have CATEGORY, AD and CATEGORY_AD table, typical many to many relationship. Somehow nothing is inserted into CATEGORY_AD table. What am I missing?
In Category.java:
#ManyToMany
#JoinTable(name = "CATEGORY_AD", joinColumns = {
#JoinColumn(name = "CATEGORY_ID", referencedColumnName = "ID") }, inverseJoinColumns = {
#JoinColumn(name = "AD_ID", referencedColumnName = "ID") })
private List<Ad> ads;
In Ad.java:
#ManyToMany(mappedBy = "ads")
private List<Category> categories;
In my Service class:
Category laCat = new Category();
laCat.setId(categoryId);
laCat.getAds().add(ad);
ad.getCategories().add(laCat);
ad = adRepository.saveAndFlush(ad);
You are saving the 'owned' side instead of the 'owning' side.
Every ManyToMany relationship must have an owner, which in your case is Category. You can see that because in Category you have the definition of the #JoinTable in the ads List, and in Ad you refer to that list by #ManyToMany(mappedBy = "ads").
Whenever you save the owning side of the relationship, this will trigger a save on the join table too, but not the other way around. So saving the owned side (Ad) will do nothing on the CATEGORY_ADtable.
You should do something like this:
Category laCat = new Category();
laCat.setId(categoryId);
laCat.getAds().add(ad);
// Next line will insert on CATEGORY and CATEGORY_AD tables
laCat = categoryRepository.saveAndFlush(category);
// We add the category to the ad object to keep both sides in sync
ad.getCategories().add(laCat);
You can see that even if a save on Category triggers a save on the join table, it's still our responsibility manually add the category to the categories Listin the ad object so both sides are in sync, having the same elements. And there's no need to save the ad object already.
Try to save Category object also (before you invoke ad.getCategories().add(laCat);)
Please use hibernate #Cascade annotations on List<Category> categories;
import org.hibernate.annotations.Cascade;
import org.hibernate.annotations.CascadeType;
#ManyToMany(mappedBy = "ads")
#Cascade(value = CascadeType.SAVE_UPDATE)
private List<Category> categories;
If you prefer using JPA annotations then you can try this:
#ManyToMany(mappedBy = "ads", cascade = CascadeType.PERSIST)
#Cascade(value = CascadeType.SAVE_UPDATE)
private List<Category> categories;
Please check this thread: what is cascading in hibernate
There are two tables with #OneToMany and #ManyToOne bidirectional relation, like this:
#Entity
public class Asset {
private int id;
private int count;
#OneToMany
private Set<Dealing> dealings;
...
}
#Entity
public class Dealing {
private int id;
...
#ManyToOne
#JoinColumn(name = "customer_id", nullable = false, updatable = false)
private Customer customer;
#ManyToOne
#JoinColumn(name = "product_id", nullable = false, updatable = false)
private Product product;
#ManyToOne(cascade = CascadeType.ALL)
private Asset asset;
}
all things sound OK, but when I want to search data using Restriction like this,
session.createCriteria(Asset.class).add(Restrictions.eq("dealings.customer.id", customerId)).add(Restrictions.eq("dealing.product.id", productId)).list();
In this level I get this error,
could not resolve property: dealings.customer of: com.project.foo.model.Asset
one of the solutions are to change my strategy but i wasted time to find this,btw I don't have any idea about it, do you ?
First of all, you don't have a bidirectional OneToMany association, but two unrelated unidirectional associations. In a bidirectional OneToMany association the One side must be marked as the inverse of the Many side using the mappedBy attribute:
#OneToMany(mappedBy = "asset")
private Set<Dealing> dealings;
Second, using the criteria API for such static queries is overkill, and leads to code that is harder to read than necessary.I would simply use HQL which is much easier to read. Criteria should be used for dynamic queries, IMHO, but not for static ones:
select asset from Asset asset
inner join asset.dealings dealing
where dealing.customer.id = :customerId
and dealing.product.id = :productId
Whether you use HQL or Criteria, you can't use asset.dealings.customer, since asset.dealings is a collection. A collection doesn't have a customer attribute. To be able to reference properties from the Dealing entity, you need a join, as shown in the above HQL query. And it's the same for Criteria:
Criteria criteria = session.createCriteria(Asset.class, "asset");
criteria.createAlias("asset.dealings", "dealing"); // that's an inner join
criteria.add(Restrictions.eq("dealing.customer.id", customerId);
criteria.add(Restrictions.eq("dealing.product.id", productId);