Just face the N+1 query problem with such Spring Data repository
public interface ToDoRepository extends CrudRepository<ToDo, Long> {
#Query("select new com.package.repr.ToDoRepr(t) from ToDo t " +
"where t.user.id = :userId")
List<ToDoRepr> findToDosByUserId(#Param("userId") Long userId);
}
I see in the logs one such query
Hibernate:
select
todo0_.id as col_0_0_
from
todos todo0_
where
todo0_.user_id=?
]
And N such queries
Hibernate:
select
todo0_.id as id1_0_0_,
todo0_.description as descript2_0_0_,
todo0_.target_date as target_d3_0_0_,
todo0_.user_id as user_id4_0_0_,
user1_.id as id1_1_1_,
user1_.password as password2_1_1_,
user1_.username as username3_1_1_
from
todos todo0_
left outer join
users user1_
on todo0_.user_id=user1_.id
where
todo0_.id=?
ToDoRepr is a simple POJO. With constructor which accepts ToDo entity as a parameter.
Here are two JPA entities I use in this query
#Entity
#Table(name = "todos")
public class ToDo {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column
private String description;
#ManyToOne
#JoinColumn(name = "user_id")
private User user;
#Column
private LocalDate targetDate;
// geters, setters, etc.
}
#Entity
#Table(name = "users")
public class User {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(unique = true, nullable = false)
private String username;
#Column(nullable = false)
private String password;
#OneToMany(
mappedBy = "user",
cascade = CascadeType.ALL,
orphanRemoval = true
)
private List<ToDo> todos;
// geters, setters, etc.
}
UPD. It possible to resolve the problem by that query but why it's not working with constructor which accepts entity as a parameter?
public interface ToDoRepository extends CrudRepository<ToDo, Long> {
#Query("select new com.package.repr.ToDoRepr(t.id, t.description, t.user.username, t.targetDate) " +
"from ToDo t " +
"where t.user.id = :userId")
List<ToDoRepr> findToDosByUserId(#Param("userId") Long userId);
}
This is a very common question so I created the article Eliminate Spring Hibernate N+1 queries detailing the solutions
The best practice with Hibernate is to define all associations as Lazy to avoid fetching it when you don't need it.
For more reasons, check Vlad Mihalcea's article https://vladmihalcea.com/eager-fetching-is-a-code-smell/
For fixing your issue, in your class ToDo, you should define the ManyToOne as Lazy:
#Entity
#Table(name = "todos")
public class ToDo {
...
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "user_id")
private User user;
...
// geters, setters, etc.
}
If you need to access the user in your ToDoRepr, it will not be loaded by default so you need to add it to your query:
JPQL, use JOIN FETCH:
public interface ToDoRepository extends CrudRepository<ToDo, Long> {
#Query("select new com.package.repr.ToDoRepr(t) " +
"from ToDo t " +
"inner join fetch t.user " +
"where t.user.id = :userId")
List<ToDoRepr> findToDosByUserId(#Param("userId") Long userId);
}
JPA, use EntityGraph:
public interface ToDoRepository extends CrudRepository<ToDo, Long> {
#EntityGraph(attributePaths = {"user"})
List<ToDoRepr> findToDosByUser_Id(Long userId);
}
I want to collect here some workaround about this my own question. There is a simple solution without explicit JPQL queries. Spring Data JPA can treat any POJO with proper getters and setters as a projection.
Just that works perfectly for me
public interface ToDoRepository extends CrudRepository<ToDo, Long> {
List<ToDoRepr> findToDosByUser_Id(Long userId);
}
Related
I currently set my entities in Hibernate using OnetoMany joins within my entities. I need to pull InvTran documents nested within my InventoryReceiptsLine Result. My issue is that I don't want to add data to these tables this way and rather setup a custom query to pull this type of report. My goal is to get the following output where the invTran populates if there is an existing order assignment to InventoryReceiptLine.
{
"receipt_number": 5000027,
"items": [
{
"receipt_number": 5000027,
"inv_mast_uid": 22428,
"qty_received": 100,
"unit_of_measure": "EA",
"item_id": "TRI620-104-706",
"item_desc": "HSS104 CLAMP 4-8/64\"-7\"",
"invTran": []
},
{
"receipt_number": 5000027,
"inv_mast_uid": 13628,
"qty_received": 200,
"unit_of_measure": "EA",
"item_id": "DIXHSS72",
"item_desc": "ALL STAINLESS WORMGEAR CLAMPS",
"invTran": []
},
{
"receipt_number": 5000027,
"inv_mast_uid": 22412,
"qty_received": 100,
"unit_of_measure": "EA",
"item_id": "TRI620-008-706",
"item_desc": "620-008 CLAMP (HAS 8)",
"invTran": [
{
"transaction_number": 4245,
"document_no": 1000064,
"qty_allocated": 51,
"unit_of_measure": "EA",
"inv_mast_uid": 22412,
"oe": null,
"receipt_id": 5000027
}
]
}
]
}
I tried setting up my repository as follows to eliminate the need for the OneToMany join on the InvTran object but I get the following error
"Cannot read field "value" because "s1" is null [SELECT IHDR FROM com.emrsinc.patch.models.InventoryReceiptsHDR IHDR left join fetch InventoryReceiptsLine IRL on IRL.receipt_number = IHDR.receipt_number left join fetch InvTran on InvTran.sub_document_no = IRL.receipt_number WHERE IHDR.receipt_number = :receipt]"
when building by query as
#Query(value = "SELECT IHDR " +
"FROM InventoryReceiptsHDR IHDR " +
"left join fetch InventoryReceiptsLine IRL on IRL.receipt_number = IHDR.receipt_number " +
"left join fetch InvTran on InvTran.sub_document_no = IRL.receipt_number " +
"WHERE IHDR.receipt_number = :receipt")
InventoryReceiptsHDR getReceipt(#Param("receipt") int receipt);
Inventory ReceiptsHDR class
#Entity
#Getter
#Table(name = "inventory_receipts_hdr")
public class InventoryReceiptsHDR implements Serializable {
#Id
private Integer receipt_number;
#OneToMany
#JoinColumn(name = "receipt_number")
#JsonProperty("items")
private Set<InventoryReceiptsLine> inventoryReceiptsLineList;
}
InventoryReceiptsLine class
#Entity
#Getter
#Table(name = "inventory_receipts_line")
public class InventoryReceiptsLine implements Serializable {
#Id
#JsonBackReference
private Long line_number;
private Long receipt_number;
private Long inv_mast_uid;
private Long qty_received;
private String unit_of_measure;
#OneToOne
#JoinColumn(name = "inv_mast_uid", insertable = false, updatable = false)
#JsonUnwrapped
private InvMast invMast;
#OneToMany
#JoinColumns({
#JoinColumn(name = "inv_mast_uid", referencedColumnName = "inv_mast_uid"),
#JoinColumn(name="sub_document_no", referencedColumnName = "receipt_number")
})
private Set<InvTran> invTran;
}
InvTran class
#Entity
#Getter
#Table(name = "inv_tran")
public class InvTran implements Serializable {
#Id
private Integer transaction_number;
private Integer document_no;
private Integer qty_allocated;
private String unit_of_measure;
#JsonProperty(value = "receipt_id")
private Integer sub_document_no;
private Integer inv_mast_uid;
#OneToOne()
#JoinColumn(name = "transaction_number", referencedColumnName = "order_no")
private OeHDR oe;
}
What you want here is an ad-hoc join and you can't fetch entities this way, you need a DTO.
I think this is a perfect use case for Blaze-Persistence Entity Views.
I created the library to allow easy mapping between JPA models and custom interface or abstract class defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure(domain model) the way you like and map attributes(getters) via JPQL expressions to the entity model.
A DTO model for your use case could look like the following with Blaze-Persistence Entity-Views:
#EntityView(InventoryReceiptsHDR.class)
public interface InventoryReceiptsHDRDto {
#IdMapping
Integer getId();
Set<InventoryReceiptsLineDto> getInventoryReceiptsLineList();
#EntityView(InventoryReceiptsLine.class)
interface InventoryReceiptsLineDto {
#IdMapping("line_number")
Long getId();
Long getReceipt_number();
#Mapping("InvTran[sub_document_no = VIEW(receipt_number)]")
Set<InvTranDto> getInvTran();
}
#EntityView(InvTran.class)
interface InvTranDto {
#IdMapping("transaction_number")
Integer getId();
Integer getDocument_no();
}
}
Querying is a matter of applying the entity view to a query, the simplest being just a query by id.
InventoryReceiptsHDRDto a = entityViewManager.find(entityManager, InventoryReceiptsHDRDto.class, id);
The Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
Page<InventoryReceiptsHDRDto> findAll(Pageable pageable);
The best part is, it will only fetch the state that is actually necessary!
I have two (Hibernate-based) Spring Data JPA domain classes, the "One" side Customer.class:
#Entity
#Table(name = "sys_customer")
#Data
public class Customer implements Serializable {
#Id
#Column(name = "cust_id")
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "cust_name")
private String customerName;
#OneToMany(cascade = CascadeType.ALL, mappedBy = "customer")
private Set<Order> orders;
}
and the "Many" side Order.class:
#Entity
#Table(name = "sys_order")
#Getter
#Setter
#AllArgsConstructor
#NoArgsConstructor
public class Order implements Serializable {
#Id
#Column(name = "order_id")
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "order_name")
private String orderName;
#ManyToOne
#JoinColumn(name = "order_cust_id", referencedColumnName = "cust_id")
private Customer customer;
public Order( String orderName) {
this.orderName = orderName;
}
public Order(String orderName, Customer customer) {
this.orderName = orderName;
this.customer = customer;
}
}
I have OrderRepository interface which extends JpaRepository interface and JpaSpecificationExecutor interface:
public interface OrderRepository extends JpaRepository<Order, Long>, JpaSpecificationExecutor<Order> {
}
I have a OrderSpecification.class with the static method searchByCustomerName:
public class OrderSpecification {
public static Specification<Order> searchByCustomerName(String customerName) {
return new Specification<Order>() {
#Override
public Predicate toPredicate(Root<Order> root, CriteriaQuery<?> query, CriteriaBuilder criteriaBuilder) {
Join<Order, Customer> join = root.join("customer");
return criteriaBuilder.like(join.get("customerName"), "%" + customerName + "%");
//return criteriaBuilder.like(root.get("customer").get("customerName"), "%" + customerName + "%");
}
};
}
}
To find the differences between get("property") chain and Join, I wrote a simple test method and comment out the above OrderSpecificatin.class code
#Test
#Transactional
public void testFindOrderByCustomerName(){
String name = "adam";
List<Order> orders = orderRepository.findAll(OrderSpecification.searchByCustomerName(name));
for(Order order: orders){
Customer customer = order.getCustomer();
log.info(new StringBuilder().append(customer.getId()).append(" ").append(customer.getCustomerName()).toString());
}
}
I found that:
get("property") chain use a cross-join(which is very bad performancing) while Join use inner-join(since ManyToOne() by default is Fetch= FetchType.EAGER)
/* get("property") chain: Hibernate: select order0_.order_id as
order_id1_1_, order0_.order_cust_id as order_cu3_1_,
order0_.order_name as order_na2_1_ from sys_order order0_ cross join
sys_customer customer1_ where order0_.order_cust_id=customer1_.cust_id
and (customer1_.cust_name like ?) Hibernate: select customer0_.cust_id
as cust_id1_0_0_, customer0_.cust_name as cust_nam2_0_0_ from
sys_customer customer0_ where customer0_.cust_id=? */
/** * "Join": * Hibernate: select order0_.order_id as order_id1_1_,
order0_.order_cust_id as order_cu3_1_, order0_.order_name as
order_na2_1_ from sys_order order0_ inner join sys_customer customer1_
on order0_.order_cust_id=customer1_.cust_id where customer1_.cust_name
like ? * Hibernate: select customer0_.cust_id as cust_id1_0_0_,
customer0_.cust_name as cust_nam2_0_0_ from sys_customer customer0_
where customer0_.cust_id=? */
My questions are:
Can I specify the Join type(inner, all three outers) or Fetch Type(LAZY, EAGER) when using get("property") chain approach to avoid cross-join?
What scenario/best practice should I use get("chain") or always stay in Join?
Does the approach OrderSpecification.class with static method obey a good OOP design pattern?
You can't specify the join type for paths. It will use INNER join semantics by default and that is mandated by the JPA specification. If you want a different join type, you will have to create joins explicitly. The fact that using get renders as cross joins is a limitation of the old query model of Hibernate, but Hibernate 6.0 will fix this. The semantics are the same though and the query planner of your database should be able to treat both queries the same way. Maybe you just need to update your database version?
There is no "best practice" i.e. this really depends on your needs. Explicit joins are just that, explicit. So multiple calls to join will create multiple joins in SQL.
As for the OOP question, I think this is fine, yes.
I am trying to understand hibernate filters, i thought that the filter is applied even if the query is not started from the filtered entity and can be applied if i just join to it.
My entities:
#Table(name = "SPM_SECTION", schema = "TEST")
#GenericGenerator(name = "MODSEC_ID.GEN", strategy = "uuid2")
public class ModuleSection implements Serializable {
private AcademicClass academicClass;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "CLASS_ID")
public AcademicClass getAcademicClass() {
return academicClass;
}
public void setAcademicClass(AcademicClass academicClass) {
this.academicClass = academicClass;
}
}
#Entity
#GenericGenerator(name = "AC_CLASS_ID.GEN", strategy = "uuid2")
#Where(clause="1=1")
#FilterDef(name= Resources.SECURITY_FILTER_NAME, parameters={#ParamDef(name=Resources.SECURITY_FILTER_PARAMETER, type="string")})
#Filter(name=Resources.SECURITY_FILTER_NAME, condition = "DISCRIMINATOR_ID in (:"+Resources.SECURITY_FILTER_PARAMETER+")")
#Table(name = "ACADEMIC_CLASS", schema = "TEST", uniqueConstraints = #UniqueConstraint(columnNames = {"OUS_ID", "YEAR_ID",
"PERIOD_ID", "SHIFT_ID", "SEMESTER", "CODE" }))
public class AcademicClass implements java.io.Serializable {
//I tried by having the association here, i also tried without it.
private Set<ModuleSection> sections = new HashSet<>(0);
#OneToMany(fetch = FetchType.LAZY, mappedBy = "academicClass")
public Set<ModuleSection> getSections() {
return this.sections;
}
public void setSections(Set<ModuleSection> sections) {
this.sections = sections;
}
}
The filter is enabled through an interceptor and the parameter list is fetched from the database for security.
When i execute a query like this:
em.createQuery("select acc from AcademicClass acc ...........", AcademicClass.class)
.getResultList();
the filter is applied. But i also want the filter to be applied when my query starts from ModuleSection:
em.createQuery("select ms from ModuleSection ms join ms.academicClass acc", AcademicClass.class)
.getResultList();
In above query the filter is not applied.
The academicClass in ModuleSection entity is nullable but i also have other entities not null where the above case does not work.
I also tried to apply the #Filter or #FilterJoinTable in module section property with no luck:
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "CLASS_ID")
#Filter(name=Resources.SECURITY_FILTER_NAME, condition = "DISCRIMINATOR_ID in (:"+Resources.SECURITY_FILTER_PARAMETER+")")
#FilterJoinTable(name=Resources.SECURITY_FILTER_NAME, condition = "DISCRIMINATOR_ID in (:"+Resources.SECURITY_FILTER_PARAMETER+")")
public AcademicClass getAcademicClass() {
return academicClass;
}
My questions:
Are filters meant to filter only the entity in the from clause? does the filter apply in join entities?
If I want to implement the above should I also add a DISCRIMINATOR_ID in ModuleSection and add the filter to that entity starting the query from there?
There is a silence about it in the hibernate documentation, but it looks like this is true that #Filter is applied only to the from clause.
Assuming that we have the following mapping:
#Entity
#Table(name = "ACADEMIC_CLASS")
#FilterDef(
name="isAccessible",
parameters = #ParamDef(
name="sec",
type="string"
)
)
#Filter(
name="isAccessible",
condition="{acClass}.discriminator_id in (:sec)",
aliases = {
#SqlFragmentAlias(alias = "acClass", table= "TEST_SCHEMA.ACADEMIC_CLASS")
}
)
public class AcademicClass
{
#Id
#Column(name = "class_id")
private Long id;
#OneToMany(fetch = FetchType.LAZY, mappedBy = "academicClass")
private Set<ModuleSection> sections;
// getters/setters
}
#Entity
#Table(name = "SPM_SECTION")
public class ModuleSection
{
#Id
#Column(name = "sec_id")
private Long id;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "sec_class_id")
private AcademicClass academicClass;
// getters/setters
}
When we run the following query:
session
.enableFilter("isAccessible")
.setParameter("sec", "A1");
List<AcademicClass> classes = session.createQuery(
"select ac from ModuleSection ms join ms.academicClass ac",
AcademicClass.class
).getResultList();
The filter is not applied. It should happen in the JoinSequence.toJoinFragment. The filterCondition is empty in this case.
But for the rewritten in the following way query:
List<AcademicClass> classes = session.createQuery(
"select ac from ModuleSection ms, AcademicClass ac where ms.academicClass = ac",
AcademicClass.class
).getResultList();
We will have:
and as result the following query will be generated:
Hibernate:
/* select
ac
from
ModuleSection ms,
AcademicClass ac
where
ms.academicClass = ac
*/
select
academiccl1_.class_id as class_id1_0_
from TEST_SCHEMA.SPM_SECTION modulesect0_ cross
join TEST_SCHEMA.ACADEMIC_CLASS academiccl1_
where academiccl1_.discriminator_id in (?)
and modulesect0_.sec_class_id=academiccl1_.class_id
So, as a workaround you can rewrite your query in this way.
The #FilterJoinTable annotation can be used only if you have a link table between the parent entity and the child table.
Have a "full Entity" class:
#Entity(name = "vacancy_dec_to_words")
public class VacancyDescriptionToWords {
#Id
#Column(name = "id")
#GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;
#JoinColumn(name = "vacancy_description_id")
#ManyToOne(cascade = CascadeType.ALL)
private VacancyDescription vacancyDescription;
#JoinColumn(name = "words_id")
#ManyToOne
private Words words;
#Column(name = "qty")
private int qty;
#Column(name = "create_date")
private Date date;
//...getters and setters
In some methods I need use only 2 column from this database: word_id and qty
I try the follow way:
Projections
https://docs.spring.io/spring-data/jpa/docs/2.1.2.RELEASE/reference/html/#projections
public interface QtyWords {
Long getWords();
Integer getQty();
}
JpaReposytory:
*Query, that I use tested and it workable, I use him in JpaRepository:
#Repository
public interface SmallVDTWRepository extends JpaRepository<VacancyDescriptionToWords, Long> {
#Query(nativeQuery = true,
value = "SELECT sum(qty), words_id FROM vacancy_desc_to_words WHERE vacancy_description_id IN (" +
"SELECT id FROM vacancy_description WHERE vacancy_id IN (" +
"SELECT id FROM vacancy WHERE explorer_id = :exp))" +
"GROUP BY words_id")
List<QtyWords> getDistinctWordsByExplorer(#Param("exp") long exp);
}
But I get some interesting result when I get list of entities:
List<QtyWords> list = vdtwService.getByExplorerId(72);
I am not get any exceptions, but I have the list with are unknowns objects. This objects contains my data, which I need(qty and words_id), but I cannot get them from him.
Can I use this method (Projection) to implement this task and, in general, how to correctly implement the 'Light Entity' in this case?
Spring provides two mechanisms that can be used to limit data to be fetched.
Projections
Projections can help you to reduce data, retrieved from database, by setting what exactly attributes you want to fetch.
Example:
#Entity
class Person {
#Id UUID id;
String firstname, lastname;
#OneToOne
Address address;
}
#Entity
static class Address {
#Id UUID id;
String zipCode, city, street;
}
interface NamesOnly {
String getFirstname();
String getLastname();
}
#Repository
interface PersonRepository extends Repository<Person, UUID> {
Collection<NamesOnly> findByLastname(String lastname);
}
Entity graph
Annotation EntityGraph can help you to reduce amount of queries to database, by setting what exactly related entities you need to fetch.
Example:
#Entity
#NamedEntityGraph(name = "GroupInfo.detail", attributeNodes = #NamedAttributeNode("members"))
public class GroupInfo {
#Id UUID id;
#ManyToMany //default fetch mode is lazy.
List<GroupMember> members = new ArrayList<GroupMember>();
}
#Repository
public interface GroupRepository extends CrudRepository<GroupInfo, String> {
#EntityGraph(value = "GroupInfo.detail", type = EntityGraphType.LOAD)
GroupInfo getByGroupName(String name); //Despite of GroupInfo.members has FetchType = LAZY, it will be fetched because of using EntityGraph
}
There are two types of EntityGraph:
EntityGraphType.LOAD - is used to specify an entity graph, attributes that are specified by attribute nodes of the entity graph are treated as FetchType.EAGER and attributes that are not specified are treated according to their specified or default FetchType.
EntityGraphType.FETCH - is used to specify an entity graph, attributes that are specified by attribute nodes of the entity graph are treated as FetchType.EAGER and attributes that are not specified are treated as FetchType.LAZY.
PS: Also remember that you can set lazy fetch type: #ManyToOne(fetch = FetchType.LAZY) and JPA will not fetching child entities when parent is being fetched.
I am trying to create a dynamic query with Specification with two entities which have bidirectional relation. The entities are:
#Entity
#Table("SUPPLIERS")
public class Supplier implements Serializable {
#Id
Column("ID")
private Long id;
#Id
Column("COMPANY_ID")
private Long companyId;
}
#Entity
#Table("EMPLOYEES")
public class Employee implements Serializable {
#Id
private Long id;
#ManyToOne
#JoinColumns({
#JoinColumn(name = "FIRM_ID", referencedColumnName = "ID"),
#JoinColumn(name = "FIRM_COMPANY_ID", referencedColumnName = "COMPANY_ID")
})
private Supplier supplier;
}
When I want to select employees based on their supplier,
return new Specification<Employee>() {
#Override
public Predicate toPredicate(Root<Employee> root, CriteriaQuery<?> query, CriteriaBuilder cb) {
Long[] supplierCodes = {1L, 2L};
Subquery<Supplier> supplierBasicSubquery = query.subquery(Supplier.class);
Root<Supplier> supplierBasicRoot = supplierBasicSubquery.from(Supplier.class);
Join<Employee, Supplier> sqTfV = root.join("supplier", JoinType.INNER);
supplierBasicSubquery.select(sqTfV).where(sqTfV.<Long>get("id").in(supplierCodes));
return root.<Supplier>get("supplier").in(supplierBasicSubquery);
}
};
When its executed, it generates SQL like:
SELECT ....
FROM EMPLOYEES E
INNER JOIN ....
WHERE (E.FIRM_ID, E.FIRM_COMPANY_ID) in
(SELECT (s.ID, s.COMPANY_ID) FROM SUPPLIERS WHERE SUPPLIER.ID in (1, 2))
As you can see, the inner select columns are surrounded by parenthesis which causes Oracle to throw exception:
java.sql.SQLSyntaxErrorException: ORA-00920: invalid relational operator
How can I fix this issue, any suggestions?
Thanks a lot.