I am using Spring Data JPA + Hibernate for a webapp. For a particular domain model A, we have a 1-to-many association in another domain B. Such that A will have a Set getB() and B will have A getA().
While querying for a A graph, I see hibernate is using 1+n queries. A single outer join query for fetching the A graph, but then 'n' queries for setting A in each B.
Am I missing any pattern here? Since all the childs have the same parent, is not somehow possible to avoid these 'n' queries?
#MappedSuperclass
#Data
public abstract class Batch implements Serializable {
private static final long serialVersionUID = 1L;
#OneToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "batch_id", referencedColumnName = "batch_id")
protected BatchID batchId;
}
/*
//The parent class in a simplified form
*/
#Entity
#Table(name = "DRYRUN")
#Inheritance(strategy=InheritanceType.TABLE_PER_CLASS)
public class DryrunBatch extends Batch {
/**
*
*/
private static final long serialVersionUID = -1596595930859735318L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Getter#Setter
protected Long id;
public DryrunTNStatus newTNStatus()
{
final DryrunTNStatus tn = new DryrunTNStatus();
tn.setBatch(this);
getTnStatus().add(tn);
return tn;
}
#OneToMany(fetch = FetchType.LAZY, mappedBy = "batch")
#Getter#Setter
private Set tnStatus = new HashSet();
}
//The child class in a simplified form
#Entity
#Table(name = "DRYRUN_TN_STATUS")
#Data
public class DryrunTNStatus implements Serializable{
/**
*
*/
private static final long serialVersionUID = -4388406636444350023L;
public DryrunTNStatus(String accountNo, String telNo) {
super();
this.accountNo = accountNo;
this.telNo = telNo;
}
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "BATCH_ID", referencedColumnName = "BATCH_ID")
private DryrunBatch batch;
public DryrunTNStatus()
{
}
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
protected Long id;
}
The code to fetch the object graph using JpaRepository. Using Spring JPA support to enforce an outer join. I preferred this over Hibernate's #Fetch annotation.
DryrunBatch drBatch = drBatchRepo.findOne(new Specification() {
#Override
public Predicate toPredicate(Root root, CriteriaQuery query,
CriteriaBuilder cb) {
query.distinct(true);
root.fetch("tnStatus", JoinType.LEFT);
return cb.equal(root.get("batchId").get("id"),
batch.getId());
}
});
And finally the hibernate queries from log. I am running a junit that fetches a parent with 10 childs from DB.
//this query can fetch data for the complete graph??
Hibernate: select distinct dryrunbatc0_.id as id1_6_0_, tnstatus1_.id as id1_9_1_[etc..] from dryrun dryrunbatc0_ left outer join dryrun_tn_status tnstatus1_ on dryrunbatc0_.batch_id=tnstatus1_.batch_id where dryrunbatc0_.batch_id=15
//and then 10 queries like
Hibernate: select dryrunbatc0_.id as id1_6_3_, [etc..] from dryrun dryrunbatc0_ left outer join batch_id batchid1_ on dryrunbatc0_.batch_id=batchid1_.batch_id inner join users user2_ on dryrunbatc0_.created_by=user2_.login_id left outer join dryrun_tn_status tnstatus3_ on dryrunbatc0_.batch_id=tnstatus3_.batch_id where dryrunbatc0_.batch_id=?
You've encountered the famous N+1 problem with lazy loading. There is no JPA standard way to tackle this, however, every JPA provider provides means to turn on "Batch fetching", which will load all lazy references at once instead loading each in a single SQL query.
Here is information on how to turn it on in hibernate.
Here is an article with explanation of how batch fetching works and examples using eclipselink.
Related
I have multiple entities that i want to fetch in single query, because N+1 queries last too long.
For example SQL join query lasts 5 seconds on DB, but elcipselink persistence fetching lasts 50-80 seconds due to N+1 fetching.
I found out that LEFT JOIN FETCH is not working as soon as #ManyToOne relation is implemented.
Does anyone know solution to LEFT JOIN FETCH for this case?
Please find below simplified entities.
#Entity
#Table(name="SITUATION_DATA")
#NamedQuery(name="SituationData.findAll", query="SELECT s FROM SituationData s")
public class DatexSituationData implements Serializable {
private static final long serialVersionUID = 1L;
//bi-directional many-to-one association to SituationRecord
#OneToMany(mappedBy="datexSituationData", cascade = CascadeType.PERSIST, fetch = FetchType.EAGER)
#JoinFetch(value=JoinFetchType.OUTER)
private List<SituationRecord> situationRecords;
}
#Entity
#Table(name="SituationRecord")
#NamedQuery(name="SituationRecord.findAll", query="SELECT s FROM SituationRecord s")
public class SituationRecord implements Serializable {
private static final long serialVersionUID = 1L;
#OneToMany(mappedBy="situationRecord", cascade = CascadeType.PERSIST, fetch = FetchType.EAGER)
#JoinFetch(value=JoinFetchType.OUTER)
private List<SituationRecordComment> situationRecordComment;
#OneToMany(mappedBy="situationRecord", cascade = CascadeType.PERSIST, fetch = FetchType.EAGER)
#JoinFetch(value=JoinFetchType.OUTER)
private List<SituationRecordTypeElement> situationRecordTypeElements;
//bi-directional many-to-one association to SituationLocation
#ManyToOne(cascade = CascadeType.PERSIST, fetch = FetchType.EAGER)
#JoinFetch(value=JoinFetchType.OUTER)
#JoinColumn(name="ID_LOKACIJE")
private SituationLocation situationLocation;
//bi-directional many-to-one association to DatexSituationData
#ManyToOne()
#JoinColumns({
#JoinColumn(name="SITUATION_ID", referencedColumnName="ID", nullable=false),
#JoinColumn(name="SITUATION_VERSION", referencedColumnName="VERSION", nullable=false)
})
private DatexSituationData datexSituationData;
}
#Entity
#Table(name="SITUATION_LOCATIONS")
#NamedQuery(name="SituationLocation.findAll", query="SELECT s FROM SituationLocation s")
public class SituationLocation implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy=GenerationType.SEQUENCE, generator="situation_location_seq")
#SequenceGenerator(name="situation_location_seq", sequenceName="SEQ_SITUATION_LOCATION", allocationSize=1)
#Column(name="ID_LOKACIJE", unique=true, nullable=false)
private long idLokacije;
//bi-directional many-to-one association to SituationRecord
#OneToMany(mappedBy="situationLocation", cascade = CascadeType.PERSIST)
private List<SituationRecord> situationRecords;
}
This is how i fetch it.
I have tried all of the below combinations, but every combinations makes query for each row (object) in SituationData, or in some cases for each joined SituationData join SituationRecord.
String sQuery =
//"select * from SITUATION_DATA t";
//"SELECT * FROM (select t.*, rank() over(partition by t.id order by version desc) rnk from SITUATION_DATA t) where rnk = 1";
"SELECT ds FROM SituationData ds LEFT JOIN FETCH ds.situationRecords sr LEFT JOIN FETCH sr.situationLocation sl LEFT JOIN FETCH sr.situationRecordTypeElements sre LEFT JOIN FETCH sr.situationRecordComment src";
EntityManager em = Emf.getInstance().getFactory().createEntityManager();
//Query q = em.createNativeQuery(sQuery, DatexSituationData.class);
Query q = em.createQuery(sQuery, DatexSituationData.class);
// q.setHint("eclipselink.LEFT_FETCH", "t.situationRecords.situationRecordComment");
q.setHint("eclipselink.LEFT_FETCH", "ds.sr.sl");
q.setHint("eclipselink.LEFT_FETCH", "ds.sr.sre");
q.setHint("eclipselink.LEFT_FETCH", "ds.sr.src");
// q.setHint("eclipselink.JDBC_FETCH_SIZE", "100");
lResult = q.getResultList();
Since you are not explicitly state EclipseLink version being used, I'll just assume version 2.6.
As per the official EclipseLink documentation, eclipselink.LEFT_FETCH is not a supported query hint. You are probably trying to use eclipselink.join-fetch or eclipselink.left-join-fetch as documented here.
The proper way to use this hint is along the lines of:
String sQuery = "SELECT ds FROM DatexSituationData ds";
EntityManager em = emf.getInstance().getFactory().createEntityManager();
TypedQuery q = em.createQuery(sQuery, DatexSituationData.class);
q.setHint("eclipselink.join-fetch", "ds.situationRecords");
q.setHint("eclipselink.join-fetch", "ds.situationLocation");
// ...
lResult = q.getResultList();
Another approach would be batch fetching.
This would work as:
//...
em
.createQuery("SELECT ds FROM DatexSituationData ds")
.setHint("eclipselink.batch", "ds.situationRecords")
//...
.setHint("eclipselink.batch.type", "IN")
.setHint("eclipselink.batch_size", "1000");
//...
You would probably test both approaches and gather some metrics; then decide which one works better (in terms of performance) for your DB schema and usage patterns.
I have two (Hibernate-based) Spring Data JPA domain classes, the "One" side Customer.class:
#Entity
#Table(name = "sys_customer")
#Data
public class Customer implements Serializable {
#Id
#Column(name = "cust_id")
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "cust_name")
private String customerName;
#OneToMany(cascade = CascadeType.ALL, mappedBy = "customer")
private Set<Order> orders;
}
and the "Many" side Order.class:
#Entity
#Table(name = "sys_order")
#Getter
#Setter
#AllArgsConstructor
#NoArgsConstructor
public class Order implements Serializable {
#Id
#Column(name = "order_id")
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "order_name")
private String orderName;
#ManyToOne
#JoinColumn(name = "order_cust_id", referencedColumnName = "cust_id")
private Customer customer;
public Order( String orderName) {
this.orderName = orderName;
}
public Order(String orderName, Customer customer) {
this.orderName = orderName;
this.customer = customer;
}
}
I have OrderRepository interface which extends JpaRepository interface and JpaSpecificationExecutor interface:
public interface OrderRepository extends JpaRepository<Order, Long>, JpaSpecificationExecutor<Order> {
}
I have a OrderSpecification.class with the static method searchByCustomerName:
public class OrderSpecification {
public static Specification<Order> searchByCustomerName(String customerName) {
return new Specification<Order>() {
#Override
public Predicate toPredicate(Root<Order> root, CriteriaQuery<?> query, CriteriaBuilder criteriaBuilder) {
Join<Order, Customer> join = root.join("customer");
return criteriaBuilder.like(join.get("customerName"), "%" + customerName + "%");
//return criteriaBuilder.like(root.get("customer").get("customerName"), "%" + customerName + "%");
}
};
}
}
To find the differences between get("property") chain and Join, I wrote a simple test method and comment out the above OrderSpecificatin.class code
#Test
#Transactional
public void testFindOrderByCustomerName(){
String name = "adam";
List<Order> orders = orderRepository.findAll(OrderSpecification.searchByCustomerName(name));
for(Order order: orders){
Customer customer = order.getCustomer();
log.info(new StringBuilder().append(customer.getId()).append(" ").append(customer.getCustomerName()).toString());
}
}
I found that:
get("property") chain use a cross-join(which is very bad performancing) while Join use inner-join(since ManyToOne() by default is Fetch= FetchType.EAGER)
/* get("property") chain: Hibernate: select order0_.order_id as
order_id1_1_, order0_.order_cust_id as order_cu3_1_,
order0_.order_name as order_na2_1_ from sys_order order0_ cross join
sys_customer customer1_ where order0_.order_cust_id=customer1_.cust_id
and (customer1_.cust_name like ?) Hibernate: select customer0_.cust_id
as cust_id1_0_0_, customer0_.cust_name as cust_nam2_0_0_ from
sys_customer customer0_ where customer0_.cust_id=? */
/** * "Join": * Hibernate: select order0_.order_id as order_id1_1_,
order0_.order_cust_id as order_cu3_1_, order0_.order_name as
order_na2_1_ from sys_order order0_ inner join sys_customer customer1_
on order0_.order_cust_id=customer1_.cust_id where customer1_.cust_name
like ? * Hibernate: select customer0_.cust_id as cust_id1_0_0_,
customer0_.cust_name as cust_nam2_0_0_ from sys_customer customer0_
where customer0_.cust_id=? */
My questions are:
Can I specify the Join type(inner, all three outers) or Fetch Type(LAZY, EAGER) when using get("property") chain approach to avoid cross-join?
What scenario/best practice should I use get("chain") or always stay in Join?
Does the approach OrderSpecification.class with static method obey a good OOP design pattern?
You can't specify the join type for paths. It will use INNER join semantics by default and that is mandated by the JPA specification. If you want a different join type, you will have to create joins explicitly. The fact that using get renders as cross joins is a limitation of the old query model of Hibernate, but Hibernate 6.0 will fix this. The semantics are the same though and the query planner of your database should be able to treat both queries the same way. Maybe you just need to update your database version?
There is no "best practice" i.e. this really depends on your needs. Explicit joins are just that, explicit. So multiple calls to join will create multiple joins in SQL.
As for the OOP question, I think this is fine, yes.
Hello This is my 2 tables:
record and submission.
In submission, it has 1 composite primary key:(submission_id, question_id). One submission number can have several questions number. For example:
And as for record, it has a composite primary key:(student_id, exam_id). It looks like this:
I want to join these 2 tables like MySQL:
select * from record
left join submission
on record.submission_id = submission.submission_id.
But in hibernate, I have successfully join these 2 tables, but it gives me the following hql:
Hibernate:
select
...all columns...
from
record record0_
inner join
submission submission1_
on record0_.submission_id=submission1_.submission_id
and record0_.question_id=submission1_.question_id
where
1=1
In this case, I will get 0 rows in the result.
I don't want it use "and record0_.question_id=submission1_.question_id" after on clause, because there is no question_id in my record table.
But I have to add all primary keys into the #joinColumns() when I add Submission attribute in Record class, like this:
// Record class
#Getter
#Setter
#ToString
#AllArgsConstructor
#NoArgsConstructor
#Entity
#Table(name = "record")
public class Record implements java.io.Serializable{
private static final long serialVersionUID = 1L;
// Other columns I don't need to show
#Column(name = "submission_id")
private Integer submissionId;
#OneToOne(fetch = FetchType.LAZY)
#JoinColumns({
#JoinColumn(name = "submission_id", referencedColumnName = "submission_id",insertable=false, updatable=false),
#JoinColumn(name = "question_id", referencedColumnName = "question_id",insertable=false, updatable=false)
})
private Submission submission;
}
My Submission class like this:
#Getter
#Setter
#ToString
#AllArgsConstructor
#NoArgsConstructor
#Entity
#Table(name = "submission")
public class Submission implements java.io.Serializable{
private static final long serialVersionUID = 1L;
#Id
#Column(name = "submission_id")
private Integer submissionId;
#Id
#Column(name = "question_id")
private Integer questionId;
#OneToOne(fetch = FetchType.LAZY, mappedBy = "submission")
private Record record;
}
Anyone can give me some advice?
-------- How I combine these tables-------
Actually, I join 4 tables and all these joins have the same problem declared above.
Code below is how i combine these 4 tables (record, submission, question, optional)
#Override
public List<RcdSubQuesOpt> getRcdSubQuesOpt(int studentID, int examId) {
Session session = this.getSession();
// RcdSubQuesOpt --> this is a class to store attributes from different tables(classes)
List<RcdSubQuesOpt> results;
Transaction transaction = null;
transaction = session.beginTransaction();
CriteriaBuilder criteriaBuilder = session.getCriteriaBuilder();
CriteriaQuery<RcdSubQuesOpt> criteriaQuery = criteriaBuilder.createQuery(RcdSubQuesOpt.class);
// To combine these tables use join
Root<Record> pRoot = criteriaQuery.from(Record.class);
Join<Record, Submission> rcd2sub = pRoot.join(Record_.submission);
Join<Submission, Question> sub2que = rcd2sub.join(Submission_.question);
Join<Question, Optional> que2opt = sub2que.join(Question_.optional);
// Attributes in RcdSubQuesOpt class
// get these columns from result and assign them to RcdSubQuesOpt class
criteriaQuery.multiselect(
pRoot.get("studentId"),
pRoot.get("examId"),
rcd2sub.get("questionId"),
rcd2sub.get("stuAnswer"),
sub2que.get("content"),
que2opt.get("content"),
que2opt.get("answer"));
// Predicate predicate = pRoot.get("examId").equals(1);
criteriaQuery.where();
results = session.createQuery(criteriaQuery).getResultList();
transaction.commit();
return results;
}
You haven't mentioned how you retrieve that data using hibernate. Have you tried trying to use #Query (select r from Record left join Submission sub on r.submissionId = sub.id where ...") ?
you have defined a #OneToOne relation in your record class. Apparantly thats wrong, since there exists more then one entry in your submission table for one record. So change this to #OneToMany and the respective relation in the submission class to #ManyToOne.
Besides your entities are not well named and mapped. Submission is in fact more of a question or an answer to it, because a line in that table does not represent one submission, which would be the expected meaning.
I have a simple spring boot rest app connected with mySQL db and I'm trying to optimize number of queries within simple function:
List<Message> messages = messagesRepository.findBySenderIdOrReceiverIdOrderByTimeDesc(senderId, receiverId);
MessagesRepository:
public interface MessagesRepository extends CrudRepository<Message, Long> {
List<Message> findBySenderIdOrReceiverIdOrderByTimeDesc(Long senderId, Long receiverId);
}
Message:
#Entity
#Table(name="s_messages")
public class Message implements Serializable
{
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private Long id;
#Transient
private int internalId;
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name="senderId", referencedColumnName = "id", updatable=false, insertable=false)
private ProfileLite sender;
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name="receiverId", referencedColumnName = "id", updatable=false, insertable=false)
private ProfileLite receiver;
#Column(columnDefinition="TEXT")
private String message;
private long time;
private MessageStatus status;
}
ProfileLite:
#Entity
#Table(name="s_profiles")
public class ProfileLite implements Comparable<ProfileLite>
{
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private Long id;
private String nickname;
private String country;
private String thumb;
private Gender gender;
}
After executing method mentioned above, hibernate generates about 40 SQL's (based on 40 profiles) like this:
SQL Log - PasteBin
so first collecting messages and then for each message creates another sql to gather profile.
Is it any possibility to push hibernate to create just one simple sql instead of 40 like: select * from s_messages m join s_profiles s1 on m.sender_id = s1.id join s_profiles s2 m_receiver_id = s2.id ? (pseudo code)
Thanks!
This could be a n + 1 problem.
You can use a JOIN FETCH in your JPA query to fix this.
A "fetch" join allows associations or collections of values to be initialized along with their parent objects using a single select. This is particularly useful in the case of a collection. It effectively overrides the outer join and lazy declarations of the mapping file for associations and collections.
Update your JPA repository like so
public interface MessagesRepository extends CrudRepository<Message, Long> {
#Query("Select m from Message m join fetch m.sender ms join fetch m.receiver mr where ms.id = :senderId or mr.id = :receiverId order by m.time desc")
List<Message> findBySenderIdOrReceiverIdOrderByTimeDesc(Long senderId, Long receiverId);
}
For a more detailed explanation check out this answer.
PS: I havent tested the query.
I have a Company entity that I fetch with a JPQL query with Hibernate. The entity has a many-to-many association with a Keyword entity. Since the join table has an additional column is_active, this table has been mapped to a CompanyKeyword entity. So the association is like this:
Company <-- CompanyKeyword --> Keyword
Now, the association from the Company entity is lazy, and it is not initialized by my JPQL query, as I want to avoid creating a cartesian product performance problem. That is why I want to initialize the association after running the JPQL query, e.g. like this:
#Service
class CompanyServiceImpl implements CompanyService {
#Autowired
private CompanyRepository companyRepository;
#Transactional
public Company findOne(int companyId) {
Company company = this.companyRepository.findOneWithSomeCustomQuery(companyId);
Hibernate.initialize(company.companyKeywords());
return company;
}
}
For a "normal" many-to-many association, this would work great, as all of the associated entities would be fetched in a single query. However, since I have an entity between Company and Keyword, Hibernate will only initialize the first part of the association, i.e. from Company to CompanyKeyword, and not from CompanyKeyword to Keyword. I hope that makes sense. I am looking for a way to initialize this association all the way without having to do something like this:
Company company = this.companyRepository.findOneWithSomeCustomQuery(companyId);
Hibernate.initialize(company.getCompanyKeywords());
for (CompanyKeyword ck : company.getCompanyKeywords()) {
Hibernate.initialize(ck.getKeyword());
}
The above code is neither clean, nor good in terms of performance. If possible, I would like to stick to my current approach of using a JPQL query to fetch my Company entity and then initializing certain associations afterwards; it would take quite a bit of refactoring to change this in my project. Should I just "manually" fetch the association with a second JPQL query, or is there a better way of doing it that I haven't thought of?
Below are my mappings. Thanks in advance!
Company
#Entity
#Table(name = "company")
public class Company implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column
private int id;
#Size(max = 20)
#OneToMany(fetch = FetchType.LAZY, mappedBy = "company")
private Set<CompanyKeyword> companyKeywords = new HashSet<>();
// Getters and setters
}
CompanyKeyword
#Entity
#Table(name = "company_service")
#IdClass(CompanyServicePK.class)
public class CompanyKeyword implements Serializable {
#Id
#ManyToOne(fetch = FetchType.LAZY, targetEntity = Company.class)
#JoinColumn(name = "company_id")
private Company company;
#Id
#ManyToOne(fetch = FetchType.LAZY, targetEntity = Keyword.class)
#JoinColumn(name = "keyword_id")
private Keyword keyword;
#Column(nullable = true)
private boolean isActive;
// Getters and setters
}
CompanyKeywordPK
public class CompanyServicePK implements Serializable {
private Company company;
private Service service;
public CompanyServicePK() { }
public CompanyServicePK(Company company, Service service) {
this.company = company;
this.service = service;
}
// Getters and setters
// hashCode()
// equals()
}
Keyword
#Entity
#Table(name = "keyword")
public class Keyword {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column
private int id;
// Fields and getters/setters
}
You'll indeed need to execute an additional JPQL query, fetching the company with its companyKeyWords and with the keyword of each CompanyKeyWord.
You could also doing it by simply looping and initializing every entity, and still avoid executing too many queries, by enabling batch fetching.