I have a performance issue on search queries with multiple joins on the table with 250000+ records. The best time that I achieve is 1.5 seconds with default pagination and sorting provided by JPA. Also, I tried to add indexes on columns, but the time remains the same because of joins. Is there any way to boost the performance of the query?
"select new com.app.e_library.service.dto.BookDto(book.id,book.isbn," +
" book.title, book.publicationYear, book.pageCount, genre.name," +
" book.bookStatus, publisher.publisherName, author.name) " +
"from BookEntity book " +
"inner join book.bookGenre genre " +
"inner join book.publisher publisher " +
"inner join book.author author " +
"where book.isbn like :key% or " +
"book.title like :key% or " +
"trim(book.publicationYear) like :key% or " +
"genre.name like :key% or " +
"publisher.publisherName like :key% or " +
"author.name like :key%"
And query generated by hibernate.
Hibernate:
select
bookentity0_.id as col_0_0_,
bookentity0_.isbn as col_1_0_,
bookentity0_.title as col_2_0_,
bookentity0_.publication_year as col_3_0_,
bookentity0_.page_count as col_4_0_,
bookgenree1_.name as col_5_0_,
bookentity0_.book_status as col_6_0_,
publishere2_.name as col_7_0_,
authorenti3_.name as col_8_0_
from
book bookentity0_
inner join
book_genre bookgenree1_
on bookentity0_.genre_id=bookgenree1_.id
inner join
publisher publishere2_
on bookentity0_.publisher_id=publishere2_.id
inner join
author authorenti3_
on bookentity0_.author_id=authorenti3_.id
where
bookentity0_.isbn like ?
or bookentity0_.title like ?
or trim(bookentity0_.publication_year) like ?
or bookgenree1_.name like ?
or publishere2_.name like ?
or authorenti3_.name like ?
order by
bookentity0_.id asc limit ?
Hibernate:
select
count(bookentity0_.id) as col_0_0_
from
book bookentity0_
inner join
book_genre bookgenree1_
on bookentity0_.genre_id=bookgenree1_.id
inner join
publisher publishere2_
on bookentity0_.publisher_id=publishere2_.id
inner join
author authorenti3_
on bookentity0_.author_id=authorenti3_.id
where
bookentity0_.isbn like ?
or bookentity0_.title like ?
or trim(bookentity0_.publication_year) like ?
or bookgenree1_.name like ?
or publishere2_.name like ?
or authorenti3_.name like ?
Book Entity
#Entity
#Table(
name = "book",
uniqueConstraints = {
#UniqueConstraint(name = "book_isbn_unique",column Names = "isbn")
},
indexes = {
#Index(name = "isbn_index", columnList = "isbn"),
#Index(name = "title_index", columnList = "title"),
}
)
public class BookEntity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "isbn", nullable = false)
#NotBlank
private String isbn;
#Column(name = "title", nullable = false)
#NotBlank
private String title;
#Column(name = "publication_year")
#Valid
private short publicationYear;
#Column(name = "page_count")
#Valid
#Range(min = 50, max = 5000)
private int pageCount;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name="genre_id",referencedColumnName = "id", nullable=false)
#ToString.Exclude
private BookGenreEntity bookGenre;
#Column(name = "book_status", nullable = false)
#Enumerated(EnumType.STRING)
private BookStatusType bookStatus;
#ManyToOne(cascade = CascadeType.ALL, fetch = FetchType.LAZY)
#JoinColumn(name = "publisher_id", referencedColumnName = "id", nullable = false)
#NonNull
#ToString.Exclude
private PublisherEntity publisher;
#OneToOne(cascade = CascadeType.ALL, fetch = FetchType.LAZY)
#JoinColumn(name = "pick_detail_id", referencedColumnName = "id")
#ToString.Exclude
private PickDetailEntity pickDetail;
#ManyToOne(cascade = CascadeType.ALL, fetch = FetchType.LAZY)
#JoinColumn(name = "author_id", referencedColumnName = "id", nullable = false)
#NonNull
#ToString.Exclude
private AuthorEntity author;
#OneToOne(cascade = CascadeType.ALL, fetch = FetchType.LAZY)
#JoinColumn(name = "book_image_id", referencedColumnName = "id")
#ToString.Exclude
private BookImageEntity bookImage;
Book Genre Entity
#Entity
#Table(
name = "book_genre",
uniqueConstraints = {
#UniqueConstraint(name = "book_genre_name_unique", columnNames = "name")
},
indexes = {
#Index(name = "name_index", columnList = "name"),
}
)
public class BookGenreEntity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id")
private Long id;
#Column(name = "name", nullable = false)
#NotBlank
private String name;
#OneToMany(
targetEntity = BookEntity.class,
mappedBy = "bookGenre",
cascade=CascadeType.ALL,
fetch = FetchType.LAZY,
orphanRemoval = true)
#ToString.Exclude
private List<BookEntity> books;
Publisher Entity
#Entity
#Table(
name = "publisher",
uniqueConstraints = {
#UniqueConstraint(name = "publisher_name_unique", columnNames = "name")
},
indexes = {
#Index(name = "name_index", columnList = "name")
}
)
public class PublisherEntity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "name", nullable = false)
#NotBlank
private String publisherName;
#OneToMany(
targetEntity = BookEntity.class,
mappedBy = "publisher",
cascade=CascadeType.ALL,
fetch = FetchType.LAZY)
#ToString.Exclude
private List<BookEntity> books;
Author Entity
#Entity
#Table(
name = "author",
uniqueConstraints = {
#UniqueConstraint(name = "author_name_unique", columnNames = "name")
},
indexes = {
#Index(name = "name_index", columnList = "name")
}
)
public class AuthorEntity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "name", nullable = false)
#NotBlank
private String name;
#OneToMany(
targetEntity = BookEntity.class,
mappedBy = "author",
cascade=CascadeType.ALL,
fetch = FetchType.LAZY)
#ToString.Exclude
private List<BookEntity> books;
You don't have to specify the referencedColumnName in your #JoinColumn
You have a #UniqueConstraint on BookEntity#isbn so there's no need to add an extra index.
You need an index on BookEntity#publication_year :
indexes = {
#Index(name = "title_index", columnList = "title"),
#Index(name = "publication_year_index", columnList = "publication_year"),
}
You can try this query :
#NamedQuery(name="BookEntity.search", query="select book from BookEntity book"
+ " inner join fetch book.genre genre"
+ " inner join fetch book.publisher publisher"
+ " inner join fetch book.author or author"
+ " left join fetch book.pickDetail"
+ " where book.isbn like :key"
+ " or book.title like :key"
+ " or book.publicationYear = :year"
+ " or genre.name like :key"
+ " or publisher.publisherName like :key"
+ " or author.name like :key")
And this function :
private EntityManager em;
public List<BookEntity> search(String key, short year) {
return em.createNamedQuery("BookEntity.search", BookEntity.class)
.setParameter("key", key + "%")
.setParameter("year", year)
.getResultList();
}
Are you aware that you are doing a sensitive case search ? That's not very friendly. Also, the user can't search with a word in the middle of the title.
To avoid this you have to use something like :
upper(book.title) like :key
And
setParameter("key", "%" + key.toUpperCase() + "%")
But this will trigger a full scan.
Related
I am trying to sort my API result by a nested entity field.
My Entities look like that:
#Entity
#Table(name = "book_item")
public class BookItem {
#Id
#Column(name = "id", nullable = false)
#GeneratedValue(strategy = GenerationType.AUTO)
private UUID id;
#ManyToOne
#NotNull
private Book book;
...
}
#Entity
#Table(name = "book")
#Cache(usage = CacheConcurrencyStrategy.READ_WRITE)
public class Book {
#Id
#Column(name = "id")
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "sequenceGenerator")
#SequenceGenerator(name = "sequenceGenerator")
private Long id;
#Column(name = "title", nullable = false)
private String title;
private String subtitle;
#ManyToMany(fetch = FetchType.EAGER, cascade = { CascadeType.PERSIST, CascadeType.MERGE })
#JoinTable(name = "rel_book_author", joinColumns = #JoinColumn(name = "book_id"), inverseJoinColumns = #JoinColumn(name = "author_id"))
#Cache(usage = CacheConcurrencyStrategy.READ_WRITE)
#JsonIgnoreProperties(value = { "books" }, allowSetters = true)
private Set<Author> authors = new HashSet<>();
#Column(name = "isbn", nullable = false)
private String isbn;
#Column(name = "pages", nullable = false)
private Integer pages;
...
}
Now I want to query all BookItems sorted by the Book titles.
My repository lloks like that:
public interface BookItemRepository extends JpaRepository<BookItem, UUID> {
#Query(
"select distinct b from BookItem b left join b.book.authors authors where (upper(authors.name) like upper(concat('%', ?2, '%')) or authors is NULL ) and upper(b.book.title) like upper(concat('%', ?1, '%'))"
)
Page<BookItem> findAllByTitleAndAuthor(String title, String name, Pageable pageable);
}
I've implemented an Angular component which does the following request:
GET http://localhost:9000/api/book/items?size=5&sort=title,asc
I get a 500 error code:
org.hibernate.QueryException: could not resolve property: title of: org.pickwicksoft.libraary.domain.BookItem [select distinct b from org.pickwicksoft.libraary.domain.BookItem b left join b.book.authors authors where (upper(authors.name) like upper(concat('%', ?2, '%')) or authors is NULL ) and upper(b.book.title) like upper(concat('%', ?1, '%')) order by b.title asc, b.id asc]; nested exception is java.lang.IllegalArgumentException
I also tried the following and it also does not work:
GET http://localhost:9000/api/book/items?size=5&sort=book_title,asc
How can I solve this problem?
Simple project to solve business task:
#Entity
#Table(name = "parent")
public class Parent {
#Id
#SequenceGenerator(name = "SEQ_GEN", sequenceName = "SEQ_JUST_FOR_TEST", allocationSize = 1)
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "SEQ_GEN")
#Column(name = "id", nullable = false)
private Long id;
#Column(name = "name", nullable = false)
private String name;
#OneToMany(fetch = FetchType.EAGER, cascade = {CascadeType.MERGE, CascadeType.PERSIST})
private List<Child> childs;
repository method:
#Query("select parent from Parent parent"
+ " join parent.childs childs "
+ " where parent.id = :id "
+ " and :search is null or childs.name = :search ")
Page<Parent>getPageById(Pageable pageable, #Param("id")Long id, #Param("search") String search);
and controller:
#GetMapping("/parents/{id}/{search}")
public ResponseEntity<Page<Parent> >parents(Pageable pageable, #PathVariable(value = "id") final String id,
#PathVariable(value = "search") final String search) {
Page<Parent>parentPage = parentRepository.getPageById(pageable, Long.getLong(id), search);
return new ResponseEntity<>(parentPage, HttpStatus.OK);
Is it possible to filter child collection and retrieve only childs with specific attribute, here it's a name for brievety?
I have two table named recipe and tags, joined with many to many relationship
This is recipe table:
#Data
#NoArgsConstructor
#Entity()
#Table(name = "recipes",
uniqueConstraints = {
#UniqueConstraint(name = "recipe_title_unique", columnNames = "title")
})
public class RecipeEntity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id", updatable = false)
private Long id;
#ManyToMany(fetch = FetchType.EAGER)
#JoinTable(
joinColumns = #JoinColumn(name = "recipe_id", referencedColumnName = "id"),
inverseJoinColumns = #JoinColumn(name = "tag_id", referencedColumnName = "id")
)
private Set<TagEntity> tags;
And this is tags table:
#Data
#NoArgsConstructor
#Entity()
#Table(name = "tags", uniqueConstraints = {
#UniqueConstraint(name = "tag_unique", columnNames = "name")
})
public class TagEntity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id", updatable = false)
private int id;
#Column(name = "name", nullable = false)
private String name;
ERD Diagram for many to many relationship
My question is, how I access recipes_tags table with JPQL?
For example I want to know how many recipe that use tag with ID 1.
#Query(value = "SELECT " +
"COUNT t.tag_id " +
"FROM recipes_tags t +
"WHERE t.tag_id = :tagId")
Int getCount(#Param("tagId") int tagId);
I'm trying to apply the where condition on the related entity, but the result set contains all the related entity data. It appears like the filter is ignored.
I have the following entities:
Entity Audit:
#Entity
#Table(name = "entity_audit")
#Cache(usage = CacheConcurrencyStrategy.NONSTRICT_READ_WRITE)
#org.springframework.data.elasticsearch.annotations.Document(indexName = "entityaudit")
public class EntityAudit implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#org.springframework.data.elasticsearch.annotations.Field(type = FieldType.Keyword)
private Long id;
#NotNull
#Column(name = "entity_id", nullable = false)
private Long entityId;
#NotNull
#Column(name = "entity_class_name", nullable = false)
private String entityClassName;
#NotNull
#Column(name = "entity_name", nullable = false)
private String entityName;
#NotNull
#Enumerated(EnumType.STRING)
#Column(name = "action_type", nullable = false)
private EntityAuditType actionType;
#NotNull
#Column(name = "timestamp", nullable = false)
private Instant timestamp;
#NotNull
#Column(name = "user", nullable = false)
private String user;
#NotNull
#Column(name = "transaction_uuid", nullable = false)
private String transactionUuid;
#OneToMany(cascade = CascadeType.ALL, orphanRemoval = true)
#JoinColumn(name = "entity_audit_id")
#Cache(usage = CacheConcurrencyStrategy.NONSTRICT_READ_WRITE)
private Set<EntityAuditUpdateData> entityAuditUpdateData = new HashSet<>();
#OneToMany(cascade = CascadeType.ALL, orphanRemoval = true)
#JoinColumn(name = "entity_audit_id")
#Cache(usage = CacheConcurrencyStrategy.NONSTRICT_READ_WRITE)
private Set<EntityAuditStatus> entityAuditStatuses = new HashSet<>();
Getters and setters...
Entity Audit Status
#Entity
#Table(name = "entity_audit_status")
#Cache(usage = CacheConcurrencyStrategy.NONSTRICT_READ_WRITE)
#org.springframework.data.elasticsearch.annotations.Document(indexName = "entityauditstatus")
public class EntityAuditStatus implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#org.springframework.data.elasticsearch.annotations.Field(type = FieldType.Keyword)
private Long id;
#NotNull
#Column(name = "user_login", nullable = false)
private String userLogin;
#NotNull
#Column(name = "jhi_read", nullable = false)
private Boolean read;
#ManyToOne
private EntityAudit entityAudit;
Getters and setters...
I'm trying to achieve this query:
#Query("select distinct entityAudit from EntityAudit entityAudit " +
"join entityAudit.entityAuditStatuses entityAuditStatus " +
"where entityAuditStatus.userLogin =:userLogin " +
"order by entityAudit.timestamp desc")
Page<EntityAudit> retrieveAllByUserLogin(#Param(value = "userLogin") String userLogin, Pageable pageable);
But when I retrieve the data the EntityAuditStatuses are not filtered. I don't understand where the problem is.
Note: I removed the date property from the minimum reproducible example.
Use left join fetch instead of left join to make sure the dependent entityAuditStatuses are fetched as part of the join query itself, and not as multiple queries after finding the entityAudit. And since the result needs to be paginated, an additional countQuery will need to be specified (without the fetch). Working Query -
#Query(value = "select entityAudit from EntityAudit entityAudit " +
"left join fetch entityAudit.entityAuditStatuses entityAuditStatus " +
"where entityAuditStatus.userLogin = :userLogin ",
countQuery = "select entityAudit from EntityAudit entityAudit " +
"left join entityAudit.entityAuditStatuses entityAuditStatus " +
"where entityAuditStatus.userLogin = :userLogin ")
Without left join fetch, three queries are being generated - one which fetches the entityAuditId 1 (based on the userLogin 1) and then two more to fetch the entityAuditStatuses (from the entity_audit_status table only without the join) given the entityAuditId 1.
That is why, when you ask for userLogin = '1' - you retrieve the EntityAudit 1 which brings with it - entityAuditStatus 1 - entityAuditStatus 3 (which has userLogin = '2')
After adding left join fetch, there is only one query using join as per the defined entity relationships. So the results are correctly fetched.
I want to get some fields and then set it to my Test.entity. My SQL query:
query = "SELECT t.id as tId, t.test_name, t.duration, q.id as qId, " +
"q.question as question, q.is_multichoice as is_multichoice, " +
"q.is_open as is_open, a.id as aId, a.answer_text as answer_text FROM result r " +
"JOIN test t ON r.test_id = t.id " +
"JOIN user u ON r.user_id = u.id " +
"JOIN question q ON t.id = q.test_id JOIN answer a ON q.id = a.question_id " +
"WHERE t.id = :testId AND u.id = :userId AND r.permission = :permissionId " +
"AND q.archived = false AND a.archived = false", resultClass = com.bionic.entities.Test.class)
Test Entity:
public class Test {
#Id
#Column(name = "id")
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
#Column(name = "duration", nullable = false)
private int duration;
#Column(name = "test_name", nullable = false, unique = true)
private String testName;
#Column(name = "archived", nullable = false)
private boolean archived;
#OneToMany(mappedBy = "test", fetch = FetchType.EAGER)
private Set<Question> questions;
#ManyToMany(mappedBy = "tests")
private Set<User> users;
Question Entity:
public class Question {
#Id
#Column(name = "id")
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
#Column(name = "is_multichoice", nullable = false)
private boolean isMultichoice;
#Column(name = "is_open", nullable = false)
private boolean isOpen;
#Column(name = "picture")
private String picture;
#Column(name = "question")
private String question;
#ManyToOne
#JoinColumn(name = "test_id", nullable = false)
private Test test;
#Column(name = "archived", nullable = false)
private boolean isArchived;
#OneToMany(mappedBy = "question", fetch = FetchType.EAGER)
private Set<Answer> answers;
Answer Entity:
public class Answer {
#Id
#Column(name = "id")
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
#Column(name = "answer_text", nullable = false)
private String answerText;
#Column(name = "mark", nullable = false)
private int mark;
#ManyToOne
#JoinColumn(name = "question_id")
private Question question;
#Column(name = "picture")
private String picture;
#Column(name = "archived", nullable = false)
private boolean isArchived;
However, after executing this query i am getting exeption :
Caused by: java.sql.SQLException: Column 'id' not found.
DAO.class:
public Test getCurrentTest(long id, long testId, long permissionId) {
Query query = em.createNamedQuery("getCurrentTestById");
query.setParameter("userId", id);
query.setParameter("testId", testId);
query.setParameter("permissionId", permissionId);
return (Test) query.getSingleResult();
}
What am i doing wrong?
Your query doesn't return a field named id. It has fields named aId, qId, and tId.
You need to use the correct column names in your entities. For example, in your Test entity, you declared a column named id. Except your query doesn't return a column named id, it returns a column named tId. See below for an example of what needs to be changed.
public class Test {
#tId
#Column(name = "tId")
#GeneratedValue(strategy = GenerationType.AUTO)
private long tId;
....