efficiently loading collection of collections from database in hibernate - java

I have an web application with hibernate which manages data in multiple languages. Currently basically every request generates a shower of select statements on the languagetranslations. The models are roughly as following:
Data <1-1> Placeholder <1-many> languageTranslation <many-1> language
If I query for all/many Dataobjects, I see lots of single selects which select one languageTranslation for the placeholder. The SQL I optimally would want to generate:
SELECT * FROM data join placeholder join languagetranslation
WHERE data.placeholder_id = placeholder.id
AND languagetranslation.placeholder_id = placeholder.id
AND languagetranslation.language_id = ?
so that I get every data with placeholder with translation in one single call. The languagetranslations have an composite primary key of language_id and placeholder_id.
I have no HBM file, everything is managed with annotations. Modelcode (only relevant sections are shown):
#Entity
public class Data {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#OneToOne(fetch = FetchType.EAGER, cascade = CascadeType.ALL, optional = false)
#Fetch(FetchMode.JOIN)
private Placeholder content;
}
public class Placeholder {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#OneToMany(mappedBy = "primaryKey.placeholder", cascade = CascadeType.ALL, fetch = FetchType.EAGER, orphanRemoval = true)
#Fetch(FetchMode.JOIN)
private Set<LanguageTranslation> languageTranslations = new HashSet<>();
}
public class LanguageTranslation {
#EmbeddedId
private LanguageTranslationPK primaryKey = new LanguageTranslationPK();
#Type(type = "org.hibernate.type.StringClobType")
private String text;
}
#Embeddable
public class LanguageTranslationPK {
#ManyToOne(fetch = FetchType.EAGER)
#Fetch(FetchMode.JOIN)
private TextPlaceholder textPlaceholder;
#ManyToOne(fetch = FetchType.EAGER)
#Fetch(FetchMode.JOIN)
private Language language;
}
public class Language {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
}
I experimented with FetchType and FetchMode but couldn't generate the behavior I want, it always single selects for single languageTranslations.
I also tried multiple ways to query, criteria based, HQL, and raw SQL. My current raw SQL query is the following:
String sql_query = "select data.*, lt.* from Data as data join languagetranslation as lt on data.content_id = lt.textplaceholder_id";
Query q = getSession().createSQLQuery(sql_query).addEntity("data", Data.class).addJoin("data.content_id", "data.title").addJoin("lt", "data.content.languageTranslations").setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY);
return q.list();
Am I doing something generally wrong here? How can I convince hibernate to get all entities in one single database call? Or is there some other methods to improve performance in my case (e.g. batch selecting)?

You may create proxy pojo which have your all entity variables with getter setter and constructor. then initialize this constructor in hibernate query so that you just get all needed data from database.
import com.proxy;
class userProxy{
private string name;
private string password;
private string address;
private int pincode;
private byte[] profilePic;
private int age;
public userProxy(string name,string password){
this.name = name;
this.password = password;
}
//Getter and setter of all variable...
}
Then use this constructor to Hibernate query like
select new com.proxy.userProxy(user.name,user.password) from usertable

Am I doing something generally wrong here?
No, you are not. That is how Hibernate works.
How can I convince hibernate to get all entities in one single database call
You have to use HQL or SQL query to do that. You do not need to have HBM file. It can be done through #NamedQueries / #NamedQuery annotation with list method.
There are many samples on Internet as example simple one:
http://www.mkyong.com/hibernate/hibernate-named-query-examples/

Related

Translate native sql to JPA code or JPQL and make it pageable and sortable

I have two Entities like Below:
#Entity
#Table(name="tb_sm_config")
class Config {
#Id
private Long id;
private String code;
private String name;
#Enumerated(EnumType.STRING)
private State state;
#JsonIgnore
#OneToMany(mappedBy = "config", fetch = FetchType.LAZY)
private List<ConfigItem> items;
}
#Entity
#Table(name="tb_sm_config_item")
class ConfigItem {
#Id
private Long id;
private String code;
private String name;
#Enumerated(EnumType.STRING)
private State state;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "config_id", referencedColumnName = "id", foreignKey = #ForeignKey(ConstraintMode.NO_CONSTRAINT))
private Config config;
}
the State is an Enum, there are three states : VALID, INVALID, DELETED
Then I define a ConfigVO like below:
class ConfigVO {
private Long id;
private String code;
private String name;
private State state;
private Long validItemCount;
private Long invalidItemCount;
pirvate Long deletedItemCount;
}
I want to provide fuzzy query based on config.name, config.code and precise query based on config.state and return a list of ConfigVO with pageable and sortable.
I know native sql will like :
select a.id, a.code, a.name, a.state,
coalesce(b._valid, 0) validItemCount,
coalesce(b._invalid, 0) invalidItemCount,
coalesce(b._deleted, 0) deletedItemCount
from tb_sm_config a left join
(select config_id, sum(if(state = 'VALID', 1, 0)) _valid, sum(if(state = 'INVALID', 1, 0)) _invalid, sum(if(state = 'DELETED', 1, 0)) _deleted from tb_sm_config_item group by config_id) b
on a.id = b.config_id
where code like %:code% and name like %:name% and state = :state
But here are the problems I don't know how to deal with:
The frontend will not always pass the code, name, state to query, so these three parameters is nullable.
If I use the #Query(nativeQuery=true, value=xxxx) in the ConfigRepository interface method, I don't know how to deal with the null value and the paging and sorting.
Is that any possible to achieve this goal through JpaSpecificationExecutor interface, Example interface, or something else?
I want to query base on the code, name, state, sometimes they will be null and want to count the total number of individual configItem states and pageable and sortable.

How do i get a Spring data query to satisfy some relations in a projection (ie pass a Collection to a projections)?

So im trying to make a projection query in spring data. This is my model (in order not to make a huge post with a spam of classes, ill omit constructors, and some annotations):
public class TutorialDAO implements Serializable {
private UUID id;
private LocalDateTime created;
private String createdBy;
private LocalDateTime lastModified;
private String lastModifiedBy;
private int version;
private boolean exclusive;
#ManyToMany(mappedBy = "tutorials", cascade = CascadeType.PERSIST, fetch = FetchType.EAGER)
private Set<TopicDAO> topics = new HashSet<>();
#ElementCollection(fetch = FetchType.EAGER)
private Set<SectionDAO> sections = new HashSet<>();
#ManyToMany(cascade = CascadeType.PERSIST, fetch = FetchType.LAZY)
#JoinTable(name = "tutorial_courses", joinColumns = #JoinColumn(name = "tutorial_id"), inverseJoinColumns = #JoinColumn(name = "course_id"))
private Set<CourseDAO> courses = new HashSet<>();
#OneToMany(mappedBy = "tutorial", cascade = { CascadeType.DETACH, CascadeType.MERGE, CascadeType.PERSIST,
CascadeType.REFRESH }, orphanRemoval = true, fetch = FetchType.EAGER)
#MapKey(name = "localizedId.locale")
private Map<String, LocalizableTutorial> localizations = new HashMap<>();
public class TopicDAO implements Serializable {
private UUID id;
private String topic;
#ManyToMany(cascade = CascadeType.PERSIST, fetch = FetchType.LAZY)
private Set<TutorialDAO> tutorials = new HashSet<>();
}
public class LocalizedTutorialDAO {
private UUID id;
private LocalDateTime created;
private String createdBy;
private LocalDateTime lastModified;
private String lastModifiedBy;
private int version;
private boolean exclusive;
private String name, description;
private Set<TopicDAO> topics = new HashSet<>();
private Set<SectionDAO> sections = new HashSet<>();
public LocalizedTutorialDAO(UUID id, LocalDateTime created, String createdBy, LocalDateTime lastModified,
String lastModifiedBy, int version, boolean exclusive, String name, String description,
Set<TopicDAO> topics) {
super(id, created, createdBy, lastModified, lastModifiedBy, version);
System.out.println(topics);
System.out.println();
this.name = name;
this.description = description;
this.topics = topics;
}
}
Im trying to make a projection query that "fills" the LocalizedTutorialDAO class, which means i want the topics relation for example (same happens with sections, but since its a similar ill just mention topics).
I have this #Query annotation:
select new LocalizedTutorialDAO(t.id, t.created, t.createdBy, t.lastModified, t.lastModifiedBy, t.version, t.exclusive, (VALUE(l)).name, (VALUE(l)).description, topics) from tutorial t join t.localizations l join t.topics as topics where (VALUE(l)).name like %:name% and (KEY(l)) = :lang
To clear up, the TutorialDAO object has multi-language support, and my query gets the "normal" variables in TutorialDAO as well as the topics and sections relations. Then it gets the name and description from a given language for example i want to see a tutorial which name has "cooking fish" in portuguese - "pt". Now this part, concerning name and description is working, ive tested it without getting topics and sections and the entity gets populated correctly.
My issue here has been ive tried native sql, and spring data non native query and i can never get it to work.
Like this, the current query, it says:
org.hibernate.hql.internal.ast.QuerySyntaxException: Unable to locate appropriate constructor on class [com.fullstack.daos.projections.LocalizedTutorialDAO]. Expected arguments are: java.util.UUID, java.time.LocalDateTime, java.lang.String, java.time.LocalDateTime, java.lang.String, int, boolean, java.lang.String, java.lang.String, com.fullstack.daos.TopicDAO [select new com.fullstack.daos.projections.LocalizedTutorialDAO(t.id, t.created, t.createdBy, t.lastModified, t.lastModifiedBy, t.version, t.exclusive, (VALUE(l)).name, (VALUE(l)).description, topics) from com.fullstack.daos.TutorialDAO t join t.localizations l join t.topics as topics where (VALUE(l)).name like :name and (KEY(l)) = :lang]
And if i try the native where instead of putting topic on the constructor i put topic.id, topic.topic it never works always because it says it cant convert TopicDAO to Set. Is there any way to tell hibernate how to do this?
For example, the findById() query auto generated by spring works, generating this sql query:
select
tutorialda0_.id as id1_33_0_,
tutorialda0_.created as created2_33_0_,
tutorialda0_.created_by as created_3_33_0_,
tutorialda0_.last_modified as last_mod4_33_0_,
tutorialda0_.last_modified_by as last_mod5_33_0_,
tutorialda0_.version as version6_33_0_,
tutorialda0_.exclusive as exclusiv7_33_0_,
localizati1_.id as id1_8_1_,
localizati1_.locale as locale2_8_1_,
localizati1_.locale as formula413_1_,
localizati1_.id as id1_8_2_,
localizati1_.locale as locale2_8_2_,
localizati1_.description as descript3_8_2_,
localizati1_.name as name4_8_2_,
sections2_.tutorial_id as tutorial1_34_3_,
sectiondao3_.id as sections2_34_3_,
sectiondao3_.id as id1_22_4_,
sectiondao3_.parent_id as parent_i2_22_4_,
children4_.parent_id as parent_i2_22_5_,
children4_.id as id1_22_5_,
children4_.id as id1_22_6_,
children4_.parent_id as parent_i2_22_6_,
topics5_.tutorials_id as tutorial2_31_7_,
topicdao6_.id as topics_i1_31_7_,
topicdao6_.id as id1_27_8_,
topicdao6_.topic as topic2_27_8_
from
tutorials tutorialda0_
left outer join
localized_tutorial localizati1_
on tutorialda0_.id=localizati1_.id
left outer join
tutorials_sections sections2_
on tutorialda0_.id=sections2_.tutorial_id
left outer join
sections sectiondao3_
on sections2_.sections_id=sectiondao3_.id
left outer join
sections children4_
on sectiondao3_.id=children4_.parent_id
left outer join
topics_tutorials topics5_
on tutorialda0_.id=topics5_.tutorials_id
left outer join
topics topicdao6_
on topics5_.topics_id=topicdao6_.id
where
tutorialda0_.id=?
That's not possible with JPQL/HQL constructor expressions or Spring Data Projections as these concepts do not support collections.
I think this is a perfect use case for Blaze-Persistence Entity Views.
I created the library to allow easy mapping between JPA models and custom interface or abstract class defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure(domain model) the way you like and map attributes(getters) via JPQL expressions to the entity model.
A DTO model for your use case could look like the following with Blaze-Persistence Entity-Views:
#EntityView(Tutorial.class)
public interface LocalizedTutorialDAO {
#IdMapping
UUID getId();
LocalDateTime getCreated();
String getCreatedBy();
LocalDateTime getLastModified();
String getLastModifiedBy();
int getVersion();
boolean isExclusive();
#Mapping("localizations[:lang].name")
#AttributeFilter(ContainsFilter.class)
String getName();
#Mapping("localizations[:lang].description")
String getDescription();
Set<TopicDAO> getTopics();
#EntityView(Topic.class)
interface TopicDAO {
#IdMapping
UUID getId();
String getTopic();
}
}
Querying is a matter of applying the entity view to a query, the simplest being just a query by id.
LocalizedTutorialDAO a = entityViewManager.find(entityManager, LocalizedTutorialDAO.class, id);
The Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
Page<LocalizedTutorialDAO> findAll(Pageable pageable, EntityViewSettingProcessor<LocalizedTutorialDAO> processor);
and use it like:
Page<LocalizedTutorialDAO> p = repo.findAll(pageable, setting -> setting.addAttributeFilter("name", nameParameter));
The best part is, it will only fetch the state that is actually necessary!

How to stop Hibernate from eagerly fetching a relationship when it is mapped using a column (referencedColumnName) different than the primary key?

I'm mapping a relationship that does not use the entity's primary key. Using "referencedColumnName" with a column different than the primary key causes hibernate to eagerly fetch the association, by issuing an extra select, even when it's tagged with FetchType.LAZY.
My goal is to make it behave like a regular mapping, meaning it wouldn't issue an extra query every time I need to query the main entity.
I have already tried using #LazyToOne(LazyToOneOption.NO_PROXY), which sorts out the problem, but it does not operate well with Jackson's (JSON parsing library) module "jackson-datatype-hibernate5", which skips hibernate lazy proxies when serializing the results.
Here is a scenario almost like the one I have that causes the problem:
Entities:
#Entity(name = "Book")
#Table(name = "book")
public class Book
implements Serializable {
#Id
#GeneratedValue
private Long id;
private String title;
private String author;
#NaturalId
private String isbn;
//Getters and setters omitted for brevity
}
#Entity(name = "Publication")
#Table(name = "publication")
public class Publication {
#Id
#GeneratedValue
private Long id;
private String publisher;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(
name = "isbn",
referencedColumnName = "isbn"
)
private Book book;
#Column(
name = "price_in_cents",
nullable = false
)
private Integer priceCents;
private String currency;
//Getters and setters omitted for brevity
}
Repository (Spring-Data, but you could try directly with the EntityManager):
#Repository
public interface PublicationRepository extends JpaReadRepository <Publication, Long>
{
#Query ("SELECT d FROM Publication d WHERE d.publisher = ?1 ")
Optional <Publication> findByPublisher (String isbn);
}
Thanks
The only way to achieve what you are looking for is by moving the annotatation #Id to the isbn property.
You can leave the #GeneratedValue on the autoincrement property.
Notes:
1 - Make sure that your equals/hc are following the OID(Object ID) on your domain case the "NaturalId" ISBN.
2 - It will be good to ensure if possible on DB level that your natural ID has unique contraint on it.

Spring Boot / JPA / mySQL - many to one relationship creates too many SQL queries

I have a simple spring boot rest app connected with mySQL db and I'm trying to optimize number of queries within simple function:
List<Message> messages = messagesRepository.findBySenderIdOrReceiverIdOrderByTimeDesc(senderId, receiverId);
MessagesRepository:
public interface MessagesRepository extends CrudRepository<Message, Long> {
List<Message> findBySenderIdOrReceiverIdOrderByTimeDesc(Long senderId, Long receiverId);
}
Message:
#Entity
#Table(name="s_messages")
public class Message implements Serializable
{
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private Long id;
#Transient
private int internalId;
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name="senderId", referencedColumnName = "id", updatable=false, insertable=false)
private ProfileLite sender;
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name="receiverId", referencedColumnName = "id", updatable=false, insertable=false)
private ProfileLite receiver;
#Column(columnDefinition="TEXT")
private String message;
private long time;
private MessageStatus status;
}
ProfileLite:
#Entity
#Table(name="s_profiles")
public class ProfileLite implements Comparable<ProfileLite>
{
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private Long id;
private String nickname;
private String country;
private String thumb;
private Gender gender;
}
After executing method mentioned above, hibernate generates about 40 SQL's (based on 40 profiles) like this:
SQL Log - PasteBin
so first collecting messages and then for each message creates another sql to gather profile.
Is it any possibility to push hibernate to create just one simple sql instead of 40 like: select * from s_messages m join s_profiles s1 on m.sender_id = s1.id join s_profiles s2 m_receiver_id = s2.id ? (pseudo code)
Thanks!
This could be a n + 1 problem.
You can use a JOIN FETCH in your JPA query to fix this.
A "fetch" join allows associations or collections of values to be initialized along with their parent objects using a single select. This is particularly useful in the case of a collection. It effectively overrides the outer join and lazy declarations of the mapping file for associations and collections.
Update your JPA repository like so
public interface MessagesRepository extends CrudRepository<Message, Long> {
#Query("Select m from Message m join fetch m.sender ms join fetch m.receiver mr where ms.id = :senderId or mr.id = :receiverId order by m.time desc")
List<Message> findBySenderIdOrReceiverIdOrderByTimeDesc(Long senderId, Long receiverId);
}
For a more detailed explanation check out this answer.
PS: I havent tested the query.

Hibernate #Where annotation on a ManyToOne relation

I have recently started to refactor my project because I had to add an extra column to some of my table. The extra column is an Enum (Pending, or Active).
Because of that change I would need now to refactor ALL my queries to only retrieves a row if the status is ACTIVE.
After some research I found that we can annotate an Entity with the #Where annotation. it works fine where I use it on a simple column but my table look like this:
#Where(clause = 'state='ACTIVE'")
#Entity
public class Place {
#Column(name="id_place")
private String placeId;
#Column(name="name")
private String palceName;
#OneToMany(mappedBy = "place")
private Set<PlaceTag> placeTag;
...
...
}
#Where(clause = 'state='ACTIVE'")
#Entity
public class Tag {
#Column(name="id_tag")
private String tagId;
#Column(name="name")
private String tagName;
#OneToMany(mappedBy = "tag")
private Set<PlaceTag> placeTag;
...
...
}
#Where(clause = 'poi.state='ACTIVE' AND tag.state='ACTIVE")
#Entity
public class PlaceTag {
#Column(name="id")
private String id;
#ManyToOne(cascade = CascadeType.DETACH, fetch = FetchType.LAZY)
#JoinColumn(name = "place_id")
private Place place;
#ManyToOne(cascade = CascadeType.DETACH, fetch = FetchType.LAZY)
#JoinColumn(name = "tag_id")
private Tag tag;
...
...
}
Now my question would be how can make this statement ONLY return the places and tags that are ACTIVE ?
SELECT pt FROM PlaceTag pt;
Is this possible? Or will I have to write the query Explicitly ?
Thank you
As you already discovered, or simply use cases the #Where clause is just fine, but in your case, you want to filter PlaceTag by the place and tag too, so a joined is required in this situation.
So, you can keep the #Where clause for Place and Tag, while for PlaceTags you need to use a JPQL query:
select pt
from PlaceTag pt
join pt.tag t
join pt.place p
where
t.state='ACTIVE' and p.state='ACTIVE'
At least until #WhereJoinTable annotation is made to work for many-to-one associations too.

Categories

Resources