Develop Reference

JPA query always returns all rows rather than rows that match criteria

I have a JPA query written like this:
public interface MyDataRepository extends CrudRepository<MyData, String> {
#Query("select md " +
"from MyData md " +
"join fetch md.child c " +
"where c.date = :date")
List<MyData> getMyDataOfDate(#NotNull LocalDate date);
#Query("select md " +
"from MyData md " +
"join fetch md.child c " +
"where c.name = :name")
List<MyData> getMyDataOfName(#NotNull String name);
#Query("select md " +
"from MyData md " +
"join fetch md.child c " +
"where md.type = :type")
List<MyData> getMyDataOfType(#NotNull String type);
}
Class MyData and Child are defined as:
class MyData {
String id;
String type;
#ManyToOne
#JoinColumn(name = "CHILD_ID", referencedColumnName = "ID", nullable = false, updatable = false)
Child child;
}
class Child {
String id;
String name;
LocalDate date;
}
The problem is that whenever I call the getMyDataOfDate method or getMyDataOfName method, they always return ALL rows rather than the rows that matches the where condition, as if the where clause never exists.
However, the getMyDataOfType method works fine. The difference of this method is that the where condition is on a property of md, not c.
What did I do wrong?

JPA does not allow filtering on join fetches. Reasons being is that when you specify a join fetch, you are telling JPA to fetch the parent and all its children defined by that relationship in the managed entities it returns. If filtering were allowed, the list of children, the relationship in the parent, might not reflect what is actually in the database. Take the case of Parent with many children
"Select parents from Parent p fetch join p.children c where c.firstName = 'Bob'"
For such a query, when you get a list of parents and calling getChildren on them, do you expect to see all their children or a list that only contains children named Bob? If the later (which is the only way to do so), how should JPA handle changes to a parents children list, and know what to do with the not-fetched children?
This is why JPA doesn't allow filtering over fetch joins, and they restrict it across all relationships to be consistent. If you want the parents who have children with the firstName of 'Bob', it would look like:
"Select parents from Parent p join p.children c where c.firstName = 'Bob'"
Every parent returned will be a complete representation of its state in the database based on its mappings; so accessing parent.getChildren will return the current state of its children list and not something affected by the way it was fetched.

How can I solve memory error in Hibernate Jpa? [duplicate]

I'm getting a warning in the Server log "firstResult/maxResults specified with collection fetch; applying in memory!". However everything working fine. But I don't want this warning.
My code is
public employee find(int id) {
return (employee) getEntityManager().createQuery(QUERY).setParameter("id", id).getSingleResult();
}
My query is
QUERY = "from employee as emp left join fetch emp.salary left join fetch emp.department where emp.id = :id"

Although you are getting valid results, the SQL query fetches all data and it's not as efficient as it should.
So, you have two options.
Fixing the issue with two SQL queries that can fetch entities in read-write mode
The easiest way to fix this issue is to execute two queries:
. The first query will fetch the root entity identifiers matching the provided filtering criteria.
. The second query will use the previously extracted root entity identifiers to fetch the parent and the child entities.
This approach is very easy to implement and looks as follows:
List<Long> postIds = entityManager
.createQuery(
"select p.id " +
"from Post p " +
"where p.title like :titlePattern " +
"order by p.createdOn", Long.class)
.setParameter(
"titlePattern",
"High-Performance Java Persistence %"
)
.setMaxResults(5)
.getResultList();
List<Post> posts = entityManager
.createQuery(
"select distinct p " +
"from Post p " +
"left join fetch p.comments " +
"where p.id in (:postIds) " +
"order by p.createdOn", Post.class)
.setParameter("postIds", postIds)
.setHint(
"hibernate.query.passDistinctThrough",
false
)
.getResultList();
Fixing the issue with one SQL query that can only fetch entities in read-only mode
The second approach is to use SDENSE_RANK over the result set of parent and child entities that match our filtering criteria and restrict the output for the first N post entries only.
The SQL query can look as follows:
#NamedNativeQuery(
name = "PostWithCommentByRank",
query =
"SELECT * " +
"FROM ( " +
" SELECT *, dense_rank() OVER (ORDER BY \"p.created_on\", \"p.id\") rank " +
" FROM ( " +
" SELECT p.id AS \"p.id\", " +
" p.created_on AS \"p.created_on\", " +
" p.title AS \"p.title\", " +
" pc.id as \"pc.id\", " +
" pc.created_on AS \"pc.created_on\", " +
" pc.review AS \"pc.review\", " +
" pc.post_id AS \"pc.post_id\" " +
" FROM post p " +
" LEFT JOIN post_comment pc ON p.id = pc.post_id " +
" WHERE p.title LIKE :titlePattern " +
" ORDER BY p.created_on " +
" ) p_pc " +
") p_pc_r " +
"WHERE p_pc_r.rank <= :rank ",
resultSetMapping = "PostWithCommentByRankMapping"
)
#SqlResultSetMapping(
name = "PostWithCommentByRankMapping",
entities = {
#EntityResult(
entityClass = Post.class,
fields = {
#FieldResult(name = "id", column = "p.id"),
#FieldResult(name = "createdOn", column = "p.created_on"),
#FieldResult(name = "title", column = "p.title"),
}
),
#EntityResult(
entityClass = PostComment.class,
fields = {
#FieldResult(name = "id", column = "pc.id"),
#FieldResult(name = "createdOn", column = "pc.created_on"),
#FieldResult(name = "review", column = "pc.review"),
#FieldResult(name = "post", column = "pc.post_id"),
}
)
}
)
The #NamedNativeQuery fetches all Post entities matching the provided title along with their associated PostComment child entities. The DENSE_RANK Window Function is used to assign the rank for each Post and PostComment joined record so that we can later filter just the amount of Post records we are interested in fetching.
The SqlResultSetMapping provides the mapping between the SQL-level column aliases and the JPA entity properties that need to be populated.
Now, we can execute the PostWithCommentByRank #NamedNativeQuery like this:
List<Post> posts = entityManager
.createNamedQuery("PostWithCommentByRank")
.setParameter(
"titlePattern",
"High-Performance Java Persistence %"
)
.setParameter(
"rank",
5
)
.unwrap(NativeQuery.class)
.setResultTransformer(
new DistinctPostResultTransformer(entityManager)
)
.getResultList();
Now, by default, a native SQL query like the PostWithCommentByRank one would fetch the Post and the PostComment in the same JDBC row, so we will end up with an Object[] containing both entities.
However, we want to transform the tabular Object[] array into a tree of parent-child entities, and for this reason, we need to use the Hibernate ResultTransformer.
The DistinctPostResultTransformer looks as follows:
public class DistinctPostResultTransformer
extends BasicTransformerAdapter {
private final EntityManager entityManager;
public DistinctPostResultTransformer(
EntityManager entityManager) {
this.entityManager = entityManager;
}
#Override
public List transformList(
List list) {
Map<Serializable, Identifiable> identifiableMap =
new LinkedHashMap<>(list.size());
for (Object entityArray : list) {
if (Object[].class.isAssignableFrom(entityArray.getClass())) {
Post post = null;
PostComment comment = null;
Object[] tuples = (Object[]) entityArray;
for (Object tuple : tuples) {
if(tuple instanceof Identifiable) {
entityManager.detach(tuple);
if (tuple instanceof Post) {
post = (Post) tuple;
}
else if (tuple instanceof PostComment) {
comment = (PostComment) tuple;
}
else {
throw new UnsupportedOperationException(
"Tuple " + tuple.getClass() + " is not supported!"
);
}
}
}
if (post != null) {
if (!identifiableMap.containsKey(post.getId())) {
identifiableMap.put(post.getId(), post);
post.setComments(new ArrayList<>());
}
if (comment != null) {
post.addComment(comment);
}
}
}
}
return new ArrayList<>(identifiableMap.values());
}
}
The DistinctPostResultTransformer must detach the entities being fetched because we are overwriting the child collection and we don’t want that to be propagated as an entity state transition:
post.setComments(new ArrayList<>());

Reason for this warning is that when fetch join is used, order in result sets is defined only by ID of selected entity (and not by join fetched).
If this sorting in memory is causing problems, do not use firsResult/maxResults with JOIN FETCH.

To avoid this WARNING you have to change the call getSingleResult to
getResultList().get(0)

This warning tells you Hibernate is performing in memory java pagination. This can cause high JVM memory consumption.
Since a developer can miss this warning, I contributed to Hibernate by adding a flag allowing to throw an exception instead of logging the warning (https://hibernate.atlassian.net/browse/HHH-9965).
The flag is hibernate.query.fail_on_pagination_over_collection_fetch.
I recommend everyone to enable it.
The flag is defined in org.hibernate.cfg.AvailableSettings :
/**
* Raises an exception when in-memory pagination over collection fetch is about to be performed.
* Disabled by default. Set to true to enable.
*
* #since 5.2.13
*/
String FAIL_ON_PAGINATION_OVER_COLLECTION_FETCH = "hibernate.query.fail_on_pagination_over_collection_fetch";

the problem is you will get cartesian product doing JOIN. The offset will cut your recordset without looking if you are still on same root identity class

I guess the emp has many departments which is a One to Many relationship. Hibernate will fetch many rows for this query with fetched department records. So the order of result set can not be decided until it has really fetch the results to the memory. So the pagination will be done in memory.
If you do not want to fetch the departments with emp, but still want to do some query based on the department, you can achieve the result with out warning (without doing ordering in the memory). For that simply you have to remove the "fetch" clause. So something like as follows:
QUERY = "from employee as emp left join emp.salary sal left join emp.department dep where emp.id = :id and dep.name = 'testing' and sal.salary > 5000 "

As others pointed out, you should generally avoid using "JOIN FETCH" and firstResult/maxResults together.
If your query requires it, you can use .stream() to eliminate warning and avoid potential OOM exception.
try (Stream<ENTITY> stream = em.createQuery(QUERY).stream()) {
ENTITY first = stream.findFirst().orElse(null); // equivalents .getSingleResult()
}
// Stream returned is an IO stream that needs to be closed manually.

JPQL include elementCollection map in select statement

I have an #Entity class Company with several attributes, referencing a companies Table in my db. One of them represents a Map companyProperties where the companies table is extended by a company_properties table, and the properties are saved in key-value format.
#Entity
#Table(name = "companies")
public class Company extends AbstractEntity {
private static final String TABLE_NAME = "companies";
#Id
#GeneratedValue(generator = TABLE_NAME + SEQUENCE_SUFFIX)
#SequenceGenerator(name = TABLE_NAME + SEQUENCE_SUFFIX, sequenceName = TABLE_NAME + SEQUENCE_SUFFIX, allocationSize = SEQUENCE_ALLOCATION_SIZE)
private Long id;
//some attributes
#ElementCollection
#CollectionTable(name = "company_properties", joinColumns = #JoinColumn(name = "companyid"))
#MapKeyColumn(name = "propname")
#Column(name = "propvalue")
private Map<String, String> companyProperties;
//getters and setters
}
The entity manager is able to perform properly find clauses
Company company = entityManager.find(Company.class, companyId);
However, I am not able to perform JPQL Queries in this entity and retrieve the Map accordingly. Since the object is big, I just need to select some of the attributes in my entity class. I also do not want to filter by companyProperties but to retrieve all of them coming with the proper assigned companyid Foreign Key. What I have tried to do is the following:
TypedQuery<Company> query = entityManager.createQuery("SELECT c.id, c.name, c.companyProperties " +
"FROM Company as c where c.id = :id", Company.class);
query.setParameter("id", companyId);
Company result = query.getSingleResult();
The error I get is:
java.lang.IllegalArgumentException: An exception occurred while creating a query in EntityManager:
Exception Description: Problem compiling [SELECT c.id, c.name, c.companyProperties FROM Company as c where c.id = :id]. [21, 40] The state field path 'c.companyProperties' cannot be resolved to a collection type.
org.eclipse.persistence.internal.jpa.EntityManagerImpl.createQuery(EntityManagerImpl.java:1616)
org.eclipse.persistence.internal.jpa.EntityManagerImpl.createQuery(EntityManagerImpl.java:1636)
com.sun.enterprise.container.common.impl.EntityManagerWrapper.createQuery(EntityManagerWrapper.java:476)
Trying to do it with joins (the furthest point I got was with
Query query = entityManager.createQuery("SELECT c.id, c.name, p " +
"FROM Company c LEFT JOIN c.companyProperties p where c.id = :id");
does not give me either the correct results (it only returns the value of the property and not a list of them with key-value).
How can I define the right query to do this?

Your JPA syntax looks off to me. In your first query you were selecting individual fields in the Company entity. But this isn't how JPA works; when you query you get back the entire object, with which you can access any field you want. I propose the following code instead:
TypedQuery<Company> query = entityManager.createQuery("from Company as c where c.id = :id", Company.class);
query.setParameter("id", companyId);
Company result = query.getSingleResult();
Similarly, for the second join query I suggest the following code:
Query query = entityManager.createQuery("SELECT c" +
"FROM Company c LEFT JOIN c.companyProperties p WHERE c.id = :id");
query.setParameter("id", companyId);
List<Company> companies = query.getResultList();
The reason why only select a Company and not a property entity is that properties would appear as a collection inside the Company class. Assuming a one to many exists between companies and properties, you could access the propeties from each Company entity.

You are expecting to get a complete Company object when doing select only on particular fields, which is not possible. If you really want to save some memory (which in most cases would not be that much of a success) and select only some field, then you should expect a List<Object[]>:
List<Object[]> results = entityManager.createQuery("SELECT c.id, c.name, p " +
"FROM Company c LEFT JOIN c.companyProperties p where c.id = :id")
.setParameter("id", companyId)
.getResultList();
Here the results will contain a single array of the selected fields. You can use getSingleResult, but be aware that it will throw an exception if no results were found.

Hibernate Criteria with self join

Have been learning Hibernate, Spring and JPA the last week and got stuck on trying to create a Criteria for the following scenario:
Let's say I have 2 tables:
Game
id
PlayedGame
id
account_ref -> reference to some account table
game_id -> reference to the game
Entity mapping:
Game {
#id
Long id;
#OneToMany(mappedBy = "game")
Set<Player> players;
}
PlayedGame {
#id
Long id;
Long account_ref;
#ManyToOne
#JoinColumn(name = "game_id", referencedColumnName = "id")
Game game;
}
Now I want to query for the following scenario:
- I want to find all games a specific player (P) has played where player (A) was a part of it. More specifically that two people belonging to the same game
In SQL this could be done with something like (which the query should be):
SELECT DISTINCT p1.* FROM Player as p1
INNER JOIN Player as p2 ON p1.game_id=p2.game_id
WHERE p1.account_ref=P AND p2.account_ref=A
Can this be done neatly with Criteria in Hibernate?

Maybe possible with Hibernate's Criteria API, but not straight forward.
A simple case would require the same association path to be join twice (one for A and one for P):
Criteria gameCriteria = ((HibernateEntityManager) em).getSession().createCriteria(Game.class);
Criteria playedGamesOfACriteria = gameCriteria.createCriteria("playedGames", "pga");
Criteria accountOfACriteria = playedGamesOfACriteria.createCriteria("account", "a");
accountOfACriteria.add(Restrictions.idEq(a.id));
Criteria playedGamesOfPCriteria = gameCriteria.createCriteria("playedGames", "pgp");
Criteria accountOfPCriteria = playedGamesOfPCriteria.createCriteria("account", "p");
accountOfPCriteria.add(Restrictions.idEq(p.id));
return gameCriteria.list();
This won't work due to HHH-879.
But you can use a JPA query:
Query q = em.createQuery(
"select g "
+ "from Game g "
+ "join g.playedGames pga "
+ "join pga.account a "
+ "join g.playedGames pgp "
+ "join pgp.account p "
+ "where a = ?1 and p = ?2"
);
q.setParameter(1, a);
q.setParameter(2, p);
return q.getResultList();
This is even less code.
Also: Some consider the Criteria API to be deprecated.

How can I avoid the Warning "firstResult/maxResults specified with collection fetch; applying in memory!" when using Hibernate?

I'm getting a warning in the Server log "firstResult/maxResults specified with collection fetch; applying in memory!". However everything working fine. But I don't want this warning.
My code is
public employee find(int id) {
return (employee) getEntityManager().createQuery(QUERY).setParameter("id", id).getSingleResult();
}
My query is
QUERY = "from employee as emp left join fetch emp.salary left join fetch emp.department where emp.id = :id"

Although you are getting valid results, the SQL query fetches all data and it's not as efficient as it should.
So, you have two options.
Fixing the issue with two SQL queries that can fetch entities in read-write mode
The easiest way to fix this issue is to execute two queries:
. The first query will fetch the root entity identifiers matching the provided filtering criteria.
. The second query will use the previously extracted root entity identifiers to fetch the parent and the child entities.
This approach is very easy to implement and looks as follows:
List<Long> postIds = entityManager
.createQuery(
"select p.id " +
"from Post p " +
"where p.title like :titlePattern " +
"order by p.createdOn", Long.class)
.setParameter(
"titlePattern",
"High-Performance Java Persistence %"
)
.setMaxResults(5)
.getResultList();
List<Post> posts = entityManager
.createQuery(
"select distinct p " +
"from Post p " +
"left join fetch p.comments " +
"where p.id in (:postIds) " +
"order by p.createdOn", Post.class)
.setParameter("postIds", postIds)
.setHint(
"hibernate.query.passDistinctThrough",
false
)
.getResultList();
Fixing the issue with one SQL query that can only fetch entities in read-only mode
The second approach is to use SDENSE_RANK over the result set of parent and child entities that match our filtering criteria and restrict the output for the first N post entries only.
The SQL query can look as follows:
#NamedNativeQuery(
name = "PostWithCommentByRank",
query =
"SELECT * " +
"FROM ( " +
" SELECT *, dense_rank() OVER (ORDER BY \"p.created_on\", \"p.id\") rank " +
" FROM ( " +
" SELECT p.id AS \"p.id\", " +
" p.created_on AS \"p.created_on\", " +
" p.title AS \"p.title\", " +
" pc.id as \"pc.id\", " +
" pc.created_on AS \"pc.created_on\", " +
" pc.review AS \"pc.review\", " +
" pc.post_id AS \"pc.post_id\" " +
" FROM post p " +
" LEFT JOIN post_comment pc ON p.id = pc.post_id " +
" WHERE p.title LIKE :titlePattern " +
" ORDER BY p.created_on " +
" ) p_pc " +
") p_pc_r " +
"WHERE p_pc_r.rank <= :rank ",
resultSetMapping = "PostWithCommentByRankMapping"
)
#SqlResultSetMapping(
name = "PostWithCommentByRankMapping",
entities = {
#EntityResult(
entityClass = Post.class,
fields = {
#FieldResult(name = "id", column = "p.id"),
#FieldResult(name = "createdOn", column = "p.created_on"),
#FieldResult(name = "title", column = "p.title"),
}
),
#EntityResult(
entityClass = PostComment.class,
fields = {
#FieldResult(name = "id", column = "pc.id"),
#FieldResult(name = "createdOn", column = "pc.created_on"),
#FieldResult(name = "review", column = "pc.review"),
#FieldResult(name = "post", column = "pc.post_id"),
}
)
}
)
The #NamedNativeQuery fetches all Post entities matching the provided title along with their associated PostComment child entities. The DENSE_RANK Window Function is used to assign the rank for each Post and PostComment joined record so that we can later filter just the amount of Post records we are interested in fetching.
The SqlResultSetMapping provides the mapping between the SQL-level column aliases and the JPA entity properties that need to be populated.
Now, we can execute the PostWithCommentByRank #NamedNativeQuery like this:
List<Post> posts = entityManager
.createNamedQuery("PostWithCommentByRank")
.setParameter(
"titlePattern",
"High-Performance Java Persistence %"
)
.setParameter(
"rank",
5
)
.unwrap(NativeQuery.class)
.setResultTransformer(
new DistinctPostResultTransformer(entityManager)
)
.getResultList();
Now, by default, a native SQL query like the PostWithCommentByRank one would fetch the Post and the PostComment in the same JDBC row, so we will end up with an Object[] containing both entities.
However, we want to transform the tabular Object[] array into a tree of parent-child entities, and for this reason, we need to use the Hibernate ResultTransformer.
The DistinctPostResultTransformer looks as follows:
public class DistinctPostResultTransformer
extends BasicTransformerAdapter {
private final EntityManager entityManager;
public DistinctPostResultTransformer(
EntityManager entityManager) {
this.entityManager = entityManager;
}
#Override
public List transformList(
List list) {
Map<Serializable, Identifiable> identifiableMap =
new LinkedHashMap<>(list.size());
for (Object entityArray : list) {
if (Object[].class.isAssignableFrom(entityArray.getClass())) {
Post post = null;
PostComment comment = null;
Object[] tuples = (Object[]) entityArray;
for (Object tuple : tuples) {
if(tuple instanceof Identifiable) {
entityManager.detach(tuple);
if (tuple instanceof Post) {
post = (Post) tuple;
}
else if (tuple instanceof PostComment) {
comment = (PostComment) tuple;
}
else {
throw new UnsupportedOperationException(
"Tuple " + tuple.getClass() + " is not supported!"
);
}
}
}
if (post != null) {
if (!identifiableMap.containsKey(post.getId())) {
identifiableMap.put(post.getId(), post);
post.setComments(new ArrayList<>());
}
if (comment != null) {
post.addComment(comment);
}
}
}
}
return new ArrayList<>(identifiableMap.values());
}
}
The DistinctPostResultTransformer must detach the entities being fetched because we are overwriting the child collection and we don’t want that to be propagated as an entity state transition:
post.setComments(new ArrayList<>());

Reason for this warning is that when fetch join is used, order in result sets is defined only by ID of selected entity (and not by join fetched).
If this sorting in memory is causing problems, do not use firsResult/maxResults with JOIN FETCH.

To avoid this WARNING you have to change the call getSingleResult to
getResultList().get(0)

This warning tells you Hibernate is performing in memory java pagination. This can cause high JVM memory consumption.
Since a developer can miss this warning, I contributed to Hibernate by adding a flag allowing to throw an exception instead of logging the warning (https://hibernate.atlassian.net/browse/HHH-9965).
The flag is hibernate.query.fail_on_pagination_over_collection_fetch.
I recommend everyone to enable it.
The flag is defined in org.hibernate.cfg.AvailableSettings :
/**
* Raises an exception when in-memory pagination over collection fetch is about to be performed.
* Disabled by default. Set to true to enable.
*
* #since 5.2.13
*/
String FAIL_ON_PAGINATION_OVER_COLLECTION_FETCH = "hibernate.query.fail_on_pagination_over_collection_fetch";

the problem is you will get cartesian product doing JOIN. The offset will cut your recordset without looking if you are still on same root identity class

I guess the emp has many departments which is a One to Many relationship. Hibernate will fetch many rows for this query with fetched department records. So the order of result set can not be decided until it has really fetch the results to the memory. So the pagination will be done in memory.
If you do not want to fetch the departments with emp, but still want to do some query based on the department, you can achieve the result with out warning (without doing ordering in the memory). For that simply you have to remove the "fetch" clause. So something like as follows:
QUERY = "from employee as emp left join emp.salary sal left join emp.department dep where emp.id = :id and dep.name = 'testing' and sal.salary > 5000 "

As others pointed out, you should generally avoid using "JOIN FETCH" and firstResult/maxResults together.
If your query requires it, you can use .stream() to eliminate warning and avoid potential OOM exception.
try (Stream<ENTITY> stream = em.createQuery(QUERY).stream()) {
ENTITY first = stream.findFirst().orElse(null); // equivalents .getSingleResult()
}
// Stream returned is an IO stream that needs to be closed manually.