Develop Reference

Spring Boot JPA Native Query Object Mapping

I have a Model called CommunityProfile. This model contains two child relationships; player (type User), and rank (type Rank).
The default spring boot JPA-generated query is taking approximately 9s to fetch 200 records, which is rather slow. By using the following MySQL query, I can return the data I need rather quickly:
SELECT cp.*, r.*, u.* FROM community_profiles cp
LEFT JOIN users u ON cp.player_id = u.id
LEFT JOIN ranks r ON cp.rank_id = r.id
WHERE cp.community_id = 1
How can I make my repository map the results to their correct Objects/Models?
I have tried using a non-native query, like this:
#Query("SELECT cp FROM CommunityProfile cp " +
"LEFT JOIN FETCH cp.player u " +
"LEFT JOIN FETCH cp.rank r " +
"WHERE cp.communityId = :communityId")
List<CommunityProfile> findByCommunityIdWithJoin(#Param("communityId") Integer communityId);
However, this is still quite slow in comparison, resulting in an 800-900ms response. For comparison, my current Laravel application can return the same data in a 400-ms cold start.
Any tips are appreciated, thank you
==UPDATE==
After trying the suggested #Index annotation, I still don't really see any performance gains. Did I implement correctly?
#Entity
#Table(name = "community_profiles", indexes = #Index(name = "cp_ci_idx", columnList = "community_id"))
public class CommunityProfile {

If your JPA query is working, and you are just asking about performance, you may add the following index:
CREATE INDEX idx ON community_profiles(community_id);
This index should allow MySQL to filter off records which are not part of the result set.
From JPA itself you may use:
#Table(indexes = #Index(name = "idx", columnList = "community_id"))
public class CommunityProfile {
// ...
}

Have you tried EntityManager
#PersistenceContext(type = PersistenceContextType.TRANSACTION)
private EntityManager entityManager;
List<CommunityProfile> findByCommunityIdWithJoin(Integer communityId){
String query = ""SELECT cp FROM CommunityProfile cp " +
"LEFT JOIN FETCH cp.player u " +
"LEFT JOIN FETCH cp.rank r " +
"WHERE cp.communityId = :communityId"
List<CommunityProfile> list = entityManager.createNativeQuery(query, CommunityProfile.class)
.setParameter("communityId",communityId)
.getResultList();
entityManager.clear();
return list
}
Once I used this kind of native query inside loop and it constantly returned cash values bu entityManagaer.clear() clears cash. This is for info only)
Or create an Index on specific columns when you are defining entity classes like:
#Table(indexes = {
#Index(columnList = "firstName"),
#Index(name = "fn_index", columnList = "firstName"),
#Index(name = "mulitIndex1", columnList = "firstName, lastName")
...
}
For Non-entity #Index you can check documentation

spring data jpa, native query not setting query parameter

I have a Spring JPA repository, with a native query defined, the query appears to execute but doesn't produce any results.
Repository class
#Repository
public interface AddressBaseRepository extends JpaRepository<Address, String> {
#Query(value = "select ADDRESSES.UPRN,ADDRESSES.FID,ADDRESSES.ORGANISATION,ADDRESSES.SUB_BUILDING,BUILDINGS.NAME as BUILDING,ADDRESSES.BUILDING_NUMBER,STREETS.NAME as STREET,TOWNS.NAME as TOWN,LOCALITY.NAME as LOCALITY,POSTCODES.POSTCODE,"
+ "EASTING,"
+ "NORTHING,"
+ "LATITUDE,"
+ "LONGITUDE"
+ " FROM ADDRESSES "
+ " INNER JOIN STREETS ON STREETS.ID = STREET"
+ " INNER JOIN TOWNS ON TOWNS.ID = TOWN"
+ " INNER JOIN POSTCODES ON POSTCODES.ID = POST_CODE"
+ " LEFT JOIN LOCALITY ON LOCALITY.ID = LOCALITY"
+ " LEFT JOIN BUILDINGS ON BUILDINGS.ID = BUILDING_NAME"
+ " LEFT JOIN ORGANISATIONS ON ORGANISATIONS.ID = ORGANISATION"
+ " WHERE ADDRESSES.UPRN = ?1", nativeQuery = true)
List<Address> getAddressByUprn(String uprn);
}
On inspection of the logs it can be seen that the query parameter is not being set
Hibernate: select ADDRESSES.UPRN,ADDRESSES.FID,ADDRESSES.ORGANISATION,ADDRESSES.SUB_BUILDING,BUILDINGS.NAME as BUILDING,ADDRESSES.BUILDING_NUMBER,STREETS.NAME as STREET,TOWNS.NAME as TOWN,LOCALITY.NAME as LOCALITY,POSTCODES.POSTCODE,EASTING,NORTHING,LATITUDE,LONGITUDE FROM ADDRESSES INNER JOIN STREETS ON STREETS.ID = STREET INNER JOIN TOWNS ON TOWNS.ID = TOWN INNER JOIN POSTCODES ON POSTCODES.ID = POST_CODE LEFT JOIN LOCALITY ON LOCALITY.ID = LOCALITY LEFT JOIN BUILDINGS ON BUILDINGS.ID = BUILDING_NAME LEFT JOIN ORGANISATIONS ON ORGANISATIONS.ID = ORGANISATION WHERE ADDRESSES.UPRN = ?
Could somebody point me in the right direction as to why the parameter is not being set, I have also tried named parameters and get the same result.

The SQL gets send with a placeholder and the values for the bind parameters get send separately. This is not the reason why you don't see the expected results. Configure logging correctly and you'll be able to see the bound parameters.
See How to print a query string with parameter values when using Hibernate for how to do that for Hiberante.

Filter rows in DataSet using JPA/Hibernate

I have the following code which is used to retrieve data from multiple tables (using joins) and then mapping every row into a DTOList but I also need to apply filters based on user preferences: per table1.name or table2.name, table3, etc.
So I just want to know what would be the best way to do it in terms of performance and best practices;
retrieving all rows and then apply the filters with lambdas (easier)
change the query to a dynamic query with Criteria or something else?
Any other solution=?
#Repository
public class ArchiveRepository {
#Autowired
EntityManager em;
String queryStr = "select wsr.id as sampleid, s.id as slideid, tb.name as batchname, k.lot_number as kitlot, " +
" 'STRING' as slidetype, tb.worklist_name as worklist, wsr.final_call as results, " +
" wa.final_pattern_ids as patterns, 'edited/yesno' as edited, wsr.last_modified_by as user, wsr.last_modified_date as time " +
" from slide s " +
" left join table2 tb on s.test_batch_id = tb.id " +
" left join table3 k on tb.kit_lot_id = k.id " +
" left join table4 w on s.id = w.slide_id " +
" left join tabl5 pw on pw.well_id = w.id " +
" left join tabl6 cw on cw.well_id = w.id " +
" left join tabl7 wsr on wsr.patient_well_sample_id = pw.id or wsr.control_sample_id = cw.id " +
" left join (select * from *** (other subselect)) wa on wa.well_sample_id = wsr.**id or wa.well_sample_id = wsr.**id " +
"where tb.state = 'STATENEEDED'";
public ArchiveDataListDTO getArchiveData(){
Query query = em.createNativeQuery(queryStr);
ArchiveDataListDTO archiveDataListDTO = new ArchiveDataListDTO();
List<Object[]> resultL = (List<Object[]>)query.getResultList();
for( Object[] o : resultL){
archiveDataListDTO.addArchiveDataRow(
new ArchiveDataDTO((String)o[0], String.valueOf(o[1]), (String) o[2], (String) o[3], (String) o[4], (String) o[5],
(String) o[6], (String) o[7], (String) o[8], (String) o[9], (String) o[10]));
}
return archiveDataListDTO;
}
}
**
note I struggled some with the code cause I wanted to apply #sqlresultsetmapping to avoid manual results mapping but it just didn´t work, most of the examples out there are when you have an entity in the DB but in this case I retrieve from many tables.**
Thanks so much

2 .- change the query to a dynamic query with Criteria or something else?

I ended up creating the query on the fly; depending on the filters I get from UI i assemble the query with Java and send it to DB, it´s easier since this required many tables...

Spring Data JPA: How not to repeat myself in countQueries?

I am using Spring Data JPA repositories (1.7.2) and I am typically facing the following scenario:
entities have lazy-loaded collections
those collections are sometimes eagerly fetched (via JPAQL fetch join)
repositories often return Page<Foo> instead of List<Foo>
I need to provide countQuery to every #Query that uses fetch joins on a repository that returns a Page. This issue has been discussed in this StackOverflow question
My typical repository method looks like this:
#Query(value = "SELECT e FROM Employee e LEFT JOIN FETCH e.addresses a " +
"WHERE e.company.id = :companyId " +
"AND e.deleted = false " +
"AND e.primaryAddress.deleted = false " +
"ORDER BY e.id, a.id",
countQuery="SELECT count(e) FROM Employee e WHERE e.companyId = :companyId AND e.deleted = false AND e.primaryAddress.deleted = false"
)
Page<Employee> findAllEmployeesWithAddressesForCompany(#Param("companyId") long companyId, Pageable pageable);
Obviously, it's not very DRY. You can tell that I am repeating all of the conditions in both value and countQuery parameters. How do I stay DRY here?

You could do something like this
public interface MyRepository extends JpaRepository {
public static final String WHERE_PART = "e.companyId = :companyId AND e.deleted = false AND e.primaryAddress.deleted = false ";
#Query(value = "SELECT e FROM Employee e LEFT JOIN FETCH e.addresses a " +
"WHERE " + MyRepository.WHERE_PART
"ORDER BY e.id, a.id",
countQuery="SELECT count(e) FROM Employee e WHERE " + MyRepository.WHERE_PART
)
Page<Employee> findAllEmployeesWithAddressesForCompany(#Param("companyId") long companyId, Pageable pageable);

How can I avoid the Warning "firstResult/maxResults specified with collection fetch; applying in memory!" when using Hibernate?

I'm getting a warning in the Server log "firstResult/maxResults specified with collection fetch; applying in memory!". However everything working fine. But I don't want this warning.
My code is
public employee find(int id) {
return (employee) getEntityManager().createQuery(QUERY).setParameter("id", id).getSingleResult();
}
My query is
QUERY = "from employee as emp left join fetch emp.salary left join fetch emp.department where emp.id = :id"

Although you are getting valid results, the SQL query fetches all data and it's not as efficient as it should.
So, you have two options.
Fixing the issue with two SQL queries that can fetch entities in read-write mode
The easiest way to fix this issue is to execute two queries:
. The first query will fetch the root entity identifiers matching the provided filtering criteria.
. The second query will use the previously extracted root entity identifiers to fetch the parent and the child entities.
This approach is very easy to implement and looks as follows:
List<Long> postIds = entityManager
.createQuery(
"select p.id " +
"from Post p " +
"where p.title like :titlePattern " +
"order by p.createdOn", Long.class)
.setParameter(
"titlePattern",
"High-Performance Java Persistence %"
)
.setMaxResults(5)
.getResultList();
List<Post> posts = entityManager
.createQuery(
"select distinct p " +
"from Post p " +
"left join fetch p.comments " +
"where p.id in (:postIds) " +
"order by p.createdOn", Post.class)
.setParameter("postIds", postIds)
.setHint(
"hibernate.query.passDistinctThrough",
false
)
.getResultList();
Fixing the issue with one SQL query that can only fetch entities in read-only mode
The second approach is to use SDENSE_RANK over the result set of parent and child entities that match our filtering criteria and restrict the output for the first N post entries only.
The SQL query can look as follows:
#NamedNativeQuery(
name = "PostWithCommentByRank",
query =
"SELECT * " +
"FROM ( " +
" SELECT *, dense_rank() OVER (ORDER BY \"p.created_on\", \"p.id\") rank " +
" FROM ( " +
" SELECT p.id AS \"p.id\", " +
" p.created_on AS \"p.created_on\", " +
" p.title AS \"p.title\", " +
" pc.id as \"pc.id\", " +
" pc.created_on AS \"pc.created_on\", " +
" pc.review AS \"pc.review\", " +
" pc.post_id AS \"pc.post_id\" " +
" FROM post p " +
" LEFT JOIN post_comment pc ON p.id = pc.post_id " +
" WHERE p.title LIKE :titlePattern " +
" ORDER BY p.created_on " +
" ) p_pc " +
") p_pc_r " +
"WHERE p_pc_r.rank <= :rank ",
resultSetMapping = "PostWithCommentByRankMapping"
)
#SqlResultSetMapping(
name = "PostWithCommentByRankMapping",
entities = {
#EntityResult(
entityClass = Post.class,
fields = {
#FieldResult(name = "id", column = "p.id"),
#FieldResult(name = "createdOn", column = "p.created_on"),
#FieldResult(name = "title", column = "p.title"),
}
),
#EntityResult(
entityClass = PostComment.class,
fields = {
#FieldResult(name = "id", column = "pc.id"),
#FieldResult(name = "createdOn", column = "pc.created_on"),
#FieldResult(name = "review", column = "pc.review"),
#FieldResult(name = "post", column = "pc.post_id"),
}
)
}
)
The #NamedNativeQuery fetches all Post entities matching the provided title along with their associated PostComment child entities. The DENSE_RANK Window Function is used to assign the rank for each Post and PostComment joined record so that we can later filter just the amount of Post records we are interested in fetching.
The SqlResultSetMapping provides the mapping between the SQL-level column aliases and the JPA entity properties that need to be populated.
Now, we can execute the PostWithCommentByRank #NamedNativeQuery like this:
List<Post> posts = entityManager
.createNamedQuery("PostWithCommentByRank")
.setParameter(
"titlePattern",
"High-Performance Java Persistence %"
)
.setParameter(
"rank",
5
)
.unwrap(NativeQuery.class)
.setResultTransformer(
new DistinctPostResultTransformer(entityManager)
)
.getResultList();
Now, by default, a native SQL query like the PostWithCommentByRank one would fetch the Post and the PostComment in the same JDBC row, so we will end up with an Object[] containing both entities.
However, we want to transform the tabular Object[] array into a tree of parent-child entities, and for this reason, we need to use the Hibernate ResultTransformer.
The DistinctPostResultTransformer looks as follows:
public class DistinctPostResultTransformer
extends BasicTransformerAdapter {
private final EntityManager entityManager;
public DistinctPostResultTransformer(
EntityManager entityManager) {
this.entityManager = entityManager;
}
#Override
public List transformList(
List list) {
Map<Serializable, Identifiable> identifiableMap =
new LinkedHashMap<>(list.size());
for (Object entityArray : list) {
if (Object[].class.isAssignableFrom(entityArray.getClass())) {
Post post = null;
PostComment comment = null;
Object[] tuples = (Object[]) entityArray;
for (Object tuple : tuples) {
if(tuple instanceof Identifiable) {
entityManager.detach(tuple);
if (tuple instanceof Post) {
post = (Post) tuple;
}
else if (tuple instanceof PostComment) {
comment = (PostComment) tuple;
}
else {
throw new UnsupportedOperationException(
"Tuple " + tuple.getClass() + " is not supported!"
);
}
}
}
if (post != null) {
if (!identifiableMap.containsKey(post.getId())) {
identifiableMap.put(post.getId(), post);
post.setComments(new ArrayList<>());
}
if (comment != null) {
post.addComment(comment);
}
}
}
}
return new ArrayList<>(identifiableMap.values());
}
}
The DistinctPostResultTransformer must detach the entities being fetched because we are overwriting the child collection and we don’t want that to be propagated as an entity state transition:
post.setComments(new ArrayList<>());

Reason for this warning is that when fetch join is used, order in result sets is defined only by ID of selected entity (and not by join fetched).
If this sorting in memory is causing problems, do not use firsResult/maxResults with JOIN FETCH.

To avoid this WARNING you have to change the call getSingleResult to
getResultList().get(0)

This warning tells you Hibernate is performing in memory java pagination. This can cause high JVM memory consumption.
Since a developer can miss this warning, I contributed to Hibernate by adding a flag allowing to throw an exception instead of logging the warning (https://hibernate.atlassian.net/browse/HHH-9965).
The flag is hibernate.query.fail_on_pagination_over_collection_fetch.
I recommend everyone to enable it.
The flag is defined in org.hibernate.cfg.AvailableSettings :
/**
* Raises an exception when in-memory pagination over collection fetch is about to be performed.
* Disabled by default. Set to true to enable.
*
* #since 5.2.13
*/
String FAIL_ON_PAGINATION_OVER_COLLECTION_FETCH = "hibernate.query.fail_on_pagination_over_collection_fetch";

the problem is you will get cartesian product doing JOIN. The offset will cut your recordset without looking if you are still on same root identity class

I guess the emp has many departments which is a One to Many relationship. Hibernate will fetch many rows for this query with fetched department records. So the order of result set can not be decided until it has really fetch the results to the memory. So the pagination will be done in memory.
If you do not want to fetch the departments with emp, but still want to do some query based on the department, you can achieve the result with out warning (without doing ordering in the memory). For that simply you have to remove the "fetch" clause. So something like as follows:
QUERY = "from employee as emp left join emp.salary sal left join emp.department dep where emp.id = :id and dep.name = 'testing' and sal.salary > 5000 "

As others pointed out, you should generally avoid using "JOIN FETCH" and firstResult/maxResults together.
If your query requires it, you can use .stream() to eliminate warning and avoid potential OOM exception.
try (Stream<ENTITY> stream = em.createQuery(QUERY).stream()) {
ENTITY first = stream.findFirst().orElse(null); // equivalents .getSingleResult()
}
// Stream returned is an IO stream that needs to be closed manually.