Develop Reference

Spring Data JPA Distinct Returning Duplicate Values

In my Spring boot application I have a query which should return a distinct List of Focus' (works perfectly in MySQL)
#Query(value = "SELECT DISTINCT * FROM Focus F " +
"JOIN FrameworkFoci FF on FF.focusID = F.focusID " +
"JOIN FocusGroups FG on FF.frameworkID = FG.frameworkID " +
"JOIN GroupMembers GM on FG.groupID = GM.groupID " +
"JOIN Users U on GM.userID = U.userID " +
"WHERE U.userID = :userID", nativeQuery = true)
List<Focus> findByUserID(#Param("userID") Long userID);
However this does not return distinct values, duplicates are contained in the resulting list. Another issue is that I can't return a whole entity using #Query annotation - changing my query to SELECT DISTINCT(F) FROM Focus F gives the error java.sql.SQLSyntaxErrorException: Unknown column 'F' in 'field list'.
Furthermore, I tried changing adjusting the query to the following
#Query(value = "SELECT DISTINCT * FROM FrameworkFoci FF " +
"JOIN FocusGroups FG on FF.frameworkID = FG.frameworkID " +
"JOIN GroupMembers GM on FG.groupID = GM.groupID " +
"JOIN Users U on GM.userID = U.userID " +
"WHERE U.userID = :userID", nativeQuery = true)
however this produced the error java.sql.SQLException: Column 'focusCategory' not found.
Why is the query not returning distinct values? And why can't I return a whole entity nor use the second query?
Focus Entity:
#Entity
public class Focus {
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private Long focusID;
#Column(name = "focusCategory")
private String focusCategory;
private String focusName;
private String focusExplanation;
#OneToMany(mappedBy = "focus")
private Set<Rating> ratings;
#ManyToMany
#JoinTable(name = "FrameworkFoci",
joinColumns = #JoinColumn(
name = "focusID"),
inverseJoinColumns = #JoinColumn(
name = "frameworkID"))
private Set<Framework> frameworks;
//image
protected Focus(){}
public Focus(String focusName, String focusCategory, String focusExplanation) {
this.focusCategory = focusCategory;
this.focusName = focusName;
this.focusExplanation = focusExplanation;
}
public Focus(String focusCategory, String focusName, String focusExplanation, Set<Rating> ratings){
this.focusCategory = focusCategory;
this.focusName = focusName;
this.focusExplanation = focusExplanation;
this.ratings = ratings;
}
public Long getFocusId() {
return focusID;
}
public void setFocusId(Long focusID) {
this.focusID = focusID;
}
public String getFocusCategory() {
return focusCategory;
}
public void setFocusCategory(String focusCategory) {
this.focusCategory = focusCategory;
}
EDIT:
I've switched from SQL to JPQL with the following query:
#Query(value = "SELECT DISTINCT focus FROM Focus focus " +
"WHERE focus.frameworks.groups.groupMembers.user.userID =:userID ")
I now get an error org.hibernate.QueryException: illegal attempt to dereference collection [focus0_.focusID.frameworks] with element property reference [groups]
Framework entity:
#Entity
#JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator.class,
property = "frameworkID")
public class Framework {
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private Long frameworkID;
private String frameworkName;
#ManyToMany
#JoinTable(name = "FrameworkFoci",
joinColumns = #JoinColumn(
name = "frameworkID"),
inverseJoinColumns = #JoinColumn(
name = "focusID"))
private Set<Focus> frameworkFoci = new HashSet<>();
#OneToMany(mappedBy = "framework", fetch = FetchType.EAGER, cascade = CascadeType.REMOVE)
private Set<Group> groups;
public Framework(){}

The following query solves the issue
#Query(value = "SELECT DISTINCT focus FROM Focus focus " +
"JOIN focus.frameworks frameworks " +
"JOIN frameworks.groups groups " +
"JOIN groups.groupMembers groupMembers "+
"WHERE groupMembers.userID =:userID ")
List<Focus> findByUserID(#Param("userID") Long userID);
Frameworks and GroupMembers are collections and hence needed to be joined, otherwise illegal attempt to dereference collection [focus0_.focusID.frameworks] with element property reference [groups] was produced

you should write your query like this:
'
SELECT DISTINCT f FROM Focus F '

The problem stems from you using SQL by specifying nativeQuery = true. SQL doesn't know about entities, just tables.
Since you presumably have many FrameworkFoci rows (and rows in all the other tables) for each Focus row, each Focus row gets repeated for each matching row in FrameworkFoci. This kind of duplicates the Focus row but the resulting rows are still distinct, because they differ in the columns from the other tables.
And then each row gets turned into a Focus entity, probably with a single element in the framework set.
So therefore query doesn't so much return duplicate results as results split into multiple entities.
Fortunately the solution should be fairly simple: Use JPQL which should be perfectly possible, since you're using only simple joins.
The following should give you a start:
#Query(value = "SELECT DISTINCT * FROM Focus F " +
"WHERE F.framework.groupMembers.user.id=:userID")
List<Focus> findByUserID(#Param("userID") Long userID);

Java Spring hibernate HQL where clause not working

I have a HQL query with a JOIN but the where clause (instrPrice.date BETWEEN :dateFrom AND :dateTo ) on the joined entity doesn't work. The query always returns all the records of instrumentPrice instead of limiting the result by the dates.
NamedQuery
#NamedQuery(name = "findAllPrices",
query = "SELECT DISTINCT taPat FROM TaPatternInstrument taPat "
+ "LEFT JOIN FETCH taPat.instrument instr "
+ "LEFT JOIN instr.instrumentPriceList instrPrice "
+ "WHERE taPat.id = :taPatternInstrumentId "
+ "AND instrPrice.date BETWEEN :dateFrom AND :dateTo ")
Service which calls the Query
public TaPatternInstrument findAllPrices(int taPatternInstrumentId, LocalDate dateFrom, LocalDate dateTo) {
TypedQuery<TaPatternInstrument> typedQuery = createNamedQuery("findAllPrices",
TaPatternInstrument.class);
typedQuery.setParameter("taPatternInstrumentId", taPatternInstrumentId);
typedQuery.setParameter("dateFrom", dateFrom);
typedQuery.setParameter("dateTo", dateTo);
return typedQuery.getSingleResult();
}
Entities
public abstract class BaseEntity implements Serializable {
#Id
#Column(name = "id")
#GeneratedValue(strategy =
GenerationType.IDENTITY)
protected int id; ...
}
public class TaPatternInstrument extends BaseEntity {
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "instrument_id", nullable = false, foreignKey = #ForeignKey(name =
"tapatterninstrument_instrument_fk"))
private Instrument instrument;
}
public class Instrument extends BaseEntity {
#OneToMany(mappedBy = "instrument", fetch = FetchType.LAZY, cascade = CascadeType.ALL)
private List<InstrumentPrice> instrumentPriceList;
}
Generated SQL
SELECT DISTINCT tapatterni0_.id AS id1_34_0_,
...
FROM tapatterninstrument tapatterni0_
LEFT OUTER JOIN instrument instrument1_
ON tapatterni0_.instrument_id = instrument1_.id
LEFT OUTER JOIN instrumentprice instrument2_
ON instrument1_.id = instrument2_.instrument_id
WHERE tapatterni0_.id = ?
AND ( instrument2_.date BETWEEN ? AND ? )

The solution is to add a FETCH on instrumentPriceList : LEFT JOIN FETCH instr.instrumentPriceList instrPrice
#NamedQuery(name = "findAllPrices",
query = "SELECT DISTINCT taPat FROM TaPatternInstrument taPat "
+ "LEFT JOIN FETCH taPat.instrument instr "
+ "LEFT JOIN FETCH instr.instrumentPriceList instrPrice "
+ "LEFT JOIN taPat.taPatternInstrumentPriceList taPatpr "
+ "WHERE taPat.id = :taPatternInstrumentId "
+ "AND instrPrice.date BETWEEN :dateFrom AND :dateTo ")
The FETCH forces Hibernate to retrieve the Entity (InstrumentPrice) immediately at the first DB request. And thus the where clause is taken into account.
Without FETCH, the Entity InstrumentPrice is only retrieved from the DB when the method getInstrumentPriceList of the Entity Instrument is called (an additional call to the DB is performed). And with this additional call to the DB, the where clause is not taken into account anymore, thus retrieving all records from Entity instrumentPrice.

JPQL, get a list of objects which have a child list, avoid n+1 requests and only choose specific fields

I have the following entity Project:
#Data
#NoArgsConstructor
#Entity
#ToString(exclude = "roles")
#JsonInclude(JsonInclude.Include.NON_NULL)
public class Project {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
#Column(unique = true)
private String name;
private String description;
private Boolean isArchived;
private LocalDate archivedDate;
private LocalDate creationDate;
#Column(nullable = false, columnDefinition = "BOOLEAN DEFAULT FALSE")
private Boolean invoicingActivated;
#ManyToOne
#NotNull
private Order order;
#OneToOne(cascade = CascadeType.ALL)
private DefaultDailyEntrySettings defaultDailyEntrySettings;
#OneToMany(mappedBy = "project", cascade = CascadeType.ALL, orphanRemoval = true)
private List<ProjectEmployee> projectEmployees;
}
I want to get all projects. Each project should also have their list of projectEmployees.
Thats the entity ProjectEmployee:
#Data
#Table(uniqueConstraints = {#UniqueConstraint(columnNames = {"employee_id", "project_id"})})
#NoArgsConstructor
#ToString(exclude = "project")
#JsonInclude(JsonInclude.Include.NON_NULL)
public class ProjectEmployee {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private long id;
#ManyToOne
#JsonIgnore
#JsonProperty(access = JsonProperty.Access.WRITE_ONLY)
#NotNull
private Project project;
#ManyToOne
#NotNull
private Employee employee;
#ManyToOne
private ProjectEmployeeRole projectEmployeeRole;
}
To avoid n+1 queries, i wrote the following query:
#Query("SELECT project FROM Project project JOIN FETCH project.order ord JOIN FETCH ord.customer " +
"LEFT JOIN FETCH project.projectEmployees projectEmployee LEFT JOIN FETCH project.defaultDailyEntrySettings " +
"LEFT JOIN FETCH projectEmployee.employee LEFT JOIN FETCH projectEmployee.projectEmployeeRole")
List<Project> findAllProjectsInOneQuery();
This works, but it returns all properties of every single object. For example i´m only interested in the id and the name of ord.customer, i dont need all the other fields of ord.customer in this case. The problem with getting all the fields this way is that theres a lot of data being transmitted which i do not need in this case. To only select the ones i need and reduce the amount of data im sending over the internet i could do something like this:
#Query("SELECT new de.project.Project(project.id, project.name, " +
"project.description, project.isArchived, project.archivedDate, " +
"project.creationDate, project.invoicingActivated, project.order.id, " +
"project.order.name, project.order.customer.id, project.order.customer.name) " +
"FROM Project project")
List<Project> findAllMinimal();
But as you see, i can´t get project.projectEmployees that way, because it´s a list and i do not think im able to pass a list via the constructor this way.
I tried:
#Query("SELECT new de.project.Project(project.id, project.name, " +
"project.description, project.isArchived, project.archivedDate, " +
"project.creationDate, project.invoicingActivated, project.order.id, " +
"project.order.name, project.order.customer.id, project.order.customer.name, " +
"projectEmployee.id) " +
"FROM Project project JOIN project.projectEmployees projectEmployee")
List<Project> findAllMinimal();
But projectEmployee.id is just the id of the first projectEmployee, i do not think im able to pass all projectEmployees this way.
Is there any way to get all projects with their projectEmployees(and the other properties i listed in the query above) and specify which fields i would like to get? It does not have to be one query, a constant number of queries is fine. It should obviosuly just avoid n+1 queries.
Edit:
I came up with a workaround. Im using the following two queries:
#Query("SELECT new de.project.Project(project.id, project.name, " +
"project.description, project.isArchived, project.archivedDate, " +
"project.creationDate, project.invoicingActivated, project.order.id, " +
"project.order.name, project.order.customer.id, project.order.customer.name) " +
"FROM Project project")
List<Project> findAllMinimal();
#Query("SELECT DISTINCT new de.projectemployee.ProjectEmployee(projectEmployee.id, " +
"projectEmployee.employee.id, projectEmployee.employee.email, " +
"projectEmployee.employee.firstName, projectEmployee.employee.lastName, " +
"projectEmployee.employee.address, projectEmployee.employee.weeklyHoursEnabled, " +
"projectEmployee.employee.weeklyHours, projectEmployee.employee.isArchived, " +
"projectEmployee.employee.archivedDate, projectEmployee.project.id, projectEmployeeRole.id, " +
"projectEmployeeRole.name, projectEmployeeRole.hourlyWage) FROM ProjectEmployee projectEmployee " +
"LEFT JOIN projectEmployee.projectEmployeeRole projectEmployeeRole " +
"WHERE projectEmployee.project IN :projects")
List<ProjectEmployee> findByProjects(#Param("projects") List<Project> projects);
To give every project his projectEmployees i need some additional java code:
List<Project> projects = projectRepository.findAllMinimal();
List<ProjectEmployee> projectEmployees = projectEmployeeRepository.findByProjects(projects);
Map<Long, List<ProjectEmployee>> projectIdToProjectEmployeesMap = new HashMap<>();
for (ProjectEmployee projectEmployee : projectEmployees) {
List<ProjectEmployee> projectEmployeesToBeSaved = projectIdToProjectEmployeesMap.getOrDefault(projectEmployee.getProject().getId(), new ArrayList<>());
projectEmployeesToBeSaved.add(projectEmployee);
projectIdToProjectEmployeesMap.put(projectEmployee.getProject().getId(), projectEmployeesToBeSaved);
}
projects.forEach(project -> project.setProjectEmployees(projectIdToProjectEmployeesMap.get(project.getId())));
return projects;
So, ye as you see im able to achieve my goal of getting all projects with their projectEmployees in a constant number of queries(2) and only select the fields i need. The downside is that i have a javacode running with O(n) complexity. But i reduced the size of the data im transmitting by over 90%, so i guess its worth.
It´s hard to believe that a java code algorithm like the one i used is required to find a solution for my problem, so if anyone finds a better solution(with just sql queries) which is capable of doing the above stated, please share it.

Didn't verify it for your case, but at a first glance you may take advantage of the first level cache. After you perform
List<Project> projects = projectRepository.findAllMinimal();
List<ProjectEmployee> projectEmployees = projectEmployeeRepository.findByProjects(projects);
Any access to projectEmployee.getProject() should return its project from the cache avoiding n+1 problem. So if you can leave project.projectEmployees List unpopulated, you can drop your "custom javacode".

JPA #OneToMany: Returning only some values from map in a query

I'm trying to run a JPA query to return only specific fields from my entity, rather than the entire entity (for performance reasons).
Within this entity is this:
#OneToMany(cascade = { CascadeType.ALL }, mappedBy = "helper", fetch = FetchType.EAGER)
#MapKeyColumn(name = "year")
public Map<Integer, DutyHistory> getDutyHistoryList() {
return dutyHistoryList;
}
I'd like, within my query, to return multiple values from this map e.g. fields from the DutyHistory object for the last 3 years.
My question is, what's the query syntax for this? I'm mapping the returned values to a POJO as below:
#Query(value = "SELECT new com.castlemon.helpers.dto.ReportHelper(h.helperId, h.firstName, h.secondName"
+ ", h.sex, h.service, h.dateOfBirth, h.schoolGroup, h.orientationRequired, h.notes as adminNotes "
+ ", h.primaryDuty.dutyName as primaryDuty, h.secondDuty, h.secondaryDuty.dutyName as secondaryDuty "
+ " WHERE h.travelling = 1")
public List<ReportHelper> getTravellingHelperDetails();

You should create another query with your "year" parameter
#Query(value = "SELECT new com.castlemon.helpers.dto.ReportHelper(h.helperId, h.firstName, h.secondName"
+ ", h.sex, h.service, h.dateOfBirth, h.schoolGroup, h.orientationRequired, h.notes as adminNotes "
+ ", h.primaryDuty.dutyName as primaryDuty, h.secondDuty, h.secondaryDuty.dutyName as secondaryDuty "
+ " WHERE h.travelling = 1 AND h.year >= :yearLimit")
public List<ReportHelper> getTravellingHelperDetailsUntilYear(String yearLimit);

Hibernate, projection and eagerly fetching a collection

I have two Entities
#Entity
#Table(name = "steps")
public class Step {
#Id
#Column(nullable = false, insertable = true, updatable = true)
private long id;
//more fields
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "result_id", referencedColumnName = "id")
private Result result;
and
#Entity
#Table(name = "results")
public class Result {
#Id
#Column(nullable = false, insertable = true, updatable = true)
private long id;
//more fields
#OneToMany(mappedBy = "result", fetch = FetchType.EAGER)
private List<Step> steps;
Now I need to get all results and their associated steps.
I will want all the steps so an eager fetch seems best, and I know all results will have steps so an inner join will be fine.
I will want to get only some more fields from results, but not all so I can maximize the use of an index, so projection will be needed.
To accomplish this I have tried two methods:
Query query = getSession().createQuery(
"select distinct result from Result as result " +
"inner join result.steps " +
"where result.monitorId = :id " +
"and result.completed between :from and :to ");
query.setLong("id", monitorId);
query.setTimestamp("from", from);
query.setTimestamp("to", to);
for (Result result : query.list()) {
result.getSteps()
//work work
}
Hibernate does a join like I wanted but when I start to iterate over the results, Hibernate logs one more select query per step I'm using.
(I also haven't found a good way for projection if I'm going this route?)
The second approach I have tried seems great so far:
Criteria criteria = getSession().createCriteria(Result.class);
criteria.setProjection(Projections.projectionList()
.add(Projections.property("completed"), "completed")
.add(Projections.property("steps"), "steps"));
criteria.add(Restrictions.eq("someId", someId));
criteria.add(Restrictions.between("completed", from, to));
criteria.setFetchMode("steps", FetchMode.JOIN);
criteria.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY);
criteria.setResultTransformer(Transformers.aliasToBean(Result.class));
The projections work great as long as I don't include steps. I'm guessing that has to do with it not belonging to the actual database table? When I do try with steps I get
java.lang.ArrayIndexOutOfBoundsException: 2
on criteria.list();
So my main question is, how do I best do eager fetching of some but not all columns in results + a collection of steps belonging to each result?
Help please? :)