Develop Reference

problems with OneToMany including a filter clause in spring jpa

I currently get unexpected results in my MYSQL8/H2 test-case when using on a #OneToMany relationship in spring jpa. I want to filter in a list of TKBColumn-tables inside my TKBData table using JPQL. I expect to get one TKBData-table with the filtered TKBColumn but I always get the TKBData-table with ALL TKBColumn (unfiltered). When I using a SQL command it works!
I got no Idea whats the problem here, why it always give me the TKBData-table with always ALL TKBColumn-tables inside.
Native Query (This works):
SELECT d.id,c.name FROM TKBDATA d LEFT JOIN TKBDATA_TKBCOLUMN dc ON d.ID = dc.TKBDATA_ID LEFT JOIN TKBCOLUMN c ON c.ID = dc.COLUMNS_ID WHERE c.name = 'column1';
Output
ID NAME
7b6ec910-3e53-40a3-9221-ee60e75c8d67 column1
JPQL Query (Not works):
select d from TKBData d LEFT JOIN d.columns c WHERE c.name = :name
Output:
id: e892bc28-c35f-4fc8-9b09-387f97a758d8, name:column1
id: 069cc76b-3487-4ad8-a4ae-6568694e2287, name:column2
Table 'TKBData'
public class TKBData {
#Id
#Builder.Default
private String id = UUID.randomUUID().toString();
...
#OneToMany(fetch = FetchType.EAGER, cascade = CascadeType.ALL, orphanRemoval = true)
#Builder.Default
private Set<TKBColumn> columns = Sets.newHashSet();
...
}
Table 'TKBColumn'
public class TKBColumn {
#Id
#Builder.Default
private String id = UUID.randomUUID().toString();
...
}
Spring Data Repository
#Service
public interface KBDataRepository extends CrudRepository<TKBData, String>, KBDataCustomRepository {
#Query("select d from TKBData d LEFT JOIN d.columns c WHERE c.name = :name")
public TKBData filterByColumn(#Param("name") String name);
}
Spring JPA Generated H2 Tables (relevant)
CREATE CACHED TABLE "PUBLIC"."TKBCOLUMN"(
"ID" VARCHAR(255) NOT NULL,
"NAME" VARCHAR(255),
...
)
CREATE CACHED TABLE "PUBLIC"."TKBDATA_TKBCOLUMN"(
"TKBDATA_ID" VARCHAR(255) NOT NULL,
"COLUMNS_ID" VARCHAR(255) NOT NULL
)
CREATE CACHED TABLE "PUBLIC"."TKBDATA"(
"ID" VARCHAR(255) NOT NULL,
...
)
Relevant Content of tables which are generated at the start of the test class
Table: TKBDATA
ID
726004cf-5cab-4b1d-bb3f-466ba22622e9
Table: TKBDATA_TKBCOLUMN
TKBDATA_ID COLUMNS_ID
726004cf-5cab-4b1d-bb3f-466ba22622e9 7b4e4ea8-4ff9-4668-8882-67ff93b595ca
726004cf-5cab-4b1d-bb3f-466ba22622e9 d670e813-0466-48a8-be54-ee992cf28462
Table: TKBCOLUMN
ID DATAORDER NAME OWNERID
d670e813-0466-48a8-be54-ee992cf28462 0 column1 16e01046-9a84-4651-98d8-4e3e358212eb
7b4e4ea8-4ff9-4668-8882-67ff93b595ca 1 column2 16e01046-9a84-4651-98d8-4e3e358212eb
For more informations you can find the github repository here: https://github.com/fo0/ScrumTool
Test class: https://github.com/fo0/ScrumTool/blob/master/ScrumTool/src/test/java/com/fo0/vaadin/scrumtool/test/data/TKBDataColumnFilterTest.java
Edit:
The solution for this was to use a native query, because of the design of JPA and how it works with objects, thats why my use-case has exactly this problem.

Meaning of select d from TKBData d JOIN d.columns c WHERE c.name = column1 is
Find a TKBData object where it has an associated column object for which name is column1
Once its decided which TKBData has at least one column object for which name is column1, then it will return all its associated column objects which you don't have control over in JPA. ( see My answer to another question ). Alternative is to write native sql and return custom non entity objects
For example, you have TKBDATA_1 with column1 and column2 associated, you also have TKBDATA_2 with column3 associated.
When you run your query, it will ignore TKBDATA_2 and decides to return TKBDATA_1 as it has atleast one column object with name= column2. But after that you don't have control over which associated column objects to return for TKBDATA_1 and JPA will return all associated column objects
If you are not sure of the reason, read about hibernate session.How it provides unique presentation of any associated entry in memory. It is the foundation for its dirty checking and repeatable read
Update your #OneToMany as follows
#OneToMany(fetch = FetchType.EAGER,
cascade = CascadeType.ALL, orphanRemoval = true)
#Builder.Default
#JoinTable(name = "TKBDATA_TKBCOLUMN",
joinColumns = #JoinColumn(name = "TKBDATA_ID"),
inverseJoinColumns = #JoinColumn(name = "COLUMNS_ID"))
private Set<TKBColumn> columns = Sets.newHashSet();
When it comes to JPA query language, I would like to think in terms of query a collection of in-memory objects.
So now try to describe the meaning of the following two queries in terms of objects.
select d from TKBData d LEFT JOIN d.columns c WHERE c.name = :name
vs
select d from TKBData d JOIN d.columns c WHERE c.name = :name
Don't forget unlike in sql where you are select any columns here you have said you want to select TKBData objects and restricting which TKBData objects to return.
So to achieve the same result as of your native sql, use the second JPA query
Note:
Even though you used a left join in your sql query, it is effectively an inner join sql query because you also applied a where condition to the most right table on that join.

Use the DISTINCT JPQL keyword
#Query("select distinct d from TKBData d LEFT JOIN d.columns c WHERE c.name = :name")
public TKBData filterByColumn(#Param("name") String name);
Or use JPA method naming query
public TKBData findByColumnsName(String name);

Problems mapping Hibernate entities - native query containing left join with condition

This should be straight-forward though can't get my Hibernate entities to play nice for the following scenario with a simple two table structure:
I'm attempting to get all config names and matching config values for a given currency code (and null's where not matching).. so have written a native query to retrieve the following like so:
SELECT * FROM CONFIG_NAME LEFT JOIN CONFIG_VALUE ON CONFIG_NAME.ID =
CONFIG_VALUE.CONFIG_ID AND CONFIG_VALUE.CURRENCY_CODE = '<CURRENCY_CODE>'
ORDER BY CONFIG_NAME.ID
This query doesn't seem to play nice with my Hibernate mapping as it appears to be essentially ignoring the CURRENCY_CODE clause in the join.
Essentially, for the following subset of data:
CONFIG_NAME:
CONFIG_VALUE:
There is no value defined for 'FREE_SHIPPING_ENABLED' for 'USD' so running the query above for both currency code returns as expected:
QUERY RESULTS FOR 'CAD':
QUERY RESULTS FOR 'USD':
I'm running the above query as a native query in a JpaRepository for the ConfigName entity. But what I appear to be getting is that it seems to ignore the currency_code clause in the JOIN condition. As the list of config values defined has both values for USD and CAD where they're populated. Is there an Hibernate annotation to factor this in that I'm unaware of?
It's worth bearing in mind there will only ever be ONE value defined for each config for a given currency - there's a unique constraint across CONFIG_VALUE.CONFIG_ID/CONFIG_VALUE.CURRENCY_CODE so potentially ConfigValue on the ConfigName entity would not need to be a map.
Mappings as are follows:
ConfigName - Entity
#OneToMany(mappedBy = "config")
private Set<ConfigValue> configValue;
ConfigValue - Entity
#ManyToOne(optional = false)
#JoinColumn(name="CONFIG_ID")
#Property(policy=PojomaticPolicy.NONE)
private ConfigName config;
Doesn't need to be strictly unidirectional either.. as I'm only concerned with the values from the ConfigName entity either being populated or null.
Think I'm missing something simple, so hope someone can help.
EDIT: Am querying using JpaRepository:
Am using JpaRepository to query:
#Repository
public interface ConfigNameRepository extends JpaRepository<ConfigName, Long>
{
static final String SQL_QUERY = "SELECT * FROM CONFIG_NAME "
+ "LEFT JOIN CONFIG_VALUE ON CONFIG_NAME.ID = CONFIG_VALUE.CONFIG_ID "
+ "AND CONFIG_VALUE.CURRENCY_CODE = ?1 ORDER BY CONFIG_NAME.ID";
#Query(value = SQL_QUERY, nativeQuery = true)
List<ConfigName> findConfigValuesByCurrencyCode(final String currencyCode);
}

As mentioned by #Ouney, your JPA relations are not taken in account if you use a native query.
You declared a SELECT * and List<ConfigName> (the real sql result contains ConfigName+ConfigValue). So with this query, Hibernate fetchs all the ConfigName. Then, when you try to access to the set of configValue, it fetchs all the related ConfigValue.
I think this should be better/easier to use a JPQL query instead (but you need Hibernate 5.1+) :
SELECT n, v
FROM ConfigName n
LEFT JOIN ConfigValue v
ON v.config = n AND v.currencyCode = :currencyCode
ORDER BY n.id
With this method signature :
List<Object[]> findConfigValuesByCurrencyCode(#Param("currencyCode") String currencyCode);
Where the result will be :
o[0] // ConfigName
o[1] // ConfigValue (nullable)
You may want to do this prettier with a wrapper :
SELECT new my.package.MyWrapper(n, v)
...
MyWrapper constructor :
public MyWrapper(ConfigName configName, ConfigValue configValue) {
...
}
Method signature with the wrapper :
List<MyWrapper> findConfigValuesByCurrencyCode(#Param("currencyCode") String currencyCode);
(update)
I think in this case, your query can be :
SELECT n, v // or new my.package.MyWrapper(n, v)
FROM ConfigName n
LEFT JOIN n.configValue v
WITH v.currencyCode = :currencyCode
ORDER BY n.id

Join optional column with JPA and hibernate

I am working with hibernate through JPA (backend for testing is h2, but the same issue happens on other engines) and have encountered a problem when joining optional columns and filtering on them.
I have the following data model:
#Entity
public class Ticket {
#Id
private long id;
#ManyToOne(optional = true)
#Nullable
private Assignee assignee;
}
#Entity
public class Assignee {
#Id
private long id;
private String name;
}
And three entities:
Assignee{id = 1, name = kitty}
Ticket{id = 1, assignee = null}
Ticket{id = 2, assignee = 1}
Now, I am querying tickets with jpql:
select t from Ticket t yields both tickets, as expected.
select t from Ticket t where t.assignee is null yields ticket 1 only, as expected.
select t from Ticket t where t.assignee.name = :name with name=kitty yields ticket 2 only, as expected.
However, linking the two filters together in an OR clause does not behave as expected: select t from Ticket t where (t.assignee is null or t.assignee.name = :name) with name=kitty only yields ticket 2, while the query should match ticket 1 as well (because assignee may be null). When checking the hibernate debug log, the following SQL query is generated:
SELECT
ticket0_.id AS id1_1_,
ticket0_.assignee_id AS assignee2_1_
FROM Ticket ticket0_ CROSS JOIN Assignee assignee1_
WHERE ticket0_.assignee_id = assignee1_.id AND (ticket0_.assignee_id IS NULL OR assignee1_.name = ?)
The condition ticket0_.assignee_id = assignee1_.id is obviously never satisfied for ticket 1 since it has no assignee, so hibernate translated this query incorrectly.
Is there any way for me to fix this?

On your SELECT statement, you indicated this expression t.assignee.name. Though you have not explicitly used a JOIN operation in your SELECT statement, traversing from the Ticket entity to the Assignee entity to get the name property will require a NATURAL JOIN between the 2 entities. Thus, you will see a ticket0_.assignee_id = assignee1_.id in your output SQL.
You can rewrite you query:
SELECT t from Ticket t WHERE (t.assignee IS NULL) OR (t.assignee IS NOT NULL AND t.assignee.name = :name)
Or try using an OUTER JOIN instead:
SELECT t FROM Ticket t LEFT JOIN Assignee a WHERE a.name = :name

JPA / EclipseLink: Joined WHERE query does not give expected result

Now that's very confusing... I have a JPA entity Order that references an entity User. The User can be either buyer or seller of the Order.
Because both buyer and seller can enter additional information for an order, I moved that to an extra entity OrderUserData. There might or might not be a corresponding OrderUserData object, but IF one exists, the user should only be able to see the entry they created (based on USER_ID) and not the one of the other party.
The entities look like this:
#Entity
#Table(name = "T_ORDER")
public class Order {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#ManyToOne
#JoinColumn(name = "SELLER_ID")
private User seller;
#ManyToOne
#JoinColumn(name = "BUYER_ID")
private User buyer;
#OneToMany(mappedBy = "order", fetch = FetchType.EAGER)
private List<OrderUserData> userData = new ArrayList<>();
//..
}
--
#Entity
#Table(name = "T_ORDER_USERDATA")
public class OrderUserData {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#ManyToOne
#JoinColumn(name = "ORDER_ID")
private Order order;
#ManyToOne
#JoinColumn(name = "USER_ID")
private User user;
private String comment;
//...
}
( User is not very exciting, just ID and basic name fields )
Now when I'm trying to select the appropriate data to display in the website, I have a problem:
String qry = "SELECT o FROM Order o LEFT JOIN o.userData ud "
+ " WHERE (o.seller.id = :userId OR o.buyer.id = :userId)"
+ " AND ( ud.user IS NULL OR ud.user.id = :userId )";
TypedQuery<Order> query = em.createQuery(qry, Order.class);
query.setParameter("userId", userId);
Let's say I execute this, setting userId to 2:
My Database looks like this:
ORDER
=====
ID SELLER_ID BUYER_ID
1 1 2
2 2 3
3 3 1
ORDER_USERDATA
===============
ID ORDER_ID USER_ID COMMENT
1 1 1 Comment that only user 1 should see
2 1 2 Comment that only user 2 should see
But unlike you would expect, when executing the above query, both records are included in the userData list! It seems like JPA is executing two queries (despite the EAGER fetch) and ignoring the WHERE on the second one. Why is that? And what other solution than to loop through the userData list on Java level and kick out the entry that the appropriate user should not see?

There is no way to load OrderUserData objects inside an Order object using a query. Maybe you're confusing the ORM functionality, mapping rows in the database to Java objects, with the query functionality.
Mapping means 1-1 correspondence between rows and objects, hence Order objects always contain all OrderUserData objects for each OrderUserData row related to Order rows.
The fetch type is just a loading strategy, determining at which time are the related objects fetched, as soon as the containing object is loaded (EAGER) or as soon as the contained objects are accessed (LAZY).
You can obtain your list issuing a query on OrderUserData objects with the proper filters and getting Order objects from each of them, i.e.
SELECT ud FROM OrderUserData ud WHERE (ud.order.seller.id = :userId
OR ud.order.buyer.id = :userId) AND ( ud.user IS NULL OR ud.user.id =
:userId )

your query seems to work well as it selects properly Order entity. Then JPA fetch all the OrderUserData child of the selected Order : that's because oneToMany join is not filtered.
I don't think it is possible to modelize pre-filtered oneToMany with eclipseLink (like Hibernate #FILTER), so you should remove it and map orderUserDataId field only. Then you can fetch your entities in 1 query, but they will not be linked
SELECT o, ud FROM Order o, o.userData ud WHERE (o.seller.id = :userId OR o.buyer.id = :userId) AND ( ud.orderUserDataId = o.id and (ud.user IS NULL OR ud.user.id = :userId) )";
On the other hand, if the oneToMany is required by other use cases, then you can create 2 different Order entities :
1 "OrderLight" without the oneToMany
1 "OrderFull" with the oneToMany, derived from OrderLight.

While user3580357 and remigio have already given the correct answer as to why this doesn't work, might I suggest that you create a view on database level.
Something like (might need to be adapted for your needs or RDBMS):
CREATE OR REPLACE VIEW
ORDER_WITH_USERDATA
AS
SELECT o.*, oud.*
FROM ORDER o
LEFT JOIN ORDER_USERDATA oud
ON o.id = oud.order_id
This will essentially give you two different "logical" records for every order. You can then create an additional JPA entity that works on this view and do your SELECT/WHERE... without needing to (LEFT)JOIN at all.

problem with "select new Object ... join ... where"

I'm having a problem with an HQL query
Three classes
ClassOne is my BusinessObject
public class ClassOne {
private int id;
private int status;
private Set<ClassTwo> classTwos;
+ other fields/getters/setters/constructor etc
}
ClassTwo is referenced in a set of ClassOne and is kind of the history of an object of ClassOne
public class ClassTwo {
private int id;
private int oldStatus;
private int newStatus;
private String message;
//+ getters/setters/constructor etc
}
ClassThree is my DTO/VO with just one classTwo (not the whole history)
public class ClassThree {
private int id;
private int status;
private ClassTwo classTwo;
public ClassThree(int pId, int pStatus, ClassTwo pClassTwo) {
id=pId;
status=pStatus;
classTwo=pClassTwo;
}
//+ getters/setters etc
}
Now I'd like to create an HQL query like this:
I'd like to get all objects of ClassThree with a certain status and if it exists the newest ClassTwo with a certain newStatus.
For example:
I'd like to get all the DTOs (ClassThree) of ClassOne whose status is now 1, but earlier in their history it has been 2 and I'd like to have the latest ClassTwo object which has 2 as newStatus.
SELECT new ClassThree(c1.id, c1.status, c2)
FROM ClassOne c1
LEFT JOIN c1.classtwos c2 (...)
and (...) is where I don't know what to do, I'm not even sure if it's a join / join fetch
Looked around and tried quite a lot already, but no clue. Especially with the join fetch I get some Hibernate errors like org.hibernate.QueryException: query specified join fetching, but the owner of the fetched association was not present in the select list.
Fetching the BusinessObject like that is no problem
SELECT distinct(c1)
FROM ClassOne c1
LEFT OUTER JOIN FETCH c1.classtwos c2
and I get the ClassTwos as my field.
Thanks in advance,
Jacob
P.S.: One thing might be important, ClassTwo has no reference to ClassOne!!
P.P.S : The simple SQL query which resolves my problem looks more or less like that:
select * from classone as c1 left join (select * from classtwo where newstatus = 2) c2 on c1.id=c2.id_classone whete c1.status = 1
This query works and gets all the information needed on my PostGreSQL DB, but I'd really like to have an HQL to continue to work with, especially for maintenance reasons and so on...
Update with workaround solution:
Getting the ids of all the ClassOnes with a status 1
Collection<Integer> ids = null;
ids = (Collection<Integer>) getHibernateTemplate().execute(
new HibernateCallback() {
public Object doInHibernate(Session pSession) throws HibernateException, SQLException {
return getDocumentIds(pSession, pStatus);
}
}
);
Now I get all the DTOs which have been in status 2 (thanks to Ivan) with:
Named query Document.dto.with.transfer
SELECT new DocumentDTO(d.id, d.status, histo)
FROM Document d
LEFT JOIN d.histories histo
WHERE
d.id in (:ids)
AND
(histo.id =
SELECT MAX(innerhisto.id)
FROM Document innerd
JOIN innerd.histories innerhisto
WHERE d.id = innerd.id AND innerhisto.newStatus = 21)
(in my code I use some named queries)
List<DocumentDTO> lRes = new ArrayList<DocumentDTO>();
Query lQuery = getSession(false).getNamedQuery("Document.dto.with.transfer");
lQuery.setParameterList("ids", ids);
lResultList.addAll(lQuery.list());
afterwards I remove all the IDs already found from my list ids
for (DocumentDTO dto : lResultList) {
ids.remove(dto .getId());
}
I do a third query using a second constructor for the DTO, initializing my history with a dummy-object.
Named query Document.dto.simple
SELECT new DocumentDTO(d.id, d.status)
FROM Document d
WHERE
d.id in (:ids)
(another named query)
lQuery = getSession(false).getNamedQuery("Document.dto.simple");
lQuery.setParameterList("ids", ids);
lResultList.addAll(lQuery.list());
and it's done.

To include Documents with no history we should use LEFT JOIN and test for empty collection, then we use subquery (SELECT COUNT(...)) to detect all documents that have never been in status 2. The last OR-clause is for fetching the last history with the specified status.
Here is the HQL query:
SELECT new DocumentDto(doc.id, doc.status, hist)
FROM Document doc
LEFT JOIN doc.histories hist
WHERE doc.status = :docStatus
AND (size(doc.histories) = 0
OR (SELECT COUNT(innerhist)
FROM Document innerdoc JOIN innerdoc.histories innerhist
WHERE innerdoc.id=doc.id AND innerhist.newStatus = :historyStatus) = 0
OR (hist.newStatus = :historyStatus AND hist.id =
(SELECT max(innerhist.id)
FROM Document innerdoc
JOIN innerdoc.histories innerhist
WHERE innerdoc.status = :docStatus AND innerhist.newStatus = :historyStatus))
Then call setParameter("historyStatus", 2) and setParameter("docStatus", 1) on your query to get the correct result.
That's it!
Please note, I've made an assumption, that we can use a value of id attribute of History as an indicator of the order in which objects were put in your database.