Sorting and paging with Hibernate's Criteria api comes with a big restriction, it needs to retrieve distinct results from the database. Tools provided by the api like DistinctRootTransformer won't work, because it is applied after retrieving the entities from db and thus breaks paging and sorting. The only way to get distinct results of a query with restrictions on an association is by limiting the resultset by a DetachedCriteria:
DetachedCriteria dc = DetachedCriteria.forClass(Household.class, "h")
.createAlias("cats", "c", JoinType.LEFT_OUTER_JOIN)
.add(Restrictions.or(
Restrictions.isEmpty("cats"),
Restrictions.ne("c.name", "Sylvester")
))
.setProjection(Projections.distinct(Projections.property("h.id")));
Criteria criteria = session.createCriteria(Household.class)
.add(Property.forName("id").in(dc));
...apply paging, sorting and filtering to criteria.
Does anybody know a better approach such as omitting subqueries and use joins without breaking pagination? My goal is to find a solution that is reusable, like passing only a criteria to another method that applies paging, sorting and filtering.
Update:
The following code does not work. Because of the join I have to use a Resulttransformer, to get distinct results. However, it is applied after sorting and paging.
Criteria criteria = session.createCriteria(Household.class)
.createAlias("cats", "c", JoinType.LEFT_OUTER_JOIN)
.add(Restrictions.ne("c.name","Sylvester"))
.setFirstResult((page - 1) * pagesize)
.setMaxResults(pagesize)
.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY);
E.g. debugging the sql the database would return something like that:
household_id=1,...cat_ids={1,2};household_id=1,...cat_ids={1,2};household_id=2,...cat_ids={1};
In this example, setting pagesize to 1 and viewing page 2 should return the uid 2, because there are only two distinct users. But as you can see in the database output, it returns the wrong uid 1, because Resulttransformers kicks in afterwards.
If you create DetachedCriteria in some method and then work with it, you can try to convert DetachedCriteria to Criteria using something like:
DetachedCriteria detachedCriteria = DetachedCriteria.forClass(Your.class);
// then add all your restrictions and convert
Criteria criteria = detachedCriteria.getExecutableCriteria(session);
And then apply you pagination
criteria.setFirstResult(pageable.getPage() * pageable.getSize());
criteria.setMaxResults(pageable.getSize());
And then execute criteria. For example:
List<T> result = criteria.list();
When I need to paginate, I never use DetachedCriteria, but instead normal Criteria, controlling first and max results.
From view I determine which page I need to show and once I have prepared the critera I configure this way:
criteria.setMaxResults(lazyQuery.getPageSize());
criteria.setFirstResult(layQuery.getStart());
lazyQuery is an object of my own model, used for view and business logic. This works perfect.
Related
I'm trying to optimize a complex operation involving all the Books and Videos in multiple Libraries (this is not the actual domain, for nondisclosure reasons).
The code originally used Criteria to load all the Libraries, then loaded the member Books lazily by iterating. I basically added a couple of FetchMode clauses:
List<Library> library = session
.createCriteria(Library.class)
.setFetchMode("books", FetchMode.JOIN)
.setFetchMode("videoShelves.videos", FetchMode.JOIN)
.list();
The second FetchMode clause appears to work, or at least doesn't result in obvious problems.
The first, though, blows out the number of Libraries from 6 to 248. So it looks to me as if maybe each Library is replicated once for each Book it has.
What are the conditions under which Hibernate might create unexpected duplicate instances when a FetchMode is added to the query?
The reason of this behaviour is an outer join, as #TimBiegeleisen suggested.
For more information refer
Hibernate does not return distinct results for a query with outer join fetching enabled for a collection (even if I use the distinct keyword)?
Possible solutions from the link provided above
Using Set
List<Library> library = ...;
return new ArrayList<Library>(new LinkedHashSet<Library>(library));
Using the Criteria.DISTINCT_ROOT_ENTITY result transformer
List<Library> library = session
.createCriteria(Library.class)
.setFetchMode("books", FetchMode.JOIN)
.setFetchMode("videoShelves.videos", FetchMode.JOIN).
.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY)
.list();
But, the best solution is not fetching all data in one request of course. For an example, if you want to show a list of libraries (with number of books, for an example) you can use projections to load only data you want to.
Using setFetchMode() will, by default, result in an outer join, which is what you are currently observing. If you want an INNER JOIN between the three tables then try this instead:
List<Library> library = session.createCriteria(Library.class, "library")
.createAlias("library.book", "books")
.createAlias("books.video", "videos").
.list();
I'm using GAE's datastore and JPA persistence framework to store my entities. Though when attempting to retreive some specific entities I run into the problem mentioned below.
The following exception is thrown when invoking the getResultList() method on my TypedQuery: javax.persistence.PersistenceException: Illegal argument
EntityManager em = Persistence.createEntityManagerFactory("test-persistence")
.createEntityManager();
String q = "SELECT c FROM c TestBord c WHERE c.publiclyAvailible=true
AND c.avarageRating='5'
AND c.user LIKE 'user%'
AND c.nameBord LIKE 'bord%'";
TypedQuery<TestBord> tq = em.createQuery(q, TestBord.class);
List<TestBord> l = tq.getResultList();
As also shown above, here is the query I'm using:
SELECT c FROM c KvCBord c WHERE c.publiclyAvailible=true
AND c.avarageRating='5'
AND c.user LIKE 'user%'
AND c.nameBord LIKE 'bord%'
It seems to break when I use two LIKE clauses, anybody have any ideas on how to work around this problem, or knows how to properly rewrite the query?
NOTE: Works fine with just one LIKE clause though.
AppEngine translates your GQL query into a low level Datastore API query. According to the Restrictions on queries Java docs, "Inequality filters are limited to at most one property". This is usually because of index selection. The LIKE operator becomes an inequality filter and cannot apply to both .user and .nameBord properties in the same query.
It doesn't work because App Engine has restrictions in its queries. Queries results come from indexes, and you can't have indexes that support 2 or more inequality filters.
I recommend using Search API for that searches. It's easy, practical, fast, and you can do more complex searches:
https://developers.google.com/appengine/docs/java/search/
This seems to be a well known problem for years as can be read here:
http://blog.xebia.com/2008/12/11/sorting-and-pagination-with-hibernate-criteria-how-it-can-go-wrong-with-joins/
And even finds reference in hibernate faqs:
https://community.jboss.org/wiki/HibernateFAQ-AdvancedProblems#Hibernate_does_not_return_distinct_results_for_a_query_with_outer_join_fetching_enabled_for_a_collection_even_if_I_use_the_distinct_keyword
This has also been discussed previously on SO
How to get distinct results in hibernate with joins and row-based limiting (paging)?
The problem is that even after going through all these resources, I have not been able to resolve my issue, which seems to be a little different from this standard problem, although I am not sure.
The standard solution proposed here involves creating two queries, first one for getting distinct IDs and then using those in a higher level query to get the desired pagination. The hibernate classes in my case are something like
A
- aId
- Set<B>
B
- bId
It appears to me that the subquery seems to be working fine for me and is being able to get the distinct aIds but the outer query which is supposed to do the pagination is again fetching the duplicates and thus the distinct in subquery is having no effect.
Assuming I have one A object which has a set of four B objects, My analysis is that because of introduction of set, while fetching data for
session.createCriteria(A.class).list();
hibernate is populating four references in the list pointing to just one object. Because of this the standard solution is failing for me.
Could someone please help in coming up with a solution for this case?
Edit: I have decided to go for doing pagination by ourselves from the distinct resultset. The other equally bad way could have been to lazy load the B objects but that would have required separate queries for all the A objects to fetch corresponding B objects
Consider using DistinctRootEntity result transformer like this
session.createCriteria(A.class)
.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY).list();
UPDATE
The samples of queries for one-to-many associations.
public Collection<Long> getIDsOfAs(int pageNumber, int pageSize) {
Session session = getCurrentSession();
Criteria criteria = session.createCriteria(A.class)
.setProjection(Projections.id())
.addOrder(Order.asc("id"));
if(pageNumber >= 0 && pageSize > 0) {
criteria.setMaxResults(pageSize);
criteria.setFirstResult(pageNumber * pageSize);
}
#SuppressWarnings("unchecked")
Collection<Long> ids = criteria.list();
return ids;
}
public Collection<A> getAs(int pageNumber, int pageSize) {
Collection<A> as = Collections.emptyList();
Collection<Long> ids = getIDsOfAs(pageNumber, pageSize);
if(!ids.isEmpty()) {
Session session = getCurrentSession();
Criteria criteria = session.createCriteria(A.class)
.add(Restrictions.in("id", ids))
.addOrder(Order.asc("id"))
.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY);
#SuppressWarnings("unchecked")
as = criteria.list();
}
return as;
}
You mention the reason you're seeing this problem is because Set<B> is fetched eagerly. If you're paginating, chances are you don't need the B's for each A, so it might be better to fetch them lazily.
However, this same problem occurs when you join the B's into the query to make a selection.
In some cases, you will not only want to paginate, but also sort on other fields than the ID. I think the way to do this is to formulate the query like this:
Criteria filter = session.createCriteria(A.class)
.add(... any criteria you want to filter on, including aliases etc ...);
filter.setProjection(Projections.id());
Criteria paginate = session.createCriteria(A.class)
.add(Subqueries.in("id", filter))
.addOrder(Order.desc("foo"))
.setMaxResults(max)
.setFirstResult(first);
return paginate.list();
(pseudocode, didn't check if the syntax is exactly right but you get the idea)
I answered this here: Pagination with Hibernate Criteria and DISTINCT_ROOT_ENTITY
You need to do 3 things, 1) get the total count, 2) get the ids of the rows you want, and then 3) get your data for the ids found in step 2. It is really not all that bad once you get the order correct, and you can even create a generic method and send it a detached criteria object to make it more abstract.
I used groupBy property to achieve this. Hope it works.
Criteria filter = session.createCriteria(A.class);
filter.setProjection(Projections.groupProperty("aId"));
//filter.add(Restrictions.eq()); add restrictions if any
filter.setFirstResult(pageNum*pageSize).setMaxResults(pageSize).addOrder(Order.desc("aId"));
Criteria criteria = session.createCriteria(A.class);
criteria.add(Restrictions.in("aId",filter.list())).addOrder(Order.desc("aId"));
return criteria.list();
Are there any good utils/frameworks which could generate Java Object for SQL Query?
QueryDsl automatically creates Query Objects from your Hibernate, JPA or JDO classes, but also from your DB schema.
Querying with Querydsl SQL is as
simple as this :
QCustomer customer = new QCustomer("c");
SQLTemplates dialect = new HSQLDBTemplates(); // SQL-dialect
SQLQuery query = new SQLQueryImpl(connection, dialect);
List<String> lastNames = query.from(customer)
.where(customer.firstName.eq("Bob"))
.list(customer.lastName);
It also supports subqueries:
To create a subquery you create a
SQLSubQuery instance, define the query
parameters via from, where etc and use
unique or list to create a subquery,
which is just a type-safe Querydsl
expression for the query. unique is
used for a unique (single) result and
list for a list result.
query.from(customer).where(
customer.status.eq(
new SQLSubQuery().from(customer2).unique(customer2.status.max()))
.list(customer.all())
Another example
query.from(customer).where(
customer.status.in(new SQLSubQuery().from(status).where(
status.level.lt(3)).list(status.id))
.list(customer.all())
I don't know its gonna be enough helpful but, as you asked for utils, I would suggest you to read about the QUERY OBJECT PATTERN (P of EAA, M. Fowler), if you have time to implement something, its a good beginning, otherwise you may lookfor any ORM framework.
I am using torque to do that. There is an example(Tutorial) which show what it can do at http://db.apache.org/torque/releases/torque-3.3/tutorial/step5.html
But what exactly is it you want? Do you just a simple way to serialize/unserialize objects to the database, and load them based on a primary/foreign key, or do you need to issue really complicated queries?
So, I have a rather complex query I am trying to make using the Hibernate Criteria API. I have the following entity classes:
Code
Gift
GiftVendor
GiftVendorStatus
with the following relationships:
Code 1<>1 Gift
Gift 1<>* GiftVendor
GiftVendor 1<>1 GiftVendorStatus
I need to build a Criteria query that returns a List of Code objects, but that restricts it to only Codes that have a Gift that have at least one GiftVendor with a GiftVendorStatus of Online. Here is the code I am using to build the criteria:
Criteria base = CodeDao.getBaseCriteria();
base.createAlias("gift","gift");
base.createAlias("gift.giftVendor","giftVendor");
base.createAlias("giftVendor.giftVendorStatus","giftVendorStatus");
base.add(Restrictions.like("giftVendorStatus.description", "Online%"));
return base.list();
This gives me a List of Code objects, restricted as I would expect. However, it also does additional queries to build out all of the unused relationships of the Gift object, even though I have all of the mappings set up with a fetch mode of Lazy. This results in 4 additional, separate queries for each of my 10000+ results.
I have code to do the query using HQL that works as expected:
String hql = "select c FROM Code c inner join c.gift g inner join g.giftVendors gv inner join gv.giftVendorStatus gvs" +
" WHERE gvs.description like :desc";
HashMap<String,Object> params = new HashMap<String, Object>();
params.put("desc", "Online%");
return performQuery(hql, params);
That code gives me a List of Code objects, as expected, without doing all of the extra queries to populate the Gift object. How do I tell Hibernate not to do those extra queries with the Criteria API?
UPDATE: The problem here is not the retrieval of the Gift table, but rather unrelated one-to-one relationships from the Gift table. For example, Gift has a one-to-one relationship to GiftCommentAggregateCache. This table is in no way related to this particular query, so I would expect lazy initialization rules to apply, and the query to GiftCommentAggregateCache to not occur unless a read is attempted. However, with the Criteria query written out as above, it makes that separate query to populate the model object for GiftCommentAggregateCache.
If I use:
base.setFetchMode("gift.giftCommentAggregateCache", FetchMode.JOIN);
then I do not have any problems. However, that means that in order for this to work as I would expect, I need to add that line for every single unused one-to-one relationship that Gift has. Any ideas as to why the Lazy rules specified in the mappings are not coming into play here?
There are a few different things I have tried:
base.setFetchMode("gift", FetchMode.LAZY); // Still does additional queries
and
base.setFetchMode("gift", FetchMode.SELECT); // Still does additional queries
and
base.setResultTransformer(Criteria.DISTINCT_ROOT_ENTITY); // Still does additional queries
and
base.setResultTransformer(Criteria.ROOT_ENTITY); // Still does additional queries
and
base.setResultTransformer(Criteria.ALIAS_TO_ENTITY_MAP); // Still does additional queries
and
base.setProjection(Projections.property("gift")); // Does not do additional queries, but incorrectly returns a List of Gift objects, instead of a List of Code objects
From the Hibernate documentation:
Join fetching: Hibernate retrieves the associated instance or collection in the same SELECT, using an OUTER JOIN.
In other words, try using base.setFetchMode("gift", FetchMode.JOIN)
I hope that helps.
Cheers