I have a JPA select where you receibe a parameter then we can search using some attributes (username, email, identifier) The user only have a text field to write the criteria text.
The problem is perfomance, in the database We have about 9Millions of users registereds, and the search is too slow, using JPA,
Form:
- Value (Input text)
User sends the value in the form (He doesn't say if he is using username, email or identifier)
User (Table) Fields:
- Identifier
- Name
- Email
JPA Query:
select u from UserEntity u where u.alias LIKE lower(:query) OR u.email LIKE lower(:query) OR lower(u.identifier) LIKE lower(:query) ORDER BY u.alias
I don't know what the best method to improve the speed of the search, (We have some indexes in the table in these fields), if we remove the lower in the u.identifier field the speed improves a lot (almost instant). But we can have the identifier in a lot of ways (migrations, registers, manual client inserts..)
We have about 9 Millions of users registered, and the search is too slow, using JPA
Please note the search will be too slow regardless the technology involved in executing the query just because the SQL query itself is too slow. That being said the problem is not JPA but how to improve the SQL query.
Combining LOWER() function with LIKE operator adds a lot of overhead because the RDBMS must apply a text function to 3 fields before analyse the LIKE match.
IMHO a good approach is using a VIEW to leverage the LOWER() part on the 3 fields and then execute the query over this view (BTW mapping a View with JPA is just as simple as mapping a Table):
CRAETE VIEW user_view AS SELECT id, lower(identifier) AS identifier, lower(alias) AS alias, lower(email) AS email FROM user;
Then create an entity for searching:
#Entity
#Table("user_view")
public class UserView {
#Basic private Long id;
#Basic private String identifier;
#Basic private String alias;
#Basic private String email;
// getters and setters as required
}
And finally the JPQL query:
String jqpl = "SELECT u FROM UserView u WHERE u.alias LIKE :query OR u.email LIKE :query OR u.identifier LIKE :query";
Note that you can pass query parameter directly in lower case as a Query parameter and you can order the result list after the query execution. Both will result in the RDBMS making less effort and consequently improving the overall response time.
You can solve it using different techniques.
One is using a collate that ignores case. For example in MySQL https://dev.mysql.com/doc/refman/5.7/en/case-sensitivity.html
Another strategy is to have a derived field with an index. In DB2, for example, you can create a field 'lower_identifier generated always as lower(identifier)' and an index in the field and it will use it automatically when doing a lower search. You don't have to map the field in JPA.
Related
Is there opportunity to filter query results by complex custom criteria that appears to be only a java function? I would like this criteria functions take a part between entity creation and placing it into result collection.
For instance, I have following entity and query
#Entity
#NamedQueries{
#NamedQuery(name="myquery",query="...")
}
class MyEntity{
#Id
public long id;
#Column(name="NAME")
public String name;
#Column(name="description")
public String description;
}
I can execute myquery and specify paging parameters to get result set with fixed size. But I want to make some additional complex filtering, that can not be expressed by query. If I make some post query processing function and use it to filter query results I would break paging invariant, page size. It would not be nice and convenience.
There's no simple way to do that. The setFirstResult() and setMaxResults() methods are applied on the generated SQL query. In MySQL, for example, LIMIT (10, 10) is added to the where clause of the query). So if you filter the results after the query has been executed, you'll always get a smaller number of results.
NamedQueries is the wrong approach to achieve this.
You need to build the query on the fly.
EntityManager em=....
Query queryObject = em.createQuery(queryString);
or
Query queryObject = em.createNativeQuery(queryString);
I'm just getting to grips with JPA in a simple Java web app running on Glassfish 3 (Persistence provider is EclipseLink). So far, I'm really liking it (bugs in netbeans/glassfish interaction aside) but there's a thing that I want to be able to do that I'm not sure how to do.
I've got an entity class (Article) that's mapped to a database table (article). I'm trying to do a query on the database that returns a calculated column, but I can't figure out how to set up a property of the Article class so that the property gets filled by the column value when I call the query.
If I do a regular "select id,title,body from article" query, I get a list of Article objects fine, with the id, title and body properties filled. This works fine.
However, if I do the below:
Query q = em.createNativeQuery("select id,title,shorttitle,datestamp,body,true as published, ts_headline(body,q,'ShortWord=0') as headline, type from articles,to_tsquery('english',?) as q where idxfti ## q order by ts_rank(idxfti,q) desc",Article.class);
(this is a fulltext search using tsearch2 on Postgres - it's a db-specific function, so I'm using a NativeQuery)
You can see I'm fetching a calculated column, called headline. How do I add a headline property to my Article class so that it gets populated by this query?
So far, I've tried setting it to be #Transient, but that just ends up with it being null all the time.
There are probably no good ways to do it, only manually:
Object[] r = (Object[]) em.createNativeQuery(
"select id,title,shorttitle,datestamp,body,true as published, ts_headline(body,q,'ShortWord=0') as headline, type from articles,to_tsquery('english',?) as q where idxfti ## q order by ts_rank(idxfti,q) desc","ArticleWithHeadline")
.setParameter(...).getSingleResult();
Article a = (Article) r[0];
a.setHeadline((String) r[1]);
-
#Entity
#SqlResultSetMapping(
name = "ArticleWithHeadline",
entities = #EntityResult(entityClass = Article.class),
columns = #ColumnResult(name = "HEADLINE"))
public class Article {
#Transient
private String headline;
...
}
AFAIK, JPA doesn't offer standardized support for calculated attributes. With Hibernate, one would use a Formula but EclipseLink doesn't have a direct equivalent. James Sutherland made some suggestions in Re: Virtual columns (#Formula of Hibernate) though:
There is no direct equivalent (please
log an enhancement), but depending on
what you want to do, there are ways to
accomplish the same thing.
EclipseLink defines a
TransformationMapping which can map a
computed value from multiple field
values, or access the database.
You can override the SQL for any CRUD
operation for a class using its
descriptor's DescriptorQueryManager.
You could define a VIEW on your
database that performs the function
and map your Entity to the view
instead of the table.
You can also perform minor
translations using Converters or
property get/set methods.
Also have a look at the enhancement request that has a solution using a DescriptorEventListener in the comments.
All this is non standard JPA of course.
I have this class mapped
#Entity
#Table(name = "USERS")
public class User {
private long id;
private String userName;
}
and I make a query:
Query query = session.createQuery("select id, userName, count(userName) from User order by count(userName) desc");
return query.list();
How can I access the values returned by the query?
I mean, how should I treat the query.list()? As a User or what?
To strictly answer your question, queries that specify a property of a class in the select clause (and optionally call aggregate functions) return "scalar" results i.e. a Object[] (or a List<Object[]>). See 10.4.1.3. Scalar results.
But your current query doesn't work. You'll need something like this:
select u.userName, count(u.userName)
from User2633514 u
group by u.userName
order by count(u.userName) desc
I'm not sure how Hibernate handles aggregates and counts, but I'm not sure if your query is going to work at all. You're trying to select a aggregate (i.e. the "count(userName)"), but you don't have a "group by" clause for userName.
If the query does in fact work, and Hibernate can figure out what to do with it, the results you get back will most likely be a raw Object[], because Hibernate will not be able to map your "count(userName)" data into any field on your mapped objects.
Overall, when you get into using aggregates in queries, Hibernate can get a little more tricky, since you're no longer mapping tables/columns directly into classes/fields. It might be a good idea to read up more on how to do aggregates in Hibernate, from their documentation.
I have project use EJB 3.0 and implement Toplink framework for model layer.
When using EJBQL to process data, I see it seems have some limitation:
It cannot process datatime such as find a part of date such as day, month or year
It cannot find datetime among from...to
It cannot comparison datetime field
It cannot map a class not entity to a customize native select query because I want to get List data from SELECT statement but when I query in case join 2 or more table and map the object output into a class but impossible
#PersistenceContext private
EntityManager em;
em.createNativeQuery("SELECT
a.usertype , b.username, b.userpass
FROM tablea a, tableb b WHERE a.id =
b.id,MyClass.class).getResultList
.....
class MyClass(){
String usertype;
String username;
String userpass;
}
Could you help me any ideas?
Thank in advance!
It can not, do it in your code. Otherwise, you need to use something database specific on one side of your condition.
It can, why not. You can use between :fromDate and :toDate in the query, or use > :fromDate and < :toDate, in the NamedQuery. Where is the problem.
It can. Similar to the last one, use = sign instead
It can using #SqlResultSetMapping. Refer to this.
I have two classes, Person and Company, derived from another class Contact. They are represented as polymorphically in two tables (Person and Company). The simplified classes look like this:
public abstract class Contact {
Integer id;
public abstract String getDisplayName();
}
public class Person extends Contact {
String firstName;
String lastName;
public String getDisplayName() {
return firstName + " " + lastName;
}
}
public class Company extends Contact {
String name;
public String getDisplayName() {
return name;
}
}
The problem is that I need to make a query finding all contacts with displayName containing a certain string. I can't make the query using displayName because it is not part of either table. Any ideas on how to do this query?
Because you do the concatenation in the Java class, there is no way that Hibernate can really help you with this one, sorry. It can simply not see what you are doing in this method, since it is in fact not related to persistence at all.
The solution depends on how you mapped the inheritance of these classes:
If it is table-per-hierarchy you can use this approach: Write a SQL where clause for a criteria query, and then use a case statement:
s.createCriteria(Contact.class)
.add(Restrictions.sqlRestriction("? = case when type='Person' then firstName || ' '|| lastName else name end"))
.list();
If it is table per-concrete-subclass, then you are better of writing two queries (since that is what Hibernate will do anyway).
You could create a new column in the Contact table containing the respective displayName, which you could fill via a Hibernate Interceptor so it would always contain the right string automatically.
The alternative would be having two queries, one for the Person and one for the Company table, each containing the respective search logic. You may have to use native queries to achieve looking for a concatenated string via a LIKE query (I'm not a HQL expert, though, it may well be possible).
If you have large tables, you should alternatively think about full-text indexing, as LIKE '%...%' queries require a full table scan unless your database supports full text indexes.
If you change displayName to be a mapped property (set to the name column in Company and to a formula like first||' '||last in Person), then can query for Contract and Hibernate will run two queries both of which now have a displayName. You will get back a List of two Lists, one containing Companies and one containing Persons so you'll have to merge them back together. I think you need to query by the full package name of Contract or set up a typedef to tell Hibernate about it.