What seems almost natural in simple SQL is impossible in mongodb.
Given a simple document:
{
"total_units" : 100,
"purchased_unit" : 60
}
I would like to query the collection, using spring data Criteria class, where "total_units > purchased_units".
To my understanding it should be as trivial as any other condition.
Found nothing to support this on Spring api.
You can use the following pattern:
Criteria criteria = new Criteria() {
#Override
public DBObject getCriteriaObject() {
DBObject obj = new BasicDBObject();
obj.put("$where", "this.total_units > this.purchased_units");
return obj;
}
};
Query query = Query.query(criteria);
I don't think Spring Data API supports this yet but you may need to wrap the $where query in your Java native DbObject. Note, your query performance will be fairly compromised since it evaluates Javascript code on every record so combine with indexed queries if you can.
Native Mongodb query:
db.collection.find({ "$where": "this.total_units > this.purchased_units" });
Native Java query:
DBObject obj = new BasicDBObject();
obj.put( "$where", "this.total_units > this.purchased_units");
Some considerations you have to look at when using $where:
Do not use global variables.
$where evaluates JavaScript and cannot take advantage of indexes.
Therefore, query performance improves when you express your query
using the standard MongoDB operators (e.g., $gt, $in). In general, you
should use $where only when you can’t express your query using another
operator. If you must use $where, try to include at least one other
standard query operator to filter the result set. Using $where alone
requires a table scan. Using normal non-$where query statements
provides the following performance advantages:
MongoDB will evaluate non-$where components of query before $where
statements. If the non-$where statements match no documents, MongoDB
will not perform any query evaluation using $where. The non-$where
query statements may use an index.
As far as I know you can't do
query.addCriteria(Criteria.where("total_units").gt("purchased_units"));
but would go with your suggestion to create an additional computed field say computed_units that is the difference between total_units and purchased_units which you can then query as:
Query query = new Query();
query.addCriteria(Criteria.where("computed_units").gt(0));
mongoOperation.find(query, CustomClass.class);
Thanks #Andrew Onischenko for the historic good answer.
On more recent version of spring-data-mongodb (ex. 2.1.9.RELEASE), I had to write the same pattern like below:
import org.bson.Document;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.Query;
// (...)
Criteria criteria = new Criteria() {
#Override
public Document getCriteriaObject() {
Document doc = new Document();
doc.put("$where", "this.total_units > this.purchased_units");
return doc;
}
};
Query query = Query.query(criteria);
One way is this:
Criteria c = Criteria.where("total_units").gt("$purchased_unit");
AggregationOperation matchOperation = Aggregation.match(c);
Aggregation aggregation = Aggregation.newAggregation(matchOperation);
mongoTemplate.aggregate(aggregation, "collectionNameInStringOnly", ReturnTypeEntity.class);
Remember to put collection name in string so as to match the spellings of fields mentioned in criteria with fields in database collection.
Related
I got a warning from Atlas Mongo saying:
Do not use the $regex operator when using a case-insensitive index for
your query. The $regex implementation is not collation-aware and
cannot utilize case-insensitive indexes. Instead, we recommend Atlas
Search queries that use the $search aggregation pipeline stage.
Source: https://www.mongodb.com/docs/atlas/schema-suggestions/case-insensitive-regex/
In my Java code i'm using this:
import org.springframework.data.mongodb.core.query.Criteria;
Criteria criteria = new Criteria();
...
criteria.and("search").regex(data.getText(), "i"); // i means case-insensitive
...
return Query.query(criteria);
Obviously this is slow performance as I get a warning. How to I apply the collation?
What is the best performance approach for this so I remove the i option from the regex?
The best practice to query data with case-insensitive option is using text index and search operator instead of regex
Create index-case-insensitive for fields you want to query:
TextIndexDefinition textIndex = new TextIndexDefinitionBuilder()
.onField("title", 2F)
.onField("content")
.build();
MongoTemplate template = … // obtain MongoTemplate
template.indexOps(CookingRecipe.class).ensureIndex(textIndex);
Then use text index to query data:
TextCriteria criteria = TextCriteria.forDefaultLanguage()
.matchingAny("coffee", "cake");
Query query = TextQuery.queryText(criteria)
.sortByScore()
.with(new PageRequest(0, 5)); //top 5 documents matching “coffee” or “cake”
Kindly refer $text $search your Documents with Spring Data MongoDB for more information
I have a query that returns a Postgres array of UUIDs:
SELECT e.id, e.date,
ARRAY
(
SELECT cs.api_id FROM event_condition_set ecs
JOIN condition_set cs on cs.id = ecs.condition_set_id
WHERE ecs.event_id = e.id
) AS condition_set_ids,
...
And then create and run this query as a native query: Query query = entityManager.createNativeQuery(queryString);
Since Hibernate can normally not deal with these Postgres arrays, I use Vlad's Hibernate Types library.
However, currently I need to register this UUIDArrayType globally in my application:
public class PostgreSQL95CustomDialect extends PostgreSQL95Dialect {
public PostgreSQL95CustomDialect() {
super();
this.registerHibernateType(Types.ARRAY, UUIDArrayType.class.getName());
}
}
Aside from the fact this is a bit ugly, it also leaves no room for other types of arrays.
(Note I also tried registering a generic ListArrayType but this throws a NullPointerException during execution of the query.)
I have also tried registering it as a scalar type:
query.unwrap(org.hibernate.query.NativeQuery.class)
.addScalar("condition_set_ids", UUIDArrayType.INSTANCE);
But this makes the entire query only return a single UUID, which is very strange and seems bugged.
Is there a way to ONLY use this UUIDArrayType specifically in this query?
(Please don't suggest using array_agg because the performance is terrible for this case)
you can call native queries using custom Hibernate types as follows:
String myJsonbData = ...;
String[] myStringArr = ...;
final String queryStr = "select your_function(?, ?, ...)"; // ? for each param
entityManager
.createNativeQuery(queryStr)
.setParameter(1, new TypedParameterValue(JsonBinaryType.INSTANCE, myJsonbData))
.setParameter(2, new TypedParameterValue(StringArrayType.INSTANCE, myStringArr));
This is just an example, but as a rule of thumb, you need to instantiate a new TypedParameterValue.
Answering my own question here. After waiting for a while and updating to the most recent library version (I'm on 2.19.2 right now) I don't have any issues anymore with the scalar types registration as I mentioned in my question, i.e.:
query.unwrap(org.hibernate.query.NativeQuery.class)
.addScalar("condition_set_ids", UUIDArrayType.INSTANCE);
So it appears to just have been a bug and I can now avoid the global registration in favor of using scalars.
Currently I am using java to connect to MONGODB,
I want to write this sql query in mongodb using java driver:
select * from tableA where name like("%ab%")
is their any solution to perform the same task through java,
the query in mongodb is very simple i know, the query is
db.collection.find({name:/ab/})
but how to perform same task in java
Current I am using pattern matching to perform the task and code is
DBObject A = QueryBuilder.start("name").is(Pattern.compile("ab",
Pattern.CASE_INSENSITIVE)).get();
but it makes query very slow I think , does a solution exist that does not use pattern matching?
Can use Regular Expressions. Take a look at the following:
http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-RegularExpressions
Make sure you understand the potential performance impacts!
DBObject A = QueryBuilder.start("name").is(Pattern.compile("ab",
Pattern.CASE_INSENSITIVE)).get();
I think this is one of the possible solution, you need to create index to achieve those.
Why do you fear the regular expressions? Once the expression is compiled they are very fast, and if the expression is "ab" the result is similar to a function that search a substring in a string.
However to do what you need you have 2 possibilities:
The first one, using regular expression, as you mention in your question. And I believe this is the best solution.
The second one, using the $where queries.
With $where queries you can specify expression like these
db.foo.find({"$where" : "this.x + this.y == 10"})
db.foo.find({"$where" : "function() { return this.x + this.y == 10; }"})
and so you can use the JavaScript .indexOf() on string fields.
Code snippet using the $regex clause (as mentioned by mikeycgto)
String searchString = "ab";
DBCollection coll = db.getCollection("yourCollection");
query.put("name",
new BasicDBObject("$regex", String.format(".*((?i)%s).*", searchString)) );
DBCursor cur = coll.find(query);
while (cur.hasNext()) {
DBObject dbObj = cur.next();
// your code to read the DBObject ..
}
As long as you are not opening and closing the connection per method call, the query should be fast.
I have an object that was stored via mongo-java-driver. Object uses java.util.UUID for its _id field. Following is presentation of object via mongo shell:
> db.b.find()
{ "_id" : BinData(3,"zUOYY2AE8WZqigtb/Tqztw==") }
I have a requirement to process searching via $where clause. I use following code to do it:
Mongo m = new Mongo();
DBCollection coll = m.getDB("a").getCollection("b");
coll.save(new BasicDBObject("_id", UUID.randomUUID()));
// ??? - don't know what should be specified
DBObject query = new BasicDBObject("$where", "this[\"_id\"] == " + ???);
coll.find(query).count()
The question is what should I specify instead of ??? to make it work?
Thanks for any help.
My invesigation shown that only one way to do it is rewriting a query in object based way (I mean migration of $where clause part to BasicDBObject based query). In such case mongo-java-driver supports java.util.UUID without any additional effort.
I found some hint in Toplink
Query query = em.createQuery("SELECT e FROM Employee e ORDER BY e.lastName ASC, e.firstName ASC");
query.setHint("eclipselink.cursor.scrollable", true);
ScrollableCursor scrollableCursor = (ScrollableCursor)query.getSingleResult();
List<Employee> emps = scrollableCursor.next(10);
is there are jpa/hibernate alternative?
To my knowledge, there is nothing standard in JPA for that.
With Hibernate, the closest alternative I'm aware of would be the Query / ScrollableResults APIs. From the documentation:
10.4.1.6. Scrollable iteration
If your JDBC driver supports
scrollable ResultSets, the Query
interface can be used to obtain a
ScrollableResults object that allows
flexible navigation of the query
results.
Query q = sess.createQuery("select cat.name, cat from DomesticCat cat " +
"order by cat.name");
ScrollableResults cats = q.scroll();
if ( cats.first() ) {
// find the first name on each page of an alphabetical list of cats by name
firstNamesOfPages = new ArrayList();
do {
String name = cats.getString(0);
firstNamesOfPages.add(name);
}
while ( cats.scroll(PAGE_SIZE) );
// Now get the first page of cats
pageOfCats = new ArrayList();
cats.beforeFirst();
int i=0;
while( ( PAGE_SIZE > i++ ) && cats.next() ) pageOfCats.add( cats.get(1) );
}
cats.close()
Note that an open database connection
and cursor is required for this
functionality. Use
setMaxResult()/setFirstResult() if you
need offline pagination functionality.
Judging from the other answers JPA does not support scrolling directly, but if you use Hibernate as JPA implementation you can do
javax.persistence.Query query = entityManager.createQuery("select foo from bar");
org.hibernate.Query hquery = query.unwrap(org.hibernate.Query);
ScrollableResults results = hquery.scroll(ScrollMode.FORWARD_ONLY);
That accesses the underlying Hibernate api for the scrolling but you can use all the features of JPA querying. (At least for criteria queries the JPA api has some features that are not in the old Hibernate api.)
When processing large number of entities in a large project code based on List<E> instances,
I has to write a really limited List implementation with only Iterator support to browse a ScrollableResults without refactoring all services implementations and method prototypes using List<E>.
This implementation is available in my IterableListScrollableResults.java Gist
It also regularly flushes Hibernate entities from session. Here is a way to use it, for instance when exporting all non archived entities from DB as a text file with a for loop:
Criteria criteria = getCurrentSession().createCriteria(LargeVolumeEntity.class);
criteria.add(Restrictions.eq("archived", Boolean.FALSE));
criteria.setReadOnly(true);
criteria.setCacheable(false);
List<E> result = new IterableListScrollableResults<E>(getCurrentSession(),
criteria.scroll(ScrollMode.FORWARD_ONLY));
for(E entity : result) {
dumpEntity(file, entity);
}
With the hope it may help
In JPA you can use query.setFirstResult and query.setMaxResults
Also using Spring Data would be an option. There you can specify the query and pass, as a parameter, a "PageRequest" in which you indicate the page size and the page number:
Page<User> users = repository.findAll(new PageRequest(1, 20));
For this you need to extend a PagingAndSortingRepository.
Just as another alternative for paging over the results.
Of course, underneath, it's using Hibernate, Toplink or whatever JPA implementation you configure.