I am having a bit of trouble with projecting and paging with mongooperations. I always get empty result. The query criteria working fine without aggregation. I tryed the same code without paging(skip&limit) and without sort but I still get the empty result.
My code:
public List<ProfileBasic> findAllActiveUsersByGenderAndAgeBetweenProjectedPage(Gender gender, int fromAge, int toAge, int pageSize, int page) {
Criteria criteria = Criteria.where("gender").is(gender).and("confirmed").is(true)
.and("dateOfBirth").lte(LocalDate.now().minusYears(fromAge))
.gte(LocalDate.now().minusYears(toAge));
MatchOperation match = Aggregation.match(criteria);
ProjectionOperation project = Aggregation.project()
.and("id").as("id")
.and("name").as("name")
.and("lastName").as("lastName")
.and("gender").as("gender")
.and("dateOfBirth").as("dateOfBirth")
.and("lastVisit").as("lastVisit");
SkipOperation skip = new SkipOperation(pageSize*(page-1));
LimitOperation limit = new LimitOperation(pageSize);
SortOperation sort = new SortOperation(new Sort(Sort.Direction.DESC, "lastVisit"));
Aggregation aggregate = Aggregation.newAggregation(project, match, skip, limit, sort);
return operations.aggregate(aggregate, User.class, ProfileBasic.class).getMappedResults();
}
I would appreciate any help.
When we do Aggregation in MongoDB data is processed in stages and the output of one stage is provided as input to the next stage.
In my code first stage was project and not match(query):
Aggregation aggregate = Aggregation.newAggregation(project, match, skip, limit, sort);
Thats why it didn't work.
I changed it to:
Aggregation aggregate = Aggregation.newAggregation(match, skip, limit, project, sort);
And now it works fine.
Related
I'm writing spring boot application. I have the HashTree structure, sorted by specific field. And I have a specific method that iterates through the tree, so if it finds a value (used as comparator) it returns it and does not check for other values.
With code it will look something like this:
class Storage {
TreeSet<Book> tree = new TreeSet<>(Comparator.comparingInt(Book::getNumberOfPages));
public Book getBookWithSmallestNumberOfPagesButGreaterThanSpecific(int number) {
iterator = tree.iterator();
while (iterator.hasNext()) {
Book book = iterator.next();
if (book.getNumberOfPages() > number) {
return book;
}
}
return null;
}
}
Please don't jurge that code, it's just for example, real code is much more complicated.
So I want to do the same thing using Spring JPA, and store Books in database.
My plan is:
create index on numberOfPages column in order to iterate it in asc order
get a coursor to the first database record
return current element or get next element in a cursor.
The thing is I dont know how to get a cursor, all JPA methods return all elements in a List<>.
Please challenge my solution. I don't know if I move in a right direction. I'm pretty new to JPA.
You can use Pageable,
Pageable pageable = PageRequest.of(page, size, direction, property);
List<ENTITY> findByProjectId(String projectId, Pageable pageable);
The official Spring documentation provides the following example (slightly simplified here) for grouping results from a Solr query:
Field field = new SimpleField("popularity");
Query query = new SimpleQuery("inStock:true");
SimpleQuery groupQuery = new SimpleQuery(new SimpleStringCriteria("*:*"));
GroupOptions groupOptions = new GroupOptions()
.addGroupByField(field)
.addGroupByQuery(query);
groupQuery.setGroupOptions(groupOptions);
GroupPage<Product> page = solrTemplate.queryForGroupPage("collection-1", query, Product.class);
However, when I try this, it ignores the search conditions (inStock:true) and just performs the grouping on all results ("*:*")
There are several things about this example I had to modify to get it working. First of all, as you may have noticed, the argument query should actually be groupQuery in the queryForGroupPage call. I also removed the Query and put the actual search conditions in groupQuery. The working version is as follows, where groupByField is a SimpleField:
SimpleQuery groupQuery = new SimpleQuery(conditions);
GroupOptions groupOptions = new GroupOptions()
.addGroupByField(groupByField);
groupQuery.setGroupOptions(groupOptions);
return solrTemplate.queryForGroupPage(groupQuery, YourObjectHere.class);
Im trying to retrieve a document with a specific '_id' and a single embedded document with another specific '_id'.
my document is represent a catalog and it contains an array of courses.
example data:
'_id': ObjectId('1111'),
'name': 'example catalog',
...
...
'courses': [
{
'_id': ObjectId('2222'),
'name': 'my course',
...
},
{
....
}
In mongod I run this aggregation query, and get back what I wish for:
db.getCollection('catalogs').aggregate(
{ $match: { '_id': ObjectId('58e8da206ca4f710bab6ef74') } },
{ $unwind: '$courses' },
{ $match: { 'courses._id': ObjectId('58d65541495c851c1703c57f') } })
As I mentioned earlier, I've get back I single catalog instance with a single course instance within.
In my java repo, I was trying to do the same:
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.match(Criteria.where(Catalog.ID_FIELD).is(catalogId)),
Aggregation.unwind(Catalog.COURSES_FIELD, true),
Aggregation.match(Criteria.where(Catalog.COURSES_FIELD + '.' + Course.ID_FIELD).is(embeddedCourseId))
);
AggregationResults<Catalog> results = mongoTemplate.aggregate(aggregation,
Catalog.class, Catalog.class);
List<Catalog> catalog = results.getMappedResults();
But unfortunately, I've got an instance of my 'example catalog' with empty array of courses.
While debugging, I've found that inside results, there are two props that returns back.
first one is what I've used, called mappedResults (represents the converted object returning from mongoDB) - contains an empty array of courses.
the other one is the rawResults, (represents the data as DBObject) - contains the specific course I query for
my Catalog class contains an ArrayList (if that make any difference).
Please help and let me know what should I do to convert the results properly, or if I did something wrong in my code.
You can try below options. The key is to preserve the structure when mapping the response.
Regular Queries:
Using $positional projection
Query query = new Query();
query.addCriteria(Criteria.where("id").is(new ObjectId("58e8da206ca4f710bab6ef74")).and("courses.id").is(new ObjectId("58d65541495c851c1703c57f")));
query.fields().include("name").position("courses", 1);
List<Course> courses = mongoTemplate.find(query, Course.class);
Using $elemMatch projection
Query query = new Query();
query.addCriteria(Criteria.where("id").is(new ObjectId("58e8da206ca4f710bab6ef74")));
query.fields().include("name").elemMatch("courses", Criteria.where("_id").is(new ObjectId("58d65541495c851c1703c57f") ) );
List<Course> Course = mongoTemplate.find(query, Course.class);
Aggregation
Mongo Version >= 3.4 & Spring 1.5.2 Boot / Spring 1.10.1 Mongo.
You can use $addFields stage which will overwrite the courses field with the $filter value while keeping all the existing properties. I couldn't find any addFields builder in current spring version. So I have to use AggregationOperation to create a new one.
AggregationOperation addFields = new AggregationOperation() {
#Override
public DBObject toDBObject(AggregationOperationContext aggregationOperationContext) {
DBObject dbObject =
new BasicDBObject("courses",
new BasicDBObject("$filter",
new BasicDBObject("input", "$$courses").
append("as", "course").
append("cond",
new BasicDBObject("$eq", Arrays.<Object>asList("$$course._id", new ObjectId("58d65541495c851c1703c57f"))))));
return new BasicDBObject("$addFields", dbObject);
}
};
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.match(Criteria.where("_id").is(new ObjectId("58e8da206ca4f710bab6ef74"))),
addFields
);
Mongo Version = 3.2 & Spring 1.5.2 Boot / Spring 1.10.1 Mongo..
The idea is still same as above but this pipeline uses $project so you'll have to add all the fields that you want to keep in final response. Also used spring helper methods to create the $filter pipeline.
Aggregation aggregation = newAggregation(
Aggregation.match(Criteria.where("id").is(new ObjectId("58e8da206ca4f710bab6ef74"))),
Aggregation.project("name")
.and(ArrayOperators.Filter.filter("courses").as("course")
.by(ComparisonOperators.Eq.valueOf("course._id").equalToValue(new ObjectId("58d65541495c851c1703c57f")))
).as("courses")
);
Mongo Version <= 2.6
You'll have to use $unwind and add a course field to have spring map it correctly.
The problem that you have here is that your Catalog class has a courses field which maps to a List/ArrayList. But when your aggregation query unwinds the courses array, it is going to output the courses field as a sub-document. The Spring mapper doesn't know how to deal with that because it doesn't match your Catalog object structure.
You haven't fully defined your problem here, but what would probably make more sense is if you had the aggregation return a Course object rather than a Catalog object. In order to do that you're going to need to add a projection stage to your aggregation pipeline so that the result looks exactly like a single Course object. The key is that the data coming back from MongoDB needs to match your object structure.
I want to execute a query in java where path and _id are two fields of the mongo document.
I want to get results list where these two fields are equal in the document.
I have tried using the following query.But could not retrieve the results properly.Received empty list which is not the case.
List<Metadata> MetadataList= ops.find(new Query(Criteria.where("path").is("_id")), Metadata.class);
How to get results where two field values are equal in mongo.
What you are looking for is the $where operator in MongoDB. Standard query operations do not compare the values of one field against another. In order to do this, you need to employ the JavaScript evaluation server side which can actually compare the two field values:
BasicQuery query = new BasicQuery(
new BasicDBObject("$where", "return this._id == this.path")
);
<Metadata> MetadataList = ops.find(query, Metadata.class);
Or you can do the same thing with native operators through the $redact pipeline stage available to the aggregation framework.
Pretty sure there is no $redact support in spring mongo as yet, but you can wrap the aggregation operation with a class to do so:
public class CustomAggregationOperation implements AggregationOperation {
private DBObject operation;
public CustomAggregattionOperation (DBObject operation) {
this.operation = operation;
}
#Override
public DBObject toDBObject(AggregationOperationContext context) {
return context.getMappedObject(operation);
}
}
And use it like this:
Aggregation aggregation = newAggregation(
new CustomAggregationOperation(
new BasicDBObject(
"$redact",
new BasicDBObject("$cond",
new BasicDBObject()
.append("if", new BasicDBObject(
"$eq", Arrays.asList("$_id", "$path")
))
.append("then", "$$KEEP")
.append("else", "$$PRUNE")
)
)
)
);
AggregationResults<Metadata> results = ops.aggregate(
(TypedAggregation<Metadata>) aggregation, Metadata.class);
So basic MongoDB query operations do not compare field values against each other. To do this you need to follow one of the methods here.
You can use BasicDBObject to add condition.
Try something
BasicDBObject query = new BasicDBObject("path", new BasicDBObject("$eq", "_id");
collection.find(query);
Please refer the below link for more information
http://mongodb.github.io/mongo-java-driver/2.13/getting-started/quick-tour/
I am trying to limit the document size returned by my query.i want lets say 10 documents back only,any my query normally displays 22,how would i go buy setting a limit for the returned output. i am aware i can just limit the list size by creating a list and adding to that list however i want to do it on the query level.
My Query: Thanks in advance :)
ueryBuilder raceGenderQuery = QueryBuilders.boolQuery()
.must(termQuery("lep_etg_desc", "indian"))
.must(termQuery("lep_gen_desc", "male"));
Set<String> suburbanLocationSet = new HashSet<String>();
suburbanLocationSet.add("queensburgh");
suburbanLocationSet.add("umhlanga");
suburbanLocationSet.add("tongaat");
suburbanLocationSet.add("phoenix");
suburbanLocationSet.add("shallcross");
suburbanLocationSet.add("balito");
//Build the necessary location query.
QueryBuilder locationQuery = QueryBuilders.boolQuery().must(termsQuery("lep_suburb_home", suburbanLocationSet));
//Combine all Queries so that its filtered to get exact results.
FilteredQueryBuilder finalSearchQuery = QueryBuilders.filteredQuery(QueryBuilders.boolQuery().must(raceGenderQuery).must(locationQuery), FilterBuilders.boolFilter().must(FilterBuilders.rangeFilter("lep_age").gte(25).lte(45)).must(FilterBuilders.rangeFilter("lep_max_income").gte(25000).lte(45000)));
//Run Query through elasticsearch iterating through documents in the traceps index for query matches.
List<Leads> finalLeadsList = new ArrayList<Leads>();
for (Leads leads : this.leadsRepository.search(finalSearchQuery)) {
finalLeadsList.add(leads);
}
I think this is what you want:
SearchResponse response = client.prepareSearch().setSearchType(SearchType.QUERY_THEN_FETCH).setSize(10).setQuery(finalSearchQuery).execute().get
You have to use QUERY_THEN_FETCH for it to return exactly size results because otherwise it gets size results from each shard.