So I came up with a query that I'm not really sure how to translate into Spring Data. This is the query:
db.collection.aggregate([{
{
$group: {
_id: "$field",
count: {
$sum: "$count"
},
data: {
"$addToSet": "$$ROOT"
}
}
},
{
$sort: {
count: -1
}
},
{
$limit: 10
},
{
$unwind: "$data"
}
])
The problematic part is unwind. In this raw query I'm putting the original document("$$ROOT") that is being processed into data, so after the set is being processed I can just unwind it and end up with just the documents that I really want. This works fine from the shell. The problem is that I'm not seeing how to do the same operation in Java/Kotlin, I can just add to the set without having a way to reference that data later. Could someone help me out and write how this would look in Java/Kotlin with Spring Data?
You can try this
public List<Object> test() {
Aggregation aggregation = Aggregation.newAggregation(
group("field")
.sum("count").as("count")
.addToSet("$$ROOT").as("data"),
sort(Sort.Direction.DESC, "count"),
limit(10),
unwind("data")
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(YOUR_COLLECTION.class), Object.class).getMappedResults();
}
Related
This is my query on Elasticsearch,
GET index101/_search
{
"query": {
"function_score": {
"boost_mode": "replace",
"query": {
"constant_score": {
"filter": {
"terms": {
"fields": ["767","434", "101", "222"]
}
}
}
},
"pqcode_score": {
"descriptors": [
{
"descriptor": "base64string"
}
],
"pqparams": {
"bucket_field": "fields",
"pqcode_field": "fields2",
"distance_function": "similarity",
"model": "random"
}
}
}
}
}
Looked into the documentation of Elasticsearch with Java, but couldn't find anything that could resolve this query in Java.
I created a JSON file, and got the input query in the jsonObject and then passed it as a parameter to searchSourceBuilder.query().
But it gives the error that the jsonObject can't be converted into QueryBuilder.
How can we go ahead with this query in Java?
Is there any other workaround for this?
looks like pqcode_score is your custom key in the Elasticsearch query, and you are trying to add the custom component/constructs in your Elasticsearch query thats not possible, hence you are getting the error.
You need to use the constructs in your Elasticsearch query thats supported by Elasticsearch.
I have a requirement of fetching data from mongodb and being done using Java Spring Reactive Mongodb library. I am using the following code for this:
Criteria criteria = Criteria.where(QUERYFIELD1).in(listOfIds)
.andOperator(Criteria.where(QUERYFIELD2).gte(ChildDate));
Query query = Query.query(criteria).noCursorTimeout();
reactiveMongoTemplate.find(query, Document.class, COLLECTIONNAME);
Where QUERYFIELD1 is "ChildId" and QUERYFIELD2 is a "ChildDate". Following is the structure of my document:
{
"_id": {
"$oid": "6296968fa63757a93e1cd123"
},
"Level1": {
"Level2": [
{
"ChildId": "1234",
"ChildDate": {
"$date": "2021-04-01T04:00:00.000Z"
}
},
{
"ChildId": "5678",
"ChildDate": {
"$date": "2017-05-16T04:00:00.000Z"
}
},
{
"ChildId": "3456",
"ChildDate": {
"$date": "2008-09-16T04:00:00.000Z"
}
},
{
"ChildDate": {
"$date": "2022-06-01T04:00:00.000Z"
},
"ChildId": "7891"
}
]
}
}
I am trying to find a document which should match the criteria within the Objects under Level2. For e.g. if My criteria has ChildId as "3456" and ChildDate as "2022-06-01T04:00:00.000Z" then I should get empty results as ChildId is matching with Object3 and ChildDate is matching with Object4. But when I use below query, I get 1 record as the match:
{ "Level1.Level2.ChildId" : "3456", "Level1.Level2.ChildDate" : { $gt: new Date("2022-01-01T05:00:00.000+00:00")}}
I am trying to achieve this using Spring Reactive MongoDB. Please help.
You can use $elemMatch for finding the documents that their array includes an item which matches the conditions:
db.collection.find({
"Level1.Level2": {
$elemMatch: {
ChildId: "3456",
ChildDate: {$gt: new Date("2008-09-16T05:00:00.000Z")}
}
}
})
See how it works on the playground example
Looking at com.mongodb.reactivestreams.client.MongoCollection interface we see that aggregation can be invoked using list of Bson elements.
public interface MongoCollection<TDocument> {
...
AggregatePublisher<TDocument> aggregate(List<? extends Bson> list);
}
It is clear how to use it when aggregation steps are JSONs (see Example with simple JSONs)
Unfortunately, when any aggregation step contains a function (which is allowed by native MongoDB query), for instance, $accumulator the same approach cannot be applied due to it causes violation of Bson format (org.bson.json.JsonParseException) (see Example with functions)
What is the best way to convert a native MongoDB aggregation query into a result in Java?
(suppose that queries are complex and it is not expedient to rewrite them with Mongo aggregation builders in Java)
Example with simple JSONs:
ReactiveMongoOperations mongo = /* ... */;
var match = BasicDBObject.parse("{ $match: {name: \"Jack\"} }");
var project = BasicDBObject.parse("{ $project: {_id: 0, age: 1, name: 1} }";
var queryParts = List.of(match, project);
Flux<PersonInfo> infoFlux = mongo
.getCollection("person")
.flatMapMany(person -> person.aggregate(queryParts).toFlux())
.map(it -> objectMapper.readValue(it.toJson(), PersonInfo.class))
.collectList()
Example with functions:
// here for conciseness it is just a counting accumulator; generally functions are more complex
var match = BasicDBObject.parse("""
{
$group: {
_id: "$token",
count: {
$accumulator: {
init: function() {
return {owned: 0, total: 0}
},
accumulate: function(state, owner) {
return {
total: state.total + 1
}
},
accumulateArgs: ["$owner"],
merge: function(a, b) {
return {
total: a.total + b.total
}
},
lang: "js"
}
},
minPriceEth: {$min: "$priceEth"}
}
}
""");
I have a query like this (simplified):
db.collection.aggregate([
{ $match: { main_id: ObjectId("58f0f67f50c6af16709fd2c7") } },
{
$group: {
_id: "$name",
count: { $sum: 1 },
sum: { $sum: { $add: ["$P31", "$P32"] } }
}
}
])
I do this query from Java, and I want to map it on my class, but I don't want _id to be mapped on name field. Because if I do something like this:
#JsonProperty("_id")
private String name;
then when I save this data back to mongo (after some modification) the data is saved with name as _id while I want a real Id to be generated.
So, how can I rename _id after $group operation?
You can achieve this by adding a $project stage at the end of your pipeline like this :
{ $project: {
_id: 0,
name: "$_id",
count: 1,
sum: 1
}
}
try it online: mongoplayground.net/p/QpVyh-0I-bP
From mongo v3.4 you could use $addFields in conjunction with $project to avoid to write all the fields in $project that could be very tedious.
This happen in $project because if you include specifically a field, the other fields will be automatically excluded.
Example:
{
$addFields: { my_new_id_name: "$_id" }
},
{
$project: { _id: 0 }
}
db.report.aggregate(
{
$group: {_id: '$name'}
},
{
$project:{
name:"$_id",
_id:false} }
)
Starting in Mongo 4.2, you can use a combination of $set / $unset stages:
// { x: 1, z: "a" }
// { x: 2, z: "b" }
db.collection.aggregate([
{ $set: { y: "$x" } },
{ $unset: "x" }
])
// { y: 1, z: "a" }
// { y: 2, z: "b" }
The $set stage adds the new field to documents and the $unset stage removes/excludes the field to be renamed from documents.
if you are using find method you can't do this, but if you using aggregation it is very easy like this:
db.collectionName.aggregate([
{
$project: {
newName: "$existingKeyName"
}
}
]);
As all of the answers are written the solution in MongoDB query despite the question seeks the solution in Java, posting my approach using Java for posterities.
After the grouping, we can rename the _id fieldname using
Projections.computed("<expected field name>", "$_id")))
To Transform the core part of the query mentioned in the question to Java
Bson mainIdMatch = match(eq("main_id", new ObjectId("58f0f67f50c6af16709fd2c7")));
Bson group = Aggregates.group("$name", Accumulators.sum("count", 1L));
Bson project = Aggregates.project(Projections.fields(Projections.excludeId(),
Projections.computed("name", "$_id")));
reportMongoCollection.aggregate(Arrays.asList(mainIdMatch, group, project))
.into(new ArrayList<>());
To answer specifically, I have added an excerpt from the above code snippet, where I am renaming _id field value as name using Projections.computed("name", "$_id") which map the values of _id which we got as a result of grouping to the field called name. Also, we should exclude the id using Projections.excludeId().
Aggregates.project(Projections.fields(Projections.excludeId(),
Projections.computed("name", "$_id")))
Running on ElasticSearch 2.0 connecting via the Java API. I've got the following query working via the REST API and can't figure out how to do this using the Java API.
{
"query": {
"query_string": {
"query": "myfield:*"
}
},
"aggs" : {
"foo_low": {
"filter" : {
"query" : {
"query_string" : {
"query": "myfield:[1 TO 5]"
}
}
}
},
"foo_high": {
"filter" : {
"query" : {
"query_string" : {
"query": "myfield:[6 TO 10]"
}
}
}
}
}
}
I've had a look at the examples using the addAggregation method but not sure how to pass in the query_string part.
As a bit of a background, was originally using Solr so have multiple Solr facet queries that need to be translated to ElasticSearch. The facet queries are a bit more complicated then I've shown in the example, with multiple fields and conditions referenced in each Solr facet query which is why I want to use the Lucene query with query_string.
Any ideas gratefully received! Thanks.
Since it looks like myfield is an integer field, you could use a range filter instead of a query_string which is more intended for text matching. Since you have two ranges you're interested in, I suggest to use the range aggregation which allows you to define several range buckets (note that the to parameter is not included in the range). Your query would then go like this:
{
"query": {
"query_string": {
"query": "myfield:*"
}
},
"aggs": {
"high_low": {
"range": {
"field": "myfield",
"keyed": true,
"ranges": [
{
"key": "foo_low",
"from": 1,
"to": 6
},
{
"key": "foo_high",
"from": 6,
"to": 11
}
]
}
}
}
}
Translated into Java code, it goes like this:
// 1. bootstrap the query
SearchRequestBuilder search = node.client().prepareSearch()
.setSize(0).setFrom(0)
.setQuery(QueryBuilders.queryStringQuery("myfield:*"));
// 2. create the range aggregation
RangeBuilder rangeAgg = AggregationBuilders.range("high_low").field("myfield");
rangeAgg.addRange("foo_low", 1, 6);
rangeAgg.addRange("foo_high", 6, 11);
search.addAggregation(rangeAgg);
// 3. execute the query
SearchResponse response = search.execute().actionGet();
** UPDATE **
As requested, here is the Java code that will generate the exact query you posted:
// 1. bootstrap the query
SearchRequestBuilder search = node.client().prepareSearch()
.setSize(0).setFrom(0)
.setQuery(QueryBuilders.queryStringQuery("myfield:*"));
// 2. create the filter aggregations
FilterAggregationBuilder lowAgg = AggregationBuilders
.filter("foo_low")
.filter(QueryBuilders.queryStringQuery("myfield:[1 TO 5]"));
search.addAggregation(lowAgg);
FilterAggregationBuilder highAgg = AggregationBuilders
.filter("foo_high")
.filter(QueryBuilders.queryStringQuery("myfield:[6 TO 10]"));
search.addAggregation(highAgg);
// 3. execute the query
SearchResponse response = search.execute().actionGet();