How to fetch the MongoDB collection data unique with two-column emp_country and emp_city and return the array with these two data in the array format.
I need pagination also with country and city. So, when I got unique data, then I can apply pagination as well. Expected output as below mentioned in example and also need for pagination but data should be unique in response.
Sample data:
[
{
"id":"1",
"emp_name":"emp1",
"data":[{
"emp_country":"country1",
"emp_city":"city1"
},
{
"emp_country":"country1",
"emp_city":"city2"
}]
},
{
"id":"2",
"emp_name":"emp2",
"data":[{
"emp_country":"country2",
"emp_city":"city2"
}]
},
{
"id":"3",
"emp_name":"emp3",
"data":[{
"emp_country":"country1",
"emp_city":"city1"
}]
},
{
"id":"4",
"emp_name":"emp4",
"data":[{
"emp_country":"country1",
"emp_city":"city2"
}]
}
]
Expected output:
[
{
"emp_country":"country1",
"emp_city":"city1"
},
{
"emp_country":"country2",
"emp_city":"city2"
},
{
"emp_country":"country1",
"emp_city":"city2"
}
]
How to achieve the above result using Java and MongoDB?
If you have only country and city in your data document
Test code here
db.collection.aggregate([
{
"$unwind": "$data"
},
{
"$group": {
"_id": "$data"
}
},
{
"$replaceRoot": {
"newRoot": "$_id"
}
}
])
If you have possible more than the field country and city in your data document
Test code here
db.collection.aggregate([
{
"$unwind": "$data"
},
{
"$group": {
"_id": {
"emp_country": "$data.emp_country",
"emp_city": "$data.emp_city"
}
}
},
{
"$replaceRoot": {
"newRoot": "$_id"
}
}
])
To write it in Java you can do it with Document and construct the query, or with the query builder
See here for examples
In your example it's unclear what should happen if you have fourth document with data:
{
"id":"4",
"emp_name":"emp4",
"data":{
"emp_country":"country1",
"emp_city":"city4"
}
}
For a result with 2 lists of unique values of each field you can use $addToSet. In your case it would like as this in MongoDB:
db.collection.aggregate([{
$group: {
_id: null,
'data.emp_country': {$addToSet: 'data.emp_country'},
'data.emp_city': {$addToSet: 'data.emp_city'},
}
}])
and the result will look like this:
{
_id: null,
'data.emp_country': ['country1', 'country2'],
'data.emp_city': ['city1', 'city2']
}
Related
How to write Custom Query for Mongo DB to get the distinct data Need to write in Java but I need to check if is it possible with the query as well without using aggregation pipeline.
Sample Data:
[
{
"id":1,
"empName":"emp1",
"emp_city":"city1"
},
{
"id":2,
"empName":"emp2",
"emp_city":"city1"
},
{
"id":3,
"empName":"emp1",
"emp_city":"city1"
},
{
"id":4,
"empName":"emp1",
"emp_city":"city2"
}
]
Expected Output:
[
{
"empName":"emp1",
"emp_city":"city1"
},
{
"empName":"emp1",
"emp_city":"city2"
},
{
"empName":"emp2",
"emp_city":"city1"
}
]
For what you are trying to archive I would suggest using a group by, by the two fields (empName and emp_city),
Here you have and example https://sqlserverguides.com/mongodb-group-by-multiple-fields/
use this :
db.collection.aggregate([
{
$group: {
_id: {
empName: "$empName",
emp_city: "$emp_city"
}
}
},
{
"$replaceRoot": {
"newRoot": "$_id"
}
}
])
https://mongoplayground.net/p/d8i7iOuvfsR
If we have collection "orders"
{"_id":"1",
"currency": "USD"//...
}
And collection "order_items"
{"_id":"1",
"order":"1",
"product":"p1",
"totalItemAmount": 25
},
{"_id":"2",
"order":"1",
"product":"p2",
"totalItemAmount": 50
}
How to get such result
[{
"orderId": "1",
"currency": "USD",
"products": ["p1", "p2"]
"totalOrderAmount": "75"
}]
using Java (with or without Spring)?
You can achieve that using an aggregation like this:
db.orders.aggregate([
{
"$lookup": {
"from": "order_items",
"as": "items",
"localField": "_id",
"foreignField": "order"
}
},
{
"$unwind": "$items"
},
{
"$group": {
"_id": "$_id",
"totalOrderAmount": {
"$sum": "$items.totalItemAmount"
},
"products": {
"$push": "$items.product"
}
}
},
{
"$project": {
"_id": 0,
"orderId": "$_id",
"currency": 1,
"totalOrderAmount": {
"$toString": "$totalOrderAmount"
},
"products": 1
}
}
])
I don't know which driver you're using in Java but it's not hard to adapt it to the language you're using
I have this collection of documents:
[
{
"name": "name1",
"data": [
{
"numbers": ["1","2","3"]
}
]
},
{
"name": "name2",
"data": [
{
"numbers": ["2","5","3"]
}
]
},
{
"name": "name3",
"data": [
{
"numbers": ["1","5","2"]
}
]
},
{
"name": "name4",
"data": [
{
"numbers": ["1","4","3"]
}
]
},
{
"name": "name5",
"data": [
{
"numbers": ["1","2"]
}
]
}
]
I want to get all documents of this collection when an array passed as a parameter is a subset of data.numbers.
This is the aggregation that I'm using.
db.testing.aggregate(
[
{ "$match" : { "data.numbers" : { "$exists" : true } } },
{ "$project" : { "is_subset" : { "$filter" : { "input" : "$data", "as" : "d", "cond" : { "$setIsSubset" :[ ["1"],"$$d.numbers"] } } } } },
{ "$match" : { "is_subset.0" : { "$exists" : true } } }]
);
I'm trying to reproduce the above aggregation in Spring Data MongoDB.
How to pass an array as parameter in $filter and $setIsSubset functions?
operations.aggregate(
newAggregation(Testing.class,
match(where("data.numbers").exists(true)),
project().and(
filter("data")
.as("d")
.by(???))
.as("is_subset"),
match(where("is_subset.0").exists(true))
), Testing.class);
I solve my issue.
operations.aggregate(
newAggregation(Testing.class,
match(where("data.numbers").exists(true)),
project("id", "name").and(
filter("data")
.as("d")
.by(context -> new Document("$setIsSubset", Arrays.asList(numbers, "$$d.numbers"))))
.as("is_subset"),
match(where("is_subset.0").exists(true))
), Testing.class);
I created a Document with the content that I needed in the $filter condition.
new Document("$setIsSubset", Arrays.asList(numbers, "$$d.numbers"))
I have the following data structure
[{
"id": "1c7bbebd-bc3d-4352-9ac0-98c01d13189d",
"version": 0,
"groups": [
{
"internalName": "Admin group",
"fields": [
{
"internalName": "Is verified",
"uiProperties": {
"isShow": true
}
},
{
"internalName": "Hide",
"uiProperties": {
"isHide": false
}
},
...
]
},
...
]
},
{
"id": "2b7bbebd-bc3d-4352-9ac0-98c01d13189d",
"version": 0,
"groups": [
{
"internalName": "User group",
"fields": [
{
"internalName": "Is verified",
"uiProperties": {
"isShow": true
}
},
{
"internalName": "Blocked",
"uiProperties": {
"isBlocked": true
}
},
...
]
},
...
]
},
...
]
Internal names of the fields can be repeated. I want to group by group.field.internalName and cut the array(for pagination) and get the output like:
{
"totalCount": 3,
"items": [
{
"internalName": "Blocked"
},
{
"internalName": "Hide"
},
{
"internalName": "Is verified"
}
]}
I wrote a query that works,
db.layouts.aggregate(
{
$unwind : "$groups"
},
{
$unwind : "$groups.fields"
},
{
$group: {
"_id" : {
"internalName" : "$groups.fields.internalName",
},
"internalName" : {
$first : "$groups.fields.internalName"
}
}
},
{
$group: {
"_id" : null,
"items" : {
$push : "$$ROOT"
},
"totalCount" : {
$sum : 1
}
}
},
{
$project: {
"items" : {
$slice : [ "$items", 0, 20 ]
},
"totalCount": 1
}
})
but I have the problem of translating it to java api. Notice that i need to use mongoTemplate approach. Here is what i have and where i'm struck
final List<AggregationOperation> aggregationOperations = new ArrayList<>();
aggregationOperations.add(unwind("groups"));
aggregationOperations.add(unwind("groups.fields"));
aggregationOperations.add(
group("groups.fields.internalName")
.first("groups.fields.internalName").as("internalName")
);
aggregationOperations.add(
group()
.push("$$ROOT").as("fields")
.sum("1").as("totalCount") // ERROR only string ref can be placed, but i need a number?
);
aggregationOperations.add(
project()
.andInclude("totalCount")
.and("fields").slice(size, page * size)
);
final Aggregation aggregation = newAggregation(aggregationOperations);
mongoTemplate.aggregate(aggregation, LAYOUTS, FieldLites.class).getMappedResults()
With this query i have the problem with sum(), because i can place only a String ref by api(but need a number) and with project operation - got an exception
java.lang.IllegalArgumentException: Invalid reference 'totalCount'!] with root cause
Can you help me with this query translation?
You can use count
group()
.push("$$ROOT").as("fields")
.count().as("totalCount")
I need to export customer records from database of mongoDB. Exported customer records should not have duplicated values. "firstName+lastName+code" is the key to DE-duped the record and If there are two records present in database with same key then I need to give preference to source field with value other than email.
customer (id,firstName,lastName,code,source) collection is this.
If there are record 3 records with same unique key and 3 different sources then i need to choose only one record between 2 sources(TV,internet){or if there are n number of sources i need the one record only}not with the 'email'(as email will be choosen when only one record is present with the unique key and source is email)
query using:
db.customer.aggregate([
{
"$match": {
"active": true,
"dealerCode": { "$in": ["111391"] },
"source": { "$in": ["email", "TV", "internet"] }
}
},
{
$group: {
"_id": {
"firstName": "$personalInfo.firstName",
"lastName": "$personalInfo.lastName",
"code": "$vehicle.code"
},
"source": {
$addToSet: { "source": "$source" }
}
}
},
{
$redact:
{
$cond: [
{ $eq: [{ $ifNull: ["$source", "other"] }, "email"] },
"$$PRUNE",
"$$DESCEND"
]
}
},
{
$project:
{
"source":
{
$map:
{
"input": {
$cond: [
{ $eq: [{ $size: "$source" }, 0] },
[{ "source": "email" }],
"$source"
]
},
"as": "inp",
"in": "$$inp.source"
}
},
"record": { "_id": 1 }
}
}
])
sample output:
{ "_id" : { "firstName" : "sGI6YaJ36WRfI4xuJQzI7A==", "lastName" : "99eQ7i+uTOqO8X+IPW+NOA==", "code" : "1GTHK23688F113955" }, "source" : ["internet"] }
{ "_id" : { "firstName" : "WYDROTF/9vs9O7XhdIKd5Q==", "lastName" : "BM18Uq/ltcbdx0UJOXh7Sw==", "code" : "1G4GE5GV5AF180133" }, "source" : ["internet"] }
{ "_id" : { "firstName" : "id+U2gYNHQaNQRWXpe34MA==", "lastName" : "AIs1G33QnH9RB0nupJEvjw==", "code" : "1G4GE5EV0AF177966" }, "source" : ["internet"] }
{ "_id" : { "firstName" : "qhreJVuUA5l8lnBPVhMAdw==", "lastName" : "petb0Qx3YPfebSioY0wL9w==", "code" : "1G1AL55F277253143" }, "source" : ["TV"] }
{ "_id" : { "firstName" : "qhreJVuUA5l8lnBPVhMAdw==", "lastName" : "6LB/NmhbfqTagbOnHFGoog==", "code" : "1GCVKREC0EZ168134" }, "source" : ["TV", "internet"] }
This is a problem with this query please suggest :(
Your code doesn't work, because $cond is not an accumulator operator. Only these accumulator operators, can be used in a $group stage.
Assuming your records contain not more than two possible values of source as you mention in your question, you could add a conditional $project stage and modify the $group stage as,
Code:
db.customer.aggregate([
{
$group: {
"_id": {
"id": "$id",
"firstName": "$firstName",
"lastName": "$lastName",
"code": "$code"
},
"sourceA": { $first: "$source" },
"sourceB": { $last: "$source" }
}
},
{
$project: {
"source": {
$cond: [
{ $eq: ["$sourceA", "email"] },
"$sourceB",
"$sourceA"
]
}
}
}
])
In case there can be more that two possible values for source, then you could do the following:
Group by the id, firstName, lastName and code. Accumulate
the unique values of source, using the $addToSet operator.
Use $redact to keep only the values other than email.
Project the required fields, if the source array is empty(all the elements have been removed), add a
value email to it.
Unwind the source field to list it as a field and not an array.
(optional)
Code:
db.customer.aggregate([
{
$group: {
"_id": {
"id": "$id",
"firstName": "$firstName",
"lastName": "$lastName",
"code": "$code"
},
"sourceArr": { $addToSet: { "source": "$source" } }
}
},
{
$redact: {
$cond: [
{ $eq: [{ $ifNull: ["$source", "other"] }, "email"] },
"$$PRUNE",
"$$DESCEND"
]
}
},
{
$project: {
"source": {
$map: {
"input":
{
$cond: [
{ $eq: [{ $size: "$sourceArr" }, 0] },
[{ "source": "item" }],
"$sourceArr"]
},
"as": "inp",
"in": "$$inp.source"
}
}
}
}
])