Develop Reference

How to fetch unique data on multiple column using MongoDB and Java

How to fetch the MongoDB collection data unique with two-column emp_country and emp_city and return the array with these two data in the array format.
I need pagination also with country and city. So, when I got unique data, then I can apply pagination as well. Expected output as below mentioned in example and also need for pagination but data should be unique in response.
Sample data:
[
{
"id":"1",
"emp_name":"emp1",
"data":[{
"emp_country":"country1",
"emp_city":"city1"
},
{
"emp_country":"country1",
"emp_city":"city2"
}]
},
{
"id":"2",
"emp_name":"emp2",
"data":[{
"emp_country":"country2",
"emp_city":"city2"
}]
},
{
"id":"3",
"emp_name":"emp3",
"data":[{
"emp_country":"country1",
"emp_city":"city1"
}]
},
{
"id":"4",
"emp_name":"emp4",
"data":[{
"emp_country":"country1",
"emp_city":"city2"
}]
}
]
Expected output:
[
{
"emp_country":"country1",
"emp_city":"city1"
},
{
"emp_country":"country2",
"emp_city":"city2"
},
{
"emp_country":"country1",
"emp_city":"city2"
}
]
How to achieve the above result using Java and MongoDB?

If you have only country and city in your data document
Test code here
db.collection.aggregate([
{
"$unwind": "$data"
},
{
"$group": {
"_id": "$data"
}
},
{
"$replaceRoot": {
"newRoot": "$_id"
}
}
])
If you have possible more than the field country and city in your data document
Test code here
db.collection.aggregate([
{
"$unwind": "$data"
},
{
"$group": {
"_id": {
"emp_country": "$data.emp_country",
"emp_city": "$data.emp_city"
}
}
},
{
"$replaceRoot": {
"newRoot": "$_id"
}
}
])
To write it in Java you can do it with Document and construct the query, or with the query builder
See here for examples

In your example it's unclear what should happen if you have fourth document with data:
{
"id":"4",
"emp_name":"emp4",
"data":{
"emp_country":"country1",
"emp_city":"city4"
}
}
For a result with 2 lists of unique values of each field you can use $addToSet. In your case it would like as this in MongoDB:
db.collection.aggregate([{
$group: {
_id: null,
'data.emp_country': {$addToSet: 'data.emp_country'},
'data.emp_city': {$addToSet: 'data.emp_city'},
}
}])
and the result will look like this:
{
_id: null,
'data.emp_country': ['country1', 'country2'],
'data.emp_city': ['city1', 'city2']
}

Passing array as parameter in filter function with spring-data-mongodb

I have this collection of documents:
[
{
"name": "name1",
"data": [
{
"numbers": ["1","2","3"]
}
]
},
{
"name": "name2",
"data": [
{
"numbers": ["2","5","3"]
}
]
},
{
"name": "name3",
"data": [
{
"numbers": ["1","5","2"]
}
]
},
{
"name": "name4",
"data": [
{
"numbers": ["1","4","3"]
}
]
},
{
"name": "name5",
"data": [
{
"numbers": ["1","2"]
}
]
}
]
I want to get all documents of this collection when an array passed as a parameter is a subset of data.numbers.
This is the aggregation that I'm using.
db.testing.aggregate(
[
{ "$match" : { "data.numbers" : { "$exists" : true } } },
{ "$project" : { "is_subset" : { "$filter" : { "input" : "$data", "as" : "d", "cond" : { "$setIsSubset" :[ ["1"],"$$d.numbers"] } } } } },
{ "$match" : { "is_subset.0" : { "$exists" : true } } }]
);
I'm trying to reproduce the above aggregation in Spring Data MongoDB.
How to pass an array as parameter in $filter and $setIsSubset functions?
operations.aggregate(
newAggregation(Testing.class,
match(where("data.numbers").exists(true)),
project().and(
filter("data")
.as("d")
.by(???))
.as("is_subset"),
match(where("is_subset.0").exists(true))
), Testing.class);

I solve my issue.
operations.aggregate(
newAggregation(Testing.class,
match(where("data.numbers").exists(true)),
project("id", "name").and(
filter("data")
.as("d")
.by(context -> new Document("$setIsSubset", Arrays.asList(numbers, "$$d.numbers"))))
.as("is_subset"),
match(where("is_subset.0").exists(true))
), Testing.class);
I created a Document with the content that I needed in the $filter condition.
new Document("$setIsSubset", Arrays.asList(numbers, "$$d.numbers"))

Mongo db java driver query convert

I have the following data structure
[{
"id": "1c7bbebd-bc3d-4352-9ac0-98c01d13189d",
"version": 0,
"groups": [
{
"internalName": "Admin group",
"fields": [
{
"internalName": "Is verified",
"uiProperties": {
"isShow": true
}
},
{
"internalName": "Hide",
"uiProperties": {
"isHide": false
}
},
...
]
},
...
]
},
{
"id": "2b7bbebd-bc3d-4352-9ac0-98c01d13189d",
"version": 0,
"groups": [
{
"internalName": "User group",
"fields": [
{
"internalName": "Is verified",
"uiProperties": {
"isShow": true
}
},
{
"internalName": "Blocked",
"uiProperties": {
"isBlocked": true
}
},
...
]
},
...
]
},
...
]
Internal names of the fields can be repeated. I want to group by group.field.internalName and cut the array(for pagination) and get the output like:
{
"totalCount": 3,
"items": [
{
"internalName": "Blocked"
},
{
"internalName": "Hide"
},
{
"internalName": "Is verified"
}
]}
I wrote a query that works,
db.layouts.aggregate(
{
$unwind : "$groups"
},
{
$unwind : "$groups.fields"
},
{
$group: {
"_id" : {
"internalName" : "$groups.fields.internalName",
},
"internalName" : {
$first : "$groups.fields.internalName"
}
}
},
{
$group: {
"_id" : null,
"items" : {
$push : "$$ROOT"
},
"totalCount" : {
$sum : 1
}
}
},
{
$project: {
"items" : {
$slice : [ "$items", 0, 20 ]
},
"totalCount": 1
}
})
but I have the problem of translating it to java api. Notice that i need to use mongoTemplate approach. Here is what i have and where i'm struck
final List<AggregationOperation> aggregationOperations = new ArrayList<>();
aggregationOperations.add(unwind("groups"));
aggregationOperations.add(unwind("groups.fields"));
aggregationOperations.add(
group("groups.fields.internalName")
.first("groups.fields.internalName").as("internalName")
);
aggregationOperations.add(
group()
.push("$$ROOT").as("fields")
.sum("1").as("totalCount") // ERROR only string ref can be placed, but i need a number?
);
aggregationOperations.add(
project()
.andInclude("totalCount")
.and("fields").slice(size, page * size)
);
final Aggregation aggregation = newAggregation(aggregationOperations);
mongoTemplate.aggregate(aggregation, LAYOUTS, FieldLites.class).getMappedResults()
With this query i have the problem with sum(), because i can place only a String ref by api(but need a number) and with project operation - got an exception
java.lang.IllegalArgumentException: Invalid reference 'totalCount'!] with root cause
Can you help me with this query translation?

You can use count
group()
.push("$$ROOT").as("fields")
.count().as("totalCount")

Elasticsearch complex query using Java API

I am trying to query on ElasticSearch using Java API, my query is:
curl -XGET 'http://localhost:9200/logstash-*/_search?search_type=count' -d '
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"and" : [
{
"range": {
"timestamp": {
"gte": "2015-08-20",
"lt": "2015-08-21",
"format": "yyyy-MM-dd",
"time_zone": "+8:00"
}
}
},
{"query": {
"match": {
"request": {
"query": "/v2/brand"
}
}
}
},
{"term": { "response" : "200"}
}
]
}
}
},
"aggs": {
"group_by_device_id": {
"terms": {
"field": "clientip"
}
}
}
}'
The similar sql logic is:
select distinct(clientip) from table where timestamp between '2015-08-20' and '2015-08-21' and request like '/v2/brand%' and response = '200'
How to implement it using Java API?
Please guide I am new to ElasticSearch. Thanks in advance!

I have resolved the problem, below is my codes:
SearchResponse scrollResp1 = client.prepareSearch("logstash-*").setSearchType(SearchType.SCAN).
setQuery(QueryBuilders.filteredQuery(QueryBuilders.matchAllQuery(),
FilterBuilders.andFilter(FilterBuilders.termFilter("response", "200")
, FilterBuilders.rangeFilter("timestamp").gte(startDate).lt
(endDate), FilterBuilders.queryFilter
(QueryBuilders.matchQuery("request", "signup"))
)))
.addAggregation(AggregationBuilders.terms
("group_by_client_ip").size(0).field("clientip")).get();

MongoDB aggregation $let expression to Java Driver

I have following $let expression :
{
"$let": {
"vars": {
"h": {
"$hour": "$Date"
}
},
"in": {
"$cond": {
"if": {
"$lt": [
"$$h",
6
]
},
"then": "Night",
"else": {
"$cond": {
"if": {
"$lt": [
"$$h",
12
]
},
"then": "Morning",
"else": {
"$cond": {
"if": {
"$lt": [
"$$h",
18
]
},
"then": "Afternoon",
"else": "Evening"
}
}
}
}
}
}
}
}
I am unable to figure out any way to convert this shell operation to Java driver version. I tried MongoDB java driver docs and even Google but unable to find any way for it.
Any help on what could be the JAVA implementation for the same.
Thank you in advance!!!