I'm working with documents that contain music playlists.
Each document has this structure:
{
"user_id": "5858",
"playlists": [
{
"name": "My Playlist",
"guild_ids": ["7575"],
"items": [
{
"title": "title",
"url": "url",
"duration": 200000
}
]
}
]
}
I would like to extract all playlists from the same guild.
But the thing is that i'd like the results to be returned in a single document. One single document with a list of playlists.
The expected result for guild_id=5656 would be like this:
{
"playlists": [
{
"name": "My Playlist",
"guild_ids": ["5656"],
"items": [
{
"title": "title",
"url": "url",
"duration": 200000
}
]
},
// other playlists where guild_ids contains "5656"
]
}
I tried to use aggregation but i always get the same number of documents as the number of unique user_ids. I get the playlists grouped by user_id.
The following query can get us the expected output:
db.collection.aggregate([
{
$unwind:"$playlists"
},
{
$match:{
"playlists.guild_ids":{
$in:["7575"]
}
}
},
{
$group:{
"_id":null,
"playlists":{
$push: "$playlists"
}
}
},
{
$project:{
"_id":0
}
}
]).pretty()
Data set:
{
"_id" : ObjectId("5d88225e38db7cf8d3f75cd6"),
"user_id" : "5858",
"playlists" : [
{
"name" : "My Playlist",
"guild_ids" : [
"7575"
],
"items" : [
{
"title" : "title",
"url" : "url",
"duration" : 200000
}
]
}
]
}
{
"_id" : ObjectId("5d88225e38db7cf8d3f75cd7"),
"user_id" : "5858",
"playlists" : [
{
"name" : "My Playlist 2",
"guild_ids" : [
"1234"
],
"items" : [
{
"title" : "title",
"url" : "url",
"duration" : 200000
}
]
}
]
}
{
"_id" : ObjectId("5d88225e38db7cf8d3f75cd8"),
"user_id" : "5858",
"playlists" : [
{
"name" : "My Playlist 3",
"guild_ids" : [
"7575"
],
"items" : [
{
"title" : "title",
"url" : "url",
"duration" : 200000
}
]
}
]
}
Output:
{
"playlists" : [
{
"name" : "My Playlist",
"guild_ids" : [
"7575"
],
"items" : [
{
"title" : "title",
"url" : "url",
"duration" : 200000
}
]
},
{
"name" : "My Playlist 3",
"guild_ids" : [
"7575"
],
"items" : [
{
"title" : "title",
"url" : "url",
"duration" : 200000
}
]
}
]
}
Query analysis: We are unwinding the playlists, filtering only those which has 7575 guild ID and then grouping them all again.
Related
This is my sample es index document:
"hits" : [
{
"_index" : "project_note",
"_type" : "project_note",
"_id" : "19",
"_score" : 1.0,
"_source" : {
"createTime" : "2021-10-04T13:43:55.330",
"createTimeInMs" : 1633333435330,
"createdBy" : "test",
"editTime" : "2021-10-04T13:43:55.330",
"editTimeInMs" : 1633333435330,
"editedBy" : "test",
"versionId" : 1,
"id" : "19",
"organizationId" : "28",
"accessLevel" : "PUBLIC",
"status" : "ACTIVE",
"projectId" : "95",
"userId" : 129,
"noteType" : "SYSTEM_GENERATED",
"projectDemographicLogId" : "1"
},
{
"_index" : "project_note",
"_type" : "project_note",
"_id" : "19",
"_score" : 1.0,
"_source" : {
"createTime" : "2021-10-04T13:43:55.330",
"createTimeInMs" : 1633333435330,
"createdBy" : "test",
"editTime" : "2021-10-04T13:43:55.330",
"editTimeInMs" : 1633333435330,
"editedBy" : "test",
"versionId" : 1,
"id" : "19",
"organizationId" : "28",
"accessLevel" : "PUBLIC",
"status" : "ACTIVE",
"projectId" : "95",
"userId" : 129
}
]
In the first doc, it has noteType but in the second, I don't have that field stored in db.
I want to exclude the documents where noteType==null or noteType is absent.
But, I am getting only the docs which have noteType="SYSTEM_GENERATED"
My approach:
{
"query":
{
"bool" : {
"must" : [
{
"term" : {
"projectId" : {
"value" : "95",
"boost" : 1.0
}
}
},
{
"range" : {
"createTimeInMs" : {
"from" : null,
"to" : 1633594455000,
"include_lower" : true,
"include_upper" : true,
"boost" : 1.0
}
}
}
],
"must_not" : [
{
"term" : {
"noteType" : {
"value" : "SYSTEM_GENERATED",
"boost" : 1.0
}
}
}
],
"adjust_pure_negative" : true,
"boost" : 1.0
}
}
}
Equivalent java code:
BoolQueryBuilder queryBuilder= QueryBuilders.boolQuery();
queryBuilder.must(QueryBuilders.termQuery("projectId", requestInfo.getProjectId()));
queryBuilder.must(rangeQuery("createTimeInMs").lte(requestInfo.getCreateTimeInMs()));
if(!requestInfo.isIncludeLog()) {
queryBuilder.mustNot(QueryBuilders.termQuery("noteType", Defs.SYSTEM_NOTE_TYPE));
}
If only the must_not part of the query is used (excluding the must part)
{
"query": {
"bool": {
"must_not": [
{
"term": {
"noteType.keyword": {
"value": "SYSTEM_GENERATED",
"boost": 1.0
}
}
}
],
"adjust_pure_negative": true,
"boost": 1.0
}
}
}
The search result is similar to what you expect to get
"hits": [
{
"_index": "69477995",
"_type": "_doc",
"_id": "2",
"_score": 0.0,
"_source": {
"createTime": "2021-09-26T15:54:08.373",
"createTimeInMs": 1632650048373,
"createdBy": "test",
"editTime": "2021-09-26T15:54:08.373",
"editTimeInMs": 1632650048373,
"editedBy": "test",
"versionId": 1,
"id": "18",
"note": "note-1, simple note ",
"organizationId": "28",
"accessLevel": "PUBLIC",
"status": "ACTIVE",
"taskId": "5",
"userId": 129
}
}
]
I have a following mongoDB document structure -
db.menus.findOne()
{
"_id" : ObjectId("5cf25412326c3f4f26df039b"),
"restaurantId" : "301728",
"items" : [
{
"itemId" : "CEBM4H41JR",
"name" : "Crun Chicken",
"imageUrl" : "",
"price" : 572,
"attributes" : [
"Tasty",
"Spicy"
]
},
{
"itemId" : "53Q0XS3HPR",
"name" : "Devils Chicken",
"imageUrl" : "",
"price" : 595,
"attributes" : [
"Gravy",
"Salty"
]
}
]
}
I am trying to write a query to get all the menus based on the "attributes" field under "items" in the document.
I have done the following to get the menus if "name" of "items" is given and I am getting a result -
db.menus.find({ 'items' : {$elemMatch : {'name' : {$regex : "Chicken Thali", $options: 'i' }}}}).pretty()
I have tried this for getting the result for attributes but this is not working -
db.menus.find({'items' : {$elemMatch : {'attributes' : {$all : [{$regex : "Tasty", $options: 'i' }]}}}})
How do I get the list and I also want to write this query for mongoRepository in a spring boot application?
Further, based on the restaurantId's obtained, I have to query restaurant collection in order to find all the restaurants in restaurants collection having the following structure -
{
"_id" : ObjectId("5cf2540e326c3f4f26de93dd"),
"restaurantId" : "301728",
"name" : "Desire Foods",
"imageUrl" : "https://b.zmtcdn.com/data/pictures/8/301728/d690ccb500d746530f56e1d637949da2_featured_v2.jpg",
"latitude" : 28.4900591,
"longitude" : 77.3066401,
"attributes" : [
"Chinese",
" Fast Food",
" Bakery"
],
"opensAt" : "09:30",
"closesAt" : "22:30"
}
Is the whole operation possible in a single query?
I think you can modify your query to use $in instead of $all.
To achieve your intended result, you can try:
db.collection.aggregate([
{
"$match": {
"items": {
"$elemMatch": {
"attributes": {
"$in": [
"Tasty"
]
}
}
}
}
},
{
"$lookup": {
"from": "restaurant",
"localField": "restaurantId",
"foreignField": "restaurantId",
"as": "restaurants"
}
},
{
"$unwind": "restaurants"
},
{
"$replaceRoot": { "newRoot": "$restaurants" }
}
])
Use $match at appropriate stages as needed to limit the documents pulled in memory
I've indexed documents like bellow in elasticsearch.
{
"category": "clothing (f)",
"description": "Women's Unstoppable Graphic T-Shirt - Women’s Short Sleeve Shirt",
"name": "Women's Unstoppable Graphic T-Shirt",
"price": "$34.99"
}
There are categories like clothing (m), clothing (f) etc. I am trying to exclude the cloting (m) category items if the search is for female items. The query I am trying is:
{
"query": {
"bool": {
"must": [
{
"match": {
"description": "women's black shirt"
}
}
],
"must_not": [
{
"term": {
"category": "clothing (m)"
}
}
]
}
},
"from": 0,
"size": 50
}
But this is not working as expected. There are always few results with clothing (m) document with other documents. How can I exclude documents which have a particular category?
In order to exclude a specific term (exact match) you will have to use keyword datatype.
Keyword datatypes are typically used for filtering (Find me all blog posts where status is published), for sorting, and for aggregations. Keyword fields are only searchable by their exact value.
Keyword Datatype
Your current query catches clothing (m) in the results because when you indexed your documents they were analyzed with elasticsearch standard analyzer which analyzes clothing (m) as clothing and (m).
In your query you searched for category as text datatype.
Text datatype fields are analyzed, that is they are passed through an analyzer to convert the string into a list of individual terms before being indexed.
Run this command:
POST my_index/_analyze
{
"text": ["clothing (m)"]
}
Results:
{
"tokens" : [
{
"token" : "clothing",
"start_offset" : 0,
"end_offset" : 8,
"type" : "<ALPHANUM>",
"position" : 0
},
{
"token" : "m",
"start_offset" : 10,
"end_offset" : 11,
"type" : "<ALPHANUM>",
"position" : 1
}
]
}
A working example:
Assuming you mappings look like that:
{
"my_index" : {
"mappings" : {
"properties" : {
"category" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"description" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"price" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
}
Let's post a few documents:
POST my_index/_doc/1
{
"category": "clothing (m)",
"description": "Women's Unstoppable Graphic T-Shirt - Women’s Short Sleeve Shirt",
"name": "Women's Unstoppable Graphic T-Shirt",
"price": "$34.99"
}
POST my_index/_doc/2
{
"category": "clothing (f)",
"description": "Women's Unstoppable Graphic T-Shirt - Women’s Short Sleeve Shirt",
"name": "Women's Unstoppable Graphic T-Shirt",
"price": "$34.99"
}
Now our query should look like this:
GET my_index/_search
{
"query": {
"bool": {
"must": {
"match": {
"description": "women's black shirt"
}
},
"filter": {
"bool": {
"must_not": {
"term": {
"category.keyword": "clothing (m)"
}
}
}
}
}
},
"from": 0,
"size": 50
}
The results:
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.43301374,
"hits" : [
{
"_index" : "my_index",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.43301374,
"_source" : {
"category" : "clothing (f)",
"description" : "Women's Unstoppable Graphic T-Shirt - Women’s Short Sleeve Shirt",
"name" : "Women's Unstoppable Graphic T-Shirt",
"price" : "$34.99"
}
}
]
}
}
Results without using keyword
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 0.43301374,
"hits" : [
{
"_index" : "my_index",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.43301374,
"_source" : {
"category" : "clothing (f)",
"description" : "Women's Unstoppable Graphic T-Shirt - Women’s Short Sleeve Shirt",
"name" : "Women's Unstoppable Graphic T-Shirt",
"price" : "$34.99"
}
},
{
"_index" : "my_index",
"_type" : "_doc",
"_id" : "2",
"_score" : 0.43301374,
"_source" : {
"category" : "clothing (m)",
"description" : "Women's Unstoppable Graphic T-Shirt - Women’s Short Sleeve Shirt",
"name" : "Women's Unstoppable Graphic T-Shirt",
"price" : "$34.99"
}
}
]
}
}
As you can see from the last results we got also clothing (m).
BTW don't use term for text datatype. use match.
Hope this helps.
I want to update request fields only in an array using java.This is my existing document in mongo db:
{
"_id": "6691e5068dwe335w42cb0a699650f",
"Opportunity_Owner": "Self",
"Account_Name": "IC",
"Lead_Source": "Callbox",
"Opportunity_Name": "name1 ",
"Stage": "Proposal",
"Stage_Status": "A",
"1555570551211": [],
"1555556165153": [],
"1555556059584": [{
"id": "1557389940585",
"Notes": "Note 1"
},
{
"id": "1557389945398",
"Notes": "Hi Bobby "
},
{
"id": "1557389978181",
"Notes": "Spoken to Bobby."
},
{
"id": "1557389990159",
"Notes": "plan to call on 29/Apr"
}
],
"createdBy": "2c18b8dbb7d74a41a66f53a90117480a",
"createdDate": "1562911250917"
}
Request payload:
{
"_id" : "6691e5068dwe335w42cb0a699650f",
"Stage_Status" : "I",
"1555556059584" : [
{
"id" : "1557389940585",
"Notes" : "updated note 123"
}
]
}
I am trying to update "Stage_Status" and "1555556059584.Notes" at a time using $set.I am able to update "Stage_Status" but "1555556059584" array is going to reset with what i have updated with last one.
expected output:
{
"_id" : "6691e5068dwe335w42cb0a699650f",
"Opportunity_Owner" : "Self",
"Account_Name" : "IC",
"Lead_Source" : "Callbox",
"Opportunity_Name" : "name1 ",
"Stage" : "Proposal",
"Stage_Status" : "I",
"1555570551211" : [],
"1555556165153" : [],
"1555556059584" : [
{
"id" : "1557389940585",
"Notes" : "updated note 123"
},
{
"id" : "1557389945398",
"Notes" : "Hi Bobby "
},
{
"id" : "1557389978181",
"Notes" : "Spoken to Bobby."
},
{
"id" : "1557389990159",
"Notes" : "plan to call on 29/Apr"
}
],
"createdBy" : "2c18b8dbb7d74a41a66f53a90117480a",
"createdDate" : "1562911250917"
}
can any one please help me to figure it out in java.
I guess you wanted to update Stage_Status and 1555556059584.Notes at Once .
here is a demo about it
> db.student.find()
{ "_id" : ObjectId("5d2c09ea8ed60ae70d3dd76b"), "name" : "bigbang", "courses" : [ { "name" : "en", "classRoom" : "9001" }, { "name" : "math", "classRoom" : "1001" } ] }
> db.student.update({name:'bigbang','courses.name':'en'},{ $set: {'courses.$.classRoom':'1009',name :"course"} })
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.student.find()
{ "_id" : ObjectId("5d2c09ea8ed60ae70d3dd76b"), "name" : "course", "courses" : [ { "name" : "en", "classRoom" : "1009" }, { "name" : "math", "classRoom" : "1001" } ] }
the java demo is like this
collection.updateOne(and(eq("Stage_Status","A"),eq("1555556059584.id","1557389940585")),new Document("$set" ,new Document("Stage_Status","YOUR_NEW_VALUE").append("1555556059584.$.Notes","YOUR_NEW_VALUE")));
you must set the 1555556059584.id to let the diver know which element to be update .
In my application I need to get all entries from my sqlite database and generate this json to create a node in a drupal site, I´m new in Json please tell me the best way to achieve this :
{ "body" : [ ],
"changed" : "1393522835",
"cid" : "0",
"comment" : "1",
"comment_count" : "0",
"created" : "1393522835",
"data" : "a:1:{s:7:\"contact\";i:0;}",
"field_associatedterm" : { "und" : [ { "target_id" : "2" } ] },
"field_fat" : { "und" : [ { "value" : "24.4" } ] },
"field_creationts" : { "und" : [ { "value" : "1391016769" } ] },
"field_key" : { "und" : [ { "format" : null,
"safe_value" : "12312312-1123123",
"value" : "12312312-1123123"
} ] },
"field_devicename" : { "und" : [ { "format" : null,
"safe_value" : "iPhone",
"value" : "iPhone"
} ] },
"field_devices" : [ ],
"field_editts" : [ ],
"field_entrydate" : { "und" : [ { "date_type" : "datetime",
"timezone" : "Europe/London",
"timezone_db" : "Europe/London",
"value" : "2013-06-05 09:45:00"
} ] },
"field_humor" : { "und" : [ { "value" : "1" } ] },
"field_synchts" : [ ],
"field_text" : { "und" : [ { "format" : null,
"safe_value" : """",
"value" : "\"\""
} ] },
"field_timezonename" : [ ],
"field_timezoneoffset" : [ ],
"field_useruniqueid" : [ ],
"field_number" : { "und" : [ { "value" : "12" } ] },
"language" : "pt-pt",
"last_comment_name" : null,
"last_comment_timestamp" : "1393522835",
"last_comment_uid" : "1",
"log" : "",
"metatags" : [ ],
"name" : "admin",
"nid" : "302",
"picture" : "0",
"promote" : "1",
"revision_timestamp" : "1393522835",
"revision_uid" : "1",
"status" : "1",
"sticky" : "0",
"title" : "2013-06-05 08:39:01 +0000",
"tnid" : "0",
"translate" : "0",
"type" : "entry",
"uid" : "1",
"vid" : "302"
},
This is not hard if you use some tools to generate the code from json strings.Like this one:http://jsongen.byingtondesign.com/.
I'm sure there are more useful tools to do this work.