Related
This is my sample es index document:
"hits" : [
{
"_index" : "project_note",
"_type" : "project_note",
"_id" : "19",
"_score" : 1.0,
"_source" : {
"createTime" : "2021-10-04T13:43:55.330",
"createTimeInMs" : 1633333435330,
"createdBy" : "test",
"editTime" : "2021-10-04T13:43:55.330",
"editTimeInMs" : 1633333435330,
"editedBy" : "test",
"versionId" : 1,
"id" : "19",
"organizationId" : "28",
"accessLevel" : "PUBLIC",
"status" : "ACTIVE",
"projectId" : "95",
"userId" : 129,
"noteType" : "SYSTEM_GENERATED",
"projectDemographicLogId" : "1"
},
{
"_index" : "project_note",
"_type" : "project_note",
"_id" : "19",
"_score" : 1.0,
"_source" : {
"createTime" : "2021-10-04T13:43:55.330",
"createTimeInMs" : 1633333435330,
"createdBy" : "test",
"editTime" : "2021-10-04T13:43:55.330",
"editTimeInMs" : 1633333435330,
"editedBy" : "test",
"versionId" : 1,
"id" : "19",
"organizationId" : "28",
"accessLevel" : "PUBLIC",
"status" : "ACTIVE",
"projectId" : "95",
"userId" : 129
}
]
In the first doc, it has noteType but in the second, I don't have that field stored in db.
I want to exclude the documents where noteType==null or noteType is absent.
But, I am getting only the docs which have noteType="SYSTEM_GENERATED"
My approach:
{
"query":
{
"bool" : {
"must" : [
{
"term" : {
"projectId" : {
"value" : "95",
"boost" : 1.0
}
}
},
{
"range" : {
"createTimeInMs" : {
"from" : null,
"to" : 1633594455000,
"include_lower" : true,
"include_upper" : true,
"boost" : 1.0
}
}
}
],
"must_not" : [
{
"term" : {
"noteType" : {
"value" : "SYSTEM_GENERATED",
"boost" : 1.0
}
}
}
],
"adjust_pure_negative" : true,
"boost" : 1.0
}
}
}
Equivalent java code:
BoolQueryBuilder queryBuilder= QueryBuilders.boolQuery();
queryBuilder.must(QueryBuilders.termQuery("projectId", requestInfo.getProjectId()));
queryBuilder.must(rangeQuery("createTimeInMs").lte(requestInfo.getCreateTimeInMs()));
if(!requestInfo.isIncludeLog()) {
queryBuilder.mustNot(QueryBuilders.termQuery("noteType", Defs.SYSTEM_NOTE_TYPE));
}
If only the must_not part of the query is used (excluding the must part)
{
"query": {
"bool": {
"must_not": [
{
"term": {
"noteType.keyword": {
"value": "SYSTEM_GENERATED",
"boost": 1.0
}
}
}
],
"adjust_pure_negative": true,
"boost": 1.0
}
}
}
The search result is similar to what you expect to get
"hits": [
{
"_index": "69477995",
"_type": "_doc",
"_id": "2",
"_score": 0.0,
"_source": {
"createTime": "2021-09-26T15:54:08.373",
"createTimeInMs": 1632650048373,
"createdBy": "test",
"editTime": "2021-09-26T15:54:08.373",
"editTimeInMs": 1632650048373,
"editedBy": "test",
"versionId": 1,
"id": "18",
"note": "note-1, simple note ",
"organizationId": "28",
"accessLevel": "PUBLIC",
"status": "ACTIVE",
"taskId": "5",
"userId": 129
}
}
]
Here is my test case and I am individually pasing this junit it is passing but when I am doing maven build it is showing some encoding error in json file, Please check the error message I provide in below
#Test
public void testRemoveEmoji() throws Exception {
Items items = getSampleItems("/json/ItemObjectWithEmojiContent.json");
List<Item> itemList = items.getItemsList();
for(Item item : itemList) {
assertNotNull(item);
this.starcImportProcessor.removeEmojis(item);
assertEquals("(Open -> In Progress -> Open -> Closed)", item.getDescription());
for(ItemComment itemComments : item.getItemComments()) {
assertEquals("Test comment", itemComments.getComment());
}
}
getSampleItems code :
private Items getSampleItems(final String path) throws JsonParseException, JsonMappingException, IOException {
Items downloadedItems = new Items();
JSONArray inputWorkitem = new JSONArray(new String(getJSONFromFile(path)));
final ObjectMapper objectMapper = new ObjectMapper();
List<Item> items = objectMapper.readValue(inputWorkitem.toString(), new TypeReference<List<Item>>() {});
downloadedItems.setItemsList(items);
return downloadedItems;
}
private static byte[] getJSONFromFile(final String jsonFilePath) throws IOException {
Resource resource = new ClassPathResource(jsonFilePath);
File file = resource.getFile();
return Files.readAllBytes(file.toPath());
}
Here is json file :
[
{
"id" : 233565,
"name" : "Hafmap DaimlerBosch Test4 CRL Reject",
"createdAt" : "2020-09-29T16:07:23.918",
"modifiedAt" : "2021-07-08T19:03:12.477",
"version" : "15",
"description" : "(Open -> 😀In Progress -> Open -> Closed)",
"descriptionFormat" : "Wiki",
"assignedAt" : "2020-10-01T15:09:26.022",
"typeName" : "Bug",
"endDate" : "2020-10-08T14:40:00.000",
"closedAt" : "2020-10-08T14:40:34.385",
"ordinal" : 0,
"createdBy" : {
"id" : 1255,
"name" : "CHRBOOT",
"type" : "UserReference",
"email" : "christian.bootz#daimler.com",
"value" : null
},
"modifiedBy" : {
"id" : 3,
"name" : "computed.update",
"type" : "UserReference",
"email" : "codebeamer#intland.com",
"value" : null
},
"tracker" : {
"id" : 269438,
"name" : "Change Request Light",
"type" : "TrackerReference",
"email" : null,
"value" : null
},
"priority" : {
"id" : 0,
"name" : "Unset",
"type" : "ChoiceOptionReference",
"email" : null,
"value" : null
},
"status" : {
"id" : 7,
"name" : "Closed",
"type" : "ChoiceOptionReference",
"email" : null,
"value" : null
},
"severities" : [ {
"id" : 3,
"name" : "VoCA Prio 3",
"type" : "ChoiceOptionReference",
"email" : null,
"value" : null
} ],
"resolutions" : [ ],
"children" : [ ],
"subjects" : [ ],
"assignedTo" : [ {
"id" : 3463,
"name" : "PID8CBD",
"type" : "UserReference",
"email" : "christian.bootz#daimler.com",
"value" : null
} ],
"owners" : null,
"comments" : [ {
"id" : 675988,
"name" : "attachment mit leerzeichen.txt",
"type" : "CommentReference",
"email" : null,
"value" : null
}, {
"id" : 675990,
"name" : "Comment-1601992066937",
"type" : "CommentReference",
"email" : null,
"value" : null
}, {
"id" : 675991,
"name" : "Comment-1601992069782",
"type" : "CommentReference",
"email" : null,
"value" : null
} ],
"categories" : [ ],
"platforms" : [ ],
"customFields" : [ {
"fieldId" : 1001,
"name" : "Sync Allowed",
"values" : [ {
"id" : 1,
"name" : "Supplier External Tool",
"type" : "ChoiceOptionReference",
"email" : null,
"value" : null
} ],
"type" : "ChoiceFieldValue",
"value" : null
}, {
"fieldId" : 1003,
"name" : "Top Ten",
"values" : [ {
"id" : 2,
"name" : "No",
"type" : "ChoiceOptionReference",
"email" : null,
"value" : null
} ],
"type" : "ChoiceFieldValue",
"value" : null
}, {
"fieldId" : 1012,
"name" : "Supplier Status",
"values" : [ {
"id" : 7,
"name" : "Supplier Closed",
"type" : "ChoiceOptionReference",
"email" : null,
"value" : null
} ],
"type" : "ChoiceFieldValue",
"value" : null
}, {
"fieldId" : 1015,
"name" : "Verification By Test Group",
"values" : [ {
"id" : 2,
"name" : "No",
"type" : "ChoiceOptionReference",
"email" : null,
"value" : null
} ],
"type" : "ChoiceFieldValue",
"value" : null
}, {
"fieldId" : 1019,
"name" : "Closing Reason",
"values" : [ {
"id" : 6,
"name" : "No Defect / Issue",
"type" : "ChoiceOptionReference",
"email" : null,
"value" : null
} ],
"type" : "ChoiceFieldValue",
"value" : null
}, {
"fieldId" : 1020,
"name" : "Reject Reason",
"values" : [ {
"id" : 4,
"name" : "Not Reproducible",
"type" : "ChoiceOptionReference",
"email" : null,
"value" : null
} ],
"type" : "ChoiceFieldValue",
"value" : null
}, {
"fieldId" : 1028,
"name" : "Supplier",
"values" : [ {
"id" : 594222,
"name" : "Supplier_Bosch_MAP_ECU",
"type" : "UserGroupReference",
"email" : null,
"value" : null
} ],
"type" : "ChoiceFieldValue",
"value" : null
}, {
"fieldId" : 10000,
"name" : "DEFAULT_PERMISSION_1",
"values" : null,
"type" : "TextFieldValue",
"value" : "1"
}, {
"fieldId" : 10002,
"name" : "Created On Date",
"values" : null,
"type" : "DateFieldValue",
"value" : "2020-09-29T02:00:00.000"
}, {
"fieldId" : 10006,
"name" : "Domain",
"values" : null,
"type" : "TextFieldValue",
"value" : "Automated driving"
}, {
"fieldId" : 10007,
"name" : "Supplier ID",
"values" : null,
"type" : "TextFieldValue",
"value" : "DYMAPDAIEX-155919"
}, {
"fieldId" : 10024,
"name" : "Last Status Change",
"values" : null,
"type" : "DateFieldValue",
"value" : "2020-09-29T18:07:23.000"
}, {
"fieldId" : 10026,
"name" : "DEFAULT_PERMISSION_READONLY_INPROGRESS",
"values" : null,
"type" : "TextFieldValue",
"value" : "1"
} ],
"attachmentData" : [ {
"id" : 675988,
"name" : "attachment mit leerzeichen.txt",
"description" : null,
"descriptionFormat" : "PlainText",
"createdAt" : "2020-10-06T13:47:45.956",
"createdBy" : {
"id" : 3463,
"name" : "PID8CBD",
"type" : "UserReference",
"email" : "christian.bootz#daimler.com",
"value" : null
},
"modifiedAt" : "2020-10-06T13:47:45.956",
"modifiedBy" : {
"id" : 3463,
"name" : "PID8CBD",
"type" : "UserReference",
"email" : "christian.bootz#daimler.com",
"value" : null
},
"version" : 1,
"size" : 317,
"md5" : "A2DE48D274A08AD80D19E8829D0093D4",
"filePath" : null
} ],
"itemComments" : [ {
"id" : 675990,
"name" : "Comment-1601992066937",
"createdAt" : "2020-10-06T13:47:46.937",
"modifiedAt" : null,
"version" : "1",
"createdBy" : {
"id" : 3463,
"name" : "PID8CBD",
"type" : "UserReference",
"email" : "christian.bootz#daimler.com",
"value" : null
},
"modifiedBy" : null,
"comment" : "😀Test comment",
"commentFormat" : "PlainText",
"parent" : null,
"attachments" : [ ]
} ]
}
]
Here is the error when i am doing maven build here some encoding error it is showing:
Could you please help me out how to resolve this issue
testRemoveEmoji(com.bosch.edexas.starc.processor.STARCImportProcessorTest) Time elapsed: 0.008 sec <<< FAILURE!
org.junit.ComparisonFailure: expected:<(Open -> []In Progress -> Open ...> but was:<(Open -> **[😀]**In Progress -> Open ...>
at org.junit.Assert.assertEquals(Assert.java:125)
at org.junit.Assert.assertEquals(Assert.java:147)
at com.bosch.edexas.starc.processor.STARCImportProcessorTest.testRemoveEmoji(STARCImportProcessorTest.java:195)
OpenJDK 64-Bit Server VM warning: ignoring option UseSplitVerifier; support was removed in 8.0
Results :
Failed tests:
STARCImportProcessorTest.testRemoveEmoji:195 expected:<(Open -> []In Progress -> Open ...> but was:<(Open -> **[????]**In Progress -> Open ...>
Please do the following when you are trying to make a new String() when your getJSONFromFile(path) returns a byte[] array.
new String(getJSONFromFile(path), StandardCharsets.UTF_8);
I'm working with documents that contain music playlists.
Each document has this structure:
{
"user_id": "5858",
"playlists": [
{
"name": "My Playlist",
"guild_ids": ["7575"],
"items": [
{
"title": "title",
"url": "url",
"duration": 200000
}
]
}
]
}
I would like to extract all playlists from the same guild.
But the thing is that i'd like the results to be returned in a single document. One single document with a list of playlists.
The expected result for guild_id=5656 would be like this:
{
"playlists": [
{
"name": "My Playlist",
"guild_ids": ["5656"],
"items": [
{
"title": "title",
"url": "url",
"duration": 200000
}
]
},
// other playlists where guild_ids contains "5656"
]
}
I tried to use aggregation but i always get the same number of documents as the number of unique user_ids. I get the playlists grouped by user_id.
The following query can get us the expected output:
db.collection.aggregate([
{
$unwind:"$playlists"
},
{
$match:{
"playlists.guild_ids":{
$in:["7575"]
}
}
},
{
$group:{
"_id":null,
"playlists":{
$push: "$playlists"
}
}
},
{
$project:{
"_id":0
}
}
]).pretty()
Data set:
{
"_id" : ObjectId("5d88225e38db7cf8d3f75cd6"),
"user_id" : "5858",
"playlists" : [
{
"name" : "My Playlist",
"guild_ids" : [
"7575"
],
"items" : [
{
"title" : "title",
"url" : "url",
"duration" : 200000
}
]
}
]
}
{
"_id" : ObjectId("5d88225e38db7cf8d3f75cd7"),
"user_id" : "5858",
"playlists" : [
{
"name" : "My Playlist 2",
"guild_ids" : [
"1234"
],
"items" : [
{
"title" : "title",
"url" : "url",
"duration" : 200000
}
]
}
]
}
{
"_id" : ObjectId("5d88225e38db7cf8d3f75cd8"),
"user_id" : "5858",
"playlists" : [
{
"name" : "My Playlist 3",
"guild_ids" : [
"7575"
],
"items" : [
{
"title" : "title",
"url" : "url",
"duration" : 200000
}
]
}
]
}
Output:
{
"playlists" : [
{
"name" : "My Playlist",
"guild_ids" : [
"7575"
],
"items" : [
{
"title" : "title",
"url" : "url",
"duration" : 200000
}
]
},
{
"name" : "My Playlist 3",
"guild_ids" : [
"7575"
],
"items" : [
{
"title" : "title",
"url" : "url",
"duration" : 200000
}
]
}
]
}
Query analysis: We are unwinding the playlists, filtering only those which has 7575 guild ID and then grouping them all again.
I want to update request fields only in an array using java.This is my existing document in mongo db:
{
"_id": "6691e5068dwe335w42cb0a699650f",
"Opportunity_Owner": "Self",
"Account_Name": "IC",
"Lead_Source": "Callbox",
"Opportunity_Name": "name1 ",
"Stage": "Proposal",
"Stage_Status": "A",
"1555570551211": [],
"1555556165153": [],
"1555556059584": [{
"id": "1557389940585",
"Notes": "Note 1"
},
{
"id": "1557389945398",
"Notes": "Hi Bobby "
},
{
"id": "1557389978181",
"Notes": "Spoken to Bobby."
},
{
"id": "1557389990159",
"Notes": "plan to call on 29/Apr"
}
],
"createdBy": "2c18b8dbb7d74a41a66f53a90117480a",
"createdDate": "1562911250917"
}
Request payload:
{
"_id" : "6691e5068dwe335w42cb0a699650f",
"Stage_Status" : "I",
"1555556059584" : [
{
"id" : "1557389940585",
"Notes" : "updated note 123"
}
]
}
I am trying to update "Stage_Status" and "1555556059584.Notes" at a time using $set.I am able to update "Stage_Status" but "1555556059584" array is going to reset with what i have updated with last one.
expected output:
{
"_id" : "6691e5068dwe335w42cb0a699650f",
"Opportunity_Owner" : "Self",
"Account_Name" : "IC",
"Lead_Source" : "Callbox",
"Opportunity_Name" : "name1 ",
"Stage" : "Proposal",
"Stage_Status" : "I",
"1555570551211" : [],
"1555556165153" : [],
"1555556059584" : [
{
"id" : "1557389940585",
"Notes" : "updated note 123"
},
{
"id" : "1557389945398",
"Notes" : "Hi Bobby "
},
{
"id" : "1557389978181",
"Notes" : "Spoken to Bobby."
},
{
"id" : "1557389990159",
"Notes" : "plan to call on 29/Apr"
}
],
"createdBy" : "2c18b8dbb7d74a41a66f53a90117480a",
"createdDate" : "1562911250917"
}
can any one please help me to figure it out in java.
I guess you wanted to update Stage_Status and 1555556059584.Notes at Once .
here is a demo about it
> db.student.find()
{ "_id" : ObjectId("5d2c09ea8ed60ae70d3dd76b"), "name" : "bigbang", "courses" : [ { "name" : "en", "classRoom" : "9001" }, { "name" : "math", "classRoom" : "1001" } ] }
> db.student.update({name:'bigbang','courses.name':'en'},{ $set: {'courses.$.classRoom':'1009',name :"course"} })
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.student.find()
{ "_id" : ObjectId("5d2c09ea8ed60ae70d3dd76b"), "name" : "course", "courses" : [ { "name" : "en", "classRoom" : "1009" }, { "name" : "math", "classRoom" : "1001" } ] }
the java demo is like this
collection.updateOne(and(eq("Stage_Status","A"),eq("1555556059584.id","1557389940585")),new Document("$set" ,new Document("Stage_Status","YOUR_NEW_VALUE").append("1555556059584.$.Notes","YOUR_NEW_VALUE")));
you must set the 1555556059584.id to let the diver know which element to be update .
My mongodb records are like in this link Updating nested array inside array mongodb and sample records are as below and want to update a field in the nested document "parameter" array provided it satisfies some conditions (_id : "04", operations._id : "100" and operations.parameters.pid : "012"), this update query UPDATES wrong nested record (operations.parameters.pid : '011') , please help where I am going wrong:
{
"_id" : "04",
"name" : "test service 4",
"id" : "04",
"version" : "0.0.1",
"title" : "testing",
"description" : "test",
"protocol" : "test",
"operations" : [
{
"_id" : "99",
"oName" : "test op 52222222222",
"sid" : "04",
"name" : "test op 52222222222",
"oid" : "99",
"description" : "testing",
"returntype" : "test",
"parameters" : [
{
"oName" : "Param1",
"name" : "Param1",
"pid" : "011",
"type" : "582",
"description" : "testing",
"value" : "",
"version" : 1.0
},
{
"oName" : "Param2",
"name" : "Param2",
"pid" : "012",
"type" : "58222",
"description" : "testing",
"value" : "",
"version" : 2.0
}
]
},
{
"_id" : "100",
"oName" : "test op 909090",
"sid" : "05",
"name" : "test op 90909",
"oid" : "1009",
"description" : "testing",
"returntype" : "test",
"parameters" : [
{
"oName" : "Param1",
"name" : "Param1",
"pid" : "011",
"type" : "582",
"description" : "testing",
"value" : "",
"version" : 1.0
},
{
"oName" : "Param2",
"name" : "Param2",
"pid" : "012",
"type" : "58222",
"description" : "testing",
"value" : "",
"version" : 2.0
}
]
},
{
"_id" : "101",
"oName" : "test op 52222222222",
"sid" : "04",
"name" : "test op 52222222222",
"oid" : "99",
"description" : "testing",
"returntype" : "test",
"parameters" : [
{
"oName" : "Param1",
"name" : "Param1",
"pid" : "011",
"type" : "582",
"description" : "testing",
"value" : "",
"version" : 1.0
},
{
"oName" : "Param2",
"name" : "Param2",
"pid" : "012",
"type" : "58222",
"description" : "testing",
"value" : "",
"version" : 1.0
}
]
},
{
"_id" : "102",
"oName" : "test op 909090",
"sid" : "05",
"name" : "test op 90909",
"oid" : "1009",
"description" : "testing",
"returntype" : "test",
"parameters" : [
{
"oName" : "Param1",
"name" : "Param1",
"pid" : "011",
"type" : "582",
"description" : "testing",
"value" : "",
"version" : 1.0
},
{
"oName" : "Param2",
"name" : "Param2",
"pid" : "012",
"type" : "58222",
"description" : "testing",
"value" : "",
"version" : 2.0
}
]
}
]
}
My update query is as follows :
db.foo.update(
{ $and : [{'_id':'04'},
{'operations._id':'100' },
{'operations.parameters.pid': '012'}]},
{
"$set": {
"operations.1.parameters.$.dummy": "foo"
}
}
)
I am using mongodb 3.6.2 referred to https://docs.mongodb.com/master/reference/operator/update/positional-filtered/
Sample record from this link :
{
"_id" : 1.0,
"grades" : [
{
"type" : "quiz",
"questions" : [
10.0,
8.0,
5.0
]
},
{
"type" : "quiz",
"questions" : [
8.0,
9.0,
6.0
]
},
{
"type" : "hw",
"questions" : [
5.0,
4.0,
3.0
]
},
{
"type" : "exam",
"questions" : [
25.0,
10.0,
23.0,
0.0
]
}
]
}
Example from this link
db.student3.update(
{},
{ $inc: { "grades.$[t].questions.$[score]": 2 } },
{ arrayFilters: [ { "t.type": "quiz" } , { "score": { $gte: 8 } } ], multi: true}
)
ERror I got from robo-3t :
cannot use the part (grades of grades.$[t].questions.$[score]) to traverse the element ({grades: [ { type: "quiz", questions: [ 10.0, 8.0, 5.0 ] }, { type: "quiz", questions: [ 8.0, 9.0, 6.0 ] }, { type: "hw", questions: [ 5.0, 4.0, 3.0 ] }, { type: "exam", questions: [ 25.0, 10.0, 23.0, 0.0 ] } ]})
Please help;
Regards
Kris
In your update operation
"$set": {
"operations.1.parameters.$.dummy": "foo"
}
refers to the 1st element in operations that is an element with "_id" : "100", and within parameters array, the $ updates the first element in array.
You need to consider using mongodb 3.6 if you want to update nested array elements using $[] to update all matching elements.
One possible way to do this in 3.4 version is fetching the required sub-document and do matches and updates on the application side.