Develop Reference

compare and add the list of elements into another list

I am trying to compare and add the list of objects inside another list using java8 stream API.
Below are the sample json.
studentList:
[{
sid:1,
id:10,
sname :"xyz"
sPassOut : "2019"
}, {
sid:2,
id:20,
sname :"abc"
sPassOut : "2020"
},{
sid:3,
id:30,
sname :"ppp"
sPassOut : "2021"
},
{
sid:1,
id:10,
sname :"ppp"
sPassOut : "2021"
},
{
sid:2,
id:20,
sname :"ppp"
sPassOut : "2021"
}]
myList:
[{
id:10,
status:"active",
studentList:[]
},{id:20,
status:"active",
studentList:[]
},{id:30,
status:"active",
studentList:[]
},{id:40,
status:"active",
studentList:[]
}]
I want to add the studentList elements in the myList for each matching id value.
tried the below code, but it is not giving the expected result.
studentList.forEach(studList -> {
studList.setStudentList(myList.stream().collect(Collectors.toMap(MyList:: getId)));
});
Expected JSON:
[
{
id: 10,
status: "active",
studentList: [
{
sid: 1,
id: 10,
sname : "xyz"
sPassOut : "2019"
},
{
sid: 4,
id: 10,
sname : "ppp"
sPassOut : "2021"
}
]
},
{id: 20,
status: "active",
studentList: [
{
sid: 2,
id: 20,
sname : "abc"
sPassOut : "2020"
},
{
sid: 5,
id: 20,
sname : "ppp"
sPassOut : "2021"
}
]
},
{id: 30,
status: "active",
studentList: [
{
sid: 3,
id: 30,
sname : "ppp"
sPassOut : "2021"
}
]
},
{id: 40,
status: "active",
studentList: []
}
]
I wan to add the elements from studentList grouped together with matching id value in the myList as shown in the sample json above.

In order to get the required student list using the stream api, you need to filter the students to get only students with a certain id and then collect them into a list. An example of this is shown below:
studentList.stream().filter(student -> student.id == ID_TO_MATCH).collect(Collectors.toList())
In the code above, ID_TO_MATCH represents the the student id to find for each section of myList. See the following example of getting ID_TO_MATCH:
myList.forEach(section -> {
int ID_TO_MATCH = section.id;
};
These code snippets can be combined to produce the solution:
myList.forEach(section -> {
int id = section.id;
List<Student> studentsFound = studentList.stream().filter(student -> student.id == id).collect(Collectors.toList())
section.setStudentList(studentsFound);
};

Search match text in Elasticsearch SpringBoot by using percentage

I'm a new Elasticsearch SpringBoot here. I don't know how to search match text in Elasticsearch SpringBoot by using percentage. For example, I have a text "Hello world". Can I set a percentage of 50% or 70% to match with my text? I try with property minimumShouldMatch already but it seems doesn't work for my case right now.
Anyone help me please, Thank

You could use should query, split your search phrase by term, and set minimum_should_match according to your pourcentage
Example query
{
"query": {
"bool": {
"should": [
{
"term": {
"my_field": "hello"
}
},
{
"term": {
"my_field": "world"
}
},
{
"term": {
"my_field": "i'm"
}
},
{
"term": {
"my_field": "alive"
}
}
],
"minimum_should_match": 2
}
}
}
Will find hello world, hello alive etc...
To split a text in terms you should use _analyse of your index
Analyze and split terms
POST myindex/_analyze
{
"field": "my_field",
"text": "hello world i'm alive"
}
Which give you result like that to populate your query, and match term analyser with the query, if for example you use custom analyzer
{
"tokens" : [
{
"token" : "hello",
"start_offset" : 0,
"end_offset" : 5,
"type" : "<ALPHANUM>",
"position" : 0
},
{
"token" : "world",
"start_offset" : 6,
"end_offset" : 11,
"type" : "<ALPHANUM>",
"position" : 1
},
{
"token" : "i'm",
"start_offset" : 12,
"end_offset" : 15,
"type" : "<ALPHANUM>",
"position" : 2
},
{
"token" : "alive",
"start_offset" : 16,
"end_offset" : 21,
"type" : "<ALPHANUM>",
"position" : 3
}
]
}

How to write a MongoDB lookup query between 2 collections using Java?

For example, create an example collection classes with the following document:
db.classes.insert( [
{ _id: 1, title: "Reading is ...", enrollmentlist: [ "giraffe2", "pandabear", "artie" ], days: ["M", "W", "F"] },
{ _id: 2, title: "But Writing ...", enrollmentlist: [ "giraffe1", "artie" ], days: ["T", "F"] }
])
Create another collection members with the following documents:
db.members.insert( [
{ _id: 1, name: "artie", joined: new Date("2016-05-01"), status: "A" },
{ _id: 2, name: "giraffe", joined: new Date("2017-05-01"), status: "D" },
{ _id: 3, name: "giraffe1", joined: new Date("2017-10-01"), status: "A" },
{ _id: 4, name: "panda", joined: new Date("2018-10-11"), status: "A" },
{ _id: 5, name: "pandabear", joined: new Date("2018-12-01"), status: "A" },
{ _id: 6, name: "giraffe2", joined: new Date("2018-12-01"), status: "D" }
])
The following aggregation operation joins documents in the classes collection with the members collection, matching on the members field to the name field:
db.classes.aggregate([
{
$lookup:
{
from: "members",
localField: "enrollmentlist",
foreignField: "name",
as: "enrollee_info"
}
}
])
The operation returns the following:
{
"_id" : 1,
"title" : "Reading is ...",
"enrollmentlist" : [ "giraffe2", "pandabear", "artie" ],
"days" : [ "M", "W", "F" ],
"enrollee_info" : [
{ "_id" : 1, "name" : "artie", "joined" : ISODate("2016-05-01T00:00:00Z"), "status" : "A" },
{ "_id" : 5, "name" : "pandabear", "joined" : ISODate("2018-12-01T00:00:00Z"), "status" : "A" },
{ "_id" : 6, "name" : "giraffe2", "joined" : ISODate("2018-12-01T00:00:00Z"), "status" : "D" }
]
}
{
"_id" : 2,
"title" : "But Writing ...",
"enrollmentlist" : [ "giraffe1", "artie" ],
"days" : [ "T", "F" ],
"enrollee_info" : [
{ "_id" : 1, "name" : "artie", "joined" : ISODate("2016-05-01T00:00:00Z"), "status" : "A" },
{ "_id" : 3, "name" : "giraffe1", "joined" : ISODate("2017-10-01T00:00:00Z"), "status" : "A" }
]
}
How would you write this in Java with MongoDB?

Elasticsearch composite group by queries across the documents

We have an elastic search document which has a dimension called city. Each document will have only one value for city field. I have a scenario where I need to query the person based on the city or cities.
Documents in Elasticsearch
{
person_id: "1",
property_value : 25000,
city: "Bangalore"
}
{
person_id: "2",
property_value : 100000,
city: "Bangalore"
}
{
person_id: "1",
property_value : 15000,
city: "Delhi"
}
Note: The aggregation should be performed on property_value and group by on person_id.
For eg.,
If I query for Bangalore it should return document with person_id 1 and 2.
If I query for both Delhi and Bangalore it should return this
{
person_id: "1",
property_value : 40000,
city: ["Bangalore", "Delhi"]
}

Looking at your data, I've come up with a sample mapping, request query and the response.
Mapping:
PUT my_index_city
{
"mappings": {
"properties": {
"person_id":{
"type": "keyword"
},
"city":{
"type":"text",
"fields":{
"keyword":{
"type": "keyword"
}
}
},
"property_value":{
"type": "long"
}
}
}
}
Sample Request:
Note that I've made use of simple query string to filter the documents having Bangalore and Delhi.
For aggregation I've made use of Terms Aggregation on person_id and Sum Aggregation on the property_value field.
POST my_index_city/_search
{
"size": 0,
"query": {
"query_string": {
"default_field": "city",
"query": "Bangalore Delhi"
}
},
"aggs": {
"my_person": {
"terms": {
"field": "person_id",
"size": 10,
"min_doc_count": 2
},
"aggs": {
"sum_property_value": {
"sum": {
"field": "property_value"
}
}
}
}
}
}
Sample Response:
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"my_person" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "1",
"doc_count" : 2,
"sum_property_value" : {
"value" : 40000.0
}
}
]
}
}
}
Note: This query would only work if the person_id has multiple documents but each document having unique/different city value.
What I mean to say is, if the person_id has multiple documents with same city, the aggregation would not give right answer.
Updated Answer:
There is no direct way to achieve what you are looking for unless you modify the mapping. What I've done is, made use of nested datatype and ingested all the documents for person_id as a single document.
Mapping:
PUT my_sample_city_index
{
"mappings": {
"properties": {
"person_id":{
"type": "keyword"
},
"property_details":{
"type":"nested", <------ Note this
"properties": {
"city":{
"type": "text",
"fields":{
"keyword":{
"type":"keyword"
}
}
},
"property_value":{
"type": "long"
}
}
}
}
}
}
Sample Documents:
POST my_sample_city_index/_doc/1
{
"person_id": "1",
"property_details":[
{
"property_value" : 25000,
"city": "Bangalore"
},
{
"property_value" : 15000,
"city": "Delhi"
}
]
}
POST my_sample_city_index/_doc/2
{
"person_id": "2",
"property_details":[
{
"property_value" : 100000,
"city": "Bangalore"
}
]
}
Aggregation Query:
POST my_sample_city_index/_search
{
"size": 0,
"query": {
"nested": {
"path": "property_details",
"query": {
"query_string": {
"default_field": "property_details.city",
"query": "bangalore delhi"
}
}
}
},
"aggs": {
"persons": {
"terms": {
"field": "person_id",
"size": 10
},
"aggs": {
"property_sum": {
"nested": { <------ Note this
"path": "property_details"
},
"aggs": {
"total_sum": {
"sum": {
"field": "property_details.property_value"
}
}
}
}
}
}
}
}
Note that I've applied initially a term query on person_id post which I've applied Nested Aggregation, further on which I've applied metric sum aggregation query.
This should also work correctly if a person has multiple properties in the same city.
Response:
{
"took" : 31,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"persons" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "1",
"doc_count" : 1,
"property_sum" : {
"doc_count" : 2,
"total_sum" : {
"value" : 40000.0
}
}
},
{
"key" : "2",
"doc_count" : 1,
"property_sum" : {
"doc_count" : 1,
"total_sum" : {
"value" : 100000.0
}
}
}
]
}
}
}
Let me know if this helps!

How to use MongoDB Java driver to group by dayOfYear on ISODate attributes?

How to use mongodb java driver to compare dayOfYear of two ISODate objects?
Here are my docs
{"name": "hello", "count": 4, "TIMESTAMP": ISODate("2017-10-02T02:00:35.098Z")}
{"name": "hello", "count": 5, "TIMESTAMP": ISODate("2017-10-02T02:00:35.098Z")}
{"name": "goodbye", "count": 6, "TIMESTAMP": ISODate("2017-10-01T02:00:35.098Z")}
{"name": "foo", "count": 6, "TIMESTAMP": ISODate("2017-10-02T02:00:35.098Z")}
I want to compare the day in "TIMESTAMP" to perform some aggregation
Bson match = Aggregates.match(eq("name": "hello"));
Bson group = Aggregates.group(new Document("name", "$name"), Accumulators.sum("total", 1));
collection.aggregate(Arrays.asList(match, group))
Now I am not sure how to do this aggregation for all the records that belongs to particular day?
so my expected result for "2017-10-02" is
[{"_id": {"name":"hello"}, "total": 9}, {"_id": {"name":"foo"}, "total": 6}]

Given the following documents:
{"name": "hello", "count": 4, "TIMESTAMP": ISODate("2017-10-02T02:00:35.098Z")}
{"name": "hello", "count": 5, "TIMESTAMP": ISODate("2017-10-02T02:00:35.098Z")}
{"name": "goodbye", "count": 6, "TIMESTAMP": ISODate("2017-10-01T02:00:35.098Z")}
{"name": "foo", "count": 6, "TIMESTAMP": ISODate("2017-10-02T02:00:35.098Z")}
The following command ...
db.getCollection('dayOfYear').aggregate([
// project dayOfYear as an attribute
{ $project: { name: 1, count: 1, dayOfYear: { $dayOfYear: "$TIMESTAMP" } } },
// match documents with dayOfYear=275
{ $match: { dayOfYear: 275 } },
// sum the count attribute for the selected day and name
{ $group : { _id : { name: "$name" }, total: { $sum: "$count" } } }
])
... will return:
{
"_id" : {
"name" : "foo"
},
"total" : 6
}
{
"_id" : {
"name" : "hello"
},
"total" : 9
}
I think this meets the requirement expressed in your OP.
Here's the same command expressed using the MongoDB Java driver:
MongoCollection<Document> collection = mongoClient.getDatabase("stackoverflow").getCollection("dayOfYear");
Document project = new Document("name", 1)
.append("count", 1)
.append("dayOfYear", new Document("$dayOfYear", "$TIMESTAMP"));
Document dayOfYearMatch = new Document("dayOfYear", 275);
Document grouping = new Document("_id", "$name").append("total", new Document("$sum", "$count"));
AggregateIterable<Document> documents = collection.aggregate(Arrays.asList(
new Document("$project", project),
new Document("$match", dayOfYearMatch),
new Document("$group", grouping)
));
for (Document document : documents) {
logger.info("{}", document.toJson());
}
Update based on this comment:
One of the problems with project is that it only include fields you specify . The above input is just an example. I have 100 fields in my doc I can't sepecify every single one so if I use project I have to specify all 100 fields in addition to "dayOfYear" field. – user1870400 11 mins ago
You can use the following command to return the same output but without a $project stage:
db.getCollection('dayOfYear').aggregate([
// ignore any documents which do not match dayOfYear=275
{ "$redact": {
"$cond": {
if: { $eq: [ { $dayOfYear: "$TIMESTAMP" }, 275 ] },
"then": "$$KEEP",
"else": "$$PRUNE"
}
}},
// sum the count attribute for the selected day
{ $group : { _id : { name: "$name" }, total: { $sum: "$count" } } }
])
Here's that command in its 'Java form':
MongoCollection<Document> collection = mongoClient.getDatabase("stackoverflow").getCollection("dayOfYear");
Document redact = new Document("$cond", new Document("if", new Document("$eq", Arrays.asList(new Document("$dayOfYear", "$TIMESTAMP"), 275)))
.append("then", "$$KEEP")
.append("else", "$$PRUNE"));
Document grouping = new Document("_id", "$name").append("total", new Document("$sum", "$count"));
AggregateIterable<Document> documents = collection.aggregate(Arrays.asList(
new Document("$redact", redact),
new Document("$group", grouping)
));
for (Document document : documents) {
logger.info("{}", document.toJson());
}
Note: Depending on the size of your collection/your non functional requirements/etc you may want to consider the performance of these solutions and either (a) add a match stage before you start projecting/redacting or (b) extract dayOfYear into its own attribute so that you can avoid this complexity entirely.