How do I get sum of a field in solr 4.8 - java

This is my response data:
"response": {
"numFound": 2,
"start": 0,
"docs": [
{
"total_amount": 10,
"id": "2"
},
{
"total_amount": 10,
"id": "1"
}
]
}
I want to get sum of total_amount. I tried facet query also. But I did't get sum. I got some blog on this but that is for solr 5.1. http://yonik.com/solr-facet-functions/


You can use the stats functionality to get this information. Just put the followed parameters in your query:
stats=true&stats.field=total_amount
Your response will be like that:
"response": {
"numFound": 2,
"start": 0,
"docs": [
{
"id": "1",
"total_amount": 15
},
{
"id": "2",
"total_amount": 12
}
]
},
"stats": {
"stats_fields": {
"total_amount": {
"min": 12,
"max": 15,
"count": 2,
"missing": 0,
"sum": 27,
"sumOfSquares": 369,
"mean": 13.5,
"stddev": 2.1213203435596424,
"facets": {}
}
}
Note that you have lots of information around the total_amount field including the sum.

Related

Group by with count in mongo db using spring data

I am new to mongo with spring data. i am trying to achive grouping for one of my data collection which is like below
[{
"id": 1,
"validNumber": true,
"validEmail": true
},
{
"id": 2,
"validNumber": false,
"validEmail": false
},
{
"id": 3,
"validNumber": true,
"validEmail": false
},
{
"id": 4,
"validNumber": false,
"validEmail": true
}
]
I am trying to group to above data set to get the total number of validNumber and valid emails . expected response should be something like below
{
"total": 4,
"validNumber": 2,
"inValidNumber": 2,
"validEmail": 1,
"inValidEmail": 3
}
I have written the below mongo query to achive the same
db.collection.aggregate([
{
"$group": {
"_id": null,
"total": {
"$sum": 1
},
"validEmail": {
"$sum": {
"$cond": [{
"$eq": ["$validEmail", true]
}, 1, 0]
}
},
"inValidEmail": {
"$sum": {
"$cond": [{
"$eq": ["$validEmail", false]
}, 1, 0]
}
},
"validNumber": {
"$sum": {
"$cond": [{
"$eq": ["$validNumber", true]
}, 1, 0]
}
},
"inValidNumber": {
"$sum": {
"$cond": [{
"$eq": ["$validNumber", false]
}, 1, 0]
}
}
}
},
{
"$project": {
_id: 0,
}
}
]);
How to implement the above query using Spring data JPA.
Thanks in advance
I tried to implement using the Aggregate function looking at the below example from here,but unable to add conditions as in my mongo query.
collection.aggregate(Aggregates.group("$stars", Accumulators.sum("count", 1))));

How to get a child object in response with java streams

Alrighty I've been banging my head in a wall whole day and cant solve this issue. I am trying to find an id in a object list which is like 3 levels down the hierarchy with java stream. I know how to do it with for loop but I need to get it with stream.
json response is
"NumberOfOwners": 1,
"CurrentPage": 1,
"TotalItems": 1,
"TotalPages": 1,
"PageSize": 1,
"PageItems": [
{
"Id": 1560,
"Title": "PlsWrk",
"IsSubmitted": true,
"Owner": {
"Branch": null,
"Position": null,
"Id": null,
"FirstName": null,
"LastName": null,
"ParentName": null,
"Gender": null,
"Image": null,
"LoginStatusId": 0,
"EmployeeStatus": 0,
"CompanyName": null,
"IsSubcontractor": false
},
"KeyResults": [
{
"Id": 5032,
"Title": "asdf1",
"OverallStatus": 2,
"MonthKeyResults": [
{
"Id": 12484,
"Month": 9,
"Status": 3,
"Progress": "REaplace1"
},
{
"Id": 12483,
"Month": 8,
"Status": 3,
"Progress": "sadf4"
},
{
"Id": 12482,
"Month": 7,
"Status": 1,
"Progress": "On Track1"
}
]
},
{
"Id": 5033,
"Title": "asdf2",
"OverallStatus": 1,
"MonthKeyResults": [
{
"Id": 12485,
"Month": 7,
"Status": 2,
"Progress": "Recovery2"
},
{
"Id": 12487,
"Month": 9,
"Status": 2,
"Progress": "asdfreas"
},
{
"Id": 12486,
"Month": 8,
"Status": 1,
"Progress": "asdf5"
}
]
},
{
"Id": 5034,
"Title": "asdf3",
"OverallStatus": 2,
"MonthKeyResults": [
{
"Id": 12490,
"Month": 9,
"Status": 1,
"Progress": "asdafa"
},
{
"Id": 12489,
"Month": 8,
"Status": 2,
"Progress": "asdf6"
},
{
"Id": 12488,
"Month": 7,
"Status": 3,
"Progress": "Replace3"
}
]
}
]
}
]
Precisely I want stream to return MonthyKeyResult object with a specific id
Atm I am here
public static MonthKeyResults getOkrMonthlyProgressById(PageItems okr, Integer monthlyProgressId){
//here i get KeyResults object list
List<KeyResults> keyResult = okr.getKeyResults().stream().collect(Collectors.toList());
//and now I am trying to get all MonthKeyResults
//objects but I not doing it right
List<MonthKeyResults> monthKeyResults = keyResult.stream().
filter(monthKeyResult -> monthKeyResult.
getMonthKeyResults().
stream().collect(Collectors.toList()));
//and then I am thinking of going trough the monthKeyResults
//list with stream and finding Id I need and returning that
//whole object with something like this
MonthKeyResults mKeyResults = monthKeyResults.stream().filter(id -> id.getId().
equals(monthlyProgressId)).findAny().orElse(null);
return mKeyResult
}
I got one more question
I've separated getting the final object as you see in 3 streams, is it possible to get it in one go or you need to separate this objects like this and go trough them separately?

Probably you're looking for something like:
return okr.getKeyResults().stream()
.flatMap(keyResult -> keyResult.getMonthKeyResults().stream())
.filter(monthKeyResult -> monthKeyResult.getId().equals(monthlyProgressId))
.findAny()
.orElse(null);
Edit: fixed typo.

Is it possible to turn an array of objects into a map during deserialisation using Gson?

Using the Steam API found here api.steampowered.com/IEconItems_440/GetPlayerItems/v0001/ I get a JSON string like the one below. As you can see, it contains an array of item objects. What I'd like to do is to turn this array into a Map where the defindex is the key and the value is the corresponding item object at point of deserialisation using GSON.
Is this possible or do I have to do that after it's created the objects and populated the array?
"result": {
"status": 1,
"num_backpack_slots": 800,
"items": [
{
"id": 12222222,
"original_id": 333333333,
"defindex": 45,
"level": 10,
"quality": 3,
"inventory": 2147483922,
"quantity": 1,
"origin": 0,
"style": 0,
"attributes": [
{
//...
]
},
{
"id": 3332222222,
"original_id": 554545465,
"defindex": 116,
"level": 10,
"quality": 6,
"inventory": 2147483865,
"quantity": 1,
"origin": 0,
"equipped": [
{
"class": 6,
"slot": 7
},
{
"class": 8,
"slot": 7
}
]
,
"style": 1,
"attributes": [
//...
]
},
{
"id": 4658518468,
"original_id": 897545164648,
"defindex": 130,
"level": 5,
"quality": 3,
"inventory": 2147484134,
"quantity": 1,
"origin": 0,
"attributes": [
{
//...
]
}
]

JSON Object changes randomly from api

I get data from an API in which the JSON object changes randomly, like if it is at "position": 1, now it will change randomly to "number": 1. So, how can I check in my application if the object is at "position": 1 or "number": 1 and use it as int?
JSON :-
{
"now": [{
"time": {
"starts_in": 0,
"ends_in": 79580,
"starts_at": 0,
"ends_at": "2018-01-21T08:00:00.788Z"
},
"coins": {
"free": 8,
"first_win": 16,
"max": 52,
"collected": 0
},
"unk1": -88317689,
"position": 1,
"xp_multiplier": 0,
"location_scid": {
"scid_type": 15,
"scid_id": 1
},
"tid": "TID_WANTED_3",
"location": "Terre",
"mode": {
"name": "Bty",
"color": "#0884FA",
"description": " The team wins!"
},
"unk4": 0,
"info": "",
"unk5": 0,
"unk6": 0
}, {
"time": {
"starts_in": 0,
"ends_in": 36380,
"starts_at": 0,
"ends_at": "2018-01-20T20:00:00.788Z"
},
"coins": {
"free": 24,
"first_win": 0,
"max": 32,
"collected": 0
}
}],
"later": [{
"time": {
"starts_in": 79580,
"ends_in": 165980,
"starts_at": "2018-01-21T08:00:00.788Z",
"ends_at": "2018-01-22T08:00:00.788Z"
},
"coins": {
"free": 8,
"first_win": 16,
"max": 52,
"collected": 0
},
"unk1": -88217689,
"position": 1,
"xp_multiplier": 0,
"location_scid": {
"scid_type": 15,
"scid_id": 7
},
"tid": "TID_GOLDRUSH_1",
"location": "Mine",
"mode": {
"name": "Grab",
"color": "#AA57CF",
"description": " An. "
}
}]
}
Thanks in advance :)

JSONObject c = //Your jsonObject;
String position = c.getInt("position");
String number = c.getInt("number");
if(position!=null){
//TODO You know it is position and it's int value
}else if(number!=null){
//TODO You know it is number and it's int value
}else{
//TODO Its neither of two
}

If you're using Gson to convert your JSON to a class, you can use both position and number as attributes for your destination class.
After that, check which one is null and which one is not and use that is not null as your number.

Just try this one,
JSONObject object = (Your jsonObject);
if(object.has("position")){
** do your code here **
}else if(object.has("number"){
** do your code here **
}

Elastic search cross fields, edge ngram analyzer

I have 999 documents which I am using for experimenting with elastic search.
There is a field f4 in my type mapping which is analyzed and has following settings for analyzer :
"myNGramAnalyzer" => [
"type" => "custom",
"char_filter" => ["html_strip"],
"tokenizer" => "standard",
"filter" => ["lowercase","standard","asciifolding","stop","snowball","ngram_filter"]
]
My filter is as below :
"filter" => [
"ngram_filter" => [
"type" => "edgeNGram",
"min_gram" => "2",
"max_gram" => "20"
]
]
I have value for field f4 as "Proj1", "Proj2", "Proj3"...... so on.
Now when I try to do search using cross fields for "proj1" string, I was expecting document with "Proj1" to be returned at the top of the response with max score. But it doesn't. Rest all the data is almost same in content.
Also I don't understand why it matches all 999 document?
Following is my search :
{
"index": "myindex",
"type": "mytype",
"body": {
"query": {
"multi_match": {
"query": "proj1",
"type": "cross_fields",
"operator": "and",
"fields": "f*"
}
},
"filter": {
"term": {
"deleted": "0"
}
}
}
}
My search response is :
{
"took": 12,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 999,
"max_score": 1,
"hits": [{
"_index": "myindex",
"_type": "mytype",
"_id": "42",
"_score": 1,
"_source": {
"f1": "396","f2": "125650","f3": "BH.1511AI.001",
"f4": "Proj42",
"f5": "BH.1511AI.001","f6": "","f7": "","f8": "","f9": "","f10": "","f11": "","f12": "","f13": "","f14": "","f15": "","f16": "09/05/16 | 01:02PM | User","deleted": "0"
}
}, {
"_index": "myindex",
"_type": "mytype",
"_id": "47",
"_score": 1,
"_source": {
"f1": "396","f2": "137946","f3": "BH.152096.001",
"f4": "Proj47",
"f5": "BH.1511AI.001","f6": "","f7": "","f8": "","f9": "","f10": "","f11": "","f12": "","f13": "","f14": "","f15": "","f16": "09/05/16 | 01:02PM | User","deleted": "0"
}
},
//.......
//.......
//MANY RECORDS IN BETWEEN HERE
//.......
//.......
{
"_index": myindex,
"_type": "mytype",
"_id": "1",
"_score": 1,
"_source": {
"f1": "396","f2": "142095","f3": "BH.705215.001",
"f4": "Proj1",
"f5": "BH.1511AI.001","f6": "","f7": "","f8": "","f9": "","f10": "","f11": "","f12": "","f13": "","f14": "","f15": "","f16": "09/05/16 | 01:02PM | User","deleted": "0"
}
//.......
//.......
//MANY RECORDS IN BETWEEN HERE
//.......
//.......
}]
}
}
Any thing that I am doing wrong or missing? (Apologies for lengthy question, but I thought to give all possible information discarding unnecessary other code).
EDITED :
Term vector response
{
"_index": "myindex",
"_type": "mytype",
"_id": "10",
"_version": 1,
"found": true,
"took": 9,
"term_vectors": {
"f4": {
"field_statistics": {
"sum_doc_freq": 5886,
"doc_count": 999,
"sum_ttf": 5886
},
"terms": {
"pr": {
"doc_freq": 999,
"ttf": 999,
"term_freq": 1,
"tokens": [{
"position": 0,
"start_offset": 0,
"end_offset": 6
}]
},
"pro": {
"doc_freq": 999,
"ttf": 999,
"term_freq": 1,
"tokens": [{
"position": 0,
"start_offset": 0,
"end_offset": 6
}]
},
"proj": {
"doc_freq": 999,
"ttf": 999,
"term_freq": 1,
"tokens": [{
"position": 0,
"start_offset": 0,
"end_offset": 6
}]
},
"proj1": {
"doc_freq": 111,
"ttf": 111,
"term_freq": 1,
"tokens": [{
"position": 0,
"start_offset": 0,
"end_offset": 6
}]
},
"proj10": {
"doc_freq": 11,
"ttf": 11,
"term_freq": 1,
"tokens": [{
"position": 0,
"start_offset": 0,
"end_offset": 6
}]
}
}
}
}
}
EDITED 2
Mappings for field f4
"f4" : {
"type" : "string",
"index_analyzer" : "myNGramAnalyzer",
"search_analyzer" : "standard"
}
I have updated to use standard analyzer for query time, which has improved the results but still not what I expected.
Instead of 999 (all documents) now it return 111 documents like "Proj1", "Proj11", "Proj111"......"Proj1", "Proj181"......... etc.
Still "Proj1" is in between the results and not at the top.

There is no index_analyzer (at least not from Elasticsearch version 1.7). For mapping parameters you can use analyzer and search_analyzer.
Try the following steps in order to make it work.
Create myindex with analyzer settings:
PUT /myindex
{
"settings": {
"analysis": {
"filter": {
"ngram_filter": {
"type": "edge_ngram",
"min_gram": 2,
"max_gram": 20
}
},
"analyzer": {
"myNGramAnalyzer": {
"type": "custom",
"tokenizer": "standard",
"char_filter": "html_strip",
"filter": [
"lowercase",
"standard",
"asciifolding",
"stop",
"snowball",
"ngram_filter"
]
}
}
}
}
}
Add mappings to mytype (to make it short I just mapped the relevant fields):
PUT /myindex/_mapping/mytype
{
"properties": {
"f1": {
"type": "string"
},
"f4": {
"type": "string",
"analyzer": "myNGramAnalyzer",
"search_analyzer": "standard"
},
"deleted": {
"type": "string"
}
}
}
Index some data:
PUT myindex/mytype/1
{
"f1":"396",
"f4":"Proj12" ,
"deleted": "0"
}
PUT myindex/mytype/2
{
"f1":"42",
"f4":"Proj22" ,
"deleted": "1"
}
Now try your query:
GET myindex/mytype/_search
{
"query": {
"multi_match": {
"query": "proj1",
"type": "cross_fields",
"operator": "and",
"fields": "f*"
}
},
"filter": {
"term": {
"deleted": "0"
}
}
}
It should return document #1. It worked for me with Sense. I am using Elasticsearch 2.X versions.
Hope I have managed to help :)

After hours of spending time to find a solution to this, I finally made it work.
So I kept everything same as mentioned in my question, using n gram analzyer while indexing data. The only thing I had to change was, to use the all field in my search query as a bool query with my existing multi-match query.
Now my result for search text Proj1 would return me results in an order such as Proj1, Proj121, Proj11, etc.
Although this does not return the exact order like Proj1, Proj11, Proj121, etc, but still it closely resembles the result that I wanted.

Categories

Resources