I created an Index for a few documents. However I am facing trouble while running for a search query as it is showing zero hits. Find the indexing file, indexed data and and Java code below:
Mapping.json:
{
"settings": {
"index": {
"max_ngram_diff": 39
},
"analysis": {
"normalizer": {
"custom_normalizer": {
"type": "custom",
"char_filter": [],
"filter": [
"lowercase",
"asciifolding"
]
}
},
"analyzer": {
"custom_analyzer": {
"tokenizer": "custom_tokenizer",
"filter": [
"lowercase"
]
},
"autocomplete_search": {
"type": "custom",
"tokenizer": "keyword",
"filter": "lowercase"
}
},
"tokenizer": {
"custom_tokenizer": {
"type": "ngram",
"min_gram": 1,
"max_gram": 40,
"token_chars": [
"letter",
"digit",
"whitespace",
"punctuation",
"symbol"
]
}
}
}
},
"mappings": {
"type_name": {
"properties": {
"primaryTerm": {
"type": "text",
"analyzer": "custom_analyzer",
"search_analyzer": "autocomplete_search",
"fielddata": "true",
"fields": {
"raw": {
"type": "keyword",
"normalizer": "custom_normalizer"
}
}
},
"entityType": {
"type": "keyword",
"normalizer": "custom_normalizer"
},
"variants": {
"type": "text",
"analyzer": "custom_analyzer",
"search_analyzer": "autocomplete_search",
"fielddata": "true",
"fields": {
"raw": {
"type": "keyword",
"normalizer": "custom_normalizer"
}
}
}
}
}
}
}
Elastic data.json:
{
"took": 82,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 4,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "relmap",
"_type": "type_name",
"_id": "X4NutXcBKX-KYFzehi4s",
"_score": 1,
"_source": {
"entityType": "Agency",
"primaryTerm": "K2IPR INTELLECTUAL PROPERTY ATTORNEYS",
"variants": [
"K2IPR",
"K2I.P.R. (PATENT & TRADEMARK ATTRONEYS)",
"K2I.P.R",
"K2.I.P.R",
"K2 I.P.R",
"K2 I.P.R",
"K2 IPR",
"K2 I. P. R."
]
}
},
{
"_index": "relmap",
"_type": "type_name",
"_id": "YINutXcBKX-KYFzeoy7V",
"_score": 1,
"_source": {
"entityType": "Agency",
"primaryTerm": "EPIPHANY IP SOLUTIONS PVT. LTD.",
"variants": [
"EPIPHANY IP SOLUTIONS PRIVATE LIMITED",
"EPIPHANY IP SOLUTIONS PVT. LTD"
]
}
},
{
"_index": "relmap",
"_type": "type_name",
"_id": "YYNutXcBKX-KYFzepC44",
"_score": 1,
"_source": {
"entityType": "Agency",
"primaryTerm": "Natural Remedies Private Limited",
"variants": [
"NATURAL REMEDIES PVT. LTD",
"NATURAL REMEDIES PVT LTD",
"NATURAL REMEDIES"
]
}
},
{
"_index": "relmap",
"_type": "type_name",
"_id": "YoNutXcBKX-KYFzepC5i",
"_score": 1,
"_source": {
"entityType": "Agency",
"primaryTerm": "NICHE, INTELLECTUAL PROPERTY OFFICES",
"variants": [
"NICHE INTELLECTUAL PROPERTY OFFICES",
"NICHE INTELLECTUAL PROPERTY OFFICECS",
"NICHE INTELLECTUAL PROPERTY LAW OFFICES"
]
}
}
]
}
}
Search query:
String query="{\"query_string\":{\"query\":\""+searchTerm.toUpperCase()+"\",\"default_operator\":\"AND\",\"default_field\":\"primaryTerm\"}}";
Java Code for elastic Query:
SearchResponse searchResponse=null;
try {
String queryJson=json;
System.out.println("queryJson:"+queryJson);
//System.out.println(indexName);
SearchRequest searchRequest = new SearchRequest(indexName);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.from(start)
.size(length)
.query(QueryBuilders.wrapperQuery(queryJson));
searchRequest.source(searchSourceBuilder);
searchResponse = elasticsearchRestTemplate.getClient().search(searchRequest, RequestOptions.DEFAULT);
For a Search Term = 'Ni' in an autosuggest drpDown I am getting zero hits for the field 'primaryTerm'
I also tried the termQuerybuilder instead of wrapper query:
SearchRequestBuilder searchRequestBuilder = client.prepareSearch("relmap");
SearchResponse searchResponse = searchRequestBuilder.
setQuery(QueryBuilders.boolQuery().must(QueryBuilders.termQuery("variants", "NATURAL REMEDIES PVT. LTD"))).setFrom(0).setSize(1000).setExplain(true).setTrackTotalHits(true).execute().actionGet();
SearchHits searchHits = searchResponse.getHits();
Interestinlgy I am getting the desired output with elastic url.
Elastic URI code:
http://localhost:9201/relmap/_search?q=NIC&df=primaryTerm&explain=true&default_operator=AND
Can anyone help let me know the fault /correct way to get the hits from the query
Related
Here is the object cuptured with kibana, which should be returned with java api
{
"_index": "backend:nips-000320",
"_type": "_doc",
"_id": "qhcOVoQBFSMoMXEJjh46",
"_version": 1,
"_score": 0,
"_source": {
"severity": 5,
"priority": 37,
"#geo": {
"Source": {
"Continent": {
"Code": "EU",
"GeoNameID": 6255148,
"Name": "Europe"
},
"Country": {
"IsInEuropeanUnion": false,
"IsoCode": "UA",
"GeoNameID": 690791,
"Name": "Ukraine"
}
}
},
"timestamp": "2022-11-08T07:04:52Z"
},
"fields": {}
}
The problem is that I can't find a way to do this kind of nested query with java code and get this:
"#geo"."Source"."Country"."IsoCode" grouped by the value of "IsoCode".
like this :
[\{"iso_code": "US", "count": 15}, \{"iso_code": "AE", "count": 102}, {}...]
I am trying with QueryBuilder api, but no results yet. Any ideas would be appreciated.
I want to get the max and min value of the emp salary based on currency. every employee has a salary range based on currency also all the details in response should be unique. while I am using aggregation function min and max but it fetches the max and min value of salary amount but I need to get max and min based on currency field.
Sample Data:
[
{
"id": "1",
"emp_name": "emp1",
"data": [
{
"emp_country": "country1",
"emp_city": "city1",
"salary": [
{
"currency": "INR",
"amount": 5000
},
{
"currency": "INR",
"amount": 600
},
{
"currency": "MXN",
"amount": 400
}
]
},
{
"emp_country": "country1",
"emp_city": "city2",
"salary": [
{
"currency": "DOLLER",
"amount": 5000
},
{
"currency": "DOLLER",
"amount": 200
},
{
"currency": "MXN",
"amount": 400
}
]
}
]
},
{
"id": "2",
"emp_name": "emp2",
"data": [
{
"emp_country": "country2",
"emp_city": "city2",
"salary": [
{
"currency": "INR",
"amount": 5000
},
{
"currency": "MXN",
"amount": 200
},
{
"currency": "MXN",
"amount": 400
}
]
}
]
},
{
"id": "3",
"emp_name": "emp3",
"data": [
{
"emp_country": "country1",
"emp_city": "city1",
"salary": [
{
"currency": "MXN",
"amount": 400
}
]
}
]
},
{
"id": "4",
"emp_name": "emp4",
"data": [
{
"emp_country": "country1",
"emp_city": "city2",
"salary": [
{
"currency": "DOLLER",
"amount": 200
}
]
}
]
}
]
Expected Output:
city, country, the name should be unique and salary have max and min based on currency.
[
{
"emp_city": "city1",
"emp_country": "country1",
"emp_name": "emp1",
"emp_salary": [{
"currency": "INR",
"max": 5000,
"min": 600
},
{
"currency": "MXN",
"max": 400,
"min": 400
}]
},
{
"emp_city": "city2",
"emp_country": "country1",
"emp_name": "emp1",
"emp_salary":[ {
"currency": "DOLLER",
"max": 5000,
"min": 200
},
{
"currency": "MXN",
"max": 400,
"min": 400
}]
},
{
"emp_city": "city2",
"emp_country": "country2",
"emp_name": "emp2",
"emp_salary": [{
"currency": "INR",
"max": 5000,
"min": 5000
},
{
"currency": "MXN",
"max": 400,
"min": 200
}]
},
{
"emp_city": "city1",
"emp_country": "country1",
"emp_name": "emp3",
"emp_salary": [{
"currency": "MXN",
"max": 400,
"min": 400
}]
},
{
"emp_city": "city2",
"emp_country": "country1",
"emp_name": "emp4",
"emp_salary": [{
"currency": "DOLLER",
"max": 200,
"min": 200
}]
}
]
We can achieve the above result using an aggregation query.
$unwind - to deconstruct array of salary
$group - group by emp city, country, name, currency with min and max value of salary
$group - use another to push in emp_salary
$project - to show fields in your format
db.collection.aggregate([
{
"$unwind": {
"path": "$data"
}
},
{
"$unwind": {
"path": "$data.salary"
}
},
{
"$group": {
"_id": {
"emp_city": "$data.emp_city",
"emp_country": "$data.emp_country",
"emp_name": "$emp_name",
"currency": "$data.salary.currency",
},
"max": {
"$max": "$data.salary.amount"
},
"min": {
"$min": "$data.salary.amount"
}
}
},
{
"$group": {
"_id": {
"emp_city": "$_id.emp_city",
"emp_country": "$_id.emp_country",
"emp_name": "$_id.emp_name",
},
"emp_salary": {
"$push": {
"currency": "$_id.currency",
"min": "$min",
"max": "$max"
}
}
}
},
{
"$project": {
"_id": 0,
"emp_city": "$_id.emp_city",
"emp_country": "$_id.emp_country",
"emp_name": "$_id.emp_name",
"emp_salary": 1,
}
}
])
Mongo Playground
In highlighting search for my application how search works for us is if we pass pageSize it will only return one matching field from record.
For example
There are 4 records
remaining information added in comments as I am unable add code here please advice how can I achieve this requirement
If you need to highlight the result of all matching fields of each document, then you can achieve this in the following way:
Adding a working example with index data, mapping, search query, and search result
Index Mapping:
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "my_tokenizer"
}
},
"tokenizer": {
"my_tokenizer": {
"type": "edge_ngram",
"min_gram": 2,
"max_gram": 20,
"token_chars": [
"letter",
"digit"
]
}
}
},
"max_ngram_diff": 50
},
"mappings": {
"properties": {
"name": {
"type": "text",
"analyzer": "my_analyzer"
},
"lastname": {
"type": "text",
"analyzer": "my_analyzer"
},
"firstname": {
"type": "text",
"analyzer": "my_analyzer"
}
}
}
}
Index Data:
{
"name": "jhon",
"Age": 24,
"lastname": "willam",
"firstname": "henry"
}
{
"name": "kellin",
"Age": 24,
"lastname": "kevin",
"firstname": "mathew"
}
{
"name": "keeper",
"Age": 24,
"lastname": "Jr",
"firstname": "gomez"
}
{
"name": "Asif",
"Age": 24,
"lastname": "peter",
"firstname": "willaim kemp"
}
Search Query:
{
"query": {
"multi_match": {
"query": "ke"
}
},
"highlight": {
"fields": {
"*": {
"type": "plain",
"fragment_size": 20,
"pre_tags": "<span class='bold'>",
"post_tags": "</span>",
"number_of_fragments": 1
}
}
}
}
Search Result:
"hits": [
{
"_index": "64996939",
"_type": "_doc",
"_id": "2",
"_score": 1.1374959,
"_source": {
"name": "kellin",
"Age": 24,
"lastname": "kevin",
"firstname": "mathew"
},
"highlight": {
"name": [
"<span class='bold'>ke</span>llin"
], <-- note this
"lastname": [
"<span class='bold'>ke</span>vin"
]
}
},
{
"_index": "64996939",
"_type": "_doc",
"_id": "4",
"_score": 0.9552834,
"_source": {
"name": "Asif",
"Age": 24,
"lastname": "peter",
"firstname": "willaim kemp"
},
"highlight": {
"firstname": [
"willaim <span class='bold'>ke</span>mp" <-- note this
]
}
},
{
"_index": "64996939",
"_type": "_doc",
"_id": "3",
"_score": 0.62883455,
"_source": {
"name": "keeper",
"Age": 24,
"lastname": "Jr",
"firstname": "gomez"
},
"highlight": {
"name": [
"<span class='bold'>ke</span>eper" <--note this
]
}
}
]
How can I query all the advert using the field
("group": { "times": [{"startTime", "startTime"}]})
no matter the date is, query is base on hours-range.
My index is like (for example "startTime": "2016-12-13T09:00:00+08:00",
"endTime": "2016-12-13T22:00:00+08:00", but I just care about the "09:00:00" and "22:00:00"):
{
"_index": "advert",
"_type": "advert",
"_id": "6217903907666941952",
"_score": 7.730222,
"_source": {
"id": "6217903907666941952",
"promotionUrl": "http://www.baidu.com",
"summary": "test",
"user": {
"createdBy": "ag-kf",
"createdDate": "2016-12-05T11:48:52+08:00",
"lastModifiedBy": "admin",
"lastModifiedDate": "2016-12-13T14:10:52+08:00",
"id": "6211385611895442432",
"login": "ag-ggz02",
"email": "testing0000019#21cn.com",
"region": null,
"activated": true,
"locked": false,
"langKey": "zh-cn"
},
"group": {
"createdBy": "ag-ggz02",
"id": "6214281669688469248",
"name": "AGggz002推广测试02",
"dailyLimit": 500,
"deliveryChannelsLimit": 5,
"startDate": "2017-01-12T00:00:00+08:00",
"endDate": "2017-01-19T00:00:00+08:00",
"paymentMode": "CPC",
"deliveryMode": "SPEEDUP",
"status": "VALID",
"times": [
{
"createdBy": "ag-ggz02",
"createdDate": "2017-01-12T14:40:59+08:00",
"lastModifiedBy": "ag-ggz02",
"lastModifiedDate": "2017-01-12T14:40:59+08:00",
"id": "6225199665533659392",
"startTime": "2016-12-13T09:00:00+08:00",
"endTime": "2016-12-13T22:00:00+08:00"
}
]
}
}}
Below is my example search response with 4 results retrieved.
{
"took": 13,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 4,
"max_score": 0.41753215,
"hits": [
{
"_index": "google_a804f89b-d32e-426a-a79a-ea83d65c98ea",
"_type": "viz_dashlet",
"_id": "/shared/Report_google_Shared",
"_score": 0.41753215,
"fields": {
"lastModified": [
1461738428007
],
"dir": [
"/shared"
],
"filename": [
"Report_google_Shared"
]
},
"highlight": {
"filename": [
"Report_google_Shared"
]
}
},
{
"_index": "google_a804f89b-d32e-426a-a79a-ea83d65c98ea",
"_type": "viz_dashlet",
"_id": "/shared/Report_Gmail_Shared",
"_score": 0.41753215,
"fields": {
"lastModified": [
1461738618676
],
"dir": [
"/shared"
],
"filename": [
"Report_Gmail_Shared"
]
},
"highlight": {
"filename": [
"Report_Gmail_Shared"
]
}
},
{
"_index": "google_a804f89b-d32e-426a-a79a-ea83d65c98ea",
"_type": "viz_dashlet",
"_id": "/private/hitesh/Report_Gmail_Private",
"_score": 0.1883173,
"fields": {
"lastModified": [
1461738629888
],
"dir": [
"/private/hitesh"
],
"filename": [
"Report_Gmail_Private"
]
},
"highlight": {
"filename": [
"Report_Gmail_Private"
]
}
},
{
"_index": "google_a804f89b-d32e-426a-a79a-ea83d65c98ea",
"_type": "viz_dashlet",
"_id": "/private/dholaria/Report_google_Private",
"_score": 0.1883173,
"fields": {
"lastModified": [
1461738451720
],
"dir": [
"/private/dholaria"
],
"filename": [
"Report_google_Private"
]
},
"highlight": {
"filename": [
"Report_google_Private"
]
}
}
]
}
}
Now, I want to filter the above search results basis the specific "dir" field value as per the below criteria.
Include the search result in the response if and only if:
If "dir" field value equals to either of: "/shared" or "/private/hitesh"
Else if "dir" field value starts with either of: "/shared/" or
"/private/hitesh/"
How can I achieve the above functionality in ElasticSearch?
PS: Below is my example mapping.
{
"google_a804f89b-d32e-426a-a79a-ea83d65c98ea": {
"mappings": {
"viz_dashlet": {
"properties": {
"charts": {
"type": "string",
"index_analyzer": "report_index_analyzer",
"search_analyzer": "report_search_analyzer",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
},
"columnLabels": {
"type": "string",
"index_analyzer": "report_index_analyzer",
"search_analyzer": "report_search_analyzer",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
},
"columnNames": {
"type": "string",
"index_analyzer": "report_index_analyzer",
"search_analyzer": "report_search_analyzer",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
},
"creator": {
"type": "string",
"index_analyzer": "report_index_analyzer",
"search_analyzer": "report_search_analyzer",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
},
"dimensions": {
"type": "string",
"index_analyzer": "report_index_analyzer",
"search_analyzer": "report_search_analyzer",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
},
"dir": {
"type": "string",
"index_analyzer": "report_index_analyzer",
"search_analyzer": "report_search_analyzer",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
},
"expressions": {
"type": "string",
"index_analyzer": "report_index_analyzer",
"search_analyzer": "report_search_analyzer",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
},
"filename": {
"type": "string",
"index_analyzer": "whitespace_index",
"search_analyzer": "whitespace_search",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
},
"lastModified": {
"type": "date",
"format": "date_hour_minute_second"
},
"measures": {
"type": "string",
"index_analyzer": "report_index_analyzer",
"search_analyzer": "report_search_analyzer",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
},
"promptFilters": {
"type": "string",
"index_analyzer": "report_index_analyzer",
"search_analyzer": "report_search_analyzer",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}
}
Try this query:
{
"query": {
"bool": {
"should": [
{
"term": {
"dir.raw": {
"value": "/shared"
}
}
},
{
"term": {
"dir.raw": {
"value": "/private/hitesh"
}
}
},
{
"match_phrase_prefix": {
"dir.raw": "/shared"
}
},
{
"match_phrase_prefix": {
"dir.raw": "/private/hitesh"
}
}
]
}
}
}