I have following query which is used for almost all search terms.
Query
GET test_partial/_search
{
"query": {
"function_score": {
"query": {
"bool": {
"filter": [],
"must": [
{
"multi_match": {
"fields": [
"title^30",
"description^10"
],
"operator": "and",
"query": "pamers diap",
"type": "most_fields"
}
}
]
}
}
}
}
}
Document
[
{
"title": "Huggies diapers"
},
{
"title": "Huggies wipes"
},
{
"title": "papmpers wipes"
},
{
"title": "natureval diapers"
}
]
If you check query "operator": "and" it works perfectly fine in terms of relevancy for all other search terms.
I have no pampers diapers document (I get no results)
But I have few documents with Huggies diapers and pampers wipes
If I change "operator": "or" I get both documents in results.
To keep relevancy top, I need to keep operator=and and switch to "OR" when no results. To achieve this I need to make 2 ES calls, is there a way we can specify conditional query when no results switch to "OR" to avoid 2 calls to ES?
Complementing my comment, I would try something like the query below. I have accuracy with match and recovery with multi-match.
{
"query": {
"function_score": {
"query": {
"bool": {
"filter": [],
"should": [
{
"match": {
"title": {
"query": "natureval diapers",
"operator": "and",
"boost": 50
}
}
},
{
"match": {
"description": {
"query": "natureval diapers",
"operator": "and",
"boost": 30
}
}
},
{
"multi_match": {
"fields": [
"title^30",
"description^10"
],
"operator": "or",
"query": "natureval diapers",
"type": "most_fields"
}
}
]
}
}
}
}
}
Related
I have below data structure in ElastiSearch.
[{
"name": "Kapil",
"age": 32,
"hobbies": ["Cricket", "Football", "Swimming"]
},
{
"name": "John",
"age": 33,
"hobbies": ["Baseball", "Football", "Swimming"]
},
{
"name": "Vick",
"age": 30,
"hobbies": ["Baseball", "Karate", "Swimming"]
}]
I want to get all records from the data in following order:
Get all users which has Football as hobby and sort them by age desc.
Get all other users sort by age desc.
So expecting result in following order John, Kapil and Vick.
I used below query to get result for #1.
{
"size": 500,
"query": {
"bool": {
"must": [
{
"match_phrase": {
"hobbies.keyword": "Football"
}
}
]
}
},
"sort": [
{
"age": {
"order": "desc"
}
}
]
}
and used below for point #2
{
"size": 500,
"query": {
"bool": {
"must_not": [
{
"match_phrase": {
"hobbies.keyword": "Football"
}
}
]
}
},
"sort": [
{
"age": {
"order": "desc"
}
}
]
}
With above, I am not able to maintain the paging logic. It also requires me to execute both queries separately. Can Someone please help how to achieve this?
Try this out:
{
"query": {
"bool": {
"must": {
"match_all": {} <-- retrieve all docs
},
"should": { <-- give higher score to docs that match this clause
"match_phrase": {
"hobbies.keyword": "Football"
}
}
}
},
"sort": [
"_score", <-- sort by doc score first
{
"age": { <-- when score is equal then sort by age
"order": "desc"
}
}
]
}
I am trying to configure elastic search with synonyms.
These are my settings:
"analysis": {
"analyzer": {
"category_synonym": {
"tokenizer": "whitespace",
"filter": [
"synonym_filter"
]
}
},
"filter": {
"synonym_filter": {
"type": "synonym",
"synonyms_path": "synonyms.txt"
}
}
}
Mappings config:
"category": {
"properties": {
"name": {
"type":"string",
"search_analyzer" : "category_synonym",
"index_analyzer" : "standard",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
And the list of my synonyms
film => video,
ooh => panels , poster,
commercial => advertisement,
print => magazine
I must say that I am using Elasticsearch Java API.
I am using QueryBuilders.queryStringQuery because this is the only way how I set analyzers to my request.
So, when I am making:
QueryBuilders.queryStringQuery("name:film").analyzer(analyzer)
It returns me
[
{
"id": 71,
"name": "Pitch video",
"description": "... ",
"parent": null
},
{
"id": 25,
"name": "Video",
"description": "... ",
"parent": null
}
]
That is perfect for me, but when I am calling something like this
QueryBuilders.queryStringQuery("name:vid").analyzer(analyzer)
I expect that it should return same objects, but there is nothing: []
So, I added asterisk to queryStringQuery:
QueryBuilders.queryStringQuery("name:vid*").analyzer(analyzer)
Works well, but now
QueryBuilders.queryStringQuery("name:film*").analyzer(analyzer)
returns me []
So, how can I configure my elastic search that it will return same objects when I am searching video, vid, film and fil?
Thanks in advance!
Hm, I don't think Elasticsearch will know to "translate" fil into vid :-). So, I think you need edgeNGrams for this, both at indexing and search time.
PUT test
{
"settings": {
"analysis": {
"analyzer": {
"category_synonym": {
"tokenizer": "whitespace",
"filter": [
"synonym_filter",
"my_edgeNGram_filter"
]
},
"standard_edgeNGram": {
"tokenizer": "standard",
"filter": [
"lowercase",
"synonym_filter",
"my_edgeNGram_filter"
]
}
},
"filter": {
"synonym_filter": {
"type": "synonym",
"synonyms_path": "synonyms.txt"
},
"my_edgeNGram_filter": {
"type": "edgeNGram",
"min_gram": 2,
"max_gram": 8
}
}
}
},
"mappings": {
"test": {
"properties": {
"name": {
"type": "string",
"analyzer": "category_synonym",
"index_analyzer": "standard_edgeNGram",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}
POST test/test/1
{"name": "Pitch video"}
POST test/test/2
{"name": "Video"}
GET /test/test/_search
{
"query": {
"query_string": {
"query": "name:fil"
}
}
}
I have JsonObjects that i search with Elasticsearch from a Java Application, using the Java API to build searchQueries. The objects contain a field called "such" that contains a searchString with which the JsonObject should be found, for example a simple searchString would be "STVBBM160A". Besides the usual characters a-Z 0-9 the searchString could also look like the following examples:
"STV-157ABR", "F-G/42-W3" or "DDM000.074.6652"
The search should return results already when only the first characters are put into a searchfield, which it does for a search like "F-G/42"
My Problem: The search sometimes doesn't return results at all, but when the last character is typed it finds the right document.
What i tried: First I wanted to use a WildcardQuery where the query would be "typedStuff*", but the WildcardQuery didn't return any results at all, as soon as I typed anything but * (It used to work for other searchFields with other values)
Now I am using a QueryStringQuery, which also takes the input and puts a * character to the end. By escaping the QueryString, I am able to search for Strings like "F-G/42" and so on, but the search for "DDM000.074.6652" doesn't return any results until elasticsearch has the whole String to search. Also, when i type "STV" all results with "STV-xxxxx" (containing the "-" after STV) are returned, but not the object with "STVBBM160A", again until the whole String is given for the search (without showing any results inbetween as soon as the searchString is "STVB")
This is the query I'm using right now:
{
"size": 1000,
"min_score": 1,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "MY_DATA_TYPE",
"fields": [
"doc.db_doc_type"
]
}
},
{
"query_string": {
"query": "MY_SPECIFIC_TYPE",
"fields": [
"doc.db_doc_specific"
]
}
}
],
"should": {
"query_string": {
"query": "STV*",
"fields": [
"doc.such"
],
"boost": 3,
"escape": true
}
}
}
}
}
This is the old Query with the WildCardQuery, which doesn't return any results at all unless there is no queryString but *:
{
"size": 50,
"min_score": 1,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "MY_DATA_TYPE",
"fields": [
"doc.db_doc_type"
]
}
},
{
"query_string": {
"query": "MY_SPECIFIC_TYPE",
"fields": [
"doc.db_doc_specific"
]
}
}
],
"should": {
"wildcard": {
"doc.such": {
"wildcard": "STV*",
"boost": 3
}
}
}
}
}
}
When using a PrefixQuery, the search also doesn't return any results at all (with and without the *):
{
"size": 50,
"min_score": 1,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "MY_DATA_TYPE",
"fields": [
"doc.db_doc_type"
]
}
},
{
"query_string": {
"query": "MY_SPECIFIC_TYPE",
"fields": [
"doc.db_doc_specific"
]
}
}
],
"should": {
"prefix": {
"doc.such": {
"prefix": "HSTKV*",
"boost": 3
}
}
}
}
}
}
How can this query be changed to achieve the goal of getting all results starting with the specified String, no matter if the field doc.such also contains Numbers or special chars like "_" or "." or "/" ?
Thanks in advance
As soon as you want to query prefixes, suffixes or substring in a serious way, you need to leverage nGrams. In your case, since you're only after prefixes, an edgeNGram tokenizer would be in order. You need to change the settings of your index to be like this one:
PUT your_index
{
"settings": {
"analysis": {
"analyzer": {
"prefix_analyzer": {
"tokenizer": "prefix_tokenizer",
"filter": [
"lowercase"
]
},
"search_prefix_analyzer": {
"tokenizer": "keyword",
"filter": [
"lowercase"
]
}
},
"tokenizer": {
"prefix_tokenizer": {
"type": "edgeNGram",
"min_gram": "1",
"max_gram": "25"
}
}
}
},
"mappings": {
"your_type": {
"properties": {
"doc": {
"properties": {
"such": {
"type": "string",
"fields": {
"starts_with": {
"type": "string",
"analyzer": "prefix_analyzer",
"search_analyzer": "search_prefix_analyzer"
}
}
}
}
}
}
}
}
}
What will happen with this analyzer is that when indexing F-G/42-W3 the following tokens will be indexed: f, f-, f-g, f-g/, f-g/4, f-g/42, f-g/42-, f-g/42-w, f-g/42-w3.
At search time, we'll simply lowercase the user input and the prefix will be matched against the indexed tokens.
Then your query can simply be transformed to a match query:
{
"size": 1000,
"min_score": 1,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "MY_DATA_TYPE",
"fields": [
"doc.db_doc_type"
]
}
},
{
"query_string": {
"query": "MY_SPECIFIC_TYPE",
"fields": [
"doc.db_doc_specific"
]
}
}
],
"should": {
"match": {
"doc.such": {
"query": "F-G/4"
}
}
}
}
}
}
I am using ElasticSearch 2.3.1 on Ubuntu 16.04.
The mapping is:
{
"settings": {
"analysis": {
"filter": {
"2gramsto3_filter": {
"type": "ngram",
"min_gram": 2,
"max_gram": 3
}
},
"analyzer": {
"2gramsto3": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"lowercase",
"2gramsto3_filter"
]
}
}
}
},
"mappings": {
"agents": {
"properties": {
"presentation": {
"type": "string",
"analyzer": "2gramsto3"
},
"cv": {
"type": "string",
"analyzer": "2gramsto3"
}
}
}
}
The query is:
{
"size": 20,
"from": 0,
"query": {
"bool": {
"filter": [
{
"bool": {
"must": [
[
{
"match": {
"cv": "folletto"
}
},
{
"match": {
"cv": " psicologia"
}
},
{
"match": {
"cv": " tenacia"
}
}
]
]
}
}
]
}
}
}
It found 14567 documents but the score is always "_score": 0
I read the filters have the score, so, why not in this case?
Thank you!
The score is not calculated for filters. You need to use a normal query if you need scores.
Just take into account implications pointed out at the documentation below.
Ref doc: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-filter-context.html
I have the following document indexed but when I run the search it's not returning anything, I was wondering if its an issue with the query. I am trying to search for any of the nested messages that have the word dogs in it. Here is the document:
{
"_index": "thread_and_messages",
"_type": "thread",
"_id": "3",
"_score": 1.0,
"_source": {
"thread_id": 3,
"thread_name": "I play the guitar",
"created": "Wed Apr 13 2016",
"thread_view": 2,
"first_nick": "Test User",
"messages": [{
"message_text": " I like dogs",
"message_id": 13,
"message_nick": "Test"
}],
"site_name": "Test Site"
}
}
Here is the query I am running when I run the curl command:
{
"function_score": {
"functions": [{
"field_value_factor": {
"field": "thread_view",
"modifier": "log1p",
"factor": 2
}
}],
{"query": {
"bool": {
"should": [{
"match": {
"thread_name": "dogs"
}
}, {
"nested": {
"path": "messages",
"query": {
"bool": {
"should": [{
"match": {
"messages.message_text": "dogs"
}
}]
}
},
"inner_hits": {}
}
}]
}
}
}
}
The mapping you have plus the sample document with a slightly modified query works for me:
curl -XGET "http://localhost:9200/thread_and_messages/thread/_search" -d'
{
"query": {
"function_score": {
"functions": [
{
"field_value_factor": {
"field": "thread_view",
"modifier": "log1p",
"factor": 2
}
}
],
"query": {
"bool": {
"should": [
{
"match": {
"thread_name": "dogs"
}
},
{
"nested": {
"path": "messages",
"query": {
"bool": {
"should": [
{
"match": {
"messages.message_text": "dogs"
}
}
]
}
},
"inner_hits": {}
}
}
]
}
}
}
}
}'