I work on a project that uses elasticsearch 5.2. The code is in java and I use elasticsearch java client 5.2.
In this project, I have a field called hash, a 7-character code containing uppercase letters, lowercase letters, and numbers (in English language).
I want to do two searches on this field:
check the existence of a hash "ErTg1Qh" (case sensitive)
find hashes that sub string s contains in them (for example, sub string "tg" exist in the hash "ErTg1Qh").
For the hash field, I chose the keyword datatype.
I used matchQuery function for first search as below:
String hash = "ErTg1Qh";
QueryBuilders.matchQuery("hash", hash)
and queryStringQuery function for the second search as below:
queryString = "hash:*" + subString + "*";
QueryBuilders.queryStringQuery(queryString)
but, the second one does not work properly.
How can I handle these two kinds of search on a field?
One of your query requires to be case sensitive while the second one is case insensitive. So I'll suggest you to use sub field for the hash field. Your main field will be analyzed with lowercase analyzer and one will store raw data i.e. the exact hash. So your index will look like below:
PUT /test
{
"settings": {
"number_of_shards": "1",
"number_of_replicas": "0",
"analysis": {
"analyzer": {
"custom_lowercase": {
"filter": [
"lowercase"
],
"type": "custom",
"tokenizer": "keyword"
}
}
}
},
"mappings": {
"_doc": {
"properties": {
"hash": {
"type": "text",
"analyzer": "custom_lowercase",
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
}
}
Query to check the existence of a hash "ErTg1Qh" (case sensitive)
POST /test/_doc/_search
{
"query": {
"match": {
"hash.keyword": "ErTg1Qh"
}
}
}
Query to find hashes that sub string s contains in them
POST /test/_doc/_search
{
"query": {
"query_string": {
"query": "*tg*"
}
}
}
Related
Let's assume there's a map corresponding to the following structure:
{
"lists": [
{
"list": [
{
"letter": "a"
},
{
"letter": "b"
}
]
},
{
"list": [
{
"letter": "c"
},
{
"letter": "d"
}
]
}
]
}
There's an easy way to get all lists using SpEL (#root['lists']) or all letters of the first list ("#root['lists'][0]['list']") or the first letter of the first list ("#root['lists'][0]['list'][0]"). Also, there's a projection mechanism which allows using construction like "#root['lists'].![#this['list']]" to convert each item of lists to a result of the projection expression.
However, given all these possibilities, I still failed to come up with an expression allowing me to extract all letters from both lists. For the example above, I'd like to get
[
{
"letter": "a"
},
{
"letter": "b"
},
{
"letter": "c"
},
{
"letter": "d"
}
]
I tried to use the projection mechanism to achieve my goal but it didn't really help. The problem I see is that every time SpEL detects a list, it applies the projection expression to each element of the list so any structure of nested lists can't be changed this way.
The problem I solve can be easily solved using JsonPath, however, I assume I could get something wrong and didn't notice a way to achieve the same result using SpEL. I would be happy to listen to any ideas.
In my documents, I have a field which has an array of String.
like
weapon : ["Bland Blade", "Defender Quelthalas", "Thousand Lies", "Frozen Bonespike"]
I would like to get all the documents whose weapon field has "Frozen Bonespike".
not field which has one word - like "Frozen" or "Bonespike"
and not even field which contains "Frozen Bonespike" - like "Winter Frozen Bonespike"
I would like to get a field which has exactly equal string with query word
Have you any idea?
To achieve that you have to set your weapon field as type keyword and then run a terms search against it.
By default elasticsearch will create both text type field (lets say weapon) and keyword (weapon.keyword)
Ingesting document
POST test_seo/_doc
{
"weapon": [
"Bland Blade",
"Defender Quelthalas",
"Thousand Lies",
"Frozen Bonespike"
]
}
Query
POST test_seo/_search
{
"query": {
"term": {
"weapon.keyword": {
"value": "Frozen Bonespike"
}
}
}
}
If you want to search by many different weapons you can use a terms query
POST test_seo/_search
{
"query": {
"terms": {
"weapon.keyword": [
"Bland Blade",
"Thousand Lies"
]
}
}
}
We have an index with a keyword field that is very often an ip address, but not always. We'd like to be able to search this index on that field using not just keywords but also CIDR notation, which is supported only for fields of type 'ip'. On the surface, this looks like a use case for multi-fields, so we have an index with mappings:
{
"mappings": {
"my_field": {
"type": "keyword"
"fields": {
"ip": {
"type": "ip"
"ignore_malformed": true
}
}
}
}
}
So, when our application has a set of non-ip addresses, ip addresses, and CIDR-notation blocks/ranges of ip addresses and needs to query by them, our application splits that set into one set with non-ip addresses and another with ip addresses/CIDR-notation blocks and makes two separate terms filters from them in my query, like so:
{
"query": {
"bool": {
"filter": [
{
"terms": {
"my_field.ip": [
"123.123.123.0/24",
"192.168.0.1",
"192.168.16.255",
"192.169.1.0/24"
]
}
},
{
"terms": {
"my_field": [
"someDomain.com",
"notAnIp.net"
]
}
}
]
}
}
}
This returns no records. I assume because it's not treating the two filters as an OR. I want all of the records that match the first terms query PLUS all of the records that match the second terms query. What is the best way to achieve this? Hoping my application doesn't need to send two separate requests.
Try out this below query:
{
"query": {
"bool": {
"should": [
{
"bool": {
"filter": {
"terms": {
"my_field.ip": [
"123.123.123.0/24",
"192.168.0.1",
"192.168.16.255",
"192.169.1.0/24"
]
}
}
}
},
{
"bool": {
"filter": {
"terms": {
"my_field": [
"someDomain.com",
"notAnIp.net"
]
}
}
}
}
],
"minimum_should_match": 1
}
}
}
You misunderstood the filter clause, it will filter the documents without should or must clause in your boolean query, please refer to boolean queries official doc for more info.
when you use should or must clause with a filter then its meaning is changed and it will start returning the documents, refer to #ESCoder answer on how to achieve it.
Also please note as you are using only the filter clause, the score will not be calculated and all the documents in your search results will have the same score as 0, which might not be something you want, if you want to score to happen then you need to use the query clause.
I am new to ElasticSearch and lookig currently into must and must_not type of search criteria.
I have the following query DSL:
curl -XPOST 'localhost:9200/employee/_search?pretty' -d '
{
"query": {
"bool": {
"must": [
{ "match": { "age": "40" } }
],
"must_not": [
{ "match": { "state": "ID" } }
]
}
}
}'
Now my question is , is there any way i can optimize the above query?
Like, if i have to search for multiple must and must_not criteria, the query would become very large and difficult to manage. Is there any way i can combine both must and must_not under a single search phrase?
As far as I know you could use "should" to put several matches inside a "bool" statement (not sure why exactly it does not work with directly one "bool" statement):
Edit Actually to have several matches, without a too complex request, what is below should work normally:
{
"query": {
"bool":{
"must": [
{"match":{"age":"40"}},
{"match":{"name": "John"}}
]
, "must_not": [
{"match":{"age":"40"}},
{"match":{"name": "John"}}
]
}
}
}
Unfortunately, must and must_not cannot be combined
I have a BasicDBObject of following structure in JAVA
"category_id": "keyboard",
"ontology_id": "mobile",
"path": "#mobile#keyboard",
"date": "2013/008/01 12:30:00",
"hour": {
"12": {
"total_docs": 1
}
}
I need to increment total_docs value by 1 but i am not able to access that object. How should i proceed in this case
never mind, we just need to append the value in string and it's done