Develop Reference

Elastic Query multi_match conditional query when no results

I have following query which is used for almost all search terms.
Query
GET test_partial/_search
{
"query": {
"function_score": {
"query": {
"bool": {
"filter": [],
"must": [
{
"multi_match": {
"fields": [
"title^30",
"description^10"
],
"operator": "and",
"query": "pamers diap",
"type": "most_fields"
}
}
]
}
}
}
}
}
Document
[
{
"title": "Huggies diapers"
},
{
"title": "Huggies wipes"
},
{
"title": "papmpers wipes"
},
{
"title": "natureval diapers"
}
]
If you check query "operator": "and" it works perfectly fine in terms of relevancy for all other search terms.
I have no pampers diapers document (I get no results)
But I have few documents with Huggies diapers and pampers wipes
If I change "operator": "or" I get both documents in results.
To keep relevancy top, I need to keep operator=and and switch to "OR" when no results. To achieve this I need to make 2 ES calls, is there a way we can specify conditional query when no results switch to "OR" to avoid 2 calls to ES?

Complementing my comment, I would try something like the query below. I have accuracy with match and recovery with multi-match.
{
"query": {
"function_score": {
"query": {
"bool": {
"filter": [],
"should": [
{
"match": {
"title": {
"query": "natureval diapers",
"operator": "and",
"boost": 50
}
}
},
{
"match": {
"description": {
"query": "natureval diapers",
"operator": "and",
"boost": 30
}
}
},
{
"multi_match": {
"fields": [
"title^30",
"description^10"
],
"operator": "or",
"query": "natureval diapers",
"type": "most_fields"
}
}
]
}
}
}
}
}

MongoDB: how to add the specific fields from one query with alias to result of another query using Java (with or without Spring)?

If we have collection "orders"
{"_id":"1",
"currency": "USD"//...
}
And collection "order_items"
{"_id":"1",
"order":"1",
"product":"p1",
"totalItemAmount": 25
},
{"_id":"2",
"order":"1",
"product":"p2",
"totalItemAmount": 50
}
How to get such result
[{
"orderId": "1",
"currency": "USD",
"products": ["p1", "p2"]
"totalOrderAmount": "75"
}]
using Java (with or without Spring)?

You can achieve that using an aggregation like this:
db.orders.aggregate([
{
"$lookup": {
"from": "order_items",
"as": "items",
"localField": "_id",
"foreignField": "order"
}
},
{
"$unwind": "$items"
},
{
"$group": {
"_id": "$_id",
"totalOrderAmount": {
"$sum": "$items.totalItemAmount"
},
"products": {
"$push": "$items.product"
}
}
},
{
"$project": {
"_id": 0,
"orderId": "$_id",
"currency": 1,
"totalOrderAmount": {
"$toString": "$totalOrderAmount"
},
"products": 1
}
}
])
I don't know which driver you're using in Java but it's not hard to adapt it to the language you're using

Why one of the field of response has "_id" field's value?

I'm working on Elasticsearch Java API, and I faced weird problem.
below is the data stored:
"_index": "my_index",
"_type": "SEC",
"_id": "1111111111111111",
"_score": 0,
"_source": {
"LOG_NO": 2222222222222222
}
And I call the request like below:
QueryBuilder queryBuilder = QueryBuilders.boolQuery().filter(query);
request.setQuery(queryBuilder);
SearchResponse searchResponse = request.get();
This is the search response:
"_index":"my_index",
"_type":"SEC",
"_id":"1111111111111111",
"_score":null,
"_source": {
"LOG_NO":1111111111111111,
}
As you can see, the "LOG_NO" of the response should be '2222222222222222' not '1111111111111111'.
the parameter 'query' is QueryBuilder and value is like below:
{
"bool": {
"must": [
{
"range": {
"LOG_GEN_TIME": {
"from": "2018-11-01 00:00:00+09:00",
"to": "2018-11-01 23:59:59+09:00",
"include_lower": true,
"include_upper": true,
"boost": 1
}
}
},
{
"bool": {
"must": [
{
"term": {
"ASSET_IP": "xx.xxx.xxx.xxx"
}
},
{
"term": {
"DST_PORT": "xx"
}
}
]
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
}
I don't understand what is the problem.
Any comments would be appreciated, Thanks.
-- Edited for #Val
"_index": "my_index",
"_type": "SEC",
"_id": "9197340043548295192",
"_score": null,
"_source": {
"ASSET_IP": "xx.xxx.xx.xxx",
"LOG_NO": 9197340043548295200,
"LOG_GEN_TIME": "2018-11-01 23:10:53+09:00",
"SRC_IP": "xx.xxx.xx.xxx",
"SRC_PORT": xx,
"DST_IP": "xx.xxx.xx.xxx",
"DST_PORT": xx,
"DESCRIPTION": "log",
"DST_NATION_CD": "USA",
}
}
this is the full document that I expect to be returned and the only "LOG_NO" field makes problem.

I found that the problem is javascript.
Because JS can't express the parameter that exceed the range of number, it was transformed.
My ES setting was the integer field "LOG_NO" is set as a index and the "_id" is string expression of "LOG_NO".
So there was no problem, except the integer number looks different on web.
Hope others would not suffer the same problem.

ElasticSearch - JavaApi searching by each character instead of term (word)

Am fetching documents from elastic search using java api, i have the following code in my elastic search documents and am trying to search it with the following pattern.
code : MS-VMA1615-0D
Input : MS-VMA1615-0D -- Am getting the results (MS-VMA1615-0D).
Input : VMA1615 -- Am getting the results (MS-VMA1615-0D) .
Input : VMA -- Am getting the results (MS-VMA1615-0D) .
But, if i give input like below, am not getting results.
Input : V -- Am not getting the results.
INPUT : MS -- Am not getting the results.
INPUT : -V -- Am not getting the results.
INPUT : 615 -- Am not getting the results.
Am expecting to return the code MS-VMA1615-0D. In simple, am trying to search character by character instead of term (word).
It should not return the code MS-VMA1615-0D for the following cases, Because its not matching with my code.
Input : VK -- should not return the results.
INPUT : MS3 -- should not return the results.
Please find my below java code that am using
private final String INDEX = "products";
private final String TYPE = "doc";
SearchRequest searchRequest = new SearchRequest(INDEX);
searchRequest.types(TYPE);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
QueryStringQueryBuilder qsQueryBuilder = new QueryStringQueryBuilder(code);
qsQueryBuilder.defaultField("code");
searchSourceBuilder.query(qsQueryBuilder);
searchSourceBuilder.size(50);
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = null;
try {
searchResponse = SearchEngineClient.getInstance().search(searchRequest);
} catch (IOException e) {
e.getLocalizedMessage();
}
Item item = null;
SearchHit[] searchHits = searchResponse.getHits().getHits();
Please find my mapping details :
PUT products
{
"settings": {
"analysis": {
"analyzer": {
"custom_analyzer": {
"type": "custom",
"tokenizer": "my_pattern_tokenizer",
"char_filter": [
"html_strip"
],
"filter": [
"lowercase",
"asciifolding"
]
}
},
"tokenizer": {
"my_pattern_tokenizer": {
"type": "pattern",
"pattern": "-|\\d"
}
}
}
},
"mappings": {
"doc": {
"properties": {
"code": {
"type": "text",
"analyzer": "custom_analyzer"
}
}
}
}
}
After Update with new Answer :
This is my request via Java API
'SearchRequest{searchType=QUERY_THEN_FETCH, indices=[products], indicesOptions=IndicesOptions[id=38, ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false], types=[doc], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=0, batchedReduceSize=512, preFilterShardSize=128, source={"size":50,"query":{"match_phrase":{"code":{"query":"1615","slop":0,"boost":1.0}}}}}
' . But am getting response as null

Follow up: ElasticSearch - JavaApi searching not happening without (*) in my input query
Your mapping should look like:
PUT products
{
"settings": {
"analysis": {
"analyzer": {
"custom_analyzer": {
"type": "custom",
"tokenizer": "ngram",
"char_filter": [
"html_strip"
],
"filter": [
"lowercase",
"asciifolding"
]
}
}
}
},
"mappings": {
"doc": {
"properties": {
"code": {
"type": "text",
"analyzer": "custom_analyzer"
}
}
}
}
}
And you should be using a match_phrase query.
In Kibana:
GET products/_search
{
"query": {
"match_phrase": {
"code": "V"
}
}
}
will return the result:
"hits": [
{
"_index": "products",
"_type": "doc",
"_id": "EoGtdGQBqdof7JidJkM_",
"_score": 0.2876821,
"_source": {
"code": "MS-VMA1615-0D"
}
}
]
But this:
GET products/_search
{
"query": {
"match_phrase": {
"code": "VK"
}
}
}
wont:
{
"took": 10,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}
}
Based on your comment:
Instead of using a Query string:
QueryStringQueryBuilder qsQueryBuilder = new QueryStringQueryBuilder(code);
qsQueryBuilder.defaultField("code");
searchSourceBuilder.query(qsQueryBuilder);
searchSourceBuilder.size(50);
searchRequest.source(searchSourceBuilder);
Use a match phrase query:
QueryBuilder query = QueryBuilders.matchPhraseQuery("code", code);
searchSourceBuilder.query(query);
searchSourceBuilder.size(50);
searchRequest.source(searchSourceBuilder);

elasticsearch aggs groupby field and orderby score

currently I implemented it from postman but I can't implement it from JAVA code. Below is the post json body.
I only want to have the full-text search for Innovation. And groupby email and orderby score. But seems the sum of the score still didn't work. Who can help me out? It worked now.
{
"from": 0,
"size": 10,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "Innovation"
}
}
]
}
},
"highlight": {
"require_field_match": false,
"pre_tags" : [ "<b>" ],
"post_tags" : [ "</b>" ],
"order" : "score",
"highlight_filter" : false,
"fields": {
"*": {}
}
},
"aggs": {
"group_by_emails": {
"terms": { "field": "email" }
}
}
}
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.from(0);
searchSourceBuilder.size(10);
searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
QueryStringQueryBuilder queryString = new QueryStringQueryBuilder(keyword);
searchSourceBuilder.query(queryString);
Script script = new Script("_score;");
AggregationBuilder aggregation = AggregationBuilders
.terms("agg")
.field("email")
.order(BucketOrder.aggregation("sum_score", false))
.subAggregation(AggregationBuilders.sum("sum_score").script(script))
;
searchSourceBuilder.aggregation(aggregation);
System.out.println(searchSourceBuilder.toString());
SearchRequest searchRequest = new SearchRequest(indexName);
searchRequest.source(searchSourceBuilder);