Develop Reference

query data from elasticsearch using java highlevelrestclient

How to query data from elasticsearch based on the property that is present inside the actual object.
Format of data stored in elsticsearch:
{
"principals": [
{
"id": 1,
"account": {
"account_id": 2
}
}
]
}
Search query in postman:
{
"query": {
"terms": {
"account_id": [
1
]
}
}
}
This is returning the required result in postman.
How to achieve the same in java using highlevelrestclient.

I am not sure how your above search query worked in fetching corresponding document.
But I had indexed and searched your document through this way :
mapping:
{
"mappings": {
"properties": {
"principals": {
"properties": {
"id": { "type": "integer" },
"account": {
"properties": {
"account_id": { "type": "integer" }
}
}
}
}
}
}
}
search query:
{
"query": {
"terms": {
"principals.account.account_id": [2]
}
}
}
Search result :
"hits": [
{
"_index": "nestedindex",
"_type": "_doc",
"_id": "2",
"_score": 1.0,
"_source": {
"principals": [
{
"id": 1,
"account": {
"account_id": 2
}
}
]
}
}
]
Search query through Elasticsearch Resthighlevelclient
SearchRequest searchRequest = new SearchRequest("testIndex"); //in place of "testIndex" you can give your index name
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
List<Integer> accountIds = new ArrayList<Integer>();
accountIds.add(2);
sourceBuilder.query(QueryBuilders.termsQuery("principals.account.account_id", accountIds));
sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
searchRequest.source(sourceBuilder);
SearchResponse searchResponse =
client.search(searchRequest, RequestOptions.DEFAULT); //client is ES client
return searchResponse; //you can read your hits through searchResponse.getHits().getHits()
ElasticSearch client can be instantiated in spring-boot application by creating configuration file in your project and autowiring the client where required:
#Configuration
#Primary
public class ElasticsearchConfig {
private RestHighLevelClient restHighLevelClient;
#Bean(destroyMethod = "close")
public RestHighLevelClient client() {
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("localhost", 9200, "http")));
return client;
}

ElasticSearch - matchPhraseQuery API to search with multiple fields

Am searching for specific field which is having ngram tokenizer and am querying that field (code) using matchPhraseQuery and it is working fine.
Now, i want to search with 3 field. How can we do this.?
Please find my java code which am searching for only one field (code).
SearchRequest searchRequest = new SearchRequest(INDEX);
searchRequest.types(TYPE);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
QueryBuilder qb = QueryBuilders.matchPhraseQuery("code", code);
searchSourceBuilder.query(qb);
searchSourceBuilder.size(10);
searchRequest.source(searchSourceBuilder);
Please find my mappings details below :
PUT products
{
"settings": {
"analysis": {
"analyzer": {
"custom_analyzer": {
"type": "custom",
"tokenizer": "ngram",
"char_filter": [
"html_strip"
],
"filter": [
"lowercase",
"asciifolding"
]
}
}
}
},
"mappings": {
"doc": {
"properties": {
"code": {
"type": "text",
"analyzer": "custom_analyzer"
},
"attribute" : {
"type" : "text",
"analyzer" : "custom_analyzer"
},
"term" : {
"type" : "text",
"analyzer" : "custom_analyzer"
}
}
}
}
}
Now, i want to make a search query for 3 fields code, attribute, term
I have tried the below java code, which is not working as expected :
BoolQueryBuilder orQuery = QueryBuilders.boolQuery();
QueryBuilder qb1 = QueryBuilders.matchPhraseQuery("catalog_keywords", keyword);
QueryBuilder qb2 = QueryBuilders.matchPhraseQuery("product_keywords", keyword);
orQuery.should(qb1);
orQuery.should(qb2);
orQuery.minimumShouldMatch(1);
searchSourceBuilder.query(orQuery);
searchSourceBuilder.size(10);
searchRequest.source(searchSourceBuilder);
My Input Query :
Logi
Output am getting like :
"Materiały, programy doborowe | Marketing | Katalogi, broszury"
Which is totally irrelavant to my query. Expected results is, Logiciels
And my field having value with delimiters |, so i just want only the exact match of the word/character. It should not print with all the delimiters and all.

Use bool query :
GET products/_search
{
"query": {
"bool": {
"must": [
{
"match_phrase": {
"FIELD": "PHRASE"
}
},
{
"match_phrase": {
"FIELD": "PHRASE"
}
},
{
"match_phrase": {
"FIELD": "PHRASE"
}
}
]
}
}
}

elasticsearch aggs groupby field and orderby score

currently I implemented it from postman but I can't implement it from JAVA code. Below is the post json body.
I only want to have the full-text search for Innovation. And groupby email and orderby score. But seems the sum of the score still didn't work. Who can help me out? It worked now.
{
"from": 0,
"size": 10,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "Innovation"
}
}
]
}
},
"highlight": {
"require_field_match": false,
"pre_tags" : [ "<b>" ],
"post_tags" : [ "</b>" ],
"order" : "score",
"highlight_filter" : false,
"fields": {
"*": {}
}
},
"aggs": {
"group_by_emails": {
"terms": { "field": "email" }
}
}
}
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.from(0);
searchSourceBuilder.size(10);
searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
QueryStringQueryBuilder queryString = new QueryStringQueryBuilder(keyword);
searchSourceBuilder.query(queryString);
Script script = new Script("_score;");
AggregationBuilder aggregation = AggregationBuilders
.terms("agg")
.field("email")
.order(BucketOrder.aggregation("sum_score", false))
.subAggregation(AggregationBuilders.sum("sum_score").script(script))
;
searchSourceBuilder.aggregation(aggregation);
System.out.println(searchSourceBuilder.toString());
SearchRequest searchRequest = new SearchRequest(indexName);
searchRequest.source(searchSourceBuilder);

Elasticsearch: Filter Query not working

I've been working to change my query to a filter in Elasticsearch using the java api. I've made sure that the fields I am running searches on are set to "not_analzed".
Here is the java code for the filter:
FilterBuilder andFilter = FilterBuilders.andFilter(FilterBuilders.termFilter("thread_name", keyword), FilterBuilders.termFilter("site_name", "test_site"));
QueryBuilder filteredQuery = QueryBuilders.filteredQuery(QueryBuilders.matchAllQuery(), andFilter);
SearchResponse threadResponse = client.prepareSearch("thread_and_messages").setQuery(filteredQuery).setSize(20).setFrom(firstRowOffset).execute().actionGet();
This then gets translated to this in JSON:
"filtered" : {
"query" : {
"match_all" : { }
},
"filter" : {
"and" : {
"filters" : [ {
"term" : {
"thread_name" : "apple"
}
}, {
"term" : {
"site_name" : "test_site"
}
} ]
}
}
}
When I try to do any searches it won't return the document I'm looking for which is thread_name: Apple and site_name: test_site.
Also when I run this using a curl command it won't find that document either.
Can anyone see what I'm doing wrong here?

You can use must clause.That will perform and operation
"query": {
"bool": {
"must": [
{
"query_string": {
"default_field": "thread_name",
"query": "apple"
}
},
{
"query_string": {
"default_field": "site_name",
"query": "test_site"
}
}
]
}
}

Elasticsearch complex query using Java API

I am trying to query on ElasticSearch using Java API, my query is:
curl -XGET 'http://localhost:9200/logstash-*/_search?search_type=count' -d '
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"and" : [
{
"range": {
"timestamp": {
"gte": "2015-08-20",
"lt": "2015-08-21",
"format": "yyyy-MM-dd",
"time_zone": "+8:00"
}
}
},
{"query": {
"match": {
"request": {
"query": "/v2/brand"
}
}
}
},
{"term": { "response" : "200"}
}
]
}
}
},
"aggs": {
"group_by_device_id": {
"terms": {
"field": "clientip"
}
}
}
}'
The similar sql logic is:
select distinct(clientip) from table where timestamp between '2015-08-20' and '2015-08-21' and request like '/v2/brand%' and response = '200'
How to implement it using Java API?
Please guide I am new to ElasticSearch. Thanks in advance!

I have resolved the problem, below is my codes:
SearchResponse scrollResp1 = client.prepareSearch("logstash-*").setSearchType(SearchType.SCAN).
setQuery(QueryBuilders.filteredQuery(QueryBuilders.matchAllQuery(),
FilterBuilders.andFilter(FilterBuilders.termFilter("response", "200")
, FilterBuilders.rangeFilter("timestamp").gte(startDate).lt
(endDate), FilterBuilders.queryFilter
(QueryBuilders.matchQuery("request", "signup"))
)))
.addAggregation(AggregationBuilders.terms
("group_by_client_ip").size(0).field("clientip")).get();