I am trying to connect to elasticsearch and do some basic query in 6.3.2 version.
The code I am trying is this:
RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("localhost", 30100, "http")));
SearchRequest sr = new SearchRequest(INDEX);
sr.indicesOptions(IndicesOptions.lenientExpandOpen());
SearchSourceBuilder ssb = new SearchSourceBuilder();
sr.source(ssb);
MatchQueryBuilder builder = QueryBuilders.matchQuery("logLevel.keyword", "ERROR");
QueryBuilder qb = QueryBuilders.boolQuery().must(builder);
ssb.query(qb);
SearchResponse response = null;
try {
response = client.search(sr);
System.out.println("total hits ::: " + response.getHits().getTotalHits());
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
client.close();
} catch (IOException e) {
e.printStackTrace();
}
}
System.out.println(response);
UPDATE
As suggested i am using the query only now. I tried the build query from APIs and i see the result but for some reason response.getHits().getTotalHits() is returning zero. The generated query looks like below and is giving me the expected result in kibana i.e. a total count of 1 :
GET /_search
{
"from": 0,
"query": {
"bool": {
"must": [
{
"match": {
"logLevel.keyword": {
"query": "ERROR",
"operator": "OR",
"prefix_length": 0,
"max_expansions": 50,
"fuzzy_transpositions": true,
"lenient": false,
"zero_terms_query": "NONE",
"auto_generate_synonyms_phrase_query": true,
"boost": 1
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
}
}
Am missing some configuration for the restclient ?
I don't see why you need all the parts in the query that you posted (like max_expansions etc). A simple query like below, should do the job if you only need the count of documents that match the criteria.
curl -XGET "http://localhost:9200/index_name/_search" -d'
{
"size": 0,
"query": {
"bool": {
"must": [
{
"match": {
"logLevel": "ERROR"
}
},
{
"range": {
"date": {
"gte": "enter_date_here"
}
}
}
]
}
}
}'
reference: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html
Related
GET feeds/_search
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "comment",
"query": {
"match": {
"comment.c_text": "This is mateen"
}
},"inner_hits": {}
}
},
{
"term": {
"title.keyword": {
"value": "This is mateen"
}
}
},
{
"term": {
"body.keyword": {
"value": "This is mateen"
}
}
}
]
}
}
}
Mapping is as follows:
PUT feeds
{
"mappings": {
"properties": {
"comment":{
"type": "nested"
}
}
}
}
I am using Elasticsearch 7.17.3. For searching all documents of Elasticsearch in my springboot I have written the following code that gives me the exact output:
public List<feed> searchAllDocuments() throws IOException {
SearchRequest searchRequest = SearchRequest.of(s -> s.index(indexName));
SearchResponse searchResponse = elasticsearchClient.search(searchRequest, feed.class);
List<Hit> hits = searchResponse.hits().hits();
List<feed> feeds = new ArrayList<>();
feed f=null;
for (Hit object : hits) {
f = (feed) object.source();
feeds.add(f);
}
return feeds;
}
Can anyone help me convert the query into springboot application? I am new to it and need your guidance
If you use the new Java Api Client try the code bellow:
Query nestedQuery = NestedQuery.of(nq ->
nq.path("comment")
.innerHits(InnerHits.of(ih -> ih))
.query(MatchQuery.of(mq -> mq.field("comment.c_text").query("This is mateen"))._toQuery())
)._toQuery();
Query termQueryTitle = TermQuery.of(
tq -> tq.field("title.keyword").value("This is mateen")
)._toQuery();
Query termQueryBody = TermQuery.of(
tq -> tq.field("body.keyword").value("This is mateen")
)._toQuery();
Query boolQuery = BoolQuery.of(bq -> bq.should(nestedQuery, termQueryBody, termQueryTitle))._toQuery();
SearchRequest searchRequest = SearchRequest.of(
s -> s.index("idx_name").query(boolQuery)
);
var response = client.search(searchRequest, Feed.class);
this is my elasticsearch query, which generated by kibana
"aggs": {
"3": {
"terms": {
"field": "policy.weight",
"order": {
"1": "asc"
},
"size": 5
},
"aggs": {
"1": {
"cardinality": {
"field": "item.item_doc_id.keyword"
}
},
"2": {
"terms": {
"field": "policy.name.keyword",
"order": {
"1": "desc"
},
"size": 100
},
"aggs": {
"1": {
"cardinality": {
"field": "item.item_doc_id.keyword"
}
},
"4": {
"terms": {
"field": "policy.description.keyword",
"order": {
"1": "desc"
},
"size": 100
},
"aggs": {
"1": {
"cardinality": {
"field": "item.item_doc_id.keyword"
}
}
}
}
}
}
}
}
}
and this is the same query with java api, which I'm implementing
public BucketList getListOfBucketsTimeRestrictedEV(BucketListInfo bucketListInfo) {
final SearchRequest searchRequest = new SearchRequest(bucketListInfo.getIndexName());
final String field1 = "policy.name.keyword";
final String field2 = "policy.description.keyword";
final String field3 = "policy.weight";
final String field4 = "item.item_doc_id.keyword";
final SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
.query(getRangeQueryBuilderWithOptionalFilter(bucketListInfo.getTimestampFieldName(), bucketListInfo.getFilterFieldName(), bucketListInfo.getFilterFieldValue(),
bucketListInfo.getFrom(), bucketListInfo.getTo())).size(100);
TermsAggregationBuilder aggregationBuilder;
aggregationBuilder = AggregationBuilders.terms(field3).field(field3).size(5).order(BucketOrder.count(true));
searchSourceBuilder.aggregation(aggregationBuilder);
aggregationBuilder = AggregationBuilders.terms(field4).field(field4);
searchSourceBuilder.aggregation(aggregationBuilder);
aggregationBuilder = AggregationBuilders.terms(field1).field(field1).size(100);
searchSourceBuilder.aggregation(aggregationBuilder);
aggregationBuilder = AggregationBuilders.terms(field2).field(field2).size(100);
searchSourceBuilder.aggregation(aggregationBuilder);
// aggregationBuilder.subAggregation(AggregationBuilders.terms(subfield).field(subfield));
// searchSourceBuilder.aggregation(aggregationBuilder);
searchRequest.source(searchSourceBuilder);
try {
final SearchResponse response = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
BucketList bucketList = new BucketList();
final Terms terms = response.getAggregations().get(field1);
bucketList.getBuckets().add(terms.getBuckets());
return bucketList;
} catch (Exception e) {
log.error(e.getMessage(), e);
return null;
}
}
Now the question:
Why I'm getting not desired results, say getting more rows than expected?
I'm also using subAggregation here, but I think the subAggregation doesn't work somehow. But I'm debugging and observe that the subAggregation is added in "aggregationBuilder.factoryBuilder container.
Sould I actually use here subAggregation, or just aggregation is enough?
I was new to Elasticsearch and Java. I was trying to convert a nested elasticsearch query which has both MUST and SHOULD functions. Can someone help on this.
GET /list/_search
{
"size": 12,
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"multi_match": {
"query": "city hed",
"type": "bool_prefix",
"fields": [
"cityName":"",
"countryCodeName":"",
"iso":""
]
}
},
{
"multi_match": {
"query": "city hed",
"fuzziness": "AUTO",
"fields": [
"cityName*"
]
}
}
]
}
},
{
"bool": {
"should": [
{
"match": {
"iso": ""
}
},
{
"match": {
"iso": ""
}
}
]
}
}
]
}
}
}
need method logic for it like how it will be in Java API
If you use Java High Client try this:
SearchRequest searchRequest = new SearchRequest("idx_name");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
MultiMatchQueryBuilder matchQueryBuilder = QueryBuilders
.multiMatchQuery("city hed")
.type(Type.PHRASE_PREFIX)
.field("cityName")
.field("countryCodeName")
.field("iso");
MultiMatchQueryBuilder matchQueryBuilder2 = QueryBuilders
.multiMatchQuery("city hed")
.fuzziness("2")
.field("cityName*");
MatchQueryBuilder matchQuery01 = QueryBuilders.matchQuery("iso", "");
MatchQueryBuilder matchQuery02 = QueryBuilders.matchQuery("iso", "");
BoolQueryBuilder bool01 = QueryBuilders.boolQuery().should(matchQueryBuilder).should(matchQueryBuilder2);
BoolQueryBuilder bool02 = QueryBuilders.boolQuery().should(matchQuery01).should(matchQuery02);
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery().must(bool01).must(bool02);
searchSourceBuilder.query(boolQueryBuilder);
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = getClient().search(searchRequest, RequestOptions.DEFAULT);
I want to build the following ES query using BoolQueryBuilders and Aggregator, but I am unable to do that.
{
"size": 0,
"query": {
"bool": {
"should": {
"bool": {
"filter": [
{
"terms": {
"country": [
"France",
"China"
]
}
},
{
"term": {
"lang": "en"
}
}
]
}
}
}
},
"aggs": {
"group_by_country": {
"terms": {
"field": "country",
"size": 0
},
"aggs": {
"top_hits_country": {
"top_hits": {
"size": 1
}
}
}
}
}
}
I am able to build this query without the aggregator, in the following manner -
BoolQueryBuilder innerEntityQueryBuilder = new BoolQueryBuilder();
BoolQueryBuilder queryBuilder = new BoolQueryBuilder()
.filter(QueryBuilders.termsQuery("country", countries))
.filter(QueryBuilders.termQuery("lang", "en"));
innerEntityQueryBuilder.should(queryBuilder);
How do I add the aggregate part as well?
like this：
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("index1");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.size(0);
BoolQueryBuilder bqb = new BoolQueryBuilder()
.filter(QueryBuilders.termsQuery("country", "France","China"))
.filter(QueryBuilders.termQuery("lang", "en"));
searchSourceBuilder.query(bqb);
TermsAggregationBuilder group = AggregationBuilders
.terms("group_by_country").field("country")
.size(0);
TopHitsAggregationBuilder topHit = AggregationBuilders.topHits("top_hits_country").size(1);
group.subAggregation(topHit);
searchSourceBuilder.aggregation(group);
searchRequest.source(searchSourceBuilder);
So I am just trying to add data in my indice via java transport client api . My code runs without any exceptions.
It successfully connects to my server, I can see the connection details on my console.
I listed all indices available on the server and it did show that too.
Then I tried to insert docs into my indice called logs
code runs without any exceptions.
I am lost here and have no idea what I am missing. The data is not getting inserted in the indice. Any help is appreciated.
Below is my code :
public static void main(String[] args) {
try {
ElasticSearchMain es = new ElasticSearchMain();
ElasticSearchMain.configureClient();
JSONObject data = new JSONObject();
data.put("serverip", "bhavik");
data.put("classname", "bhavik");
data.put("methodname", "bhavik");
data.put("exception", "bhavik");
data.put("logexception", "4896681231231232");
data.put("timestamp", "1900-00-00");
es.setIndex(data.toString(), "logs");
} catch (Exception e) {
e.printStackTrace();
}
}
public IndexRequestBuilder setIndex(String data,String IndexName){
try{
return client.prepareIndex(IndexName, "event").setSource(data);
}catch(Exception e){
e.printStackTrace();
}
return null;
}
public static void configureClient(){
try{
if(client==null){
String address = "localhost";
int port = 9300;
BasicConfigurator.configure();
client = TransportClient.builder().build()
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName(address), port));
String[] indices = client.admin().indices().getIndex(new GetIndexRequest()).actionGet().getIndices();
for (String s : indices) {
System.out.println("indice ->> " + s);
}
}
}catch(Exception e ){
e.printStackTrace();
EmgrLog.AppendExceptionToLog(e);
}
}
Indice mapping :
PUT logs
{
"mappings": {
"event": {
"properties": {
"serverip": {
"type": "integer",
"index": "not_analyzed"
},
"classname": {
"type": "string",
"analyzer": "english"
},
"methodname": {
"type": "string",
"index": "not_analyzed"
},
"exception": {
"type": "string",
"index": "not_analyzed"
},
"logexception": {
"type": "string"
},
"timestamp":{
"type": "date"
}
}
}
}
}
This code :
public IndexRequestBuilder setIndex(String data,String IndexName){
try{
return client.prepareIndex(IndexName, "event").setSource(data);
}catch(Exception e){
e.printStackTrace();
}
return null;
}
Returns a "request builder", but this request is never built, nor executed, and logically, nothing happens as a consequence.
To submit the request, you have to send it to the elasticsearch instance.
Elasticsearch JavaAPI documentation proposes this code :
IndexResponse response = client.prepareIndex("twitter", "tweet", "1")
.setSource(jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elasticsearch")
.endObject()
)
.get();
Note the get() at the end of the snippet is what triggers the call.