I am having a problem where when I run a search on elastic using the java api I get back results... but when I try and extract values from the results there are no fields.
ElasticSearch v5.3.1
Elastic API: org.elasticsearch.client:transport v5.3.0
My code:
SearchRequestBuilder srb = client.prepareSearch("documents").setTypes("documents").setSearchType(SearchType.QUERY_THEN_FETCH).setQuery(qb).setFrom(0).setSize((10)).setExplain(false);
srb.addDocValueField("title.raw");
SearchResponse response = srb.get();
response.getHits().forEach(new Consumer<SearchHit>() {
#Override
public void accept(SearchHit hit) {
System.out.println(hit);
Map<String, SearchHitField> fields = hit.getFields();
Object obj = fields.get("title.raw").getValue();
}
});
When the forEach runs the obj is always coming back null. Fields has an item in it with the key of title.raw and it has a SearchHitField.
Fields is only used when you are trying to obtain stored fields. By default you should obtain fields using the source. With the following code samples I'll try to explain it.
PUT documents
{
"settings": {
"number_of_replicas": 0,
"number_of_shards": 1
},
"mappings": {
"document": {
"properties": {
"title": {
"type": "text",
"store": true
},
"description": {
"type": "text"
}
}
}
}
}
PUT documents/document/_bulk
{"index": {}}
{"title": "help me", "description": "description of help me"}
{"index": {}}
{"title": "help me two", "description": "another description of help me"}
GET documents/_search
{
"query": {
"match": {
"title": "help"
}
},
"stored_fields": ["title"],
"_source": {
"includes": ["description"]
}
}
Next the response, notice the difference between the stored field Title and the normal field description.
{
"_index": "documents",
"_type": "document",
"_id": "AVuciB12YLj8D0X3N5We",
"_score": 0.14638957,
"_source": {
"description": "another description of help me"
},
"fields": {
"title": [
"help me two"
]
}
}
Related
I'm constructing a message to be sent through my spring boot application.
I was testing out the templates and I have created one where most of the elements are static except a link that needs to be generated by the code and added to the Json.
Currently the Json message looks like this:
{
"blocks": [
{
"type": "context",
"elements": [
{
"type": "image",
"image_url": "https://api.slack.com/img/blocks/bkb_template_images/highpriority.png",
"alt_text": "High Priority"
},
{
"type": "mrkdwn",
"text": "*High Priority*"
}
]
},
{
"type": "divider"
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*Hercules Platform Status Response failed Messages*"
}
},
{
"type": "divider"
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*Please click the link to download the file*"
}
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*<LINK|SOME_LINK>*"
}
}
]
}
I'm not sure how to construct this Json in my spring boot application. Since most of it is static, should I just load this template as a string and append the last link section?
I'm not able to figure out the slack classes in java to build such a message.
I am trying to post a share with entity location and resolved Url in content entities, Even though i am pointing resolvedUrl to a image, it is not considering that image instead fetching 'og:image' from entity location. I don't see any use of resolvedUrl.
Suggest on how to use resolvedUrl ie. image with custom url.
Body:
{
"owner": "urn:li:organization:xxxxxxx",
"distribution": {
"linkedInDistributionTarget": {}
},
"content": {
"contentEntities": [{
"entityLocation": "https://www.cricbuzz.com/cricket-news/107673/ipl-2019-rishabh-pant-shikhar-dhawan-propel-delhi-capitals-to-the-top-rr-vs-dc-rajasthan-royals",
"thumbnail": [{
"resolvedUrl": "https://www.cricbuzz.com/a/img/v1/595x396/i1/c168531/watson-led-csks-chase-of-176.jpg"
}]
}],
"description": "Description of attachment",
"title": "Title of the attachment"
},
"text": {
"text": "Hii Kaushik from builder"
}
}
Hello I think it might be because of a typo: It should be "thumbnails" instead of "thumbnail", see if it works.
Link: https://learn.microsoft.com/en-us/linkedin/marketing/integrations/community-management/shares/share-api#post-shares
{
"owner": "urn:li:organization:xxxxxxx",
"distribution": {
"linkedInDistributionTarget": {}
},
"content": {
"contentEntities": [{
"entityLocation": "https://www.cricbuzz.com/cricket-news/107673/ipl-2019-rishabh-pant-shikhar-dhawan-propel-delhi-capitals-to-the-top-rr-vs-dc-rajasthan-royals",
"thumbnails": [{
"resolvedUrl": "https://www.cricbuzz.com/a/img/v1/595x396/i1/c168531/watson-led-csks-chase-of-176.jpg"
}]
}],
"description": "Description of attachment",
"title": "Title of the attachment"
},
"text": {
"text": "Hii Kaushik from builder"
}
}
Hope this helps.
I'm working on Elasticsearch Java API, and I faced weird problem.
below is the data stored:
"_index": "my_index",
"_type": "SEC",
"_id": "1111111111111111",
"_score": 0,
"_source": {
"LOG_NO": 2222222222222222
}
And I call the request like below:
QueryBuilder queryBuilder = QueryBuilders.boolQuery().filter(query);
request.setQuery(queryBuilder);
SearchResponse searchResponse = request.get();
This is the search response:
"_index":"my_index",
"_type":"SEC",
"_id":"1111111111111111",
"_score":null,
"_source": {
"LOG_NO":1111111111111111,
}
As you can see, the "LOG_NO" of the response should be '2222222222222222' not '1111111111111111'.
the parameter 'query' is QueryBuilder and value is like below:
{
"bool": {
"must": [
{
"range": {
"LOG_GEN_TIME": {
"from": "2018-11-01 00:00:00+09:00",
"to": "2018-11-01 23:59:59+09:00",
"include_lower": true,
"include_upper": true,
"boost": 1
}
}
},
{
"bool": {
"must": [
{
"term": {
"ASSET_IP": "xx.xxx.xxx.xxx"
}
},
{
"term": {
"DST_PORT": "xx"
}
}
]
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
}
I don't understand what is the problem.
Any comments would be appreciated, Thanks.
-- Edited for #Val
"_index": "my_index",
"_type": "SEC",
"_id": "9197340043548295192",
"_score": null,
"_source": {
"ASSET_IP": "xx.xxx.xx.xxx",
"LOG_NO": 9197340043548295200,
"LOG_GEN_TIME": "2018-11-01 23:10:53+09:00",
"SRC_IP": "xx.xxx.xx.xxx",
"SRC_PORT": xx,
"DST_IP": "xx.xxx.xx.xxx",
"DST_PORT": xx,
"DESCRIPTION": "log",
"DST_NATION_CD": "USA",
}
}
this is the full document that I expect to be returned and the only "LOG_NO" field makes problem.
I found that the problem is javascript.
Because JS can't express the parameter that exceed the range of number, it was transformed.
My ES setting was the integer field "LOG_NO" is set as a index and the "_id" is string expression of "LOG_NO".
So there was no problem, except the integer number looks different on web.
Hope others would not suffer the same problem.
I have created the below normalizer for the field code to query the elasticsearch based on both upper and lower case.
PUT my_index12
{
"settings": {
"analysis": {
"normalizer": {
"my_normalizer": {
"type": "custom",
"char_filter": [],
"filter": ["lowercase", "asciifolding"]
}
}
}
},
"mappings": {
"doc": {
"properties": {
"code": {
"type": "keyword",
"normalizer": "my_normalizer"
}
}
}
}
}
And am trying to search using wildcard query, am not getting any results, without normalizer am able to get with the uppercase, exactly how it is there in elasticsearch
get my_index12/_search
{
"query": {
"wildcard": {
"code.keyword": {
"value": "*AB-7000-5000-Wk-21*"
}
}
}
}
Please find my index below
{
"_index": "my_index12",
"_type": "doc",
"_id": "2",
"_score": 1,
"_source": {
"code": "ABCq123S"
}
},
{
"_index": "my_index12",
"_type": "doc",
"_id": "1",
"_score": 1,
"_source": {
"code": "AB-7000-5000-Wk-21"
}
}
If i try to do the mapping for code.keyword
"mappings": {
"doc": {
"properties": {
"code.keyword": {
"type": "keyword",
"normalizer": "my_normalizer"
}
}
am getting the below error while inserting documents into the index
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "object mapping for [code] tried to parse field [code] as object, but found a concrete value"
}
],
"type": "mapper_parsing_exception",
"reason": "object mapping for [code] tried to parse field [code] as object, but found a concrete value"
},
"status": 400
}
I have the below mapping structure for my Elasticsearch index.
{
"users": {
"mappings": {
"user-type": {
"properties": {
"lastModifiedBy": {
"type": "string"
},
"lastModifiedDate": {
"type": "date",
"format": "dateOptionalTime"
},
"details": {
"type": "nested",
"properties": {
"lastModifiedBy": {
"type": "string"
},
"lastModifiedDate": {
"type": "date",
"format": "dateOptionalTime"
},
"views": {
"type": "nested",
"properties": {
"id": {
"type": "string"
},
"name": {
"type": "string"
},
"properties": {
"properties": {
"name": {
"type": "string"
},
"type": {
"type": "string"
},
"value": {
"type": "string"
}
}
}
}
}
}
}
}
}
}
}
}
Basically I want to retrieve ONLY the view object inside details based on index id & view id(details.views.id).
I have tried with the below java code.But seems to be not working.
SearchRequestBuilder srq = this.client.prepareSearch(this.indexName)
.setTypes(this.type)
.setQuery(QueryBuilders.termQuery("_id", sid))
.setPostFilter(FilterBuilders.nestedFilter("details.views",
FilterBuilders.termFilter("details.views.id", id)));
Below is the query structure for this java code.
{
"query": {
"term": {
"_id": "123"
}
},
"post_filter": {
"nested": {
"filter": {
"term": {
"details.views.id": "def"
}
},
"path": "details.views"
}
}
}
Since details is nested and view is nested inside details, you basically need two nested filters as well (one for each level) + the constraint on the _id field is best done with the ids query. The query DSL would look like this:
{
"query": {
"ids": {
"values": [
"123"
]
}
},
"post_filter": {
"nested": {
"filter": {
"nested": {
"path": "details.view",
"filter": {
"term": {
"details.views.id": "def"
}
}
}
},
"path": "details"
}
}
}
Translating this into Java code yields:
// 2nd-level nested filter
FilterBuilder detailsView = FilterBuilders.nestedFilter("details.views",
FilterBuilders.termFilter("details.views.id", id));
// 1st-level nested filter
FilterBuilder details = FilterBuilders.nestedFilter("details", detailsView);
// ids constraint
IdsQueryBuilder ids = QueryBuilders.idsQuery(this.type).addIds("123");
SearchRequestBuilder srq = this.client.prepareSearch(this.indexName)
.setTypes(this.type)
.setQuery(ids)
.setPostFilter(details);
PS: I second what #Paul said, i.e. always play around with the query DSL first and when you know you have zeroed in on the exact query you need, then you can translate it to the Java form.