Develop Reference

Elasticsearch match partial text in Java

I have field in elasticsearch "name" which contains for example:
"some text"
I would like to do query which match any partial of this text, for example "some", "so" or "xt"
I was trying do this by queryString:
public Page<View> search(String name, Pageable pageable) {
QueryBuilder queryBuilder = buildQueryFrom(name);
HighlightBuilder highlightBuilder = buildHighlig();
SearchRequestBuilder searchRequest = client.prepareSearch()
.setQuery(queryBuilder)
.highlighter(highlightBuilder)
.setFetchSource(null, new String[]{CV_OCR});
addPageable(searchRequest, pageable);
SearchResponse response = searchRequest.execute().actionGet();
return mapper.mapResponse(response, pageable).map(this::View);
}
private QueryBuilder buildQueryFrom(String text) {
BoolQueryBuilder queryBuilder = boolQuery();
queryBuilder.must(queryStringQuery("*" + text + "*").field("name"));
return queryBuilder;
}
but if I try to search for example "%" or another special character the results are different than my expectations. I was trying also use regex:
private QueryBuilder buildQueryFrom(String text) {
BoolQueryBuilder queryBuilder = boolQuery();
queryBuilder.must(regexpQuery("name", quoteStringAndSetCaseInsensitive(text).pattern()));
return queryBuilder;
}
private Pattern quoteStringAndSetCaseInsensitive(String value) {
return Pattern.compile(Pattern.quote(value), Pattern.CASE_INSENSITIVE);
}
but it didn't work. Is there any other possibility for searching like this?

How to use QueryStringQueryBuilder

I've tried to use QueryStringQueryBuilder in a very simple case, but I dont understand why I get another result than the result I get from Kibana. What am I doing wrong?
Kibana:
GET .../_search
{
"query": {
"query_string" : {
"query" : "\"this is a query\"",
"lenient": true,
"default_operator": "OR"
}
}
}
Java:
private Optional<QueryStringQueryBuilder> parseQuery(String query) {
if (query.equals("")) {
return Optional.empty();
}
QueryStringQueryBuilder queryBuilder = QueryBuilders.queryStringQuery(query);
queryBuilder.lenient(true);
queryBuilder.defaultOperator(Operator.OR);
return Optional.of(queryBuilder);
}
Result from kibana: totalhits = 3336. Result from Java: totalhits = 10018.
EDIT:
This method calls parseQuery. Input is "this is a query".
public Optional<SearchRequestBuilder> getRequestBuilderByQuery(SearchQuery query) {
SearchRequestBuilder builder = getBuilderWithMaxHits(query.getMaxHits());
builder.setFetchSource(Globals.getFIELDS(query.isIncludeStory()), new String[0]);
parseQuery(query.getQuery()).ifPresent(builder::setQuery);
return Optional.of(builder);
}

I dont know what your input parameter 'query' in this case contains.
But i think you want to set queryBuilder.queryName(String queryName) in your QueryStringQueryBuilder.
From the JavaDocs:
queryName(String queryName):
Sets the query name for the filter that can be used when searching for matched_filters per hit.

Ok, I found the problem.
The query for kibana had quotation marks. Which meant it was processed differently than just a normal String. QueryStringQuerybuilder seems to be parsing the query by itself with "",AND,OR,NOT. This is magic!

How to Query ES using Rabbitmq spark stram

I am using Spark2. I am trying to get the stream of search text from Rabbitmq and query againt Elasticsearch.
params.put("hosts", "IP");
params.put("queueName", "query");
params.put("exchangeName", "Exchangequery");
params.put("vHost", "/");
params.put("userName", "test");
params.put("password", "test");
Function<byte[], String> messageHandler = new Function<byte[], String>() {
public String call(byte[] message) {
return new String(message);
}
};
JavaReceiverInputDStream<String> messages = RabbitMQUtils.createJavaStream(jssc, String.class, params, messageHandler);
messages.foreachRDD();
above code receives stram from rabbitmq. But i am not sure how to connect ES and query for the stream batch. One thing is, If i use messages.foreachRDD(); and query the elasticsearch for each input item then it will affect the performance.
Always i will query elasticsearch using only one field. For example
My stram messages has the input like
apple
orange
i have a index in es fruit and i want to query like ?q=apple or orange. I know i have to frame the query using should in elasticsearch. My question is how can i query against ES using the value received from Rabbitmq stream

The code makes only one call to the elasticsearch server (basically it constructs a single query with a lot of should clauses)
public static void main(String[] args) throws UnknownHostException {
Client client = TransportClient.builder().build()
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("host1"), 9300))
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("host2"), 9300));
List<String> messages = new ArrayList<>();
messages.add("apple");
messages.add("orange");
String index = "fruit";
String fieldName = "fruit_type";
BoolQueryBuilder query = QueryBuilders.boolQuery();
for (String message : messages) {
query.should(QueryBuilders.matchQuery(fieldName, message));
// alternative if you are not analyzing fields
// query.should(QueryBuilders.termQuery(fieldName, message));
}
int size = 60; //you may want to change this since it defaults to 10
SearchResponse response = client.prepareSearch(index).setQuery(query).setSize(size).execute().actionGet();
long totalHits = response.getHits().getTotalHits();
System.out.println("Found " + totalHits + " documents");
for (SearchHit hit : response.getHits().getHits()) {
System.out.println(hit.getSource());
}
}
Query generated:
{
"bool" : {
"should" : [ {
"match" : {
"fruit_type" : {
"query" : "apple",
"type" : "boolean"
}
}
}, {
"match" : {
"fruit_type" : {
"query" : "orange",
"type" : "boolean"
}
}
} ]
}
}

elasticsearch QueryBuilder with dynamic list value in term query

I have a code like below where I'm doing multiple must in bool query. Here I'm passing the must term queries in field "address". Now the ip address will come to me as a list from other api and I have to pass for all the ip's in the list as a must term query. Here I'm not getting a way how to pass the address values dynamically when creating the QueryBuilder.
Please suggest how to do this.
public static SearchResponse searchResultWithAggregation(String es_index,
String es_type, List<String> ipList, String queryRangeTime) {
Client client = ESClientFactory.getInstance();
QueryBuilder qb = QueryBuilders.boolQuery()
.must(QueryBuilders.termQuery("address", "10.203.238.138"))
.must(QueryBuilders.termQuery("address", "10.203.238.137"))
.must(QueryBuilders.termQuery("address", "10.203.238.136"))
.mustNot(QueryBuilders.termQuery("address", "10.203.238.140"))
.should(QueryBuilders.termQuery("client", ""));
queryRangeTime = "now-" + queryRangeTime + "m";
FilterBuilder fb = FilterBuilders.rangeFilter("#timestamp")
.from(queryRangeTime).to("now");
SearchResponse response = client
.prepareSearch(es_index)
.setTypes(es_type)
.setQuery(qb)
.setPostFilter(fb)
.addAggregation(
AggregationBuilders.avg("cpu_average").field("value"))
.setSize(10).execute().actionGet();
System.out.println(response.toString());
return response;
}

You can use the terms query to pass multiple values for single field.
create a string array or set. and pass it to the terms query.
Set<String> address = new HashSet<String>();
address.add("10.203.238.138");
address.add("10.203.238.137");
address.add("10.203.238.136");
if(address!=null)
QueryBuilder qb = QueryBuilders.boolQuery()
.must(QueryBuilders.termsQuery("address",address))
.mustNot(QueryBuilders.termQuery("address", "10.203.238.140"))
.should(QueryBuilders.termQuery("client", ""));
else
QueryBuilder qb = QueryBuilders.boolQuery()
.mustNot(QueryBuilders.termQuery("address", "10.203.238.140"))
.should(QueryBuilders.termQuery("client", ""));
Hope it helps..!

If you use TermsQuery for address array/set, it will return any documents that match with at least one or more of the provided terms.
List<String> address = new ArrayList<String>();
address.add("10.203.238.138");
address.add("10.203.238.137");
address.add("10.203.238.136");
BoolQueryBuilder qb = QueryBuilders.boolQuery();
qb.mustNot(QueryBuilders.termQuery("address", "10.203.238.140"));
qb.should(QueryBuilders.termQuery("client", ""));
for(String add: Address){
qb.must(QueryBuilders.termsQuery("address",add));
}

Query Elasticsearch using String in java

I am using Elasticsearch maven jar file to query Elasticsearch. But now I want to query the elasticsearch using full generated query string:
query :
{
"bool" : {
"must" : [ {
"term" : {
"title" : "mercedes"
}
}, {
"term" : {
"Doors" : "2"
}
} ]
}
}
How do I use the above query string to query elasticsearch in java?

Following code prepares a boolquery. You should create a SearchRequestBuilder to execute it.
BoolQueryBuilder boolQuery = new BoolQueryBuilder();
boolQuery.must(QueryBuilders.termQuery("title", "mercedes"));
boolQuery.must(QueryBuilders.termQuery("Doors", "2"));
If you want to use query as string without building it in code, you can use following;
String myQuery = "Your Query Here";
SearchSourceBuilder ssb= new SearchSourceBuilder();
search.query(myQuery);
SearchRequestBuilder srb; // You should define srb before next steps
srb.internalBuilder(ssb);
SearchResponse response = srb.execute().actionGet();

I use the following code to query Elasticsearch using JSON string.
SearchResponse response = client.prepareSearch(yourIndexName)
.setSource(yourJsonQueryString)
.execute().actionGet();