I started to use ElasticSearch in my test project and cant figure out hot to create search for all fields. For instance we have some words as a search query and i want to find all indexed object in ElasticSearch, using Java API.
My obj have: id, name, adress, etc
I searched for this kind of info and wrote this:
Node node = nodeBuilder().node();
Client client = node.client();
RegexpFilterBuilder qFilter = FilterBuilders.regexpFilter("_all", (".*" + query + ".*").replace(" ", ".*"));
SearchResponse response = client.prepareSearch(index)
.setTypes(type)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setPostFilter(qFilter)
.setFrom(0).setSize(100).setExplain(true)
.execute()
.actionGet();
SearchHit[] results = response.getHits().getHits();
System.out.println("Current results: " + results.length);
I also tried to use one field:
SearchResponse response = client.prepareSearch(index)
.setTypes(type)
.setSearchType(SearchType.QUERY_AND_FETCH)
.setQuery(termQuery(field, value))
.setFrom(0).setSize(100).setExplain(true)
.execute()
.actionGet();
I always get 0 result.
Can you show me, how to do this in right way in java?
Ok, i spent more time on documentation and i found a solution, hope it helps to someone else!
You just need to use QueryBuilders.multiMatchQuery, the value is our searching word and other strings are the columns where to search to.
SearchResponse response = client.prepareSearch(index)
.setTypes(type)
.setSearchType(SearchType.QUERY_AND_FETCH)
.setQuery(QueryBuilders.multiMatchQuery(value,
"name", "address1", "city", "postalCode",
"countryCode", "airportCode", "locationDescription",
"shortDescription"
))
.setFrom(0).setSize(100).setExplain(true)
.execute()
.actionGet();
Related
I am writing a REST application (javax.ws.rs) that takes search requests from clients and submits them to the Elasticsearch high-level API. I want the clients
(browser based javascript mostly) to be able to compose their searches using the Elasticsearch REST API instructions.
The REST end point is defined like this:
#Path("list")
#POST
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public Response list(Map<String, Object> req) {
...
The following code would implement a security layer function and then pass the query on to SearchRequest object pretty much unchanged. So I don't want to build queries using QueryBuilders here.
I have tried the instructions in this article but it doesn't work. I think the createParser method has changed since that example was written. If someone could review this and suggest a solution that would be much appreciated.
UPDATE: Using ES 7.2 I have come up with the following code. There have been many changes in the API not all of which I understand but here is what seems like it should work.
XContentBuilder xcb = XContentFactory.contentBuilder(Requests.CONTENT_TYPE);
xcb.map(req);
String json = Strings.toString(xcb);
XContentParser parser = JsonXContent.jsonXContent.createParser(
NamedXContentRegistry.EMPTY, LoggingDeprecationHandler.INSTANCE, json);
SearchSourceBuilder ssb = new SearchSourceBuilder();
ssb.parseXContent(parser);
SearchRequest sr = new SearchRequest(Log.INDEX);
sr.source(ssb);
SearchResponse resp = client.search(sr, RequestOptions.DEFAULT);
I get an IOException off of the call to parseXContent. Looking with the debugger the string json has unprintable characters in it. Any suggestions?
I found a code pattern that works and it seems a bit convoluted but logical. There is no documentation anywhere that would lead you to this. This was pieced together from some fragments posted in message boards here and there.
try {
// convert the Map into a JSON string to parse. Alternatively
// you could just take the string directly from the HTTP request
// but the Map form makes it easy to manipulate.
XContentBuilder xcb = XContentFactory.jsonBuilder();
xcb.map(req);
String json = Strings.toString(xcb);
// Create an XContentParser and borrow a NamedXContentRegistry from
// the SearchModule class. Without that the parser has no way of
// knowing the query syntax.
SearchModule sm = new SearchModule(Settings.EMPTY, false, Collections.emptyList());
XContentParser parser = XContentFactory.xContent(XContentType.JSON)
.createParser(new NamedXContentRegistry(sm.getNamedXContents()),
LoggingDeprecationHandler.INSTANCE,
json);
// Finally we can create our SearchSourceBuilder and feed it the
// parser to ingest the request. This can throw and IllegalArgumentException
// if something isn't right with the JSON that we started with.
SearchSourceBuilder ssb = new SearchSourceBuilder();
ssb.parseXContent(parser);
// Now create a search request and use it
SearchRequest sr = new SearchRequest(Log.INDEX);
sr.source(ssb);
SearchResponse resp = client.search(sr, RequestOptions.DEFAULT);
I have tested this with a number of different JSON queries from the client and they all seem to work the way the direct REST API would. Here is an example:
{
from: 0,
size: 1000,
query: {
match_all: { boost: 1 }
},
sort: [
{ timestamp: { 'order': 'asc' } }
]
}
Hopefully this post will save someone else from the painful search I went through. I would appreciate any comments from anyone who could suggest a better way of doing this.
XContentBuilder xcb = XContentFactory.jsonBuilder();
xcb.map(req);
String json = Strings.toString(xcb);
SearchSourceBuilder ssb = new SearchSourceBuilder();
ssb.query(QueryBuilders.wrapperQuery(json));
I am looking to build a query filter in java for Elsaticsearch 5.2 to match any of the keywords wether they are author supplied or system generated. I am trying to do a wild card path but it does not return any results. I am using standard dynamic mapping in Elasticsearch.
{
"content": {
"title": "The History of Cats",
"description": "A brief history of cats.",
"keywords": {
"author": ["cat"],
"system": ["felis", "animalia"]
}
}
}
Ideally, if a user searches with the keyword cat or felis they should get a hit on this record. Currently, I do not receive any hits. Is this something that can be achieved through this query builder or another? Do I need to tweak mappings?
BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
queryBuilder.filter(QueryBuilders.matchQuery("content.keywords.*", keyword));
SearchRequestBuilder searchRequestBuilder = this.client.prepareSearch("my-index")
.setTypes("article")
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(queryBuilder)
.setFrom(0)
.setSize(10)
.addSort(SortBuilders.scoreSort());
SearchResponse searchResponse = searchRequestBuilder.execute().actionGet();
return searchResponse;
Thanks
Drew
After some digging, this can be achieved by changing the match query to a multi-match query
BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
queryBuilder.filter(QueryBuilders.multiMatchQuery(keyword, "content.keywords.*"));
SearchRequestBuilder searchRequestBuilder = this.client.prepareSearch("my-index")
.setTypes("article")
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(queryBuilder)
.setFrom(0)
.setSize(10)
.addSort(SortBuilders.scoreSort());
SearchResponse searchResponse = searchRequestBuilder.execute().actionGet();
return searchResponse;
I used ElasticSearch java client. I do search with query_string, and I get response, but score always be 1.0 .
code is:
String query = "{\"query\": {\"query_string\": {\"query\": \"weblog data4\"}}}";
SearchRequestBuilder builder = client
.prepareSearch("flume-2016-08-10")
.setQuery(query)
.addHighlightedField("*")
.setHighlighterRequireFieldMatch(false)
.setFrom(0).setSize(60).setExplain(true);
SearchResponse response = builder.execute().actionGet();
System.out.println(response.toString());
System.out.println(response.getHits().getAt(0).getSource());
System.out.println(response.getHits().getAt(0).getHighlightFields());
client.close();
result is:
However, I do search in elaseticsearch-head, I get response with correct score.
So, How do I get correct score with java?
This solved my problem:
change:
String query = "{\"query\": {\"query_string\": {\"query\": \"weblog data4\"}}}";
to:
String query = "{\"query_string\": {\"query\": \"weblog data4\"}}";
more details:
answer by jpountz
I am having trouble making this URL query work in Java, it does not return any results. but from the browser it returns all the results, here is the URL that returns result:
_search?pretty&q=*357*+AND+account_id:574fe92c9179a809fd76f0b8+AND+invalid:false
And here is my code (does not return any results):
FilterBuilder[] filtersArray = new FilterBuilder[2];
filtersArray[0] = FilterBuilders.termFilter("account_id", "574fe92c9179a809fd76f0b8");
filtersArray[1] = FilterBuilders.termFilter("invalid", false);
QueryBuilder query = QueryBuilders.filteredQuery(QueryBuilders.simpleQueryStringQuery("*357*"), FilterBuilders.andFilter(filtersArray));
SearchResponse response = esClient.prepareSearch(SecurityManager.getNamespace())
.addSort("created_time", SortOrder.DESC)
.setTypes(dataType)
.setQuery(query)
.addFields("_id")
.setFrom(page * size)
.setSize(size)
.setExplain(false)
.execute()
.actionGet();
Can someone tell me what is the best way to translate the URL query into a java query?
First off, the URL query you should use is this one
?q=*357*+AND+account_id:574fe92c9179a809fd76f0b8+AND+invalid:false
otherwise you'll have no constraint on account_id and invalid
Then, the exact translation of this new URL query in Java is
QueryBuilder query = QueryBuilders.queryStringQuery("*357* AND account_id:574fe92c9179a809fd76f0b8 AND invalid:false");
SearchResponse response = esClient.prepareSearch(SecurityManager.getNamespace())
.addSort("created_time", SortOrder.DESC)
.setTypes(dataType)
.setQuery(query)
.addFields("_id")
.setFrom(page * size)
.setSize(size)
.setExplain(false)
.execute()
.actionGet();
Notes:
queryStringQuery and not simpleQueryStringQuery
no filters as they are all in the query string already
This is my old code which is working fine
qb = QueryBuilders.queryString(query.replaceAll(" ", " OR ").replaceAll(",", " AND ").replaceAll("!", " NOT "));
FilterBuilder fb = FilterBuilders.andFilter(FilterBuilders.rangeFilter("Experiance").from(smonth).to(emonth));
FilteredQueryBuilder fqBuilder = QueryBuilders.filteredQuery(qb, fb);
org.elasticsearch.action.search.SearchResponse searchHits = node.client()
.prepareSearch(name)
.setQuery(fqBuilder)
In this piece of code, I am searching those data whose experience between smonth and emonth.
Now i need to add more filters in my search, So i move to 'NativeSearchQueryBuilder'. After modification, I write this code:-
qb = QueryBuilders.queryString(query.replaceAll(" ", " OR ").replaceAll(",", " AND ").replaceAll("!", " NOT "));
FilterBuilder fb = FilterBuilders.andFilter(FilterBuilders.rangeFilter("Experiance").from(smonth).to(emonth));
NativeSearchQueryBuilder builder = new NativeSearchQueryBuilder();
builder.withQuery(qb);
AndFilterBuilder filters = null;
filters = new AndFilterBuilder(fb);
filters.add(FilterBuilders.andFilter(FilterBuilders.boolFilter().must(FilterBuilders.termFilter("providedZipcode", zipcode)))); //third filter
builder.withFilter(filters);
org.elasticsearch.action.search.SearchResponse searchHits = node.client()
.prepareSearch(name)
.setQuery(builder.build().getQuery());
But when i am searching then filter is not working. I am not getting data according to experience and zipcode.
Got it.
We can use setPostFilter
https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/search.html
org.elasticsearch.action.search.SearchResponse searchHits = node.client()
.prepareSearch(name)
// .setIndices(name)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(fqBuilder)
.setPostFilter(FilterBuilders.andFilter(FilterBuilders.boolFilter().must(FilterBuilders.termFilter("providedZipcode", zipcode)))); // Filter
But i have another problem. If i didnt pass zipcode then it's not showing data. I want to get all data if i not pass any zipcode.
You look like you would actually benefit more from using a bool filter, with two condiditons in addition to the query string. We will be searching all documents that hopefully match the providedZipcode but definitely have experience ranging from smonth to emonth. In my answer, I used a bool filter to implement this logic.
Post Filter in your above answer isn't really the functionality that you are looking for. It executes your request using the original query, and then filters the results of that original query by the second filter.
This is how I wrote up your request:
QueryStringQueryBuilder qb = QueryBuilders.queryString(query
.replaceAll(" ", " OR ")
.replaceAll(",", " AND ")
.replaceAll("!", " NOT "));
FilterBuilder fb = FilterBuilders.boolFilter()
.should(FilterBuilders.termFilter("providedZipcode", zipcode))
.must(FilterBuilders.rangeFilter("Experiance").from(smonth).to(emonth));
FilteredQueryBuilder fqb = new FilteredQueryBuilder(qb, fb);