I want to enable search for a bunch of fields for one of my entities. Therefore, I added hibernate search to my spring boot project. When I load data into the database, I can see that Elasticsearch contains that data as expected in the index running
curl localhost:9200/myindex/_search?pretty
I can run queries like
curl localhost:9200/myindex/_search?pretty&q=name:test
and receive the expected results.
I would like to give consumers of my API the option to run arbitrary queries like "name:test" against the index, so that
curl "localhost:8086/myentity/search/querySearch?query=name:test"
would return the same results as before in the direct query.
Here's what I am trying but whatever I do, I get 0 results:
public List<MyEntity> querySearch(String queryString) {
QueryParser queryParser = new MultiFieldQueryParser(ALL_FIELDS, new SimpleAnalyzer());
queryParser.setDefaultOperator(QueryParser.AND_OPERATOR);
org.apache.lucene.search.Query query = queryParser.parse(QueryParser.escape(queryString));
FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(this.entityManager);
javax.persistence.Query persistenceQuery =
fullTextEntityManager.createFullTextQuery(query, MyEntity.class);
return persistenceQuery.getResultList();
}
By calling QueryParser.escape(queryString), you remove the meaning of operators such as :. So if the user enters name:test, you will end up looking for documents that contain name:test (literally), instead of looking for documents whose name field contains test.
Remove that escape and everything should work as you want.
By the way, you're essentially using Lucene to parse a query that will then be sent to Elasticsearch. An easier solution would be to send the query to Elasticsearch directly, especially if you do not need to prevent users from accessing some fields.
public List<Training> querySearch(String queryString) {
FullTextEntityManager fullTextEm = Search.getFullTextEntityManager(this.entityManager);
QueryDescriptor query = ElasticsearchQueries.fromQueryString(queryString);
javax.persistence.Query persistenceQuery = fullTextEm.createFullTextQuery(query, Training.class);
return persistenceQuery.getResultList();
}
See https://docs.jboss.org/hibernate/search/5.11/reference/en-US/html_single/#_queries
Related
I'm using hibernate-search-elasticsearch 5.8.2.Final and I can't figure out how to get script fields:
https://www.elastic.co/guide/en/elasticsearch/reference/5.6/search-request-script-fields.html
Is there any way to accomplish this functionality?
This is not possible in Hibernate Search 5.8.
In Hibernate Search 5.10 you could get direct access to the REST client, send a REST request to Elasticsearch and get the result as a JSON string that you would have to parse yourself, but it is very low-level and you would not benefit from the Hibernate Search search APIs at all (no query DSL, no managed entity loading, no direct translation entity type => index name, ...).
If you want better support for this feature, don't hesitate to open a ticket on our JIRA, describing in details what you are trying to achieve and how you would have expected to be able to do that. We are currently working on Search 6.0 which brings a lot of improvements, in particular when it comes to using native features of Elasticsearch, so it just might be something we could slip into our backlog.
EDIT: I forgot to mention that, while you cannot use server-side scripts, you can still get the full source from your documents, and do some parsing in your application to achieve a similar result. This will work even in Search 5.8:
FullTextEntityManager fullTextEm = Search.getFullTextEntityManager(entityManager);
FullTextQuery query = fullTextEm.createFullTextQuery(
qb.keyword()
.onField( "tags" )
.matching( "round-based" )
.createQuery(),
VideoGame.class
)
.setProjection( ElasticsearchProjectionConstants.SCORE, ElasticsearchProjectionConstants.SOURCE );
Object[] projections = (Object[]) query.getSingleResult();
for (Object projection : projections) {
float score = (float) projection[0];
String source = (String) projection[1];
}
See this section of the documentation.
Hi I am trying to do query on elastic search by following the sql query and I want to implement same logic using Java API
select * from log , web where l.loghost = w.webhost and #datetime between '2016-05-20' AND '2016-05-25'
log and web are different types, and indices are set to logstash-log-* and logstash-web*, #timestamp format looks like "2016-05-20T17:14:01.037Z"
Now I have the following Java code but i don't know how to set between two dates ,so it does not return expected output
SearchResponse response = client.prepareSearch("logstash-log-*","logstash-web-*")
.setTypes("log","web")
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setFetchSource(new String[]{"*"}, null)
.setQuery(QueryBuilders.queryStringQuery("1.2.3.4").field("*_host"))// Query
.execute()
.actionGet();
Please guide I am new to Elastic search. Thanks in advance.
You need to combine a range query with your query_string query inside a bool/filter query:
QueryStringQueryBuilder qs = QueryBuilders.queryStringQuery("1.2.3.4").field("*_host");
RangeQueryBuilder range = QueryBuilders.rangeQuery("#timestamp")
.gte("2016-05-20T00:00:00.000Z")
.lte("2016-05-25T00:00:00.000Z");
and then
...
.setQuery(QueryBuilders.boolQuery().filter(qs).filter(range))
...
Is it possible not just sort results of query(com.google.appengine.api.search.Query), but also filter them? Like for example in this query com.google.appengine.api.datastore.Query:
Filter heightMinFilter =
new FilterPredicate("height",
FilterOperator.GREATER_THAN_OR_EQUAL,
minHeight);
Query q = new Query("Person").setFilter(heightMinFilter);
Search query has options that are needed for sorting, but I also want to filter result, because there are too many of them and I need only specific.
The filter is part of the query string.
A query string specifies conditions for the values of one or more
document fields. When you search an index you get back only those
documents in the index with fields that satisfy the query.
See example below or find more details here.
String queryString = "product: piano AND price < 5000";
Query query = Query.newBuilder().build(queryString);
Results<ScoredDocument> results = getIndex().search(queryString);
What seems almost natural in simple SQL is impossible in mongodb.
Given a simple document:
{
"total_units" : 100,
"purchased_unit" : 60
}
I would like to query the collection, using spring data Criteria class, where "total_units > purchased_units".
To my understanding it should be as trivial as any other condition.
Found nothing to support this on Spring api.
You can use the following pattern:
Criteria criteria = new Criteria() {
#Override
public DBObject getCriteriaObject() {
DBObject obj = new BasicDBObject();
obj.put("$where", "this.total_units > this.purchased_units");
return obj;
}
};
Query query = Query.query(criteria);
I don't think Spring Data API supports this yet but you may need to wrap the $where query in your Java native DbObject. Note, your query performance will be fairly compromised since it evaluates Javascript code on every record so combine with indexed queries if you can.
Native Mongodb query:
db.collection.find({ "$where": "this.total_units > this.purchased_units" });
Native Java query:
DBObject obj = new BasicDBObject();
obj.put( "$where", "this.total_units > this.purchased_units");
Some considerations you have to look at when using $where:
Do not use global variables.
$where evaluates JavaScript and cannot take advantage of indexes.
Therefore, query performance improves when you express your query
using the standard MongoDB operators (e.g., $gt, $in). In general, you
should use $where only when you can’t express your query using another
operator. If you must use $where, try to include at least one other
standard query operator to filter the result set. Using $where alone
requires a table scan. Using normal non-$where query statements
provides the following performance advantages:
MongoDB will evaluate non-$where components of query before $where
statements. If the non-$where statements match no documents, MongoDB
will not perform any query evaluation using $where. The non-$where
query statements may use an index.
As far as I know you can't do
query.addCriteria(Criteria.where("total_units").gt("purchased_units"));
but would go with your suggestion to create an additional computed field say computed_units that is the difference between total_units and purchased_units which you can then query as:
Query query = new Query();
query.addCriteria(Criteria.where("computed_units").gt(0));
mongoOperation.find(query, CustomClass.class);
Thanks #Andrew Onischenko for the historic good answer.
On more recent version of spring-data-mongodb (ex. 2.1.9.RELEASE), I had to write the same pattern like below:
import org.bson.Document;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.Query;
// (...)
Criteria criteria = new Criteria() {
#Override
public Document getCriteriaObject() {
Document doc = new Document();
doc.put("$where", "this.total_units > this.purchased_units");
return doc;
}
};
Query query = Query.query(criteria);
One way is this:
Criteria c = Criteria.where("total_units").gt("$purchased_unit");
AggregationOperation matchOperation = Aggregation.match(c);
Aggregation aggregation = Aggregation.newAggregation(matchOperation);
mongoTemplate.aggregate(aggregation, "collectionNameInStringOnly", ReturnTypeEntity.class);
Remember to put collection name in string so as to match the spellings of fields mentioned in criteria with fields in database collection.
I am trying to perform aggregations on values filtered by some conditions. I am using ElasticSearchTemplate.query() method of spring data too execute query and get the results in result extractor.
I am getting the hits correctly (i.e. filters are applied and docs matching those values are only retrieved.). However, aggregations are performed on all the docs. I believe aggregations should be applied to filtered values only. Following is the code I am using:
SearchQuery query = //get the query
SearchResponse hits = template.query(query, new ResultsExtractor<SearchResponse>() {
#Override
public SearchResponse extract(SearchResponse response) {
return response;
}
});
To debug the problem further, I wrote the code to execute the query rather than using spring data. Following is the code:
SearchRequestBuilder builder = esSetup.client().prepareSearch("document");
builder.setQuery(QueryBuilders.filteredQuery(QueryBuilders.matchAllQuery(), query.getFilter()));
builder.addFields(query.getFields().toArray(new String[query.getFields().size()]));
for(AbstractAggregationBuilder aggregation : query.getAggregations()){
builder.addAggregation(aggregation);
}
SearchResponse response = builder.get();
To my surprise, this query executed correctly and filters were applied on aggregates as well. To analyze further, I went through the code of elasticsearchtemplate and found that it uses setPostFilter method to set the filter. I then modified my code to set the filter that way:
SearchRequestBuilder builder = esSetup.client().prepareSearch("document");
// builder.setQuery(QueryBuilders.filteredQuery(QueryBuilders.matchAllQuery(), query.getFilter()));
builder.setPostFilter(query.getFilter());
builder.addFields(query.getFields().toArray(new String[query.getFields().size()]));
for(AbstractAggregationBuilder aggregation : query.getAggregations()){
builder.addAggregation(aggregation);
}
SearchResponse response = builder.get();
When I executed above code, it showed same behavior as spring data! (i.e. filters were applied on query but not aggregates.
Is this a bug of spring data es? If not, then, is there any other method which I should be using to retrieve the data the way I want?
Thanks in advance.
This behaviour is by design in Elasticsearch.
In very simple words, input to aggregations AND post filter is the set of documents that match the query section of the request body. Hence aggregations are not applied over the filtered documents.
However, if you do want aggregations to be applied over the filtered documents, "move the filters inside the query section", that is, use filtered query. Now output of the query section will be the filtered set of documents and aggregations will apply on them as expected.
So for your requirements, use filtered query instead of post filter.