I use spring-data-elasticsearch framework to get query result from elasticsearch server, the java code like this:
public void testQuery() {
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withFields("createDate","updateDate").withQuery(matchAllQuery()).withPageable(new PageRequest(0,Integer.MAX_VALUE)).build();
List<Entity> list = template.queryForList(searchQuery, Entity.class);
for (Entity e : list) {
System.out.println(e.getCreateDate());
System.out.println(e.getUpdateDate());
}
}
I get the raw query log in server, like this:
{"from":0,"size":10,"query":{"match_all":{}},"fields":["createDate","updateDate"]}
As per the query log, spring-data-elasticsearch will add size limit to the query. "from":0, "size":10, How can I avoid it to add the size limit?
You don't want to do this, you could use the findAll functionality on a repository that returns an Iterable. I think the best way to obtain all items is to use the scan/scroll functionality. Maybe the following code block can put you in the right direction:
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(QueryBuilders.matchAllQuery())
.withIndices("customer")
.withTypes("customermodel")
.withSearchType(SearchType.SCAN)
.withPageable(new PageRequest(0, NUM_ITEMS_PER_SCROLL))
.build();
String scrollId = elasticsearchTemplate.scan(searchQuery, SCROLL_TIME_IN_MILLIS, false);
boolean hasRecords = true;
while (hasRecords) {
Page<CustomerModel> page = elasticsearchTemplate.scroll(scrollId, SCROLL_TIME_IN_MILLIS, CustomerModel.class);
if (page != null) {
// DO something with the records
hasRecords = (page.getContent().size() == NUM_ITEMS_PER_SCROLL);
} else {
hasRecords = false;
}
}
Related
I'm trying to return random element in Spring using Query.
I have this:
#Override
public List<AdventureHolidays> findRandomTrekking() {
Query query = new Query();
query.addCriteria(Criteria.where("typeOfAdventureHolidays").is("trekking"));
return mongoTemplate.find(query, AdventureHolidays.class);
}
But this return me all elements that match my criteria,
I tried with:
return mongoTemplate.findOne(query, AdventureHolidays.class); but then I have required type List provided AdventureHoliday
Also I was using and tried with this, but on this way elements appear twice sometimes:
#Aggregation(pipeline = {"{'$match':{'typeOfAdventureHolidays':'trekking'}}", "{$sample:
{size:1}}"})
So I find a way with this Query, but its listing me all documents while I want just one random from collection
After some discussion this is what OP asked for:
private static Queue<AdventureHolidays> elementsToReturn = new LinkedList<>();
public AdventureHolidays findRandomTrekking() {
if (elementsToReturn.size() == 0) { //fetch data from db
Query query = new Query();
query.addCriteria(Criteria.where("typeOfAdventureHolidays")
.is("trekking"));
List<AdventureHolidays> newData = mongoTemplate.find(query, AdventureHolidays.class)
Collections.shuffle(newData);
elementsToReturn.addAll(newData);
}
return elementsToReturn.poll(); //this will crash if database is empty
}
Original answer.
You need to change return type of a method:
public AdventureHolidays findRandomTrekking() {
Query query = new Query();
query.addCriteria(Criteria.where("typeOfAdventureHolidays").is("trekking"));
return mongoTemplate.findOne(query, AdventureHolidays.class);
}
I have a function that returns a list of templates(unsorted); can someone tell me how I can get the list in a sorted format?
public List<FormTemplate> listDomainTemplates(Integer id) {
Domain domain = domainService.getDomain(id);
if (domain == null) {
return new ArrayList<>();
}
CriteriaBuilder cb = sessionFactory.getCurrentSession().getCriteriaBuilder();
CriteriaQuery<FormTemplate> query = cb.createQuery(FormTemplate.class);
Root<FormTemplate> application = query.from(FormTemplate.class);
query.select(application);
Predicate predicate = cb.equal(application.get("domain"), domain);
query.where(predicate);
Query<FormTemplate> q = sessionFactory.getCurrentSession().createQuery(query);
return q.getResultList();
}
Please experiment with following:
query.orderBy(cb.asc(application.get(...));
... - should point to FormTemplate field to sort, I suppose.
I have this implementation to remove documents based on some id using ReactiveMongoTemplate. I'm trying to get the size of the the list of impacted documents but it always returns 0, and since it is reactive I'm not sure how to get the number of records deleted
#Override
public int deleteMongoDataForGivenId(Long id) {
int deletedRecords = 0;
Query query = new Query();
query.addCriteria(where("id").is(id));
Flux<Object> deletedDocs = reactiveMongoTemplate.findAllAndRemove(query, Object.class, "SomeCollection");
if(!deletedDocs.collectList().block().isEmpty()) {
List<Object> listOfRecords = deletedDocs.collectList().block();
deletedRecords = listOfRecords.size();
}
}
Doing it the reactive way would be to return a Mono<Long> instead of blocking and unpacking the mono into an long or int:
public Mono<Long> deleteMongoDataForGivenId(Long id) {
Query query = new Query();
query.addCriteria(where("id").is(id));
return reactiveMongoTemplate
.findAllAndRemove(query, MyDocument.class, "SomeCollection")
.count();
}
Having to use a blocking method defies the purpose of reactive programming, but if you don't really have a choice, you can do the following:
public Long deleteMongoDataForGivenId(Long id) {
Query query = new Query();
query.addCriteria(where("id").is(id));
return reactiveMongoTemplate
.findAllAndRemove(query, MyDocument.class, "SomeCollection")
.count()
// Please don't do this!!!
.share().block();
}
I'm trying to see and use the invidual _score of each hit when doing a search by a SearchQuery. This is, among other things, to know in what range of scores my searches result in. But other than setting a MinScore using searchQuery.withMinScore(float); I can't find any method for handling the scores of search.
#Override
public Page<Website> listsearch(SearchBody searchBody, int size, int page) {
BoolQueryBuilder qb = QueryBuilders.boolQuery();
for(SearchUnit unit:searchBody.getSearchBody()){
if(unit.isPriority()) {
qb.must(matchQuery("_all", unit.getWord()).operator(MatchQueryBuilder.Operator.AND)
.fuzziness(Fuzziness.AUTO));
}else {
qb.should(termQuery("_all", unit.getWord())
.boost(unit.getWeight()));
}
}
for(SearchUnit ExUnit:searchBody.getExcludeBody()){
qb.mustNot(matchPhraseQuery("_all",ExUnit.getWord()));
}
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withIndices("websites_v1")
.withTypes("website")
.withQuery(qb)
.withMinScore(0.05F)//Magical minscore
.withPageable(new PageRequest(page, size))
.build();
Page<Website> search = searchRepository.search(searchQuery);
return search;
}
The search function used is from org.springframework.data.elasticsearch.repository; defined as
Page<T> search(SearchQuery var1);
So my question is there anyway I can access the score of each returned object in the Page? Or do I need to switch my query method to something else to achive that?
This is not possible with the Spring Data ElasticSearch repositories.
You need to autowire an EntityMapper and an ElasticSearchTemplate and extract the score yourself. Something like this should work:
Pageable pageRequest = new PageRequest(0, 10);
Page<Website> result = elasticSearchTemplate.query(searchQuery, new ResultsExtractor<Page<Website>>() {
#Override
public Page<Website> extract(SearchResponse response) {
List<Website> content = new ArrayList<>();
SearchHit[] hits = response.getHits().getHits();
for (SearchHit hit : hits) {
Website website = entityMapper.mapToObject(hit, Website.class);
content.add(website);
float documentScore = hit.getScore(); // <---- score of a hit
}
return new PageImpl<Website>(content, pageRequest, response.getHits().getTotalHits());
}
});
I am trying to add multiple 'and' conditions to criteria in Spring Data but not able to figure out what am I doing wrong.
Please refer the following code :
Criteria criteria = new Criteria();
criteria.andOperator(Criteria.where("siteCode").is(siteCode));
if(paymentMode != null) {
criteria.andOperator(Criteria.where("paymentMode").is(paymentMode));
}
if(planCode != null) {
criteria.andOperator(Criteria.where("packageCode").is(planCode));
}
if(status){
criteria.andOperator(Criteria.where("expiryDateTime").gt(new Date()));
} else {
criteria.andOperator(Criteria.where("expiryDateTime").lte(new Date()));
}
Query query = new Query(criteria);
List<UserPackage> userPackageList = mongoTemplate.find(query, UserPackage.class);
I found out what I was doing wrong. You don't need to use method 'andOperator' and another Criteria inside.
It would simply work by using 'and' method and then chain the following operator method you want to use.
Sharing the working solution to help other newcomers like me.
Criteria criteria = new Criteria();
criteria = criteria.and("siteCode").is(siteCode);
if (paymentMode != null) {
criteria = criteria.and("paymentMode").is(paymentMode);
}
if (planCode != null) {
criteria = criteria.and("packageCode").is(planCode);
}
if (status) {
criteria = criteria.and("expiryDateTime").gt(new Date());
} else {
criteria = criteria.and("expiryDateTime").lte(new Date());
}
Query query = new Query(criteria);
Just in case if you want to add conditions for a date between and match another field, below is what I used.
Criteria criteria = Criteria.where("date").gte(from).andOperator(Criteria.where("date").lt(to));
if (StringUtils.isNotBlank(manager)) {
criteria = criteria.and("manager").is(manager);
}