Develop Reference

How to return one random element by Query

I'm trying to return random element in Spring using Query.
I have this:
#Override
public List<AdventureHolidays> findRandomTrekking() {
Query query = new Query();
query.addCriteria(Criteria.where("typeOfAdventureHolidays").is("trekking"));
return mongoTemplate.find(query, AdventureHolidays.class);
}
But this return me all elements that match my criteria,
I tried with:
return mongoTemplate.findOne(query, AdventureHolidays.class); but then I have required type List provided AdventureHoliday
Also I was using and tried with this, but on this way elements appear twice sometimes:
#Aggregation(pipeline = {"{'$match':{'typeOfAdventureHolidays':'trekking'}}", "{$sample:
{size:1}}"})
So I find a way with this Query, but its listing me all documents while I want just one random from collection

After some discussion this is what OP asked for:
private static Queue<AdventureHolidays> elementsToReturn = new LinkedList<>();
public AdventureHolidays findRandomTrekking() {
if (elementsToReturn.size() == 0) { //fetch data from db
Query query = new Query();
query.addCriteria(Criteria.where("typeOfAdventureHolidays")
.is("trekking"));
List<AdventureHolidays> newData = mongoTemplate.find(query, AdventureHolidays.class)
Collections.shuffle(newData);
elementsToReturn.addAll(newData);
}
return elementsToReturn.poll(); //this will crash if database is empty
}
Original answer.
You need to change return type of a method:
public AdventureHolidays findRandomTrekking() {
Query query = new Query();
query.addCriteria(Criteria.where("typeOfAdventureHolidays").is("trekking"));
return mongoTemplate.findOne(query, AdventureHolidays.class);
}

ReactiveMongoTemplate does not return number of documents removed

I have this implementation to remove documents based on some id using ReactiveMongoTemplate. I'm trying to get the size of the the list of impacted documents but it always returns 0, and since it is reactive I'm not sure how to get the number of records deleted
#Override
public int deleteMongoDataForGivenId(Long id) {
int deletedRecords = 0;
Query query = new Query();
query.addCriteria(where("id").is(id));
Flux<Object> deletedDocs = reactiveMongoTemplate.findAllAndRemove(query, Object.class, "SomeCollection");
if(!deletedDocs.collectList().block().isEmpty()) {
List<Object> listOfRecords = deletedDocs.collectList().block();
deletedRecords = listOfRecords.size();
}
}

Doing it the reactive way would be to return a Mono<Long> instead of blocking and unpacking the mono into an long or int:
public Mono<Long> deleteMongoDataForGivenId(Long id) {
Query query = new Query();
query.addCriteria(where("id").is(id));
return reactiveMongoTemplate
.findAllAndRemove(query, MyDocument.class, "SomeCollection")
.count();
}
Having to use a blocking method defies the purpose of reactive programming, but if you don't really have a choice, you can do the following:
public Long deleteMongoDataForGivenId(Long id) {
Query query = new Query();
query.addCriteria(where("id").is(id));
return reactiveMongoTemplate
.findAllAndRemove(query, MyDocument.class, "SomeCollection")
.count()
// Please don't do this!!!
.share().block();
}

Score of each hit with spring SearchQuery ElasticSearch

I'm trying to see and use the invidual _score of each hit when doing a search by a SearchQuery. This is, among other things, to know in what range of scores my searches result in. But other than setting a MinScore using searchQuery.withMinScore(float); I can't find any method for handling the scores of search.
#Override
public Page<Website> listsearch(SearchBody searchBody, int size, int page) {
BoolQueryBuilder qb = QueryBuilders.boolQuery();
for(SearchUnit unit:searchBody.getSearchBody()){
if(unit.isPriority()) {
qb.must(matchQuery("_all", unit.getWord()).operator(MatchQueryBuilder.Operator.AND)
.fuzziness(Fuzziness.AUTO));
}else {
qb.should(termQuery("_all", unit.getWord())
.boost(unit.getWeight()));
}
}
for(SearchUnit ExUnit:searchBody.getExcludeBody()){
qb.mustNot(matchPhraseQuery("_all",ExUnit.getWord()));
}
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withIndices("websites_v1")
.withTypes("website")
.withQuery(qb)
.withMinScore(0.05F)//Magical minscore
.withPageable(new PageRequest(page, size))
.build();
Page<Website> search = searchRepository.search(searchQuery);
return search;
}
The search function used is from org.springframework.data.elasticsearch.repository; defined as
Page<T> search(SearchQuery var1);
So my question is there anyway I can access the score of each returned object in the Page? Or do I need to switch my query method to something else to achive that?

This is not possible with the Spring Data ElasticSearch repositories.
You need to autowire an EntityMapper and an ElasticSearchTemplate and extract the score yourself. Something like this should work:
Pageable pageRequest = new PageRequest(0, 10);
Page<Website> result = elasticSearchTemplate.query(searchQuery, new ResultsExtractor<Page<Website>>() {
#Override
public Page<Website> extract(SearchResponse response) {
List<Website> content = new ArrayList<>();
SearchHit[] hits = response.getHits().getHits();
for (SearchHit hit : hits) {
Website website = entityMapper.mapToObject(hit, Website.class);
content.add(website);
float documentScore = hit.getScore(); // <---- score of a hit
}
return new PageImpl<Website>(content, pageRequest, response.getHits().getTotalHits());
}
});

how to disable page query in Spring-data-elasticsearch

I use spring-data-elasticsearch framework to get query result from elasticsearch server, the java code like this:
public void testQuery() {
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withFields("createDate","updateDate").withQuery(matchAllQuery()).withPageable(new PageRequest(0,Integer.MAX_VALUE)).build();
List<Entity> list = template.queryForList(searchQuery, Entity.class);
for (Entity e : list) {
System.out.println(e.getCreateDate());
System.out.println(e.getUpdateDate());
}
}
I get the raw query log in server, like this:
{"from":0,"size":10,"query":{"match_all":{}},"fields":["createDate","updateDate"]}
As per the query log, spring-data-elasticsearch will add size limit to the query. "from":0, "size":10, How can I avoid it to add the size limit?

You don't want to do this, you could use the findAll functionality on a repository that returns an Iterable. I think the best way to obtain all items is to use the scan/scroll functionality. Maybe the following code block can put you in the right direction:
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(QueryBuilders.matchAllQuery())
.withIndices("customer")
.withTypes("customermodel")
.withSearchType(SearchType.SCAN)
.withPageable(new PageRequest(0, NUM_ITEMS_PER_SCROLL))
.build();
String scrollId = elasticsearchTemplate.scan(searchQuery, SCROLL_TIME_IN_MILLIS, false);
boolean hasRecords = true;
while (hasRecords) {
Page<CustomerModel> page = elasticsearchTemplate.scroll(scrollId, SCROLL_TIME_IN_MILLIS, CustomerModel.class);
if (page != null) {
// DO something with the records
hasRecords = (page.getContent().size() == NUM_ITEMS_PER_SCROLL);
} else {
hasRecords = false;
}
}

scan count returns significantly less number for a dynamodb table

I'm running a sample java program to query a dynamodb table, the table has about 90000 items but when i get the scan count from java it shows only 1994 items
ScanRequest scanRequest = new ScanRequest().withTableName(tableName);
ScanResult result = client.scan(scanRequest);
System.out.println("#items:" + result.getScannedCount());
the program outputs #items:1994
but the detail from amazon aws console shows:
Item Count*: 89249
any idea?
thanks

scanning or querying dynamodb only returns maximum of 1MB of data.
the count is the number of return items fit in 1MB. in order to get the whole table, you should aggressively scan the database until the value LastEvaluatedKey is null

Set your book object with correct hash key value, and use DynamoDBMapper to get the count.
DynamoDBQueryExpression<Book> queryExpression = new DynamoDBQueryExpression<Book>()
.withHashKeyValues(book);
dynamoDbMapper.count(Book.class, queryExpression);

This should help . Worked for me
AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard()
.withRegion("your region").build();
DynamoDB dynamoDB = new DynamoDB(client);
TableDescription tableDescription = dynamoDB.getTable("table name").describe();
tableDescription.getItemCount();

Based on answer from nightograph
private ArrayList<String> fetchItems() {
ArrayList<String> ids = new ArrayList<>();
ScanResult result = null;
do {
ScanRequest req = new ScanRequest();
req.setTableName("table_name");
if (result != null) {
req.setExclusiveStartKey(result.getLastEvaluatedKey());
}
result = amazonDynamoDBClient.scan(req);
List<Map<String, AttributeValue>> rows = result.getItems();
for (Map<String, AttributeValue> map : rows) {
AttributeValue v = map.get("rangeKey");
String id = v.getS();
ids.add(id);
}
} while (result.getLastEvaluatedKey() != null);
System.out.println("Result size: " + ids.size());
return ids;
}

I agreed with nightograph. I thinks this link is useful.
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryAndScan.html
I just tested with this example. Anyway this is the Dyanamodb v2.
final ScanRequest scanRequest = new ScanRequest()
.withTableName("table_name");
final ScanResult result = dynamoDB.scan(scanRequest);
return result.getCount();