I am doing a search to bring all registered cars 100 by 100 in a paginated way.
Pageable pageable = PageRequest.of(0, 100);
Page<Car> listOfcars = carService.findAll(pageable);
For me to get to the next page, I'm doing it via while:
while(listOfcars.hasNext()){
listOfcars = carService.findAll(listOfcars.nextPageable());
carService.start(listOfcars.getContent());
log.info("New batch sent" + listOfcars.getTotalElements());
}
There is a way to do this via the java stream, searching for the data on a page and right after that already searching for the sequences without having to do it via while ?
you can try by knowing total of available pages (maybe a total count will be needed first) and store each page numeric representation in a collection (let's say pages = [1, 2, ... n] ) then you can create a method OperateListOfCars and call your logic for each page with pages.stream()... maybe create a CustomPageResult wrapper and put there the pages collection will help to have the code more clean.
You don't need stream or loop here, you can use getContent method to return all content in a List from Page
Returns the page content as List.
Pageable pageable = PageRequest.of(0, 100);
Page<Car> listOfcars = carService.findAll(pageable);
List<Car> cars = listOfCars.getContent();
Related
I'm developing an application in Quarkus that integrates with the DynamoDB database. I have a query method that returns a list and I'd like this list to be paginated, but it would have to be done manually by passing the parameters.
I chose to use DynamoDBMapper because it gives more possibilities to work with lists of objects and the level of complexity is lower.
Does anyone have any idea how to do this pagination manually in the function?
DynamoDBScanExpression scanExpression = new DynamoDBScanExpression()
.withLimit(pageSize)
.withExclusiveStartKey(paginationToken);
PaginatedScanList<YourModel> result = mapper.scan(YourModel.class, scanExpression);
String nextPaginationToken = result.getLastEvaluatedKey();
You can pass the pageSize and paginationToken as parameters to your query method. The nextPaginationToken can be returned along with the results, to be used for the next page.
DynamoDB Mapper paginates by iterating over the results, by lazily loading the dataset:
By default, the scan method returns a "lazy-loaded" collection. It initially returns only one page of results, and then makes a service call for the next page if needed. To obtain all the matching items, iterate over the result collection.
Ref
For example:
List<Customer> result = mapper.scan(Customer.class, scanExpression);
for ( Customer cust : result ) {
System.out.println(cust.getId());
}
To Scan manually page by page you can use ScanPage
final DynamoDBScanExpression scanPageExpression = new DynamoDBScanExpression()
.withLimit(limit);
do {
ScanResultPage<MyClass> scanPage = mapper.scanPage(MyClass.class, scanPageExpression);
scanPage.getResults().forEach(System.out::println);
System.out.println("LastEvaluatedKey=" + scanPage.getLastEvaluatedKey());
scanPageExpression.setExclusiveStartKey(scanPage.getLastEvaluatedKey());
} while (scanPageExpression.getExclusiveStartKey() != null);
Ref
Ref
I have a Users endpoint to get all the users in MySQL sorted by a boolean value using pagination, as the number keeps growing I found out it's skipping some users and duplicating others here's some screenshots in postman where a user with the same id (95) is duplicating in 2 different pages
Also sometimes i can't find some users in all the pages
Page 2
Page 3
ServiceImpl
#Override
public PagedResponse<?> findAll(UserPrincipal currentUser, int page, int size) {
AppUtils.validatePageNumberAndSize(page, size);
//return users with 'isBoard' true first
Pageable pageable = PageRequest.of(page, size, Sort.Direction.DESC, "isBoard");
if (currentUser.getAuthorities().contains(new SimpleGrantedAuthority(RoleName.ROLE_ADMIN.toString()))
|| currentUser.getAuthorities().contains(new SimpleGrantedAuthority(RoleName.ROLE_SUPER_ADMIN.toString()))) {
Page<MEUser> contentPage = userRepository.findAll(pageable);
List<MEUser> list = contentPage.getNumberOfElements() == 0 ? Collections.emptyList() : contentPage.getContent();
return new PagedResponse<>(list.stream()
.map(user -> mapToAllUsersDTO(user, new AllUserDTO()))
.collect(Collectors.toList()), contentPage.getNumber(), contentPage.getSize(), contentPage.getTotalElements(), contentPage.getTotalPages(), contentPage.isLast());
} else {
Page<MEUser> contentPage = userRepository.findByMemberTrue(pageable);
List<MEUser> list = contentPage.getNumberOfElements() == 0 ? Collections.emptyList() : contentPage.getContent();
return new PagedResponse<>(list.stream()
.map(user -> mapToAllUsersDTO(user, new AllUserDTO()))
.collect(Collectors.toList()), contentPage.getNumber(), contentPage.getSize(), contentPage.getTotalElements(), contentPage.getTotalPages(), contentPage.isLast());
}
}
This is a database problem rather than in your code. The database does not ensure consistent results when there's multiple rows with the same value of an ordered field.
When using pagination with LIMIT and OFFSET you have to supply an order by field with unique values, so in your case you have to sort by isBoard and id to ensure consistent result.
This is because the if condition is validating only ADMIN and SUPER_USER. user 95 may have other roles that come in else conditions. Please check once and provide the database info for Roles and Users.
i added the id condition to the pagination:
Pageable pageable =
PageRequest.of(page, size, Sort.by("isBoard").descending().and(Sort.by("id")));
currently I am developing an api which uses Spring Data Pagination and I came to the problem that I passing a Pageable object as a request to my repository with which page I want to receive and how many elements should be on it. And with that my object looks like: Pageable pageable = PageRequest.of(0, 2); - so I want first page with two elements. In my database are 3 objects so therefore there will be 2 pages. But what ide shows me is: screenshot. Can anyone tell me why it shows that content is an array of 3 elements but actually I asked for 2 elements?
#Override
public List<NotificationDto> getLast30NotificationsFor(ApplicationUser user) {
Pageable pageable = PageRequest.of(0, 2);
Page<Notification> first30ByOwnerIdOrderByCreatedAtDesc = notificationRepository.findFirst30ByOwnerIdOrderByCreatedAtDesc(user.getId(), pageable);
List<Notification> notifications = new ArrayList<>(first30ByOwnerIdOrderByCreatedAtDesc.getContent());
return notifications.stream()
.map(NotificationToDtoMapper::map)
.collect(toList());
}
Try:
#Override
public List<NotificationDto> getLast30NotificationsFor(ApplicationUser user) {
Pageable page = PageRequest.of(0,2, Sort.by("createdAt").descending());
return notificationRepository.findByOwnerId(user.getId(), page)
.getContent()
.stream()
.map(NotificationToDtoMapper::map)
.collect(toList());
}
I am trying to implement pagination feature in Spring Data JPA.
I am referring this Blog
My Controller contains following code :
#RequestMapping(value="/organizationData", method = RequestMethod.GET)
public String list(Pageable pageable, Model model){
Page<Organization> members = this.OrganizationRepository.findAll(pageable);
model.addAttribute("members", members.getContent());
float nrOfPages = members.getTotalPages();
model.addAttribute("maxPages", nrOfPages);
return "members/list";
}
My DAO is following :
#Query(value="select m from Member m", countQuery="select count(m) from Member m")
Page<Organization> findMembers(Pageable pageable);
I am able to show first 20 records, how do I show next 20???
Is there any other pagination example that I can refer??
The constructors of Pageable are deprecated, use of() instead:
Pageable pageable = PageRequest.of(0, 20);
I've seen similar problem last week, but can't find it so I'll answer directly.
Your problem is that you specify the parameters too late. Pageable works the following way: you create Pageable object with certain properties. You can at least specify:
Page size,
Page number,
Sorting.
So let's assume that we have:
PageRequest p = new PageRequest(2, 20);
the above passed to the query will filter the results so only results from 21th to 40th will be returned.
You don't apply Pageable on result. You pass it with the query.
Edit:
Constructors of PageRequest are deprecated. Use Pageable pageable = PageRequest.of(2, 20);
Pageable object by default comes with size 20, page 0, and unsorted
So if you want the next page in front end the url can be sent with query params page, size,sort and these u can test it on postman.
You can use Page, List or Slice.
If you dont need the number of pages, and only need to know if the next page exists, use Slice, since it does not do the "count" query:
for (int i = 0; ; i++) {
Slice<Organization> pageOrganization = organizationRepository.find(PageRequest.of(0, 100));
List<Organization> organizationList = pageOrganization.getContent();
for (Organization org : organizationList) {
// code
}
if (!pageOrganization.hasNext()) {
break;
}
}
I am trying to limit the document size returned by my query.i want lets say 10 documents back only,any my query normally displays 22,how would i go buy setting a limit for the returned output. i am aware i can just limit the list size by creating a list and adding to that list however i want to do it on the query level.
My Query: Thanks in advance :)
ueryBuilder raceGenderQuery = QueryBuilders.boolQuery()
.must(termQuery("lep_etg_desc", "indian"))
.must(termQuery("lep_gen_desc", "male"));
Set<String> suburbanLocationSet = new HashSet<String>();
suburbanLocationSet.add("queensburgh");
suburbanLocationSet.add("umhlanga");
suburbanLocationSet.add("tongaat");
suburbanLocationSet.add("phoenix");
suburbanLocationSet.add("shallcross");
suburbanLocationSet.add("balito");
//Build the necessary location query.
QueryBuilder locationQuery = QueryBuilders.boolQuery().must(termsQuery("lep_suburb_home", suburbanLocationSet));
//Combine all Queries so that its filtered to get exact results.
FilteredQueryBuilder finalSearchQuery = QueryBuilders.filteredQuery(QueryBuilders.boolQuery().must(raceGenderQuery).must(locationQuery), FilterBuilders.boolFilter().must(FilterBuilders.rangeFilter("lep_age").gte(25).lte(45)).must(FilterBuilders.rangeFilter("lep_max_income").gte(25000).lte(45000)));
//Run Query through elasticsearch iterating through documents in the traceps index for query matches.
List<Leads> finalLeadsList = new ArrayList<Leads>();
for (Leads leads : this.leadsRepository.search(finalSearchQuery)) {
finalLeadsList.add(leads);
}
I think this is what you want:
SearchResponse response = client.prepareSearch().setSearchType(SearchType.QUERY_THEN_FETCH).setSize(10).setQuery(finalSearchQuery).execute().get
You have to use QUERY_THEN_FETCH for it to return exactly size results because otherwise it gets size results from each shard.