I have to write some REST service which should handle a million entries and return a response to the user in JSON format. I'm writing some REST-controller using Spring and make pagination.
public List<ContactDto> getAllContacts() {
double countItems = contactRepo.count();
int pages = (int) Math.ceil(countItems / totalItemsPerPage);
List<Contact> contacts = new ArrayList<>();
for (int i = 0; i < pages; i++) {
Page<Contact> page = contactRepo.findAll(PageRequest.of(i, totalItemsPerPage));
contacts.addAll(page.stream().collect(Collectors.toList()));
}
return contacts.stream()
.map(entity -> new ContactDto(entity.getId(), entity.getName()))
.collect(Collectors.toList());
}
I'm new in spring and pagination.
In this approach is have a sense or I'm doing something wrong?
I mean I want to know I'm using pagination write or wrong?
Thanks for the help!
It seems that you are collecting all the Contacts from all the pages and that does not make sense as you are storing all the data in memory negating all the lazy loading benefints.
I would suggest the following:
1.Rest controller should be able to accept pageNumber and pageSize arguments:
#GetMapping(value="/uri/{pageNumber}/{pageSize}")
public List<Contact> getContactsPage(#PathVariable("pageNumber") final Integer pageNumber, #PathVariable("pageSize") final Integer pageSize) {
//service or repository call
}
2.Repository interface should extent PagingAndSortingRepository:
public interface ContactRepository extends PagingAndSortingRepository<Contact, Long> {
Page<Contact> fingAll(Pageable pageable);
}
3.In your service or in controller directly create a Pageable object and pass it as ContactRepository#fingAll() argument:
final Pageable contactsPageable = PageRequest.of(pageNumber, pageSize);
4.Map Page to DTO if necessary.
You should take a look at Spring Data Rest.
Related
I have a global search I will search with one keyword but need to get results with all the matching columns of a table.
Page<A> a = null;
a = zRepo.getResultByNameSearch(searchText)
a = zRepo.getResultByNumberSeach(searchText)
a = zRepo.getRsultByProjectSearch(searchText)
#Query("select * from a_table x where x.name like :searchText")
Page<A> getResultByNameSearch(#Param("searchText") String searchText, Pageable pageable);
#Query("select * from a_table where x.number like :searchText")
Page<A> getResultByNumberSearch(#Param("searchText") String searchText, Pageable pageable);
#Query("select * from a_table where x.project like :searchText")
Page<A> getResultByProjectSearch(#Param("searchText") String searchText, Pageable pageable);
So each repository call queries and fetches the same table but according to the searchText.
Let's assume name = "Company910", number = "XX910", project = "910".
Now I'm searching for "910" and want to get results with all the 3 values. Page<a> will be having all the columns of a_table with the list of results as per "Company910", "XX910", "910".
How to implement this or is there any other way where I can maintain a single query instead of three different for name, number and project?
For the first part of how to get the combined result. I will do something like this.
Create repository interface as you wrote but using spring data jpa intead of raw query
user CompletableFuture with each method call
combine the result into a dto
return this combined result
#Repository
public interface SampleDocumentRepository extends JpaRepository<SampleDocument, Integer> {
Page<SampleDocument> findByNameContains(String name, Pageable pageable);
Page<SampleDocument> findByNumberContains(String number, Pageable pageable);
Page<SampleDocument> findByProjectContains(String project, Pageable pageable);
}
Service class where result are combined together
#Service
#AllArgsConstructor
public class SampleDocumentService {
private final SampleDocumentRepository repository;
#Transactional(readOnly = true)
public CompletableFuture<ResultDto> search(String query) {
PageRequest page = PageRequest.of(0, 20);
CompletableFuture<Page<SampleDocument>> nameSearch = CompletableFuture.supplyAsync(() -> repository.findByNameContains(query, page));
CompletableFuture<Page<SampleDocument>> numberSearch = CompletableFuture.supplyAsync(() -> repository.findByNumberContains(query, page));
CompletableFuture<Page<SampleDocument>> projectSearch = CompletableFuture.supplyAsync(() -> repository.findByProjectContains(query, page));
return CompletableFuture.allOf(nameSearch, numberSearch, projectSearch)
.thenApply(unused -> new ResultDto(nameSearch.join(), numberSearch.join(), projectSearch.join()));
}
}
Then the call from the service
#GetMapping("/search")
public CompletableFuture<ResultDto> search(#RequestParam("query") String query) {
return service.search(query);
}
call using your query argument
http://localhost:8080/sample/search?query=hello
To answer the second part, if you want to check if the query is present in any of the columns you can write JPA query combining the Or operator like this.
Page<SampleDocument> findByNameContainsOrNumberContainsOrProjectContains(String name, String number, String project, Pageable pageable);
Caller would be something like this
#Transactional(readOnly = true)
public CompletableFuture<ResultDto> searchAll(String query) {
PageRequest page = PageRequest.of(0, 20);
CompletableFuture<Page<SampleDocument>> nameSearch = CompletableFuture.supplyAsync(() ->
repository.findByNameContainsOrNumberContainsOrProjectContains(query, query, query, page));
return CompletableFuture.supplyAsync(() -> new ResultDto(nameSearch.join(), null, null));
}
If you want to use Async with spring data and completable please follow this link
I have developed following three APIs for my spring-boot application:
#GetMapping(value = "/state-transitions/searchWithFromState")
public ResponseEntity<List<TcStateTransitionDTO>>
searchWithFromState(
#RequestParam(value = "fromStateId") String fromStateId) {
return ResponseEntity.ok(stateTransitionService.findByFromState(fromStateId));
}
#GetMapping(value = "/state-transitions/searchWithFromStateAndToState")
public ResponseEntity<List<TcStateTransitionDTO>>
searchWithFromStateAndToState(
#RequestParam(value = "fromStateId") String fromStateId,
#RequestParam(value = "toStateId") String toStateId) {
return ResponseEntity.ok(stateTransitionService
.findByFromStateAndToState(fromStateId, toStateId));
}
#GetMapping(value = "/state-transitions/searchWithFromStateAndAction")
public ResponseEntity<List<TcStateTransitionDTO>>
searchWithFromStateAndAction(
#RequestParam(value = "fromStateId") String fromStateId,
#RequestParam(value = "actionId") String actionId) {
return ResponseEntity.ok(stateTransitionService
.findByFromStateAndAction(fromStateId, actionId));
}
These APIs are working perfectly. But I am wondering if is there any way to write these APIs in a better fashion. I am thinking this because, if say, there are n params to search, in this way, I will end up in write 2^n-1 number of APIs.
Could anyone please help here? Thanks.
You can receive variable number of parameters if you receive them as a Map<String,Object> form like this:
#GetMapping(value = "/search")
public ResponseEntity<Page<TcStateTransitionDTO>> search(#RequestParam Map<String, Object> params) {
return ResponseEntity.ok(stateTransitionService.searchByParams(params));
}
You can create a criteria map and generate dynamic query based on the parameters using criteriaBuilder. If you're using JPA, then just pass the specification generated through criteriabuilder to the findAll method.
public Page<TcStateTransitionDTO> searchByParams(Map<String, Object> params) {
PageRequest pageRequest = generatePageRequestFromParams(params);
Specifications specifications = getSearchSpecifications(params);
return repository.findAll(specifications, pageRequest);
}
This getSpecification(Map<String, Object> params) method that I've mentioned is the main gameplayer here. The main trick is to write this method efficiently. I would suggest to read the above link to know more about CriteriaBuilder and do a bit study on specifications.
I am using projection, and I need to format the date variable in list of objects, but Page<> type response is read only, so I can not iterate and modify the objects. What Should I do?
#Override
public Page<OrderDto> findAll(Pageable pageable) {
return OrderDao.findByIsEnabledTrue(pageable, OrderDto.class);
}
I tried like this:
#Override
public Page<OrderDto> findAll(Pageable pageable) {
Page<OrderDto> page = orderDao.findByIsEnabledTrue(pageable, OrderDto.class);
List<OrderDto> orderDtos = page.getContent();
orderDtos.stream().forEach(it ->{
it.setFormattedCreatedAt(new SimpleDateFormat("dd-M-yyyy").format(it.getCreatedAt()));
});
page.getContent().clear();
page.getContent().addAll(orderDtos);
return page;
}
page.getContent() returns an unmodifiable list. The best approach is to use map method provided by page:
#Override
public Page<OrderDto> findAll(Pageable pageable) {
Page<OrderDto> page = orderDao.findByIsEnabledTrue(pageable, OrderDto.class);
page = page.map(this :: transformOrderDto);
return page;
}
private OrderDto transformOrderDto(final OrderDto order) {
order. setFormattedCreatedAt(new SimpleDateFormat("dd-M-yyyy").format(order.getCreatedAt()));
return order;
}
Since 1.10, Page has supported a map method that is specifically meant to let you transform the objects contained in it.
What's the best practice to create search function in spring boot with spring data jpa?
#GetMapping("/search")
public List<Hotel> getAllByCriteria(#RequestParam MultiValueMap<String, String> criteria) {
if (criteria.containsKey("cityPublicId")) {
String cityPublicId = criteria.getFirst("cityPublicId");
if (criteria.containsKey("amenity")) {
List<String> amenities = criteria.get("amenity");
return svc.findAllByCityAndAmenities(cityPublicId, amenities);
}
return svc.findAllByCity(cityPublicId);
}
//currently only support one amenity filtration
else if (criteria.containsKey("amenity")) {
return svc.findAllByAmenities(criteria.get("amenity"));
}
return null;
}
Currently I have to identify all possible combination of criteria to use corresponding method, Is there a universal way to handle all condition? Or at least not hardcoding all possible combination.
PS: If I want to filter result by multiple amenities, may I use findByAmenitiesContains(set)? Where a Hotel entity has a set of amenity. Do I have to create custom query using #query?
Thanks.
You basically have the following options:
create the query programmatically from the input data using a custom method. This gives you maximum flexibility but also requires the most work.
Use a specification. Almost the same flexibility and almost as much work.
Use query by example. Very little work, limited flexibility.
Regarding the PS: The capabilities of query derivation are well documented.
AFAIR you can use different request payload entities to handle the same endpoint
#GetMapping(path = "/search", params = { "cityId" })
public List<Hotel> getAllByCriteria(ByCityPublicId byCity) {
return svc.findAllByCity(byCity.getCityPublicId())
}
#GetMapping(path = "/search", params = { "cityId", "amenity" })
public List<Hotel> getAllByCriteria(ByCityPublicIdAndAmenity byCityAndAmenitities) {
return svc.findAllByCityAndAmenities(byCityAndAmenitities.getCityPublicId(), byCityAndAmenitities.getAmenitities())
}
#GetMapping(path = "/search", params = { "amenity" })
public List<Hotel> getAllByCriteria(ByAmenity byAmenity) {
return svc.findAllByAmenities(byAmenity.getAmenity());
}
I have a Spring Boot 1.5.2 project which up till now has been sending lists back to the view. I would like to use pagination instead of lists. I began to change the code in the service layer and repository, but it has not been a simple case of changing from List<T> to Page<T>.
In particular on the service layer I was returning different List<T> based on user role and then passing this list to a method that converts to Dto before sending it back to controller.
Spring Boot 1.5.2 appears to use Spring-data-jpa:1.11
Controller
#GetMapping("/dashboard/sale")
public String dashboard(#RequestParam(name = "p", defaultValue = "1") int pageNumber, Model model, HttpServletRequest request) {
List<SaleDashboard> listSaleDashboard = saleService.getPage(pageNumber);
model.addAttribute("listSaleDashboard", listSaleDashboard);
return "dashboard";
}
Service Layer
public List<SaleDashboard> getPage(int pageNumber) {
PageRequest request = new PageRequest(pageNumber - 1, PAGESIZE, Sort.Direction.ASC, "id");
List<Sale> listSale = new ArrayList<>();
if (roles.contains("ROLE_ADMIN")) {
listSale = saleRepository.findBySomeProperty(user.getUserDetails().getReportDept());
}
if (roles.contains("ROLE_USER")) {
listSale = saleRepository.findByListOfCreatingUser(userList);
}
List<SaleDashboard> listSaleDashboard = createSaleDashboard(listSale);
return listSaleDashboard;
}
public List<SaleDashboard> createSaleDashboard(List<Sale> sales) {
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("dd/MM/yyyy");
List<SaleDashboard> listSaleDashboard = new ArrayList<>();
for (Sale sale: sales) {
SaleDashboard saleDashboard = new SaleDashboard();
saleDashboard.setSaleId(sale.getId());
// ETC
listSaleDashboard.add(saleDashboard);
}
return listSaleDashboard;
}
In the service layer above I began to use PageRequest but thats as far as I got.
Repository
public interface SaleRepository extends JpaRepository<Sale, Long> , PagingAndSortingRepository<Sale, Long> {
#Query("SELECT e FROM Sale e WHERE e.creatingUser in (:userList)")
Page<Sale> findByListOfCreatingUser(#Param("userList") List<User> users, Pageable pageable);
}
How would I implement a similar service layer but use Page<T> instead of List<T>?