How to read documents from elasticsearch using Spring Data? - java

I have created an index (house) with a type "apartments" that contains 20 documents. I uploaded the Json as a binary file into elasticsearch using postman. I have a Spring Boot project that has the following classes:
EsConfig.java - I have configured the clustername which is the default name in the application.properties file.
#Configuration
#EnableElasticsearchRepositories(basePackages = "com.search.repository")
public class EsConfig {
#Value("${elasticsearch.clustername}")
private String EsClusterName;
#Bean
public Client esClient() throws UnknownHostException {
Settings esSettings = Settings.builder()
.put("cluster.name", EsClusterName)
.put("client.transport.sniff", true)
.put("client.transport.ignore_cluster_name", false)
.build();
TransportClient client = new PreBuiltTransportClient(esSettings)
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"), 9300));
return client;
}
#Bean
public ElasticsearchOperations elasticsearchTemplate() throws Exception{
return new ElasticsearchTemplate(esClient());
}
}
Apartments.java - This is my data model. The documents have the below fields in elasticsearch.
#Document(indexName = "house", type = "apartments")
#JsonIgnoreProperties(ignoreUnknown=true)
public class Apartments {
#Id
private String id;
#JsonProperty("Apartment_Name")
private String apartmentName;
#JsonProperty("Apartment_ID")
private String apartmentId;
#JsonProperty("Area_Name")
private String areaName;
//constructors along with getters and setters
}
ApartmentSearchRepository.java - This is an interface that extends the ElasticsearchRepository interface to perform crud operations.
public interface ApartmentSearchRepository extends ElasticsearchRepository<Apartments, String> {
List<Apartments> findByApartmentName(String apartmentName);
}
EsApartmentService.java -
#Service
public class EsApartmentService {
#Autowired
ApartmentSearchRepository apartmentSearchRepository;
public List<Apartments> getApartmentByName(String apartmentName) {
return apartmentSearchRepository.findByApartmentName(apartmentName);
}
}
ApartmentController.java - I have created an endpoint that should give back those 20 documents from elasticsearch. (Also, Apartment is a POJO in my project and Apartments is the data model.)
#Autowired
EsApartmentService esApartmentService;
#GetMapping(path = "/search",produces = "application/json")
public Set<Apartment> searchApartmentByName(
#RequestParam(value = "apartmentName", defaultValue = "") String apartmentName) throws IOException {
List<Apartment> apartments= new ArrayList<>();
esApartmentService.getApartmentByName(apartmentName).forEach(apartment-> {
apartments.add(new Apartment(apartment.getApartmentName(), apartment.getApartmentId(), apartment.getAreaName()));
});
return apartments.stream()
.collect(Collectors.toCollection(() -> new TreeSet<>(Comparator.comparing(Apartment::getApartmentId))));
}
This code gives back a status of 200 but with an empty response. I tried debugging but it seems that it is unable to read those documents from elasticsearch. I went through a couple of solutions but most of them have set the document data from within the code itself.
I am unable to retrieve those documents by hitting the endpoint I specified in the controller. Can someone let me know what I could be missing out on? Thanks! :)
Edit: The screenshot below shows the query and response in Postman.

As far I know, you are able to use #JsonProperty in order to map the POJO to the query response but you're loosing the ability to use the dynamic finder methods (findBy*) of spring data. The dynamic finders generation of spring data relies on reflection and there is where the field names in your POJO become important.
Would you mind to change the field names of you POJO or in your documents to verify this? Or just define a custom query? There is also a powerfull java api where you can define more complex queries: https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#elasticsearch.misc.filter

As mentioned above by #ibexit, I removed #JsonProperty and used the native search query builder in my service. Also, it was not taking Apartment_Name and worked when I gave apartment_Name. (seems like Elasticsearch has case issues so I gave it in Camel Case.)
My changes:
Apartments.java - Removed #JsonProperty
#Document(indexName = "house", type = "apartments")
//#JsonIgnoreProperties(ignoreUnknown=true)
public class Apartments {
#Id
private String id;
//#JsonProperty("apartment_ID")
private String apartment_ID;
//#JsonProperty("Area_Name")
private String area_Name;
//#JsonProperty("Apartment_Name")
private String apartment_Name;
}
EsApartmentService.java -
#Service
public class EsApartmentService {
#Autowired
private ElasticsearchTemplate elasticsearchTemplate;
public List<Apartments> getApartmentByName(String apartmentName) {
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(org.elasticsearch.index.query.QueryBuilders
.matchQuery("apartment_Name", apartmentName)).build();
Page<Apartments> sampleEntities =
elasticsearchTemplate.queryForPage(searchQuery,Apartments.class);
return sampleEntities.getContent();
}
}
Removed ApartmentSearchRepository.java file.
These changes gave me the required response! :)

Related

How to return required parameters as json response from pojo class in spring boot?

what I am trying to do is,
If I take one pojo class like
#Entity
#Table(name = "property_table")
public class Property {
#Id
#Column(name = "property_id")
#GeneratedValue(strategy = GenerationType.IDENTITY)
private int propertyId;
#Column(name = "property_name")
private String propertyName;
#Column(name = "property_type")
private String propertyType;
}
In RestController I wrote Two Methods like
#GetMapping(value = "/getProperties", produces = { "application/json",
"application/xml" }, consumes = { "application/xml", "application/json" })
#ResponseBody
public List<Property> getProperties() {
//some code
}
#GetMapping(value = "/getPropertyById", produces = { "application/json",
"application/xml" }, consumes = { "application/xml", "application/json" })
#ResponseBody
public Property getPropertyById() {
//some code
}
So, hear what I am trying to do is
for first api method I want return json like some parameters from Property pojo class i.e., like
for getProperties api method
{
"property":[
{
"propertyId":001,
"propertyName":"PROPERTY 1"
},
{
"propertyId":002,
"propertyName":"PROPERTY 2"
}
],
In the Above json I want to return only two parameters i.e propertyId,propertyName and remaining parameter i.e propertyType I dont want to retun in json.
How to return like that?
and for the second api method I want to return all three parameters. i.e., like below
for getPropertyById api method
{
"propertyId":001,
"propertyName":"PROPERTY 1",
"propertyType:"PROPERTY_TYPE 1"
},
how to maintain different json response using same pojo class with different parameters for different api methods.
please help me to solve this isuue.
Thanks.
REST API under/over-fetching is a well-known problem. There's only two (classical ways) to handle that.
The first one is to build one model per each attribute visibility state. So, in your case, you'll need to create two different models (this kind of models are called DTO - Data Transfert Object). One model will have a propertyType attribute, the other will not. The model Property you've shared shows that you use the same class as entity and as transfert object. This solution will add some complexity to your app because you will have to implement some mappers to convert your entity to a corresponding DTO.
The second one is to accept that you send an attribute that will not be useful (be aware of the over-fetching). This solution is often the most adopted one. The cons of this solution is when you don't want to send something to your client (imagine a User model, you want to get the password from your client but you don't want to sent it back to it). Another obvious negative point is that the transactions will be larger but it is negligible in most cases
I would strongly advice you to keep your #Entity isolated in the 'db' layer. So that changes on the database side don't affect your API and vice versa. Also, you will have much better control over what data is exposed in your API. For your needs you can create 2 true DTOs, like PropertyDto and PropertyDetailsDto (or using private fields and getters/setters).
public class PropertyDto {
public String propertyId;
public String propertyName;
}
public class PropertyDetailsDto extends PropertyDto {
public String propertyType;
}
Map your #Entity to a specific dto corresponding to your needs.
EDIT
public List<PropertyDto> getProperties() {
return toPropertyDtos(repository.findAll());
}
public PropertyDetailsDto getPropertyById(Long id) {
return toPropertyDetailsDto(repository.findBy(id));
}
in some Mapper.java
...
public static List<PropertyDto> toPropertyDtos(List<Property> properties) {
return properties.stream()
.map(Mapper::toPropertyDto)
.collect(toList());
}
private static PropertyDto toPropertyDto(Property property) {
PropertyDto dto = new PropertyDto();
dto.propertyId = property.propertyId;
dto.propertyName = property.propertyName;
return dto;
}
// same stuff for `toPropertyDetailsDto`, you could extract common mapping parts in a separate private method inside `Mapper`
...

Dynamic filtering of a field in the response from a RESTful webservice enpoint that returns a List of domain objects

Given a RESTful web service developed using the Spring Boot framework, I wanted a way to suppress the birthDate of all Users in the response. This is what I implemented after looking around for a solution :
#RestController
public class UserResource {
#Autowired
private UserDAOservice userDAOService;
#GetMapping("/users")
public MappingJacksonValue users() {
List<User> users = userDAOService.findAll();
SimpleBeanPropertyFilter filter = SimpleBeanPropertyFilter
.filterOutAllExcept("id", "name");
FilterProvider filters = new SimpleFilterProvider().addFilter(
"UserBirthDateFilter", filter);
MappingJacksonValue mapping = new MappingJacksonValue(users);
mapping.setFilters(filters);
return mapping;
}
}
However, when I hit the rest end point in the browser, I can still see the birth date of the user in the response :
{
"id": 1,
"name": "Adam",
"birthDate": "1980-03-31T16:56:28.926+0000"
}
Question 1 : What API can I use to achieve my objective?
Next, assuming that I want to adhere to HATEOAS in combination with filtering, how can I go about doing this. I am unable to figure out the APIs that can be used for using these two features together :
#GetMapping("/users/{id}")
public EntityModel<User> users(#PathVariable Integer id) {
User user = userDAOService.findById(id);
if (user == null) {
throw new ResourceNotFoundException("id-" + id);
}
EntityModel<User> model = new EntityModel<>(user);
WebMvcLinkBuilder linkTo = linkTo(methodOn(this.getClass()).users());
model.add(linkTo.withRel("all-users"));
//how do I combine EntityModel with filtering?
return model;
}
Question 2 : How do I combine EntityModel with MappingJacksonValue?
Note : I am aware of #JsonIgnore annotation but that would apply the filter for all end points that use the domain; however, I want to restrict the filtering only to the two endpoints above.
Turns out for this to work, I have to add the #JsonFilter annotation above the DTO and provide the same name that was used while creating the SimpleFilterProvider.
#JsonFilter("UserBirthDateFilter")
public class User {
private Integer id;
#Size(min=2, message="user name must be atleast 2 characters")
#ApiModelProperty(notes="user name must be atleast 2 characters")
private String name;
#Past
#ApiModelProperty(notes="birth date cannot be in the past")
private Date birthDate;
//other methods
}
There is an easier way to do this, on your transfer object (the class you are sending back to the client), you can simply use the #JsonIgnore annotation to make sure the field is not serialized, and therefore sent to the client. So simply add #JsonIgnore inside your User class for your birthDay field.
You can also read more here about this approach:
https://www.baeldung.com/jackson-ignore-properties-on-serialization
If you need to return a different object for different endpoints (User without birthDay in your case, only for specific) you should create separate transfer objects and use those for their respective endpoints. You can pass your original entity (User) in the constructor to those classes and copy over all fields needed.
You can use Jackson's #JsonView feature. With this, you can tell a certain request mapping to produce serialized JSON with chosen set of properties.
public class View {
interface UserDetails {}
}
public class User {
#JsonView(View.UserDetails.class)
private Long id;
#JsonView(View.UserDetails.class)
private String name;
private String birthdate;
}
Controller be like
#JsonView(View.UserDetails.class)
#GetMapping("/users")
public MappingJacksonValue users() {
....
}
For question 2, I had the exact same question as you did, and here's what I did. It seems to be working:
#GetMapping(path = "/users/{id}")
public MappingJacksonValue retrieveUser(#PathVariable int id){
User user = service.findOne(id);
if(user==null){
throw new UserNotFoundException("id-"+id);
}
//"all-users", SERVER_PATH + "/users"
EntityModel<User> resource = EntityModel.of(user);
WebMvcLinkBuilder linkTo =
linkTo(methodOn(this.getClass()).retrieveAllUsers());
resource.add(linkTo.withRel("all-users"));
SimpleBeanPropertyFilter filter = SimpleBeanPropertyFilter.filterOutAllExcept("id");
FilterProvider filters = new SimpleFilterProvider().addFilter("UserFilter",filter);
MappingJacksonValue mapping = new MappingJacksonValue(resource);
mapping.setFilters(filters);
return mapping;
}
Response for HTTP GET localhost:8080/users/1
{
"id": 1,
"links": [
{
"rel": "all-users",
"href": "http://localhost:8080/users"
}
]}

How to Manually Generate Instance of Swagger ApiModel

If I have the following class:
#ApiModel
public class SomeEvent {
#ApiModelProperty(
value = "The unique identifier of this event.",
example = "MY_EVENT"
)
private final String eventType;
#JsonCreator
public SomeEvent(
#JsonProperty(value = "eventType", required = true) String eventType) {
this.eventType = eventType;
}
public String getEventType() {
return eventType;
}
}
I would like to use Swagger to generate an instance and serialize it to the following JSON:
{
"eventType": "MY_EVENT"
}
By using this class as a #RequestBody in my Spring Boot controller, the expected serialized form is displayed perfectly on the Swagger UI.
However, I'd like to be able to generate an instance (where the instance's fields contain the values from the example in the #ApiModelProperty) using code. I've searched around and could not find a single example of how to do this, assuming it's possible to achieve.
Note: I do not want to just create an instance using new SomeEvent("MY_EVENT");, but rather use Swagger to grab the examples.

Spring Boot RestTemplate Composite Response Empty List

So I'm developping some microservices in JAVA using Spring Boot and I'm facing some problems involving the objects I'm using.
So I have a data service which is the DB interface and a scheduling service which will be called by the frontend.
Both work with their own Response and Request objects eventhough at this point they are basically the same.
please ignore that there are no getters and setters in the code below.
Data-Service
#RestController
#RequestMapping("")
public class DataServiceResource {
#GetMapping(...)
public ResponseEntity<JobDetailsResponse> getJobDetailsSingleDate(#PathVariable("singledate") final String date) {
...
return response;
}
}
JobDetailsResponse
#JsonIgnoreProperties(ignoreUnknown = true)
public class JobDetailsResponse {
private Object requestSent;
private List<Job> jobsFound;
private boolean hasError;
private String errorMessage;
private LocalDateTime dataTimestamp;
}
JobDetailsSingleDateRequest
#JsonIgnoreProperties(ignoreUnknown = true)
public class JobDetailsSingleDateRequest {
private String dateFrom;
}
Scheduling Service
#RestController
#RequestMapping("")
public class SchedulingServiceResource {
...
#Autowired
private RestTemplate restTemplate;
#GetMapping(...)
public ResponseEntity<ReportDetailsResponse> getReportDetailsSingleDate(#PathVariable("singledate") final String singledate) {
ResponseEntity<ReportDetailsResponse> quoteResponse = this.restTemplate.exchange(DATA_SERVICE_JOB_DETAILS_SINGLE_DATE_URL + singledate, HttpMethod.GET,
null, new ParameterizedTypeReference<ReportDetailsResponse>() {});
...
return response;
}
ReportDetailsSingleDateRequest
#JsonIgnoreProperties(ignoreUnknown = true)
public class ReportDetailsSingleDateRequest {
private String dateFrom;
}
ReportDetailsResponse
#JsonIgnoreProperties(ignoreUnknown = true)
public class ReportDetailsResponse {
private Object requestSent;
private List<Job> jobsFound;
private boolean hasError;
private String errorMessage;
private LocalDateTime dataTimestamp;
}
So when I go through the quoteResponse.getBody().getJobsFound() method to check the data I got from the Data Service My List of jobs is empty.
I read that If the objects are equal in definition, spring would use reflection to pass the values, but in my case its not woking.
Is there a way to consume the microservice without having to add the data service dependency to the scheduling service?
Sorry for the long post but, until now I haven't found a proper example for my case. All the examples I found work with List as return of the microservice.
Thanks in advance.

Spring Boot - Spring Data - Jersey - No results of repositories during JUnit integration test

struggling long time with that issue and I have the weird feeling it has something to do on how I am setting my #Transactional annotation.
So what do I want to do?
I am preparing some data and save them with the available repositories in the database.
This can be found here in my FormTest class in the prepareExampleApplication method
#RunWith(SpringRunner.class)
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#ActiveProfiles("test")
#DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD)
public class FormTest {
#Autowired
private ApplicationRepository applicationRepository;
#Autowired
private AppActionRepository appActionRepository;
#Autowired
private RoleRepository roleRepository;
#Autowired
private TestRestTemplate restTemplate;
#Autowired
private FormRepository formRepository;
#Autowired
private FormElementRepository formElementRepository;
#Autowired
private SectionRepository sectionRepository;
private Application application;
private InputField refInputField;
private Select refSelectBox;
#Before
public void prepareExampleApplication() {
Form form = formRepository.save(ModelFactory.getForm("Project"));
Application application = ModelFactory.getApplication("Example", form);
this.application = applicationRepository.save(application);
Role role = new Role();
role.setRoleName("ADMIN");
role.setApp(application);
role = roleRepository.save(role);
Section section = ModelFactory.getSection(form, null, null);
section = formElementRepository.save(section);
InputField inputField = ModelFactory.getInputField(form, section, section);
refInputField = formElementRepository.save(inputField);
//once again. Just for my own eyes to see if it is there
Iterable<Form> all = formRepository.findAll();
// lot more stuff
}
#Test
#Transactional
public void testUserInput() {
// first create a new container to give inouts
Long id = this.application.getEntity().getId();
// for the sake of debugging I am using the formRepo to really SEARCH for the persisted form AND IT IS THERE!!!
Form byId = formRepository.findById(id).orElseThrow(NotFoundException::new);
URI uri = this.restTemplate.postForLocation("/api/form/" + id + "/forminstance", null);
long containerId = TestHelper.extractId(uri);
}
}
The next thing I am doing having this data is to use the restTemplate and dispatch a post request to a REST service. You can find the POST call in the test method. For debugging reasons - and to see that the repo is working - I am REALLY using the repository to get the id of the form instead of using the class field that had been filled with the preparation method. AND the form will be returned!!
Within my rest service I am using the formRepository once again, looking for the entity I have found before in the test class. But THIS TIME the repository does not return anything. Only null. I have tried SO MANY different things with setting #Transactional on different locations, but whatever I do the formRepository within the REST service does only give me back 0 entites and a null. Here is the REST service
#Service
#Api("FormService")
#Path("/form")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public class FormService {
#Autowired
private FormInstanceRepository formInstanceRepository;
#Autowired
private FormRepository formRepository;
#Autowired
private FormElementRepository formElementRepository;
#Autowired
private UserInputRepository userInputRepository;
#POST
#Path("{id}/forminstance")
#Transactional
public Response createFormInstance(#Context UriInfo info, #PathParam("id") long formId) {
// returns ALWAYS 0 elements
Iterable<Form> all = formRepository.findAll();
// returns always null
Form form = formRepository.findById(formId).orElse(null);
FormInstance formInstance = new FormInstance();
formInstance.setForm(form);
FormInstance save = formInstanceRepository.save(formInstance);
UriBuilder builder = info.getAbsolutePathBuilder();
builder.path(Long.toString(save.getId()));
return Response.created(builder.build()).build();
}
IF you know the answer I am really interested in the explanation to understand my error. I am using an in-memory H2 db for the tests.
Adding the Form entity and FormRepository, too
#Entity
#Data
public class Form {
#Id
#GeneratedValue
private Long id;
private String name;
}
-
public interface FormRepository extends CrudRepository<Form, Long> {
}
Thanks in advance for your help!!

Categories

Resources