Develop Reference

Spring Data Elasticsearch: Convert a String to an Object (and vice versa) using ValueConverter and dot-notation

I have kind of a combination-follow up question to [1] and [2].
I have a POJO with a field I want to persist in - and read from - Elasticsearch:
#Document
public class MyPojo {
private String level3;
// getters/setters...
}
For convenience and because the property is also being persisted (flatly) into postgres, the property level3 should be a String, however it should be written into ES as a nested object (because the ES index is defined elsewhere).
The current solution is unsatisfactory:
#Document
#Entity
public class MyPojo {
#Column(name = "level3)
#Field(name = "level3", type = FieldType.Keyword)
#ValueConverter(MyConverter.class)
private String level3;
// getters/setters...
}
with the object path "level1.level2.level3" hardcoded within MyConverter, which converts from Map<String, Object> to String (read) and from String to Map<String, Object> (write). Because we potentially need to do this on multiple fields, this is not a really viable solution.
I'd rather do something like this:
#Document
#Entity
public class MyPojo {
#Column(name = "level3)
#Field(name = "level1.level2.level3", type = FieldType.Keyword)
#ValueConverter(MyConverter2.class)
private String level3;
// getters/setters...
}
which does not work (writing works fine, while reading we get the "null is not a map" error from [2]).
Is this at all possible (if I understood [2] correctly, no)? If not, is there another way to achieve what I want without hardcoding and an extra converter per field?
Can I somehow access the #Field annotation within MyConverter (e.g. the name), or can I somehow supply additional arguments to MyConverter?
[1] Spring data elasticsearch embedded field mapping
[2] Spring Elasticsearch: Adding fields to working POJO class causes IllegalArgumentException: null is not a Map

No error when violating constraints after inserting data in spring-boot application with spring data jpa

I'm currently learning Spring-Boot and Spring-Data-JPA.
I'm using a postgresql database for storing the data.
My goal is to store ingredients with a unique and custom ID (you just type it in when creating it), but when another ingredient with the same ID gets inserted, there should be some kind of error. In my understanding, this is what happens when I use the #Id annotation, hibernate also logs the correct create table statement.
This is my Ingredient class:
public class Ingredient {
#Id
#Column(name = "ingredient_id")
private String ingredient_id;
#Column(name = "name")
private String name;
#Column(name = "curr_stock")
private double curr_stock;
#Column(name = "opt_stock")
private double opt_stock;
#Column(name = "unit")
private String unit;
#Column(name = "price_per_unit")
private double price_per_unit;
#Column(name = "supplier")
private String supplier;
-- ... getters, setters, constructors (they work fine, I can insert and get the data)
}
My controller looks like this:
#RestController
#RequestMapping(path = "api/v1/ingredient")
public class IngredientController {
private final IngredientService ingredientService;
#Autowired
public IngredientController(IngredientService ingredientService) {
this.ingredientService = ingredientService;
}
#GetMapping
public List<Ingredient> getIngredients(){
return ingredientService.getIngredients();
}
#PostMapping
public void registerNewStudent(#RequestBody Ingredient ingredient) {
ingredientService.saveIngredient(ingredient);
}
}
And my service class just uses the save() method from the JpaRepository to store new ingredients.
To this point I had the feeling, that I understood the whole thing, but when sending two post-requests to my application, each one containing an ingredient with the id "1234", and then showing all ingredients with a get request, the first ingredient just got replaced by the second one and there was no error or smth. like that in between.
Sending direct sql insert statements to the database with the same values throws an error, because the primary key constraint gets violated, just as it should be. Exactly this should have happened after the second post request (in my understanding).
What did I get wrong?
Update:
From the terminal output and the answers I got below, it is now clear, that the save() method can be understood as "insert or update if primary key is already existing".
But is there a better way around this than just error-handle every time when saving a new entry by hand?

The save method will create or update the entry if the id already exists. I'd switch to auto generating the ID when inserting, instead of manually creating the IDs. That would prevent the issue you have

When saving a new ingredient, jpa will perform an update if the value contained in the “id” field is already in the table.
A nice way through which you can achieve what you want is
ingredientRepository.findById(ingredientDTO.getIngredientId()).
ifPresentOrElse( ingredientEntity-> ResponseEntity.badRequest().build(), () -> ingredientRepository.save(ingredientDTO));
You can return an error if the entity is already in the table otherwise (empty lambda), you can save the new row

This is a downside to using CrudRepository save() on an entity where the id is set by the application.
Under the hood EntityManager.persist() will only be called if the id is null otherwise EntityManager.merge() is called.
Using the EntityManager directly gives you more fine grained control and you can call the persist method in your application when required

Can I directly map the data from an external source on it to persist on DB?

I am pretty new in Spring Data JPA and ORM in general. I have the following architectural doubt.
Lets consider this entity class:
#Entity // This tells Hibernate to make a table out of this class
public class Order {
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
private int id;
#Column(name = "name")
private String fullName;
private String address;
private String product;
#Column(name = "order_date_time")
private String orderDate;
private Double quantity;
// getters, setters
}
This class is mapped on my order database table.
In my application data came from an Excel document that I parse via Apace POI and than I have to persist on the database.
My doubt is: can I directly use this entity class to map an Excel row using Apache POI in order to persist the Excel rows as order table records? Or is better to use another DTO class to read the rows from Excel and than use this DTO to set the field values of my entity class?
An entity class can contain a constructor using fields?

Can I directly use this entity class to map an Excel row using Apache
POI in order to persist the Excel rows as order table records?
Yes you can.
Or is better to use another DTO class to read the rows from Excel and
than use this DTO to set the field values of my entity class?
It's certainly common to have a DTO layer between the two, but it's not required so it's up to you.
An entity class can contain a constructor using fields?
Yes, but at least Hibernate wants a non-private default constructor as well, so remember to create a protected Order() {} (or any visibility modifier besides private) in addition to your parameterized constructor.

I'm not a heavy user of Apache POI, but I do know it's used to manipulate MS files.
So, here are my two cents, in your use case, you can just read it and map directly to the Entity class as it doesn't expose an API to the external world.
However, if you were building a REST/SOAP API, I recommend you put a DTO in between so you don't mistakenly expose things that shouldn't be exposed.

From architectural point of view better to have a DTO class and encapsulate some logic there.
class CsvOrder {
private String fullName;
private String address;
public CsvRecord(String[] record) {
fullName = get(record, FULLNAME_INDEX);
address = get(record, ADDRESS_INDEX);
}
public Order toOrder() {
Order result = new Order();
result.setFullName(fullName);
return result;
}
}
public static <T> T get(T[] arr, int index) {
final T notFound = null;
return index < size(arr) ? arr[index] : notFound;
}
public static <T> int size(T[] array) {
return array == null ? 0 : array.length;
}
You can put a static method toOrder() to OrderServiceMapper, if you want to totally decouple layers
class OrderServiceMapper {
public static Order toOrder(CsvOrder order) {
Order result = new Order();
result.setFullName(order.getFullName());
return result;
}
}
Also, use Integer in place of int for id. Better to use Long everywhere
// This tells Spring to add this class to Hibernate configuration during auto scan
#Entity
public class Order {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Integer id;
}

Spring boot elastic search repository

I have several indices where I save products:
product-example1
product-example2
product-example3
product-example4
product-example5
I have the a document in elastic search that has the same structure and it can used for different indices:
#Data
#org.springframework.data.elasticsearch.annotations.Document(indexName = "", type = "", createIndex = false)
public class ProductDocument {
#Id
private String id;
private String title;
private String seller;
private String releaseDate;
....
}
So basically I want to use same settings, same mappings same search service.
So I made indexName and type parametric in spring boot java, instead of creating 5 classes extending ProductDocument ?
#Autowired
private final ElasticsearchTemplate elasticsearchTemplate;
this.elasticsearchTemplate.createIndex("product-example1", loadFile("/files/settings.json"));
this.elasticsearchTemplate.putMapping("product-example1", "product-type1", loadFile("/files/mapping.json"));
this.elasticsearchTemplate.createIndex("product-example2", loadFile("/files/settings.json"));
this.elasticsearchTemplate.putMapping("product-example2", "product-type2", loadFile("/files/mapping.json"));
......
Now I want to create a ProductRepository but I don't have a class with defined index name. If I use generic class:
public interface DocumentRepository extends ElasticsearchRepository<ProductDocument, String> {
}
I get the error which is totally understable cause I created the index names in dynamic way:
lang.IllegalArgumentException: Unknown indexName. Make sure the indexName is defined. e.g #ProductDocument(indexName="foo")
So is it possible somehow to create repository for indexes that I created in dynamic way as described above, and pass the index name and type as parameter ?
Please help! I'm stuck.

Ignore JsonIgnore in Elasticsearch

I am developing an application which uses Spring-boot, a relational database and Elasticsearch.
I use JSON serialization at 2 differents places in the code:
In the response of the REST API.
When the code interacts with Elasticsearch.
There are some properties that I need in Elasticsearch but that I want to hide to the application user (e.g. internal ids coming from the relational database).
Here is an example of entity :
#Document
public class MyElasticsearchEntity {
#Id
private Long id; //I want to hide this to the user.
private String name;
private String description;
}
Problem : When the object it persisted in Elasticsearch, it gets serialized as JSON. Hence, fields with #JsonIgnore are ignored when serialized to Elasticsearch.
Up to now, I found 2 unsatisfying solutions :
Solution 1 : Use #JsonProperty like this :
#Id
#JsonProperty(access = JsonProperty.Access.READ_ONLY)
private Long id;
The id gets written in Elasticsearch and is nullified in the JSON response :
{
"id" : null,
"name" : "abc",
"description" : null
}
So it works but the application user still sees that this property exists. This is messy.
Solution 2 : Cutomize the object mapper to ignore null values
Spring-boot has a built-in option for that :
spring.jackson.serialization-inclusion=NON_NULL
Problem : it suppresses all non-null properties, not only those that I want to ignore. Suppose that the field description of the previous entity is empty, the JSON response will be :
{
"name" : "abc"
}
And this is problematic for the UI.
So is there a way to ignore such field only in the JSON response?

You could use Jackson JsonView for your purpose. You can define one view which will be used to serialize pojo for the application user :
Create the views as class, one public and one private:
class Views {
static class Public { }
static class Private extends Public { }
}
Then uses the view in your Pojo as an annotation:
#Id
#JsonView(Views.Private.class) String name;
private Long id;
#JsonView(Views.Public.class) String name;
private String publicField;
and then serialize your pojo for the application user using the view:
objectMapper.writeValueUsingView(out, beanInstance, Views.Public.class);
This is one example of many others on how view can fit your question. Eg. you can use too objectMapper.configure(SerializationConfig.Feature.DEFAULT_VIEW_INCLUSION, false); to exclude field without view annotation and remove the Private view.