Small question on Spring Boot, and how to use a design pattern combined with Spring #Value configuration in order to select the appropriate #Repository please.
Setup: A springboot project which does nothing but save a pojo. The "difficulty" is the need to choose where to save the pojo, based on some info from inside the payload request.
I started with a first straightforward version, which looks like this:
#RestController
public class ControllerVersionOne {
#Autowired private ElasticRepository elasticRepository;
#Autowired private MongoDbRepository mongoRepository;
#Autowired private RedisRepository redisRepository;
//imagine many more other repositories
//imagine many more other repositories
//imagine many more other repositories
#PostMapping(path = "/save")
public String save(#RequestBody MyRequest myRequest) {
String whereToSave = myRequest.getWhereToSave();
MyPojo myPojo = new MyPojo(UUID.randomUUID().toString(), myRequest.getValue());
if (whereToSave.equals("elastic")) {
return elasticRepository.save(myPojo).toString();
} else if (whereToSave.equals("mongo")) {
return mongoRepository.save(myPojo).toString();
} else if (whereToSave.equals("redis")) {
return redisRepository.save(myPojo).toString();
// imagine many more if
// imagine many more if
// imagine many more if
} else {
return "unknown destination";
}
}
With the appropriate #Configuration and #Repository for each and every databases. I am showing 3 here, but imagine many. The project has a way to inject future #Configuration and #Repository as well (the question is not here actually)
#Configuration
public class ElasticConfiguration extends ElasticsearchConfiguration {
#Repository
public interface ElasticRepository extends CrudRepository<MyPojo, String> {
#Configuration
public class MongoConfiguration extends AbstractMongoClientConfiguration {
#Repository
public interface MongoDbRepository extends MongoRepository<MyPojo, String> {
#Configuration
public class RedisConfiguration {
#Repository
public interface RedisRepository {
Please note, some of the repositories are not children of CrudRepository. There is no direct ___Repository which can cover everything.
And this first version is working fine. Very happy, meaning I am able to save the pojo to where it should be saved, as I am getting the correct repository bean, using this if else structure.
In my opinion, this structure is not very elegant (if it ok if we have different opinion here), especially, not flexible at all (need to hardcode each and every possible repository, again imagine many).
This is why I went to refactor and change to this second version:
#RestController
public class ControllerVersionTwo {
private ElasticRepository elasticRepository;
private MongoDbRepository mongoRepository;
private RedisRepository redisRepository;
private Map<String, Function<MyPojo, MyPojo>> designPattern;
#Autowired
public ControllerVersionTwo(ElasticRepository elasticRepository, MongoDbRepository mongoRepository, RedisRepository redisRepository) {
this.elasticRepository = elasticRepository;
this.mongoRepository = mongoRepository;
this.redisRepository = redisRepository;
// many more repositories
designPattern = new HashMap<>();
designPattern.put("elastic", myPojo -> elasticRepository.save(myPojo));
designPattern.put("mongo", myPojo -> mongoRepository.save(myPojo));
designPattern.put("redis", myPojo -> redisRepository.save(myPojo));
//many more put
}
#PostMapping(path = "/save")
public String save(#RequestBody MyRequest myRequest) {
String whereToSave = myRequest.getWhereToSave();
MyPojo myPojo = new MyPojo(UUID.randomUUID().toString(), myRequest.getValue());
return designPattern.get(whereToSave).apply(myPojo).toString();
}
As you can see, I am leveraging a design pattern refactoring the if-else into a hashmap.
This post is not about if-else vs hashmap by the way.
Working fine, but please note, the map is a Map<String, Function<MyPojo, MyPojo>>, as I cannot construct a map of Map<String, #Repository>.
With this second version, the if-else is being refactored, but again, we need to hardcode the hashmap.
This is why I am having the idea to build a third version, where I can configure the map itself, via a spring boot property #Value for Map:
Here is what I tried:
#RestController
public class ControllerVersionThree {
#Value("#{${configuration.design.pattern.map}}")
Map<String, String> configurationDesignPatternMap;
private Map<String, Function<MyPojo, MyPojo>> designPatternStrategy;
public ControllerVersionThree() {
convertConfigurationDesignPatternMapToDesignPatternStrategy(configurationDesignPatternMap, designPatternStrategy);
}
private void convertConfigurationDesignPatternMapToDesignPatternStrategy(Map<String, String> configurationDesignPatternMap, Map<String, Function<MyPojo, MyPojo>> designPatternStrategy) {
// convert configurationDesignPatternMap
// {elastic:ElasticRepository, mongo:MongoDbRepository , redis:RedisRepository , ...}
// to a map where I can directly get the appropriate repository based on the key
}
#PostMapping(path = "/save")
public String save(#RequestBody MyRequest myRequest) {
String whereToSave = myRequest.getWhereToSave();
MyPojo myPojo = new MyPojo(UUID.randomUUID().toString(), myRequest.getValue());
return designPatternStrategy.get(whereToSave).apply(myPojo).toString();
}
And I would configure in the property file:
configuration.design.pattern.map={elastic:ElasticRepository, mongo:MongoDbRepository , saveToRedis:RedisRepositry, redis:RedisRepository , ...}
And tomorrow, I would be able to configure add or remove the future repository target.
configuration.design.pattern.map={elastic:ElasticRepository, anotherElasticKeyForSameElasticRepository, redis:RedisRepository , postgre:PostGreRepository}
Unfortunately, I am stuck.
What is the correct code in order to leverage a configurable property for mapping a key with it's "which #Repository to use" please?
Thank you for your help.
You can create a base repository to be extended by all your repositories:
public interface BaseRepository {
MyPojo save(MyPojo onboarding);
}
so you will have a bunch of repositories like:
#Repository("repoA")
public interface ARepository extends JpaRepository<MyPojo, String>, BaseRepository {
}
#Repository("repoB")
public interface BRepository extends JpaRepository<MyPojo, String>, BaseRepository {
}
...
Those repositories will be provided by a factory:
public interface BaseRepositoryFactory {
BaseRepository getBaseRepository(String whereToSave);
}
that you must configure in a ServiceLocatorFactoryBean:
#Bean
public ServiceLocatorFactoryBean baseRepositoryBean() {
ServiceLocatorFactoryBean serviceLocatorFactoryBean = new ServiceLocatorFactoryBean();
serviceLocatorFactoryBean.setServiceLocatorInterface(BaseRepositoryFactory.class);
return serviceLocatorFactoryBean;
}
Now you can inject the factory wherever you need and get the repo want:
#Autowired
private BaseRepositoryFactory baseRepositoryFactory;
...
baseRepositoryFactory.getBaseRepository("repoA").save(myPojo);
...
Hope it helps.
Short answer:
create a shared interface
create multiple sub-class of this interface (one per storage) using different spring component names
Use a map to deal with aliases
use Spring context to retrieve the right bean by alias (instead of creating a custom factory)
Now adding a new storage is only adding a new Repository classes with a name
Explanation:
As mentioned in the other answer you first need to define a common interface as you can't use the CrudRepository.save(...).
In my example I reuse the same signature as the save method to avoid re-implementing it in the sub-classes of CrudRepository.
public interface MyInterface<T> {
<S extends T> S save(S entity);
}
Redis Repository:
#Repository("redis") // Here is the name of the redis repo
public class RedisRepository implements MyInterface<MyPojo> {
#Override
public <S extends MyPojo> S save(S entity) {
entity.setValue(entity.getValue() + " saved by redis");
return entity;
}
}
For the other CrudRepository no need to provide an implementation:
#Repository("elastic") // Here is the name of the elastic repo
public interface ElasticRepository extends CrudRepository<MyPojo, String>, MyInterface<MyPojo> {
}
Create a configuration for your aliases in application.yml
configuration:
design:
pattern:
map:
redis: redis
saveToRedisPlease: redis
elastic: elastic
Create a custom properties to retrieve the map:
#Component
#ConfigurationProperties(prefix = "configuration.design.pattern")
public class PatternProperties {
private Map<String, String> map;
public String getRepoName(String alias) {
return map.get(alias);
}
public Map<String, String> getMap() {
return map;
}
public void setMap(Map<String, String> map) {
this.map = map;
}
}
Now create the version three of your repository with the injection of SpringContext:
#RestController
public class ControllerVersionThree {
private final ApplicationContext context;
private PatternProperties designPatternMap;
public ControllerVersionThree(ApplicationContext context,
PatternProperties designPatternMap) {
this.context = context;
this.designPatternMap = designPatternMap;
}
#PostMapping(path = "/save")
public String save(#RequestBody MyRequest myRequest) {
String whereToSave = myRequest.getWhereToSave();
MyPojo myPojo = new MyPojo(UUID.randomUUID().toString(), myRequest.getValue());
String repoName = designPatternMap.getRepoName(whereToSave);
MyInterface<MyPojo> repo = context.getBean(repoName, MyInterface.class);
return repo.save(myPojo).toString();
}
}
You can check that this is working with a test:
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.client.TestRestTemplate;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.springframework.http.HttpEntity;
import static org.junit.jupiter.api.Assertions.assertEquals;
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class ControllerVersionThreeTest {
#LocalServerPort
private int port;
#Autowired
private TestRestTemplate restTemplate;
#Test
void testSaveByRedis() {
// Given: here 'redis' is the name of the spring beans
HttpEntity<MyRequest> request = new HttpEntity<>(new MyRequest("redis", "aValue"));
// When
String response = restTemplate.postForObject("http://localhost:" + port + "/save", request, String.class);
// Then
assertEquals("MyPojo{value='aValue saved by redis'}", response);
}
#Test
void testSaveByRedisAlias() {
// Given: here 'saveToRedisPlease' is an alias name of the spring beans
HttpEntity<MyRequest> request = new HttpEntity<>(new MyRequest("saveToRedisPlease", "aValue"));
// When
String response = restTemplate.postForObject("http://localhost:" + port + "/save", request, String.class);
// Then
assertEquals("MyPojo{value='aValue saved by redis'}", response);
}
}
Have you tried creating a configuration class to create your repository map
#Configuration
public class MyConfiguration {
#Bean
public Map repositoryMap() {
Map<String, ? extends Repository> repositoryMap = new HashMap<>();
repositoryMap.put('redis', new RedisRepository());
repositoryMap.put('mongo', new MongoRepository());
repositoryMap.put('elastic', new ElasticRepository());
return Collections.unmodifiableMap(repositoryMap);
}
}
Then you could have the following in your rest controller
#RestController
#Configuration
public class ControllerVersionFour {
#Autowired
private Map<String, ? extends Repository> repositoryMap;
#PostMapping(path = "/save/{dbname}")
public String save(#RequestBody MyRequest myRequest, #PathVariable("dbname") String dbname) {
MyPojo myPojo = new MyPojo(UUID.randomUUID().toString(), myRequest.getValue());
return repisitoryMap.get(dbname).save(myPojo);
}
It might be better to have the db as a path/query parameter instead of having it in the request body. That way you may or may not be able to just save the request body depending on your use case instead of creating another pojo.
This post may also be useful for autowiring a map
I have a batch job written using Spring Batch
I have a config file below:
#Configuration
public class ObjectMapperConfig {
#Bean
public ObjectMapper objectMapper(){
return new ObjectMapper();
}
}
I have Json Line aggregator as below:
public class JsonLineAggregator<T> implements LineAggregator<T> {
private final ObjectMapper mapper = new ObjectMapper();
#Override
public String aggregate(final T item) {
String result = null;
try {
result = mapper.writeValueAsString(item);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
return result;
}
}
I want to inject objectMapper and don't want to create it inside the JsonLineAggregator class. Any idea how can I implement it using DI ?
You can use #Autowired annotation to inject the dependency in spring like this:
#Autowired
private final ObjectMapper mapper;
And Spring will inject the ObjectMapper bean created by you in ObjectMapperConfig class
(Before start question, my English might not be enough to describe all clearly. Kindly let me know if you don't understand.)
I am trying to send an data object from A spring project(producer) to B spring project(consumer) by Kafka.
Problem is that data objects in A and B are having different classpath. So B project data class is not able to map A project's field.
But two objects have same fields. So i want to get object from A project as an argument on B project.
Error message
Listener failed; nested exception is
org.springframework.kafka.support.serializer.DeserializationException: failed to deserialize; nested exception is
org.springframework.messaging.converter.MessageConversionException: failed to resolve class name. Class not found [com.example.springboot.DTO.kafka.PostViewCountDTO]; nested exception is
java.lang.ClassNotFoundException: com.example.springboot.DTO.kafka.PostViewCountDTO
build.gradle
implementation 'org.apache.kafka:kafka-clients:2.8.0'
implementation 'org.apache.kafka:kafka_2.13:2.8.0'
implementation 'org.springframework.boot:spring-boot-starter-web:2.5.3'
Data class (using A and B projects)
public class PostViewCountDTO implements Serializable {
private static final long serialVersionUID = 1L;
#NotNull
private long postNo;
}
producer config
#Configuration
public class PostViewProducerConfig {
#Value("${spring.kafka.producer.bootstrap-servers}")
private String bootstrapServer;
#Bean
public Map<String,Object> postViewProducerConfigs() {
return JsonSerializer.getStringObjectMap(bootstrapServer);
}
#Bean
public ProducerFactory<String, PostViewCountDTO> postViewCountDTOProducerFactory() {
return new DefaultKafkaProducerFactory<>(postViewProducerConfigs());
}
#Bean
public KafkaTemplate<String, PostViewCountDTO> postViewDTOKafkaTemplate() {
return new KafkaTemplate<>(postViewCountDTOProducerFactory());
}
}
Common JsonSerializer class
public class JsonSerializer {
static Map<String, Object> getStringObjectMap(String bootstrapServer) {
Map<String, Object> props = new HashMap<>();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServer);
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, org.springframework.kafka.support.serializer.JsonSerializer.class);
return props;
}
}
consumer config
#Configuration
#RequiredArgsConstructor
public class PostViewConsumerConfig {
#Value("${spring.kafka.consumer.bootstrap-servers}")
private String bootstrapServer;
#Bean
public Map<String,Object> postViewConsumerConfigs() {
return JsonDeserializer.getStringObjectMap(bootstrapServer);
}
#Bean
public ConsumerFactory<String, PostViewCountDTO> postViewCountDTO_ConsumerFactory() {
return new DefaultKafkaConsumerFactory<>(postViewConsumerConfigs());
}
#Bean
public ConcurrentKafkaListenerContainerFactory<String, PostViewCountDTO> postViewCountListener() {
ConcurrentKafkaListenerContainerFactory<String, PostViewCountDTO> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(postViewCountDTO_ConsumerFactory());
return factory;
}
#Bean
public StringJsonMessageConverter jsonConverter() {
return new StringJsonMessageConverter();
}
}
produce
#Async
public void sendPostNo(PostViewCountDTO postViewCountDTO) {
postViewKafkaTemplate.send(topic_viewCount, null, postViewCountDTO);
}
consume
#KafkaListener(topics = topic_viewCount, groupId = groupId, containerFactory = "postViewCountListener")
public void consume(#Payload PostViewCountDTO postViewCountDTO) {
...
}
You need to add type mappings to the serializer and deserializer
https://docs.spring.io/spring-kafka/docs/current/reference/html/#serdes-mapping-types
On the producer side, map com.a.PostViewCountDTO to PostViewCountDTO.
On the consumer side, map com.b.PostViewCountDTO to PostViewCountDTO.
Generally: i'm reading serialized object (as JSONs) from Kafka Stream and trying to save it to Redis using Spring Data repository.
After a two calls (objects has not been saved to Redis) to repository.save() i get StackOverFlowError:
Exception in thread "processOffers-applicationId-1c24ef63-baae-47b9-beb7-5e6517736bc4-StreamThread-1" java.lang.StackOverflowError
at org.springframework.data.util.Lazy.get(Lazy.java:94)
at org.springframework.data.mapping.model.AnnotationBasedPersistentProperty.usePropertyAccess(AnnotationBasedPersistentProperty.java:277)
at org.springframework.data.mapping.model.BeanWrapper.getProperty(BeanWrapper.java:134)
at org.springframework.data.mapping.model.BeanWrapper.getProperty(BeanWrapper.java:115)
at org.springframework.data.redis.core.convert.MappingRedisConverter.lambda$writeInternal$2(MappingRedisConverter.java:601)
at org.springframework.data.mapping.model.BasicPersistentEntity.doWithProperties(BasicPersistentEntity.java:353)
at org.springframework.data.redis.core.convert.MappingRedisConverter.writeInternal(MappingRedisConverter.java:597)
at org.springframework.data.redis.core.convert.MappingRedisConverter.lambda$writeInternal$2(MappingRedisConverter.java:639)
Serialized POJO look like this:
#Data
#With
#NoArgsConstructor
#AllArgsConstructor
#RedisHash("students")
public class Student {
#Id
#JsonProperty("student_id")
private long id;
#JsonProperty("entities")
private Map<String, Object> entities = new HashMap<>();
}
Map entities contains 100+ Entries, with nested maps (objects).
Interesting part: if i make map empty everything works fine and data instantly saved to Redis.
Corresponding repository for POJO:
#Repository
public interface StudentRepository extends CrudRepository<Student, Long> {
}
Also, i've defined RedisCustomConversion for Long id field:
#Component
#ReadingConverter
public class BytesToLongConverter implements Converter<byte[], Long> {
#Override
public Long convert(final byte[] source) {
ByteBuffer buffer = ByteBuffer.allocate(Long.BYTES);
buffer.put(source);
buffer.flip();
return buffer.getLong();
}
}
#Component
#WritingConverter
public class LongToBytesConverter implements Converter<Long, byte[]> {
#Override
public byte[] convert(final Long source) {
ByteBuffer buffer = ByteBuffer.allocate(Long.BYTES);
buffer.putLong(source);
return buffer.array();
}
}
Redis configuration class looks like this:
#Configuration
#EnableRedisRepositories
public class RedisConfiguration {
#Bean
#Primary
public RedisProperties redisProperties() {
return new RedisProperties();
}
#Bean
public RedisConnectionFactory redisConnectionFactory() {
var config = new RedisStandaloneConfiguration();
var props = redisProperties();
config.setHostName(props.getHost());
config.setPort(props.getPort());
return new JedisConnectionFactory(config);
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
var template = new RedisTemplate<String, Object>();
template.setConnectionFactory(redisConnectionFactory());
template.setDefaultSerializer(new GenericJackson2JsonRedisSerializer());
return template;
}
#Bean
public RedisCustomConversions redisCustomConversions(LongToBytesConverter longToBytes,
BytesToLongConverter bytesToLong) {
return new RedisCustomConversions(Arrays.asList(longToBytes, bytesToLong));
}
}
UPD:
I've found this issue on Spring Data Redis Jira, but the resolution set as "Fixed", so it's seems strange to me.
I've defined custom WritingConverter and ReadingConverter for my inner map in POJO using GenericJackson2JsonRedisSerializer and everything worked out!
Code:
#Component
#WritingConverter
public class FieldsToBytesConverter implements Converter<Map<String, Object>, byte[]> {
private final RedisSerializer serializer;
public FieldsToBytesConverter() {
serializer = new GenericJackson2JsonRedisSerializer();
}
#Override
public byte[] convert(Map<String, Object> value) {
return serializer.serialize(value);
}
}
#Component
#ReadingConverter
public class BytesToFieldsConverter implements Converter<byte[], Map<String, Object>> {
private final GenericJackson2JsonRedisSerializer serializer;
public BytesToFieldsConverter() {
serializer = new GenericJackson2JsonRedisSerializer();
}
#Override
public Map<String, Object> convert(byte[] value) {
return (Map<String, Object>) serializer.deserialize(value);
}
}
I'm trying to add some extra validation logic on my REST beans using annotations. This is just an example, but the point is that the annotation is to be used on multiple REST resource objects / DTO's.
I was hoping for a solution like this:
public class Entity {
#NotNull // JSR-303
private String name;
#Phone // Custom phonenumber that has to exist in a database
private String phoneNumber;
}
#Component
public class PhoneNumberValidator implements Validator { // Spring Validator
#Autowired
private PhoneRepository repository;
public boolean supports(Class<?> clazz) {
return true;
}
public void validate(Object target, Errors errors) {
Phone annotation = // find fields with annotations by iterating over target.getClass().getFields().getAnnotation
Object fieldValue = // how do i do this? I can easily get the annotation, but now I wish to do a call to repository checking if the field value exists.
}
}
Did you try JSR 303 bean validator implementations like hibernate validator
e.g. is available here http://www.codejava.net/frameworks/spring/spring-mvc-form-validation-example-with-bean-validation-api
Maven Module A:
public interface RestValidator<A extends Annotation, T> extends ConstraintValidator<A, T>
public interface PhoneValidator extends RestValidator<PhoneNumber, String>
#Target(FIELD)
#Retention(RUNTIME)
#Constraint(validatedBy = PhoneValidator.class) // This usually doesnt work since its a interface
public #interface PhoneNumber {
// JSR-303 required fields (payload, message, group)
}
public class Person {
#PhoneNumber
private String phoneNumber;
}
Maven Module B:
#Bean
LocalValidatorFactoryBean configurationPropertiesValidator(ApplicationContext context, AutowireCapableBeanFactory factory) {
LocalValidatorFactoryBean factoryBean = new LocalValidatorFactoryBean();
factoryBean.setConstraintValidatorFactory(factory(context, factory));
return factoryBean;
}
private ConstraintValidatorFactory factory(final ApplicationContext context, final AutowireCapableBeanFactory factory) {
return new ConstraintValidatorFactory() {
#Override
public <T extends ConstraintValidator<?, ?>> T getInstance(Class<T> key) {
if (RestValidator.class.isAssignableFrom(key)) {
return context.getBean(key);
} else {
return factory.createBean(key);
}
}
#Override
public void releaseInstance(ConstraintValidator<?, ?> instance) {
if (!(instance instanceof RestValidator<?, ?>)) {
factory.destroyBean(instance);
}
}
};
}
#Bean
WebMvcConfigurerAdapter webMvcConfigurerAdapter(final LocalValidatorFactoryBean validatorFactoryBean) {
return new WebMvcConfigurerAdapter() { // Adds the validator to MVC
#Override
public Validator getValidator() {
return validatorFactoryBean;
}
};
}
Then I have a #Component implementation of PhoneValidator that has a Scope = Prototype.
I hate this solution, and I think Spring SHOULD look up on Interface implementations by default, but I'm sure some people that are a lot smarter than me made the decision not to.