I'm experiencing really strange behavior from Spring. I have a #Bean which returns a Map. However when the Bean is #Autowired in, the key for the map is different than what was assigned in the #Bean method. my #Bean has two input parameters which are also Spring Beans from another configuration class. Once #Autowired The Keys of my map are changed to match the name of the #Bean methods passed in as dependencies im my Map returning Bean. The #Beanin question is located in an #ConfigurationProperties class where I am extracting some values from my application.yml file which are all returning correctly.
#Component
#ConfigurationProperties(prefix = "channel-broker")
#EnableConfigurationProperties
public class ChannelLookupConfig {
private String messageDeliveryChannelKey;
private String otherDeliveryChannelKey;
public String getMessageDeliveryChannelKey() {
return messageDeliveryChannelKey;
}
public void setMessageDeliveryChannelKey(String messageDeliveryChannelKey) {
this.messageDeliveryChannelKey = messageDeliveryChannelKey;
}
public String getOtherDeliveryChannelKey() {
return otherDeliveryChannelKey;
}
public void setOtherDeliveryChannelKey(String OtherDeliveryChannelKey) {
this.otherDeliveryChannelKey = OtherDeliveryChannelKey;
}
#Bean
public Map<String, MessageDeliveryClient> channelCallerLookup(MessageDeliveryClient MessageDispatcherClient, MessageDeliveryClient otherDeliveryClient) {
Map<String, MessageDeliveryClient> channelCallerLookup = new HashMap<>();
channelCallerLookup.put(messageDeliveryChannelKey, MessageDispatcherClient);
channelCallerLookup.put(otherDeliveryChannelKey, otherDeliveryClient);
return channelCallerLookup;
}
}
My second config file
#Configuration
public class Config {
#Bean
public MessageDeliveryClient MessageDispatcherClient() {
MessageDeliveryClient client = MessageDeliveryClient.builder()
.awsAccessKey(destinationSqsAccessKey)
.awsSecretKey(destinationSqsSecretKey)
.awsRegion(destinationSqsRegion)
.destinationQueueName(destinationSqsName)
.build();
return client;
}
#Bean
public MessageDeliveryClient otherPickerDeliveryClient() {
MessageDeliveryClient client = MessageDeliveryClient.builder()
.awsAccessKey(destinationSqsAccessKey)
.awsSecretKey(destinationSqsSecretKey)
.awsRegion(destinationSqsRegion)
.destinationQueueName(destinationOtherPickerSqsName)
.build();
return client;
}
}
Autowired in for use as such:
public class SimpleCustomerMessageDeliveryBrokerImpl implements CustomerMessageDeliveryBroker {
private Map<String, MessageDeliveryClient> channelCallerLookup = new HashMap<>();
#Autowired
public void setBrokerConfiguration(BrokerConfiguration brokerConfiguration) {
this.brokerConfiguration = brokerConfiguration;
}
}
the Map should contain 2 elements the first with a key equal to the value in String messageDeliveryChannelKey and the second with a key equal to the value in String otherDeliveryChannelKey. However the keys are always set equal to the name of the #Beans methods which are passed into my score. Even if I change the method names to nonsense the map's keys will equal that value.
How can I prevent this behavior from happening
This was occurring because of default Spring behavior. To work around this I applied a Wrapper around the return Map.
Changed my Bean to this
#Bean
public ChannelCallerLookup channelCallerLookup(MessageDeliveryClient messageDispatcherClient, MessageDeliveryClient otherPickerDeliveryClient) {
HashMap<String, MessageDeliveryClient> channelCallerLookup = new HashMap<>();
channelCallerLookup.put(CHANNEL1_KEY, messageDispatcherClient);
channelCallerLookup.put(CHANNEL1_KEY2, otherPickerDeliveryClient);
ChannelCallerLookup callerLookup = new ChannelCallerLookup(channelCallerLookup);
return callerLookup;
}
Created This Wrapper Class
public class ChannelCallerLookup {
Map<String, MessageDeliveryClient> lookupMap;
public ChannelCallerLookup(Map<String, MessageDeliveryClient> lookupMap) {
this.lookupMap = lookupMap;
}
public Map<String, MessageDeliveryClient> getLookupMap() {
return lookupMap;
}
public MessageDeliveryClient get(String key){
return lookupMap.get(key);
}
}
Related
(Before start question, my English might not be enough to describe all clearly. Kindly let me know if you don't understand.)
I am trying to send an data object from A spring project(producer) to B spring project(consumer) by Kafka.
Problem is that data objects in A and B are having different classpath. So B project data class is not able to map A project's field.
But two objects have same fields. So i want to get object from A project as an argument on B project.
Error message
Listener failed; nested exception is
org.springframework.kafka.support.serializer.DeserializationException: failed to deserialize; nested exception is
org.springframework.messaging.converter.MessageConversionException: failed to resolve class name. Class not found [com.example.springboot.DTO.kafka.PostViewCountDTO]; nested exception is
java.lang.ClassNotFoundException: com.example.springboot.DTO.kafka.PostViewCountDTO
build.gradle
implementation 'org.apache.kafka:kafka-clients:2.8.0'
implementation 'org.apache.kafka:kafka_2.13:2.8.0'
implementation 'org.springframework.boot:spring-boot-starter-web:2.5.3'
Data class (using A and B projects)
public class PostViewCountDTO implements Serializable {
private static final long serialVersionUID = 1L;
#NotNull
private long postNo;
}
producer config
#Configuration
public class PostViewProducerConfig {
#Value("${spring.kafka.producer.bootstrap-servers}")
private String bootstrapServer;
#Bean
public Map<String,Object> postViewProducerConfigs() {
return JsonSerializer.getStringObjectMap(bootstrapServer);
}
#Bean
public ProducerFactory<String, PostViewCountDTO> postViewCountDTOProducerFactory() {
return new DefaultKafkaProducerFactory<>(postViewProducerConfigs());
}
#Bean
public KafkaTemplate<String, PostViewCountDTO> postViewDTOKafkaTemplate() {
return new KafkaTemplate<>(postViewCountDTOProducerFactory());
}
}
Common JsonSerializer class
public class JsonSerializer {
static Map<String, Object> getStringObjectMap(String bootstrapServer) {
Map<String, Object> props = new HashMap<>();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServer);
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, org.springframework.kafka.support.serializer.JsonSerializer.class);
return props;
}
}
consumer config
#Configuration
#RequiredArgsConstructor
public class PostViewConsumerConfig {
#Value("${spring.kafka.consumer.bootstrap-servers}")
private String bootstrapServer;
#Bean
public Map<String,Object> postViewConsumerConfigs() {
return JsonDeserializer.getStringObjectMap(bootstrapServer);
}
#Bean
public ConsumerFactory<String, PostViewCountDTO> postViewCountDTO_ConsumerFactory() {
return new DefaultKafkaConsumerFactory<>(postViewConsumerConfigs());
}
#Bean
public ConcurrentKafkaListenerContainerFactory<String, PostViewCountDTO> postViewCountListener() {
ConcurrentKafkaListenerContainerFactory<String, PostViewCountDTO> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(postViewCountDTO_ConsumerFactory());
return factory;
}
#Bean
public StringJsonMessageConverter jsonConverter() {
return new StringJsonMessageConverter();
}
}
produce
#Async
public void sendPostNo(PostViewCountDTO postViewCountDTO) {
postViewKafkaTemplate.send(topic_viewCount, null, postViewCountDTO);
}
consume
#KafkaListener(topics = topic_viewCount, groupId = groupId, containerFactory = "postViewCountListener")
public void consume(#Payload PostViewCountDTO postViewCountDTO) {
...
}
You need to add type mappings to the serializer and deserializer
https://docs.spring.io/spring-kafka/docs/current/reference/html/#serdes-mapping-types
On the producer side, map com.a.PostViewCountDTO to PostViewCountDTO.
On the consumer side, map com.b.PostViewCountDTO to PostViewCountDTO.
I am trying to use HashMap by putting some key-value pair (very very basic). But I am facing a weird issue. The key that I am putting there is not visible when I iterate over it rather it creates the key with the class name. Here is the code snippet:
#Configuration
public class AppConfig {
#Bean
public Map<String, Command> allCommands(Command command1, Command command2) {
Map<String, Command> map = new HashMap<>();
map.put("first", command1);
map.put("second", command2);
return map;
}
#Bean
public Map<String, RouterStrategy> allStrategies(KafkaRouter kafkaRouter) {
Map<String, RouterStrategy> map = new HashMap<>();
map.put("kafka", kafkaRouter);
return map;
}
}
The class which is using this bean:
#Service
public class Postman {
private Map<String, RouterStrategy> allStrategies;
#Autowired
public Postman(Map<String, RouterStrategy> allStrategies) {
this.allStrategies = allStrategies;
}
public void process(String strategy,Envelope envelope) throws Exception{
allStrategies.forEach((k,v) -> logger.info(k + " -> " + v));
RouterStrategy routerStrategy = allStrategies.get(strategy);
routerStrategy.routeMessage(envelope);
}
}
The method which calls this process() method is as below:
public class SomeCommand implements Command {
Postman postman;
public SomeCommand(Postman postman) {
this.postman = postman;
}
#Override
public void execute(Envelope envelope) throws Exception {
postman.process("kafka",envelope);
}
}
Now, the problem here is if I run the code, process() method of the Postman class should print the strategy map with "kafka" as one of the keys. But it prints below output-
applog.msg=kafkaRouter -> KafkaRouter{kafkaTemplate=org.springframework.kafka.core.KafkaTemplate#79708}
Pretty weird, whatever the key I am setting is simply ignored rather it creates this Map by using characters of the class name (KafkaRouter -> kafkaRouter). This thing is same if I add an additional key-value pair in the strategy map. For example, if I modify the strategy map like mentioned below-
#Bean
public Map<String, RouterStrategy> allStrategies(KafkaRouter kafkaRouter, RestRouter restRouter) {
Map<String, RouterStrategy> map = new HashMap<>();
map.put("kafka", kafkaRouter);
map.put("rest", restRouter);
return map;
}
Strategy interface-
public interface RouterStrategy {
void routeMessage(Envelope envelope) throws Exception;
}
Strategy implementation-
#Service
public class KafkaRouter implements RouterStrategy{
private KafkaTemplate<String, String> kafkaTemplate;
#Autowired
public KafkaRouter(KafkaTemplate<String, String> kafkaTemplate) {
this.kafkaTemplate = kafkaTemplate;
}
#Override
public void routeMessage(Envelope envelope) throws Exception {
// Logic to get additional details like Kafka topic name, etc.
sendMessage(envelope.getContent());
}
#Override
public String toString() {
return "KafkaRouter{" +
"kafkaTemplate=" + kafkaTemplate + '}';
}
// Kafka publisher logic
private void sendMessage(String content) throws Exception {
}
}
In this case, two keys are printed kafkaRouter and restRouter. Since it can't "find the key", it throws NullPointerException. Anything I am missing? TIA
Other details:
Springboot version - 2.3.10.RELEASE
JDK - OpenJDK Runtime Environment Zulu11.40+15-NV (build 11.0.7.0.101+5-LTS)
Generally: i'm reading serialized object (as JSONs) from Kafka Stream and trying to save it to Redis using Spring Data repository.
After a two calls (objects has not been saved to Redis) to repository.save() i get StackOverFlowError:
Exception in thread "processOffers-applicationId-1c24ef63-baae-47b9-beb7-5e6517736bc4-StreamThread-1" java.lang.StackOverflowError
at org.springframework.data.util.Lazy.get(Lazy.java:94)
at org.springframework.data.mapping.model.AnnotationBasedPersistentProperty.usePropertyAccess(AnnotationBasedPersistentProperty.java:277)
at org.springframework.data.mapping.model.BeanWrapper.getProperty(BeanWrapper.java:134)
at org.springframework.data.mapping.model.BeanWrapper.getProperty(BeanWrapper.java:115)
at org.springframework.data.redis.core.convert.MappingRedisConverter.lambda$writeInternal$2(MappingRedisConverter.java:601)
at org.springframework.data.mapping.model.BasicPersistentEntity.doWithProperties(BasicPersistentEntity.java:353)
at org.springframework.data.redis.core.convert.MappingRedisConverter.writeInternal(MappingRedisConverter.java:597)
at org.springframework.data.redis.core.convert.MappingRedisConverter.lambda$writeInternal$2(MappingRedisConverter.java:639)
Serialized POJO look like this:
#Data
#With
#NoArgsConstructor
#AllArgsConstructor
#RedisHash("students")
public class Student {
#Id
#JsonProperty("student_id")
private long id;
#JsonProperty("entities")
private Map<String, Object> entities = new HashMap<>();
}
Map entities contains 100+ Entries, with nested maps (objects).
Interesting part: if i make map empty everything works fine and data instantly saved to Redis.
Corresponding repository for POJO:
#Repository
public interface StudentRepository extends CrudRepository<Student, Long> {
}
Also, i've defined RedisCustomConversion for Long id field:
#Component
#ReadingConverter
public class BytesToLongConverter implements Converter<byte[], Long> {
#Override
public Long convert(final byte[] source) {
ByteBuffer buffer = ByteBuffer.allocate(Long.BYTES);
buffer.put(source);
buffer.flip();
return buffer.getLong();
}
}
#Component
#WritingConverter
public class LongToBytesConverter implements Converter<Long, byte[]> {
#Override
public byte[] convert(final Long source) {
ByteBuffer buffer = ByteBuffer.allocate(Long.BYTES);
buffer.putLong(source);
return buffer.array();
}
}
Redis configuration class looks like this:
#Configuration
#EnableRedisRepositories
public class RedisConfiguration {
#Bean
#Primary
public RedisProperties redisProperties() {
return new RedisProperties();
}
#Bean
public RedisConnectionFactory redisConnectionFactory() {
var config = new RedisStandaloneConfiguration();
var props = redisProperties();
config.setHostName(props.getHost());
config.setPort(props.getPort());
return new JedisConnectionFactory(config);
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
var template = new RedisTemplate<String, Object>();
template.setConnectionFactory(redisConnectionFactory());
template.setDefaultSerializer(new GenericJackson2JsonRedisSerializer());
return template;
}
#Bean
public RedisCustomConversions redisCustomConversions(LongToBytesConverter longToBytes,
BytesToLongConverter bytesToLong) {
return new RedisCustomConversions(Arrays.asList(longToBytes, bytesToLong));
}
}
UPD:
I've found this issue on Spring Data Redis Jira, but the resolution set as "Fixed", so it's seems strange to me.
I've defined custom WritingConverter and ReadingConverter for my inner map in POJO using GenericJackson2JsonRedisSerializer and everything worked out!
Code:
#Component
#WritingConverter
public class FieldsToBytesConverter implements Converter<Map<String, Object>, byte[]> {
private final RedisSerializer serializer;
public FieldsToBytesConverter() {
serializer = new GenericJackson2JsonRedisSerializer();
}
#Override
public byte[] convert(Map<String, Object> value) {
return serializer.serialize(value);
}
}
#Component
#ReadingConverter
public class BytesToFieldsConverter implements Converter<byte[], Map<String, Object>> {
private final GenericJackson2JsonRedisSerializer serializer;
public BytesToFieldsConverter() {
serializer = new GenericJackson2JsonRedisSerializer();
}
#Override
public Map<String, Object> convert(byte[] value) {
return (Map<String, Object>) serializer.deserialize(value);
}
}
I have the following class:
#Configuration
public class ActionsConfig {
private Map<ObjectType, List<Action>> map = new HashMap<>();
#Bean
public Action1 action1() {
return new Action1();
}
#Bean
public Action2 action2(){
return new Action2();
}
#Bean
public Action3 action3(){
return new Action3();
}
private void fillMap(){
//here I am filling my map
}
public Map<ObjectType, List<Action>> getMap(){
return this.map;
}
}
The classes Action1, Action2 and Action3 implements a common Action interface.
Than, inside my service, I autowire the ActionsConfig class and get the map.
#Service
public class BasketService {
#Autowired
private ActionsConfig actionsConfig;
...
public void doSomething(){
...
actionsConfig.getMap()...
...
}
}
Is there a way to autowire just the map, and hence to use the values inside the map directly?
You can create a method that's annotated with #Bean.
#Bean
public Map<ObjectType, List<Action>> getMap() {
Map<ObjectType, List<Action>> map = new HashMap<>();
fillMap()
return map;
}
You can then use #Autowired to autowire the map.
#Autowired
private Map<ObjectType, List<Action>> myMap;
I need to read all properties in application.properties file in a map
In the code below the property test has the respective value but the map is empty. How can I fill "map" with the values in the application.properties file without adding a prefix to the properties.
This is my application.properties file
AAPL=25
GDDY=65
test=22
I'm using #ConfigurationProperties like this
#Configuration
#ConfigurationProperties("")
#PropertySource("classpath:application.properties")
public class InitialConfiguration {
private HashMap<String, BigInteger> map = new HashMap<>();
private String test;
public HashMap<String, BigInteger> getMap() {
return map;
}
public void setMap(HashMap<String, BigInteger> map) {
this.map = map;
}
public String getTest() {
return test;
}
public void setTest(String test) {
this.test = test;
}
}
This can be achieved using the PropertiesLoaderUtils and #PostConstruct
Please check the sample below:
#Configuration
public class HelloConfiguration {
private Map<String, String> valueMap = new HashMap<>();
#PostConstruct
public void doInit() throws IOException {
Properties properties = PropertiesLoaderUtils.loadAllProperties("application.properties");
properties.keySet().forEach(key -> {
valueMap.put((String) key, properties.getProperty((String) key));
});
System.err.println("valueMap -> "+valueMap);
}
public Map<String, String> getValueMap() {
return valueMap;
}
public void setValueMap(Map<String, String> valueMap) {
this.valueMap = valueMap;
}
}
You can't do it with #ConfigurationProperties as far as I'm aware, those require a prefix to be able to load those properties within the bean.
However, if your goal is to obtain "value Y" for "property X" programmatically, you can always inject Environment and use the getProperty() method to find certain property, for example:
#Configuration
public class InitialConfiguration {
#Autowired
private Environment environment;
#PostConstruct
public void test() {
Integer aapl = environment.getProperty("AAPL", Integer.class); // 25
Integer gddy = environment.getProperty("GDDY", Integer.class); // 65
Integer test = environment.getProperty("test", Integer.class); // 22
}
}
In spring boot, if you need to get a single value from the application.proprties, you just need to use the #Value annotation with the given name
So to get AAPL value just add a class level property like this
#Value("${AAPL}")
private String aapl;
And if you need to load a full properties file as a map, I'm using the ResourceLoader to load the full file as a stream and then parse it as follows
#Autowired
public loadResources(ResourceLoader resourceLoader) throws Exception {
Resource resource = resourceLoader.getResource("classpath:myProperties.properties"));
BufferedReader br = new BufferedReader(new InputStreamReader(resource.getInputStream()));
String line;
int pos = 0;
Map<String, String> map = new HashMap<>();
while ((line = br.readLine()) != null) {
pos = line.indexOf("=");
map.put(line.substring(0, pos), line.substring( pos + 1));
}
}
Indeed you can use #ConfigurationProperties without prefix to get entire properties known to Spring application i.e. application, system and environment properties etc.
Following example creates a fully populated map as a Spring bean. Then wire / inject this bean wherever you need it.
#Configuration
class YetAnotherConfiguration {
#ConfigurationProperties /* or #ConfigurationProperties("") */
#Bean
Map<String, String> allProperties() {
return new LinkedHashMap<>();
}
}
#Autowire
void test(Map<String, String> allProperties) {
System.out.println(allProperties.get("AAPL")); // 25
...
}
#PropertySource("classpath:config.properties")
public class GlobalConfig {
public static String AAPL;
#Value("${AAPL}")
private void setDatabaseUrl(String value) {
AAPL = value;
}
}
You have to use #Value to get value from application.properties file