Develop Reference

Use YamlPropertySourceFactory for multiple yaml configurations

I currently have the following situation where elements should be filtered based on a subscription matrix which is divided onto multiple yaml files. For different processes different subscription filters shall be used.
The yamls look like the following:
process1.yaml
subscriptions:
-
attributeA: ...
attributeB: ...
-
attributeA: ...
attributeB: ...
...
process2.yaml
subscriptions:
-
attributeA: ...
attributeB: ...
-
attributeA: ...
attributeB: ...
...
Following this Baeldung link I created a YamplPropertySourceFactory and use it the following way:
#Data
#ConfigurationProperties
#EnableConfigurationProperties
public abstract class ProcessConfig {
private List<Subscription> subscriptions;
#Data
public static class Subscription {
private String attributeA;
private String attributeD;
}
static class YamlPropertySourceFactory implements PropertySourceFactory {
#Override
public org.springframework.core.env.PropertySource<?> createPropertySource(#Nullable String name, EncodedResource resource) {
YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean();
factory.setResources(resource.getResource());
Properties properties = factory.getObject();
//noinspection ConstantConditions
return new PropertiesPropertySource(resource.getResource().getFilename(), properties);
}
}
#Configuration("process1Config")
#PropertySource(value = "classpath:processmatrix/process1.yaml", factory = YamlPropertySourceFactory.class)
public static class Process1Config extends ProcessConfig {
}
#Configuration("process2Config")
#PropertySource(value = "classpath:processmatrix/process2.yaml", factory = YamlPropertySourceFactory.class)
public static class Process2Config extends ProcessConfig {
}
...
}
The issue occourring now is that the bean creation for both configurations is called on application start but only one configuration is received when autowiring in usage.
#RunWith(SpringRunner.class)
#SpringBootTest(classes = {Process1Config.class, Process2Config.class, ...})
public class ProcessConfigTest {
#Autowired
#Qualifier("process1Config")
Process1Config process1Config;
#Autowired
#Qualifier("process2Config")
Process1Config process2Config;
#Test
public void testProcess1Size() {
assertEquals(130, process1Config.getSubscriptions().size());
}
#Test
public void testProcess2Size() {
assertEquals(86, process2Config.getSubscriptions().size());
}
}
If the tests are run testProcess1Size() will pass while testProcess2Size() fails with the subscriptions().size() beeing 130.
Do you have any idea what the issue could be and how I could load multiple yaml configurations? I already tried to set the #Scope("prototype") but it didn't help. Also I tried to remove the common parent class but that didn't help either.

Spring Boot: read from yaml using #ConfigurationProperties not working with #Data

I am trying to retrieve value from application.yml. The final line below is showing kafkaConfig as null, and cannot read. How do I setup the Kafka Config and code properly, to read from the json file? We are using #Data instead of getters/setters.
KafkaConfig.java
#Configuration
#ConfigurationProperties("kafka")
#Data
public class KafkaConfig {
private String topic;
private String event;
}
Application.yml
kafka:
topic: "testTopic"
event: "testEvent"
KafkaProducerBeans.java
#Component
public class KafkaProducerBeans {
#Autowired
private KafkaConfig kafkaConfig;
public KafkaProducerBeans(KafkaConfig kafkaConfig) {
this.kafkaConfig = kafkaConfig;
}
#Bean(name = "kafkaTestClient")
public String getData() {
return kafkaConfig.getTopic(); // final line is creating null for kafka Config
}
Resource: https://codingnconcepts.com/spring-boot/spring-configuration-properties-using-yml/

I think the recommended way of binding properties to pojos is to use the #EnableConfigurationProperties annotation like so:
KafkaConfig.java
#ConfigurationProperties("kafka")
#Data
public class KafkaConfig {
private String topic;
private String event;
}
KafkaProducerBeans.java
#Component
#EnableConfigurationProperties(KafkaConfig.class)
public class KafkaProducerBeans {
private final KafkaConfig kafkaConfig;
#Autowired
public KafkaProducerBeans(KafkaConfig kafkaConfig) {
this.kafkaConfig = kafkaConfig;
}
// [...]
}
Refer to the official Spring Documentation for further details:
https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.external-config.typesafe-configuration-properties.java-bean-binding
https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.external-config.typesafe-configuration-properties.enabling-annotated-types

Add one more annotation #EnableConfigurationProperties on the class KafkaConfig
KafkaConfig.java
#Configuration
#EnableConfigurationProperties // new added annotation
#ConfigurationProperties("kafka")
#Data
public class KafkaConfig {
private String topic;
private String event;
}

#ComponentScan(basePackages ="PATH" )
PATH: the package path you want it to look for
#ComponentScan(basePackages ="PATH" )
#Configuration
#Import(value = {
KafkaProducerBeans.class
})
public class AppConfig{
}

Reading YAML properties file in spring boot via PropertySourceFactory using YamlMapFactoryBean

I'm trying to read a YAML configuration file through the spring boot #PropertySource mechanism, using a factory that utilizes the provided YamlMapFactoryBean parser.
The factory's implementation is as follows:
public class YamlPropertySourceFactory implements PropertySourceFactory {
#Override
public PropertySource<?> createPropertySource(String name, EncodedResource encodedResource) {
YamlMapFactoryBean factory = new YamlMapFactoryBean();
factory.setResources(encodedResource.getResource());
Map<String, Object> map = factory.getObject();
return new MapPropertySource(encodedResource.getResource().getDescription(), map);
}
}
The YAML configuration file (foo.yml) is:
yaml:
name: foo
aliases:
- abc
- xyz
The corresponding entity is:
#Configuration
#ConfigurationProperties(prefix = "yaml")
#PropertySource(value = "classpath:foo.yml", factory = YamlPropertySourceFactory.class)
public class YamlFooProperties {
private String name;
private List<String> aliases;
// Getters and Setters...
}
And finally, my test class is:
#RunWith(SpringRunner.class)
#SpringBootTest
public class YamlFooPropertiesTest {
#Autowired
private YamlFooProperties yamlFooProperties;
#Test
public void whenFactoryProvidedThenYamlPropertiesInjected() {
assertThat(yamlFooProperties.getName()).isEqualTo("foo");
assertThat(yamlFooProperties.getAliases()).containsExactly("abc", "xyz");
}
}
When debugging the factory, I see that the YAML file is parsed correctly and added to spring boot's propertySourceNames structure. However, when accessing it from the test in an Autowired fashion, all fields are null and the test therefore fails.

Spring-Boot multi module project load property-file

I have a Spring-Boot-Application as a multimodule-Project in maven. The structure is as follows:
Parent-Project
|--MainApplication
|--Module1
|--ModuleN
In the MainApplication project there is the main() method class annotated with #SpringBootApplication and so on. This project has, as always, an application.properties file which is loaded automatically. So I can access the values with the #Value annotation
#Value("${myapp.api-key}")
private String apiKey;
Within my Module1 I want to use a properties file as well (called module1.properties), where the modules configuration is stored. This File will only be accessed and used in the module. But I cannot get it loaded. I tried it with #Configuration and #PropertySource but no luck.
#Configuration
#PropertySource(value = "classpath:module1.properties")
public class ConfigClass {
How can I load a properties file with Spring-Boot and access the values easily? Could not find a valid solution.
My Configuration
#Configuration
#PropertySource(value = "classpath:tmdb.properties")
public class TMDbConfig {
#Value("${moviedb.tmdb.api-key}")
private String apiKey;
public String getApiKey() {
return apiKey;
}
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
}
Calling the Config
#Component
public class TMDbWarper {
#Autowired
private TMDbConfig tmdbConfig;
private TmdbApi tmdbApi;
public TMDbWarper(){
tmdbApi = new TmdbApi(tmdbConfig.getApiKey());
}
I'm getting an NullPointerException in the constructor when I autowire the warper.

For field injection:
Fields are injected right after construction of a bean, before any config methods are invoked. Such a config field does not have to be public. Refer Autowired annotation for complete usage. Use constructor injection in this case like below:
#Component
public class TMDbWarper {
private TMDbConfig tmdbConfig;
private TmdbApi tmdbApi;
#Autowired
public TMDbWarper(final TMDbConfig tmdbConfig){
this.tmdbConfig = tmdbConfig;
tmdbApi = new TmdbApi(tmdbConfig.getApiKey());
}
(or)
Use #PostConstruct to initialise like below:
#Component
public class TMDbWarper {
#Autowired
private TMDbConfig tmdbConfig;
private TmdbApi tmdbApi;
#PostConstruct
public void init() {
// any initialisation method
tmdbConfig.getConfig();
}

Autowiring is performed just after the creation of the object(after calling the constructor via reflection). So NullPointerException is expected in your constructor as tmdbConfig field would be null during invocation of constructor
You may fix this by using the #PostConstruct callback method as shown below:
#Component
public class TMDbWarper {
#Autowired
private TMDbConfig tmdbConfig;
private TmdbApi tmdbApi;
public TMDbWarper() {
}
#PostConstruct
public void init() {
tmdbApi = new TmdbApi(tmdbConfig.getApiKey());
}
public TmdbApi getTmdbApi() {
return this.tmdbApi;
}
}
Rest of your configuration seems correct to me.
Hope this helps.

Here is a Spring Boot multi-module example where you can get properties in different module.
Let's say I have main application module, dataparse-module, datasave-module.
StartApp.java in application module:
#SpringBootApplication
public class StartApp {
public static void main(String[] args) {
SpringApplication.run(StartApp.class, args);
}
}
Configuration in dataparse-module. ParseConfig.java:
#Configuration
public class ParseConfig {
#Bean
public XmlParseService xmlParseService() {
return new XmlParseService();
}
}
XmlParseService.java:
#Service
public class XmlParseService {...}
Configuration in datasave-module. SaveConfig.java:
#Configuration
#EnableConfigurationProperties(ServiceProperties.class)
#Import(ParseConfig.class)//get beans from dataparse-module - in this case XmlParseService
public class SaveConfig {
#Bean
public SaveXmlService saveXmlService() {
return new SaveXmlService();
}
}
ServiceProperties.java:
#ConfigurationProperties("datasave")
public class ServiceProperties {
private String message;
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
}
application.properties in datasave-module in resource/config folder:
datasave.message=Multi-module Maven project!
threads.xml.number=5
file.location.on.disk=D:\temp\registry
Then in datasave-module you can use all your properties either through #Value.
SaveXmlService.java:
#Service
public class SaveXmlService {
#Autowired
XmlParseService xmlParseService;
#Value("${file.location.on.disk: none}")
private String fileLocation;
#Value("${threads.xml.number: 3}")
private int numberOfXmlThreads;
...
}
Or through ServiceProperties:
Service.java:
#Component
public class Service {
#Autowired
ServiceProperties serviceProperties;
public String message() {
return serviceProperties.getMessage();
}
}

I had this situation before, I noticed that the properties file was not copied to the jar.
I made the following to get it working:
In the resources folder, I have created a unique package, then stored my application.properties file inside it. e.g: com/company/project
In the configuration file e.g: TMDBConfig.java I have referenced the full path of my .properties file:
#Configuration
#PropertySource("classpath:/com/company/project/application.properties")
public class AwsConfig
Build and run, it will work like magic.

You could autowire and use the Enviornment bean to read the property
#Configuration
#PropertySource(value = "classpath:tmdb.properties")
public class TMDbConfig {
#Autowired
private Environment env;
public String getApiKey() {
return env.getRequiredProperty("moviedb.tmdb.api-key");
}
}
This should guarantee that property is read from the context when you invoke the getApiKey() method regardless of when the #Value expression is resolved by PropertySourcesPlaceholderConfigurer.

How to load multiple properties from YML file in spring-boot application

We have a YML configuration which looks like:
datasurces:
readDataSource:
ssl-enabled: false
driver-class-name: oracle.jdbc.driver.OracleDriver
host: db1.abc.com
port: 1232
sid: ABC_DB
trust-store-fileName: abcdb.jks
connection-pool:
initial-size: 10
max-size: 20
writeDataSource:
ssl-enabled: false
driver-class-name: oracle.jdbc.driver.OracleDriver
host: db2.abc.com
port: 1232
sid: XYZ_DB
trust-store-fileName: xyzdb.jks
connection-pool:
initial-size: 10
max-size: 20
We have to load this to a custom class DataSources which looks like
#ConfigurationProperties(prefix = "datasources")
public class DataSources {
#Value("${datasources.readDataSource}")
private DataSource readDataSource;
#Value("${datasources.writeDataSource}")
private DataSource writeDataSource;
//...getters/setters
}
public class DataSource {
private String id;
private boolean sslEnabled;
private String driverClassName;
private String host;
private int port;
private String trustStoreFileName;
private ConnectionPool connectionPool;
//...getters/setters
}
public class ConnectionPool {
private int initialSize;
private int maxSize;
//...getters/setters
}
My configuration files for spring boot looks like:
#Configuration
#ComponentScan(basePackages = {"com.abc"})
#EnableAspectJAutoProxy(proxyTargetClass = true)
#EnableConfigurationProperties(DataSources.class)
#Profile({"dev"})
public class TestAppConfiguration {
}
#EnableAutoConfiguration
#SpringBootApplication
#EnableConfigurationProperties(DataSources.class)
public class TestAppInitializer {
#Autowired
private DataSources dataSources;
public static void main(final String[] args) {
SpringApplication.run(TestAppInitializer.class, args);
}
}
The unit test is:
#SpringApplicationConfiguration(classes = {TestAppInitializer.class})
#Test(groups = "categoryTests")
#ActiveProfiles("dev")
public class DataSourcesTest extends AbstractTestNGSpringContextTests {
private static final AppLogger logger = LoggerUtil.getLogger(DataSourcesTest.class);
#Autowired
private DataSources dataSources;
#Test
public void printDetails() {
logger.debug("DataSources --> {}", dataSources);
}
}
Result is not as expected.
When I remove #Value from DataSources class, both the properties readDataSource and writeDataSource are null (The DataSources class itself is not null).
When I add #Value in DataSources class, the test fails with exception
Caused by: java.lang.IllegalStateException: Cannot convert value of type [java.lang.String] to required type [com.abc.DataSource]: no matching editors or conversion strategy found
Appreciate if someone can provide some idea how to deal with this. All I want is to capture both readDataSource and writeDataSource inside a class like DataSources.

Annotate your DataSources class with #Configuration then create 2 #Bean methods annotatated with #ConfigurationProperties.
#Configuration
public class DataSources {
#Bean
#ConfigurationProperties(prefix="datasources.readDataSource")
public DataSource readDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="datasources.writeDataSource")
public DataSource writeDataSource() {
return DataSourceBuilder.create().build();
}
}
Now you have 2 datasources with the properties bound to the created DataSources. This mechanism is explained here in the Spring Boot Reference Guide.
The same would apply if you don't need a DataSource but construct your own object (although not sure why you would need that?).