Develop Reference

Use YamlPropertySourceFactory for multiple yaml configurations

I currently have the following situation where elements should be filtered based on a subscription matrix which is divided onto multiple yaml files. For different processes different subscription filters shall be used.
The yamls look like the following:
process1.yaml
subscriptions:
-
attributeA: ...
attributeB: ...
-
attributeA: ...
attributeB: ...
...
process2.yaml
subscriptions:
-
attributeA: ...
attributeB: ...
-
attributeA: ...
attributeB: ...
...
Following this Baeldung link I created a YamplPropertySourceFactory and use it the following way:
#Data
#ConfigurationProperties
#EnableConfigurationProperties
public abstract class ProcessConfig {
private List<Subscription> subscriptions;
#Data
public static class Subscription {
private String attributeA;
private String attributeD;
}
static class YamlPropertySourceFactory implements PropertySourceFactory {
#Override
public org.springframework.core.env.PropertySource<?> createPropertySource(#Nullable String name, EncodedResource resource) {
YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean();
factory.setResources(resource.getResource());
Properties properties = factory.getObject();
//noinspection ConstantConditions
return new PropertiesPropertySource(resource.getResource().getFilename(), properties);
}
}
#Configuration("process1Config")
#PropertySource(value = "classpath:processmatrix/process1.yaml", factory = YamlPropertySourceFactory.class)
public static class Process1Config extends ProcessConfig {
}
#Configuration("process2Config")
#PropertySource(value = "classpath:processmatrix/process2.yaml", factory = YamlPropertySourceFactory.class)
public static class Process2Config extends ProcessConfig {
}
...
}
The issue occourring now is that the bean creation for both configurations is called on application start but only one configuration is received when autowiring in usage.
#RunWith(SpringRunner.class)
#SpringBootTest(classes = {Process1Config.class, Process2Config.class, ...})
public class ProcessConfigTest {
#Autowired
#Qualifier("process1Config")
Process1Config process1Config;
#Autowired
#Qualifier("process2Config")
Process1Config process2Config;
#Test
public void testProcess1Size() {
assertEquals(130, process1Config.getSubscriptions().size());
}
#Test
public void testProcess2Size() {
assertEquals(86, process2Config.getSubscriptions().size());
}
}
If the tests are run testProcess1Size() will pass while testProcess2Size() fails with the subscriptions().size() beeing 130.
Do you have any idea what the issue could be and how I could load multiple yaml configurations? I already tried to set the #Scope("prototype") but it didn't help. Also I tried to remove the common parent class but that didn't help either.

Better way to use HikariCP with Hibernate in Spring Boot Project

I'm a beginner, I have a simple Spring Boot project, it's my first time using a connection pool (HikariCP in this case) and I need your help. It's working but I want to know if I'm using it the right way with Hibernate, or if there are better ways to do it, and if my Spring Boot project structure is correct.
EDIT : It's working even if I remove the class HikariCPConfig, how can I know if connection pools are working or not?
The project is as follow :
- BankManager
src/main/java
|
|__com.manager
|__BankManagerApplication.java
|__HikariCPConfig.java
|__com.manager.dao
|__ClientRepository.java
|__com.manager.entities
|__Client.java
|__com.manager.service
|__ClientServiceImpl.java
|__ClientServiceInterface.java
src/main/resources
|__application.properties
BankManagerApplication.java :
#SpringBootApplication
public class BankManagerApplication {
public static void main(String[] args) {
ApplicationContext ctx = SpringApplication.run(BankManagerApplication.class, args);
ClientServiceInterface service = ctx.getBean(ClientServiceInterface.class);
service.addClient(new Client("client1"));
service.addClient(new Client("client2"));
}
}
HikariCPConfig.java :
#Configuration
#ComponentScan
class HikariCPConfig {
#Value("${spring.datasource.username}")
private String user;
#Value("${spring.datasource.password}")
private String password;
#Value("${spring.datasource.url}")
private String dataSourceUrl;
#Value("${spring.datasource.dataSourceClassName}")
private String dataSourceClassName;
#Value("${spring.datasource.poolName}")
private String poolName;
#Value("${spring.datasource.connectionTimeout}")
private int connectionTimeout;
#Value("${spring.datasource.maxLifetime}")
private int maxLifetime;
#Value("${spring.datasource.maximumPoolSize}")
private int maximumPoolSize;
#Value("${spring.datasource.minimumIdle}")
private int minimumIdle;
#Value("${spring.datasource.idleTimeout}")
private int idleTimeout;
#Bean
public HikariDataSource primaryDataSource() {
Properties dsProps = new Properties();
dsProps.put("url", dataSourceUrl);
dsProps.put("user", user);
dsProps.put("password", password);
dsProps.put("prepStmtCacheSize",250);
dsProps.put("prepStmtCacheSqlLimit",2048);
dsProps.put("cachePrepStmts",Boolean.TRUE);
dsProps.put("useServerPrepStmts",Boolean.TRUE);
Properties configProps = new Properties();
configProps.put("dataSourceClassName", dataSourceClassName);
configProps.put("poolName",poolName);
configProps.put("maximumPoolSize",maximumPoolSize);
configProps.put("minimumIdle",minimumIdle);
configProps.put("minimumIdle",minimumIdle);
configProps.put("connectionTimeout", connectionTimeout);
configProps.put("idleTimeout", idleTimeout);
configProps.put("dataSourceProperties", dsProps);
HikariConfig hc = new HikariConfig(configProps);
HikariDataSource ds = new HikariDataSource(hc);
return ds;
}
}
ClientServiceImpl.java
#Service
public class ClientServiceImpl implements ClientServiceInterface {
#Autowired
ClientRepository clientRepository; // this class extends JPARepository
#Override
public Client addClient(Client c) {
return clientRepository.save(c);
}
}
application.properties :
server.port = 8888
spring.jpa.databasePlatform=org.hibernate.dialect.MySQL5Dialect
spring.jpa.show-sql = true
spring.jpa.hibernate.ddl-auto = update
spring.datasource.dataSourceClassName=com.mysql.jdbc.jdbc2.optional.MysqlDataSource
spring.datasource.url=jdbc:mysql://localhost:3306/bank_manager
spring.datasource.username=root
spring.datasource.password=
spring.datasource.poolName=SpringBootHikariCP
spring.datasource.maximumPoolSize=5
spring.datasource.minimumIdle=3
spring.datasource.maxLifetime=2000000
spring.datasource.connectionTimeout=30000
spring.datasource.idleTimeout=30000
spring.datasource.pool-prepared-statements=true
spring.datasource.max-open-prepared-statements=250
Thank you in advance.

You project structure is standard, so it's correct.
About Hikari:
Hikari is indeed a great choice for pooling. I'm used to work with Hikari successfully by using a smaller set of params you are applying in your case, but if it's working for you that's fine.
For more info about Hikaru setup, I recommend reading the official wiki, if you haven't already.
About property loading:
You can make use of some SpringBoot features to read the DB parameters and apply into your runtime Beans with less code. Like:
In application.properties (define a custom prefix 'myproject.db' for your pool params)
myproject.db.dataSourceClassName=com.mysql.jdbc.jdbc2.optional.MysqlDataSource
myproject.db.url=jdbc:mysql://localhost:3306/bank_manager
myproject.db.username=root
... and the other params below
Create a Spring Configuration class
#Configuration
public class MyDBConfiguration {
#Bean(name = "myProjectDataSource")
#ConfigurationProperties(prefix = "myproject.db")
public DataSource dataSource(){
//This will activate Hikari to create a new DataSource instance with all parameters you defined with 'myproject.db'
return DataSourceBuilder.create().build();
}
}
In your ClientRepository class:
#Repository
public class ClientRepository {
//The code below is optional, but will work if you want to use jdbctemplate tied to the DataSource created above. By default all Hibernate Sessions will take the DataSource generated by Spring
#Bean(name = "myProjectJdbcTemplate")
public JdbcTemplate jdbcTemplate(#Qualifier("myProjectDataSource") DataSource dataSource){
return new JdbcTemplate(dataSource);
}
}
There are other options to manage the DataSource beans creation if you are going to use 2 or more different Databases. You can vary the properties prefix for the other databases and annotate 1 Datasource only as #Primary, which is mandatory when you have more than 1 DataSources in Spring context

Spring boot #ConfigurationProperties not loaded

As the title says, I'm trying to use Typesafe Configuration Properties to load a list of DataSourceConfig objects. I have lombok for setter/getters
The main application class annotations
#Slf4j
#SpringBootApplication
#EnableConfigurationProperties
public class Application {
The configuration pojo
#Data
public class DataSourceConfig {
private String key;
private String dbname;
private String dbpath;
}
The yml file
tenantdb:
dataSourceConfig:
-
key: default
dbpath: file:eventstore/jdbc/database
dbname: defaultdb
-
key: other
dbpath: file:eventstore/jdbc/other
dbname: dslfjsdf
Finally, the Spring Configuration class with the #ConfigurationProperties annotation.
#Configuration
#Profile("hsqldb")
#ImportResource(value = { "persistence-config.xml" })
#Slf4j
#ConfigurationProperties(prefix="tenantdb", locations={"datasources.yml"})
public class HsqlConfiguration {
private List<DataSourceConfig> dataSourceConfig = new ArrayList<>();
#Bean
public List<DataSourceConfig> getDataSourceConfig() {
return dataSourceConfig;
}
With the config above, I get:
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'hsqlConfiguration': Could not bind properties to [unknown] (target=tenantdb, ignoreInvalidFields=false, ignoreUnknownFields=true, ignoreNestedProperties=false); nested exception is java.lang.NullPointerException
at org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor.postProcessBeforeInitialization(ConfigurationPropertiesBindingPostProcessor.java:303)
at org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor.postProcessBeforeInitialization(ConfigurationPropertiesBindingPostProcessor.java:250)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:408)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initia
I've tried various combinations. If I change the annotation to #ConfigurationProperties(prefix="tenantdb.dataSourceConfig"), I don't get the error but List<DataSourceConfig> is empty.
HELP!!

You should use configuration properties as simple POJO with only getters and setters and have separate HsqlConfiguration which has this properties injected.
Something like this:
#Component
#ConfigurationProperties(prefix="tenantdb", locations={"datasources.yml"})
public class TenantDbProperties {
//DataSourceConfig is POJO with key, dbpath and dbname
private List<DataSourceConfig> dataSourceConfigs;
public List<DataSourceConfig> getDataSourceConfigs(){
return dataSourceConfigs;
}
public void setDataSourceConfigs(List<DataSourceConfig> dataSourceConfigs){
this.dataSourceConfigs = dataSourceConfigs;
}
}
And in separate class have this properties injected as:
#Configuration
#Profile("hsqldb")
#ImportResource(value = { "persistence-config.xml" })
#Slf4j
public class HsqlConfiguration {
#Autowired
private TenantDbProperties tenantDbProperties;
//code goes here where you can use tenantDbProperties.getDataSourceConfigs()
}

Spring Boot - Environment #Autowired throws NullPointerException

I have a project setup using Spring Boot 0.5.0.M5.
In one of the configuration files I am trying to #Autowire Environment but that fails with a NullPointerException.
Here's what I have so far:
Application.java
#EnableAutoConfiguration
#Configuration
#ComponentScan
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
JpaConfig.java where I am trying to #Autowire Environment
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackages = "com.ui.persistence.repository")
public class JpaConfig {
private static final String DATABASE_DRIVER = "db.driver";
private static final String DATABASE_PASSWORD = "db.password";
private static final String DATABASE_URL = "db.url";
private static final String DATABASE_USERNAME = "db.username";
private static final String HIBERNATE_DIALECT = "hibernate.dialect";
private static final String HIBERNATE_SHOW_SQL = "hibernate.show_sql";
private static final String ENTITYMANAGER_PACKAGES_TO_SCAN
= "entitymanager.packages.to.scan";
#Autowired
private Environment env;
#Bean
public DataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(env.getProperty(DATABASE_DRIVER));
dataSource.setUrl(env.getProperty(DATABASE_URL));
dataSource.setUsername(env.getProperty(DATABASE_USERNAME));
dataSource.setPassword(env.getProperty(DATABASE_PASSWORD));
return dataSource;
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
LocalContainerEntityManagerFactoryBean entityManagerFactoryBean
= new LocalContainerEntityManagerFactoryBean();
entityManagerFactoryBean.setDataSource(dataSource());
entityManagerFactoryBean.setPersistenceProviderClass(
HibernatePersistence.class);
entityManagerFactoryBean.setPackagesToScan(
env.getProperty(ENTITYMANAGER_PACKAGES_TO_SCAN));
entityManagerFactoryBean.setJpaProperties(hibernateProperties());
return entityManagerFactoryBean;
}
}
I am trying to load the database properties configured in a properties file. However, the Environment is not injected and the code fails with NullPointerException. I do not have any configuration in XML files.
For the properties file I have configured PropertySourcesPlaceholderConfigurer this way:
#Configuration
#PropertySource("classpath:database.properties")
public class PropertyConfig {
#Bean
public static PropertySourcesPlaceholderConfigurer propertyPlaceHolderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
}
I have tried swapping #Autowired, #Resource and #Inject but nothing has worked so far. Would appreciate any help. Thanks.

Though your specific problem is solved, here's how to get Environment in case Spring's autowiring happens too late.
The trick is to implement org.springframework.context.EnvironmentAware; Spring then passes environment to setEnvironment() method.
This works since Spring 3.1.
An example:
#Configuration
#PropertySource("classpath:myProperties.properties")
public class MyConfiguration implements EnvironmentAware {
private Environment environment;
#Override
public void setEnvironment(final Environment environment) {
this.environment = environment;
}
public void myMethod() {
final String myPropertyValue = environment.getProperty("myProperty");
// ...
}
}
This is not as elegant as #Autowired or #Value, but it works as workaround in some situations.

I believe there were some lifecycle issues with Spring and the EntityManagerFactory, and you might have fallen foul of those (fixed in 4.0.0.RC1) - if your #Configuration class gets instantiated super early, it might not be eligible for autowiring. You can probably tell from the log output if that is the case.
Just out of interest, did you know that the functionality provided by your JpaConfig and PropertyConfig is already presetn out of the box if you use #EnableAutoConfiguration (as long as you #ComponentScan that package where your repositories are defined)? See the JPA sample in Spring Boot for an example.

I had the same problem on Spring Batch. Writers cannot autowire Environment class because Configuration class was instantiated earlier.
So I created a sort of Singleton (old manner) to instantiate Environment and I could access to it every time.
I did this implementation :
#Configuration
#PropertySource(value = { "classpath:kid-batch.properties" }, ignoreResourceNotFound = false)
public class BatchConfiguration implements EnvironmentAware {
private static Environment env;
public static String getProperty(String key) {
return env.getProperty(key);
}
#Override
public void setEnvironment(Environment env) {
BatchConfiguration.env = env;
}
}
And it works

I was having the similar issue to read properties from my application.properties file in spring boot application. I have struggled a lot to figure out the problem and make it work. Finally I have done. Here is my Constants class which will read properties values from properties file. I hope it will help to someone.
import org.springframework.context.EnvironmentAware;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
import org.springframework.context.support.PropertySourcesPlaceholderConfigurer;
import org.springframework.core.env.Environment;
#Configuration
#PropertySource("classpath:application.properties")
public class Constants implements EnvironmentAware {
static Environment environment;
#Override
public void setEnvironment(Environment environment) {
Constants.environment = environment;
}
#Bean
public static PropertySourcesPlaceholderConfigurer propertyConfigInDev() {
return new PropertySourcesPlaceholderConfigurer();
}
public static String getActiveMQHost() {
System.out.println(environment.getProperty("spring.activemq.broker-host"));
return environment.getProperty("spring.activemq.broker-host");
}
public static String getActiveMQPort() {
System.out.println(environment.getProperty("spring.activemq.broker-port"));
return environment.getProperty("spring.activemq.broker-port");
}
public static String getActiveMQUser() {
System.out.println(environment.getProperty("spring.activemq.user"));
return environment.getProperty("spring.activemq.user");
}
public static String getActiveMQPassword() {
System.out.println(environment.getProperty("spring.activemq.password"));
return environment.getProperty("spring.activemq.password");
}
}
These are the property key's declared in my application.properties,
spring.activemq.broker-host
spring.activemq.broker-port
spring.activemq.user
spring.activemq.password

In spring boot how to load a yml section as java.util.Properties

In my spring boot application, I have following configuration:
server:
host: a.com
port: 5922
enable-log: true
I want to read the above as a java.util.Properties. I tried putting following class:
#ConfigurationProperties(prefix = "server")
public class ServerConfig {
private Properties serverProps;
// ... Getter/setter
}
The boot config files look like:
#Configuration
#ComponentScan("com.demo")
#EnableAspectJAutoProxy(proxyTargetClass = true)
#EnableConfigurationProperties({ServerConfig.class})
#Profile({"dev"})
public class TestAppConfiguration {
}
#EnableAutoConfiguration
#SpringBootApplication
public class TestAppInitializer {
public static void main(final String[] args) {
SpringApplication.run(TestAppInitializer.class, args);
}
}
Unit test class:
#SpringApplicationConfiguration(classes = {TestAppInitializer.class})
#ActiveProfiles("dev")
public class ServerConfigTest extends AbstractTestNGSpringContextTests {
#Autowired
private ServerConfig serverConfig;
#Test
public void printDetails() {
logger.debug("ServerConfig.properties --> {}", serverConfig.getProperties());
}
}
The output is
ServerConfig.properties --> null.
I am not sure why is this happening. The only idea that comes to my mind is, the configuration parameters underneath server are basically flat strings. Is there any way I can read those as Properties?

#Bean({"kaptchaProperties"})
#ConfigurationProperties(prefix = "custom.kaptcha")
public Properties getProperties() {
return new Properties();
}

AFAIK, you can't load them into java.util.Properties with annotation #ConfigurationProperties.
You need to create this:
#Component
#ConfigurationProperties(prefix = "server")
public class ServerConfig {
private String host;
private String port;
private String enableLog;
// ... Getter/setter
}