Develop Reference

Spring (not boot) load multiple yml files from multiple projects

So I have read dosens af articles on how to configure Spring boot to be aware of more yml files than application.yml and how to include these - even from subprojects. It is however hard to come by articles describing the same for "pure" Spring. I think however that i'm heading in the right direction I just can't get my configuration values back.
It's a straight forward multi-project gradle build with - for simplicity - two projects. One project is the "main" spring project - ie. Spring Context is initialized in this project. The other is a "support" module with some database entities and datasource configuration. We use annotation based configuration.
I would like to be able to define a set of configuration properties in the support module and based on whatever spring profile is active, the datasource configuration is loaded accordingly.
This SA post got me quite far following the different links in the different answers and composing my solution from this. The structure and code is as follows:
mainproject
src
main
groovy
Application.groovy
resourcers
application.yml
submodule
src
main
groovy
PropertiesConfiguration.groovy
DataSource.groovy
resources
datasource.yml
The PropertiesConfiguration.groovy adds the datasource.yml by using PropertySourcesPlaceholderConfigurer:
#Configuration
class PropertiesConfiguration {
#Bean
public PropertySourcesPlaceholderConfigurer configure() {
PropertySourcesPlaceholderConfigurer configurer = new PropertySourcesPlaceholderConfigurer()
YamlPropertiesFactoryBean yamlPropertiesFactoryBean = new YamlPropertiesFactoryBean()
yamlPropertiesFactoryBean.setResources(new ClassPathResource("datasource.yml"))
configurer.setProperties(yamlPropertiesFactoryBean.getObject())
return configurer
}
}
The Datasource.groovy should then read values based on the spring profile using (code reduced for readability):
#Autowired
Environment env
datasource.username = env.getProperty("datasource.username")
The env.getProperty returns null. No matter what spring profile is active. I can access the configuration value using the #Value annotation, however then the active profile is not respected and it return a value even if it is not defined for that profile. My yml looks (something) like this:
---
spring:
profiles: development
datasource:
username: sa
password:
databaseUrl: jdbc:h2:mem:tests
databaseDriver: org.h2.Driver
I can from Application.groovy inspect my ApplicationContext using a debugger and confirm that my PropertySourcesPlaceholderConfigurer exist and the values are loaded. Inspecting applicationContext.environment.propertySources it is NOT there.
What am I missing?

Using a PropertySourcesPlaceholderConfigurer does not add properties to Environment. Using something like #PropertySource("classpath:something.properties") on the class level of your configuration class will add properties to Environment, but sadly this does not work with yaml-files.
So, you would have to manually add the properties read from the yaml file to your Environment. Here is one way to do this:
#Bean
public PropertySourcesPlaceholderConfigurer config(final ConfigurableEnvironment confenv) {
final PropertySourcesPlaceholderConfigurer configurer = new PropertySourcesPlaceholderConfigurer();
final YamlPropertiesFactoryBean yamlProperties = new YamlPropertiesFactoryBean();
yamlProperties.setResources(new ClassPathResource("datasource.yml"));
configurer.setProperties(yamlProperties.getObject());
confenv.getPropertySources().addFirst(new PropertiesPropertySource("datasource", yamlProperties.getObject()));
return configurer;
}
With this code, you can inject properties in either of these two fashions:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = PropertiesConfiguration.class)
public class ConfigTest {
#Autowired
private Environment environment;
#Value("${datasource.username}")
private String username;
#Test
public void props() {
System.out.println(environment.getProperty("datasource.username"));
System.out.println(username);
}
}
With the properties supplied in the question, this will print "sa" two times.
Edit: It doesn't seem that the PropertySourcesPlaceholderConfigurer is actually needed now, so the code can be simplified to the below and still produce the same output.
#Autowired
public void config(final ConfigurableEnvironment confenv) {
final YamlPropertiesFactoryBean yamlProperties = new YamlPropertiesFactoryBean();
yamlProperties.setResources(new ClassPathResource("datasource.yml"));
confenv.getPropertySources().addFirst(new PropertiesPropertySource("datasource", yamlProperties.getObject()));
}
Edit 2:
I see now that you are looking to use the yaml-file with multiple documents in one file, and Spring boot-style selection by profile. It does not seem to be possible using regular Spring. So I think you have to split your yaml files into several, named "datasource-{profile}.yml". Then, this should work (perhaps with some more advanced checking for multiple profiles, etc)
#Autowired
public void config(final ConfigurableEnvironment confenv) {
final YamlPropertiesFactoryBean yamlProperties = new YamlPropertiesFactoryBean();
yamlProperties.setResources(new ClassPathResource("datasource-" + confenv.getActiveProfiles()[0] + ".yml"));
confenv.getPropertySources().addFirst(new PropertiesPropertySource("datasource", yamlProperties.getObject()));
}
Edit 3:
It could also be possible to use functionality from Spring boot without doing a full conversion of your project (I haven't actually tried it on a real project though). By adding a dependency to org.springframework.boot:spring-boot:1.5.9.RELEASE I was able to get it working with the single datasource.yml and multiple profiles, like this:
#Autowired
public void config (final ConfigurableEnvironment confenv) {
final YamlPropertySourceLoader yamlPropertySourceLoader = new YamlPropertySourceLoader();
try {
final PropertySource<?> datasource =
yamlPropertySourceLoader.load("datasource",
new ClassPathResource("datasource.yml"),
confenv.getActiveProfiles()[0]);
confenv.getPropertySources().addFirst(datasource);
} catch (final IOException e) {
throw new RuntimeException("Failed to load datasource properties", e);
}
}

Configuring the database properties from outside of intellij

I want to ask as currently I have my database properties like username and password inside the persistence layer in the intellij. But I want to place it somewhere outside so if someone wants to change the password or any configuration inside database he should not have to dig inside my current structure. Now my structure is persistence then main then resources and then dbconfig properties so is there any way I can do it.

You can create a file app.properties in your resources folder with all database information you need:
# Datasource details
testapp.db.driver = org.h2.Driver
testapp.db.url = jdbc:h2:mem:test
testapp.db.username = username
testapp.db.password = password
Then you can refer to it in your Java code as:
#Configuration
#PropertySource("app.properties")
public class DataConfig {
#Autowired
private Environment env;
#Bean
public DataSource dataSource() {
BasicDataSource ds = new BasicDataSource();
ds.setDriverClassName(env.getProperty("testapp.db.driver"));
ds.setUrl(env.getProperty("testapp.db.url"));
ds.setUsername(env.getProperty("testapp.db.username"));
ds.setPassword(env.getProperty("testapp.db.password"));
return ds;
}
}

Spring Boot not reading from property file

I have tried every option on web but not able to set the values in following method:
#Configuration
#PropertySource("classpath:application.properties")
public class MyDataSource {
#Value("${db.driver}")
private String DB_DRIVER;
#Value("${db.url}")
private String DB_URL;
#Value("${db.username}")
private String DB_USERNAME;
#Value("${db.password}")
private String DB_PASSWORD;
#Bean
public static PropertySourcesPlaceholderConfigurer placeHolderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
#Bean
public DataSource getDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(DB_DRIVER);
dataSource.setUrl(DB_URL);
dataSource.setUsername(DB_USERNAME);
dataSource.setPassword(DB_PASSWORD);
return dataSource;
}
}
My application.properties is in main/resources folder and values can be seen in variables in debug mode. But on running app, it shows Property ' ' must not be empty.
EDIT: I am not sure what can be the issue in first case?
So changed the application.property file as suggested and code as below :
#Autowired
protected JdbcTemplate jdbcTemp;
public List<> getData(String id) {
return jdbcTemp.query("SELECT ........,new RowMapper());
}
But getting java.lang.NullPointerException:

If you're using Spring Boot, you can leverage application.properties file by declaring some entries:
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://localhost/test
spring.datasource.username=dbuser
spring.datasource.password=dbpass
In this way there is no need to implement a #Configuration class to setup database connection in Spring Boot.
You can deepen more here:
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html
By the way, take a look at spring.io

For the java configuration, using Environment instance to obtain the properties seems to be the preferred way, as by default ${..} placeholders are not resolved.
You may use something like this:
#Autowired
private Environment env;
#Bean
public DataSource getDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(env.getProperty("db.driver");
.....
return dataSource;
}
Reasons from the Spring Jira:
it's inconsistent. #PropertySource is the declarative counterpart to ConfigurableEnvironment#addPropertySource. We do not add a
PropertySourcesPlaceholderConfigurer in the latter case, and it would
be inconsistent to do so in the former. it will not be what the user
intended in every (or even most) cases.
It is entirely possible, and even recommended that #Configuration class users forego $ {...} property replacement entirely, in favor of
Environment#getProperty lookups within #Bean methods. For users
following this recommendation, the automatic registration of a
PropertySorucesPlaceholderConfigurer would be confusing when noticed,
and generally undesirable as it's one more moving part. Yes, it's
presence is benign, but not cost-free. a PSPC must visit every bean
definition in the container to interrogate PropertyValues, only to do
nothing in cases where users are going with the
Environment#getProperty approach.
it is solvable (and already solved) by documentation. Proper use of #PropertySource, PropertySourcesPlaceholderConfigurer and other
components is pretty comprehensively documented in the Javadoc for
#Configuration already, and reference documentation is soon to follow.

Me too was getting the error when tried to switch from MySQL to MSSQL. The actual issue was I forgot to put the MSSQL dependency in the service. I used mssql-jdbc

Mock the Spring Environment Object In Junit with #ContextConfiguration

I have got a test like this
RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = { AppConfig.class, TestConfig.class})
public class MyTest {
........
}
The AppConfig is the main config for my app, the TestConfig is the test config
which loads the test properties
#Configuration
#PropertySource("classpath:test_dev.properties")
public class DevConfig {
#Bean
public DataSource getDataDataSource() {
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName(env.getProperty("driverclass"));
dataSource.setUrl(env.getProperty("url"));
dataSource.setUsername(env.getProperty("username"));
dataSource.setPassword(env.getProperty("password"));
return dataSource;
}
}
problem is The test_dev.properties file has a encrypted password field
driverclass = ojdbc:xxx
url = xxxxxx
username = abc
password = #'"#~£$%
I need to use decryptor to decrypt it, then the decrypted password on the env object. thus env.get("password"), the real password will return
so my question is how can I mock the Environment object before the DataSource object gets crated.

in my case, I wanted to mock the password filed on the datasource object
I first let the datasource bean initialize, then in my integration test autowired the mybatis Environment (don't be confused with springs environment).
from the env you can get the datasource.
from the #beforetest section, the trick is to use Reflectiontestutils.setfield to set the password field on the datasource
alternatively you could inject the entire datasource on the target environment
but I didn't try
the key motivation for using Spring is to easily inject mock objects.

How to access non-static properties in the creation of a Spring Factory Bean?

Looking for any type of solution here, but I'll propose my thought process along the way.
Working with Spring-Mybatis (over a MySql database) and running it through the Maven embedded jetty plugin.
So it's a pretty simple issue I'm running into and easy to re-create. Basically, Mybatis-Spring requires one to define a Spring bean for the SqlSessionFactoryBean class. This class implements Spring's FactoryBean interface. To wire it up properly, one should supply it, among other things, a DataSource. I reference the JDBC connection details elsewhere in code (for Mybatis-Generator purposes) and would like to keep those details consolidated to one place: property file or something else.
For now, I have a property file, and everything built pretty standard from a Spring point-of-view. I start seeing issues when trying to boot up the Jetty server though, Mybatis-Spring seems to catch itself in a circular dependency.
I cannot say I quite know exactly what causes this circular reference, but I do know the point of failure: I set the JDBC driver to null when trying to call (from an autowired Environment env.getRequiredProperty("jdbc.driver"). It's an NPE (on the env object); the property later gets initialized as I print it out in a PostConstructor.
My understanding of this issue is that since the SqlSessionFactoryBean is a Spring factory bean (does this make it of BFPP type?), that normal autowiring and bean instantiation cannot be assumed to be available. What further solidifies this point is that if I hard-code the JDBC details (or otherwise make the SqlSessionFactoryBean method static), then Jetty is able to start-up just fine and the application works as expected. Obviously hard-coding in the JDBC details is not desired here -- I reference these details in one other spot.
tl;dr
How can I statically access Spring's environment property values, say, in a Factory Bean method? I have tried an #autowired+postconstruct route and a implement-application-context-aware approach; both to no avail.
code samples
public abstract class AbstractMybatisConfig {
#Autowired
Environment env;
String jdbcDriver;
String jdbcUrl;
String jdbcUsername;
String jdbcPassword;
// if I try the below statement, the env object throws an NPE
// jdbcDriver = env.getRequiredProperty( "jdbc.driver" );
#PostConstruct
protected void postConstruct() {
jdbcDriver = env.getRequiredProperty( "jdbc.driver" );
// this statement prints out "com.mysql.jdbc.Driver" as expected
System.out.println( jdbcDriver );
jdbcUrl = env.getRequiredProperty( "jdbc.url" );
jdbcUsername = env.getRequiredProperty( "jdbc.username" );
jdbcPassword = env.getRequiredProperty( "jdbc.password" );
}
#Bean
public DataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName( jdbcDriver );
// this statement prints out null
System.out.println( jdbcDriver );
dataSource.setUrl( jdbcUrl );
dataSource.setUsername( jdbcUsername );
dataSource.setPassword( jdbcPassword );
return dataSource;
}
#Bean // circular reference happens here b/c dataSource() sets null properties
public SqlSessionFactoryBean sqlSessionFactory() {
SqlSessionFactoryBean sqlSessionFactory = new SqlSessionFactoryBean();
sqlSessionFactory.setDataSource( dataSource() );
// ...other (unrelated) configuration goes here...
return sqlSessionFactory;
}
private static getJdbcProperties() {
Properties jdbcProps = new Properties();
// when this method is invoked, Spring complains the resource does not exist
// (and it is the same String used to identify this property file in my PropertySources annotation found in another config)
jdbcProps.load( new ClassPathResource( "classpath:META-INF/properties/mybatis.properties" ).getInputStream() );
// ...code to grab property values omitted for brevity...
return jdbcProps
}