Problem Statement
I want to load properties from a properties file in a classpath or at an external location before the beans are initialized. These properties are also a part of Bean initialization. I cannot autowire the properties from Spring's standard application.properties or its customization because the same properties file must be accessible by multiple deployables.
What I Tried
I'm aware about Spring Application Events; in fact, I'm already hooking
ContextRefreshedEvent to perform some tasks after the Spring Context is initialized (Beans are also initialized at this stage).
For my problem statement, from the description of Spring Docs ApplicationEnvironmentPreparedEvent looked promising, but the hook did not work.
#SpringBootApplication
public class App {
public static void main(String[] args) throws IOException {
SpringApplication.run(App.class, args);
}
#EventListener
public void onStartUp(ContextRefreshedEvent event) {
System.out.println("ContextRefreshedEvent"); // WORKS
}
#EventListener
public void onShutDown(ContextClosedEvent event) {
System.out.println("ContextClosedEvent"); // WORKS
}
#EventListener
public void onEvent6(ApplicationStartedEvent event) {
System.out.println("ApplicationStartedEvent"); // WORKS BUT AFTER ContextRefreshedEvent
}
#EventListener
public void onEvent3(ApplicationReadyEvent event) {
System.out.println("ApplicationReadyEvent"); // WORKS WORKS BUT AFTER ContextRefreshedEvent
}
public void onEvent1(ApplicationEnvironmentPreparedEvent event) {
System.out.println("ApplicationEnvironmentPreparedEvent"); // DOESN'T WORK
}
#EventListener
public void onEvent2(ApplicationContextInitializedEvent event) {
System.out.println("ApplicationContextInitializedEvent"); // DOESN'T WORK
}
#EventListener
public void onEvent4(ApplicationContextInitializedEvent event) {
System.out.println("ApplicationContextInitializedEvent");
}
#EventListener
public void onEvent5(ContextStartedEvent event) {
System.out.println("ContextStartedEvent");
}
}
Update
As suggested by M.Deinum in the comments, I tried adding an application context initializer like below. It doesn't seem to be working either.
public static void main(String[] args) {
new SpringApplicationBuilder()
.sources(App.class)
.initializers(applicationContext -> {
System.out.println("INSIDE CUSTOM APPLICATION INITIALIZER");
})
.run(args);
}
Update #2
While my problem statement is regarding loading properties, my question/curiosity is really about how to run some code before the classes are initialized as beans and put into Spring IoC container. Now, these beans require some property values during initialization and I can't/don't want to Autowire them because of the following reason:
As stated in comments and answers, the same can be done using Spring Boot's externalized configuration and profiles. However, I need to maintain application properties and domain-related properties separately. A base domain properties should have at least 100 properties, and the number grows over time. Both application properties and domain-related properties have a property file for different environments (dev, SIT, UAT, Production). Property files override one or more of the base properties. That's 8 property files. Now, the same app needs to be deployed into multiple geographies. That makes it 8 * n property files where n is the number of geographies. I want all the property files stored in a common module so that they can be accessed by different deployables. Environment and geography would be known in run-time as system properties.
While these might be achieved by using Spring profiles and precedence order, I want to have a programmatic control over it (I also would maintain my own property repository). Eg. I would write a convenience utility called MyPropUtil and access them like:
public class MyPropUtil {
private static Map<String, Properties> repository;
public static initialize(..) {
....
}
public static String getDomainProperty(String key) {
return repository.get("domain").getProperty(key);
}
public static String getAppProperty(String key) {
return repository.get("app").getProperty(key);
}
public static String getAndAddBasePathToAppPropertyValue(String key) {
...
}
}
#Configuration
public class MyComponent {
#Bean
public SomeClass getSomeClassBean() {
SomeClass obj = new SomeClass();
obj.someProp1(MyPropUtil.getDomainProperty('domainkey1'));
obj.someProp2(MyPropUtil.getAppProperty('appkey1'));
// For some properties
obj.someProp2(MyPropUtil.getAndAddBasePathToAppPropertyValue('some.relative.path.value'));
....
return obj;
}
}
From the docs, it seems like ApplicationEvents and ApplicationInitializers fit my need, but I am not able to get them to work for my problem statement.
Bit late to the party but hopefully I can offer a solution to your updated problem statement.
This will focus on problem of how to run some code before the classes are initialized as beans and put into Spring IoC container
One issue I notice is that you're defining your application events via the #EventListener annotation.
These are only called once all beans are initiated since these annotations are processed by EventListenerMethodProcessor which is only triggered when the context is ready (see SmartInitializingSingleton#afterSingletonsInstantiated)
As such, some of the events that occur before the context is ready. e.g. ContextStartedEvent, ApplicationContextInitializedEvent won't make it to your listener.
Instead, what you can do is extend the interface for these events directly.
#Slf4j
public class AllEvent implements ApplicationListener<ApplicationEvent> {
#Override
public void onApplicationEvent(final ApplicationEvent event) {
log.info("I am a {}", event.getClass().getSimpleName());
}
Note the missing #Component. Even bean instantiation can occur after some of these events. If you use #Component, then you'll get the following logs
I am a DataSourceSchemaCreatedEvent
I am a ContextRefreshedEvent
I am a ServletWebServerInitializedEvent
I am a ApplicationStartedEvent
I am a ApplicationReadyEvent
Still better and more instant than the annotative listeners but will still not receive the initialization events. For that, what you need to do is follow the instructions found here
To summarize,
Create directory resources/META-INF
Create file spring.factories
org.springframework.context.ApplicationListener=full.path.to.my.class.AllEvent
The result:-
I am a ApplicationContextInitializedEvent
I am a ApplicationPreparedEvent
I am a DataSourceSchemaCreatedEvent
I am a ContextRefreshedEvent
I am a ServletWebServerInitializedEvent
I am a ApplicationStartedEvent
I am a ApplicationReadyEvent
In particular, ApplicationContextInitializedEvent should allow you to perform whatever per-instantiation tasks you need.
I think Spring Cloud Config is a perfect solution for your problem statement. Detailed documentation Here
Spring Cloud Config provides server-side and client-side support for externalized configuration in a distributed system.
So you can easily manage the configurations outside of the app, as well as all the instances will use same configurations.
Create a bean that will be a properties repository and inject it in other beans requiring properties.
In your example, instead of having static methods in MyPropUtil, make the class a bean itself with instance methods. Initialize Map<String, Properties> repository in the initialize method annotated with #PostConstruct.
#Component
public class MyPropUtil {
private static final String DOMAIN_KEY = "domain";
private static final String APP_KEY = "app";
private Map<String, Properties> repository;
#PostConstruct
public void init() {
Properties domainProps = new Properties();
//domainProps.load();
repository.put(DOMAIN_KEY, domainProps);
Properties appProps = new Properties();
//appProps.load();
repository.put(APP_KEY, appProps);
}
public String getDomainProperty(String key) {
return repository.get(DOMAIN_KEY).getProperty(key);
}
public String getAppProperty(String key) {
return repository.get(APP_KEY).getProperty(key);
}
public String getAndAddBasePathToAppPropertyValue(String key) {
//...
}
}
and
#Configuration
public class MyComponent {
#Autowired
private MyPropUtil myPropUtil;
#Bean
public SomeClass getSomeClassBean() {
SomeClass obj = new SomeClass();
obj.someProp1(myPropUtil.getDomainProperty("domainkey1"));
obj.someProp2(myPropUtil.getAppProperty("appkey1"));
// For some properties
obj.someProp2(myPropUtil.getAndAddBasePathToAppPropertyValue("some.relative.path.value"));
//...
return obj;
}
}
Or you can inject MyPropUtil directly to the SomeClass:
#Component
public class SomeClass {
private final String someProp1;
private final String someProp2;
#Autowired
public SomeClass(MyPropUtil myPropUtil) {
this.someProp1 = myPropUtil.getDomainProperty("domainkey1");
this.someProp2 = myPropUtil.getAppProperty("appkey1");
}
//...
}
I feel like your main issue is that you need to maintain application properties and domain-related properties separately.
From spring's perspective, it doesn't really matter since all properties files are kinda merged together after they have been loaded in memory.
So for example, you have two files that contain some properties:
application.related=property1 # this is in application.properties
domain.related=property2 # this is in domain-specific.properties
After they have been loaded, you will get one big thing that contains all properties, if I am not mistaken, it is a org.springframework.core.env.ConfigurableEnvironment instance.
Then what you need to do is just inject the property you need using something like #Value.
For the main issue, to separate properties into different files, you just need to specify spring's spring.config.name property (via environment variable, command line or programmatically). Following the above example, it should be spring.config.name=application,domain-specific.
Furthermore, if you really want to have programmatic control, you can add a custom EnvironmentPostProcessor which exposes the ConfigurableEnvironment instance.
As explaned in this post you can add external property files like this;
public PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
PropertySourcesPlaceholderConfigurer properties = new PropertySourcesPlaceholderConfigurer();
properties.setLocation(new FileSystemResource("/Users/home/conf.properties"));
properties.setIgnoreResourceNotFound(false);
return properties;
}
If you don't want to use this, just read the property file with jackson and set the properties to System.setProperty("key","value") in the main method before spring starts.
If you don't want to use this too, take a look at the BeanPostProcessor#postProcessBeforeInitialization method. It runs before bean properties initialized by spring.
I might be missing what exactly do you mean by "Beans initialization", probably an example of such a bean in a question could be beneficial.
I think you should differentiate between properties reading part and bean initialization.
By the time of bean initialization, properties are already read and available. Thats a part of spring magic, if you wish.
That's why the following code works for example:
#Component
public class MySampleBean {
public MySampleBean(#Value("${some.prop}" String someProp) {...}
}
It doesn't matter from where do these property come (spring boot defines many different ways of these places with precedence between them), it will happen before the initialization of beans happens.
Now, lets get back to your original question:
I want to load properties from a properties file in a classpath or at external location (before the beans are initialized - irrelevant).
In spring / spring-boot there is a concept of profiles that basically allows to create a file application-foo.properties (or yaml) and when you load with --spring.profiles.active=foo it will automatically load properties defined in this application-foo.properties in addition to the regular application.properties
So you can place the stuff that you want to "load from classpath" into application-local.properties (the word local is for the sake of example only) and start the application with --spring.profiles.active=local (in the deployment script, docker file or whatever)
If you want to run the property from external location (outside the classpath) you can use: --spring.config.location=<Full-path-file>
Note that even if you put some properties into a regular application.properties and still use --spring.config.location with the same key-value pairs they will take precedence over the properties in the classpath.
Alternatively you can use only --sring.profiles.active=local or remote and do not use config locations at all.
You can configure external location directly in the command line:
java -jar app.jar --spring.config.location=file:///Users/home/config/external.properties
You can use WebApplicationInitializer to execute code before classes are initialized as beans
public class MyWebInitializer implements WebApplicationInitializer {
#Override
public void onStartup(ServletContext servletContext) throws ServletException {
var ctx = new AnnotationConfigWebApplicationContext();
ctx.register(WebConfig.class);
ctx.setServletContext(servletContext);
We create an AnnotationConfigWebApplicationContext and register a web configuration file with register().
You can check if PropertySource may help you.
Example:
#PropertySource({"classpath:persistence/persistence.properties"})
You can use this annotation on every #Configuration or #SpringBootApplication bean
It sounds like you want to take some ownership of a part of the bean initialization. Typically people think of Spring completing the bean configuration, but in your case it might be easier to consider Spring as starting it.
So, your bean has some properties you want to configure, and some that you want Spring to configure. Just annotate the ones you want Spring to configure (with #Autowire or #Inject, or whatever flavour you prefer), and then take over the control from there, using #PostConstruct or InitializingBean.
class MyMultiStageBoosterRocket {
private Foo foo;
private Bar bar;
private Cat cat;
#Autowire
public MyMultiStageBoosterRocket(Foo foo, Bar bar) {
this.foo = foo;
this.bar = bar'
}
// called *after* Spring has done its injection, but *before* the bean
// is registered in the context
#PostConstruct
public void postConstruct() {
// your magic property injection from whatever source you happen to want
ServiceLoader<CatProvider> loader = ServiceLoader.load(CatProvider.class);
// etc...
}
}
Of course your mechanism for property resolution would need to be available statically somehow, but that seems to fit with you MyPropUtil example.
Getting far more involved, you start looking at Bean Post Processors directly (#PostConstruct is a simple variant of sorts).
There's a previous question, with a useful answer, here How exactly does the Spring BeanPostProcessor work?, but for simplicity, you'd do something like
public class CustomBeanPostProcessor implements BeanPostProcessor {
#Override
public Object postProcessBeforeInitialization(Object bean, String beanName)
throws BeansException {
// fixme: detect if this bean needs fancy initialization
return bean;
}
}
Clearly #PostProcess, or InitializingBean are simpler, but the custom post processor has a big advantage... it can be injected with other Spring managed beans. That means you can Spring manage your property injection whatever-thing, and still manually manage the actual injection process.
Just try to load everything you need in main before
SpringApplication.run()
call
public static void main(String[] args) {
// before spring initialization
TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
SpringApplication.run(CyberRiskApplication.class, args);
}
You can use ApplicationEnvironmentPreparedEvent but it can't be configured using EventListener annotation. Because by this time Bean drfinitions are not loaded. See the below link on how to cofigure this event.
https://www.thetechnojournals.com/2019/10/spring-boot-application-events.html
I don't know, how to approach a solution for the following scenario.
We have a new requirement to remove DB Password from properties even though it's encrypted with Jasypt library or some other algorithms.
Instead of storing the password in properties or LDAP, we need to fetch it dynamically from Cyberark.
Password may expire in a day or two or in a week or in a month. It totally depends on Password expiration policy.
We have multiple projects. Some are web-based and some are standalone. We want to write a generic solution.
How to override getConnection method of any data source like Spring data source, Apache Basic data source (it support extending class), C3P0, DBCP or HikariCP without impacting their behavior and setting the password before hitting super.getConnection()?
super.getConnection(); // Here max attempt will be 3
Spring supports method replacement, but I don't know what will be the impact on the connection pooling framework.
Let me know if you need more details.
To solve your problem you can use spring-cloud-context library and its #RefreshScope annotation. Also, it will be needed for you to develop a bit.
1) You need a special watcher bean which will monitor if the password was changed. It will be smth like this:
#Service
public class Watcher {
private final ContextRefresher refresher;
public Watcher(ContextRefresher refresher) {
this.refresher = refresher;
}
#Scheduled(fixedDelay = 10000L)
public void monitor() {
if (/* smth changed*/) {
refresher.refresh();
}
}
}
So, when you call refresher.refresh(); all beans annotated with #RefreshContext will be disposed and recreated after the first access to them.
2) Annotate your datasource bean with #RefreshContext annotation.
3) You have to provide password to be accessed using #ConfigurationProperties annotation. You will need to create SourceLocator. It will be smth like this
#Order(0)
public class SourceLocator implements PropertySourceLocator {
#Override
public PropertySource<?> locate(Environment environment) {
//Load properties to hash map
return new MapPropertySource("props", new HashMap<>());
}
}
Also, create a file spring.factories and put the following data there:
org.springframework.cloud.bootstrap.BootstrapConfiguration=\
com.test.YourSourceLocator
4) Create properties class where your db pass will be held and refreshed.
#RefreshScope
#ConfigurationProperties(prefix="your.prefix")
public class Properties {
private String dbPassword;
}
Autowire this bean to the configuration where you create your datasource and use password from it.
In an application ther are multiple properties file for managing exception messages , alerts , and some others text these file like this :
- core-message.properties
- databaseException.properties
......
in Service layer maybe a database call occure and the database return a key that exist in one the properties files , and i want get the value and raise the exception messsage to user interface layer .
if i know that the key in wich properties file the code will be like this :
#Value("#{core['theExceptionKey']}")
public String excpetionMessage;
private void myMethod() {
throw new ExceptionClass(exceptionMessage);
}
i think spring can do that because when i use spring:message tag in jsp files spring does not know the key in witch file but it load the message correctly.
You can use Spring Environment abstraction for that.
First you need to add Property Source to your Java Configuration file
#Configuration
#PropertySource("classpath:/com/mypacakge/core-message.properties")
public class AppConfig {
Or if you have multiple properties files
#Configuration
#PropertySources({
#PropertySource("classpath:core-message.properties"),
#PropertySource("classpath:database.properties")
})
public class AppConfig {
Add PropertySourceConfigurer to to your Java Configuration file
#Bean
public static PropertySourcesPlaceholderConfigurer placeHolderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
Now let's say that in your core-message.properties you have the following data
message.name=Hello
You can retrieve this data in any bean by autowiring Environment abstraction and then calling env.getProperty()
#Autowired
Environment env;
public void m1(){
String message = env.getProperty("message.name")` // will return Hello
Environment object provides interface to configure property sources and resolve properties. It provides convenience to read from a variety of sources: properties files, system environment variable, JVM system properties, servlet context parameters, and so on, which is very useful. For example :
environment.getSystemProperties().put("message", "Hello");
System.getProperties().put("message", "Hello");
environment.getSystemProperties().get("message"); // retrieve property
environment.getPropertySources() // allows manipulation of Properties objects
Spring Reference Documentation - Environment
To get the value of the key programmatically you can use the following:
#Autowired
private Environment env;
...
String something = env.getProperty("property.key.something");
I'm working on building a custom "event" library that encapsulates the technical details of an event buffer we are planning to share with multiple consumers. Ideally, we want this library to use the spring framework (note: not spring boot), and be environmentally aware. Something I am not groking from the current docs is how to make the library environmentally aware.
For example, we want to include with the library a static configuration for the queue end points the library will publish / consume from; however, we want to enable "overriding" these queues when in the development or integration environments. Ideally, I do not want to make multiple builds that swap out what the config file is, but include them all and know to read the "right" one.
Some of the things I am not understanding;
How to pass in a "profile" when debugging (it seems the Environment object won't honor the -Dspring.active.profiles property).
How to structure the #Configuration classes so that you do not hard code #Profile(prod).
Total Spring n00b, thanks in advance!
---UPDATE: Trying to provide a more concrete example.
So I have create a basic configuration class to hold the details that would be populated by configuration files:
#Configuration
public class EventConfiguration implements EnvironmentAware{
private static Environment env = null;
#Value("${events.queue1}")
private String queue1;
#Value("${events.queue2}")
private String queue2;
#Bean
public EventDispatcher eventDispatcher() {
return new EventDispatcher(this);
}
#Override
public void setEnvironment(Environment environment) {
env = environment;
}
... getters and setters
Essentially I want to either go the yaml approach and define the queues by environment "dev", "integration", "prod"; or have 3 different files following the application-{env}.properties convention.
Then, to help me understand how this works, I threw together a quick test so I can inspect the configuration / environment:
#Test
public void testContext() {
AnnotationConfigApplicationContext ctx =
new AnnotationConfigApplicationContext();
Environment env = ctx.getEnvironment();
ctx.scan("com...events");
ctx.refresh();
EventDispatcher dispatcher = ctx.getBean(EventDispatcher.class);
}
I started the debugger with a -Dspring.profiles.active=dev, after having created an application-dev.profile available on the class path.
Am I on the right track? Seems weird to have to have that type of boiler plate code to instantiate the objects, plus it didnt work. The Environment object only showed "default" as the active profile.
This relates to this answer:
System.getProperty("catalina.base") There can be scenario where client may use any other server
another server-independent system
property yourself, you can set as a VM
argument.
-Dconfig.location=/path/to/folder
In case of Tomcat, you can set it as
JAVA_OPTS environment variable, or
edit the catalina.bat startup file or
edit the Windows Service settings
(when it's installed as Windows
Service), etc. Other servers supports
similar constructs as well.
Is this considered 'clean'? We've been doing this for years, just want to know if this is acceptable, or there is a better way to configure runtime environment.
It feels maybe dirty, but there are apart from putting it in the classpath really no better ways if the requirement is to untouch the WAR whenever you want to change the location of external configuration files.
If untouching the WAR is not a strict requirement and rebuilding the WAR is allowed (e.g. you're using an inhouse developed application with continuous integration and serveradmins are in the same line, etc), then you could also use a <context-param> in web.xml instead.
<context-param>
<param-name>config.location<param-name>
<param-value>/path/to/file</param-value>
</context-param>
It's then in any Servlet (or better, ServletContextListener) available by ServletContext#getInitParameter():
String configLocation = servletContext.getInitParameter("config.location");
File configFile = new File(configLocation, "config.properties");
// ...
My understanding is that "more clean" would be using either <servlet-param> <init-param> in web.xml or some kind of IoC solution, like Spring.
I feel this is not the cleanest of ways to attain what you want. You can use the web.xml init params or servlet params tags.
Another way is using properties file or an XML configuration file.
I just solved a similar problem in a slightly different way. Our customer wants to configure database connection details, integration server locations and ports etc. without rebuilding the war. Using environment property to point an external file containing the information may or may not be okay, but it felt a bit dirty trick. Anyway, here's a slightly more enterprisey way.
For database connections we use JNDI lookup and below is the current solution for integration server parametrization. The parameters can come from at least three different sources now:
properties-file, which is overridable with Maven profiles and requires single line of xml in spring configuration to be accessible. This is obviously inside the war file.
web.xml context-param. This is also, of course, inside the war file.
Tomcat server can override the init parameters with context.xml which can be outside the war. This happens to be the same file where JNDI context is defined, which is nice.
Below is the implementation for configuration accessor bean. It can run in servlet context and also without one (for some unit tests it makes little sense to kickstart a full-blown web server, but we nevertheless need to satisfy spring bean injections).
I don't mean this to be a perfect solution, but it is one. Didn't find anything like this with google.
#Service
public class IntegrationConfigurationImpl implements
IntegrationConfiguration, InitializingBean,
ServletContextAware, ApplicationContextAware {
private static final String SERVER_HOST_PROPERTY = "integration.server.host";
private static final String SERVER_PORT_PROPERTY = "integration.server.port";
private static final String PROPERTY_BEAN_NAME = "integrationProperties";
private ServletContext servletContext;
private ApplicationContext applicationContext;
private static final Logger log = LoggerFactory.getLogger(IntegrationConfigurationImpl.class);
private String serverHost = "foo";
private int serverPort = 42;
#Override
public String getServerHost() {
return serverHost;
}
#Override
public int getServerPort() {
return serverPort;
}
#Override
public void setServletContext(ServletContext servletContext) {
this.servletContext = servletContext;
}
#Override
public void setApplicationContext(ApplicationContext applicationContext) {
this.applicationContext = applicationContext;
}
#Override
public void afterPropertiesSet() throws Exception {
// konfiguraation validointi..
if (servletContext == null) {
log.info("servlet context not set, not running as a web application. Trying to get properties from application context");
if (applicationContext.containsBean(PROPERTY_BEAN_NAME)) {
Properties p = (Properties)applicationContext.getBean(PROPERTY_BEAN_NAME);
serverHost = p.getProperty(SERVER_HOST_PROPERTY);
serverPort = Integer.valueOf(p.getProperty(SERVER_PORT_PROPERTY)).intValue();
} else {
log.info("Property bean not found :" + PROPERTY_BEAN_NAME);
}
} else {
serverHost = servletContext.getInitParameter(SERVER_HOST_PROPERTY);
serverPort = Integer.valueOf(servletContext.getInitParameter(SERVER_PORT_PROPERTY)).intValue();
}
log.info("Using integration server " + getServerHost() + ", port " + getServerPort());
}
}
The disadvantage with having system property is you need to restart the container to modify the system parameter.
Having it as init-param in web.xml, can allow you to modify by just restarting the web app.
Having in init-param is a better way.