Develop Reference

Post-processing YAML properties in Spring based on prefix to retrieve property from REST service

I have a Spring boot configuration YAML with something like
spring:
application:
name: my-app
a: a literal
b: <<external due to special first and last chars>>
What i'm trying to do is to add some kind of resolver that will detect that the value of b is of the form <<X>> and will trigger retrieving that value from an external rest api to overwrite in memory the value that was in the YAML before it gets passed to the bean that holds the configurations at runtime
I tried and failed using an EnvironmentPostProcessor because I can't get ahold of the actual property values, just the property sources, so I can't post-process the values.
What currently works for me is in the #Configuration bean that holds the fields a and b, implement something in the setters to detect if the value that spring is trying to set starts with << and ends with >> and if so, overwrite what gets loaded into the pojo with the version that i retrieve from the rest api. This is not ideal because I end up with a lot of duplication
What's the right way to implement something like this in Spring 5? I know spring properties support references to other properties using the syntax ${a} so there must be some mechanism that already allows to add custom placeholder resolvers

I ended up changing things a bit to mark the special properties. Then I created my own PropertySource kind of like #Andreas suggested. It was all inspired by org.springframework.boot.env.RandomValuePropertySource
The trick was changing the special chars << and >> to the syntax used already by spring: ${}, but like the random resolver which uses ${random.int} I did something like ${rest.XXX}. What I didn't know before is that by doing that, Spring will invoke all the property sources a second time with a new property name coming from the placeholder value (rest.XXX in my previous example). This way in the property source I can handle the value if the name of the property starts with my prefix rest.
Here is a simplified version of my solution
public class MyPropertySource extends PropertySource<RestTemplate> {
private static final String PREFIX = "rest.";
public MyPropertySource() {
super(MyPropertySource.class.getSimpleName());
}
#Override
public Object getProperty(#Nonnull String name) {
String result = null;
if (name.startsWith(PREFIX)) {
result = getValueFromRest(name.substring(PREFIX.length()));
}
return result;
}
}
Finally, to register the property source I used an EnvironmentPostProcessor as described here. I couldn't find a simpler way that doesn't entail maintaining a new file META-INF/spring.factories

Don't know about right way, but one way to get properties from REST call is to implement your own PropertySource, that gets (and caches?) the values of specifically named properties.

Here is an hacky solution I came up with using Spring Boot 2.1.5.
Probably better to use a custom PropertyResolver
Essentially it goes like:
Grab the PropertySource I care about. For this case it's application.properties. Applications can have N number of sources so if there are other places where << >> can occur, then you'll to check them as well.
Loop through the source's values for << >>
Dynamically replace the value if match.
My properties are:
a=hello from a
b=<<I need special attention>>
My hacked ApplicationListener is:
import org.apache.commons.lang3.StringUtils;
import org.springframework.boot.context.event.ApplicationEnvironmentPreparedEvent;
import org.springframework.boot.env.OriginTrackedMapPropertySource;
import org.springframework.context.ApplicationListener;
import org.springframework.core.Ordered;
import org.springframework.core.env.MapPropertySource;
import org.springframework.core.env.PropertySource;
import org.springframework.web.client.RestTemplate;
import java.util.HashMap;
import java.util.Map;
public class EnvironmentPrepareListener implements ApplicationListener<ApplicationEnvironmentPreparedEvent>, Ordered {
private final RestTemplate restTemplate = new RestTemplate();
#Override
public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event) {
// Only focused main application.properties (or yml) configuration
// Loop through sources to figure out name
final String propertySourceName = "applicationConfig: [classpath:/application.properties]";
PropertySource<?> propertySource = event.getEnvironment().getPropertySources()
.get(propertySourceName);
Map<String, Object> source = ((OriginTrackedMapPropertySource) propertySource).getSource();
Map<String, Object> myUpdatedProps = new HashMap<>();
final String url = "https://jsonplaceholder.typicode.com/todos/1";
for (Map.Entry<String, Object> entry : source.entrySet()) {
if (isDynamic(entry.getValue())) {
String updatedValue = restTemplate.getForEntity(url, String.class).getBody();
myUpdatedProps.put(entry.getKey(), updatedValue);
}
}
if (!myUpdatedProps.isEmpty()) {
event.getEnvironment().getPropertySources()
.addBefore(
propertySourceName,
new MapPropertySource("myUpdatedProps", myUpdatedProps)
);
}
}
private boolean isDynamic(Object value) {
return StringUtils.startsWith(value.toString(), "<<")
&& StringUtils.endsWith(value.toString(), ">>");
}
#Override
public int getOrder() {
return Ordered.LOWEST_PRECEDENCE;
}
}
Hitting /test yields me:
{ "userId": 1, "id": 1, "title": "delectus aut autem", "completed": false }

Spring changing the properties file in runtime

I'm using Spring Boot and I have a properties file p.properties:
p1 = some val1
p2 = some val2
Configuration class:
#Configuration
#PropertySource("classpath:p.properties")
public class myProperties {
public myProperties () {
super();
}
#Bean
public static PropertySourcesPlaceholderConfigurer placeholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
}
And I'm using this in order to access the property:
#Value("${p1}")
private String mProperty;
Everything works great.
I want to change p1 in p.properties file from outside of the app and the next time that I'll use mProperty, it will contains the new value without restarting the app.
Is it possible?
Thanks,
Avi

You can simply use spring boot actuator.
Just add the actuator dependency in your maven/gradle config and you should be seeing live reloads when you update the property file.
Note: You won't have to restart the app but actuator will do live reloads on its own.

If you want to change the properties at runtime and don't want to restart the server then follow the below steps:
Application.properties
app.name= xyz
management.endpoints.web.exposure.include=*
Add below dependencies in pom.xml
org.springframework.boot
spring-boot-starter-actuator
org.springframework.cloud
spring-cloud-context
2.0.1.RELEASE
3)Place application.properties in config folder . The config folder must be in the location from where you will run the jar.
Add ApplcationProperties.java
#RefreshScope
#Component
#ConfigurationProperties(prefix = "app")
public class ApplcationProperties {
private String name;
//getter and setter
}
Write ApplicationController.java and inject ApplcationProperties
#Autowired
ApplcationProperties applcationProperties;
#RestController
public Class ApplicationController
{
#GetMapping("/find")
String getValue()
{
return applicationProperties.getName();
}
}
Run the spring boot application
Call localhost:XXXX/find from your browser
Output : xyz
Change the value in application.properties from xyz to abc
Using postman send a put /options request to localhost:XXXX/actuator/refresh
--Note this request should be either PUT/OPTIONS
Call localhost:XXXX/find from your browser
Output : abc

I think, in this case, it is advisable to keep it in the database so that, it can be changed & accessed seamlessly. We have a similar scenario where we keep the encrypted password for database in the properties file. While connecting to db, it needs to be decrypted. We do that by extending PropertyPlaceholderConfigurer as follows.
public class MyPropertyConfigurer extends PropertyPlaceholderConfigurer{
protected void convertProperties(Properties props){
Enumeration<?> propertyNames = props.propertyNames();
while (propertyNames.hasMoreElements()) {
String propertyName = (String) propertyNames.nextElement();
String propertyValue = props.getProperty(propertyName);
if(propertyName.indexOf("db.password") != -1){
decryptAndSetPropValue(props,propertyName,propertyValue);
}
}
}
}
But, this is done only once while loading the properties file.

Can I manually load #ConfigurationProperties without the Spring AppContext?

Is there any way to load a class marked with #ConfigurationProperties without using a Spring Context directly? Basically I want to reuse all the smart logic that Spring does but for a bean I manually instantiate outside of the Spring lifecycle.
I have a bean that loads happily in Spring (Boot) and I can inject this into my other Service beans:
#ConfigurationProperties(prefix="my")
public class MySettings {
String property1;
File property2;
}
See the spring docco for more info http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#boot-features-external-config-command-line-args
But now I need to access this bean from a class that is created outside of Spring (by Hibernate). The class is created so early in the app startup process that Spring Boot has not yet made the application context available through the classic lookup helper methods or roll-my-own static references.
So I instead want to do something like:
MySettings mySettings = new MySettings();
SpringPropertyLoadingMagicClass loader = new SpringPropertyLoadingMagicClass();
loader.populatePropertyValues(mySettings);
And have MySettings end up with all its values loaded, from the command line, system properties, app.properties, etc. Is there some class in Spring that does something like this or is it all too interwoven with the application context?
Obviously I could just load the Properties file myself, but I really want to keep Spring Boot's logic around using command line variables (e.g. --my.property1=xxx), or system variables, or application.properties or even a yaml file, as well as its logic around relaxed binding and type conversion (e.g. property2 is a File) so that it all works exactly the same as when used in the Spring context.
Possible or pipe dream?
Thanks for your help!

I had the same "issue".
Here is how I solved it in SpringBoot version 1.3.xxx and 1.4.1.
Let's say we have the following yaml configuration file:
foo:
apis:
-
name: Happy Api
path: /happyApi.json?v=bar
-
name: Grumpy Api
path: /grumpyApi.json?v=grrr
and we have the following ConfigurationProperties:
#ConfigurationProperties(prefix = "foo")
public class ApisProperties {
private List<ApiPath> apis = Lists.newArrayList();
public ApisProperties() {
}
public List<ApiPath> getApis() {
return apis;
}
public static class ApiPath {
private String name;
private String path;
public String getName() {
return name;
}
public void setName(final String aName) {
name = aName;
}
public String getPath() {
return path;
}
public void setPath(final String aPath) {
path = aPath;
}
}
}
Then, to do the "magic" things of Spring Boot programmatically (e.g. loading some properties in a static method), you can do:
private static ApisProperties apiProperties() {
try {
ClassPathResource resource;
resource = new ClassPathResource("/config/application.yml");
YamlPropertiesFactoryBean factoryBean;
factoryBean = new YamlPropertiesFactoryBean();
factoryBean.setSingleton(true); // optional depends on your use-case
factoryBean.setResources(resource);
Properties properties;
properties = factoryBean.getObject();
MutablePropertySources propertySources;
propertySources = new MutablePropertySources();
propertySources.addLast(new PropertiesPropertySource("apis", properties));
ApisProperties apisProperties;
apisProperties = new ApisProperties();
PropertiesConfigurationFactory<ApisProperties> configurationFactory;
configurationFactory = new PropertiesConfigurationFactory<>(apisProperties);
configurationFactory.setPropertySources(propertySources);
configurationFactory.setTargetName("foo"); // it's the same prefix as the one defined in the #ConfigurationProperties
configurationFactory.bindPropertiesToTarget();
return apisProperties; // apiProperties are fed with the values defined in the application.yaml
} catch (BindException e) {
throw new IllegalArgumentException(e);
}
}

Here's an update to ctranxuan's answer for Spring Boot 2.x. In our situation, we avoid spinning up a Spring context for unit tests, but do like to test our configuration classes (which is called AppConfig in this example, and its settings are prefixed by app):
public class AppConfigTest {
private static AppConfig config;
#BeforeClass
public static void init() {
YamlPropertiesFactoryBean factoryBean = new YamlPropertiesFactoryBean();
factoryBean.setResources(new ClassPathResource("application.yaml"));
Properties properties = factoryBean.getObject();
ConfigurationPropertySource propertySource = new MapConfigurationPropertySource(properties);
Binder binder = new Binder(propertySource);
config = binder.bind("app", AppConfig.class).get(); // same prefix as #ConfigurationProperties
}
}

The "magic" class you are looking for is PropertiesConfigurationFactory. But I would question your need for it - if you only need to bind once, then Spring should be able to do it for you, and if you have lifecycle issues it would be better to address those (in case they break something else).

This post is going into similar direction but extends the last answer with also validation and property placeholder resolutions.
Spring Boot Binder API support for #Value Annotations
#Value annotations in ConfigurationPropertys don't seem to bind properly though (at least if the referenced values are not part of the ConfigurationProperty's prefix namespace).

import org.springframework.boot.context.properties.bind.Binder
val binder = Binder.get(environment)
binder.bind(prefix, MySettings.class).get

Spring #Cacheable doesn't cache public methods

I have following method;
#Cacheable(value = "psgSiteToMap", key = "'P2M_'.concat(#siteName)")
public Map getSiteDetail(String siteName) {
Map map = new HashMap();
.....
//construct map variable here
.......
return map;
}
While project startup, cannot autowire class this method belongs to. If i change above method as following;
#Cacheable(value = "psgSiteToMap", key = "'P2M_'.concat(#siteName)")
private Map getSiteDetail(String siteName) {
Map map = new HashMap();
.....
//construct map variable here
................
return map;
}
public Map getSiteDetailPublic(String siteName) {
return this.getSiteDetail(siteName);
}
it works. Is there any restriction on #Cacheable annotation for public methods?
Thanks in advance

Spring AOP works only on public methods by default. You'd need AspectJ and load time or compile time weaving to make it work on private methods.
So it works in your case means that when you move the #Cacheable to the private method the proxy is not created at all and that works is autowireing, but not caching.
You probably have not set proxy-target-class property in your XML configuration or its equivalent annotation attribute. Can you please add the Spring configuration you're using and the class definition line. I'm interested if it implements any interfaces? Than I'll expand my answer with more details.

Dynamically accessing properties using Spring 3.1's property abstraction

I'm trying to dynamically access properties from Spring's Environment property abstraction.
I declare my property files like this:
<context:property-placeholder
location="classpath:server.common.properties,
classpath:server.${my-environment}.properties" />
In my property file server.test.properties, I define the following:
myKey=foo
Then, given the following code:
#Component
public class PropertyTest {
#Value("${myKey}")
private String propertyValue;
#Autowired
private PropertyResolver propertyResolver;
public function test() {
String fromResolver = propertyResolver.getProperty("myKey");
}
}
When I run this code, I end up with propertyValue='foo', but fromResolver=null;
Receiving propertyValue indicates that the properties are being read, (and I know this from other parts of my code). However, attempting to look them up dynamically is failing.
Why? How can I dynamically look up property values, without having to use #Value?

Simply adding a <context:property-placeholder/> doesn't add a new PropertySource to the Environment. If you read the article you linked completely, you'll see it suggests registering an ApplicationContextInitializer in order to add new PropertySources so they'll be available in the way you're trying to use them.

To get this to work I had to split out the reading of the properties into a #Configuration bean, as shown here.
Here's the complete example:
#Configuration
#PropertySource("classpath:/server.${env}.properties")
public class AngularEnvironmentModuleConfiguration {
private static final String PROPERTY_LIST_NAME = "angular.environment.properties";
#Autowired
private Environment environment;
#Bean(name="angularEnvironmentProperties")
public Map<String,String> getAngularEnvironmentProperties()
{
String propertiesToInclude = environment.getProperty(PROPERTY_LIST_NAME, "");
String[] propertyNames = StringUtils.split(propertiesToInclude, ",");
Map<String,String> properties = Maps.newHashMap();
for (String propertyName : propertyNames)
{
String propertyValue = environment.getProperty(propertyName);
properties.put(propertyName, propertyValue);
}
return properties;
}
}
The set of properties are then injected elsewhere, to be consumed.