Read qualified yaml file with prefix without using annotations or xml - java

My use case is a bit oddball but basically, I'd like to read a portion of a yaml file and map it to the appropriate java object in a spring application. This is a pretty common and trivial operation in spring (just use #ConfigurationProperties ).
However, in my case, I'd like to accomplish this reading earlier in the lifecycle i.e. by the time the BeanFactoryPostProcessor hooks in - in order to use the instructions specified in yml to dynamically create a number of beans.
I can get this working with application.properties but not with application.yml
I was hoping to use yml in order leverage mapping part of the yml to POJO and also utilize hierarchical mapping files and data structures (lists, maps etc).
Here's an example of how to read application.properties. https://blog.pchudzik.com/201705/dynamic-beans/
I set up a simple skeleton project at https://github.com/balamuru/yaml-loader to try out different techniques.
Any ideas ?
#Component
#EnableConfigurationProperties(SampleDataConfig.class)
class ConfigurableBeanFactory implements BeanFactoryPostProcessor, InitializingBean {
private List<String> beanInstances = new ArrayList<>();
#Override
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
final BeanDefinitionRegistry registry = (BeanDefinitionRegistry) beanFactory;
Map<String, SampleDataConfig> beans = beanFactory.getBeansOfType(SampleDataConfig.class);
System.err.println("");
beanInstances.forEach(instance -> {
registry.registerBeanDefinition(instance, BeanDefinitionBuilder
.rootBeanDefinition(SampleDataConfig.class)
.addConstructorArgValue(instance)
.getBeanDefinition());
});
}
#Override
public void afterPropertiesSet() throws Exception {
// this.beanInstances = asList(PropertiesLoaderUtils
// .loadProperties(new ClassPathResource("/application.properties"))
// .getProperty("dynamic-beans.instances", "")
// .split(","));
/**
* Rather than reading from application.properties,
* I would like to be able to load up the relevant prefix qualified segments (com.foo.bar.stuff) mapping to my POJO (SampleDataConfig,class)
* loaded from application.yml
*/
}
}
Internally, spring uses the following mechanism, but I was hoping there is an easier way to leverage this without re-inventing the spring :)
public class ConfigurationPropertiesBindingPostProcessor ...{
.
.
private void postProcessBeforeInitialization(Object bean, String beanName,
ConfigurationProperties annotation) {
Object target = bean;
PropertiesConfigurationFactory<Object> factory = new PropertiesConfigurationFactory<Object>(
target);
factory.setPropertySources(this.propertySources);
factory.setValidator(determineValidator(bean));
// If no explicit conversion service is provided we add one so that (at least)
// comma-separated arrays of convertibles can be bound automatically
factory.setConversionService(this.conversionService == null
? getDefaultConversionService() : this.conversionService);
if (annotation != null) {
factory.setIgnoreInvalidFields(annotation.ignoreInvalidFields());
factory.setIgnoreUnknownFields(annotation.ignoreUnknownFields());
factory.setExceptionIfInvalid(annotation.exceptionIfInvalid());
factory.setIgnoreNestedProperties(annotation.ignoreNestedProperties());
if (StringUtils.hasLength(annotation.prefix())) {
factory.setTargetName(annotation.prefix()); //====> use annotation prefix
}
}
try {
factory.bindPropertiesToTarget(); //===> bind properties
}
Thanks

YamlPropertiesFactoryBean yaml = new YamlPropertiesFactoryBean();
yaml.setResources(new ClassPathResource("application.yml"));
configProperty = yaml.getObject();
Set<Object> keys = configProperty.keySet();
Below is my YAML configuration, which looks like:
template:
config:
broker-urls:
- tcp://127.0.0.1:61616
- tcp://127.0.0.1:61617
- tcp://127.0.0.1:61618
qeues:
- Test
- Demo
- Qeue3
After applying above code you will get converted properties like below:
template.config.broker-urls[0]=tcp://127.0.0.1:61616
template.config.broker-urls[1]=tcp://127.0.0.1:61617
template.config.broker-urls[1]=tcp://127.0.0.1:61618
template.config.qeues[0]=Test
template.config.qeues[1]=Demo
template.config.qeues[1]=Qeue3

Related

Is it possible to register all classes within a package as Spring beans

I'm familiar with Springs Java based configuration options, including the usage of #Component and #Configuration in conjunction with #Bean annotations to register Spring beans.
However, when converting a decent size project to Spring, it can be very labor intensive to systematically touch all classes in the project and update with #Configuration #Beans or annotating each class with #Component. We have a large Groovy project to be converted and I would like to simplify the process.
My question: Is there a facility provided in Spring that allows you to tell Spring to auto-configure all valid bean candidate classes within a specific package?
If not, what other options are available?
ClassPathBeanDefinitionScanner is all you need.
public class Main {
public static void main(String[] args) {
GenericApplicationContext context = new GenericApplicationContext();
ClassPathBeanDefinitionScanner scanner = new ClassPathBeanDefinitionScanner(context, false);
scanner.addIncludeFilter((metadataReader, metadataReaderFactory) -> true);
scanner.scan("net.company.name");
context.refresh();
A a = context.getBean(A.class);
System.out.println(a.toString());
}
}
You can pass custom logic in include filter if you want. In current version every class in the provided package will be included as a bean.
But it is impossible to build a right dependency structure on your classes automagically, it really depends on the scope you want. You need to do it by your hands.
I'd do pretty much the same thing that Roman did, only I'd do it at build time, not at runtime, using code generation. The rationale here is that I strongly prefer magic to happen at build time to magic that happens at deploy time.
In the simplest version, write a main method that scans the package (instead of reflections api, I'm using Guava's ClassPath scanner) and creates a #Bean method for every class it finds.
For the Code generation, I'd use JCodeModel:
public class PackageBeanGenerator {
public static void main(String[] args) throws Exception {
String packageName = args[0];
JCodeModel codeModel = new JCodeModel();
// create class definition
JDefinedClass springConfig = codeModel._package(packageName)._class("SpringConfig");
springConfig.annotate(Configuration.class);
for (ClassPath.ClassInfo classInfo : ClassPath.from(
PackageBeanGenerator.class.getClassLoader()
).getTopLevelClasses(packageName)) {
Class<?> type = classInfo.load();
String beanName = CaseFormat.UPPER_CAMEL.to(
CaseFormat.LOWER_CAMEL,
type.getSimpleName());
JMethod beanMethod = springConfig.method(JMod.PUBLIC, type, beanName);
beanMethod.annotate(Bean.class);
beanMethod.body()._return(JExpr._new(codeModel._ref(type)));
}
// write class to file
codeModel.build(new File("/path/to/output/folder"));
}
}
You can try to use your own BeanDefinitionRegistryPostProcessor
#Component
public class CustomBeanFactoryPostProcessor implements BeanDefinitionRegistryPostProcessor {
#Override
public void postProcessBeanDefinitionRegistry(BeanDefinitionRegistry registry) throws BeansException {
Reflections reflections = new Reflections("my.package.prefix", new SubTypesScanner(false));
Set<Class<? extends Object>> allClasses = reflections.getSubTypesOf(Object.class);
for (Class clazz : allClasses) {
GenericBeanDefinition gbd = new GenericBeanDefinition();
gbd.setBeanClass(clazz);
gbd.setAttribute("attributeName", "attributeValue");
registry.registerBeanDefinition(clazz.getSimpleName() + "_Bean", gbd);
}
}
#Override
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
// Custom post process the existing bean definitions
}
}
See sample project at https://github.com/sandarkin/so-q37548350
At the risk of sounding primitive, why not just do a simple find and replace in your IDE (e.g. search for "public class" in a package and replace with "#Component public class") ? That should be much quicker than trying to do anything programatically.

Google Guice Properties Management

I would like to create a proper properties management strategy in a java webapp that relays on google guice as a DI framework.
I would like to have a mechanism answering the following 3 requirements:
I would like to be able to inject properties using guice (#Named)
I would like to be able to access properties in a static way
The mechanism should support prioritization of properties, meaning that a property can be wrapped in the deployed war with a certain value but it can also be redundant in the target system level or local file system (of the target machine I deploy on), in such a case the value in the war will be overridden by the value that exists in the target machine.
I believe this is a standard requirement. Now, using guice standard binder I can easily get the first requirement but not the other two. To get the other two I created my own class that does the following:
Wraps and exposes the binding methods of guice (those that binds properties) For example:
public static void bindString(AnnotatedBindingBuilder<String> binder, String property, String defaultValue) {
binder.annotatedWith(Names.named(property)).toInstance(getProperty(property, defaultValue));
}
Where the getProperty method knows how to handle my properties (get the value from the war or system level) and exposes the properties statically as well.
So basically as long as I'm using this utility that I created for properties bindings I'm good, it covers all my requirements but once I use the standard guice bindings I'm losing the second and third requirement.
Is there a way to override guice bindings and get all those 3 requirements?
Once I had the same challange in a spring based app and was pretty easy. I implemented ApplicationContextInitializer with the following method:
#Override
public void initialize(ConfigurableWebApplicationContext ctx) {
PropertySource<Map<String, Object>> localProps = null;
try {
localProps = new ResourcePropertySource(new ClassPathResource(LOCAL_PROPERTIES_FILE_NAME));
} catch (IOException e) {
LOG.fatal("Could not load local properties from classpath " + LOCAL_PROPERTIES_FILE_NAME);
return;
}
LOG.info("Loaded configuration from classpath local file " + LOCAL_PROPERTIES_FILE_NAME);
ctx.getEnvironment().getPropertySources().addFirst(localProps);
}
so this gave me a way to add local properties with highest priority to my Environment. In case of overlap with war properties the local ones had higher priority. In addition I exposed my Environment statically so I has static access to my properties (for services that are not managed by the container, legacy mostly).
How can I achieve this with guice?
Unfortunately, I don't think that you are going to find anything that gives you a truly clean and satisfying implementation. Especially, I don't think that you will find anything that gives you exactly what you want without implementing at least portions of it yourself.
If I had those needs, I would make sure that my injector is created in a central InjectorFactory. If you require a large number of parameters from outside to create your injector, I would simply create it once at the very beginning of my application and then cache the injector into a static final field. This would make it available to a static method. I would bind my "fall-back" property loading to an explicit provider. That way, instead of using the standard Names.bindProperties(...) method, I would bind it directly to a Provider. This provider then implements the logic that is necessary to perform the fallback or to merge multiple property files. Having the injector cached to a static field means that I can call a static method to access properties from a global-context outside of my injected classes.
Using your own provider seems initially unpleasant, but can provide some additional benefits. For starters, you can implement your fallback strategy exactly how you want. Additionally, you can add additional behaviors such as auto-reloading your property files, etc (not shown in my code sample).
public class InjectorFactory {
private static Injector injector = null;
public static synchronized Injector getOrCreateInjector() {
if(injector == null) {
injector = Guice.createInjector(new AbstractModule() {
#Override
protected void configure() {
Properties properties1 = createProperties("file1.properties");
Properties properties2 = createProperties("file2.properties");
Set<Object> propertyNames = new HashSet<Object>();
propertyNames.addAll(properties1.keySet());
propertyNames.addAll(properties2.keySet());
for (Object object : propertyNames) {
String propertyName = (String) object;
bind(String.class).annotatedWith(Names.named(propertyName)).toProvider(new StringProvider(properties1, properties2, propertyName));
}
}
private Properties createProperties(String propertyFileName) {
try {
InputStream stream = InjectorFactory.class.getResourceAsStream(propertyFileName);
try {
Properties properties = new Properties();
properties.load(stream);
return properties;
} finally {
stream.close();
}
} catch (IOException exception) {
throw new RuntimeException("Could not load properties file");
}
}
});
}
return injector;
}
public static String getProperty(String propertyName) {
return getOrCreateInjector().getInstance(Key.get(String.class, Names.named(propertyName)));
}
}
Given the above code and file1.properties:
property1=Property1Value
property2=Property2Value
And file.properties:
property2=IncorrectProperty2Value
property3=Property3Value
with the provider
public class StringProvider implements Provider<String> {
private Properties properties1;
private Properties properties2;
private String propertyName;
public StringProvider(Properties properties1, Properties properties2,
String propertyName) {
this.properties1 = properties1;
this.properties2 = properties2;
this.propertyName = propertyName;
}
public String get() {
if(properties1.containsKey(propertyName)) {
return properties1.getProperty(propertyName);
}
return properties2.getProperty(propertyName);
}
}
The following usage:
public class InjectorFactoryTest {
public static void main(String ... parameters) {
System.out.println(InjectorFactory.getProperty("property1"));
System.out.println(InjectorFactory.getProperty("property2"));
System.out.println(InjectorFactory.getProperty("property3"));
}
}
Outputs:
Property1Value
Property2Value
Property3Value

Is it possible to use the Multibinder within Guice to construct dynamic sets based on application configuration?

I have an application which relies on Properties configuration to determine whether to mix in various components or not.
For example, the configuration has boolean flags like "componentX.enabled" etc which determine whether these components should be active or not.
Currently I am using these flags in my provider methods like so:
#Provides
#Singleton
#Nullable
public ComponentX provideComponentX(Properties props) {
if (props.isComponentXEnabled()) {
return new ComponentX();
} else {
return null;
}
}
#Provides
#Singleton
public Set<Component> provideComponentSet(
#Nullable ComponentX compX,
ComponentY compY,
ComponentZ compZ
) {
Set<Component> comps = new HashSet<>();
if (compX != null) {
comps.add(compX);
}
comps.add(compY);
comps.add(compZ);
return comps;
}
This approach seems a little clunky (it relies on possible injecting null)- but is there a better way?
The only other way I can think of doing it is by using a parent injector to obtain the application Properties into my module, and then using the set Multibinder.
Then use the create child injector with the new module to complete the bootstrap process.
public class Module extends AbstractModule {
Properties props;
public Module(Properties props) {
this.props = props;
}
public void configure() {
Multibinder<Component> compBinder = Multibinder.newSetBinder(binder(), Component.class);
if (props.isComponentXEnabled()) {
compBinder.addBinding().to(ComponentX.class);
}
compBinder.addBinding().to(ComponentY.class);
compBinder.addBinding().to(ComponentZ.class);
}
}
This also seems a little clunky because it requires the use of a child injector etc.
Again, is there a better way?
Maybe I could use Netflix's Governator (https://github.com/Netflix/governator/wiki/Configuration-Mapping) to inject Configuration values into my module (not sure if that is possible or not)?
How do other people approach this problem?
The applications I've been working with recently have a properties file (or other configuration) that is used to decide which parts of the application are relevant. Our typical approach is parse those properties immediately (just to a Properties object) and construct the application module(s) from that, and they will then conditionally include other modules based on the values specified.
In a couple of places, this has grown into an "init parameters" set, with an enumeration of possible parameters:
enum InitParam {
PricesQueue("prices.queue")
}
Each enum instance is related to a property key and there is a method to get a basic string value for each parameter from Properties:
boolean presentIn(Properties props) { return props.containsKey(propertyKey); }
String valueIn(Properties props) { return props.getProperty(propertyKey); }
So this can be used like so:
public AppModule extends AbstractModule {
private final Properties config;
protected void configure() {
if (InitParam.PricesQueue.presentIn(config)) {
install(new PricesQueueConsumerModule(config));
}
}
}
Additionally, there is a module to bind all the values in the config properties to String, Optional<String> etc, allowing:
#Inject
public PricesQueueConsumer(#FromInitParam(InitParam.PricesQueue) String queueName) {
}
This will trap the queue consumer being referenced when the configuration isn't available (the module won't bind a string if the value isn't present in the config file) while still allowing the behaviour for when the value isn't present to be deferred to later (by injecting Optional<String> instead)
So this is somewhat similar to your second approach, except that I'd not considered the using-Guice-to-inject-Guice-modules approach, which seems a bit convoluted. Although probably it's essentially the same. Maybe rather than a parent/child injector you could simply create a "bootstrapping" injector to build your top-level application module, and then use that to build a completely separate injector?

Can I manually load #ConfigurationProperties without the Spring AppContext?

Is there any way to load a class marked with #ConfigurationProperties without using a Spring Context directly? Basically I want to reuse all the smart logic that Spring does but for a bean I manually instantiate outside of the Spring lifecycle.
I have a bean that loads happily in Spring (Boot) and I can inject this into my other Service beans:
#ConfigurationProperties(prefix="my")
public class MySettings {
String property1;
File property2;
}
See the spring docco for more info http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#boot-features-external-config-command-line-args
But now I need to access this bean from a class that is created outside of Spring (by Hibernate). The class is created so early in the app startup process that Spring Boot has not yet made the application context available through the classic lookup helper methods or roll-my-own static references.
So I instead want to do something like:
MySettings mySettings = new MySettings();
SpringPropertyLoadingMagicClass loader = new SpringPropertyLoadingMagicClass();
loader.populatePropertyValues(mySettings);
And have MySettings end up with all its values loaded, from the command line, system properties, app.properties, etc. Is there some class in Spring that does something like this or is it all too interwoven with the application context?
Obviously I could just load the Properties file myself, but I really want to keep Spring Boot's logic around using command line variables (e.g. --my.property1=xxx), or system variables, or application.properties or even a yaml file, as well as its logic around relaxed binding and type conversion (e.g. property2 is a File) so that it all works exactly the same as when used in the Spring context.
Possible or pipe dream?
Thanks for your help!
I had the same "issue".
Here is how I solved it in SpringBoot version 1.3.xxx and 1.4.1.
Let's say we have the following yaml configuration file:
foo:
apis:
-
name: Happy Api
path: /happyApi.json?v=bar
-
name: Grumpy Api
path: /grumpyApi.json?v=grrr
and we have the following ConfigurationProperties:
#ConfigurationProperties(prefix = "foo")
public class ApisProperties {
private List<ApiPath> apis = Lists.newArrayList();
public ApisProperties() {
}
public List<ApiPath> getApis() {
return apis;
}
public static class ApiPath {
private String name;
private String path;
public String getName() {
return name;
}
public void setName(final String aName) {
name = aName;
}
public String getPath() {
return path;
}
public void setPath(final String aPath) {
path = aPath;
}
}
}
Then, to do the "magic" things of Spring Boot programmatically (e.g. loading some properties in a static method), you can do:
private static ApisProperties apiProperties() {
try {
ClassPathResource resource;
resource = new ClassPathResource("/config/application.yml");
YamlPropertiesFactoryBean factoryBean;
factoryBean = new YamlPropertiesFactoryBean();
factoryBean.setSingleton(true); // optional depends on your use-case
factoryBean.setResources(resource);
Properties properties;
properties = factoryBean.getObject();
MutablePropertySources propertySources;
propertySources = new MutablePropertySources();
propertySources.addLast(new PropertiesPropertySource("apis", properties));
ApisProperties apisProperties;
apisProperties = new ApisProperties();
PropertiesConfigurationFactory<ApisProperties> configurationFactory;
configurationFactory = new PropertiesConfigurationFactory<>(apisProperties);
configurationFactory.setPropertySources(propertySources);
configurationFactory.setTargetName("foo"); // it's the same prefix as the one defined in the #ConfigurationProperties
configurationFactory.bindPropertiesToTarget();
return apisProperties; // apiProperties are fed with the values defined in the application.yaml
} catch (BindException e) {
throw new IllegalArgumentException(e);
}
}
Here's an update to ctranxuan's answer for Spring Boot 2.x. In our situation, we avoid spinning up a Spring context for unit tests, but do like to test our configuration classes (which is called AppConfig in this example, and its settings are prefixed by app):
public class AppConfigTest {
private static AppConfig config;
#BeforeClass
public static void init() {
YamlPropertiesFactoryBean factoryBean = new YamlPropertiesFactoryBean();
factoryBean.setResources(new ClassPathResource("application.yaml"));
Properties properties = factoryBean.getObject();
ConfigurationPropertySource propertySource = new MapConfigurationPropertySource(properties);
Binder binder = new Binder(propertySource);
config = binder.bind("app", AppConfig.class).get(); // same prefix as #ConfigurationProperties
}
}
The "magic" class you are looking for is PropertiesConfigurationFactory. But I would question your need for it - if you only need to bind once, then Spring should be able to do it for you, and if you have lifecycle issues it would be better to address those (in case they break something else).
This post is going into similar direction but extends the last answer with also validation and property placeholder resolutions.
Spring Boot Binder API support for #Value Annotations
#Value annotations in ConfigurationPropertys don't seem to bind properly though (at least if the referenced values are not part of the ConfigurationProperty's prefix namespace).
import org.springframework.boot.context.properties.bind.Binder
val binder = Binder.get(environment)
binder.bind(prefix, MySettings.class).get

Dynamically accessing properties using Spring 3.1's property abstraction

I'm trying to dynamically access properties from Spring's Environment property abstraction.
I declare my property files like this:
<context:property-placeholder
location="classpath:server.common.properties,
classpath:server.${my-environment}.properties" />
In my property file server.test.properties, I define the following:
myKey=foo
Then, given the following code:
#Component
public class PropertyTest {
#Value("${myKey}")
private String propertyValue;
#Autowired
private PropertyResolver propertyResolver;
public function test() {
String fromResolver = propertyResolver.getProperty("myKey");
}
}
When I run this code, I end up with propertyValue='foo', but fromResolver=null;
Receiving propertyValue indicates that the properties are being read, (and I know this from other parts of my code). However, attempting to look them up dynamically is failing.
Why? How can I dynamically look up property values, without having to use #Value?
Simply adding a <context:property-placeholder/> doesn't add a new PropertySource to the Environment. If you read the article you linked completely, you'll see it suggests registering an ApplicationContextInitializer in order to add new PropertySources so they'll be available in the way you're trying to use them.
To get this to work I had to split out the reading of the properties into a #Configuration bean, as shown here.
Here's the complete example:
#Configuration
#PropertySource("classpath:/server.${env}.properties")
public class AngularEnvironmentModuleConfiguration {
private static final String PROPERTY_LIST_NAME = "angular.environment.properties";
#Autowired
private Environment environment;
#Bean(name="angularEnvironmentProperties")
public Map<String,String> getAngularEnvironmentProperties()
{
String propertiesToInclude = environment.getProperty(PROPERTY_LIST_NAME, "");
String[] propertyNames = StringUtils.split(propertiesToInclude, ",");
Map<String,String> properties = Maps.newHashMap();
for (String propertyName : propertyNames)
{
String propertyValue = environment.getProperty(propertyName);
properties.put(propertyName, propertyValue);
}
return properties;
}
}
The set of properties are then injected elsewhere, to be consumed.

Categories

Resources