Is it possible to load spring-boot config from a .json file as opposed to .yaml or .properties? From looking at the documentation, this isn't supported out of the box - I'm wondering if it's possible and if so how one would go about doing it?
As noted in docs and on GitHub
YAML is a superset of JSON
So you can just create the following class in your Spring Boot project:
public class JsonPropertySourceLoader extends YamlPropertySourceLoader {
#Override
public String[] getFileExtensions() {
return new String[]{"json"};
}
}
Then create a file:
/src/main/resources/META-INF/spring.factories
with the following content:
org.springframework.boot.env.PropertySourceLoader=\
io.myapp.JsonPropertySourceLoader
And your Spring application is ready to load JSON configurations from application.json. The priority will be: .properties -> .yaml -> .json
If you have multiple apps, you can create a jar with the shared PropertySourceLoader and spring.factories file in order to include it to any project you need.
The spring boot way:
#EnableAutoConfiguration
#Configuration
#PropertySource(value = { "classpath:/properties/config.default.json" }, factory=SpringBootTest.JsonLoader.class )
public class SpringBootTest extends SpringBootServletInitializer {
#Bean
public Object test(Environment e) {
System.out.println(e.getProperty("test"));
return new Object();
}
public static void main(String[] args) {
SpringApplication.run(SpringBootTest.class);
}
public static class JsonLoader implements PropertySourceFactory {
#Override
public org.springframework.core.env.PropertySource<?> createPropertySource(String name,
EncodedResource resource) throws IOException {
Map readValue = new ObjectMapper().readValue(resource.getInputStream(), Map.class);
return new MapPropertySource("json-source", readValue);
}
}
}
Define your own PropertySourceFactory and hook it in via the #PropertySource annotation. Read the resource, set the properties, use them anywhere.
Only thing is, how do you translate nested properties. The Spring way to do that (by the way you can define Json also as a variable for properties, see: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html)
is to translate nested properties as such:
{"test": { "test2" : "x" } }
Becomes:
test.test2.x
Hope that helps,
Artur
The SPRING_APPLICATION_JSON properties can be supplied on the command line with an environment variable. For example, you could use the following line in a UN*X shell:
$ SPRING_APPLICATION_JSON='{"acme":{"name":"test"}}' java -jar
myapp.jar
In the preceding example, you end up with acme.name=test in the Spring Environment. You can also supply the JSON as spring.application.json in a System property, as shown in the following example:
$ java -Dspring.application.json='{"name":"test"}' -jar myapp.jar
You can also supply the JSON by using a command line argument, as shown in the following example:
$ java -jar myapp.jar --spring.application.json='{"name":"test"}'
You can also supply the JSON as a JNDI variable, as follows:
java:comp/env/spring.application.json.
Reference documentation: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html
2 steps
public String asYaml(String jsonString) throws JsonProcessingException, IOException {
// parse JSON
JsonNode jsonNodeTree = new ObjectMapper().readTree(jsonString);
// save it as YAML
String jsonAsYaml = new YAMLMapper().writeValueAsString(jsonNodeTree);
return jsonAsYaml;
}
Got from the post
and
public class YamlFileApplicationContextInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
#Override
public void initialize(ConfigurableApplicationContext applicationContext) {
try {
Resource resource = applicationContext.getResource("classpath:file.yml");
YamlPropertySourceLoader sourceLoader = new YamlPropertySourceLoader();
PropertySource<?> yamlTestProperties = yamlTestProperties = sourceLoader.load("yamlTestProperties", resource, null);
applicationContext.getEnvironment().getPropertySources().addFirst(yamlTestProperties);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
Got from the post
So you can combine both. Load your json as resource and convert to yaml and then add to Environment all the found properties
Related
I have defined some external properties in my main class as follows:
#PropertySources({
#PropertySource(value = "classpath:application.properties", ignoreResourceNotFound = true),
#PropertySource(value = "file:/opt/app/conf/database.properties", ignoreResourceNotFound = true),
#PropertySource(value = "file:/opt/app/conf/overrides.properties", ignoreResourceNotFound = true)
})
For some of the properties I would like to do some post processing, for example encrypting or enrichment before they are actually used by any beans. One such property is spring.datasource.password
I have done the following:
Write an ApplicationContextInitializer<ConfigurableApplicationContext> and tried to process those properties in the initialize() method
public class MyApplicationContextInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
#Override
public void initialize(ConfigurableApplicationContext applicationContext) {
ConfigurableEnvironment environment = applicationContext.getEnvironment();
String value = environment.getProperty("spring.datasource.password");
System.out.println("The value in initializer is " + value);
System.out.println("The database url in initializer is " + environment.getProperty("spring.datasource.url"));
System.out.println("The database username in initializer is " + environment.getProperty("spring.datasource.username"));
// .. other code
}
}
and included the above class in the META-INF/spring.factories as follows:
org.springframework.context.ApplicationContextInitializer=com.myapp.MyApplicationContextInitializer
I am seeing null values in all of the above properties that are printed though both database.properties and overrides.properties are present. They are the very first statements to be printed (even before the banner)
Another approach I tried is org.springframework.boot.env.EnvironmentPostProcessor and adding in META-INF/spring.factories as
org.springframework.boot.env.EnvironmentPostProcessor=com.myapp.PropertiesProcessor
But still, I get the same null value.
Interestingly, when I pass in the
-Dspring.config.location="file:/opt/app/conf/database.properties, file:/opt/app/conf/overrides.properties"
system property before executing the war, it works i.e. the values are printed.
But I do not want to manually pass in the system property at runtime. Is there a way to do it?
What is wrong with my code or approach. Are there any other ways of doing the post-processing before actually creating the beans?
I solved similar problem by adding property source with highest precedence.
Add META-INF/spring.factories with content:
org.springframework.boot.env.EnvironmentPostProcessor=
com.example.SettingsPropertiesAddingPostProcessor
SettingsPropertiesAddingPostProcessor implementation:
public class SettingsPropertiesAddingPostProcessor implements EnvironmentPostProcessor {
private static final String SETTINGS_CONFIG_PATH = "/tmp/settings.properties";
#Override
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
File settingsFile = new File(SETTINGS_CONFIG_PATH);
if (!settingsFile.exists()) {
log.debug("Config file not found, skipping adding custom property source");
return;
}
log.debug("Config file found, adding custom property source");
Properties props = loadProperties(settingsFile);
MutablePropertySources propertySources = environment.getPropertySources();
propertySources.addFirst(new PropertiesPropertySource("settings-source", props));
}
private Properties loadProperties(File f) {
FileSystemResource resource = new FileSystemResource(f);
try {
return PropertiesLoaderUtils.loadProperties(resource);
} catch (IOException ex) {
throw new IllegalStateException("Failed to load local settings from " + f.getAbsolutePath(), ex);
}
}
}
That should be all.
Following the advice of #M. Deinum, regarding using "spring.config.additional-location", I have made a workaround as follows:
#SpringBootApplication
#EnableSwagger2
public class MyApp extends SpringBootServletInitializer {
public static void main(String[] args) {
System.setProperty("spring.config.additional-location", "file:/opt/app/conf/database.properties, file:/opt/app/conf/overrides.properties");
SpringApplication springApplication = new SpringApplication(MyApp.class);
springApplication.run(args);
}
#Override
public void onStartup(ServletContext servletContext) throws ServletException {
System.out.println("Setting properties onStartup");
System.setProperty("spring.config.additional-location", "file:/opt/app/conf/database.properties, file:/opt/app/conf/overrides.properties");
super.onStartup(servletContext);
}
}
I have called the System.setProperty() in the onStartup() method of the SpringBootServletInitializer by overriding it as above and then invoked the super class' onStartup()
The latter part i.e. setting system property in onStartup method helps when the application is deployed in a web container like Tomcat.
Note: We can also append the properties to spring.config.additional-location instead of set so that other additional locations can also be added during runtime.
I'm trying to get a repository into a class annotated with #Service using the #Autowired annotation in a Spring-boot class. However, the repository turns up as a null.
Here's the relevant code:
#Service
public class ImportLicenses {
#Autowired
private LicenseRepository licenseRepository;
public static void main (String[] args) {
ImportLicenses importLicenses = new ImportLicenses();
importLicenses.listFiles("/Users/admin/Licenses/licenses");
}
private void processLicense (Path path) {
try {
count++;
BasicFileAttributes attr = Files.readAttributes(path, BasicFileAttributes.class);
FileTime fileTime = attr.lastModifiedTime();
Permission permission = new Permission(readLineByLineJava8(path.toFile().getAbsolutePath()));
LicensePojo licensePojo = new LicensePojo(permission, path);
Optional<License> licenseOptional = licenseRepository.findById(licensePojo.getId());
at which point it gets an NPE since licenceReposity is null.
I am able to access the licenseRepository in a controller class with this constructor
public LicenseController(LicenseRepository licenseRepository,
UserRepository userRepository) {
this.licenseRepository = licenseRepository;
this.userRepository = userRepository;
}
However, since I'm calling the constructor directly in the static main method, this doesn't seem like it's available. What's the best way to get the repository into this class?
Edit: Thanks for the responses. To clarify the question, I'm trying to pull this class into the structure of the existing Spring Boot application, instead of creating a separate one.
Option 1: Create a button or menu selection on the UI, and create a new controller class to run the import. This would be the simplest, but I don't want to necessarily have that on the UI.
Option 2: Code the import class create another Spring Application
#SpringBootApplication
public class ImportLicenses implements ApplicationRunner {
private final Logger logger = LoggerFactory.getLogger(LicenseGenApplication.class);
#SpringBootApplication
public static void main() {
main();
}
public static void main(String[] args) {
SpringApplication.run(ImportLiceneses.class, args);
}
#Override
public void run(ApplicationArguments args) throws Exception {
listFiles("/Users/admin//licenses");
}
public void listFiles(String path) {
try {
Files.walk(Paths.get(path))
.filter(ImportLicenses::test)
.forEach(p -> processLicense(p));
} catch (IOException e) {
e.printStackTrace();
}
}
....
}
Option 3 - Create a non-executable jar file from the existing application for use in the new application to avoid duplicating code.
Option 1 is the quickest, I'm not sure if option 2 work, Option 3 I'll take a look at to see if it's do-able.
Your application is a usual Java application. It is not a Spring Boot application.
What should you do? Make a Spring Boot application of it. For instance, create a demo app at https://start.spring.io/, compare your main class to the main class in the demo application, then adjust your main class correspondingly. Also compare your Maven or Gradle config to the config of the demo app and adjust correspondingly.
At least, your application should have an annotation #SpringBootApplication. Also, it should be launched via SpringApplication.run(...). This is a minimum. Depending on what you need, you may want to use #EnableAutoConfiguration and other configuration options.
My use case is a bit oddball but basically, I'd like to read a portion of a yaml file and map it to the appropriate java object in a spring application. This is a pretty common and trivial operation in spring (just use #ConfigurationProperties ).
However, in my case, I'd like to accomplish this reading earlier in the lifecycle i.e. by the time the BeanFactoryPostProcessor hooks in - in order to use the instructions specified in yml to dynamically create a number of beans.
I can get this working with application.properties but not with application.yml
I was hoping to use yml in order leverage mapping part of the yml to POJO and also utilize hierarchical mapping files and data structures (lists, maps etc).
Here's an example of how to read application.properties. https://blog.pchudzik.com/201705/dynamic-beans/
I set up a simple skeleton project at https://github.com/balamuru/yaml-loader to try out different techniques.
Any ideas ?
#Component
#EnableConfigurationProperties(SampleDataConfig.class)
class ConfigurableBeanFactory implements BeanFactoryPostProcessor, InitializingBean {
private List<String> beanInstances = new ArrayList<>();
#Override
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
final BeanDefinitionRegistry registry = (BeanDefinitionRegistry) beanFactory;
Map<String, SampleDataConfig> beans = beanFactory.getBeansOfType(SampleDataConfig.class);
System.err.println("");
beanInstances.forEach(instance -> {
registry.registerBeanDefinition(instance, BeanDefinitionBuilder
.rootBeanDefinition(SampleDataConfig.class)
.addConstructorArgValue(instance)
.getBeanDefinition());
});
}
#Override
public void afterPropertiesSet() throws Exception {
// this.beanInstances = asList(PropertiesLoaderUtils
// .loadProperties(new ClassPathResource("/application.properties"))
// .getProperty("dynamic-beans.instances", "")
// .split(","));
/**
* Rather than reading from application.properties,
* I would like to be able to load up the relevant prefix qualified segments (com.foo.bar.stuff) mapping to my POJO (SampleDataConfig,class)
* loaded from application.yml
*/
}
}
Internally, spring uses the following mechanism, but I was hoping there is an easier way to leverage this without re-inventing the spring :)
public class ConfigurationPropertiesBindingPostProcessor ...{
.
.
private void postProcessBeforeInitialization(Object bean, String beanName,
ConfigurationProperties annotation) {
Object target = bean;
PropertiesConfigurationFactory<Object> factory = new PropertiesConfigurationFactory<Object>(
target);
factory.setPropertySources(this.propertySources);
factory.setValidator(determineValidator(bean));
// If no explicit conversion service is provided we add one so that (at least)
// comma-separated arrays of convertibles can be bound automatically
factory.setConversionService(this.conversionService == null
? getDefaultConversionService() : this.conversionService);
if (annotation != null) {
factory.setIgnoreInvalidFields(annotation.ignoreInvalidFields());
factory.setIgnoreUnknownFields(annotation.ignoreUnknownFields());
factory.setExceptionIfInvalid(annotation.exceptionIfInvalid());
factory.setIgnoreNestedProperties(annotation.ignoreNestedProperties());
if (StringUtils.hasLength(annotation.prefix())) {
factory.setTargetName(annotation.prefix()); //====> use annotation prefix
}
}
try {
factory.bindPropertiesToTarget(); //===> bind properties
}
Thanks
YamlPropertiesFactoryBean yaml = new YamlPropertiesFactoryBean();
yaml.setResources(new ClassPathResource("application.yml"));
configProperty = yaml.getObject();
Set<Object> keys = configProperty.keySet();
Below is my YAML configuration, which looks like:
template:
config:
broker-urls:
- tcp://127.0.0.1:61616
- tcp://127.0.0.1:61617
- tcp://127.0.0.1:61618
qeues:
- Test
- Demo
- Qeue3
After applying above code you will get converted properties like below:
template.config.broker-urls[0]=tcp://127.0.0.1:61616
template.config.broker-urls[1]=tcp://127.0.0.1:61617
template.config.broker-urls[1]=tcp://127.0.0.1:61618
template.config.qeues[0]=Test
template.config.qeues[1]=Demo
template.config.qeues[1]=Qeue3
Is there any way to load a class marked with #ConfigurationProperties without using a Spring Context directly? Basically I want to reuse all the smart logic that Spring does but for a bean I manually instantiate outside of the Spring lifecycle.
I have a bean that loads happily in Spring (Boot) and I can inject this into my other Service beans:
#ConfigurationProperties(prefix="my")
public class MySettings {
String property1;
File property2;
}
See the spring docco for more info http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#boot-features-external-config-command-line-args
But now I need to access this bean from a class that is created outside of Spring (by Hibernate). The class is created so early in the app startup process that Spring Boot has not yet made the application context available through the classic lookup helper methods or roll-my-own static references.
So I instead want to do something like:
MySettings mySettings = new MySettings();
SpringPropertyLoadingMagicClass loader = new SpringPropertyLoadingMagicClass();
loader.populatePropertyValues(mySettings);
And have MySettings end up with all its values loaded, from the command line, system properties, app.properties, etc. Is there some class in Spring that does something like this or is it all too interwoven with the application context?
Obviously I could just load the Properties file myself, but I really want to keep Spring Boot's logic around using command line variables (e.g. --my.property1=xxx), or system variables, or application.properties or even a yaml file, as well as its logic around relaxed binding and type conversion (e.g. property2 is a File) so that it all works exactly the same as when used in the Spring context.
Possible or pipe dream?
Thanks for your help!
I had the same "issue".
Here is how I solved it in SpringBoot version 1.3.xxx and 1.4.1.
Let's say we have the following yaml configuration file:
foo:
apis:
-
name: Happy Api
path: /happyApi.json?v=bar
-
name: Grumpy Api
path: /grumpyApi.json?v=grrr
and we have the following ConfigurationProperties:
#ConfigurationProperties(prefix = "foo")
public class ApisProperties {
private List<ApiPath> apis = Lists.newArrayList();
public ApisProperties() {
}
public List<ApiPath> getApis() {
return apis;
}
public static class ApiPath {
private String name;
private String path;
public String getName() {
return name;
}
public void setName(final String aName) {
name = aName;
}
public String getPath() {
return path;
}
public void setPath(final String aPath) {
path = aPath;
}
}
}
Then, to do the "magic" things of Spring Boot programmatically (e.g. loading some properties in a static method), you can do:
private static ApisProperties apiProperties() {
try {
ClassPathResource resource;
resource = new ClassPathResource("/config/application.yml");
YamlPropertiesFactoryBean factoryBean;
factoryBean = new YamlPropertiesFactoryBean();
factoryBean.setSingleton(true); // optional depends on your use-case
factoryBean.setResources(resource);
Properties properties;
properties = factoryBean.getObject();
MutablePropertySources propertySources;
propertySources = new MutablePropertySources();
propertySources.addLast(new PropertiesPropertySource("apis", properties));
ApisProperties apisProperties;
apisProperties = new ApisProperties();
PropertiesConfigurationFactory<ApisProperties> configurationFactory;
configurationFactory = new PropertiesConfigurationFactory<>(apisProperties);
configurationFactory.setPropertySources(propertySources);
configurationFactory.setTargetName("foo"); // it's the same prefix as the one defined in the #ConfigurationProperties
configurationFactory.bindPropertiesToTarget();
return apisProperties; // apiProperties are fed with the values defined in the application.yaml
} catch (BindException e) {
throw new IllegalArgumentException(e);
}
}
Here's an update to ctranxuan's answer for Spring Boot 2.x. In our situation, we avoid spinning up a Spring context for unit tests, but do like to test our configuration classes (which is called AppConfig in this example, and its settings are prefixed by app):
public class AppConfigTest {
private static AppConfig config;
#BeforeClass
public static void init() {
YamlPropertiesFactoryBean factoryBean = new YamlPropertiesFactoryBean();
factoryBean.setResources(new ClassPathResource("application.yaml"));
Properties properties = factoryBean.getObject();
ConfigurationPropertySource propertySource = new MapConfigurationPropertySource(properties);
Binder binder = new Binder(propertySource);
config = binder.bind("app", AppConfig.class).get(); // same prefix as #ConfigurationProperties
}
}
The "magic" class you are looking for is PropertiesConfigurationFactory. But I would question your need for it - if you only need to bind once, then Spring should be able to do it for you, and if you have lifecycle issues it would be better to address those (in case they break something else).
This post is going into similar direction but extends the last answer with also validation and property placeholder resolutions.
Spring Boot Binder API support for #Value Annotations
#Value annotations in ConfigurationPropertys don't seem to bind properly though (at least if the referenced values are not part of the ConfigurationProperty's prefix namespace).
import org.springframework.boot.context.properties.bind.Binder
val binder = Binder.get(environment)
binder.bind(prefix, MySettings.class).get
We have implemented an extended .properties format. Such a properties file can contain an optional include property. The value of this properties is the classpaths of other properties files to load recursively.
I could configure the Spring environment for my application but I have a problem to engage this mechanism with SpringJUnit4ClassRunner.
I thought I could use the initializer property of the ContextConfiguration annotation but it looks it can only be instantiated with a no-arg constructor.
I need to give it the root file of my properties files hierarchy. It could eventually be another annotation on my test class, but again, how can I access it ?
The only think I have found so far is to set this file as a system property in my test class static initializer. ugly?:
#ActiveProfiles("qacs.controller.channels=mock")
#ContextConfiguration(initializer=ContainerTestContextInitializer.class)
public class QacsControllerTest
{
static
{
System.setProperty(ContainerTestContextInitializer.SYSTEM_PROPERTY, "classpath:com/xxx/qacs/QacsControllerTest.properties");
}
#Test
void test() {}
}
}
public class ContainerTestContextInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext>
{
public static final String SYSTEM_PROPERTY = "icomp.test.properties";
#Override
public void initialize(ConfigurableApplicationContext pApplicationContext)
{
String path = System.getProperty(SYSTEM_PROPERTY);
if (path == null)
{
throw new IllegalStateException("Missing system property " + SYSTEM_PROPERTY);
}
final DefaultPropertiesLoader loader;
loader = new DefaultPropertiesLoader(System.getProperties());
try
{
loader.load(path);
}
catch (IOException e)
{
throw new IllegalStateException(e.getMessage(), e);
}
MutablePropertySources sources = pApplicationContext.getEnvironment().getPropertySources();
MapPropertySource mps = new MapPropertySource(Launcher.ICOMP_PROPERTY_SOURCE, (Map) loader.getProperties());
sources.addFirst(mps);
}
}