i want developers to be able to locally override some configuration properties. (let's say we work on google drive and everyone should test it on its own account).
i don't want to override properties using command line (because it has to be set inside every IDE configuration and on every CLI run).
what i want is: application should use all the standard spring boot config files (application.yml etc) and also look for e.g. local.yml (on the classpath) or some file inside user.home. and those additional files should override other settings.
how to add new yml resources and order them correctly?
edit: i know spring's default orders and locations. question is about adding new ones
If you look in the Spring Boot documentation about the locations for configuration files (http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#boot-features-external-config), you can see, that they are loaded from the following places (amongst others):
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
Application properties outside of your packaged jar (application.properties and YAML variants).
There are two default locations where they are loaded from ( see http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#boot-features-external-config-application-property-files):
A /config subdirectory of the current directory.
The current directory
Current directory in this case means the working directory for the Java process (Usually the directory where the JAR is located, or in case of running with in the IDE, usually the project root folder). So the developers just can place their own configuration files in that places and they are automatically loaded (and will override properties within the JARs). Add that files to .gitignore (or .svnignore or ...) and they won't accidentally committed into your repository.
There's a new way to do this, after Spring Boot v2.4, by using spring.config.import: https://spring.io/blog/2020/08/14/config-file-processing-in-spring-boot-2-4#importing-additional-configuration
By adding this part to your application.yml file, you should be able to import the additional configuration:
spring:
config:
import: local.yml
The article also has this section:
Imports can be considered as additional documents inserted just below the document that declares them. They follow the same top-down ordering as regular multi-document files: An import will only be imported once, no matter how many times it is declared.
So the contents of local.yml should be handled as if they were appended to the end of application.yml, thereby allowing you to override any property in application.yml.
From Spring Boot Documentation : Application property files:
SpringApplication will load properties from application.properties files in the following locations and add them to the Spring Environment:
A /config subdirectory of the current directory.
The current directory
A classpath /config package
The classpath root
The list is ordered by precedence (properties defined in locations higher in the list override those defined in lower locations).
This also goes for yaml, so you everyone can add application.yml under config directory, under the directory you run the spring boot jar from.
You can also customize the extra configuration file to be local.yml if you'd like by using spring.config.location:
--spring.config.location=classpath:/application.yml,classpath:/local.yml
Note however:
spring.config.name and spring.config.location are used very early to determine which files have to be loaded so they have to be defined as an environment property (typically OS env, system property or command line argument).
To provide the configuration from external config file in spring-boot application -
-Dspring.config.location=file:/home/vfroot/Workspace/project/MODULE_HOME/application.yaml
this command can be run with terminal:
mvn clean install -Dspring.config.location
= file:/home/vfroot/Workspace/MODULE_HOME/application.yaml
or need to set in Eclipse VM argument.
Also to set the active profiles :
-Dspring.profiles.active=dev
Well, since i am new in Spring Boot & Restfull Web Services. However, i managed to add a new .yml file to mange database and server port.
Instructions that i followed:
Project File.
Other Sources
src/main/resources
default package
right click on "default package"
add new YAML FILE
Or of YAML File option not available
5. right click on "default package"
6. then in categories: other --> File Types: YAML File
Related
I have a external configuration file(out side jar). I try to run and expected
that value in external file will override value in internal file(application.properties in \resource\ - in jar file).
I read Documentation and try this:
java -jar ccgame-1.0.jar --spring.config.location=classpath:/application.properties,file:/production.properties
This not working.
My jar file at \target\ directory and my production.properties too(at \target\)
How can I resolve my problem?
Where should I put external config file ?
And what I have to do ?
Starting from Spring Boot 2.0 it's possible to use property spring.config.additional-location. With this property, you can set external config file, but properties from that config will only override the corresponding ones from internal config, leaving other properties unchanged.
More about it in docs.
If you need to completely override the whole config, then continue to use spring.config.location property instead.
By convention, Spring Boot looks for an externalized configuration file – application.properties or application.yml – in 4 predetermined locations in the following order of precedence:
/config subdirectory of the current directory
The current directory
Classpath /config package
The classpath root
You can place your application.properties in any of the 4 locations without needing to give the location of application.properties while executing the jar. If you want to given any other custom location , then you will have to provide the path of the config location while executing the jar:
java -jar -Dspring.config.location=<path-to-file> myProject.jar
Source: https://www.baeldung.com/spring-properties-file-outside-jar
According to the SpringBoot documentation, the order of configuration is as:
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
On my project I have a profile called "prod" and the following files:
application.yml (inside the jar)
application-prod.yml (inside the jar)
And I also want to override some of the properties using an external file. Since according to the docs, an external application.yml will be overridden by the internal application-prod.yml, I need to make sure that the external file is considered as a profile specific config file.
I have tried to use:
-Dspring.config.location=<my path>/application-prod.yml
and I have also tried:
-Dspring.config.location=file:<my path>/application-prod.yml
In all cases I get the value from the internal application-prod.yml
If I totally remove the internal config file then I get the value from the external (so I know that the config picks up the file).
I understand that this external file is considered as the equivalent to the generic application.yml and not a profile specific.
How can I configure it to be considered as a profile specific external config?
Found the answer:
You need to use a Directory externally to set the profile specific configuration files, not using the file directly and it needs to end in /. So it has to be:
-Dspring.profiles.active=prod
-Dspring.config.location=/<some-path>/config/ (any path that ends in /)
and in there have a :
application-prod.yml
I am in a scenario where I am deploying by spring boot as a WAR on tomcat. Here in this application I have application.properties which has database username/password , some URL which help in consuming rest services(urls vary depending on the environment). Now I need to get the DB credentials and URL's depending on the environment it is deployed to. how to achieve it.
On external container ex tomcat
You can pass configuration using Jndi variable in context.xml
This var override local property defined in application.properties
Or pass -Dspring.profiles.active=env
in tomcat startup script ,for select environment specific application.properties
if you prefer to have configuration inside the war
As per 24. Externalized Configuration it should be enough to place a profile specific properties e.g. application-dev.properties on the classpath. The property precedence is:
...
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants).
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants).
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
...
This can however work slight differently if you are packaging as JAR as per 24.3 Application Property Files. The property precedence is:
A /config subdirectory of the current directory
The current directory
A classpath /config package
The classpath root
according to the discussion with Karol, I guess using properties sources referencing file system path should be ok:
#Configuration
#PropertySource("${mywebapp.config.dir}/application.properties")
public class SpringConfig {
}
And just path JVM args at the startup of tomcat like
-Dmywebapp.config.dir=file:/etc/mywebapp
This way you can deploy one application.properties different for each environment.
As for your comment:
#PropertySource(value={"file:C:/Users/foo/apache-tomcat-8.5.28/webapps/application.properties}"})
as it's a windows system path may you have to double backslash your path: C:\\Users\\foo ...
I am able to read the properties file from tomcat/webapp location
for every environment I can ask system engineers to drop the file at the location,
I dont know if its a good solution. Please suggest
#PropertySource(value={"file:C:/Users/foo/Downloads/apache-tomcat-8.5.28/webapps/application.properties"})
I have a spring boot application that I can package in a war that I want to deploy to different environments. To automate this deployment it'd be easier to have the configuration file externalized.
Currently everything works fine with a application.properties file in src/main/resources. Then I use ´mvn install´ to build a war deployable to tomcat.
But I would like to use a .yml file that does not need to be present on mvn install but that would be read from during deployment of the war and is in the same or a directory relative to my war.
24. externalized configuration shows where spring boot will look for files and 72.3 Change the location of external properties of an application gives more detail on how to configure this but I just do not understand how to translate this to my code.
My application class looks like this:
package be.ugent.lca;
Updated below
Do I need to add a #PropertySource to this file? How would I refer to a certain relative path?
I feel like it's probably documented in there as most spring boot documentation but I just don't understand how they mean me to do this.
EDIT
Not sure if this should be a separate issue but I think it's still related.
Upon setting the os variable the error of yaml file not found went away. Yet I still get the same error again as when I had no application .properties or .yml file.
Application now looks like this:
#Configuration
**#PropertySource("file:${application_home}/application.yml")**
#ComponentScan({"be.ugent.lca","be.ugent.sherpa.configuration"})
#EnableAutoConfiguration
#EnableSpringDataWebSupport
public class Application extends SpringBootServletInitializer{
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
The application_home OS variable
$ echo $application_home
C:\Masterproef\clones\la15-lca-web\rest-service\target
My application.yml file(part it complains about):
sherpa:
package:
base: be.ugent.lca
Error upon java -jar *.war
All variations upon:
Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'sherpa.package.base' in string value "${sherpa.package.base}"
at org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:174)
at org.springframework.util.PropertyPlaceholderHelper.replacePlaceholders(PropertyPlaceholderHelper.java:126)
at org.springframework.core.env.AbstractPropertyResolver.doResolvePlaceholders(AbstractPropertyResolver.java:204)
at org.springframework.core.env.AbstractPropertyResolver.resolveRequiredPlaceholders(AbstractPropertyResolver.java:178)
at org.springframework.context.support.PropertySourcesPlaceholderConfigurer$2.resolveStringValue(PropertySourcesPlaceholderConfigurer.java:172)
at org.springframework.beans.factory.support.AbstractBeanFactory.resolveEmbeddedValue(AbstractBeanFactory.java:808)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1027)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1014)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:545)
... 142 more
Using external properties files
The answer lies in the Spring Boot Docs, I'll try to break it down for you.
First of all, no you should not use #PropertySource when working with Yaml configuration, as mentioned here under the Yaml shortcomings :
YAML files can’t be loaded via the #PropertySource annotation. So in the case that you need to load values that way, you need to use a properties file.
So, how to load propery files? That is explained here Application Property Files
One is loaded for you: application.yml , place it in one of the directories as mentioned in the link above. This is great for your general configuration.
Now for your environment specific configuration (and stuff like passwords) you want to use external property files, how to do that is also explained in that section :
If you don’t like application.properties as the configuration file name you can switch to another by specifying a spring.config.name environment property. You can also refer to an explicit location using the spring.config.location environment property (comma-separated list of directory locations, or file paths).
So you use the spring.config.location environment property.
Imagine you have an external config file: application-external.yml in the conf/ dir under your home directory, just add it like this:
-Dspring.config.location=file:${home}/conf/application-external.yml as a startup parameter of your JVM.
If you have multiple files, just seperate them with a comma. Note that you can easily use external properties like this to overwrite properties, not just add them.
I would advice to test this by getting your application to work with just your internal application.yml file , and then overwrite a (test) property in your external properties file and log the value of it somewhere.
Bind Yaml properties to objects
When working with Yaml properties I usually load them with #ConfigurationProperties, which is great when working with for example lists or a more complex property structure. (Which is why you should use Yaml properties, for straightforward properties you are maybe better of using regular property files). Read this for more information: Type-Safe Configuration properties
Extra: loading these properties in IntelliJ, Maven and JUnit tests
Sometimes you want to load these properties in your maven builds or when performing tests. Or just for local development with your IDE
If you use IntelliJ for development you can easily add this by adding it to your Tomcat Run Configuration : "Run" -> "Edit Configurations" , select your run configuration under "Tomcat Server" , check the Server tab and add it under "VM Options".
To use external configuration files in your Maven build : configure the maven surefire plugin like this in your pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Dspring.config.location=file:${home}/conf/application-external.yml</argLine>
</configuration>
</plugin>
When running JUnit tests in IntelliJ:
Run → Edit Configurations
Defaults → JUnit
add VM Options -> -ea -Dspring.config.location=file:${home}/conf/application-external.yml
Yes, you need to use #PropertySource as shown below.
The important point here is that you need to provide the application_home property (or choose any other name) as OS environment variable or System property or you can pass as a command line argument while launching Spring boot. This property tells where the configuration file (.properties or .yaml) is exactly located (example: /usr/local/my_project/ etc..)
#Configuration
#PropertySource("file:${application_home}config.properties")//or specify yaml file
#ComponentScan({"be.ugent.lca","be.ugent.sherpa.configuration"})
#EnableAutoConfiguration
#EnableSpringDataWebSupport
public class Application extends SpringBootServletInitializer{
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
There is a very simple way to achieve this.
Inside your original application.properties file you can just specify the following line:
spring.config.import=file:Directory_To_The_File/Property_Name.properties
It will automatically sync all the properties from the external property file.
Now lets say that you have a situation where you need to get properties from multiple property files. In that case, you can mention the same line in the external property file which in turn will take the remaining properties from the second property file and so on.
Consider the following example.
application.properties:
spring.config.import=file:Resources/Custom1.properties
Custom1.properties:
server.port=8090
.
.
.
spring.config.import=file:Resources/Custom2.properties
One of the easiest way to use externalized property file using system environment variable is, in application.properties file you can use following syntax:
spring.datasource.url = ${OPENSHIFT_MYSQL_DB_HOST}:${OPENSHIFT_MYSQL_DB_PORT}/"nameofDB"
spring.datasource.username = ${OPENSHIFT_MYSQL_DB_USERNAME}
spring.datasource.password = ${OPENSHIFT_MYSQL_DB_PORT}
Now, declare above used environment variables,
export OPENSHIFT_MYSQL_DB_HOST="jdbc:mysql://localhost"
export OPENSHIFT_MYSQL_DB_PORT="3306"
export OPENSHIFT_MYSQL_DB_USERNAME="root"
export OPENSHIFT_MYSQL_DB_PASSWORD="123asd"
This way you can use different value for same variable in different environments.
Use below code in your boot class:
#PropertySource({"classpath:omnicell-health.properties"})
use below code in your controller:
#Autowired
private Environment env;
I'm having difficulty trying to override a property declared in a profile-specific application properties file on the classpath with another value declared in an overrides file on the file system.
I have an auto-configured Spring-boot application (that is, using #EnableAutoconfiguration) that has multiple profiles, which I launch using PropertiesLauncher rather than JarLauncher (the reason having to do with deployment constraints - I need to deploy an exploded directory rather than an archive into a read-only filesystem.)
Within the root of my application, I have some profile-specific application properties, for example:
application-dev.properties
application-qa.properties
application-prd.properties
And let's say, for the sake of argument that application-dev.properties contains:
foo.bar=baz
foo.baz=other
For any environment, it may be necessary to override an existing property, as well as supply an absent one (like a production password, for example), and the issue I'm seeing is with overriding properties already declared in an application-${profile}.properties file on the classpath. (Supplying properties not present in the classpath file works fine, this is not the issue.)
Say I have an overrides properties file in a file system location such as:
/local/appname/dev/overrides/application.properties
and I want to override the property, foo.bar, as well as declare a new property, foo.password.
Therefore the contents of the overrides file are:
foo.bar=overridden-value
foo.password=something
When I launch the application, I use a command line something like this:
java -Dspring.config.location=file:/local/appname/dev/overrides/
-Dspring.profiles.active=dev
org.springframework.boot.loader.PropertiesLauncher
--debug &
The issue I am seeing is that although foo.password, the property not declared in the application-dev.properties file is picked up, the override of foo.bar is ignored - I still see the value, baz from application-dev.properties rather than the value, overridden-value from /local/appname/dev/overrides/application.properties.
With the --debug option enabled, I can see the ConfigFileApplicationListener logging that it has loaded both the overrides file (from the filesystem) and the profile-specific file (from the classpath), in that order.
I'm tempted into the perhaps naïve conclusion that because the overrides file is listed first, it is being loaded first then overridden by the 'default' profile-specific file from the classpath, which is listed later. I do appreciate however, that order of listing in the log doesn't necessarily correlate with behaviour. And I have tried varying the order of paths declared on the spring.config.location property, so that classpath: is listed before file:... but this hasn't helped and I't not convinced it would anyway, given that the Spring-boot documentation clearly states that the default properties locations are always searched even if you supply a value for spring.config.location.
The Spring-boot documentation is very specific about the order that properties are resolved for a Spring-boot executable JAR, in descending order of precedence:
Command line arguments.
Java System properties (System.getProperties()).
OS environment variables.
JNDI attributes from java:comp/env
A RandomValuePropertySource that only has properties in random.*.
Application properties outside of your packaged jar (application.properties including YAML and profile variants).
Application properties packaged inside your jar (application.properties including YAML and profile variants).
#PropertySource annotations on your #Configuration classes.
Default properties (specified using SpringApplication.setDefaultProperties).
Take note of lines 6 and 7 - properties outside over properties inside your jar.
What's not stated, as far as I can see, and which may be the source of my confusion/issue, is what happens when you're not using a JAR but an exploded directory (and therefore PropertiesLauncher.)
If the behaviour of an exploded directory were consistent with what's stated for a JAR, I'd expect that the values of properties declared in /local/appname/dev/overrides/application.properties would override any of the same name declared in classpath:application-dev.properties, but this doesn't seem to be the case.
Also noted from the Spring-boot documentation (appendix C.4 on PropertiesLauncher) is mention of the loader.home property, which is described as '... [the] Location of additional properties file, e.g. /opt/app (defaults to ${user.dir})'.
So I tried using loader.home instead of spring.config.location, but to no avail.
(Update: I also tried using loader.config.location and I have two notes: it seems to want a file rather than a directory (so its behaviour is not analogous with spring.config.location), and when I did supply a file path rather than the parent directory, it still didn't help.)
Can anyone spot what I'm doing wrong, or what incorrect assumption(s) I'm making?
Thanks, Dave, your suggestion was 100% correct.
If I rename the properties file in /local/appname/dev/overrides to application-dev.properties then the property values from that file do override the ones in classpath:application-dev.properties.
I was sure I had tried this combination yesterday, but I think what must have stopped it working was when I was playing around with specifying the spring.config.location and got that wrong so it wasn't looking for the override file in the right place.