Develop Reference

The best way to make app configs editable for docker image/container?

I have an app which is dockerized like this:
FROM openjdk:11-jre-slim as jdkbase
FROM jdkbase
COPY target/dependency-jars /run/dependency-jars
COPY target/resources /run/resources
ADD target/app-1.0.2.jar /run/app-1.0.2.jar
CMD java -jar run/app-1.0.2.jar
This app uses some configs from application.properties. I push this docker image into my private registry for using it inside Kubernetes cluster.
There is no problems unless i have to change any properties in application.properties. Lets say, my database URL was changed. So, I have to undate it in application.properties and then force my app to use updated configs. I've tried to edit application.properties inside the running docker container and then restart the container. As a result, after i restart my container it has edited application.properties BUT the app still uses old URL.
The only way i've found to force the app to use new configs is to commit changed container into new image and then start the new image.
It works, but it doesn't seems to me to be an optimal solution: like, after every changes in app's configs I have to recreate image, which is +300Mb of data, I have to push this new image into registry, I have to recreate Kubernetes pod from new image... It looks like too much unnecessary actions to just change one URL.
So, is there any other more optimal way to work with needs to change application.properties from time to time?

If this is a spring boot application is easy to define a variable as an environment variable. If an environment variable is present it overwrite the variable with the same name defined in your application.properties.
What happens is called externalized configuration and it is defined here :
Spring Boot allows you to externalize your configuration so you can work with the same application code in different environments. You can use properties files, YAML files, environment variables and command-line arguments to externalize configuration. Property values can be injected directly into your beans using the #Value annotation, accessed via Spring’s Environment abstraction or bound to structured objects via #ConfigurationProperties.
Spring Boot uses a very particular PropertySource order that is designed to allow sensible overriding of values. Properties are considered in the following order:
Devtools global settings properties on your home directory (~/.spring-boot-devtools.properties when devtools is active).
#TestPropertySource annotations on your tests.
#SpringBootTest#properties annotation attribute on your tests.
Command line arguments.
Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property)
ServletConfig init parameters.
ServletContext init parameters.
JNDI attributes from java:comp/env.
Java System properties (System.getProperties()).
OS environment variables.
A RandomValuePropertySource that only has properties in random.*.
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
#PropertySource annotations on your #Configuration classes.
Default properties (specified using SpringApplication.setDefaultProperties).
So you can pass at runtime an environment variable to your docker to solve this problem. You don't need to recreate the Docker image each time.

System environment properties overriding spring boot application.properties

In a spring boot application, I have a property in application.properties
spring.datasource.url: jdbc:mysql://host1:3306/test?useSSL=false&autoReconnect=true
However when I start the application, it is connecting to a totally different database.
So when I checked the env values in /actuator/env it url, it shows that the value is getting overrided by an environment property
SPRING_DATASOURCE_URL": {
"value": "jdbc:mysql://host2:3306/test",
"origin": "System Environment Property \"SPRING_DATASOURCE_URL\""
}
I have no idea where this environment property is configured.
I just want it to pick from my property file and not the environment. Is there any way to do that?

there are 17 ways to set and override properties in spring boot and they are ordered,
for example- application.properties outside the jar comes before application.properties inside the jar.
you could search potential source that override your properties in all those ways-
(ordered from strongest to weakest ) - copied from spring boot reference -https://docs.spring.io/spring-boot/docs/current/reference/html/spring-boot-features.html#boot-features-external-config
Devtools global settings properties in the $HOME/.config/spring-boot directory when devtools is active.
#TestPropertySource annotations on your tests.
properties attribute on your tests. Available on #SpringBootTest and the test annotations for testing a particular slice of your application.
Command line arguments.
Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property).
ServletConfig init parameters.
ServletContext init parameters.
JNDI attributes from java:comp/env.
Java System properties (System.getProperties()).
OS environment variables.
A RandomValuePropertySource that has properties only in random.*.
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants).
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants).
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
#PropertySource annotations on your #Configuration classes. Please note that such property sources are not added to the Environment until the application context is being refreshed. This is too late to configure certain properties such as logging.* and spring.main.* which are read before refresh begins.
Default properties (specified by setting SpringApplication.setDefaultProperties).

springboot external configuration - profile specific configuration

According to the SpringBoot documentation, the order of configuration is as:
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
On my project I have a profile called "prod" and the following files:
application.yml (inside the jar)
application-prod.yml (inside the jar)
And I also want to override some of the properties using an external file. Since according to the docs, an external application.yml will be overridden by the internal application-prod.yml, I need to make sure that the external file is considered as a profile specific config file.
I have tried to use:
-Dspring.config.location=<my path>/application-prod.yml
and I have also tried:
-Dspring.config.location=file:<my path>/application-prod.yml
In all cases I get the value from the internal application-prod.yml
If I totally remove the internal config file then I get the value from the external (so I know that the config picks up the file).
I understand that this external file is considered as the equivalent to the generic application.yml and not a profile specific.
How can I configure it to be considered as a profile specific external config?

Found the answer:
You need to use a Directory externally to set the profile specific configuration files, not using the file directly and it needs to end in /. So it has to be:
-Dspring.profiles.active=prod
-Dspring.config.location=/<some-path>/config/ (any path that ends in /)
and in there have a :
application-prod.yml

Spring boot deployed as WAR on tomcat, How to Externalize application.properties for different profiles(Dev,Test,Staging,Prod)

I am in a scenario where I am deploying by spring boot as a WAR on tomcat. Here in this application I have application.properties which has database username/password , some URL which help in consuming rest services(urls vary depending on the environment). Now I need to get the DB credentials and URL's depending on the environment it is deployed to. how to achieve it.

On external container ex tomcat
You can pass configuration using Jndi variable in context.xml
This var override local property defined in application.properties
Or pass -Dspring.profiles.active=env
in tomcat startup script ,for select environment specific application.properties
if you prefer to have configuration inside the war

As per 24. Externalized Configuration it should be enough to place a profile specific properties e.g. application-dev.properties on the classpath. The property precedence is:
...
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants).
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants).
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
...
This can however work slight differently if you are packaging as JAR as per 24.3 Application Property Files. The property precedence is:
A /config subdirectory of the current directory
The current directory
A classpath /config package
The classpath root

according to the discussion with Karol, I guess using properties sources referencing file system path should be ok:
#Configuration
#PropertySource("${mywebapp.config.dir}/application.properties")
public class SpringConfig {
}
And just path JVM args at the startup of tomcat like
-Dmywebapp.config.dir=file:/etc/mywebapp
This way you can deploy one application.properties different for each environment.
As for your comment:
#PropertySource(value={"file:C:/Users/foo/apache-tomcat-8.5.28/webapps/application.properties}"})
as it's a windows system path may you have to double backslash your path: C:\\Users\\foo ...

I am able to read the properties file from tomcat/webapp location
for every environment I can ask system engineers to drop the file at the location,
I dont know if its a good solution. Please suggest
#PropertySource(value={"file:C:/Users/foo/Downloads/apache-tomcat-8.5.28/webapps/application.properties"})

spring boot: add new yml files to application config

i want developers to be able to locally override some configuration properties. (let's say we work on google drive and everyone should test it on its own account).
i don't want to override properties using command line (because it has to be set inside every IDE configuration and on every CLI run).
what i want is: application should use all the standard spring boot config files (application.yml etc) and also look for e.g. local.yml (on the classpath) or some file inside user.home. and those additional files should override other settings.
how to add new yml resources and order them correctly?
edit: i know spring's default orders and locations. question is about adding new ones

If you look in the Spring Boot documentation about the locations for configuration files (http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#boot-features-external-config), you can see, that they are loaded from the following places (amongst others):
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
Application properties outside of your packaged jar (application.properties and YAML variants).
There are two default locations where they are loaded from ( see http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#boot-features-external-config-application-property-files):
A /config subdirectory of the current directory.
The current directory
Current directory in this case means the working directory for the Java process (Usually the directory where the JAR is located, or in case of running with in the IDE, usually the project root folder). So the developers just can place their own configuration files in that places and they are automatically loaded (and will override properties within the JARs). Add that files to .gitignore (or .svnignore or ...) and they won't accidentally committed into your repository.

There's a new way to do this, after Spring Boot v2.4, by using spring.config.import: https://spring.io/blog/2020/08/14/config-file-processing-in-spring-boot-2-4#importing-additional-configuration
By adding this part to your application.yml file, you should be able to import the additional configuration:
spring:
config:
import: local.yml
The article also has this section:
Imports can be considered as additional documents inserted just below the document that declares them. They follow the same top-down ordering as regular multi-document files: An import will only be imported once, no matter how many times it is declared.
So the contents of local.yml should be handled as if they were appended to the end of application.yml, thereby allowing you to override any property in application.yml.

From Spring Boot Documentation : Application property files:
SpringApplication will load properties from application.properties files in the following locations and add them to the Spring Environment:
A /config subdirectory of the current directory.
The current directory
A classpath /config package
The classpath root
The list is ordered by precedence (properties defined in locations higher in the list override those defined in lower locations).
This also goes for yaml, so you everyone can add application.yml under config directory, under the directory you run the spring boot jar from.
You can also customize the extra configuration file to be local.yml if you'd like by using spring.config.location:
--spring.config.location=classpath:/application.yml,classpath:/local.yml
Note however:
spring.config.name and spring.config.location are used very early to determine which files have to be loaded so they have to be defined as an environment property (typically OS env, system property or command line argument).

To provide the configuration from external config file in spring-boot application -
-Dspring.config.location=file:/home/vfroot/Workspace/project/MODULE_HOME/application.yaml
this command can be run with terminal:
mvn clean install -Dspring.config.location
= file:/home/vfroot/Workspace/MODULE_HOME/application.yaml
or need to set in Eclipse VM argument.
Also to set the active profiles :
-Dspring.profiles.active=dev

Well, since i am new in Spring Boot & Restfull Web Services. However, i managed to add a new .yml file to mange database and server port.
Instructions that i followed:
Project File.
Other Sources
src/main/resources
default package
right click on "default package"
add new YAML FILE
Or of YAML File option not available
5. right click on "default package"
6. then in categories: other --> File Types: YAML File