Develop Reference

How can I create a folder and a file before spring boot application load the application.propeties

We have a spring boot application that uses an application.properties file that contains two values that are similar to:
file.file=folder/name.jks
file.file2=folder2/name2.jks
these files are required when the application starts.
now, we need to read two environment variables to build these files, but the problem here is that we need to do it before the application loads the application.properties file
I tried to do something like :
public static void main(String[] args) {
Method.toCreateFiles();
SpringApplication.run(app.class);
}
and the environment variables are empty (I suppose that the reason is about the server configuration, and we can not change this easily).
So, we need to read the environment variables into the spring boot context, but first that the application.properties file is loaded because the path of files is needed at this point, I'd like to know some ideas to do that.

The best way to make app configs editable for docker image/container?

I have an app which is dockerized like this:
FROM openjdk:11-jre-slim as jdkbase
FROM jdkbase
COPY target/dependency-jars /run/dependency-jars
COPY target/resources /run/resources
ADD target/app-1.0.2.jar /run/app-1.0.2.jar
CMD java -jar run/app-1.0.2.jar
This app uses some configs from application.properties. I push this docker image into my private registry for using it inside Kubernetes cluster.
There is no problems unless i have to change any properties in application.properties. Lets say, my database URL was changed. So, I have to undate it in application.properties and then force my app to use updated configs. I've tried to edit application.properties inside the running docker container and then restart the container. As a result, after i restart my container it has edited application.properties BUT the app still uses old URL.
The only way i've found to force the app to use new configs is to commit changed container into new image and then start the new image.
It works, but it doesn't seems to me to be an optimal solution: like, after every changes in app's configs I have to recreate image, which is +300Mb of data, I have to push this new image into registry, I have to recreate Kubernetes pod from new image... It looks like too much unnecessary actions to just change one URL.
So, is there any other more optimal way to work with needs to change application.properties from time to time?

If this is a spring boot application is easy to define a variable as an environment variable. If an environment variable is present it overwrite the variable with the same name defined in your application.properties.
What happens is called externalized configuration and it is defined here :
Spring Boot allows you to externalize your configuration so you can work with the same application code in different environments. You can use properties files, YAML files, environment variables and command-line arguments to externalize configuration. Property values can be injected directly into your beans using the #Value annotation, accessed via Spring’s Environment abstraction or bound to structured objects via #ConfigurationProperties.
Spring Boot uses a very particular PropertySource order that is designed to allow sensible overriding of values. Properties are considered in the following order:
Devtools global settings properties on your home directory (~/.spring-boot-devtools.properties when devtools is active).
#TestPropertySource annotations on your tests.
#SpringBootTest#properties annotation attribute on your tests.
Command line arguments.
Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property)
ServletConfig init parameters.
ServletContext init parameters.
JNDI attributes from java:comp/env.
Java System properties (System.getProperties()).
OS environment variables.
A RandomValuePropertySource that only has properties in random.*.
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
#PropertySource annotations on your #Configuration classes.
Default properties (specified using SpringApplication.setDefaultProperties).
So you can pass at runtime an environment variable to your docker to solve this problem. You don't need to recreate the Docker image each time.

Spring boot deployed as WAR on tomcat, How to Externalize application.properties for different profiles(Dev,Test,Staging,Prod)

I am in a scenario where I am deploying by spring boot as a WAR on tomcat. Here in this application I have application.properties which has database username/password , some URL which help in consuming rest services(urls vary depending on the environment). Now I need to get the DB credentials and URL's depending on the environment it is deployed to. how to achieve it.

On external container ex tomcat
You can pass configuration using Jndi variable in context.xml
This var override local property defined in application.properties
Or pass -Dspring.profiles.active=env
in tomcat startup script ,for select environment specific application.properties
if you prefer to have configuration inside the war

As per 24. Externalized Configuration it should be enough to place a profile specific properties e.g. application-dev.properties on the classpath. The property precedence is:
...
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants).
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants).
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
...
This can however work slight differently if you are packaging as JAR as per 24.3 Application Property Files. The property precedence is:
A /config subdirectory of the current directory
The current directory
A classpath /config package
The classpath root

according to the discussion with Karol, I guess using properties sources referencing file system path should be ok:
#Configuration
#PropertySource("${mywebapp.config.dir}/application.properties")
public class SpringConfig {
}
And just path JVM args at the startup of tomcat like
-Dmywebapp.config.dir=file:/etc/mywebapp
This way you can deploy one application.properties different for each environment.
As for your comment:
#PropertySource(value={"file:C:/Users/foo/apache-tomcat-8.5.28/webapps/application.properties}"})
as it's a windows system path may you have to double backslash your path: C:\\Users\\foo ...

I am able to read the properties file from tomcat/webapp location
for every environment I can ask system engineers to drop the file at the location,
I dont know if its a good solution. Please suggest
#PropertySource(value={"file:C:/Users/foo/Downloads/apache-tomcat-8.5.28/webapps/application.properties"})

Spring boot external configuration of property file

I have a spring boot application that I can package in a war that I want to deploy to different environments. To automate this deployment it'd be easier to have the configuration file externalized.
Currently everything works fine with a application.properties file in src/main/resources. Then I use ´mvn install´ to build a war deployable to tomcat.
But I would like to use a .yml file that does not need to be present on mvn install but that would be read from during deployment of the war and is in the same or a directory relative to my war.
24. externalized configuration shows where spring boot will look for files and 72.3 Change the location of external properties of an application gives more detail on how to configure this but I just do not understand how to translate this to my code.
My application class looks like this:
package be.ugent.lca;
Updated below
Do I need to add a #PropertySource to this file? How would I refer to a certain relative path?
I feel like it's probably documented in there as most spring boot documentation but I just don't understand how they mean me to do this.
EDIT
Not sure if this should be a separate issue but I think it's still related.
Upon setting the os variable the error of yaml file not found went away. Yet I still get the same error again as when I had no application .properties or .yml file.
Application now looks like this:
#Configuration
**#PropertySource("file:${application_home}/application.yml")**
#ComponentScan({"be.ugent.lca","be.ugent.sherpa.configuration"})
#EnableAutoConfiguration
#EnableSpringDataWebSupport
public class Application extends SpringBootServletInitializer{
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
The application_home OS variable
$ echo $application_home
C:\Masterproef\clones\la15-lca-web\rest-service\target
My application.yml file(part it complains about):
sherpa:
package:
base: be.ugent.lca
Error upon java -jar *.war
All variations upon:
Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'sherpa.package.base' in string value "${sherpa.package.base}"
at org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:174)
at org.springframework.util.PropertyPlaceholderHelper.replacePlaceholders(PropertyPlaceholderHelper.java:126)
at org.springframework.core.env.AbstractPropertyResolver.doResolvePlaceholders(AbstractPropertyResolver.java:204)
at org.springframework.core.env.AbstractPropertyResolver.resolveRequiredPlaceholders(AbstractPropertyResolver.java:178)
at org.springframework.context.support.PropertySourcesPlaceholderConfigurer$2.resolveStringValue(PropertySourcesPlaceholderConfigurer.java:172)
at org.springframework.beans.factory.support.AbstractBeanFactory.resolveEmbeddedValue(AbstractBeanFactory.java:808)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1027)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1014)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:545)
... 142 more

Using external properties files
The answer lies in the Spring Boot Docs, I'll try to break it down for you.
First of all, no you should not use #PropertySource when working with Yaml configuration, as mentioned here under the Yaml shortcomings :
YAML files can’t be loaded via the #PropertySource annotation. So in the case that you need to load values that way, you need to use a properties file.
So, how to load propery files? That is explained here Application Property Files
One is loaded for you: application.yml , place it in one of the directories as mentioned in the link above. This is great for your general configuration.
Now for your environment specific configuration (and stuff like passwords) you want to use external property files, how to do that is also explained in that section :
If you don’t like application.properties as the configuration file name you can switch to another by specifying a spring.config.name environment property. You can also refer to an explicit location using the spring.config.location environment property (comma-separated list of directory locations, or file paths).
So you use the spring.config.location environment property.
Imagine you have an external config file: application-external.yml in the conf/ dir under your home directory, just add it like this:
-Dspring.config.location=file:${home}/conf/application-external.yml as a startup parameter of your JVM.
If you have multiple files, just seperate them with a comma. Note that you can easily use external properties like this to overwrite properties, not just add them.
I would advice to test this by getting your application to work with just your internal application.yml file , and then overwrite a (test) property in your external properties file and log the value of it somewhere.
Bind Yaml properties to objects
When working with Yaml properties I usually load them with #ConfigurationProperties, which is great when working with for example lists or a more complex property structure. (Which is why you should use Yaml properties, for straightforward properties you are maybe better of using regular property files). Read this for more information: Type-Safe Configuration properties
Extra: loading these properties in IntelliJ, Maven and JUnit tests
Sometimes you want to load these properties in your maven builds or when performing tests. Or just for local development with your IDE
If you use IntelliJ for development you can easily add this by adding it to your Tomcat Run Configuration : "Run" -> "Edit Configurations" , select your run configuration under "Tomcat Server" , check the Server tab and add it under "VM Options".
To use external configuration files in your Maven build : configure the maven surefire plugin like this in your pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Dspring.config.location=file:${home}/conf/application-external.yml</argLine>
</configuration>
</plugin>
When running JUnit tests in IntelliJ:
Run → Edit Configurations
Defaults → JUnit
add VM Options -> -ea -Dspring.config.location=file:${home}/conf/application-external.yml

Yes, you need to use #PropertySource as shown below.
The important point here is that you need to provide the application_home property (or choose any other name) as OS environment variable or System property or you can pass as a command line argument while launching Spring boot. This property tells where the configuration file (.properties or .yaml) is exactly located (example: /usr/local/my_project/ etc..)
#Configuration
#PropertySource("file:${application_home}config.properties")//or specify yaml file
#ComponentScan({"be.ugent.lca","be.ugent.sherpa.configuration"})
#EnableAutoConfiguration
#EnableSpringDataWebSupport
public class Application extends SpringBootServletInitializer{
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}

There is a very simple way to achieve this.
Inside your original application.properties file you can just specify the following line:
spring.config.import=file:Directory_To_The_File/Property_Name.properties
It will automatically sync all the properties from the external property file.
Now lets say that you have a situation where you need to get properties from multiple property files. In that case, you can mention the same line in the external property file which in turn will take the remaining properties from the second property file and so on.
Consider the following example.
application.properties:
spring.config.import=file:Resources/Custom1.properties
Custom1.properties:
server.port=8090
.
.
.
spring.config.import=file:Resources/Custom2.properties

One of the easiest way to use externalized property file using system environment variable is, in application.properties file you can use following syntax:
spring.datasource.url = ${OPENSHIFT_MYSQL_DB_HOST}:${OPENSHIFT_MYSQL_DB_PORT}/"nameofDB"
spring.datasource.username = ${OPENSHIFT_MYSQL_DB_USERNAME}
spring.datasource.password = ${OPENSHIFT_MYSQL_DB_PORT}
Now, declare above used environment variables,
export OPENSHIFT_MYSQL_DB_HOST="jdbc:mysql://localhost"
export OPENSHIFT_MYSQL_DB_PORT="3306"
export OPENSHIFT_MYSQL_DB_USERNAME="root"
export OPENSHIFT_MYSQL_DB_PASSWORD="123asd"
This way you can use different value for same variable in different environments.

Use below code in your boot class:
#PropertySource({"classpath:omnicell-health.properties"})
use below code in your controller:
#Autowired
private Environment env;

How to externalize application.properties in Tomcat webserver for Spring?

SpringApplication will load properties from application.properties
files in the following locations and add them to the Spring
Environment:
- A /config subdirectory of the current directory.
- The current directory
- A classpath /config package
- The classpath root
The list is ordered by precedence (properties defined in locations
higher in the list override those defined in lower locations).
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html#boot-features-external-config-application-property-files
Question: when running a war file on a tomcat server: how can I add an additional location for the application.properties outside the classpath or the tomcat container, like d:\application.properties?
The custom location should get highest precedence regarding the locations above.
Problem is: I could of course add a /config folder inside my exploded war in the tomcat webapps folder, but then I'd lose any custom configuration if the webapps folder is cleaned and war is redeployed.
Thus I'd like to add an additional location outside.

For me the easiest way to do it, was to place a context file inside Tomcat's config folder. For example if your application is running under root path (eg. http://your_domain.com/) you need to create a file [path_to_your_tomcat]/conf/Catalina/localhost/ROOT.xml. If your application runs in a different path, for example http://your_domain.com/example_path the file should be named like this [path_to_your_tomcat]/conf/Catalina/localhost/example_path.xml. Inside this file you can specify a path to the external application.properties file that can be placed anywhere on your hard drive.
<?xml version="1.0" encoding="UTF-8"?>
<Context>
<Environment name="spring.config.location" value="file:/path/to/your/application/properties/file/" type="java.lang.String"/>
</Context>

You can set a spring_config_location environment variable pointing to the folder that contains your application.properties file.
In the case of Tomcat you can do this by adding the following line to your <TOMCAT_HOME>/bin/setenv.sh file (create the file if missing):
export spring_config_location=/usr/local/tomcat/conf/
Place the properties file in that folder. In case you have multiple apps you can set the name of the properties file of each app to be unique. For a Spring Boot App I have done it like this:
#SpringBootApplication
public class MyApplication {
public static void main(String[] args) {
System.setProperty("spring.config.name", "my-app");
SpringApplication.run(MyApplication.class, args);
}
}
This will pick the new name when run with BOOT. To have the name configured when deployed on Tomcat too, overwrite configure of SpringBootServletInitializer like so:
public class ServletInitializer extends SpringBootServletInitializer {
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
return application.sources(MyApplication.class).properties("spring.config.name: my-app");
}
}
Then name your properties file like: my-app.properties. Instead of the default name Spring will look for that. You can put all your apps properties files in the specified folder, /usr/local/tomcat/conf/ in our sample. Your external properties will get precedence. See here for priorities: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html
UPDATE
Since Spring Boot 2 the behavior of spring_config_location has changed (from the migration guide):
it previously added a location to the list of default ones, now it
replaces the default locations. If you were relying on the way it was
handled previously, you should now use
spring.config.additional-location instead.
So based on your use case you should consider which of the properties to set as an environment variable. The new one should look like spring_config_additional-location in setenv.sh.
Where files are looked up from is described in the reference documentation too.

In tomcat/bin create setenv.sh file and in file you need to give below line in file for default properties for all the wars, And save it and restart tomcat.
export SPRING_PROFILES_ACTIVE=dev

I had to do it several times and the best approach I found is to configure an external directory as classpath resource in the container:
Then, in the directory place the resources you want to externalize and everything will run fine. To load the resource in spring, you could do this way:
<beans:bean id="externalProperties" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<beans:property name="location" value="classpath:[my-application-name]/applicationProperties.properties" />
<beans:property name="placeholderPrefix" value="!applicationProperties{" />
<beans:property name="placeholderSuffix" value="}" />
</beans:bean>
You can see that, as you said that you might want to have multiple applications deployed in each tomcat, you could simply create a directory structure in the folder you set in classpath, to maintain different application.properties for each of your war applications
In case you want to maintain dinamically the application name section in your Spring configuration, you could do it in several ways, during packaging stage in maven or even using the application context path

I finally added the following property to externalize eg secure properties:
spring.config.additional-location=/etc/tomcat/<appname>/application-production.properties

if anyone is looking for a linux solution, This is working for us:
edit tomcat startup.sh
add:
export spring_config_location=/<YOUR_PATH>/application.properties
example:
export spring_config_location=/app/conf/application.properties

for tomcat 9 on Ubuntu 18.04 and spring boot 2, create setenv.sh file under $CATALINA_HOME/bin/ working for me:
#!/bin/bash
export spring_config_additional_location="/opt/tomcat/latest/conf/application.properties"
don't forget to set file permission if needed

Using Tomcat 9.0.27
Running Spring boot 2.2.1
I did setup setenv.sh and added the line
export spring_config_location=/<PATH_TO_CONF_DIR>/application.properties
and got it to work..