How to use environment variables in flyway config file? - java

I have Flyway config file - flyway.properties , which contains basic database connection parameters:
flyway.url=jdbc:mysql://localhost
flyway.user=test
flyway.password=test
flyway.schemas=testdb
As I know exposing parameters in such config files is a bad practice. Is it possible to use environment parameters(to create .env file to define params there and to receive them in flyway config?)

You should be able to pass to the config file every possible property, including connections and passwords, so you don't have to store them in the config file. Something like this:
flyway -enterprise -url=jdbc:postgresql://localhost:5498/hamshackradio -user=postgres -password=dude1988 -configFiles="./conf/flyway.conf" migrate
That way you can use the environment variables from the command line much easier. This is the complete list of parameters. If you're calling the API, you can also use envVars(), but I don't have experience with that.

As well as using environment variables, you can use the flyway.conf file in the user profile folder, which only the current user can access. This is fine if you only access one database/server combination.
If you use several, then you can reference them as an extra .conf file in the user profile/user home area. For example, if, as a Windows user, you were accessing the development branch of a Pubs database project you might create a PubsDevelop.conf file in your user profile folder and specify it to Flyway by using a commandline parameter
flyway -configFiles="%userprofile%\PubsDevelop.conf" info
or if you dislike typing commands, set it up as an environemt variable
FLYWAY_CONFIG_FILES=%userprofile%\PubsDevelop.conf
That way, nobody can see your credentials or server name, and there is no risk of it leaking into the source control system by mistake!

Related

How can I externalize some config variables in spring boot?

I have a yaml file that contains some credentials for azure keyvault. This project is I am working on is a shared git repo so I would like to set these values as environment variables for the whole project not just for myself
application.yml:
azure:
keyvault:
uri:someUri
client-id:someClientId
client-key:someClientKey
but I want to have them set up like this:
azure:
keyvault:
uri: ${uri}
client-id:${clientId}
client-key:${clientKey}
Is there a way to set those values and have this work for others without them having to manually set these values in their environment?
You can ignore the .yaml in gitignore so you can use your config in your environment. And if you pull from others their config won't overwrite your config.
If you want if like dynamically
java -jar myapp.jar --spring.application.json='{"foo":"bar"}'
You can add variables like this in start command. Configure in your Run configuration.
The syntax you've presented above is supposed to work.
I usually use capital letters for 'convention' but other than that its fine.
However providing such a syntax effectively means that you don't want to define property values in your application so you'll have to get them from elsewhere.
From your question I understand is that your primary concern is your teammates and in your setup each one of your peers has clientId, key, etc.
In this case you can create a script that will "export" all the variables to be environment variables automatically, they'll run it only one time and it will work since than.
Another option which is kind of similar is providing a property SPRING_APPLICATION_JSON that will refer to the json with a configuration, spring boot application can read it as written in the official documentation

Running one file on multiple servers

Right now my team deals with about 4-5 different servers and about 2-3 different DB servers and we're using environmental variables to decide which server we're on and what server configuration to use.
Is there a better way to do this as my team continues to expand? I've considered compiler flags / args, but it doesn't seem as robust.
From my perspective, in java, you have basically 3 ways to crack this cookie:
Environment variables
-D JVM parameters (which are System Properties)
properties files
You've already discovered Environment Variables and that is pretty much "the unix way" to get the effect you are after; different configuration to common binary that customizes the running application for the environment it is executing on.
System Properties are really the Java "moral equivalent" of Environment Variables. They come in via -D parameters on your application's command line like...
java -Dlogback.configurationFile=/opt/dm/logback.xml -cp app.jar org.rekdev.App
Explicit Properties file processing http://docs.oracle.com/javase/tutorial/essential/environment/properties.html in Java is a third variant which you often see coupled with -D to get something like default behavior which can be overridden at runtime from the command line. That is what is basically going on with the logback.xml configuration above, the JAR file has a logback.xml inside it that will be used unless a System Property called "logback.configurationFile" exists, at which point the App will load it instead.
As you try to figure out how to keep this all in sync and working correctly in a multi-server environment, consider the use of chef http://wiki.opscode.com/display/chef/Home to do the deployments and put each specific environment's customizations under chefs control. Put the chef "recipes" in version control and, voila, full on configuration management.
SHIP IT!
I can see two scenarios
You embed all the different properties within your package (can be a war, ear, jar, or on the file system /yourapp/etc/)
You embed only one property file and this one is created during build (with ant or maven)
Say your app is named foo
Solution 1
It has the advantage that your app can be put as-is on any of the supported servers (all that have a property file in your app package).
Your properties will be named foo.dev.properties, foo.test.properties, foo.prod.properties, foo.damien.properties, foo.bob.properties.
One other advantage is that every developer working has its own dev file that he can safely push on svn/git/whatever and be sure that other developer won't destroy his config.
At runtime the application can check the -D parameter or even retrieve the hostname dinamycally, in order to load the correct property file.
Solution 2
It has the advantage that your package is not polluted by unnecessary properties files.
You must configure a lot of ant tasks/maven target in order to build for specific environment. You will have in your source directory the properties files for the environments also, but only one will be shipped with your app. This one foo.properties will only have placeholders for values, and values will be inferred within it using foo.ENV.properties with the right ant task/maven target.
At my actual job and the previous one also, we did use the solution 1, I think it brings flexibility.
Some parameter (like database user/password) were fetched directly from environment variables on the Unix servers though (so that only the server admins knew the credentials).
You can safely mix the solutions, in order to get where you feel there is the more flexibility for you and your team.

Good repository organization practice for Java properties files

I maintain a multi-platform development framework that attempts to configure environment variables based on certain system and environment information that is inferred via various means. Once I infer these variables, I store them in a Java properties file for later use. This file could also be edited by the user of my framework (a developer).
I have a class called Env that manages this properties file, and it's in a package called org.myproject.config. I'm currently storing the default properties file in src/org/myproject/config. I'm wondering, is it good practice to store a config-type file under this directory of my project? Not sure where to go with this. Any input is appreciated.
-tjw
You should use the Java Preferences API if you wish to maintain platform neutrality. Once you specify a file path, you make a lot of assumptions about the environment your application is running on.
From what we experienced, environment specific information should not be stored on a repository. Someone updates by mistake an file and when he commits other users/environments are affected. We keep environment configuration separate and local on each environment. In our case the environment variables are stored in database tables, and the only environment information we have in property files is the minimum required fields to create a JDBC connection to the location of the environment variables. updates to environments pass through a IT change request or are done via an software upgrade.

Axis2 webservice (aar archive) properties file

I'm currently developing a set of SOAP webservices over Axis2, deployed over a clustered WebLogic 10.3.2 environment.
My webservices use some user settings that I want to be editable without the need for recompiling and regenerating the AAR archive. With this in mind, I chose to put them into a properties file that is loaded and consumed in runtime.
Unfortunately, I'm having some questions about this:
As far as I know, to achieve what I want, the only option is to put the properties file into the ../axis2/WEB-INF/classes directory of each one of the deployments (on each WebLogic instance) I currently have on my clustered configuration, and then load the file, as follows (or equivalent, this has not been verified for optimization):
InputStreamReader fMainProp = new InputStreamReader(this.getClass().getResourceAsStream("myfile.properties"));
Properties mainProp = new Properties();
mainProp.load(fMainProp);
This is not as practical as I wanted it to be, because each time I want to alter some setting on the properties file, I have to edit each one of the files (deployed over different WebLogic instances) and there is a high probability of modifying one of these files without modifying the others.
What I would like to know is if there is any (better) alternative to accomplish what I want, minimizing the potential conflict of configuration that is created by distributing and replicating the properties file through multiple WebLogic instances.
Your usecase suits use of JMX.
Please see this link for details of using JMX to build custom Mbeans to manage application configurations.
http://blogs.oracle.com/WebLogicServer/2009/10/developing_custom_mbeans_to_ma.html
Then you can either opt for console or WLST script based access.
You can set a System Variable in the Weblogic to hold on the absolute property file location. At the weblogic installation you will find setDomainEnv.cmd (Windows) or setDomainEnv.sh (Linux). Inside that add (append) to JAVA_OPTIONS the system variable you need. For example
-DpropLocation=C:/somewhere/The.properties
In your code you can get the property file location by System.getProperties("propLocation"). You can add multiple System variable by adding spaces in between them.

How to Use Different META-INF/context.xml Files For Development and Production Environments

In Tomcat (and some other servlet containers) I can store information about my JDBC DataSource in META-INF/context.xml. This is very useful.
However, the settings for my JDBC DataSource can be different in my development and production environments. I'd like to know how other people deal with these differences in an elegant way, specifically how can I set up a context.xml for my development environment and one for my production environment in the most hassle-free manner.
You can create different files for specific builds. For example, create:
development.context.xml
production.context.xml
Then, you can control which context file is used in your build.xml file. Basically, setup a prompt for which type of build you would like to use. When you select development, it uses the development context file. When you select production, it uses the production context file.
I would do the same as Kevin mentioned. If you're using Maven you would use "profiles".
If you want to learn more about Maven profiles read this: Introduction to Build Profiles
Personally I wouldn't store configuration information like that in context.xml (perhaps in another properties file or something), but the general way for something like this is to have your build script package different versions of the configuration file into the WAR/EAR/whatever. You could have your build script decide whether to use the "dev" or "production" configuration file based on parameters you pass in, running different targets, etc.
Something I use often is the task in ant to replace certain tokens in files with values from a filters file; and swap which filters file is used depending on which environment I am targeting.

Categories

Resources