I maintain a multi-platform development framework that attempts to configure environment variables based on certain system and environment information that is inferred via various means. Once I infer these variables, I store them in a Java properties file for later use. This file could also be edited by the user of my framework (a developer).
I have a class called Env that manages this properties file, and it's in a package called org.myproject.config. I'm currently storing the default properties file in src/org/myproject/config. I'm wondering, is it good practice to store a config-type file under this directory of my project? Not sure where to go with this. Any input is appreciated.
-tjw
You should use the Java Preferences API if you wish to maintain platform neutrality. Once you specify a file path, you make a lot of assumptions about the environment your application is running on.
From what we experienced, environment specific information should not be stored on a repository. Someone updates by mistake an file and when he commits other users/environments are affected. We keep environment configuration separate and local on each environment. In our case the environment variables are stored in database tables, and the only environment information we have in property files is the minimum required fields to create a JDBC connection to the location of the environment variables. updates to environments pass through a IT change request or are done via an software upgrade.
Related
I have Flyway config file - flyway.properties , which contains basic database connection parameters:
flyway.url=jdbc:mysql://localhost
flyway.user=test
flyway.password=test
flyway.schemas=testdb
As I know exposing parameters in such config files is a bad practice. Is it possible to use environment parameters(to create .env file to define params there and to receive them in flyway config?)
You should be able to pass to the config file every possible property, including connections and passwords, so you don't have to store them in the config file. Something like this:
flyway -enterprise -url=jdbc:postgresql://localhost:5498/hamshackradio -user=postgres -password=dude1988 -configFiles="./conf/flyway.conf" migrate
That way you can use the environment variables from the command line much easier. This is the complete list of parameters. If you're calling the API, you can also use envVars(), but I don't have experience with that.
As well as using environment variables, you can use the flyway.conf file in the user profile folder, which only the current user can access. This is fine if you only access one database/server combination.
If you use several, then you can reference them as an extra .conf file in the user profile/user home area. For example, if, as a Windows user, you were accessing the development branch of a Pubs database project you might create a PubsDevelop.conf file in your user profile folder and specify it to Flyway by using a commandline parameter
flyway -configFiles="%userprofile%\PubsDevelop.conf" info
or if you dislike typing commands, set it up as an environemt variable
FLYWAY_CONFIG_FILES=%userprofile%\PubsDevelop.conf
That way, nobody can see your credentials or server name, and there is no risk of it leaking into the source control system by mistake!
Right now my team deals with about 4-5 different servers and about 2-3 different DB servers and we're using environmental variables to decide which server we're on and what server configuration to use.
Is there a better way to do this as my team continues to expand? I've considered compiler flags / args, but it doesn't seem as robust.
From my perspective, in java, you have basically 3 ways to crack this cookie:
Environment variables
-D JVM parameters (which are System Properties)
properties files
You've already discovered Environment Variables and that is pretty much "the unix way" to get the effect you are after; different configuration to common binary that customizes the running application for the environment it is executing on.
System Properties are really the Java "moral equivalent" of Environment Variables. They come in via -D parameters on your application's command line like...
java -Dlogback.configurationFile=/opt/dm/logback.xml -cp app.jar org.rekdev.App
Explicit Properties file processing http://docs.oracle.com/javase/tutorial/essential/environment/properties.html in Java is a third variant which you often see coupled with -D to get something like default behavior which can be overridden at runtime from the command line. That is what is basically going on with the logback.xml configuration above, the JAR file has a logback.xml inside it that will be used unless a System Property called "logback.configurationFile" exists, at which point the App will load it instead.
As you try to figure out how to keep this all in sync and working correctly in a multi-server environment, consider the use of chef http://wiki.opscode.com/display/chef/Home to do the deployments and put each specific environment's customizations under chefs control. Put the chef "recipes" in version control and, voila, full on configuration management.
SHIP IT!
I can see two scenarios
You embed all the different properties within your package (can be a war, ear, jar, or on the file system /yourapp/etc/)
You embed only one property file and this one is created during build (with ant or maven)
Say your app is named foo
Solution 1
It has the advantage that your app can be put as-is on any of the supported servers (all that have a property file in your app package).
Your properties will be named foo.dev.properties, foo.test.properties, foo.prod.properties, foo.damien.properties, foo.bob.properties.
One other advantage is that every developer working has its own dev file that he can safely push on svn/git/whatever and be sure that other developer won't destroy his config.
At runtime the application can check the -D parameter or even retrieve the hostname dinamycally, in order to load the correct property file.
Solution 2
It has the advantage that your package is not polluted by unnecessary properties files.
You must configure a lot of ant tasks/maven target in order to build for specific environment. You will have in your source directory the properties files for the environments also, but only one will be shipped with your app. This one foo.properties will only have placeholders for values, and values will be inferred within it using foo.ENV.properties with the right ant task/maven target.
At my actual job and the previous one also, we did use the solution 1, I think it brings flexibility.
Some parameter (like database user/password) were fetched directly from environment variables on the Unix servers though (so that only the server admins knew the credentials).
You can safely mix the solutions, in order to get where you feel there is the more flexibility for you and your team.
In Tomcat (and some other servlet containers) I can store information about my JDBC DataSource in META-INF/context.xml. This is very useful.
However, the settings for my JDBC DataSource can be different in my development and production environments. I'd like to know how other people deal with these differences in an elegant way, specifically how can I set up a context.xml for my development environment and one for my production environment in the most hassle-free manner.
You can create different files for specific builds. For example, create:
development.context.xml
production.context.xml
Then, you can control which context file is used in your build.xml file. Basically, setup a prompt for which type of build you would like to use. When you select development, it uses the development context file. When you select production, it uses the production context file.
I would do the same as Kevin mentioned. If you're using Maven you would use "profiles".
If you want to learn more about Maven profiles read this: Introduction to Build Profiles
Personally I wouldn't store configuration information like that in context.xml (perhaps in another properties file or something), but the general way for something like this is to have your build script package different versions of the configuration file into the WAR/EAR/whatever. You could have your build script decide whether to use the "dev" or "production" configuration file based on parameters you pass in, running different targets, etc.
Something I use often is the task in ant to replace certain tokens in files with values from a filters file; and swap which filters file is used depending on which environment I am targeting.
Where in a Windows (Vista) system should I place data that ought to be readable and writable by everyone, i.e. every user of the computer? Vista's concepts of C:\Users\xxx\AppData\something, C:\Program Files and C:\ProgramData directories and UAC are a bit confusing.
Furthermore, is there any ready solution to determine those locations with Java? I suppose that it requires some interaction with native libraries, since System.getProperties has just user.home and user.dir, neither of which is globally writable.
In vista c:\ProgramData is the place, this replaces what used to be C:\Documents and Settings\AllUsers\AppData in XP.
I'm not sure about the specifics of doing this in java.. but, the ALLUSERSPROFILE environment variable gives you the path if you can get hold of that. You should always use this instead of hard coding the path, because the folder name changes on different internationalized versions of the OS.
If you need to allow users that do not have administrator privileges to modify the global settings then the proper approach is to create an installer for the application and during the install set the permissions on the "Common Application Data" folder such that users area allowed to write to it.
See this post: Where to put common writable application files?
Unless the data is really large, you might want to take a look at the Preferences API; in particular Preferences.systemNodeForPackage. This gives you a platform-independent, system-wide, backend-agnostic storage facility, if that's what you're after.
Furthermore, is there any ready
solution to determine those locations
with Java?
You can check approrpirate environment variables, for instance %PROGRAMDATA%, %PROGRAMFILES% and others.
Where do you store user-specific and machine-specific runtime configuration data for J2SE application?
(For example, C:\Users\USERNAME\AppData\Roaming</em> on Windows and /home/username on Unix)
How do you get these locations in the filesystem in platform-independent way?
First on the format:
Java property files are good for key/value pairs (also automatically handle the newline chars). A degree of structure is possible by using 'dot notation'. Drawback is that the structure does not allow you to easily enumerate top-level configuration entities and work in drill-down manner. Best used for a small set of often tweakable environment-specific settings
XML files - quite often used for more complex configuration of various Java frameworks (notably J2EE and Spring). I would advice that you at least learn about Spring - it contains many ideas worth knowing even if you decide not to use it. If you decide to roll your own XML configuration, I'd recommend using XStream with customized serialization options or if you just need to parse some XML, take a look at XOM. BTW Spring also allows you to plug your custom XML configuration language, but it's a relatively complex task. XML configuration is best used for more complex 'internal' configuration that is not seen or tweaked by the end user.
Serialized Java objects - a quick and easy way to persist the state of an object and restore it later. Useful if you write a configuration GUI and you don't care if the configuration is human readable. Beware of compatibility issues when you evolve classes.
Preferences - introduced in Java 1.4, allow you to store typed text, numbers, byte arrays and other primitives in platform-specific storage. On Windows, that is the registry (you can choose between /Software/JavaSoft/Prefs under HKLM or HKCU). Under Unix the same API creates files under the user home or /etc. Each prefs hive can be exported and imported as XML file. You can specify custom implementation of the PreferencesFactory interface by setting the "java.util.prefs.PreferencesFactory" JVM property to your implementation class name.
In general using the prefs API can be a good or a bad thing based on your app scenario.
If you plan to have multiple versions of the same code running on the same machine with different configuration, then using the Preferences API is a bad idea.
If you plan using the application in a restricted environment (Windows domain or tightly managed Unix box) you need to make sure that you have proper access to the necessary registry keys/directories. This has caught me by surprise more than once.
Beware from roaming profiles (replicated home dirs) they make up for some funny scenarios when more than one active machines are involved.
Preferences are not as obvious as a configuration file under the application's directory. most of the desktop support staff doesn't expect and doesn't like them.
Regarding the file layout of the prefs it again depends on your application. A generic suggestion is:
Package most of your XML files inside application's JAR either in the root or under /META-INF directory. These files will be read-only and are considered private for the application.
Put the user modifiable configuration under $APP_HOME/conf . It should consist mainly of properties files and occasionally a simple XML file (XStream serialization). These files are tweaked as part of the installation process and are usually not user serviceable.
Under the user-home, in a dot-directory (i.e. '~/.myapplication') store any user configuration. The user configuration may override the one in the application conf directory. Any changes made from within the application go here (see also next point).
You can also use an $APP_HOME/var directory to store any other mutable data which is specific to this application instance (as opposed to the user). Another advantage of this approach is that you can move and backup the whole application and it's configuration by simple copy of one directory.
This illustrates some standard techniques for managing configuration. You can implement them using different libraries and tools, starting from raw JRE, adding Spring/Guice or going for a full J2EE container (possibly with embedded Spring)
Other approaches for managing configuration are:
Using multiple base directories for running multiple instances of the application using different configurations.
Using lightweight registries for centralized configuration management
A centrally managed Configuration Management Database (CMDB) file, containing the host-specific values for each machine is rsync-ed every night to all production hosts. The application uses a templated configuration and selects from the CMDB during runtime based on the current hostname.
That depends on your kind of J2SE Application:
J2SE executable JAR file (very simple): use user.home System property to find home-dir. Then make a subdir accordingly (like e.g. PGP, SVN, ... do)
Java Web Start provides very nice included methods to safe properties. Always user-specific
Finally Eclipse RCP: There you have the notion of the workspace (also derived from user.home) for users and configuration (not totally sure how to access that tricky in Vista) for computer wide usage
All these approaches are, when used with care -- use correct separatorChar -- OS neutral.
Java has a library specifically for doing this in java.util.prefs.Preferences.
Preferences userPrefs = Preferences.getUserNodeForPackage(MyClass.class); // Gets user preferences node for MyClass
Preferences systemPrefs = Preferences.getSysteNodeForPackage(MyClass.class); // Gets system preferences node for MyClass
Preferences userPrefsRoot = Preferences.getUserRoot(); // Gets user preferences root node
Preferences systemPrefsRoot = Preferences.getSystemRoot(); // Gets system preferences root node
I use this
String pathFile = null;
if(OS.contains("win")){
pathFile = System.getenv("AppData");
}else{
pathFile = System.getProperty("user.home");
}
I save the settings of my application here
C:\Users\USERNAME\AppData\ on windows
user.home (/home/USERNAME) on other platfroms
For user specific config, you could write a config file to the folder pointed to by the "user.home" system property. Would only work on that machine of course.
You might want to look at Resource Bundles.