Develop Reference

Is there a typical way to handle separate configurations for different tomcat instances in a Java app?

Our system is split up into different environments, each one as a separate Tomcat instance.
Development (Windows)
QA: Accessed by our QA department (Linux)
Production: Live site, accessible to customers. (Linux)
Each of these environments rely on separate databases, and several other web services. This means we need to keep track of the various URLs, usernames, and passwords that are all different. Many of these settings are shared across several apps, so it would be ideal to have them all in one place to remove duplication.
Right now we have config files in the application itself. We use Maven profiles to fill in the different config settings when we build the app. But this is clumsy because we have to build a different WAR for each environment.
Where is a good place to store the config files so that we can deploy the same WAR file to each server?
I've done a fair amount of research on this already. But I haven't found an solution that completely makes sense to me yet.
Separate Config Directory
Define a directory to hold config files. Such as /opt/config on linux.
I like this idea, but how do I tell Tomcat where this directory is? I see references to context.xml, but every example I've seen puts the context.xml in the META-INF folder inside the WAR. Is there a way to configure this outside the WAR?
System Property to define environment
This involves setting a system property, and then using some sort of if/else or switching logic to load the appropriate config file. This seems workable, but a bit messy. But where/how do you set this property? I typically start tomcat with ./startup.sh. Do I add arguments to that command or is there another configuration somewhere?
JNDI
I don't think this is an option for us. Every tutorial I've looked at for this seems to be dependent on LDAP or something similar. To my knowledge we don't have that available to us, and it seems like too much overhead to get set up for only a half-dozen config files.

Use system property that refers to the location where your configuration file or directory is located. In this case you can manage different environment easily and no if/else logic is needed.
You application can have hard coded value of config file path, that will allow running application without any additional system property. The application however should fail to start if mandatory data is not found.
Concerning to partial sharing of data among environments.
You can split your data into several files by categories. Some files will be shared, some other different for different environments. You can even develop your own mechanism of references between data files.
However better approach is using some ready-to-use packages. For example Spring framework supports very flexible configuration mechanism. However if you already have Spring-less application introducing this framework for configuration only seems like an overkill. In this case take a look on Apache Commons Configuration package.

Worked with my team on this and we came up with what we feel is a cleaner approach. While every tutorial I found put the context.xml inside the WAR, it can also be placed in the conf folder of the Tomcat directory.
This works for us as all our servers are Tomcat based. So each server can have it's own context.xml which has a property pointing to the config folder on that particular server.

Making an open source Java webapp easy to install and deploy on Tomcat, Jetty etc

I'm distributing a simple Java webapp as open source. The webapp needs to be configured before it can be run – a configuration file needs to be created, and the location of that configuration file needs to be made known to the webapp as a parameter in web.xml.
Now my question is how to best package and distribute the webapp in order to make it easy to install, and how to describe that installation process in the documentation. The options I can think of are:
Distribute the webapp as a WAR archive. Recommend that users deploy the WAR into their Tomcat/Jetty/whatever, then drop their configuration file into /webapps/myapp/WEB-INF, and modify /webapps/myapp/WEB-INF/web.xml accordingly
Distribute the webapp as source. Recommend that users should drop their configuration file into the /src/main/webapp/WEB-INF folder, then modify their /src/main/webapp/WEB-INF/web.xml accordingly, then build a WAR using Ant or Maven, and deploy that into their servlet container.
There are probably other options that I can't think of.
What setup is likely to be most convenient to users that need to install, configure and deploy the webapp?
Edit: I should add that the configuration file isn't just a few lines – it's a large file that contains a database schema and other stuff and is potentially generated using an external editor; so just providing good defaults isn't an option.

Externalize this configuration and maybe provide some default values. If you make a new version of your app, everybody will have to remember to back-up that configuration file, then redeploy and then copy back that file--> this is a nightmare.
There are many ways to put that configuration somewhere else. You can use Java Preferences for example.

I would say the WAR, although not requiring the configuration would likely be more convenient :)
What is it, loosely, that must be configured such that there isn't a sensible default value for everyone? URL string?

Providing an answer of my own, after more reading on the issue: JNDI seems to be the “official”, although somewhat heavyweight, way of solving this. With JNDI, a configuration option (like the location of the full config file I need) can be declared in the web.xml, and its actual value can be set in a per-webapp context.xml that lives in the /webapps directory of Tomcat (or the /contexts directory of Jetty). This setup has a bunch of advantages:
The big configuration file can live outside of the servlet container and webapp
The webapp can be updated without danger of losing the configuration
The distributed war doesn't need to be modified or rebuilt
Downside: It's sort of complicated, requires messing around with XML, and configuring JNDI on Tomcat works differently from Jetty (requiring twice as much documentation).

Maybe use a system property for the config file location. Can easily be passed on the command line as -Dorg.example.config.file=/foo/bar, in startup scripts or in Java code. I think I've seen some tools, e.g. logging frameworks, use system properties for similar things in webapps.

Java EE EAR shared location of read/write resources within clustered environment

Within a Java EE environment (happens to be WAS 6.1 but could be any application server) I need to place a XML file, which is a configuration file, so that I can read and write to it.
This needs to be available in a clustered environment so I am looking at using the class path to load the file.
I am thinking I can store this file in the EAR root, reference it in the manifest and then load and save it.
I have tried this approach by having my file in a JAR and making this available via the MANIFES and I can load the config file from the class path no problem using the following.
this.getClass().getClassLoader().getResourceAsStream("configFileName");
That loads the file that is in the JAR, which is fantastic. But if I want to edit this file, programmatically, I cannot access the JAR files location (the EAR root) it returns me an interpreted path like this:
/usr/IBM/WebSphere/AppServer/profiles/AppSrv01/installedApps/localhostNode01Cell/MyApp.ear/MyApp.war/TB_config.jar
That is not the correct location of the JAR the correct location is at MyApp.ear.
So the question is: how can I access and update (copy contents, create new, save, delete old) the JAR with my config file.
Or should I put the config file somewhere else?
What is the standard Java EE to make files that need read/write access available to WARs on a cluster?

Ok I have built a solution for this. It is more WebSphere based (our platform) but it is J2EE and I am suprised it was not mentioned. Basically I have used JMX to synchronise the nodes. The files are stored, and saved to, the deployment manager the nodes are then resynchronised using JMX calls and then the engines withing the applicaitons are restarted by calling servlets within the applications.
It works a dream
So #stacker, nodes are managed and the manager distributes files to the nodes.

The problem that you've hit is not unique. A lot of Java EE programmers can struggle with providing a "configurable" property file to administrators of a cluster. And the solution that you've chosen, well, has its limitations.
The problem with embedding a config file inside a JAR, is absolute path or the physical path of the file, in case you need to update it. If your container will not explode your EAR and WAR files, then placing the config file alongside the code is a bad idea - the administrator will have to deploy a newer version of the EAR/WAR/JAR. That is unless, of course, you can configure the container to explode the artifacts - WebLogic Server does this, I'm not sure about WAS.
There are several ways to resolve this problem:
Store the config file in a SAN that is accessible to all the nodes in the cluster via a 'canonical' path. That way, you could locate the file from any node in the cluster and update it. Remind yourself to restrict access to this directory. Although this sounds simple, it need not be - Java objects might have to be 'flushed' across nodes, once the configuration file has been updated. Moreover, you might have to cater to the scenario where property files can get edited outside the application.
Use a database. Much simpler and almost hasslefree, except that the Java objects might have to be flushed again.
Use a MBean. As good as a database, except that I haven't known a lot of people vouching for the MBean support in WAS. Also, I'm not really sure if object states can go haywire across a cluster, in this case.

You cannot write to an ear file, you should place the XML file in the DB as a text lob (large object).

Actually, as I am using WebSphere, it appears I can use the dynamic cache provided by the WebSphere deployment manager. The last chapter in the link below dicusses the use of the Dynamic Cache providing a shared object in a cluster. The configuration file is XML that is parsed as such by the engine (into a Document object) of the application and so is a Java object, thus it can be placed into the DistributedMap.
Looks like a clean solution. Thanks all for reading and your replies.
http://www.ibm.com/developerworks/websphere/library/techarticles/0606_zhou/0606_zhou.html

Can I use a single war file in multiple environments? Should I?

I have a Java web application at my work and I'd like simplify how we deploy to our DEV, QA, and PROD environments.
The application reads in a series of properties at startup, and the properties files are different for dev, qa, and prod. Whenever I want to deploy to a certain environment I drop the environment-specific properties file into my app folder, build the war, and then deploy it to one of the three tomcat 5.5 servers.
What I would like to do is have to have a single .war that has the properties for all environments, and have the app interrogate the webserver during the init process to figure out which environment the app is in, and hence which properties to load. Is there an easy way (or, failing that, a standard way) to do that?

This really depends on what you are using those properties for.
Some (like data source, for example) can be configured in the container itself (Tomcat 5.5. JNDI Resources, see JDBC sources section as well).
Others (application-specific) may indeed need to be properties. In which case your choices are:
Bundle properties within WAR file and load the appropriate subset based on some external switch (either environment variable or JVM property)
Setup a deployment process on each of your servers where war is unpacked and a property file (located in a predefined location on that server and specific to that server) is copied over to WEB-INF/classes (or other appropriate place).
As far as "is this a desirable goal" goes - yes, I think so. Having a single WAR to test in QA / staging and then deploy to production cuts out an intermediate step and thus leaves less chances for mistakes.
Update (based on comment):
Item #1 above refers to an actual environment variable (e.g. something that you set via SET ENV_NAME=QA in Windows or ENV_NAME=QA; export ENV_NAME in Linux). You can the read its value from your code using System.getenv() and load the appropriate properties file:
String targetEnvironment = System.getenv("TARGET_ENV");
String resourceFileName = "/WEB-INF/configuration-" + targetEnvironment + ".properties";
InputStream is = getServletContext().getResourceAsStream(resourceFileName);
Properties configuration = new Properties();
configuration.load(is);
But yes, you can instead define a scalar value via JNDI (see Environment Entries in Tomcat doc) instead:
<Context ...>
<Environment name="TARGET_ENV" value="DEV" type="java.lang.String" override="false"/>
</Context>
and read it within your app via
Context context = (Context) InitialContext().lookup("java:comp/env");
String targetEnvironment = (String) context.lookup("TARGET_ENV");
// the rest is the same as above
The thing is, if you will be using JNDI anyway, you might as well forgo your property files and configure everything via JNDI. Your data sources will be available to you as actual resources and basic properties will remain scalars (though they will be type safe).
Ultimately it's up to you to decide which way is better for your specific needs; both have pros and cons.

What you do is an accident waiting to happen... One day a DEV war will end up in de PROD server, and by some law superior to all laws of nature that problem will be detected at 2AM. Can't explain why this is the case, but some day that will happen. So one war is in my opinion definitely a good idea.
You can set a system property in the respective JVM's (-Dcom.yourdomain.configpath=/where/you/store/configfiles) and fetch this property with
String value = System.getProperty("com.yourdomain.configpath", "defaultvalue_if_any");
The default value could point somewhere inside the war (WEB-INF/...), or if there's no default, be used to make some logging noise during load to warn for misconfiguration). Also note that this technique is not platform dependent, so you dev machine can be a Windows box and the server a Linux machine, it can cope with both. We normally create a subdir per application in this configpath, as several applications use this system on a server, and we want to keep things tidy.
As an added bonus, you don't risk to trash manually tweaked property files on a PROD server this way. Just don't forget to include the path where the files are stored in a backup scenario.

I think a single war file is a good way to go, because its nice to have confidence that the binary you tested in DEV is exactly the same as in Production. The way we do it, we keep the configurations in a separate properties file, outside the war, but in the app server's class path.
If you want to keep all the properties inside the war (which does make deployment easier, because then you don't have to also deploy a properties file), you could keep a single properties file in the classpath that identifies the server environment type, and use that to key values in the properties file within your .war file. The external properties file may also be a good way to go for maybe some high-level configurations that don't change much and are used across a number of war files.

Probably the simplest way would be to set up an environment variable that differs between the application services and use this to determine which property file to read.
Another possibility would be to store the properties in a database, and use a datasource that exists under a standard JNDI name, but points to a different place in the various environments.

I prefer the one EAR or one WAR approach. There is something re-assuring and often required from a security standpoint about taking the exact same EAR that was just tested and moving it directly into the next state (test > Production).
There are also many options besides properties files provided by the container. Often the container has a nice UI to maintain those values and resources when you are dead and gone.
There are countless examples of using a database backed ResourceBundle.
For example, the Spring framework has several mechanisms to make that happen without much effort. Starting with PropertyPlaceholderConfigurer

Set a Systemproperty at startup that points to the location of your properties file, and then in your application pull in this property and load your settings. Another thing I do is have two properties file, something like default.properties, and external.properties. They contain the same properties, but the default.properties contains the default(works most of the time settings), this file goes in the war. Then if you deploy to an env. you look for the external.properties, if found that is used, if not then you rollback to the default.properties. This provides a nice way to override properties if needed, but also have a default setup. This works in a lot of my deployments, but may not in your scenario.

Absolutely a single WAR is the best way to go. Set the resources using the same JNDI names in each environment, and if you need to detect which environment you're in for business logic purposes, use a System property on Tomcat startup.

A single build (war) is certainly the right approach.
However, when it comes to environment specific configuration, the best way to go is to ensure that all configuration .properties files should not be pushed to all the environments. e.g. PROD properties files should be copied to DEV or UAT.
Spring profiles also should be avoided as they lead to convoluted configuration management.

How do you maintain java webapps in different staging environments?

You might have a set of properties that is used on the developer machine, which varies from developer to developer, another set for a staging environment, and yet another for the production environment.
In a Spring application you may also have beans that you want to load in a local environment but not in a production environment, and vice versa.
How do you handle this? Do you use separate files, ant/maven resource filtering or other approaches?

I just put the various properties in JNDI. This way each of the servers can be configured and I can have ONE war file.
If the list of properties is large, then I'll host the properties (or XML) files on another server. I'll use JNDI to specify the URL of the file to use.
If you are creating different app files (war/ear) for each environment, then you aren't deploying the same war/ear that you are testing.
In one of my apps, we use several REST services. I just put the root url in JNDI. Then in each environment, the server can be configured to communicate with the proper REST service for that environment.

I just use different Spring XML configuration files for each machine, and make sure that all the bits of configuration data that vary between machines is referenced by beans that load from those Spring configuration files.
For example, I have a webapp that connects to a Java RMI interface of another app. My app gets the address of this other app's RMI interface via a bean that's configured in the Spring XML config file. Both my app and the other app have dev, test, and production instances, so I have three configuration files for my app -- one that corresponds to the configuration appropriate for the production instance, one for the test instance, and one for the dev instance.
Then, the only thing that I need to keep straight is which configuration file gets deployed to which machine. So far, I haven't had any problems with the strategy of creating Ant tasks that handle copying the correct configuration file into place before generating my WAR file; thus, in the above example, I have three Ant tasks, one that generates the production WAR, one that generates the dev WAR, and one that generates the test WAR. All three tasks handle copying the right config file into the right place, and then call the same next step, which is compiling the app and creating the WAR.
Hope this makes some sense...

We use properties files specific to the environments and have the ant build select the correct set when building the jars/wars.
Environment specific things can also be handled through the directory service (JNDI), depending on your app server. We use tomcat and our DataSource is defined in Tomcat's read only JNDI implementation. Spring makes the lookup very easy.
We also use the ant strategy for building different sites (differeing content, security roles, etc) from the same source project as well.
There is one thing that causes us a little trouble with this build strategy, and that is that often files and directories don't exist until the build is run, so it can make it difficult to write true integration tests (using the same spring set up as when deployed) that are runnable from within the IDE. You also miss out on some of the IDE's ability to check for the existence of files, etc.

I use Maven to filter out the resources under src/main/resources in my project. I use this in combination with property files to pull in customized attributes in my Spring-based projects.
For default builds, I have a properties file in my home directory that Maven then uses as overrides (so things like my local Tomcat install are found correctly). Test server and production server are my other profiles. A simple -Pproduction is all it then takes to build an application for my production server.

Use different properties files and use ant replace filters which will do the replacement based on environment for which the build is done.
See http://www.devrecipes.com/2009/08/14/environment-specific-configuration-for-java-applications/

Separate configuration files, stored in the source control repository and updated by hand. Typically configuration does not change radically between one version and the next so synchronization (even by hand) isn't really a major issue.
For highly scalable systems in production environments I would seriously recommend a scheme in which configuration files are kept in templates, and as part of the build script these templates are used to render "final" configuration files (all environments should use the same process).

I recently also used Maven for alternative configurations for live or staging environments. Production configuration using Maven Profiles. Hope it helps.

I use Ant's copy with a filter file.
In the directory with the config file with variables I have a directory with a file for each environment. The build script know the env and uses the correct variable file.

I have different configuration folders holding the configurations for the target deployment, and I use ANT to select the one to use during the file copy stage.

We use different ant targets for different environments. The way we do it may be a bit inelegant but it works. We will just tell certain ant targets to filter out different resource files (which is how you could exclude certain beans from being loaded), load different database properties, and load different seed data into the database. We don't really have an ant 'expert' running around but we're able to run our builds with different configurations from a single command.

One solution I have seen used is to configure the staging environment so that it is identical to the production environment. This means each environment has a VLAN with the same IP range, and machine roles on the same IP addresses (e.g. the db cluster IP is always 192.168.1.101 in each environment). The firewalls mapped external facing addresses to the web servers, so by swapping host files on your PC the same URL could be used - http://www.myapp.com/webapp/file.jsp would go to either staging or production, depending on which hosts file you had swapped in.
I'm not sure this is an ideal solution, it's quite fiddly to maintain, but it's an interesting one to note.

Caleb P and JeeBee probably have your fastest solution. Plus you don't have to setup different services or point to files on different machines. You can specify your environment either by using a ${user.name} variable or by specifying the profile in a -D argument for Ant or Maven.
Additionally in this setup, you can have a generic properties file, and overriding properties files for the specific environments. Both Ant and Maven support these capabilities.

Don't forget to investigate PropertyPlaceholderConfigurer - this is especially useful in environments where JNDI is not available