I have a maven project which generate zip generated from ant scripts. Then this zip is deployed to specific application server again using ant script. this is very complex for maintenance. Now we move to use maven for building zip, so what is standard way to doing with respect to developer and client
How to handle application server specific deployment(e.g for weblogic and jboss) Do I need to create 2 zip for each server?
How to handle global configuration parameter like database, product specific settings. Where to put them, and how it use by developer and client?
The installation of application sever is need to integrate with build cycle or what is best practice for it?
I'll try to answer based on my experience:
I use maven profiles for this. Especially since there are beans (classes) that are specific to only one app server at a time. See http://maven.apache.org/guides/introduction/introduction-to-profiles.html
maven supports placeholders. One build for every single environment: dev/qa/prod,etc. All you need is the properties file. (this is similar to properties in ant)
You do not need (usually, unless you have integration tests that are tight to the app server - and if you do, it seems wrong) an app server for the build itself.
Related
Our team is writing a server-client application where the frontend is an Angular.js single page application which uses a Spring MVC java backend. The backend serves the application files and the REST endpoints used by the browser end. We are using Maven as the main build system for the application.
We like to take advantage of require.js and r.js to minify the app at the end, and we are also using client side dependency management. Currently we are using bower to download Javascript libraries required but it doesn't feel right to me to download client dependencies to src/main/webapp since this is a source folder. However in order to avoid rebuilding the whole frontend module each time something changes in the client files, this seems the only sensible way to us. This way we can start a web server and it will automatically pick up changes without restart, but as i said this doesn't fit Maven's folder layout.
I'm experimenting with Webjars which seems a better choice in our Maven oriented build and dependency management. Because in Servlet 3.0 containers webjar resources are provided automatically on the server container path it's very easy to use and manage them. It's also possible to create a require.js config to refer to libraries contained in webjars since they are on the webserver path the same way if they were static files, the serving is being done transparently in the client applications point of view.
My only problem is that i don't know how could we achieve r.js minification with this layout, since the source files are in jar files r.js cannot access them. Also the require.js config refers to the runtime server paths which are simply not there in build time.
I see that webjars now have some integration with Require.js + Play Framework but we are not using Play just simple Spring MVC in our case. I really hope there is a way to handle this case because i like the Webjar way of client dependency handling.
You need an asset compiler / pipeline in your build process. There are probably many options but the one I know of is wro4j: http://alexo.github.io/wro4j/
I have a java web application managed and built via maven with several independent sub projects. I build them all and finally all subproject's dependencies are added to web application. Now I'm planning to separate the production and development environment. All subprojects have somehow different connection URLs (for database, web service, etc.) and specifically these URLs will change on production environment. I am planning to use Spring's placeholder to manage those URLs and other properties on different execution environments, but what do you think, I am open to use maven assembly plugin and create different jar/war files for each subproject and main project.
Have any other ideas on that subject?
Thanks in advance.
You don't want different branches, version, deliverable for different environments. You want the same deliverable for each environment.
You already mention that you use placeholders/spring configuration then use it. Put the properties file(s) somewhere on the file-system (/etc/conf/yourapp or something) and read it from that location in your app. You might want to put in some sensible defaults (like hibernate dialect, etc.).
Another thing you might want to look into is profiles as you might want to load different beans in development and production (i.e. development local configured datasource, production jndi lookup for the datasource).
I have been working on 3 Java Spring and Hibernate projects at a time. Every time if I make any changes on any of these three project then I have to create build again and manually I upload three builds to three different servers.
Is there any mechanism to avoid this manual process? Is there a tool or script that can check if I make changes to my code in Eclipse and save the files which will then commit the code and automatically build and upload the war file to the appropriate server?
This would save me a lot of time.
You need continous integration. Maven is a build tool and won't deploy on change.
Whereas a CI tool such as jenkins will listen to your code reposiotry and every time a file is commited it will call then call whatver maven command you wish.
However re-reading your question, it looks like all you really need is a hot deploy development enivronment. Which is quite easy.
If the projects are on maven, there are plugins for all major application servers.
I'm currently working on a j2ee project that's been in beta for a while now. Right now we're just hammering out some of the issues with the deployment process. Specifically, there are a number of files embedded in the war (some xml-files and .properties) that need different versions deploying depending on whether you are in a dev, testing or production environment. Stuff like loglevels, connection pools, etc.
So I was wondering how developers here structure their process for deploying webapps. Do you offload as much configuration as you can to the application server? Do you replace the settings files programmatically before deploying? Pick a version during build process? Manually edit the wars?
Also how far do you go in providing dependencies through the application servers' static libraries and how much do you put in the war themselves? All this just to get some ideas of what the common (or perhaps best) practice is at the moment.
I think that if the properties are machine/deployment specific, then they belong on the machine. If I'm going to wrap things up in a war, it should be drop-innable, which means nothing that's specific to the machine it's running on. This idea will break if the war has machine dependent properties in it.
What I like to do is build a project with a properties.example file, each machine has a .properties that lives somewhere the war can access it.
An alternative way would be to have ant tasks, e.g. for dev-war, stage-war, prod-war and have the sets of properties part of the project, baked in in the war-build. I don't like this as much because you're going to end up having things like file locations on an individual server as part of your project build.
I work in an environment where a separate server team performs the configuration of the QA and Production servers for our applications. Each application is generally deployed on two servers in QA and three servers in Production. My dev team has discovered that it is best to minimize the amount of configuration required on the server by putting as much configuration as possible in the war (or ear). This makes server configuration easier and also minimizes the chance that the server team will incorrectly configure the server.
We don't have machine-specific configuration, but we do have environment-specific configuration (Dev, QA, and Production). We have configuration files stored in the war file that are named by environment (ex. dev.properties, qa.properties, prod.properties). We put a -D property on the server VM's java command line to specify the environment (ex. java -Dapp.env=prod ...). The application can look for the app.env system property and use it to determine the name of the properties file to use.
I suppose if you have a small number of machine-specific properties then you could specify them as -D properties as well. Commons Configuration provides an easy way to combine properties files with system properties.
We configure connection pools on the server. We name the connection pool the same for every environment and simply point the servers that are assigned to each environment to the appropriate database. The application only has to know the one connection pool name.
wrt configuration files, I think Steve's answer is the best one so far. I would add the suggestion of making the external files relative to the installation path of the war file - that way you can have multiple installations of the war in the one server with different configurations.
e.g. If my dev.war gets unpacked into /opt/tomcat/webapps/dev, then I would use ServletContext.getRealPath to find the base folder and war folder name, so then the configuration files would live in ../../config/dev relative to the war, or /opt/tomcat/config/dev for absolute.
I also agree with Bill about putting as little as possible in these external configuration files. Using the database or JMX depending on your environment to store as much as it makes sense to. Apache Commons Configuration has a nice object for handling configurations backed by a database table.
Regarding libraries, I agree with unknown to have all the libs in the WEB-INF/lib folder in the war file (self-packaged). The advantage is that each installation of the application is autonomous, and you may have different builds of the war using different versions of the libraries concurrently.
The disadvantage is that it will use more memory as each web application will have its own copy of the classes, loaded by its own class loader.
If this poses a real concern, then you could put the jars in the common library folder for your servlet container ($CATALINA_HOME/lib for tomcat). All installations of your web application running on the same server have to use the same versions of the libraries though. (Actually, that's not strictly true as you could put overriding versions in the individual WEB-INF/lib folder if necessary, but that's getting pretty messy to maintain.)
I would build an automated installer for the common libraries in this case, using InstallShield or NSIS or equivalent for your operating system. Something that can make it easy to tell if you have the most up to date set of libraries, and upgrade, downgrade, etc.
I usually make two properties files:
one for app specifics (messages, internal "magic" words) embedded in the app,
the other for environment specifics (db access, log levels & paths...) exposed on each server's classpath and "sticked" (not delivered with my app). Usually I "mavenise" or "anttise" these one to put specific values, depending on the target env.
Cool guys use JMX to maintain their app conf (conf can be modified in realtime, without redeploying), but it's too complex for my needs.
Server's (static ?) libraries: I strongly discourage server library use in my apps as it adds dependency to the server:
IMO, my app must be "self-packaged": dropping my war, and that's all. I have seen wars with 20 Mbs of jars in it, and that's not disturbing for me.
A common best-practice is to limit your external dependencies to what is offered by the J2EE dogma: the J2EE API (use of Servlets, Ejbs, Jndi, JMX, JMS...). Your app has to be "server agnostic".
Putting dependencies in your app (war, ear, wathever) is self-documenting: you know what libraries your app depends on. With server libs, you have to clearly document these dependencies as they are less obvious (and soon your developers will forget this little magic).
If you upgrade your appserver, chances that the server lib you depends on will also change. AppServer editors are not supposed to maintain compatibility on their internal libs from version to version (and most of the time, they don't).
If you use a widely-used lib embedded in your appServer (jakarta commons logging, aka jcl, comes to mind) and want to ugrade it's version to get the latest features, you take the huge risk that your appServer will not support it.
If you relies on a static server object (in a static field of a server class, e.g. a Map or a log), you'll have to reboot your appserver to clean this object. You loose the ability to hot-redeploy your app (old server object will still exists between redeployments). Using appServer-wide objects (other than those defined by J2EE) can lead to subtle bugs, especially if this object is shared between multiple apps. That's why I strongly discourage the use of objects which resides in a static field of an appServer lib.
If you absolutely need "this object in this appserver's jar", try to copy the jar in your app, hoping there's no dependency on other server's jar, and checking your app's classloading policy (I take the habit to put a "parent last" classloading policy on all my apps: I'm sure I won't be "polluted" by server's jars - but I don't know if it is a "best practice").
I put all configuration in the database. The container (Tomcat, WebSphere, etc) gives me access to the initial database connection and from then on, everything comes out of the database. This allows for multiple environments, clustering, and dynamic changes without downtime (or at least without a redeploy). Especially nice is being able to change the log level on the fly (although you'll need either an admin screen or a background refresher to pick up the changes). Obviously this only works for things that aren't required to get the app started, but generally, you can get to the database pretty quickly after startup.
You might have a set of properties that is used on the developer machine, which varies from developer to developer, another set for a staging environment, and yet another for the production environment.
In a Spring application you may also have beans that you want to load in a local environment but not in a production environment, and vice versa.
How do you handle this? Do you use separate files, ant/maven resource filtering or other approaches?
I just put the various properties in JNDI. This way each of the servers can be configured and I can have ONE war file.
If the list of properties is large, then I'll host the properties (or XML) files on another server. I'll use JNDI to specify the URL of the file to use.
If you are creating different app files (war/ear) for each environment, then you aren't deploying the same war/ear that you are testing.
In one of my apps, we use several REST services. I just put the root url in JNDI. Then in each environment, the server can be configured to communicate with the proper REST service for that environment.
I just use different Spring XML configuration files for each machine, and make sure that all the bits of configuration data that vary between machines is referenced by beans that load from those Spring configuration files.
For example, I have a webapp that connects to a Java RMI interface of another app. My app gets the address of this other app's RMI interface via a bean that's configured in the Spring XML config file. Both my app and the other app have dev, test, and production instances, so I have three configuration files for my app -- one that corresponds to the configuration appropriate for the production instance, one for the test instance, and one for the dev instance.
Then, the only thing that I need to keep straight is which configuration file gets deployed to which machine. So far, I haven't had any problems with the strategy of creating Ant tasks that handle copying the correct configuration file into place before generating my WAR file; thus, in the above example, I have three Ant tasks, one that generates the production WAR, one that generates the dev WAR, and one that generates the test WAR. All three tasks handle copying the right config file into the right place, and then call the same next step, which is compiling the app and creating the WAR.
Hope this makes some sense...
We use properties files specific to the environments and have the ant build select the correct set when building the jars/wars.
Environment specific things can also be handled through the directory service (JNDI), depending on your app server. We use tomcat and our DataSource is defined in Tomcat's read only JNDI implementation. Spring makes the lookup very easy.
We also use the ant strategy for building different sites (differeing content, security roles, etc) from the same source project as well.
There is one thing that causes us a little trouble with this build strategy, and that is that often files and directories don't exist until the build is run, so it can make it difficult to write true integration tests (using the same spring set up as when deployed) that are runnable from within the IDE. You also miss out on some of the IDE's ability to check for the existence of files, etc.
I use Maven to filter out the resources under src/main/resources in my project. I use this in combination with property files to pull in customized attributes in my Spring-based projects.
For default builds, I have a properties file in my home directory that Maven then uses as overrides (so things like my local Tomcat install are found correctly). Test server and production server are my other profiles. A simple -Pproduction is all it then takes to build an application for my production server.
Use different properties files and use ant replace filters which will do the replacement based on environment for which the build is done.
See http://www.devrecipes.com/2009/08/14/environment-specific-configuration-for-java-applications/
Separate configuration files, stored in the source control repository and updated by hand. Typically configuration does not change radically between one version and the next so synchronization (even by hand) isn't really a major issue.
For highly scalable systems in production environments I would seriously recommend a scheme in which configuration files are kept in templates, and as part of the build script these templates are used to render "final" configuration files (all environments should use the same process).
I recently also used Maven for alternative configurations for live or staging environments. Production configuration using Maven Profiles. Hope it helps.
I use Ant's copy with a filter file.
In the directory with the config file with variables I have a directory with a file for each environment. The build script know the env and uses the correct variable file.
I have different configuration folders holding the configurations for the target deployment, and I use ANT to select the one to use during the file copy stage.
We use different ant targets for different environments. The way we do it may be a bit inelegant but it works. We will just tell certain ant targets to filter out different resource files (which is how you could exclude certain beans from being loaded), load different database properties, and load different seed data into the database. We don't really have an ant 'expert' running around but we're able to run our builds with different configurations from a single command.
One solution I have seen used is to configure the staging environment so that it is identical to the production environment. This means each environment has a VLAN with the same IP range, and machine roles on the same IP addresses (e.g. the db cluster IP is always 192.168.1.101 in each environment). The firewalls mapped external facing addresses to the web servers, so by swapping host files on your PC the same URL could be used - http://www.myapp.com/webapp/file.jsp would go to either staging or production, depending on which hosts file you had swapped in.
I'm not sure this is an ideal solution, it's quite fiddly to maintain, but it's an interesting one to note.
Caleb P and JeeBee probably have your fastest solution. Plus you don't have to setup different services or point to files on different machines. You can specify your environment either by using a ${user.name} variable or by specifying the profile in a -D argument for Ant or Maven.
Additionally in this setup, you can have a generic properties file, and overriding properties files for the specific environments. Both Ant and Maven support these capabilities.
Don't forget to investigate PropertyPlaceholderConfigurer - this is especially useful in environments where JNDI is not available