Develop Reference

How to move the Liferay hooks into a directory inside webapps?

I have created many hooks for Liferay with Eclipse Java EE IDE for Web Developers.
The problem is that when I create a hook is deployed inside the webapps folder.
My webapps folder tree:
There are many hooks. How can how can I move them into a new folder inside webapps?
in order to have something like this:
webapps
calendar-portlet
hooks
blogs-hook
breadcrumb-hook
dockbar-hook
login-hook
blogs-hook
sitepages-hook
truefalsetocheckbox-hook
userinfo-hook
wiki-hook
marketplace-portlet
my-theme
resources-importer-web
ROOT
Any help is greatly appreciated! Thank you so much!

I suppose you're speaking about Tomcat's (or whatever appserver you're using) webapp folder. You cannot merge different apps into one folder: it has to keep the structure on deploy.
So what you can do is to aggregate those hooks on source level (i.e., create only one hook project in Eclipse with the Liferay IDE so you can put everything there). So you could put them into the same plugin.
Btw I would discourage using several hooks, because they might occasionally overwrite each other's behaviour. Imagine you have some sort of property defined in a-hook and b-hook in a properties file. You cannot tell which one will be used after the deployment.

Adding a bit of reasoning to rlegendi's answer, here's what you might consider:
If you use Liferay's auto-deploy mechanics, e.g. through the plugins-sdk deployment, Liferay places the hooks in tomcat's webapps folder (in case you use tomcat) and you'd also have to override this.
You might be able to configure tomcat to monitor different directories, even on the same virtual host (I've only tried this for different virtual hosts and don't know if this will work properly with Liferay), but the auto-deploy problem would persist, you'd basically have to change the whole deployment logic, or deploy manually to tomcat as you'd do with "big iron" appservers.
If you're happy about the development-time separation between the hooks, but would like some better runtime overview, you might want to merge different hooks into one for deployment. However, you'd have to do this on your own.
But my final recommendation is to not care too much about what's in the webapps directory anyway - you shouldn't rely on directory listings of this directory for knowing what's running on your appserver. Rather use proper deployment techniques that monitor your implementations, and limit access for manual changes. E.g. deploy your applications scripted and install a system that alerts you should required components not be available and running.

Making an open source Java webapp easy to install and deploy on Tomcat, Jetty etc

I'm distributing a simple Java webapp as open source. The webapp needs to be configured before it can be run – a configuration file needs to be created, and the location of that configuration file needs to be made known to the webapp as a parameter in web.xml.
Now my question is how to best package and distribute the webapp in order to make it easy to install, and how to describe that installation process in the documentation. The options I can think of are:
Distribute the webapp as a WAR archive. Recommend that users deploy the WAR into their Tomcat/Jetty/whatever, then drop their configuration file into /webapps/myapp/WEB-INF, and modify /webapps/myapp/WEB-INF/web.xml accordingly
Distribute the webapp as source. Recommend that users should drop their configuration file into the /src/main/webapp/WEB-INF folder, then modify their /src/main/webapp/WEB-INF/web.xml accordingly, then build a WAR using Ant or Maven, and deploy that into their servlet container.
There are probably other options that I can't think of.
What setup is likely to be most convenient to users that need to install, configure and deploy the webapp?
Edit: I should add that the configuration file isn't just a few lines – it's a large file that contains a database schema and other stuff and is potentially generated using an external editor; so just providing good defaults isn't an option.

Externalize this configuration and maybe provide some default values. If you make a new version of your app, everybody will have to remember to back-up that configuration file, then redeploy and then copy back that file--> this is a nightmare.
There are many ways to put that configuration somewhere else. You can use Java Preferences for example.

I would say the WAR, although not requiring the configuration would likely be more convenient :)
What is it, loosely, that must be configured such that there isn't a sensible default value for everyone? URL string?

Providing an answer of my own, after more reading on the issue: JNDI seems to be the “official”, although somewhat heavyweight, way of solving this. With JNDI, a configuration option (like the location of the full config file I need) can be declared in the web.xml, and its actual value can be set in a per-webapp context.xml that lives in the /webapps directory of Tomcat (or the /contexts directory of Jetty). This setup has a bunch of advantages:
The big configuration file can live outside of the servlet container and webapp
The webapp can be updated without danger of losing the configuration
The distributed war doesn't need to be modified or rebuilt
Downside: It's sort of complicated, requires messing around with XML, and configuring JNDI on Tomcat works differently from Jetty (requiring twice as much documentation).

Maybe use a system property for the config file location. Can easily be passed on the command line as -Dorg.example.config.file=/foo/bar, in startup scripts or in Java code. I think I've seen some tools, e.g. logging frameworks, use system properties for similar things in webapps.

Tomcat parent webapp shared by configurable children webapps

Currently, we support many clients using the same web app, but each client has a different configuration for accessing their database, setting files etc. As the client list grows, updating the web apps is becoming increasingly arduous, and the duplication of resources is a waste of memory, file space, etc..
What we'd like to do is have a parent web app which is shared by all children web apps. Then have each child web app carry only files specific to them. When the child web app starts up, Tomcat loads the web app from the parent web app and then overrides any files defined in the child web app following an identical package structure.
We've been googling around and haven't found a ready or complete solution. Solutions we've looked at:
Tomcat common/share - could handle class and JAR files, but we don't see a way to handle static and JSP resources residing above the WEB-INF dir.
CATALINA_BASE appears to be more suited for running multiple instances of Tomcat which we'd rather avoid
A Maven possible solution, but we are not big fans of Maven, so would rather avoid it also.
Anybody have suggestions or ideas on how to solve this? If Tomcat configuration is not possible, what about a different application server (such as Glassfish) or a tool for doing dynamic file updated (such as OSGi, rsync). Would like to remove the resource duplication if possible.
Thank you.

There is no such thing as "parent" or "child" webapps. It's not part of J2EE spec and AFAIK it's not supported by any application server.
That said, your problem is twofold:
1) Having shared resources. This part is pretty easy assuming "resources" means static resources (images / CSS / javascript / etc...).
If they are truly shared (e.g. you don't need to have a separate version in some of your webapps), host them elsewhere (separate "common" webapp or put Apache in front of your Tomcat and host them there.
If you do need to have "local" versions of some of those resources you may be able to do some clever conditional URL rewriting or simply write a servlet that would check whether particular resource exists locally and, if not, take it from "common" location.
Precompile your JSPs so you only have to deal with JARs.
If your Tomcat instance only hosts your apps, you can indeed put your JARs in shared (or lib in the latest version); otherwise you can deploy them with each application .
2) Simplifying deployment. I'm not really sure what the big problem is here... It's rather trivial to write an Ant (batch, shell, what have you) script that would assemble and deploy WARs based on "common" and "per-app" directory structures.
Alternatively, you may want to take a look at using JNDI to severely reduce the number of files that have to be deployed (in theory, to a single context.xml for each application).

You can build parent-child hierarchy if you use Spring at your web-apps - Using a shared parent application context in a multi-war Spring application.
I.e. you can define all shared stuff at the 'parent' context and have 'child' contexts just to use it.

If all you had was setting file and configuration changes you could manage these through the context.xml and then you can point the docBase of each application context at a common directory for all the applications to share the same source.
the drawback to this is changes to the application will require a tomcat restart.
This does not however solve your problem if you want to override logic.
A option that I am exploring for a similar scenario is to move the client custom portion into ajax widgets / gadgets. Then have it be part of the configuration files to tell the application which version of the gadget to pull for which client.
you can review documentation for having applications share a docbase here http://tomcat.apache.org/tomcat-5.5-doc/config/context.html

How do you manage embedded configuration files and libraries in java webapps?

I'm currently working on a j2ee project that's been in beta for a while now. Right now we're just hammering out some of the issues with the deployment process. Specifically, there are a number of files embedded in the war (some xml-files and .properties) that need different versions deploying depending on whether you are in a dev, testing or production environment. Stuff like loglevels, connection pools, etc.
So I was wondering how developers here structure their process for deploying webapps. Do you offload as much configuration as you can to the application server? Do you replace the settings files programmatically before deploying? Pick a version during build process? Manually edit the wars?
Also how far do you go in providing dependencies through the application servers' static libraries and how much do you put in the war themselves? All this just to get some ideas of what the common (or perhaps best) practice is at the moment.

I think that if the properties are machine/deployment specific, then they belong on the machine. If I'm going to wrap things up in a war, it should be drop-innable, which means nothing that's specific to the machine it's running on. This idea will break if the war has machine dependent properties in it.
What I like to do is build a project with a properties.example file, each machine has a .properties that lives somewhere the war can access it.
An alternative way would be to have ant tasks, e.g. for dev-war, stage-war, prod-war and have the sets of properties part of the project, baked in in the war-build. I don't like this as much because you're going to end up having things like file locations on an individual server as part of your project build.

I work in an environment where a separate server team performs the configuration of the QA and Production servers for our applications. Each application is generally deployed on two servers in QA and three servers in Production. My dev team has discovered that it is best to minimize the amount of configuration required on the server by putting as much configuration as possible in the war (or ear). This makes server configuration easier and also minimizes the chance that the server team will incorrectly configure the server.
We don't have machine-specific configuration, but we do have environment-specific configuration (Dev, QA, and Production). We have configuration files stored in the war file that are named by environment (ex. dev.properties, qa.properties, prod.properties). We put a -D property on the server VM's java command line to specify the environment (ex. java -Dapp.env=prod ...). The application can look for the app.env system property and use it to determine the name of the properties file to use.
I suppose if you have a small number of machine-specific properties then you could specify them as -D properties as well. Commons Configuration provides an easy way to combine properties files with system properties.
We configure connection pools on the server. We name the connection pool the same for every environment and simply point the servers that are assigned to each environment to the appropriate database. The application only has to know the one connection pool name.

wrt configuration files, I think Steve's answer is the best one so far. I would add the suggestion of making the external files relative to the installation path of the war file - that way you can have multiple installations of the war in the one server with different configurations.
e.g. If my dev.war gets unpacked into /opt/tomcat/webapps/dev, then I would use ServletContext.getRealPath to find the base folder and war folder name, so then the configuration files would live in ../../config/dev relative to the war, or /opt/tomcat/config/dev for absolute.
I also agree with Bill about putting as little as possible in these external configuration files. Using the database or JMX depending on your environment to store as much as it makes sense to. Apache Commons Configuration has a nice object for handling configurations backed by a database table.
Regarding libraries, I agree with unknown to have all the libs in the WEB-INF/lib folder in the war file (self-packaged). The advantage is that each installation of the application is autonomous, and you may have different builds of the war using different versions of the libraries concurrently.
The disadvantage is that it will use more memory as each web application will have its own copy of the classes, loaded by its own class loader.
If this poses a real concern, then you could put the jars in the common library folder for your servlet container ($CATALINA_HOME/lib for tomcat). All installations of your web application running on the same server have to use the same versions of the libraries though. (Actually, that's not strictly true as you could put overriding versions in the individual WEB-INF/lib folder if necessary, but that's getting pretty messy to maintain.)
I would build an automated installer for the common libraries in this case, using InstallShield or NSIS or equivalent for your operating system. Something that can make it easy to tell if you have the most up to date set of libraries, and upgrade, downgrade, etc.

I usually make two properties files:
one for app specifics (messages, internal "magic" words) embedded in the app,
the other for environment specifics (db access, log levels & paths...) exposed on each server's classpath and "sticked" (not delivered with my app). Usually I "mavenise" or "anttise" these one to put specific values, depending on the target env.
Cool guys use JMX to maintain their app conf (conf can be modified in realtime, without redeploying), but it's too complex for my needs.
Server's (static ?) libraries: I strongly discourage server library use in my apps as it adds dependency to the server:
IMO, my app must be "self-packaged": dropping my war, and that's all. I have seen wars with 20 Mbs of jars in it, and that's not disturbing for me.
A common best-practice is to limit your external dependencies to what is offered by the J2EE dogma: the J2EE API (use of Servlets, Ejbs, Jndi, JMX, JMS...). Your app has to be "server agnostic".
Putting dependencies in your app (war, ear, wathever) is self-documenting: you know what libraries your app depends on. With server libs, you have to clearly document these dependencies as they are less obvious (and soon your developers will forget this little magic).
If you upgrade your appserver, chances that the server lib you depends on will also change. AppServer editors are not supposed to maintain compatibility on their internal libs from version to version (and most of the time, they don't).
If you use a widely-used lib embedded in your appServer (jakarta commons logging, aka jcl, comes to mind) and want to ugrade it's version to get the latest features, you take the huge risk that your appServer will not support it.
If you relies on a static server object (in a static field of a server class, e.g. a Map or a log), you'll have to reboot your appserver to clean this object. You loose the ability to hot-redeploy your app (old server object will still exists between redeployments). Using appServer-wide objects (other than those defined by J2EE) can lead to subtle bugs, especially if this object is shared between multiple apps. That's why I strongly discourage the use of objects which resides in a static field of an appServer lib.
If you absolutely need "this object in this appserver's jar", try to copy the jar in your app, hoping there's no dependency on other server's jar, and checking your app's classloading policy (I take the habit to put a "parent last" classloading policy on all my apps: I'm sure I won't be "polluted" by server's jars - but I don't know if it is a "best practice").

I put all configuration in the database. The container (Tomcat, WebSphere, etc) gives me access to the initial database connection and from then on, everything comes out of the database. This allows for multiple environments, clustering, and dynamic changes without downtime (or at least without a redeploy). Especially nice is being able to change the log level on the fly (although you'll need either an admin screen or a background refresher to pick up the changes). Obviously this only works for things that aren't required to get the app started, but generally, you can get to the database pretty quickly after startup.

Place MySQL connector JAR in WEB-INF/lib of my WAR instead of in $CATALINA_HOME/common/lib?

I'm about to use MySQL with Hibernate on a Tomcat 5.5.x server.
Do I have to put mysql-connector-java-[version]-bin.jar in $CATALINA_HOME/common/lib/ or could I place it in WEB-INF/lib inside my WAR file with my other library dependencies?
It would be easier to have it in my WAR in WEB-INF/lib, as I could get it using the Maven repository that way. Are there any big draw backs to having it there instead of in the common libraries directory of Tomcat?

If your jar is in common then it's loaded globally in tomcat. Every webapp sees it. If you put it in your webapp only your webapp sees it. What I assume Boris is referring to is if some part of your global tomcat config loads something that needs a mysql connection (like a connection pool) then it's going to need the jdbc driver, so it'll need it in common. Otherwise, no.
I believe there's a security configuration you can set up in the tomcat config that stores its config in a db. If you use this, it'll need a driver.
Why would you want stuff in webapp/WEB-INF/lib? well, because it's modular, it's part of your webapp, if you move your webapp somewhere else it doesn't need an extra part of it that's part of your tomcat installation. Maintenance is much easier if you can drop your webapp directly in a stock tomcat installation. Another reason- if you have multiple webapps, they're all going to use the jars in common, which could cause library conflicts and version issues.
In general, put as little in common as you can get away with. Even if you only have a single app.