Develop Reference

Reloading jar files contents dynamically

I have one jar file in my application's class path. At run time, I add new classes to the jar file and sometimes also modify the fields/methods of the existing classes. Currently I am using URLClassLoader to load the classes dynamically. The new classes added dynamically are loaded correctly and I am able to use them at runtime. But it fails to reload the existing classes that are modified at runtime. I read many articles which states we need to explicitly handle reloading because class once loaded will not be reloaded until all the references to the class are destroyed. Also I tried out sample code that I found but none of them worked.
Can anyone suggest me a proper approach for reloading ? Any sample code for the same will be highly appreciated.

Normally to reload a class you need to unload the entire class loader. i.e. remove all references to all classes loaded for that class loader.
Another option is to use instrumentation to change the byte code of an existing class. This usually comes with limitations and changing fields is something you cannot do. i.e. the objects of that type would have to be translated somehow.
What I normally do is have services which are very quick to start/restart. This way to you easily restart a process which needs updated code ideally by pressing the Run in my IDE. This minimises deployment time as well.

In principle, a class that has already been loaded cannot be reloaded with the same classloader.
For a new load, it is necessary to create a new classloader and thus load the class.
Using URLClassLoader has one problem and that is that the jar file remains open.
If you have multiple classes loaded from one jar file by different instances of URLClassLoader and you change the jar file at runtime, you will usually get this error: java.util.zip.ZipException: ZipFile invalid LOC header (bad signature). The error may be different.
In order for the above errors not to occur, it is necessary to use the close method on all URLClassLoaders using the given jar file. But this is a solution that actually leads to a restart of the entire application.
A better solution is to modify the URLClassLoader so that the contents of the jar file are loaded into the RAM cache. This no longer affects other URLClassloaders that read data from the same jar file. The jar file can then be freely changed while the application is running. For example, you can use this modification of URLClassLoader for this purpose: in-memory URLClassLoader

Dynamically compile and include JSP(X)s outside Web Context Root

I have a web application heavily using jspx-files and jsp:include-includes which is deployed from the same jar to several hundred contexts in tomcat. Currently we can only modifying the layout of each individual instance by changing CSS or overwriting certain files in other file directories:
Let's say the webapp.jar contains a file named /css/forms.css then the webapp ensures that GETting http://mydomain/cust1/res/css/forms.css is mapped to that file. If the webapp is configured to look in the "override directory" /data/cust1/, then if there's a file named /data/cust1/css/forms.css this file is served instead from the one in the application archive. This is ensured by the webapp itself.
During the last years this has been very successful but recently I feel the pain of the restriction of the more or less static jspxs that cannot be "overridden" ;) Basically I'd like to be able to "override" some jspx files for each deployed context without compiling and deplyoing a custom webapp-jar for each context. (Having a custom something:includeJsp-tag wouldn't be a problem).
Basically the webapp should be able to provide an override to the jsp(x) compiler for individual jspx-files, e.g. take a look at the following example sutrcture:
<!-- webapp.jar:/jspx/view.jspx -->
<jsp:root ...>
<ns:customInclude src="inc/include.jspx" />
</jsp:root>
<!-- webapp.jar:/jspx/inc/include.jspx -->
<jsp:root ...>
Default-Markup from the webapp.jar
</jsp:root>
<!-- /data/cust1/jspx/inc/include.jspx -->
<jsp:root ...>
Custom markup for "cust1"
</jsp:root>
Now, when http://.../cust1/jspx/view.jspx is requested I want /data/cust1/jspx/inc/include.jspx to be compiled and executed by Tomcat.
Basically I know that actually everything necessary is already possible somewhere in Tomcat (compiling from a jspx to Java-Bytecode, including the file, ...), but I also know that Tomcat/Jasper adhere to the jsp-spec. I figured by looking at the code that this is not that easy...
So basically, does anyone know how to get a setup like this to work? Did perhaps someone already solve this? Or are there alternatives to my current approach?

I ended up implementing my own FileDirContext that is included via a <Resources>-Tag in context.xml. Actually I implemented two - an extension of org.apache.naming.resources.WARDirContext and an extension of org.apache.naming.resources.FileDirContext because I wanted to enable the same mechanisms during development with eclipse (which deploys to a file structure) and on our servers which deploy from non-exploded .war files.
Both of them can be configured via the context descriptor and can use several "virtual overlays". An overlay can either be a file system path or an external .war file (actually more like a .zip renamed to .war) or a .jar file packed into the deployed .war file.
The steps to a tuned down solution with one virtual overlay are:
Create a class MagicWARContext (magic is like the solution to every problem ;) extending from aforementioned WARDirContext.
In this class create a static inner class ProtectedMethodVisibleMaker extends FileDirContext. This allows to access certain protected methods of Tomcat's original implementation. Override doGetRealPath and doLookup.
In your own WAR context create getters and setters for a field private String overlay and a field private ProtectedMethodVisibleMaker overlayContext and instantiate a new and empty instance to this field.
In allocate() of your WAR context call overlayContext.setDocBase(overlay). Be sure to call overlayContext.release() from release() just to be sure.
Now you have to override 5 more methods of WARDirContext
doGetRealPath(String)
doLookup(String)
getAttributes(Name, String[])
list(String)
list(Name)
The resulting enumerations of both lists should of course contain entries of a virtual overlay as well as the original context. So I ended up writing my own NamingEnumeration that iterates over a list of delegate naming enumerations and ensures that each named entry is output only once.
Instantiate your Magic Context from the context descriptor:
<Resources
className="package.name.tomcatextensions.MagicWARContext"
overlay="/data/some/dir"/>
Pack your classes in a jar and copy this e.g. to tomcat/lib
Caveats
We're overriding Tomcat's internal classes. This naturally exposes this code to changes. So someone has to ensure that everything works as expected after version upgrades. Instead of some of the protected methods one can override the actual public ones, but some of them are final so you're basically out of luck.
Plus it is possible to expose non-magic subcontexts to other application layers via the lookup methods meaning they don't have that overlay anymore or don't contain the fallback resources of your deployed war. I ended up adding some warnings to the log when subcontexts are requested. During startup this happens two times, once for /WEB-INF/classes and once for /WEB-INF/lib.
In my usecase I prohibit adding virtual overlays that contain either on of those paths because I don't want to have foreign class files appearing in my web context, so this is not a problem. However some other client might be calling those methods and doing something unexpected with them. During my tests I did not find any problem, but there might occure strange effects with caching enabled or when enumerating resources and there will be problems when sub contexts are used... Of course one could create virtualized wrappers around a subcontext returned by a lookup, but I didn't do this yet as I hope this is unnecessary.
And of course this method enables you to override certain resources of the fallback webapp in un-exploding war deployments. If you just need to add external resources to the web context you can use Tomcat's aliases (see How do I add aliases to a Servlet Context in java?).
Use at your own risk and have fun.

Is it ok to put configuration files in JARs?

We are having a debate at my office around what can and cannot go in a JAR file. It has been suggested that it is poor form to have anything that is not a .class file go into a JAR. We currently have some XML configurations for Ibatis/etc, some property files.. the usual. However, there is a push to extract all such files from JARs and put them onto the local file system of each deployment machine. Does this sound reasonable?

it is poor form to have anything that
is not a .class file go into a JAR
That is nonsense. It is in fact very good form to put resources like icons and other data files that user used by the code into the JAR together with the code. This is what the getResource() and getResourceAsStream() methods of Class and ClassLoader are for, and it makes for much more robust applications than messing around with resource paths manually.
However, config files are possibly a different matter. If they're meant to be changed during or after deployment, then having them inside a JAR file is rather inconvenient, and having them in a separate directory is preferable.

If you make changes in a configuration file inside a JAR (even without altering any line of Java code), the whole JAR needs to be rebuilt and redeployed. Does this sound reasonable?

It's absolutely OK to put non-class files in a JAR file, especially resources that the application needs (images, localized strings, etc.) Knowing this, you must decide which scenario fits your situation:
If the configuration is fixed and will only change when a new JAR file is deployed, put it in the JAR.
If the configuration must be altered, either manually or by the application, store it on the filesystem.
If you choose the latter, note that it's good practice to include a default configuration in the JAR file to handle the case when the external configuration file is missing. The default can be loaded directly from the JAR or copied to the filesystem to become the new editable configuration.

It does not sound reasonable to me. I believe, that some application's configuration should be in jar file. Such things as ORM mappings, spring config, custom spring namespace XSD, other XSDs, etc.. should be in most cases in jar. It's important part of deployment artifact.
The fact, that it's not class file, does not mean, that it should be taken out of jar just because it's theoretically can be modified without building a new jar. Can you imagine a modification of *.hbm.xml in production? for me it sounds very scary.
I think some configuration, like spring xml, is meant in most cases to better organize your application and dependencies, but not to change them at runtime in production.

Do you want or expect them to be changed without a new release of the code? Then you need to extract them.
If the answer to the question in no than you shouldn't extract them, as it would allow support to tinker around with them without going through the release process. (Of course this is also possible if they are in the JAR but slightly less tempting.)
Update: But since you mentioned multiple deployment machines, there's a third option: extract them and place them in a commonly accessible area on a network drive. Manually editable config files which are replicated on several machines (but should be identical) are notorious for getting out of sync.

ORM tools (such as Hibernate or IBatis) are not really supposed to be modified once the application is deployed. So, no, I would say that it doesn't make sense for that kind of files.
Depending on your requirements, application-wide configuration files can be placed outside the Jar/War so that they can be modified without having to rebuild the jar.
Do keep in mind that modifying application parameters in production is, imho, a bad practice. Changes should be tested first in some pre-production environment.

How to load DTDs from a jar file?

The intended machine doesn't have a connection to the internet and I do not want to load it using a fixed location.
Aim: To load DTDs from a jar, the jar will be a dependency.

To load any file from the classpath (it is, the space where youre classes reside, usually a bunch of jars) you can do:
InputStream is = this.getClass().getResourceAsStream("my/package/ResourceFile.dtd");
And then you can use the input stream where you want.
Note: getResourceAsStream() loads the resource using the class loader that loaded the class. If you are making an application any class from your application (and hence loaded by the same classloader with your jars) will be fine.

There is no standard way to provide a local cache (CATALOG if I recall correctly) of DTD's.
Hence, you will need to investigate the parser that will use the local copies, and use its non-standard configuration API to let it know about these local copies so that the trip to the net is avoided.
Use the getResourceAsStream() method to pick out entries from the classpath.

Java EE EAR shared location of read/write resources within clustered environment

Within a Java EE environment (happens to be WAS 6.1 but could be any application server) I need to place a XML file, which is a configuration file, so that I can read and write to it.
This needs to be available in a clustered environment so I am looking at using the class path to load the file.
I am thinking I can store this file in the EAR root, reference it in the manifest and then load and save it.
I have tried this approach by having my file in a JAR and making this available via the MANIFES and I can load the config file from the class path no problem using the following.
this.getClass().getClassLoader().getResourceAsStream("configFileName");
That loads the file that is in the JAR, which is fantastic. But if I want to edit this file, programmatically, I cannot access the JAR files location (the EAR root) it returns me an interpreted path like this:
/usr/IBM/WebSphere/AppServer/profiles/AppSrv01/installedApps/localhostNode01Cell/MyApp.ear/MyApp.war/TB_config.jar
That is not the correct location of the JAR the correct location is at MyApp.ear.
So the question is: how can I access and update (copy contents, create new, save, delete old) the JAR with my config file.
Or should I put the config file somewhere else?
What is the standard Java EE to make files that need read/write access available to WARs on a cluster?

Ok I have built a solution for this. It is more WebSphere based (our platform) but it is J2EE and I am suprised it was not mentioned. Basically I have used JMX to synchronise the nodes. The files are stored, and saved to, the deployment manager the nodes are then resynchronised using JMX calls and then the engines withing the applicaitons are restarted by calling servlets within the applications.
It works a dream
So #stacker, nodes are managed and the manager distributes files to the nodes.

The problem that you've hit is not unique. A lot of Java EE programmers can struggle with providing a "configurable" property file to administrators of a cluster. And the solution that you've chosen, well, has its limitations.
The problem with embedding a config file inside a JAR, is absolute path or the physical path of the file, in case you need to update it. If your container will not explode your EAR and WAR files, then placing the config file alongside the code is a bad idea - the administrator will have to deploy a newer version of the EAR/WAR/JAR. That is unless, of course, you can configure the container to explode the artifacts - WebLogic Server does this, I'm not sure about WAS.
There are several ways to resolve this problem:
Store the config file in a SAN that is accessible to all the nodes in the cluster via a 'canonical' path. That way, you could locate the file from any node in the cluster and update it. Remind yourself to restrict access to this directory. Although this sounds simple, it need not be - Java objects might have to be 'flushed' across nodes, once the configuration file has been updated. Moreover, you might have to cater to the scenario where property files can get edited outside the application.
Use a database. Much simpler and almost hasslefree, except that the Java objects might have to be flushed again.
Use a MBean. As good as a database, except that I haven't known a lot of people vouching for the MBean support in WAS. Also, I'm not really sure if object states can go haywire across a cluster, in this case.

You cannot write to an ear file, you should place the XML file in the DB as a text lob (large object).

Actually, as I am using WebSphere, it appears I can use the dynamic cache provided by the WebSphere deployment manager. The last chapter in the link below dicusses the use of the Dynamic Cache providing a shared object in a cluster. The configuration file is XML that is parsed as such by the engine (into a Document object) of the application and so is a Java object, thus it can be placed into the DistributedMap.
Looks like a clean solution. Thanks all for reading and your replies.
http://www.ibm.com/developerworks/websphere/library/techarticles/0606_zhou/0606_zhou.html