Develop Reference

How to design an enterprise application with many independent processes?

How should I design an application comprised of numerous (but identical) independent processes that need to communicate data to an enterprise application and be monitored and accessible by a web interface?
Here's a more concrete example in Java:
The independent processes are multiple instances of a standalone J2SE application that receives on initialization data about a "user" entity and then starts doing stuff regarding this "user" (this is an infinite process and so any batch sort of design would be wrong here and also similarly, the starting time of these processes is irrelevant)
The enterprise application is a set of J2EE beans and web-services that implement business logic, DB access etc.. and that are (for example) hosted on GlassFish.
The web front is a set of JSPs (perhaps also on GlassFish) that work with the beans.
Now ideally, I want a way for the processes in (1) to be able to invoke methods from the beans in (2), but also for the beans in (2) to be able and update the processes (1) about things.
So these are the required flows of executions, assuming there are 10 independent process of (1) running for 10 different users (consider a "user" something easily identifiable by, say, a number):
Something happens in one of the processes of (1) and they invoke a method from the enterprise application (2) with some data.
One of the real, human, users (which was already identified by the web app) clicks something on a web-page of (3), this invokes a method in (2), and then some "magical" entity (which I have no idea how to name) finds the independent process from (1) that is responsible for this particular user and updates the process with some new data.
My best approach so far is to expose these J2SE apps by JMX and go from there, but I have one thing I don't understand - who or what should be holding a key-pair list of the sort "the process at URI X is responsible for user Y" and then directing the calls accordingly.
BTW, please feel free to give any advice outside of the Java platform (!), as long as it is a platform that can be scaled easily.
EDIT:
Also, is there a way to "host" such independent processes on some app-server? Something that will re-spawn processes if they fail, allow for deployment and monitoring of such processes on remote machines etc.?

There has been some time since I have used Java Message Service in the past so I am afraid I am not up-to-date with the technical details, but from your description it seems like it would suit your case, to handle communication between the adminstration GUI and the client processes.
There are various options (I believe you are interested for asynchronous communication) so you should take a look on the latest developments to examine yourself if it fits your case or not.
Regarding the data size that the server would exchange with the processes I believe this is a different topic and I must say that the answer depends. Would it be better to send all data in the message? Or would the message be just a notification so the client to be notified and then connect to some enterprise bean to check some new state? I would prefer the latter case but this is something you should decide based on your requirements. I wouldn't blindly exclude the first option unless I had some apparent evidence that this wouldn't work.
Regarding the scaling I don't think it can be much worse then the scaling of your rest of your beans. As much the server is concerned they processes are all clients that need to be served.
Please take the above advice with a grain of salt: I don't know specifics of your problem/design. I am speaking more about in a general way.
I hope that helps

How should I do interprocess communication between a management process and child processes?

I have five separate java processes; which are running as business logic modules. I would like to develop my process management application were i can start/ping/monitor/message child processes.
Also, it maybe sharing resources like cache etc with child processes over rest-ws or worst case rmi calls since requires additional overhead.
I was inclined toward webservice based api, which will keep sending information about business logic running within processes. The processes can be data churning, computation, notification process engines.
Any ideas?

One option is to use JMX, and publish one or more MBeans. Oracle has documentation on it. You can use the request information from the processes, or to send them signals to change their behavior.
The bare bones outline of what you would do is decide what methods you need to expose remotely in each of your child processes. Each of them should define an interface with those methods, then an implementation of that interface. Then those implementations need to be registered with the MBeanServer.
The advantage of this approach is that you will immediately get a bare-bones 'management application', because you can open JConsole against your processes and use the MBeans. If you then wish to create a separate application that will more cleanly present your data, you can do so at your leisure, without changing the child processes.
This approach does not really get you anyway to 'sharing a cache', but sharing a cache between processes (or machines) should really be a separate question (I think).

Trace java application at runtime

I'd like to trace a java application at runtime to log and later analyze every its behaviour.
Is there a possibility to hook into a java application to get runtime information like method calls (with parameters and return values) and the status of an object (i.e. its attributes and whose values)?
My goal is to get a complete understanding of the applications behaviour and how it deals with the data.

If you need highly customized logging and runtime processing, one alternative to profilers is to use aspects and load-time weaving.
We use AspectJ in this way to capture and log the authentication information for users who call a number of low-level methods for debugging purposes and to undo mistaken changes.

Use a profiler. For example JProfiler or one from this overview of opensource java profilers. Whenever I had to find deadlocks for example, these tools were priceless...

In Netbeans the profiler exist and work properly for use it see http://profiler.netbeans.org/

Maybe have a look at Glassbox a troubleshooting agent for Java applications that automatically diagnoses common problems. From Glassbox - Automated monitoring and troubleshooting using AOP:
Glassbox deploys as a war file to your
appserver and then uses AspectJ load
time weaving to monitor application
components and other artifacts, in
order to identify problems like excess
or failed remote calls, slow queries,
too many database queries, thread
contention, even what request
parameters caused failures. All this without
changing the code or the build process.
(...)
Glassbox monitors applications non-invasively by using aspects to track component interactions. We also monitor built-in JMX data, notably on a Java 5 VM we sample thread data (every 100 ms by default). As a request is processed, we summarize noteworthy events such as where time was spent and what parameters were involved in making things slow or fail. We also detect higher-level operations (such as Struts actions or Spring controllers) that we use to report on. Our AJAX Web client then provides summaries of status by operation on the machines being monitored and we generate a more detailed analysis on request. Glassbox allows monitoring clusters of servers: the Web app uses JMX Remote or direct RMI to access data from remote servers. We also provide JMX remote access to the lower-level summary statistics.
It's a nice application, give it a try.

How to start a background process in Java EE

I want to start a background process in a Java EE (OC4J 10) environment. It seems wrong to just start a Thread with "new Thread" But I can't find a good way for this.
Using a JMS queue is difficult in my special case, since my parameters for this method call are not serializable.
I also thought about using an onTimeout Timer Method on a session bean but this does not allow me to pass parameters (as far as I know).
Is there any "canon" way to handle such a task, or do I just have to revert to "new Thread" or a java.concurrent.ThreadPool.

Java EE usually attempts to removing threading from the developers concerns. (It's success at this is a completely different topic).
JMS is clearly the preferred approach to handle this.
With most parameters, you have the option of forcing or faking serialization, even if they aren't serializable by default. Depending on the data, consider wrapping it in a serializable object that can reload the data. This will clearly depend on the parameter and application.

JMS is the Java EE way of doing this. You can start your own threads if the container lets you, but that does violate the Java EE spec (you may or may not care about this).
If you don't care about Java EE generic compliance (if you would in fact resort to threads rather than deal with JMS), the Oracle container will for sure have proprietary ways of doing this (such as the OracleAS Job Scheduler).

Don't know OCJ4 in detail but I used the Thread approach and a java.util.Timer approach to perform some task in a Tomcat based application. In Java 5+ there is an option to use one of the Executor services (Sheduled, Priority).
I don't know about the onTimeout but you could pass parameters around in the session itself, the app context or in a static variable (discouraged would some say). But the name tells me it is invoked when the user's session times out and you want to do some cleanup.

Using the JMS is the right way to do it, but it's heavier weight.
The advantage you get is that if you need multiple servers, one server or whatever, once the servers are configured, your "Threading" can now be distributed to multiple machines.
It also means you don't want to send a message for a truly trivial amount of work or with a massive amount of data. Choose your interface points well.

see here for some more info:
stackoverflow.com/questions/533783/why-spawning-threads-in-j2ee-container-is-discouraged
I've been creating threads in a container (Tomcat, JBoss) with no problem, but they were really simple queues, and I don't rely on clustering.
However, EJB 3.1 will introduce asynchronous invocation that you may find useful:
http://www.theserverside.com/tt/articles/article.tss?track=NL-461&ad=700869&l=EJB3-1Maturity&asrc=EM_NLN_6665442&uid=2882457

Java EE doesn't really forbid you to create your own threads, it's the EJB spec that says "unmanaged threads" arn't allowed. The reason is that these threads are unknown to the application server and therefore the container cannot manage things like security and transactions on these threads.
Nevertheless there are lots of frameworks out there that do create their own threads. For example Quartz, Axis and Spring. Changes are your already using one of these, so it's not that bad to create your own threads as long as you're aware of the consequences. That said I agree with the others that the use of JMS or JCA is preferred over manual thread creation.
By the way, OC4J allows you to create your own threads. However it doesn't allow JNDI lookups from these unmanaged threads. You can disable this restriction by specifying the -userThreads argument.

I come from a .NET background, and JMS seems quite heavy-weight to me. Instead, I recommend Quartz, which is a background-scheduling library for Java and JEE apps. (I used Quartz.NET in my ASP.NET MVC app with much success.)

Is it necessary to unregister an MBean from the Platform MBean Server?

I've begun playing with MBeans for exposing some information about an app. Pretty much the totality of HOWTOs / Tutorials / Blog posts / Samples online teach you how to implement an MBean and register it with an MBean Server, but never mention (or only cursorily) unregistering the MBean from the server.
I'm concerned about an MBean that needs to have a reference to a fairly heavyweight object in order to provide status information about that object.
Does the Platform MBean Server maintain a weak reference to the MBean, or some other such trick, to ensure that it eventually gets GC'ed if your application no longer holds any references to it? Is unregistering generally unnecessary, thereby explaining why no one is talking about it in JMX tutorials?

You can not "weakly" register an MBean with a server (yet), thereby expecting it to be GCed when no other references to it exist.
This being said, you should definitely read some posts by Eamonn McManus on the subject.
https://web.archive.org/web/20120207140653/http://weblogs.java.net/blog/emcmanus/archive/2005/07/cleaning_up_an_1.html
It's been suggested that the JMX API
could have some explicit support for
"Weak MBeans" like this. I'm not sure
there's enough use for them to justify
including them in the API, and I'm
also not sure what a general-purpose
API for Weak MBeans would look like.
But the above shows how you can create
your own Weak MBeans if need be.
https://web.archive.org/web/20090114131740/http://weblogs.java.net/blog/emcmanus/archive/2005/07/javaone_feedbac.html
"Weak" MBeans. An MBean frequently
manages another Java object that is
the "resource" to be monitored or
controlled. But what if the only
reference to that resource is from the
MBean? Could we somehow arrange for
the MBean to disappear if the resource
is no longer referenced by anyone
else?
Turning on and off expensive MBeans.
Some MBeans may export information
that is continuously sampled and that
is expensive to gather. You don't
necessarily want those MBeans to be
running all the time. Ad hoc solutions
are easy, for instance the
setThreadContentionMonitoringEnabled
method in
java.lang.management.ThreadMXBean. But
perhaps there could be a more general
convention, such as a
setDetailLevel(int) method.