I have made test WebService which starts Thread, which writes to file timestamp every 10 second. I intentionally don't have Thread stop mehanism.
Now, if I stop test WebService, and even delete it, the Thread live in Jboss forever
(needs JBoss restart).
Is this normal that JBoss isn't aware of Threads made within WebService context ?
Within JVM, when app shuts down, all threads are killed, but here is JVM owned by Jboss,
which dynamically loads classes.
Is this a "feature" or bug ?
I am asking this, cause we have 3rd party application doing threading, and I noticed they're not shutted down on WebService destructor, so after re-publish, we have an issue.
Cleaning up resources used by a WebService, is the responsibility of the WebService itself.
Tomcat will help you by logging warnings when a WebService is not doing this properly (e.g. when the MySQL JDBC driver is leaving a thread hanging) and it will even try to clean ThreadLocals for you (see also comments in this utility class).
In your case, since you are using the 3rd party application in a WebService, you are responsible for cleaning up resources used by the 3rd party application when your WebService is unplugged. It would be nice if JBoss could at least report "resource leakage" like Tomcat does, but that would be a feature and not a bug.
I have made test WebService which starts Thread, which writes to file timestamp every 10 second. I intentionally don't have Thread stop mehanism. Now, if I stop test WebService, and even delete it, the Thread live in Jboss forever (needs JBoss restart). Is this normal that JBoss isn't aware of Threads made within WebService context ?
You aren't supposed to launch your own threads, so JBoss won't clean your stuff up for you.
See also why spawning threads in Java EE is discouraged (mostly applies to Enterprise Java Beans) or "Java EE specification and multi threading".
If the 3rd party application does its own threading and you can't change it, then it might not be a good fit for an application server. An old trick to graft an old application into Java EE is to manage its lifecycle using a ServletContextListener, which has an init and destroy method.
If you can change it though, check the Concurrency Utils API example from the question / answer I linked to above, using an ExecutorService is the modern way to manage threads so that the server is aware of your handywork.
Related
What is the Java equivalent of the .net Recycle for web apps in IIS.
This is when using Java on a linux machine outside of IIS.
Is it just to stop and start the application?
Not really - IIS and JVM work in different ways. When you say recycling in IIS, it's basically restarting the Worker process. Each Web Application deployed to IIS is under an application pool and worker process.
In case of java, it's not like that. The whole App server runs on a jvm and you have different Web applications deployed into the App server which runs within the app server.
You could use DB connection pools or Apache commons pool for pooling (Some of your expensive objects you reuse) which can be refreshed but not exactly in a way like IIS.
Even though this would be a nice feature - in reality if you ever
reach a situation of needing to refresh application pool, your
code/dll(may be 3rd party) is the culprit. There would definitely be a
memory leak which needs to be addressed! Also when you recycle the session state might be lost. Apparently users logged in would get logged out (and if they are in the middle of a transaction they might loose data).So it could lead to a very volatile situation!
Update
You could use stuff like Terracotta which handles memory management.
Usually JEE servlet containers offer the option of reloading the application. i.e. Tomcat has a Reload button for each app on the manager console.
It also has a button that triggers a full garbage collection, by the way.
Say my application is performing an operation (counting pi to 1000th decimal place). During this time another application accesses my API (REST or SOAP does it matter?) and requests data.
Will they be left hanging or does the server/jvm handle this request on a different thread (even if my application is single threaded).
I've been told different things
edit: I am running this in a web container. Currently Jetty but will probably need to move to Tomcat at least.
Thanks
The web server will spawn new threads (it will actually more likely use a thread within its pool, and you could configure it differently, but the default behavior is likely to use a pool of threads) for each request, so your callers will not be left hanging (unless you're over your server's capacity).
Your code needs to be thread safe to handle multiple requests at the same time.
Depends on your setup. Typically (though not always) java containers will spawn* a thread per request.
*Actually no they won't necessarily spawn one, most servers will work with a thread pool, but the effect is essentially the same.
Some setups, however, will not spawn a thread-per-request, for example if using cometd for long-running requests. More details on that can be found here: http://cometd.org/book/export/html/31
Update in response to updated question:
Jetty will work just fine, unless there are other reasons to move to tomcat? I like Jetty myself, very low footprint, and I find its xml configs straightforward, but that may just come from 5 years of working with it.
I'd like to run a web container where each webapp runs in its own process (JVM). Incoming requests get forwarded by a proxy webapp running on port 80 to individual webapps, each (webapp) running on its own port in its own JVM.
This will solve three problems:
Webapps using JNI (where the JNI code changes between restarts) cannot be restarted. There is no way to guarantee that the old webapp has been garbage-collected before loading the new webapp, so when the code invokes System.loadLibrary() the JVM throws: java.lang.UnsatisfiedLinkError: Native Library x already loaded in another classloader.
Libraries leak memory every time a webapp is reloaded, eventually forcing a full server restart. Tomcat has made headway in addressing this problem but it will never be completely fixed.
Faster restarts. The mechanism I'm proposing would allow near-instant webapp restarts. We no longer have to wait for the old webapp to finish unloading, which is the slowest part.
I've posted a RFE here and here. I'd like to know what you think.
Does any existing web container do this today?
I'm closing this question because I seem to have run into a dead end: http://tomcat.10.n6.nabble.com/One-process-per-webapp-td2084881.html
As a workaround, I'm manually launching a separate Jetty instance per webapp.
Can't you just deploy one app per container and then use DNS entries and reverse proxies to do the exact same thing? I believe Weblogic has something like this in the form of managed domains.
No, AFAIK, none of them do, probably because Java web containers emphasize following the servlet API - which spins off a thread per http request. What you want would be a fork at the JVM level - and that simply isn't a standard Java idiom.
If I understand correctly you are asking for the standard features for enterprise quality servers such IBM's WebSphere Network Deployment (disclaimer I work for IBM) where you can distribute applications across many JVMs, and those JVMs can in fact be distributed across many physical machines.
I'm not sure that your fundamental premise is correct though. It's not necessary to restart a whole JVM in order to deploy a new version of an application. Many app servers will use a class-loader strategy that allows them to discard a version of an app and load a new one.
I know that we should not start threads in a servlet is that threads should be managed by the container. If the container is told to shutdown if there are threads that it does not know about hanging around it wont shutdown. I take care of this by making it a daemon thread...
But other than the above "unable to shutdown" situation what other reason could there be to not allow a servlet to start threads. I have seen some mentions that if the environment is clustered it will cause a problem. But no actual walk-through of what could happen that would be BAD.
EDIT:
This is currently being done in a servlet and I am having trouble convincing the author of this code that is not a good idea. The argument that one has to understand complexity is not going to fly...
I am looking for one specific concrete case when something bad can happen, without intending it to
In my situation: the servlet in question is launches n threads and this happens in each vm on the cluster by design.
There is no transactional requirement
From the official FAQ:
Why is thread creation and management
disallowed?
The EJB specification assigns to the
EJB container the responsibility for
managing threads. Allowing enterprise
bean instances to create and manage
threads would interfere with the
container's ability to control its
components' lifecycle. Thread
management is not a business function,
it is an implementation detail, and is
typically complicated and
platform-specific. Letting the
container manage threads relieves the
enterprise bean developer of dealing
with threading issues. Multithreaded
applications are still possible, but
control of multithreading is located
in the container, not in the
enterprise bean.
That said, if the problem of startup and shutdown is not considered, it is partly a "philosophical" issue in the sense that thread is an implementation detail, and also the fact that multi-threading is considered a scalability concern, which should be managed by the app. server.
For instance, most app. servers allow the integrator to define pools and configure the number of threads, etc. An app that spawns thread itself escapes this configuration, and does not cooperate nicely in the scalability plan.
Also, if you want a single background thread in a clustered environment, it becomes tricky.
And finally, the app. server controls the transactions. If you spawn threads yourself, you must take care to understand all the details of what can be used safely or not (e.g. get a connection from the pool) and how to use UserTransaction if necessary. The idea is that you shouldn't worry about such detail if you use an app. server, but you will need to if you start dealing with threads yourself.
I've however seen web app spawning a background thread from a ServletContextListener, and guess what, that was fine, even if the app was deployed on more than one node. You just need to understand what it means to have several JVM running and make sure you support that correctly.
There are a lot of issues, depending on your use case. What if the particular server in the cluster that your thread/job is running on crashes, which makes your thread go away, would that be a bad thing? Should someone be notified? Should the job move over to another server in the cluster? Should it restart once the server starts up again? All of this, you've got to implement in your thread....or you could use JMS, which will even run in Tomcat, with the addon of ActiveMQ, or some other messaging container of your choice, and just write the code that executes your logic, and let the container worry about all the rest of this.
YMMV
I have read that it is possible with Tomcat 5.5+ to deploy a war to a Tomcat server without a restart. That sounds fantastic but I guess I am too skeptical about this functionality and it's reliability. My previous experience (with Websphere) was that it was a best practice to restart the server to avoid memory problems, etc. So I wanted to get feedback as to what pitfalls might exist with Tomcat.
(To be clear about my experience, I developed java web apps for 5 years for a large company that partitioned the app developers from the app server engineers - we used Websphere - so I don't have a lot of experience with running/configuring any app servers myself)
In general, there are multiple type of leaks and they apply to redeploy-scenarios. For production systems, it's really the best to perform restarts if possible, as there are so many different components and libraries used in todays applications that it's very hard to find them all and even harder to fix them. Esp. if you haven't got access to all source code.
Memory leaks
Thread and ThreadLocal leaks
ClassLoader leaks
System resource leaks
Connection leaks
ClassLoader leaks are the ones which bite at redeployment.
They can be caused by everything. Really, i mean everything:
Timers: Timers have Threads and Threads created at runtime inherit the current context class loader, which means the WebappClassloader of Tomcat.
ThreadLocals: ThreadLocals are bound to the thread. App servers use Thread pools. When a ThreadLocal is bound to a Thread and the Thread is given back to the pool, the ThreadLocal will stay there if nobody removes() it properly. Happens quite often and very hard to find (ThreadLocals do not have a name, except the rarely used Spring NamedThreadLocal). If the ThreadLocal holds a class loaded by the WebappClassloader, you got a ClassLoader leak.
Caches: e.g. EhCache CacheManager
Reflection: JavaBeans Introspector (e.g. holding Class or Method caches)
JDBC Drivers: they shouldn't be in the .war file anyway. Leak due to static registry
Static libraries which cache ClassLoaders, such as Commons-Logging LogFactory
Specific to Tomcat, my experience is as follows:
For simple apps with "clean" libraries, it works fine in Tomcat
Tomcat tries very hard to clean up classes loaded by the WebappClassloader. For example, all static fields of classes are set to null when a webapp is undeployed. This sometimes leads to NullPointerExceptions when code is run while the undeployment is happening, e.g. background jobs using a Logger
Tomcat has a Listener which cleans up even more stuff. Its called org.apache.catalina.core.JreMemoryLeakPreventionListener and was submitted recently to Tomcat 6.x
I wrote a blog post about my experience with leaks when doing redeployment stresstesting - trying to "fix" all possible leaks of an enterprise-grade Java Web Application.
Hot deployment is very nice as it usually is much faster than bringing the server up and down.
mhaller has written a lot about avoiding leaks. Another issue is for active users to have their session survive the application "reboot". There are several things that must be taken care of, but which all in all means that their session must be serializable and THEN deserialize properly afterwards. This can be a bit tricky if you have stateful database connections etc, but if your code is robust against database hickups anyway that shouldn't be too bad.
Also note that some IDE's allow updating code inside the WAR (in the same way as applications) when saving a modified source file, instead of having to redeploy. MyEclipse does this rather nicely.