Develop Reference

How do frameworks (like Spring) configure the Servlet container without web.xml?

(I already know the answer, but because I often find myself re-looking for the answer, so I post it here as documentation for myself and others. This is encouraged on Stackoverflow.)
Background intro
Many Servlet developers have read the book "Head First Serlet & JSP" to get their "Certified Web Component Developer Exam" or just to learn Servlets. But the book hasn't been updated since 2009, and only covers Servlet 2.4. A lot has changed since then. The current latest version is 4.0.
One of the things that changed is the boot process of servlet web applications, which can make it unclear what happens during startup and how the web application is initialized.
Question
In Servlet version 2.4 and lower, web.xml is used to fully configure web applications. But higher version seem to have other ways to configure web applications, without touching web.xml and without annotations. For example, web frameworks provided as .jar files are somehow able to hook into the Servlet container and add url mappings.
How does this mechanism work?

Introduction
In Servlet 2.4 (November 2003) a servlet container (such as Tomcat and Jetty) simply booted the web application by looking for the file WEB-INF/web.xml (the deployment descriptor). The file web.xml contains references to servlets, filters and listeners, with their associated url patterns and parameters. Using web.xml the servlet container knows exactly where to find everything and how to configure them.
Since Servlet 3.0 (December 2009), web.xml is optional, you can also use annotations or a programmatic configuration instead.
Annotations are simpler in use. They are located in the javax.servlet.annotation package, and allow you to annotate a servlet with #WebServlet, a filter with #WebFilter, and a listener with #WebListener. The servler container will then automatically find and detect these classes. Annotations however do provide less configuration features than web.xml and a programmatic configuration.
This text further focuses at how programmatic configuration can be configured and how it starts Spring MVC. It's a bit more complex than annotations, but does give you and framework designers more control over the booting process.
If you wanted to use a web framework before Servlet version 3.0, you have to add a servlet or filter to your web.xml and configure the framework from there. After this initialization, you could start writing classes (usually non-Servlet classes) known by the web framework to create a web application.
Since Servlet 3.0, the system is modular. This allows framework and library designers to initialize the servlet container without you having to configure the framework via web.xml. You can immediately start writing the web framework specific classes to create a web application without touching Servlet classes.
(It's also possible in Servlet 3.0 to create your own web.xml and still let the framework do the initialization of the framework, without it being defined in web.xml.)
How can a framework or library hook into the Servlet container automatically?
When booting up, the Servlet container first looks for the deployment descriptor located at WEB-INF/web.xml. If this file has a metadata-complete attribute set to false, or if it isn't defined at all, the container will also search for annotated classes, such as #WebServlet.
Apart from looking for web.xml and annotated classes, since Servlet 3.0, the container will also look for META-INF/web-fragment.xml files within the .jar files located in the WEB-INF/lib directory.
The file web-fragment.xml is a web fragment, which is
(quote from Java Servlet Specification)
a logical partitioning of the web application in such a way that the
frameworks being used within the web application can define all the
artifacts without asking developers to edit or add information in the
web.xml. It can include almost all the same elements that the web.xml
descriptor uses. However the top level element for the descriptor MUST
be web-fragment and the corresponding descriptor file MUST be called
web-fragment.xml. The ordering related elements also differ between
the web-fragment.xml and web.xml
The content of web-fragment.xml is like web.xml, but with a web-fragment root element instead of web-app element:
<web-fragment xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
https://java.sun.com/xml/ns/javaee/web-fragment_3_0.xsd"
version="3.0">
<filter>
<filter-name>FrameworkFilter</filter-name>
<filter-class>framework.FrameworkFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>FrameworkFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
</web-fragment>
The exact ordering of loading of a single web.xml and multiple web-fragment.xml files, can be configured using the <absolute-ordering> and <ordering>.
Apart from web fragments, there is also a programmatic method to partition the web application: by writing an implementation of the interface javax.servlet.ServletContainerInitializer. ServletContainerInitializer gives access to the ServletContext, which contains methods to programmatically add servlets, filters and listeners.
To use ServletContainerInitializer, it has to be specified in a file located in META-INF/services/javax.servlet.ServletContainerInitializer. The content of this file has to be the fully qualified path to the implementation class.
How does this work for Spring MVC?
Although Spring MVC does contain a web-fragment.xml, it doesn't define any servlets, filters or listeners. Spring uses the META-INF/services/javax.servlet.ServletContainerInitializer file to refer to its own ServletContainerInitializer implementation class, which is the class SpringServletContainerInitializer.
SpringServletContainerInitializer is a ServletContainerInitializer, so it receives a ServletContext on startup. The goal of SpringServletContainerInitializer is to pass the servletContext to a more developer friendly WebApplicationInitializer, so that you can add servlets, such as Springs DispatcherServlet, which is a front-controller that directs incoming requests to other controllers. (See spring-framework-reference about how to configure a DispatcherServlet to Spring.)
Spring MVC doesn't provide a concrete implementation of WebApplicationInitializer, only a number of abstract classes, so that you have control over the booting process. In case of Spring Boot, a concrete implementation is provided: SpringApplicationWebApplicationInitializer to reduce the amount of boilerplate code.
Further info
A detailed description about how the Servlet container booting process exactly works, can be found in the official Java Servlet Specification. Further info about Spring MVC can be found in the Spring Framework reference.

Single servlet for application

Is it possible to have an application such as shopping cart, to have a single servlet in the entire application, which will result into a single entry in web.xml? If yes, how?

Sure. There are in fact several webapp frameworks (e.g. Struts and JSF) based on a single servlet that dispatches incoming requests to application-specific controller classes after mapping URL components to controllers and parameters.

Servlet Filters - Context Initialization

When using a "global" filter, one which is located in the root lib directory of an apache tomcat webserver and applies to all web application contexts:
I was wondering if the web application contexts are initialized synchronously - one at a time, or can there be a case where the Filter.init() function is called by two different contexts in the same time.
I'm asking this since I want to initialize a database connection configuration object based on a specific FilterConfig.initParameter the first time init() is called by any of the application contexts, and I'm wondering if I should put the above in a synchronized block or not...

I think it's the internals of Tomcat, it could change any time so I'd do it in a synchronized block. It's not a big deal or performance issue because it runs only once per application context initialization.
From Java Servlet Specification 6.2.1 Filter Lifecycle:
Only one instance per <filter> declaration in the deployment
descriptor is instantiated per JVM of the container.
You could reach the same result with a custom Valve:
The Valve Component
Valve javadoc

Basic question about servlet mapping

I am new to J2EE and related stuff in general. I am trying to move a particular web application from a Sun One server deployment on to JBoss. The application is filled with a lot of servlets where each one re-directs to another.
There are way too many servlets for me to enter a mapping between each of these servlet class to a URL-mapping individually in web.xml. The application code has many redirects where they name the servlet class names itself in the redirect-URL. As of now when I run it on JBoss the redirections to URL with servlet classnames in URL don't seem to work on JBoss (it gives a 404: Not Found, probably since there is no mapping in the web.xml). So is there any config setting that I can set to allow this to happen or should manually enter each and every URLpattern-to-Servlet mapping in the web.xml?
Thank you.

There are two solutions.
As we know JBoss uses Tomcat under the hood as a servlet container. You can enable invoker servlet, that would save you from mapping the whole lot in web.xml. But beware, it will be naive to do that, and not at all encouraged.
Secondly, you can write another servlet/filter and map just that in your web.xml for every url pattern, may be. Then that new servlet of your can forward the requests to whatever servlet it should.
I hope you get my point.

Not sure what you mean by this
the application code has many redirects where they name the servlet class names itself in the redirect-URL
Do you have hardcoded urls in the servlet classes? How many servlets? If you have hardcoded urls they may all have broken because the context is slightly different, or the appname, etc.etc. Can you post an example?

Well, there are some hard-coded URLs in the code, but even if i typed in the right URLs in browser directly I still get a 404. There are about 30 servlets (a conservative approximation). Ex: http://FQDN_SERVER.com/?arg1=ABCD&arg2=XYZ
Here the servlet-classname is literally the classname of the servlet without ".class" extension, which may not be a good practice. But the code is full of such redirections and if I have to change this then I have to add a new url-pattern to each of these servlets in the web.xml and construct a new red-rect URL for each of these servlets. So is there anyway I can avoid this or will I have to go through the pain of doing the above mentioned?
Thanks,
Manoj

Sorry the URL-pattern looks this http://FQDN_SERVER.com/servlet-classname?arg1=ABCD&arg2=XYZ

Protecting internal view layer template pages in servlet applications

I have a very basic question about MVC web applications in Java.
Since the olden days of raw JSP up until current technologies like Seam, a very basic pattern has always been the internal dispatch from the controller that initially accepted the request to the view layer that creates the output to be sent to the client.
This internal dispatch is generally done (although the mechanism may be hidden through an extra layer of configuration) by asking the servlet container for a new resource using a URL. The mapping of these URL are done by the same web.xml that also defines the "real" URL to the outside.
Unless special measures are taken, it is often possible to directly access the view layer directly. Witness the Seam "registration" demo, where you can bypass "register.seam" and directly go to "registered.xhtml". This is a potential security problem. At the very least, it leaks view template source code.
I am aware that this is only a basic sample application, but it is also strange that any extra measures should need to be taken to declare these internal resources invisible to the outside.
What is the easiest way to restrict URL entry points?
Is there maybe something like the "WEB-INF" directory, a magic URL path component that can only be accessed by internal requests?

You can prevent access to internal resources by using a security-constraint in your web.xml deployment descriptor.
For example, I use the following configuration to prevent direct access to JSPs:
<!-- Prevent direct access to JSPs. -->
<security-constraint>
<web-resource-collection>
<web-resource-name>JSP templates</web-resource-name>
<url-pattern>*.jsp</url-pattern>
</web-resource-collection>
<auth-constraint/> <!-- i.e. nobody -->
</security-constraint>

I would not recommend allowing Internet requests to directly access your appserver. I'd throw a webserver in front, then in it, allow the request of certain kinds of URLs. Don't want people to go to foo.com/jsps? Restrict it once and for all there.
There's a bit of a conversation on the topic here: hiding pages behind WEB-INF?

One way to handle this would be to construct a Servlet Filter which examines the request path of every request and handles each request accordingly. Here is a link that could help get you started, JavaServer Pages (JSP) and JSTL - Access control with JSP

I have now seen a couple of applications that put their internal JSP into WEB-INF/jsp. That seems to do the trick, at least for JSP, and also for Velocity. It does not seem to work for JSF, though.