Develop Reference

Springboot why setting spring-boot-starter-tomcat as provided

When deploying a Springboot maven project(version 2.3.4.RELEASE) to an external Tomcat container,official guide says you need to mark the "spring-boot-starter-tomcat" dependency as provided ,but actually even if without doing that,the final war package which contains lib like "spring-boot-starter-tomcat","tomcat-embed-core" and "tomcat-embed-websocket" also works fine in tomcat8.5.54 or tomcat 9.0 ,so I am confused about that,"Do we really need set spring-boot-starter-tomcat as provided or not?" ,anyone could explains why?
"Traditional Deployment"

You don't want to have multiple versions of the same classes on the classpath. This can lead to many errors during runtime. For example, if you have a
public class MyServlet implements javax.servlet.Servlet
and you package both MyServlet.class and javax.servlet-api.jar (which contains javax.servlet.Servlet) into your application, you might get an error:
Servlet class MyServlet is not a javax.servlet.Servlet
What happens is: when the application classloader loads MyServlet, it looks for javax.servlet.Servlet in the application first and it finds it in javax.servlet-api.jar. The server compares this class with the one loaded by the server's classloader and concludes that they differ, since classes from different JARs are not equal. If the application is shipped without javax.servlet-api.jar this does not happen: the classloader does not find javax.servlet.Servlet in its own classpath, so it looks in the parent classloader.
Remark: This example can not actually be reproduced on Tomcat. Due probably to many incorrectly packaged applications the WebappClassLoader has an exception to the class loading rule: special classes such as those starting with javax.servlet or org.apache.tomcat are always loaded from the server's classloader (cf. the source code for a list of these exceptions).
TL;DR: due to the remark above, leaving spring-boot-starter-tomcat in the compile scope probably will do no harm, but it is a risk not worth taking.

How do optional compile-time dependencies work?

For the longest time, I thought that in Java you either had one of two types of dependencies:
Required compile-time dependencies (dependencies always required at compile time)
Possibly optional runtime dependencies (dependency that can be
resolved at runtime)
Recently, I found out that compile dependencies can be optional too. For example, commons-beanutils is listed as an optional compile dependency of JXPath.
How can this work? Can a dependency really be used at the time of compilation yet remain fully optional?
EDIT: I might have been unclear. I'm looking for a case where a dependency is used at compile-time and is at the same time fully optional, or an explanation why such a dependency is impossible.

A class can compile to an interface but the implementation of that interface is not needed during compilation. The implementation is needed during runtime.
Example commons-logging, JPA, JDBC etc which are frameworks, an application can compile based on these. At runtime an implementation is needed to execute the code. Sample implementations - Common Bean utils, Oracle thin driver, Eclipse link etc.

An extensive quote from Maven documentation describes this quite clearly:
Optional dependencies are used when it's not possible (for whatever reason) to split a project into sub-modules. The idea is that some of the dependencies are only used for certain features in the project and will not be needed if that feature isn't used. Ideally, such a feature would be split into a sub-module that depends on the core functionality project. This new subproject would have only non-optional dependencies, since you'd need them all if you decided to use the subproject's functionality.
However, since the project cannot be split up (again, for whatever reason), these dependencies are declared optional. If a user wants to use functionality related to an optional dependency, they have to redeclare that optional dependency in their own project. This is not the clearest way to handle this situation, but both optional dependencies and dependency exclusions are stop-gap solutions.
Why use optional dependencies?
Optional dependencies save space and memory. They prevent problematic jars that violate a license agreement or cause classpath issues from being bundled into a WAR, EAR, fat jar, or the like.
How do optional dependencies work?
Project-A -> Project-B
The diagram above says that Project-A depends on Project-B. When A declares B as an optional dependency in its POM, this relationship remains unchanged. It's just like a normal build where Project-B will be added in Project-A's classpath.
Project-X -> Project-A
When another project (Project-X) declares Project-A as a dependency in its POM, the optional nature of the dependency takes effect. Project-B is not included in the classpath of Project-X. You need to declare it directly in the POM of Project X for B to be included in X's classpath.
A practical example: imagine that you are a developer of a library/framework SuperLib that is built as one superlib.jar. Your library provides multiple features. Its main feature (that most of the users use) is dependency injection based on a third-party di library. However, one of your classes - EmailApi - offers features to send e-mails, using a third-party email library. Since superlib is one artifact, it needs both di and email to be compiled.
Now put yourself in the position of a user who uses superlib. They are interested in the dependency injection features. This is the core role of your library, so the dependency between superlib and di would not be optional.
However, most users are not interested in sending emails and may be bothered by having a useless email library and its dependencies added to their application (which will cause size increase of their application and may cause a dependency version clash between the dependencies of email and dependencies of the user's application). Therefore, you would mark the dependency on email as optional. As long as the user does not use your EmailApi class, everything will run fine. However, if they do use EmailApi, they will need the email dependency, otherwise the application will fail at runtime with ClassNotFoundException for whichever class from email would be referenced in EmailApi. The user of your library will need to add the email dependency explicitly in their POM.
See also When to use <optional>true</optional> and when to use <scope>provided</scope>.

What you described is actually a feature of Maven, the build tool, but not Java itself.
Without build tools, using just 'javac' you need to specify all classes or interfaces that directly used in your code. Sure there are options for dynamic class loading and even runtime compilation, but thats not on topic.
One of use-cases with separation on interface and implementation is described in previous answer, another popular case is based on classpath scanning:
if some specific class is present in classpath and/or has specific annotation - an optional module will be loaded.
That's how Spring Boot modules are loaded.

How to find all classes with my annotation | Wildfly JavaEE

What is the best way to find all classes with a specific annotation in a JavaEE deployment? I work with a WildFly ApplicationServer and have created my own annotation. At the start of my deployment I would like to find all classes with my annotation and cache them for later access. What do I have to do to protect resources?

First of all Wildfly has modular class loading structure. In Wildfly doc:
Since JBoss AS 7, Class loading is considerably different to previous versions of JBoss AS. Class loading is based on the JBoss Modules project. Instead of the more familiar hierarchical class loading environment, WildFly's class loading is based on modules that have to define explicit dependencies on other modules. Deployments in WildFly are also modules, and do not have access to classes that are defined in jars in the application server unless an explicit dependency on those classes is defined.
As described above, if you have more .ear or .war package, they cannot see classes within each other unless explicit dependency defined. Each module can see its own class therefore each module should load and cache its own classes. You can not load all classes with a single class and single method.
With the following code you can see all loaded classes in the module.
Field f = ClassLoader.class.getDeclaredField("classes");
f.setAccessible(true);
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
Vector<Class> classes = (Vector<Class>) f.get(classLoader);

REST client inside of OSGi application

I need to integrate a REST client into an existing OSGi application implemented using Apache Felix. The REST service is based on RESTeasy implementation (version 2.3.2.Final) of JAX-RS. I created a separate bundle with clients' dependencies, exporting required RESTeasy packages and importing them in the bundle where the client is used, but unfortunately I cannot get it working inside of the OSGi context.
I tried two different approaches. First one using the generic ClientRequest:
ClientRequest request = new ClientRequest(MyService.URL_TEST+"/stats");
request.body(javax.ws.rs.core.MediaType.APPLICATION_XML, stats);
ClientResponse<String> response = request.post(String.class);
The error that I get in this case is pretty weird:
[java] java.lang.RuntimeException: java.lang.ClassCastException:
org.jboss.resteasy.client.core.executors.ApacheHttpClient4Executor cannot be cast to
org.jboss.resteasy.client.ClientExecutor
where I it is known for sure that ApacheHttpClient4Executor implements the ClientExecutor interface.
When I try to use my own REST client wrapper around RESTeasy like this:
MyService myService = MyServiceClient.getInstance();
myService.saveStatistics(stats);
I get a different exception:
[java] java.lang.LinkageError: ClassCastException: attempting to
castjar:file:/D:/Development/Eclipses/eclipse_4.2_j2ee_x64/lib/jaxrs-api-2.3.2.Final.jar
!/javax/ws/rs/ext/RuntimeDelegate.classtobundle:
//78.0:1/javax/ws/rs/ext/RuntimeDelegate.class
As far as I understand, the LinkageError most probably has to do with the way RESTeasy initializes the RuntimeDelegate using some classloader tricks, which probably fall under the restrictions of OSGi framework. I get the suspicion that the java.lang.ClassCastException mentioned first has the same source.
Is there any way to get RESTeasy working inside of OSGi?
PS: discussion about a similar issue with RESTeasy, but outside of OSGi: java.lang.LinkageError: ClassCastException
Update:
these are the libraries included into restclient bundle:
activation-1.1.jar commons-codec-1.2.jar commons-httpclient-3.1.jar commons-io-2.1.jar commons-logging-1.0.4.jar flexjson-2.1.jar httpclient-4.1.2.jar httpcore-4.1.2.jar javassist-3.12.1.GA.jar jaxb-api-2.2.3.jar jaxb-impl-2.2.4.jar jaxrs-api-2.3.2.Final.jar jcip-annotations-1.0.jar jettison-1.3.1.jar jsr250-api-1.0.jar junit-4.10.jar log4j-1.2.14.jar resteasy-jaxb-provider-2.3.2.Final.jar resteasy-jaxrs-2.3.2.Final.jar resteasy-jettison-provider-2.3.2.Final.jar scannotation-1.0.3.jar slf4j-api-1.6.4.jar slf4j-log4j12-1.6.4.jar myservice-common-0.1.0.3.jar my-service-client-0.1.0.3-SNAPSHOT.jar stax-api-1.0-2.jar xmlpull-1.1.3.1.jar xpp3_min-1.1.4c.jar xstream-1.4.2.jar
These are the exports from the restclient bundle: javax.ws.rs, javax.ws.rs.ext, javax.ws.rs.core, org.jboss.resteasy.client, org.jboss.resteasy.client.cache, org.jboss.resteasy.client.extractors, org.jboss.resteasy.client.marshallers, org.jboss.resteasy.client.core.executors, javax.xml.bind.annotation, org.jboss.resteasy.plugins.providers, org.jboss.resteasy.plugins.providers.jaxb, org.jboss.resteasy.spi

Have a look at the SpringSource Bundle Repo, it's got some very useful pre-built bundles of common libraries including the Apache HTTP Client which we are using (in conjunction with gson) to do our RESTful comms.

(unfortunately a legacy module of my project still uses OSGi, but using RESTeasy 3.0.16 now)
When I need to OSGify a dependency my preferred solution now is to wrap it using the excellent Apache Ops4j Pax Tipi project.
The project provides a preconfigured Maven setup (parent POM handles the bundling) and you just have to adapt the GAV coordinates of the original project in a Tipi sub module with a org.apache.ops4j.pax.tipi prefix and build the new bundle project which draws in the original dependency, unpacks and wraps it as OSGi bundle.
You can start from an existing Tipi sub project that best matches your project setup (dependencies, etc.) and adapt any OSGi imports/exports missing (most often, these are created automatically by the maven-bundle-plugin anyway).
This worked quite well for me as long as the original project did not contain too many exotic or malformed dependencies.
However you may run into snags like transitive dependencies using the root package, as I currently experience, which can be a real show stopper (finding out which library is a real nightmare).
Unfortunately, RESTeasy seems to be affected by this, as I get exactly the same error (default package , even after declaring non-test and non-provided dependencies as optional:
The default package '.' is not permitted by the Import-Package syntax.
Upgrading the maven-bundle-plugin to the latest release 3.0.1 yields a different error (even less helpful):
[ERROR] Bundle org.ops4j.pax.tipi:org.ops4j.pax.tipi.resteasy-jaxrs:bundle:3.0.16.Final.1 : Can not parse name from bundle native code header:
[ERROR] Error(s) found in bundle configuration
Update seems to be solved by upping Tipi version in POM to 1.4.0, testing...

Is RESTEasy mandatory ?
I personally use jersey in OSGi and it is working perfectly, both as client and server.

This problem isn't limited to RESTeasy. It also occurs with Jersey.
It is occurring because you have two copies of the JAX-RS classes on the classpath.
You can see this in the LinkageError:
[java] java.lang.LinkageError: ClassCastException: attempting to cast jar:file:/D:/Development/Eclipses/eclipse_4.2_j2ee_x64/lib/jaxrs-api-2.3.2.Final.jar!/javax/ws/rs/ext/RuntimeDelegate.class to bundle://78.0:1/javax/ws/rs/ext/RuntimeDelegate.class
i.e. one copy is coming from:
D:/Development/Eclipses/eclipse_4.2_j2ee_x64/lib/jaxrs-api-2.3.2.Final.jar
and the other from the OSGI bundle.
This causes problems for the RuntimeDelegate class, which by default uses the system class loader to create the RuntimeDelegate implementation (see javax.ws.rs.ext.FactoryFinder).
The problem can also occur if the same jar is loaded via two different class loaders.
There are a couple of workarounds:
remove the jaxrs-api-2.3.2.Final.jar from the system class path
set the thread context class loader to that of your bundle, prior to making any JAX-RS calls.
The FactoryFinder will use this to load the RuntimeDelegate.
To avoid polluting your code with calls to Thread.currentThread().setContextClassLoader(myBundleClassLoader), you can wrap your JAX-RS client using a Proxy. e.g. see the Thread context classloader section of https://puredanger.github.io/tech.puredanger.com/2007/06/15/classloaders/

Spring cannot resolve type in WEB-INF/libs

I recently start a spring web project with HBase. The problem is，spring seems unable to resolve types under WEB-INF/libs. it complains about "unresolved org.apache.hadoop.conf.Configuration" which is indirectly referenced from required .class file, which is actually in a .jar under the WEB-INF/lib library. Is there anything to be set for spring container to find it？

You need to make sure that Spring itself was loaded by the same class loader: the WAR class loader. This class loader should include all JARs in WEB-INF/lib and all .class files in WEB-INF/classes.
There's a way to pass a class loader into Spring to use a different class loader than the one that loaded it (or the Thread's context class loader), but that gets more complicated.

OK I got the problem. It seems that my project depended on hadoop-core-1.0.2 whilst hbase depend on hadoop-core-1.0.0 and that confused Spring. I fixed the dependency and problem solved.