Develop Reference

Maven: how to exclude or set provided to dependencies that dragged by other dependencies

I've problem that in large java project that is using maven to build it.
I've many dependencies that the main app is using like ( 40 ).
The problem is that those dependencies are dragging more dependencies
and some of them I can't and don't want to include in my final build
this is dependencies hell!!
How do I exclude the specific dependencies in maven in my main app pom ?
I don't want it in my app final deployment.
For example if I set this in my main pom, it doesn't put all the dependencies of spring-boot-starter-tomcat that are dragged from other dependencies in provided , only the top level .
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<scope>provided</scope>
</dependency>

Maven brings in all the dependencies that a library requires and builds it for the runtime. Most cases you would want this to happen as you dont want to go through each dependency of your dependencies and then their inner dependencies (its a hellish job to do, please don't do that). But in some cases you might want to avoid bringing in few inner dependencies as you have the same at project level, for those cases maven lets you exclude them by <exclusions>.
The below sample is from maven official documentation:
<project>
...
<dependencies>
<dependency>
<groupId>sample.ProjectA</groupId>
<artifactId>Project-A</artifactId>
<version>1.0</version>
<scope>compile</scope>
<exclusions>
<exclusion> <!-- declare the exclusion here -->
<groupId>sample.ProjectB</groupId>
<artifactId>Project-B</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</project>
For deeper knowledge go here
Hope this helps !!

If you set a dependency to scope provided, all the transitive dependencies below also become provided - unless the <dependencyManagement> overwrites the versions/scopes of the dependencies. Have a look at mvn dependency:tree to figure out.
The command mvn dependency:list will list you exactly what Maven considers a dependency of your project. In your final artifact, you can either set all the unwanted dependencies as provided or figure out below which artifact they are included and write exclude statements as #vizsatiz said.

Maven: Resolve dependencies using container dependencies first

I have a plugin project which is added to other container projects as a dependency.
Now, this plugin project uses many frequent dependencies like spring-security, commons-lang, etc.
Usually, the container projects contain their own versions of such frequent dependencies. So, when we add our plugin dependency there are conflicts and the dependencies are resolved based on regular maven dependency resolver and depending on scopes and optional tags provided in the plugin project dependencies.
Is there a way where all the dependencies are resolved using the version in parent dependencies first and iff they are not available then use the version specified in plugin dependency.
Note: optional and scope runtime have a problem that these dependencies are provided by the container and thus beats the aim to provide a hassle-free single dependency to add plugin dependency.

In your plugins pom define the version of a dependency as range of the versions you know the plugin to be able to use. If a container-dependency overlaps this will be used. If no overlapping version, of the dependency both container and plugin need, can be found, an error will be produced, since the negotiation failed.
Use no special scope for the dependencies, since you want them to be included if necessary into the container,
See:
https://maven.apache.org/enforcer/enforcer-rules/versionRanges.html
And:
https://books.sonatype.com/mvnref-book/reference/pom-relationships-sect-project-dependencies.html#pom-relationships-sect-version-ranges

Assuming that your container and plugin projects use the same parent pom you could utilize the <dependencyManagement> section in the parent to define the common artifacts. This allows you to omit the version in the plugins <dependencies> section.
parent:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>group-a</groupId>
<artifactId>artifact-a</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
</dependencyManagement>
plugin/module:
<dependencies>
<dependency>
<groupId>group-a</groupId>
<artifactId>artifact-a</artifactId>
</dependency>
</dependencies>
See https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html for further details.

you can exclude it when you build a plugin project and add a dependency to maven.
This is an example. Dependency and main project have conflicted due to logging library. Below is to exclude log4j in dependency project.
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>${zk.version}</version>
<exclusions>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
</exclusions>
</dependency>
P/S: Added from my comments:
I have also developed a system which has a similar architecture with yours. I separate this system into 3 main parts: 1. Commons which contains common code and required maven dependencies, 2. The main project, 3. plugin project. You can refer this.

Sharing src/test classes with Maven without version specification for test-jar

I'm sharing src/test classes between number of modules, in a similar way described in attaching tests guide and the following question.
So, I have the following pom.xml dependencies:
<dependency>
<groupId>com.myco.app</groupId>
<artifactId>foo</artifactId>
</dependency>
<dependency>
<groupId>com.myco.app</groupId>
<artifactId>foo</artifactId>
<version>1.0.0-SNAPSHOT</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
BUT, in opposite to the question above, when attaching the test-jar, i don't want to specify the specific test-jar version. As in the compile level dependency:
<dependency>
<groupId>com.myco.app</groupId>
<artifactId>foo</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
In this case, my pom.xml become erroneous with message about the missing version. Why is this happen? Why i can specify dependency without versions but not the test-jar one? Is there a way to overcome this and make the test-jar to use the latest jar it can find?

The reason the main code dependency could be used without the version, is the existence of a "main pom" in our project that automatically generates appropriate version for each dependency in section. Therefore, each dependency can be specified without specific version number.
Test-jar dependency on the other hand, don't have it's version defined anywhere else in the transitive dependency so, the specific version must be specified.

maven dependency without version

Recently I've been working on some improvements in a project developed some time ago, and here's what I found. A lot of dependencies in the pom files go without versions specified, and yet they are resolved. The project consists of 1 root module and 2 submodules. The Aggregator pattern is used, meaning there's no dependencyManagement section at all. The upper-project simply aggregates 2 modules and that's all it does. Subprojects don't refer to it as to a parent. They have a different parent. What I can't grasp is that neither subprojects themselves nor their parent(as a matter of fact, it doesn't have dependencyManagement either) specify versions for some of the dependencies. For instance:
<dependency>
<groupId>javax.mail</groupId>
<artifactId>javax.mail-api</artifactId>
</dependency>
<dependency>
<groupId>com.sun.mail</groupId>
<artifactId>javax.mail</artifactId>
</dependency>
<dependency>
<groupId>com.sun.mail</groupId>
<artifactId>imap</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jul-to-slf4j</artifactId>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</dependency>
Can someone help me figure this out? Is maven handling versioning with some default strategy? What is that default strategy?

Ok, I think I'm gonna answer it myself. Of course I took a look at dependency:tree, but all the dependencies that I mentioned were first-level members of the tree. What I failed to notice right away, is that dependencyManagement is not present in the parent, but it is however present in the submodules, and what is more interesting it contains:
<dependency>
<groupId>io.spring.platform</groupId>
<artifactId>platform-bom</artifactId>
<version>1.0.2.RELEASE</version>
<type>pom</type>
<scope>import</scope>
</dependency>
I've never used Spring IO Platform before, so this is a totally new concept for me. As it turns out the platform includes quite a few preconfigured dependencies:
http://docs.spring.io/platform/docs/current/reference/htmlsingle/#appendix-dependency-versions

It is impossible for maven to work without defining versions of the artifacts. They should be defined somewhere in dependencyManagement tag either in the submodule or parent. Please check your pom hierarchy. Use mvn help:effective-pom in the submodule directory of the project. Also you can use mvn dependency:tree in order to find out which artifacts - along with full artifact information including version numbers - are resolved in the result of dependency management.

Use
mvn -P<my_profile_of_interest> help:effective-pom -Dverbose
Verbose mode (Since: 3.2.0) adds XML comments containing precise reference to a place where dependency declaration is coming from.

Each maven dependency defined in the pom must have a version either directly or indirectly for example, through dependencyManagement or parent. That being said, if the version is not given, then the version provided in the dependencyManagement or the parent pom will be used.
For example: in the pom (only important sections are mentioned) given below, no version is provided for the artifact jstl. However, in the "mvn dependency:tree", it shows that jstl version 1.2 is included. And looking at the spring-boot-starter-parent, for the version 2.3.3.RELEASE pom, it includes jstl version 1.2.
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.3.3.RELEASE</version>
</parent>
<dependencies>
....
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
</dependency>
....
</dependencies>

In my case if i was using Spring boot starter parent to manage all dependency and lombok version is managed by Spring boot , This problem was coming due to higher java version JAVA 11 . I exported JAVA 8 in to my compile time environment and after using JAVA 8 this problem was gone.

How to avoid duplication of derived dependencies in EAR/WAR with Maven?

To make the example as simple as possible, let's say I have classic Java EE 5 application.
Let's say I use x-lib in EAR module, and x-lib uses commons-io.
Also I use y-lib in WAR module, and y-lib uses commons-io too.
The EAR is set as provided in WAR.
The result I get is:
- app.ear
/lib
x-lib.jar
commons-io.jar
/app.war
/WEB-INF
/lib
y-lib.jar
commons-io.jar
I don't want commons-io to be packaged in app.war/WEB-INF/lib once it is packaged already in app.ear/lib.
In my war's pom.xml I have:
<dependency>
<groupId>my.group</groupId>
<artifactId>app</artifactId>
<type>ejb</artifact>
<scope>provided</scope>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>some.other.group</groupId>
<artifactId>y-lib</artifactId> <!-- This loads commons-io as compile dependency -->
<version>1.2.3</version>
</dependency>
Is there a way to tell maven that I want everything that is provided along with app ejb dependency should be set to provided and not included in WAR?
I do not want to track all those duplicated JARs and set them as provided or exclude explicitly one by one.
EDIT
I am aware of skinny-war solution. However I don't like the drwaback of duplicating dependencies in WAR and EAR. Maybe you know how to overcome this.

From the version 2.7 of the maven-ear-plugin, you can use SkinnyWars.
See here for specs: https://maven.apache.org/plugins/maven-ear-plugin/examples/skinny-wars.html
Basically, you can add the skinnyWars tag to your plugin configuration:
<plugin>
<artifactId>maven-ear-plugin</artifactId>
<version>2.10.1</version>
<configuration>
<defaultLibBundleDir>lib/</defaultLibBundleDir>
<skinnyWars>true</skinnyWars>
</configuration>
</plugin>

That SHOULD be automatic if you add the EJB module as a provided dependency to your war module and let all the dependencies of that EJB come transitively. So don't add the overlapping dependencies to your war pom, just let Maven deal with it.

Is there a way to tell maven that I want everything that is provided
along with app ejb dependency should be set to provided and not
included in WAR?
In this case a think the best option by now is to use wilcards in dependency exclusion.
It is only supported in maven 3.
Below is an example of how to declare the ejbmodule dependency from a war project:
<dependency>
<groupId>ejbModuleGroupId</groupId>
<artifactId>ejbModuleArtifactId</artifactId>
<scope>provided</scope>
<type>ejb</type>
<exclusions>
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
You can use this same strategy when declaring a dependency to a ejb-client.
Depending on your maven 3 version, you'll get a nasty warning, but in newer versions it must be fine.

You need to add an exclusion like this:
<dependency>
<groupId>some.other.group</groupId>
<artifactId>y-lib</artifactId>
<version>1.2.3</version>
<exclusions>
<exclusion>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
</exclusion>
</exclusions>
</dependency>