Develop Reference

Multi-module project: Exclude a dependency from the parent pom

We have a Maven multi-module project with some 30 child projects/module in it.
All child projects where using those two dependencies. (This is just to put something concrete in the example)
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-cassandra-reactive</artifactId>
</dependency>
Hence, we refactored and put those two dependencies to the parent POM. The snippet from above is now in the parent level. All child projects could still benefits from it, and have to only maintain the version in one place, all very happy.
We now have a 31st project which fits the business use cases and fits inside this multi module project, we believe it makes sense to have it under this same parent POM.
However, this 31st project does not need one particular dependency from the Parent POM. (In this case the Cassandra dependency, but the question is about how to exclude something from the parent)
Having this 31st project part of this multi module, he also takes this dependency (Cassandra) from the parent. How to tell this child project to exclude this dependency from the parent?
I do not want to extract this project entirely and have it separated from the multi module.
I tried putting an exclusion in ALL dependencies, but it is still there. Like in all dependencies of my child pom, I write this:
<dependency>
<groupId>some.group</groupId>
<artifactId>some.artifact</artifactId>
<exclusions>
<exclusion> <!-- declare the exclusion here -->
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-cassandra-reactive</artifactId>
</exclusion>
</exclusions>
</dependency>
I even tried doing the crazy refactor of moving the dependency away from the parent back to all children, and did 30 copy paste to the child, with my 31st project not having it. This works, but I believe there is something smarter than that.
How to exclude one particular dependency from the Parent POM please?

The way I normally do this is by using a multi-level hierarchy of poms.
This means that instead of a single parent-pom with just one row of non-parent poms underneath it, I have an entire tree of parent poms consisting of a root-parent-pom with several levels of intermediate-parent-poms as its children, ending with the non-parent-poms as the leaves of the tree.
By adding intermediate parent-poms you can have the root parent only provide the dependencies that are truly used by all children, while intermediate poms can provide additional dependencies that are only used by their children but not by children of their siblings.
So, in your case, your root parent pom would declare all dependencies except cassandra, and it would have two children: your casandraless project, and your existing parent pom which would now become an intermediate parent pom and it would only declare the cassandra dependency. So, the children of the intermediate parent pom would inherit all the dependencies except cassandra from the root, and then cassandra from the intermediate parent pom.
I have tried this, it works very well.
It is true that there may theoretically exist situations that cannot be covered, because parent poms can only form a tree, and a tree is not a graph, but in practice I never came across a situation where I wanted to arrange my dependencies in such a way that this mechanism could not cover.
As an example, take a look at my public home projects, which I have rolled all together into one repository for the sake of more easily managing them:
https://github.com/mikenakis/Public
https://github.com/mikenakis/Public/blob/master/pom.xml is the root pom.
https://github.com/mikenakis/Public/blob/master/testana/pom.xml is an intermediate parent pom,
https://github.com/mikenakis/Public/blob/master/testana/testana-console/pom.xml is a child pom. (A leaf.)
Now, in my projects it just so happens that I only declare dependencies in the leaf poms, because my projects do not really depend on many external libraries. But I could be declaring dependencies in the root pom and in the intermediate parent poms if I wanted to, and their children would be inheriting them.

Why isn't Maven overridden scope recognized transitively?

This is an interesting state of affairs in Maven that I didn't expect. Maybe someone can explain exactly why this is happening.
I have a parent POM foobar-parent that declares logback-classic with a test scope in the <dependencyManagement> section.
I have a separate project project example that has its own example-parent, which inherits from foobar-parent and also serves as a parent to its submodules.
One submodule example-foo overrides the dependency logback-classic and gives it compile scope:
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<scope>compile</scope>
</dependency>
Lastly I have another submodule example-bar which uses example-foo as a dependency.
Strangely, for the effective POM of example-bar, it shows that logback-classic has test scope!! Since example-foo declares logback-classic to be of compile scope (meaning it is required at compile time), and since example-bar has a compile-time dependency to example-foo, I expected example-bar to bring in logback-classic as a transitive dependency.
The way I interpret this is that the test scope specified in the <dependencyManagement> management section of a parent POM will override the scope of a transitive dependency from the compile scope!! Is this a correct interpretation, and is that how Maven is supposed to work?

You are right: "dependency management takes precedence over dependency mediation for transitive dependencies" (taken from Introduction to the Dependency Mechanism)

Java9 Multi-Module Maven Project Test Dependencies

I have a multi-module maven project with three modules core, utils and test-utils
Core has the following dependencies definition
<dependency>
<groupId>my.project</groupId>
<artifactId>utils</artifactId>
</dependency>
<dependency>
<groupId>my.project</groupId>
<artifactId>test-utils</artifactId>
<scope>test</scope>
</dependency>
I have added Java 9 module-info.java definitions for all three modules and core's looks like this:
module my.project.core {
requires my.project.utils;
}
However I cannot figure out how to get core's test classes to be able to see the test-utils classes during test execution. When maven-surefire-plugin attempts the test run I get class not found.
If I add a requires my.project.testutils; to core's module-info.java:
module my.project.core {
requires my.project.utils;
requires my.project.testutils; //test dependency
}
Then at compile time I get an error that the my.project.testutils module can't be found (presumably because it's only brought in as a test dependency).
How does one work with test dependencies in a Java 9 modular world? For obvious reason's I don't want my main code to pull in test dependencies. Am I missing something?

With maven and java9, if your my.project.testutils is a test scope dependency, you don't need to explicitly include(requires) it in the module descriptor.
The test dependencies are taken care via the classpath itself. So you can simply remove the testutils and it would be patched by maven while executing tests.
module my.project.core {
requires my.project.utils;
}
Refer to the slide 30 pertaining to maven-compiler-plugin.
I would also suggest you take a look at Where should I put unit tests when migrating a Java 8 project to Jigsaw and this comment by Robert confirming on the implementation that maven follows.
Edit: Created a sample project drawing an analogy that the main module is same as your core, the dependency on guava is same as your utils and the junit dependency is same as your testutils.

Why sometimes dependency not contain version property in pom.xml?

I am a newbie of Maven, currently reading Hadoop source code, and found something interesting in some pom.xml files:
Some of the dependency node do not contain version node at all.
Question: why is it like this?
for instance, this pom.xml.

Because specific version of dependency in parent pom.xml file
https://github.com/apache/hadoop/blob/trunk/pom.xml
Reference: https://maven.apache.org/guides/introduction/introduction-to-the-pom.html

As I commented at first, a pom file can have a parent (via inheritance) and such a parent may provide some governance and harmonization across all of its children. A classic case is to provide versioning for certain dependencies via a dependencyManagement section.
is used by POMs to help manage dependency information across all of its children. If the my-parent project uses dependencyManagement to define a dependency on junit:junit:4.0, then POMs inheriting from this one can set their dependency giving the groupId=junit and artifactId=junit only, then Maven will fill in the version set by the parent. The benefits of this method are obvious. Dependency details can be set in one central location, which will propagate to all inheriting POMs.
The mentioned pom has indeed a parent pom:
<parent>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-project-dist</artifactId>
<version>3.0.0-SNAPSHOT</version>
<relativePath>../../hadoop-project-dist</relativePath>
</parent>
Which in chain has another parent pom file which defines several dependencies as part of its dependencies management section.
If you really want to check the effective (merged) pom your build is using, you could run:
mvn help:effective-pom -Doutput=effective-pom.xml
And the maven-help-plugin will produce an additional pom as specified by the command above, merging the current pom file and all of its anchestors.

In Maven you can inherit from parents folder in order to merge or inherit some properties. This can be the version of the modules. Usually you have a "super" POM in the root folder of your project and you put there all the commons dependencies in order to controll them in an easier way. I.e. If you must change one module version, you only need to change in the "super" POM and not in each POM inside each subfolder that need it. If you need more information about POM inheritance the documentation has a couple of useful examples.
https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Project_Inheritance

Get classpath for integration tests of multi-module project with maven-ant-task

I have a multi-module project built with maven. I need to run the project's integration tests daily. It is not possible to do this during the standard maven build cycle, because on runtime the integration tests defined within the modules have circular dependencies, which are illegal for me to declare on their poms.
Instead, I have created a separate project named Global that lists all modules jars and test-jars as its dependencies. Global has the same parent as all the modules. The idea is that using maven-ant-tasks I will be able to get a classpath of all modules jars and test-jars and go on from there. Global's pom.xml dependency section is as follows:
<dependency>
<groupId>mygroup</groupId>
<artifactId>A</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>mygroup</groupId>
<artifactId>A</artifactId>
<version>${project.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>mygroup</groupId>
<artifactId>B</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>mygroup</groupId>
<artifactId>B</artifactId>
<version>${project.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
...etc
The problem is that I cannot seem to get a classpath that contains all jars and test-jars declared on Global's pom.xml (and their runtime dependencies) using the ant tasks available. I have tried (among other things):
<dependencies pathId="cp1" type="jar" usescope="runtime">
<pom file="${basedir}/pom.xml">
<profile id="DEV" />
</pom>
</dependencies>
[1] This one fetches all runtime dependencies. Nothing wrong with that.
<dependencies pathId="cp2">
<dependency groupId="mygroup" artifactId="Global" version="myVersion" scope="test" type="test-jar"/>
</dependencies>
[2] This one fetches all runtime dependencies along with Global-myversion-tests.jar, but no other test-jar.
<dependencies pathId="cp3" type="test-jar" usescope="test">
<pom file="${basedir}/pom.xml">
<profile id="DEV" />
</pom>
</dependencies>
[3] This one fetches nothing.
Obviously, declaring something like [2] once for each module will do the trick, but I am looking to create a setup that will not need to edit a gazillion files each time a new module is added or removed. BTW I am using maven-ant-task-2.1.3.
Thanks for any input.
---Edits for #yannisf accepted answer---
You should not ever have cyclic dependencies
I assume you mean for maven builds. Having cyclic dependencies on runtime is pretty common, for example:
Module A declares interface: UploadToDocumentManagementSystem
Module B implements it in : UploadToCoolDms (that way in the future, when the DMS system changes to CoolerDms module B can be replaced by a new implementation with no side-effects to the rest of the app).
Module B depends on A compile time (and, by definition, runtime as well)
Module A depends on B on runtime
Maven does not allow to declare this. The reason, that I can sympathize with, is that maven needs to complete build cycles (including tests) of multi-module projects in a specific order. Thing is, it is not really necessary to declare it if you get rid of any runtime dependecy to B for the tests of A (which is good practice and should happen anyway).
You should do things the maven way instead of resorting to ant-tasks
Fair enough, I can see how maven-ant-tasks was not made for this use.
In your global pom you are declaring dual types for the same artifact (jar, test-jar)
Is that a problem in general? For example module A contains some samples for its tests that I would like to use in the tests of module B as well. Is it wrong (by maven best practices standards) to declare that B depends on A jar (compile scope) and on A test-jar (test scope)? Won't an integration tests project justify to depend on a module as well as the same module's samples and resources used for its unit tests?
tl;dr version: I will attempt to rearrange the tests declared on the modules and create separate module(s) for integration tests (assuming I can get 20 developers to play ball). Thanks for the answer and for making me admit defeat and stop trying to make maven work with the project instead of making the project work with maven :).

You are trying to break the maven conventions in many ways. 1. You should not ever have cyclic dependencies, 2. You should do things the maven way instead of resorting to ant-tasks 3. In your global pom you are declaring dual types for the same artifact (jar, test-jar).
Although at first this might not seem to answer your question, you should take a step back and rethink your layout. Integration tests need all the dependencies and are much more demanding than unit tests. So, instead of trying to fit them into the existing projects, create a separate maven project in the same group, that will only host integration tests (under src/java/test, main will be blank) and will have as dependencies all the other projects.