Develop Reference

Find unresolved maven dependencies in multi-module maven project

I'd like maven to report unresolved dependencies in multi-module maven Java project which has below structure:
multi-module-java-app/
├── app1
│ ├── pom.xml
│ └── src
├── app2
│ ├── pom.xml
│ └── src
└── pom.xml
poms are at the bottom.
Background:
Maven is used as a dependency management and build tool. Artifactory is repository manager. Artifacts may be built using maven locally on a developer’s environment or on Jenkins build server. Artifactory periodically moves artifacts to special archive repository which is part of all repository virtual.
Maven caches locally built artifacts in~/.m2 directory on the computer where Maven runs whether it’s developer environment or build server.
Problem
Several issues may arise:
Local builds on developers’ VMs may succeed, but fail in Jenkins.
Local builds, Jenkins builds may succeed, but fail on another developer’s VM.
Cause
Present/missing artifacts on a developer .m2 cache, missing artifacts in build server’s .m2 cache and/or archive Artifactory repository
Proposed solution
Run [path_to_maven]mvn dependency:list. [path_to_maven] has custom maven installation (with empty .m2 cache) in the root folder of the project. Custom maven is also configured using settings.xml to use special non-archived repository (all repository without the archive).
The output is like below:
It reports unresolved dependencies as well as dependent artifacts which miss them.
However this solution has 2 main drawbacks:
.m2 missing slows down the detection significantly as all the dependencies have to be downloaded
unresolved artifacts or modules which miss them are always snapshots.
Running mvn -B -q versions:set -DnewVersion=[some_version] solves the second. This command runs during release pipeline anyway.
However it's not clear how to solve the first.
How to find unresolved maven dependencies while using .m2, so that unresolved dependencies may be detected quickly during Jenkins build after each push to feature branch?
The only idea that pops is that .m2 on build server will be synced with Artifactory.
.m2 on developers' machines can be synced as well using some sort of custom plugin that uses rsync. Is there a known plugin that does this?
The ultimate goal is to remove archive repository and let the builds fail. But, first developers need to align the dependencies to the latest versions.
root pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycompany.app</groupId>
<artifactId>multi-module-java-app</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>
<name>multi-module-java-app</name>
<!-- FIXME change it to the project's website -->
<url>http://www.example.com</url>
<pluginRepositories>
<pluginRepository>
<id>plugins</id>
<name>plugins</name>
<url>http://localhost:8081/artifactory/all</url>
</pluginRepository>
</pluginRepositories>
<repositories>
<repository>
<id>all</id>
<name>all</name>
<url>http://localhost:8081/artifactory/all</url>
</repository>
</repositories>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.example</groupId>
<artifactId>spring-boot</artifactId>
</dependency>
</dependencies>
</dependencyManagement>
<modules>
<module>app1</module>
<module>app2</module>
</modules>
</project>
app 1 pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.mycompany.app</groupId>
<artifactId>multi-module-java-app</artifactId>
<version>1.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<groupId>com.mycompany.app.app1</groupId>
<artifactId>app1</artifactId>
<version>1.0-SNAPSHOT</version>
<name>app1</name>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<finalName>${artifactId}</finalName>
</configuration>
</plugin>
</plugins>
</build>
<pluginRepositories>
<pluginRepository>
<id>plugins</id>
<name>plugins</name>
<url>http://localhost:8081/artifactory/all</url>
</pluginRepository>
</pluginRepositories>
<repositories>
<repository>
<id>all</id>
<name>all</name>
<url>http://localhost:8081/artifactory/all</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>com.example</groupId>
<artifactId>spring-boot</artifactId>
<version>0.0.1-20200510.095344-1</version>
</dependency>
</dependencies>
</project>
app2 pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.mycompany.app</groupId>
<artifactId>multi-module-java-app</artifactId>
<version>1.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<groupId>com.mycompany.app.app2</groupId>
<artifactId>app2</artifactId>
<version>1.0-SNAPSHOT</version>
<name>app2</name>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<finalName>${artifactId}</finalName>
</configuration>
</plugin>
</plugins>
</build>
<pluginRepositories>
<pluginRepository>
<id>plugins</id>
<name>plugins</name>
<url>http://localhost:8081/artifactory/all</url>
</pluginRepository>
</pluginRepositories>
<repositories>
<repository>
<id>all</id>
<name>all</name>
<url>http://localhost:8081/artifactory/all</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>com.example</groupId>
<artifactId>spring-boot</artifactId>
<version>0.0.1-20200510.095344-1</version>
</dependency>
</dependencies>
</project>

I have run into this exact issue where the developer's local build environment was different from Jenkins Slave environment. In the ideal world, the developer needs to baseline their local environment with that of the slave, or depend entirely on Jenkins job builds once initial stages of development are complete.
I appreciate the fact that you are trying to provide an automated sync feature of the .m2 repositories, it is feasible but adds scope for error and additional routine maintenance tasks, not to mention user education issues. For instance, how will you ensure that the .m2 is the latest version? In terms of maven dependencies, the developer knows best, or they may be introducing new dependencies which do not exist on the slave yet. Thereby, I suggest fixing the root problem of developer not aligning their dependencies which is more design related than automation.
Not sure if you want to take this path, but may help someone:
Eliminate the need for .m2 repositories in local machines of developers. The m2 cache creates a problem, if the developer's machine gets wiped out or corrupt and there will be a need for updates, audits and reconciliation.
Eliminate the need for .m2 in Jenkins slaves. The problem here is, multiple slaves tend to have different .m2 cache content and syncing back and forth from Artifactory and then developers also syncing it to their local sounds complicated. There is no saying that all these .m2 will be in sync at any point of time and a builds may still get executed with an n-1 version of it.
Now that there is no .m2 , we still need a place for developers to pull dependencies from. Push all your dependencies to a repo in Artifactory and actively maintain it. Use the setting.xml feature to pull the standard dependencies. If a developer is building in his local machine, via Eclipse or other, the dependencies will be available to be pulled from the IDE itself by using the same xml reference so there is no local cache maintained on the developer's machine.
When the build environment has a minor difference between the local and the Jenkins Slave This causes the .jar to be the same size or slightly different in kilobytes of size.
To identify this difference between jar files use any tools listed here in this post, it also helps developers to identify which dependency is out of sync, by themselves:
Comparing two .jar files
If this design is implemented, the Artifactory repository containing the dependencies becomes the single source of truth for dependencies, you need to work with developers to create a cadence as to how the dependencies will be updated and consumed to/from this single source of truth. I hope this is useful.

Import Maven local .jar as a lib

I made a simple Maven project for my class. According to the teachers tutorial we cannot upload it to our school repo due to some server issues, so we have to store it locally using altDeploymentRepository. I have the following pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.dpp</groupId>
<artifactId>simple_lib</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>1.6</maven.compiler.source>
<maven.compiler.target>1.6</maven.compiler.target>
</properties>
<build>
<plugins>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
<configuration>
<altDeploymentRepository>internal.repo::default::file://${project.basedir}/../${project.name}-mvn-repo</altDeploymentRepository>
</configuration>
</plugin>
</plugins>
</build>
</project>
So in the directory with my Maven project I have two directories:
sample_lib
sample_lib-mvn-repo
In the second one, deep down in : sample_lib-mvn-repo\com\dpp\sample_lib\1.0-SNAPSHOT I have a .jar file which I want to import (but not using just .jar file like passing the path to it - I need to do this "Maven way", import it as Maven lib). Can I do it if the file is not stored on any remote repository, but on my hard drive?

Running simply mvn install will install the file in your local repository. The local repository, by default, is in your home directory, under .m2\repository.
Using your pom above, after running mvn install, you would have jar (and some other files) in .m2\repository\com\dpp\sample_lib\1.0-SNAPSHOT.
To import this subsequently in another project, you would create a dependency in that project's pom like:
<dependency>
<groupId>com.dpp</groupId>
<artifactId>simple_lib</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
This all takes place only on your machine, and will not use any remote repository.

Now you have a local simulation of a repository.
You can import it using the repository tag as described in https://maven.apache.org/pom.html#Repositories. To specify a file make it a file url like file:

Yes, you can add maven repository and point it to a local directory:
<repository>
<id>local</id>
<name>local</name>
<url>file:${user.dir}/sample_lib-mvn-repo</url>
</repository>
https://maven.apache.org/guides/introduction/introduction-to-repositories.html
Given that your jar file is here sample_lib-mvn-repo\com\dpp\sample_lib\1.0-SNAPSHOT.jar, you then can add it as a dependency:
<dependency>
<groupId>com.dpp</groupId>
<artifactId>sample_lib</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>

It's not exactly what you're asking. But a quick and dirty solution is to make a POM project (no source code).
Inside the POM project you have the main and external projects.
You can simply make a dependency on the other project.

How to configure Eclipse to automatically resolve and build multiple maven modules?

Here is my situation:
I'm trying to migrate from Ant to Maven
My project has 3 artifacts: shared api (jar), web app (war), desktop swing app (jar). Latter 2 depends on shared api.
At this moment I'm trying to make web app part work. So I've created 4 poms: eftracker (root pom), eftracker-parent, eftracker-shared, eftracker-web.
If I run mvn package on eftracker all works just perfect -- I have eftracker-shared.jar and eftracker-web.war created as expected
I added tomcat7-maven-pluginto run web app with maven goal tomcat7:run to test changes made during development
I also added eftracker-shared as a project to eftracker-web build path.
My goal:
Now I want to work comfortably in Eclipse, meaning I want to change files, hit Run and in couple seconds be able to test my changes.
During development I will change both: shared and web projects.
My problem:
If I never run mvn install than an attempt to invoke tomcat7:run will lead to error: Failed to execute goal on project eftracker-web: Could not resolve dependencies for project com.skarpushin:eftracker-web:war:1.503.0: Could not find artifact com.skarpushin:eftracker-shared:jar:1.503.0 in central (https://repo.maven.apache.org/maven2)
It appears I have to mvn clean install shared project (or even on root module) each time I change it before I can execute tomcat7:run on web app and see recent changes.
Question is:
Is it possible to make this process automatic?
...OR maybe there is other way how to minimize "maven overhead" during development?
eftracker.pom
<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.skarpushin</groupId>
<artifactId>eftracker</artifactId>
<version>1.503.0</version>
<packaging>pom</packaging>
<name>eftracker</name>
<modules>
<module>eftracker-parent</module>
<module>eftracker-shared</module>
<module>eftracker-web</module>
</modules>
<properties>
<maven.deploy.skip>true</maven.deploy.skip>
</properties>
</project>
eftracker-parent/pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.skarpushin</groupId>
<version>1.503.0</version>
<artifactId>eftracker-parent</artifactId>
<packaging>pom</packaging>
<name>eftracker-parent</name>
<!-- ...some common properties, dependencies, build plugins... -->
</project>
eftracker-web/pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>com.skarpushin</groupId>
<artifactId>eftracker-parent</artifactId>
<version>1.503.0</version>
<relativePath>../eftracker-parent</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>eftracker-web</artifactId>
<packaging>war</packaging>
<build>
<plugins>
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<version>2.2</version>
<configuration>
<path>/</path>
<port>8080</port>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.4</version>
<configuration>
<warSourceDirectory>src/main/webapp</warSourceDirectory>
<warName>ROOT##${project.version}</warName>
<failOnMissingWebXml>false</failOnMissingWebXml>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>com.skarpushin</groupId>
<artifactId>eftracker-shared</artifactId>
<version>${project.version}</version>
</dependency>
<!-- ...other deps -->
</dependencies>
</project>

Try to use M2Eclipse
https://www.eclipse.org/m2e/
M2Eclipse provides tight integration for Apache Maven into the IDE
with the following features:
Launching Maven builds from within Eclipse
Dependency management for Eclipse build path based on Maven's pom.xml
Resolving Maven dependencies from the Eclipse workspace without installing to local Maven repository
Automatic downloading of the required dependencies from the remote Maven repositories
Wizards for creating new Maven projects, pom.xml and to enable Maven support on plain Java project
Quick search for dependencies in Maven remote repositories

So it appears there are 2 things needs to be done:
run mvn compile on parent project in that way all classes will appear in ../parent/target/classes folder. Note that they'll be automatically updated by Eclipse if you change source code
edit Eclipse run configuration and put this checkbox "Resolve Workspace artifacts"
Now I was able to run project as Maven build... with goal tomcat7:run and it worked without the need of parent project to be installed

Can I put the packaged file into another directory?

I am wondering if I can put a packaged JAR or WAR into another directory using Maven. How can I do that?

You could specify a directory as your delivery location:
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.myspotontheweb.demo</groupId>
<artifactId>demo</artifactId>
<version>1.0</version>
<distributionManagement>
<repository>
<id>internal.repo</id>
<name>MyCo Internal Repository</name>
<url>file:///home/me/path/to/my/repo</url>
</repository>
</distributionManagement>
</project>
Just run maven as follows
mvn clean deploy
I suspect this is not what you're looking for...It's kinda pointless.... Running the install goal places the same artifact in the following default location
$HOME/.m2/repository
Some further guidance on why you want to do this might furnish a better answer

Distributing application with its Maven POM only

Is it possible to distribute my application with its POM only ?
I have deployed my application in a remote repository and I think it would be nice if I can distribute only its POM, instead of asking the users to download the complete source first and use the POM to build the application afterwards.
The idea is that users would be able to install the application using the POM and a single Maven command.
I tried adding to the POM a downloadURL in a distributionManagement section without success. Here my experiment:
<?xml version="1.0" encoding="UTF-8"?>
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>aGroupId</groupId>
<artifactId>anArtifactId</artifactId>
<version>0.0.1</version>
<distributionManagement>
<downloadUrl>anURL</downloadUrl>
</distributionManagement>
<repositories>
<repository>
<id>someId</id>
<url>anURL</url>
</repository>
</repositories>
</project>
Thanks in advance for any feedback

If you're distributing source, then you should look at the bootstrap POM method: http://maven.apache.org/scm/maven-scm-plugin/examples/bootstrapping-with-pom.html
where the 'scm' element is key. The user has only to run scm:bootstrap to then receive the project source tree from which to build the project.
-tim

You can use the dependency:get mojo to download an artifact to a specified location:
mvn dependency:get -Dartifact=<groupid>:<artifactid>:<version> /
-Ddest=path/to/destination.jar