We have numerous Java projects, which are CI built with Jenkins. These are deployed to our own Nexus server just fine.
The problem is, we need to provide these libraries to a third party, but without the source code.
So for each project, in Nexus we have:
Releases repository for our devs (includes deployed source code)
Snapshots repositories for our devs (includes deployed source code)
Third party release repository (only JAR + POM)
(and would be good to have): Third party snapshot repository (only JAR + POM) for third party nightly builds
The question is: how is this usually handled in Jenkins/Nexus world? I'd prefer to have one single Job in Jenkins which handles the CI build and the release (artefact deployment) process "automatically".
Currently I'm using multiple <distributionManagement> profiles in our "main root pom.xml" (included by all projects):
[...]
<profiles>
<profile>
<id>default</id>
<distributionManagement>
<repository>
<id>releases</id>
<name>Release</name>
<url>http://path/to/nexus/content/repositories/releases/</url>
</repository>
<snapshotRepository>
<id>snapshots</id>
<name>Snapshot</name>
<url>http://path/to/nexus/content/repositories/snapshots/</url>
<uniqueVersion>false</uniqueVersion>
</snapshotRepository>
</distributionManagement>
</profile>
<profile>
<id>third-party</id>
<distributionManagement>
<repository>
<id>releases</id>
<name>Release</name>
<url>http://path/to/nexus/content/repositories/third-party/</url>
</repository>
<snapshotRepository>
<id>snapshots</id>
<name>Snapshot</name>
<url>http://path/to/nexus/content/repositories/third-party-snapshots/</url>
<uniqueVersion>false</uniqueVersion>
</snapshotRepository>
</distributionManagement>
</profile>
</profiles>
From the Maven docs, it seems to be no way of using multiple repositories during the same build lifecycle, not to mention the fact that we need/don't need the source based on the target repo.
I can do a trick with creating a Job in Jenkins, with the Maven "Goals and options": clean deploy -P third-party and then adding the Post-build action - "Deploy artifacts to Maven repository" with the "default" data - but in this case, only SNAPSHOTs are going to both repo and artefacts released via Jenkins Maven Release Plug-in are going into one repository only.
Any practical ideas how can I do this without overcomplicating our CI job hierarchy?
Thanks in advance!
You can just handle this all in Nexus. Create a repository target that contains a pattern like the one used in the preconfigured example "All but sources (Maven 2)" and narrow that target down even further with another pattern that restricts the groupid, artifactid and maybe even version.
Then create a privilege that uses that repository target and assign it to the user or role you want to have the respective access.
No need to do multiple deployments or some such..
See http://books.sonatype.com/nexus-book/reference/repository-targets.html
You can use Maven Wagon Plugin and upload a single jar to a remote location on deploy phase.
Related
I followed the instructions to configure a maven client to use an Artifact Registry. I was able to deploy my root pom to it successfully; however, when I have a separate project that inherits from that root pom, it fails to build because it doesn't know where to find the root pom. This makes sense: the child project has no way of knowing that the parent pom is to be found in a random Artifact Registry repo. What is the standard practice for this?
What i've done before is in every programmer's computer, configure the <repository> in their settings.xml but this doesn't scale well + it needs human intervention + it's very cumbersome when dealing with CI since now you need to configure the settings.xml of the build robot. Considering there's so many CI systems nowadays I imagine there's a convention by now. Is there a better way to accomplish this? some way that's committed in the repo and where everything works right out of the box? git clone... mvn package... done. I ask specifically about Artifact Registry because it doesn't use a static username/password but uses the maven wagon to authenticate
One potential solution is to not put the <repository> in the root pom but in every project's pom but that leads to a lot of duplication since the config + the wagon is pretty verbose
Assuming your child and parent share a common .mvn/ folder, you can add:
.mvn/extensions.xml:
<extensions xmlns="http://maven.apache.org/EXTENSIONS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/EXTENSIONS/1.0.0 http://maven.apache.org/xsd/core-extensions-1.0.0.xsd">
<!--This has to be declared here in order to allow child modules to be able to fetch older parent POMs.-->
<extension>
<groupId>com.google.cloud.artifactregistry</groupId>
<artifactId>artifactregistry-maven-wagon</artifactId>
<version>2.1.1</version>
</extension>
<!--project-settings-extension allows using project-specific .mvn/settings.xml-->
<extension>
<groupId>com.github.gzm55.maven</groupId>
<artifactId>project-settings-extension</artifactId>
<version>0.1.1</version>
</extension>
</extensions>
.mvn/settings.xml (don't forget to use your actual artifactregistry URL):
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
<profiles>
<profile>
<id>default</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<repositories>
<!--This has to be declared here in order to allow child modules to be able to fetch older parent POMs.-->
<repository>
<id>cloud-artifacts</id>
<url>artifactregistry://...</url> <!--use actual full URL here-->
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
</repository>
</repositories>
</profile>
</profiles>
</settings>
The repository in settings.xml informs Maven to consider that location when searching for artifacts. The artifactregistry-maven-wagon extension allows Maven to access artifacts via the artifactregistry:// protocol.
The other extension (com.github.gzm55.maven) causes Maven to look at the project-specific settings.xml file (you could also just pass --settings=<path to file>, perhaps in .mvn/maven.config, but that would override any user settings).
I have a maven project in bitbucket account. I need to add that the project in my local maven repository so that I can refer that project as a maven dependency in my main project's pom.xml.
That is, perform below steps
1) Clone the project
2) run maven build install This will add to local maven repository
Please let me know if there is a way to achieve the same (in similar ways to connecting to maven public repository) ?
You can achieve this by defining repositories in pom file of the project where you want to use project 1 as a dependency.
For eg.
Git clone path of Project 1: https://github.com/yogi21jan/project1
Add below code in child project:
<repositories>
<repository>
<id>YOUR-PROJECT-NAME-mvn-repo</id>
<url>https://raw.github.com/YOUR-USERNAME/YOUR-PROJECT-NAME/mvn-repo/</url>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
</repository>
</repositories>
And finally Add this project as a dependency:
<dependency>
<groupId>group id of project 1</groupId>
<artifactId>artifcat id of project 1</artifactId>
<version>required version</version>
</dependency>
Look into jitpack.io. It can create a Maven repository for a GitHub project and also other hosters like BitBucket, GitLab, Azure, Gitee.
I have choosen a random Maven based project,
here fastconverter. It not GitHub is used, the full URL has to be provided. The created jar is published then here.
I am newbie using maven m2e plugin in my eclipse and JFrog Artifactory.
As given in the instructions I downloaded JFrog Artifactory war and deployed it using Tomcat 8 Windows installer service. I am using windows 8.1 64 bit OS.
Once the artifactory been deployed through tomcat i added some jars required for my project through Deploy option in artifactory into ext-release-local and maven-clean, other such jars to plugins-relase-local.
Then i generated settings.xml file and copied it to .m2 file. My settings.xml file consists of
<settings xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd">
<profiles>
<profile>
<repositories>
<repository>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>central</id>
<name>libs-release</name>
<url>http://localhost:8082/artifactory/libs-release</url>
</repository>
<repository>
<snapshots/>
<id>snapshots</id>
<name>libs-snapshot</name>
<url>http://localhost:8082/artifactory/libs-snapshot</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>central</id>
<name>plugins-release</name>
<url>http://localhost:8082/artifactory/plugins-release</url>
</pluginRepository>
<pluginRepository>
<snapshots/>
<id>snapshots</id>
<name>plugins-snapshot</name>
<url>http://localhost:8082/artifactory/plugins-snapshot</url>
</pluginRepository>
</pluginRepositories>
<id>artifactory</id>
</profile>
</profiles>
<activeProfiles>
<activeProfile>artifactory</activeProfile>
</activeProfiles>
</settings>
And then when I tried to clean install -X -e the project I am getting the following exceptions.
[ERROR] Plugin org.apache.maven.plugins:maven-clean-plugin:2.4.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-clean-plugin:jar:2.4.1: Failure to find org.apache.maven.plugins:maven-clean-plugin:pom:2.4.1 in http://localhost:8082/artifactory/plugins-release was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]
org.apache.maven.plugin.PluginResolutionException: Plugin org.apache.maven.plugins:maven-clean-plugin:2.4.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-clean-plugin:jar:2.4.1
How to integrate Artifactory with Maven.
The primary goal is only for the very first time the jars and other dependencies should be fetched from central repository, all other times the jars should be fetched from local repos.
how to deploy jars in bundle. Example : Deploying all jars of spring 4.X in a single deployment process.
Any guide or proper documentation or rules of thumb explaining proper way of maintaining repositories etc.,
How to change admin username and password of Artifactory. creating multiple users in Artifactory with user defined roles. etc.,
More very simple best practices/ tutorial explaining how to configure and use Artifactory with Maven
One more issue with the Artifactory web UI is its often freezing and when i check it out with the tomcat service in task manager the tomcat freezes with 'stopping..'. And ended up in connection reset error in browser. I cant stop the service immediately through tomcat configurer. The tomcat is taking too long time to stop and restart. Any clues to properly use tomcat deploy?
Thanks
Your Artifactory should proxy external repositories with the most common artifacts and plugins. E.g. the maven-clean-plugin should be resolved without any problems. If Artifactory is behind proxy, you should configure it in the Admin screen as described in the User Guide.
I would generally suggest going though it, it should answer all the mentioned questions.
Regarding freezes - if you deployed the war into existing Tomcat installation without configuring its JVM paramteres, the freezes you see are full GC cycles - you are running out of memory. That brings us to the User Guide again.
When work in STS (springsource tool suite), open pom.xml file and check dependencies in "dependencies tab", add/select maven dependency will give 0 result found. We need edit pom.xml to add dependency by hand. Is there a way to let M2E know where to check repository to search dependency?
M2e by default searches maven central repository for artifacts. It downloads an index file which has artifact details. You can configure additional repositories to be searched by specifying the same in repository section of your settings.xml file. Do note that some repositories do not have this index file.
By default , all pom.xml will automatically extend the Super POM , which is located in the maven_installation_folder/lib/maven-x.x.x-uber.jar ==> package org.apache.maven.project ==> pom-4.0.0.xml . All the configuration specified in the Super POM is inherited by the POMs you created for your projects.
If you open pom-4.0.0.xml , you will find that the maven central repository http://repo1.maven.org/maven2/ is defined here . That means if you specify a <dependency> in your pom.xml , maven will try to download this dependency in the following orders:
Maven local repository (i.e your local hard disk)
Maven central repository specifed in the Super POM (i.e http://repo1.maven.org/maven2/)
Maven remote repository (Defined in the <repository> section of your pom.xml )
Normally , I will use some maven repository search engines , such as this , to find out the <dependency> of the libraries /frameworks /tools that I want to use and then paste them in the pom.xml
If the dependency exists in the Maven central repository , everything will be fine and it will be download to your local repository . However , if some of the <dependency> (eg Hibernate) cannot be found and downloaded from the Maven central repository , you can try to visit its official site to find out its repository link and paste them in the <repository> section of your pom.xml .For example , Hibernate require to define the jboss repository in the pom.xml like this:
<repositories>
<repository>
<id>repository.jboss.org</id>
<name>JBoss Repository</name>
<url>http://repository.jboss.org/nexus/content/groups/public-jboss</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
You can refer to this for the use of <repository>
So I have a project that depends on a snapshot version of another project. The dependency is:
<dependency>
<groupId>org.oop</groupId>
<artifactId>oop</artifactId>
<version>0.9.9-SNAPSHOT</version>
</dependency>
For the oop project, I did do a 'mvn clean deploy', so the snapshot version should be somewhere in the maven central repository. But when I do a mvn clean install, the snapshot dependency above cannot be resolved and I get this:
Missing:
1) org.oop:oop:jar:0.9.9-SNAPSHOT
Try downloading the file manually from the project website.
Then, install it using the command:
mvn install:install-file -DgroupId=org.oop -DartifactId=oop -Dversion=0.9.9-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file
Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=org.oop -DartifactId=oop -Dversion=0.9.9-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
Is there a way to make maven download the snapshot automatically? I must be missing something here.
EDIT1: On my settings.xml I have:
<server>
<id>sonatype-nexus-snapshots</id>
<username>XXXXXX</username>
<password>XXXXXX</password>
</server>
<server>
<id>sonatype-nexus-staging</id>
<username>XXXXXX</username>
<password>XXXXXX</password>
</server>
EDIT2:
Just add this to your ~/.m2/settings.xml:
<profiles>
<profile>
<id>allow-snapshots</id>
<activation><activeByDefault>true</activeByDefault></activation>
<repositories>
<repository>
<id>snapshots-repo</id>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
<releases><enabled>false</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
</profile>
</profiles>
To update snapshots, try with the -U option
-U,--update-snapshots Forces a check for updated
releases and snapshots on remote
repositories
However, you said:
I did do a 'mvn clean deploy', so the snapshot version should be somewhere in the maven central repository.
This is just not possible, your snapshot is going somewhere else. If I do a mvn clean deploy without configuring my personal repository I get:
Deployment failed: repository element was not specified in the POM inside distributionManagement element or in -DaltDeploymentRepository=id::layout::url parameter
To enable deployment, there is some configuration to be added to pom.xml, like for instance:
<distributionManagement>
<!-- Publish versioned releases here -->
<repository>
<id>myrepo</id>
<name>My releases</name>
<url>http://nexus.mycompany.com/nexus/content/repositories/releases</url>
</repository>
<!-- Publish snapshots here -->
<snapshotRepository>
<id>myrepo</id>
<name>my snapshots</name>
<url>http://nexus.mycompany.com/nexus/content/repositories/snapshots</url>
</snapshotRepository>
</distributionManagement>
<repositories>
<repository>
<id>myrepo</id>
<name>My Public Repository</name>
<url>http://nexus.mycompany.com/nexus/content/groups/public</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
Maven would try to download the snapshot automatically and indeed it does (as your error indicates). By default, Maven will look for newer snapshot versions once a day, but you can change that interval in your snapshot repository config (e.g. in settings.xml):
<updatePolicy>interval:5</updatePolicy>
This will make maven check every 5 minutes (if you build that often). Alternatively, you could use the -U or --update-snapshots option, to force the check manually.
However, it can't find the dependency. Could you post your repo settings and artifact config for the snapshot dependency?
Is the org.oop:oop:jar:0.9.9-SNAPSHOT artifact in your repository?
... so the snapshot version should be somewhere in the maven central repository.
No it isn't. I tried to look it up, but couldn't find it. Afaik, there's some staging mechanism, so maybe your settings are just wrong. But normally, as the others already said, you'd go and use your own repository manager like Artifactory or Nexus.
Does that dependency exists in your repository? (in pom.xml or settings.xml)?
Looks like not. By the way, take a look at your config, just you are not using -o (offline). Also you can use -U to refresh snapshots.
You can either
use a parent project which builds all your snapshots, or
deploy your snapshots to your maven build server (nexus/archiva/..) using e.g., mvn:deploy
Let's clear up terminology a bit to make sure there is no misunderstanding.
"Maven Central" (http://search.maven.org/) is a global site where you only find releases. Central doesn't accept snapshots so deploying there should give you an error.
You probably mean your local/company wide maven proxy/cache. These can also be configured to reject snapshot versions. In case of Nexus, you can also define more complex rules. In my case, I had an issue there which gave no error during mvn deploy but I could see an error in the server's logs.
Try to follow the data: Enable debug (mvn -X) to see where Maven uploads the data. Then check the server to see whether the artifacts were really uploaded. Check the server's logs for errors.
Also note that snapshot dependencies are only refreshed once a day; so this won't work:
PC #1: mvn install -> Error missing dependency
PC #2: mvn deploy
PC #1: mvn install -> Dependency is still missing because of "update once per day" policy
Try mvn install -U to force Maven to refresh its cached metadata.
I hit the issue of snapshots not updating even when setting -U on the command line. For me the issue was my client was Maven 3 and the server was Maven 2, and in Maven 3 unique snapshots are no longer supported. We had to create a new repository with timestamped snapshots to support the 3.xx clients.