Relatively new to Ivy here.
I have a growing list of projects that all publish to the same repository under the same organization. And I'm trying to find out how, if at all possible, I can declare dependencies from another project to artifacts created by all those projects.
My ivisettings.xml already has a resolver that points to that repository location. All artifacts in that repo are from the same organization. I trying to determine if there's a way to easily include artifacts in that location as dependencies from another project. I was hoping that I could do this using a wildcard for its names, like:
<dependencies>
<dependency org="my.org.name" name="**" rev="${current.iteration}" conf="master" />
</dependencies>
But of course the above doesn't work. So, any pointer/comment/pointer suggestion would be welcomed. I can't be the first guy to want to do this, so there's got to be a way to do this and maybe I'm just not using the right keywords in my searches. Thanks.
Publish an extra ivy file that lists all the other projects as a list of dependencies.
<ivy-module version="2.0">
<info organisation="my.org.name" module="projects" revision="1.0.0" status="release" publication="20130215110241"/>
<dependencies>
<dependency org="my.org.name" name="projectA" rev="1.0.0"/>
<dependency org="my.org.name" name="projectB" rev="1.0.0"/>
<dependency org="my.org.name" name="projectC" rev="1.0.0"/>
..
..
</dependencies>
</ivy-module>
Creating a single dependency against this special ivy module will then pull in the other project artifacts as transitive dependencies.
<dependency org="my.org.name" name="projects" rev="1.0.0"/>
Enhancement
You could go one further and create different configurations to categorize your projects' artifacts:
<ivy-module version="2.0">
<info organisation="my.org.name" module="projects" revision="1.0.0" status="release" publication="20130215110241"/>
<configurations>
<conf name="web-apps" description="Projects which provide functionality for webapps"/>
<conf name="standalone" description="Projects which provide functionality for stand-alone java apps"/>
</configurations>
<dependencies>
<dependency org="my.org.name" name="projectA" rev="1.0.0" conf="webapps->default"/>
<dependency org="my.org.name" name="projectB" rev="1.0.0" conf="standalone->default"/>
<dependency org="my.org.name" name="projectC" rev="1.0.0" conf="webapps->default;standalone->default"/>
</dependencies>
</ivy-module>
Configurations are an extremely useful ivy feature. Client builds can use mappings to pull in artifacts associated with either webapp or standalone development. This feature is similar to "scopes" in Maven but much more powerful.
Update: Dynamic revisions
When publishing the "projects" module, you might want to use dynamic revisions to simplify maintenance.
<ivy-module version="2.0">
<info organisation="my.org.name" module="projects"/>
<dependencies>
<dependency org="my.org.name" name="projectA" rev="latest.release"/>
<dependency org="my.org.name" name="projectB" rev="latest.release"/>
<dependency org="my.org.name" name="projectC" rev="latest.release"/>
..
..
</dependencies>
</ivy-module>
The publish task will generate and push an ivy file resolved with the latest version of each project.
You will still need to list each project, but this is in fact a good thing. It means pulling in older versions of the "projects" module will result in same number of revisions at a point in time.
Generate ivy file
Finally to completely automate this process, using a variable number projects, perhaps you could generate the "projects" ivy file?
The following is a groovy snippet that you could use:
<groovy>
import groovy.xml.MarkupBuilder
new File("build/ivy.xml").withWriter { writer ->
def xml = new MarkupBuilder(writer)
xml."ivy-module"(version:"2.0") {
info(organisation:"my.org.name", module:"projects")
dependencies() {
new File("/path/to/projects/directory").listFiles().each { dir ->
dependency(org:"my.org.name", name:dir.name, rev:"latest.release")
}
}
}
}
</groovy>
Related
I need to build a custom Ant script that builds a project based on CI output. We use Atlassian Bamboo as CI server.
Normally our projects have a dependency to our platform module, managed via Ivy/Artifactory.
Our typical dependencies.xml file contains a dependency to that module, transitively. And other potential dependencies. As an example, our core module depends on lots of Spring packages, but not on Spring Boot. If a project needs Spring Boot too, it will define its dependency in its dependencies.xml file along with <depencency org="com.acme" name="core-platform"...
My goal now is to exclude com.acme#core-platform from resolution, because I am making a different task that uses Bamboo output artifact to take the latest build of the core module and its dependencies without going through Artifactory.
This is very important because during a build I may like to change the version of a dependent package (e.g. upgrade Spring 4.3.1 to 4.3.3) and test with the proper Spring. If I simply resolve dependencies to com.acme#core-platform#latest.release, which is released on Artifactory, I won't take 4.3.3 of Spring which was committed to Git and available in core-platform's currently-building dependencies.xml. I hope my explanation is easy to understand.
So let's say I have this dependency list as an example
com.acme#core-platform#${version}
org.hibernate#hibernate-java8#5.1.0.Final
org.springframework.boot#spring-boot-starter-web#1.3.1.RELEASE
commons-collections#commons-collections#3.2.2
.... others
Full dependency is
<dependencies>
<dependency org="com.acme" name="core-platform" rev="${version}" transitive="true" conf="runtime->runtime" changing="true"/>
<dependency org="com.acme" name="core-platform" rev="${version}" transitive="true" conf="compile->compile" changing="true"/>
<dependency org="com.acme" name="core-platform" rev="${version}" transitive="true" conf="provided->provided" changing="true"/>
<dependency org="com.acme" name="core-platform" rev="${version}" transitive="true" conf="junit->junit" changing="true"/>
<dependency org="com.acme" name="core-platform" rev="${version}" transitive="true" conf="test->test" changing="true"/>
<!-- https://mvnrepository.com/artifact/org.hibernate/hibernate-java8 -->
<dependency org="org.hibernate" name="hibernate-java8" rev="5.1.0.Final" transitive="false" />
<dependency org="org.springframework.boot" name="spring-boot-starter-web" rev="1.3.1.RELEASE" transitive="false" />
<dependency org="org.springframework.boot" name="spring-boot-starter-tomcat" rev="1.3.1.RELEASE" transitive="false" />
<dependency org="org.springframework.boot" name="spring-boot-starter-validation" rev="1.3.1.RELEASE" transitive="false" />
<dependency org="commons-collections" name="commons-collections" rev="3.2.2" transitive="false" />
<!-- jackson2 libs -->
<dependency org="com.fasterxml.jackson.datatype" name="jackson-datatype-jdk8" rev="2.8.1" transitive="false" conf="runtime->*"/>
<dependency org="com.fasterxml.jackson.datatype" name="jackson-datatype-jsr310" rev="2.8.1" transitive="false" conf="runtime->*"/>
<exclude module="joda-time" />
<exclude module="jackson-datatype-joda" />
</dependencies>
I simply want to take Hibernates' Java8, commons-collections, etc.
Creating a duplicate dependencies.xml is not an option
I was considering manipulating the dependencies.xml via Ant and have it exclude the acme modules by regex. Feasible but tricky
Unfortunately I can't combine Ant task's ivy:retrieve with attributes file and element exclude, because that would have helped a looooooot
Any ideas?
It is hard to understand your requirement. I suspect that your problem could be solved by creating an additional configuration and use configuration mappings to control the downloads.
Example
This build creates two directories. The first contains the log4j dependency without transitive dependencies, the second includes the remote module's optional dependencies. If you look at the remote POM you'll see they have a different scope.
├── build.xml
├── ivy.xml
├── lib1
│ └── log4j-1.2.17.jar
└── lib2
├── activation-1.1.jar
├── geronimo-jms_1.1_spec-1.0.jar
├── log4j-1.2.17.jar
└── mail-1.4.3.jar
build.xml
<project name="demo" default="resolve" xmlns:ivy="antlib:org.apache.ivy.ant">
<target name="resolve">
<ivy:resolve/>
<ivy:retrieve pattern="lib1/[artifact]-[revision](-[classifier]).[ext]" conf="noDependencies"/>
<ivy:retrieve pattern="lib2/[artifact]-[revision](-[classifier]).[ext]" conf="withDependencies"/>
</target>
</project>
Notes:
Each "retrieve" task creates a directory containing the files that make up the configuration.
ivy.xml
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="noDependencies" description="File grouping that has no transitive dependencies"/>
<conf name="withDependencies" description="File grouping that contains dependencies"/>
</configurations>
<dependencies>
<dependency org="log4j" name="log4j" rev="1.2.17" conf="noDependencies->master; withDependencies->master,optional"/>
</dependencies>
</ivy-module>
Notes:
Note how the configuration is declared at the top of the ivy file and the dependency contains two configuration mappings
Additional
The following answer explains how ivy interprets Maven modules. It creates configurations that can be used to decide which files should be downloaded:
How are maven scopes mapped to ivy configurations by ivy
Ok, looks like the replace trick is very easy too.
Add the following markers <!-- DEPS START --> and <!-- DEPS END --> (or any of choice) between the parts of the dependencies.xml file to ignore
Hack via Ant
<copy file="dependencies.xml" tofile="ci/hacked-dependencies.xml" overwrite="true">
<filterchain>
<replacestring from="<!-- DEPS START -->" to="<!--" />
<replacestring from="<!-- DEPS END -->" to="-->" />
</filterchain>
</copy>
Example
<!-- DEPS START -->
<dependency org="com.acme" name="core-platfrom" rev="${version}" transitive="true" conf="runtime->runtime"/>
<!-- DEPS END -->
I have tried to find the right org=".. address for JaCoCo so that I can simply add the address to my ivy.xml file and it will take care of the dependency.
But I have trouble finding the right link and was wondering if anyone knew what it was?
I am going to use Ant to build my project, so I'd have to put JaCoCo to use in there.
None of the below are correct:
<!-- JaCoCo -->
<dependency org="org.jacoco.agent" name="jacoco-agent" rev="0.6.2.2" />
<dependency org="org.jacoco.core" name="jacoco-core" rev="0.6.2.2" />
<dependency org="org.jacoco.report" name="jacoco-report" rev="0.6.2.2" />
You'll only need a single dependency declaration:
<dependency org="org.jacoco" name="org.jacoco.ant" rev="0.7.2.201409121644" />
The "core", "agent" and "report" jars get pulled in as dependencies.
Jacoco ANT task jar in Maven Central
The following is an build example that uses ivy to manage Jacoco and Sonar plugins. It's worth noting how it uses a "build" configuration mapping to create a classpath for just the ANT task jars.
integrating JaCoCo in sonar using Ant
I use following dependencies:
<dependency org="org.jacoco" name="org.jacoco.ant" rev="0.6.3.201306030806"/>
<dependency org="org.jacoco" name="org.jacoco.core" rev="0.6.3.201306030806"/>
<dependency org="org.jacoco" name="org.jacoco.agent" rev="0.6.3.201306030806"/>
<dependency org="org.jacoco" name="org.jacoco.report" rev="0.6.3.201306030806"/>
I tried to configure Ivy to download source code for dependencies. It works, but only for the top-level dependency. Maven will get sources for all levels of dependencies. How can I do this using Ivy?
Here is my ivy.xml file:
<ivy-module version="2.0">
<info organisation="com.ubs" module="IMQuant"/>
<dependencies defaultconfmapping="*->default,sources">
<dependency org="junit" name="junit" rev="4.11"/>
<dependency org="org.apache.maven.surefire" name="common-java5" rev="2.17"/>
<dependency org="com.googlecode.kevinarpe-papaya" name="kevinarpe-papaya" rev="0.0.15"/>
</dependencies>
</ivy-module>
Here is my retreive statement:
<ivy:retrieve pattern="${libs.dir}/ivy/all/[artifact]-[revision]-[type].[ext]" />
Above, dependency kevinarpe-papaya requires Google Guava. I see the JAR downloaded (actually a bundle), but I don't see the sources. I checked my Ivy cache. The sources are also missing for Google Guava.
Also, dependency junit requires Hamcrest (core). Again, I see the JAR downloaded, but I don't see the sources. (Also, I checked my Ivy cache.)
You could try to specify the type:
type="sources"
E.g.:
<dependency org="junit" name="junit" type="sources" rev="4.11"/>
If you want to retrieve different types, mention the same dependendy multiple times:
<dependency org="junit" name="junit" type="sources" rev="4.11"/>
<dependency org="junit" name="junit" type="jar" rev="4.11"/>
I handle the build of a Java product with dependencies with ant.
Here is my project.properties file:
project.name=foo
project.version=1.0.0
thirdpart.commons-cli.version=1.2
thirdpart.guava.version=16.0.1
This is loaded using the following in my build.xml ant script:
<property file="project.properties"/>
I would like to loop over all properties starting by "thirdpart." and retrieve each time the name between "thirdpart." & ".version" and the value of the property.
Idea behind is then to retrieve the correct jar file from a shared server. Those informations will help me to build up the correct URL to retrieve them, while allowing me to change my dependencies version easily.
How to proceed with ant ? (Thanks for your help).
Instead of building your own dependency manager I would suggest using Apache ivy.
Several advantages. Instead of building and populating a shared server, you could download from Maven Central. Standard Maven repository managers (nexus, artifactory, archiva) can be used to host repositories inside your firewall.
Examples:
Class not found with Ant, Ivy and JUnit - error in build.xml?
How to avoid copying dependencies with Ivy
Your dependencies
Here's an example ivy.xml file to retrieve your dependencies
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile" description="Required to compile application"/>
</configurations>
<dependencies>
<!-- compile dependencies -->
<dependency org="commons-cli" name="commons-cli" rev="1.2" conf="compile->default"/>
<dependency org="com.google.guava" name="guava" rev="17.0-rc2" conf="compile->default"/>
</dependencies>
</ivy-module>
Here is my ivy.xml:
<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="2.0">
...
<dependencies>
<dependency org="spring" name="richclient" rev="1.1.0"/>
</dependencies>
</ivy-module>
And ivy-settings.xml:
<property name="ivy.local.default.root" value="/home/---/dev/Java/_libraries/_ivy" override="false"/>
<property name="ivy.local.default.ivy.pattern" value="[organisation]/[module]/[revision]/[type]s/[artifact].[ext]" override="false"/>
<property name="ivy.local.default.artifact.pattern" value="[organisation]/[module]/[revision]/[type]s/[artifact].[ext]" override="false"/>
<resolvers>
<filesystem name="local">
<ivy pattern="${ivy.local.default.root}/${ivy.local.default.ivy.pattern}" />
<artifact pattern="${ivy.local.default.root}/${ivy.local.default.artifact.pattern}" />
</filesystem>
</resolvers>
Ivy try to find
/home/---/dev/Java/_libraries/_ivy/spring/richclient/1.1.0/jars/richclient.jar
And here is the problem. Library has 4 jar files.
How to include all jars in project from one dependency in ivy.xml?
Thx
I'm assuming you've just downloaded the jars locally? It won't work unless you also write an ivy.xml file for the downloaded files, listing the artifacts that are associated with the module (See publications section of the ivy.xml doco)
Why not avoid the hassle of maintaining the your own version of someone else's module by using the maven repository provided by Spring?
Add the following to your ivy-settings.xml file:
<resolvers>
<ibiblio name="spring-rcp" m2compatible="true" root="http://spring-rich-c.sourceforge.net/maven2repository"/>
</resolvers>
While Ivy can work using dependencies on individual JAR files, it works better if you define separate ivy.xml files for the dependencies themselves, which specifies the 4 separate JAR files. This ivy.xml defines what Ivy calls a module.
Your application's ivy.xml then expresses a dependency on that module, rather than on specific JAR files.
The Ivy website has a tutorial on modules, I highly recommend reading it
http://ant.apache.org/ivy/history/latest-milestone/tutorial/conf.html