I handle the build of a Java product with dependencies with ant.
Here is my project.properties file:
project.name=foo
project.version=1.0.0
thirdpart.commons-cli.version=1.2
thirdpart.guava.version=16.0.1
This is loaded using the following in my build.xml ant script:
<property file="project.properties"/>
I would like to loop over all properties starting by "thirdpart." and retrieve each time the name between "thirdpart." & ".version" and the value of the property.
Idea behind is then to retrieve the correct jar file from a shared server. Those informations will help me to build up the correct URL to retrieve them, while allowing me to change my dependencies version easily.
How to proceed with ant ? (Thanks for your help).
Instead of building your own dependency manager I would suggest using Apache ivy.
Several advantages. Instead of building and populating a shared server, you could download from Maven Central. Standard Maven repository managers (nexus, artifactory, archiva) can be used to host repositories inside your firewall.
Examples:
Class not found with Ant, Ivy and JUnit - error in build.xml?
How to avoid copying dependencies with Ivy
Your dependencies
Here's an example ivy.xml file to retrieve your dependencies
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile" description="Required to compile application"/>
</configurations>
<dependencies>
<!-- compile dependencies -->
<dependency org="commons-cli" name="commons-cli" rev="1.2" conf="compile->default"/>
<dependency org="com.google.guava" name="guava" rev="17.0-rc2" conf="compile->default"/>
</dependencies>
</ivy-module>
Related
I have a project with both maven depedencies and a dependency on an company-internal jar file stored in our company's ivy repository. It was published with an ivy ant task. Let's call this file myco.jar. there is a source file called myco-sources.jar, and it is in the same location as myco.jar.
When I build with gradle, via Eclipse, the maven dependencies automatically download the source jar file and attach it to the main jar in the generated .classpath file. But for myco.jar, it does not automatically attach myco-sources.jar to it. Instead, myco-sources.jar is downloaded as if it was a separate dependency. So even though the source code is technically part of the classpath, I still can't step through the source code when debugging!
Here's a snippet of my dependencies:
dependencies {
implementation "com.myco:modname:${versionNo}"
implementation "com.fasterxml.jackson.datatype:jackson-datatype-jsr310:${jacksonVersion}"
...
}
How do I 'tell' gradle that the sources file is not a separate dependency but is the source file for the myco.jar file?
This is how the ivy.xml file looks like that is used to publish the myco.jar and myco-sources.jar files:
<publications>
<artifact name="myco" type="jar" ext="jar"/>
<artifact name="myco" e:classifier="sources" type="source" ext="jar"/>
<dependencies>
<dependency ... />
...
</dependencies>
and this is the ant task CI runs to publish to our artifactory repository:
<target name="publish-task" depends="xxx" description="xxx">
<ivy:resolve file="ivy.xml"/>
<ivy:publish pubrevision="${version}" status="integration" resolver="xxx" overwrite="true">
<artifacts pattern="${path}/[artifact]-[revision](-[classifier]).[ext]"/>
</ivy:publish>
<delete file="ivy-${version}.xml"/>
</target>
I am trying to use the ivy:publish task to publish my jars using the public ibiblio resolver which I understand corresponds to the Maven central repository.
It seems to sort of work, but I am eventually getting an HTTP 405: Method Not Allowed.
The relevant sections from my ivysettings.xml, ivy.xml and build.xml files are:
ivysettings.xml
This is actually the entire file:
<ivysettings>
<settings defaultResolver="public"/>
<resolvers>
<ibiblio name="public" m2compatible="true" />
</resolvers>
</ivysettings>
ivy.xml
...
<publications>
<artifact type="pom" ext="pom" conf="default"/>
<artifact type="jar" ext="jar" conf="default"/>
</publications>
...
build.xml
...
<target name="gen-pom" depends="ivy-resolve">
<ivy:makepom ivyfile="ivy.xml" pomfile="${dist.dir}/${ant.project.name}.pom">
<mapping conf="default" scope="compile"/>
</ivy:makepom>
</target>
<target name="ivy-publish" depends="jar, gen-pom" description="publish jar/source to maven repo">
<ivy:publish resolver="public" forcedeliver="true" overwrite="true" publishivy="false">
<artifacts pattern="${dist.dir}/[artifact].[ext]" />
</ivy:publish>
</target>
...
When I invoke ant ivy-publish I get:
impossible to publish artifacts for mant-tasks#mant-tasks;1.0: java.io.IOException: PUT operation to URL https://repo1.maven.org/maven2/mant-tasks/mant-tasks/1.0/mant-tasks-1.0.pom failed with status code 405: Method Not Allowed
I guess this makes sense otherwise anybody would be able to pollute the namespace. So my questions are:
is this due to my lack of credentials? I.e. would I need to add a
credentials element in my ivysettings.xml as shown in this
answer?
can I obtain credentials for ibiblio for my
insignificant module and if not what other freely available public
repositories are available to a Java developer who doesn't want to
build and maintain their own maven repository? I need to be able to pull-in my module (as an IVY dependency of other projects) from a globally available repository.
Maven central is operated by Sonatype, the creators of Maven:
http://central.sonatype.org/
JFrog operate a competing service called bintray:
https://bintray.com/
I suggest reading the documentation and obtain a credential that will allow to publish your module's file(s)
I have tried to find the right org=".. address for JaCoCo so that I can simply add the address to my ivy.xml file and it will take care of the dependency.
But I have trouble finding the right link and was wondering if anyone knew what it was?
I am going to use Ant to build my project, so I'd have to put JaCoCo to use in there.
None of the below are correct:
<!-- JaCoCo -->
<dependency org="org.jacoco.agent" name="jacoco-agent" rev="0.6.2.2" />
<dependency org="org.jacoco.core" name="jacoco-core" rev="0.6.2.2" />
<dependency org="org.jacoco.report" name="jacoco-report" rev="0.6.2.2" />
You'll only need a single dependency declaration:
<dependency org="org.jacoco" name="org.jacoco.ant" rev="0.7.2.201409121644" />
The "core", "agent" and "report" jars get pulled in as dependencies.
Jacoco ANT task jar in Maven Central
The following is an build example that uses ivy to manage Jacoco and Sonar plugins. It's worth noting how it uses a "build" configuration mapping to create a classpath for just the ANT task jars.
integrating JaCoCo in sonar using Ant
I use following dependencies:
<dependency org="org.jacoco" name="org.jacoco.ant" rev="0.6.3.201306030806"/>
<dependency org="org.jacoco" name="org.jacoco.core" rev="0.6.3.201306030806"/>
<dependency org="org.jacoco" name="org.jacoco.agent" rev="0.6.3.201306030806"/>
<dependency org="org.jacoco" name="org.jacoco.report" rev="0.6.3.201306030806"/>
Relatively new to Ivy here.
I have a growing list of projects that all publish to the same repository under the same organization. And I'm trying to find out how, if at all possible, I can declare dependencies from another project to artifacts created by all those projects.
My ivisettings.xml already has a resolver that points to that repository location. All artifacts in that repo are from the same organization. I trying to determine if there's a way to easily include artifacts in that location as dependencies from another project. I was hoping that I could do this using a wildcard for its names, like:
<dependencies>
<dependency org="my.org.name" name="**" rev="${current.iteration}" conf="master" />
</dependencies>
But of course the above doesn't work. So, any pointer/comment/pointer suggestion would be welcomed. I can't be the first guy to want to do this, so there's got to be a way to do this and maybe I'm just not using the right keywords in my searches. Thanks.
Publish an extra ivy file that lists all the other projects as a list of dependencies.
<ivy-module version="2.0">
<info organisation="my.org.name" module="projects" revision="1.0.0" status="release" publication="20130215110241"/>
<dependencies>
<dependency org="my.org.name" name="projectA" rev="1.0.0"/>
<dependency org="my.org.name" name="projectB" rev="1.0.0"/>
<dependency org="my.org.name" name="projectC" rev="1.0.0"/>
..
..
</dependencies>
</ivy-module>
Creating a single dependency against this special ivy module will then pull in the other project artifacts as transitive dependencies.
<dependency org="my.org.name" name="projects" rev="1.0.0"/>
Enhancement
You could go one further and create different configurations to categorize your projects' artifacts:
<ivy-module version="2.0">
<info organisation="my.org.name" module="projects" revision="1.0.0" status="release" publication="20130215110241"/>
<configurations>
<conf name="web-apps" description="Projects which provide functionality for webapps"/>
<conf name="standalone" description="Projects which provide functionality for stand-alone java apps"/>
</configurations>
<dependencies>
<dependency org="my.org.name" name="projectA" rev="1.0.0" conf="webapps->default"/>
<dependency org="my.org.name" name="projectB" rev="1.0.0" conf="standalone->default"/>
<dependency org="my.org.name" name="projectC" rev="1.0.0" conf="webapps->default;standalone->default"/>
</dependencies>
</ivy-module>
Configurations are an extremely useful ivy feature. Client builds can use mappings to pull in artifacts associated with either webapp or standalone development. This feature is similar to "scopes" in Maven but much more powerful.
Update: Dynamic revisions
When publishing the "projects" module, you might want to use dynamic revisions to simplify maintenance.
<ivy-module version="2.0">
<info organisation="my.org.name" module="projects"/>
<dependencies>
<dependency org="my.org.name" name="projectA" rev="latest.release"/>
<dependency org="my.org.name" name="projectB" rev="latest.release"/>
<dependency org="my.org.name" name="projectC" rev="latest.release"/>
..
..
</dependencies>
</ivy-module>
The publish task will generate and push an ivy file resolved with the latest version of each project.
You will still need to list each project, but this is in fact a good thing. It means pulling in older versions of the "projects" module will result in same number of revisions at a point in time.
Generate ivy file
Finally to completely automate this process, using a variable number projects, perhaps you could generate the "projects" ivy file?
The following is a groovy snippet that you could use:
<groovy>
import groovy.xml.MarkupBuilder
new File("build/ivy.xml").withWriter { writer ->
def xml = new MarkupBuilder(writer)
xml."ivy-module"(version:"2.0") {
info(organisation:"my.org.name", module:"projects")
dependencies() {
new File("/path/to/projects/directory").listFiles().each { dir ->
dependency(org:"my.org.name", name:dir.name, rev:"latest.release")
}
}
}
}
</groovy>
I currently have ANT_HOME located at /home/<myuser>/ant/1.8.4/ant-1.8.4.
I just downloaded the Apache Ivy tarball that includes its dependencies. I extracted it to /home/<myuser>/ivy/2.3.0-rc1/ivy-2.3.0-rc1.
I then copied /home/<myuser>/ivy/2.3.0-rc1/ivy-2.3.0-rc1/lib/*.jar to ANT_HOME/lib. If my understanding of how Ant works with plugins/extensions is correct, then Ant should now be able to access all of Ivy's tasks at runtime.
My next question is, how do I define Ivy tasks inside my Ant buildfile? Say I want to use ivy-retrieve, ivy-resolve and ivy-publish tasks. What are all the configurations I need to do (in the XML) to get these tasks working when I run my Ant build from the command-line (I will not be building through the Ant-Eclipse plugin). Thanks in advance!
First, you have to define a <taskdef> to point to the Ivy tasks.
<property environment="env"/>
<property name="ivy.home" value="${env_IVY_HOME}"/>
<taskdef resource="org/apache/ivy/ant/antlib.xml">
<classpath>
<fileset dir="${ivy.home}">
<include name="*.jar"/>
</fileset>
</classpath>
</taskdef>
That will give you access to the Ivy tasks. You'd use these tasks like this:
<cachepath pathid="main.classpath" conf="compile"/>
The problem is that your Ivy tasks names might clash with other Ant tasks. For example, there's an Ivy task <report>. To solve this, you can create an Ivy namespace. To do that, you put a reference in your namespace in the <project> entity like this:
<project name="my.proj" default="package" basedir="."
xmlns:ivy="antlib:org.apache.ivy.ant"/>
Now, when you define the Ivy tasks, you can use that antlib:org.apache.ivy.ant reference to your ivy namespace. Same taskdef as before, but with a uri field:
<property environment="env"/>
<property name="ivy.home" value="${env_IVY_HOME}"/>
<taskdef resource="org/apache/ivy/ant/antlib.xml"
uri="antlib:org.apache.ivy.ant">
<classpath>
<fileset dir="${ivy.home}">
<include name="*.jar"/>
</fileset>
</classpath>
</taskdef>
By the way, there's nothing special about that uri. I could have done this:
<project name="my.proj" default="package" basename="."
xmlns:ivy="pastrami:with.mustard">
[...]
<taskdef resource="org/apache/ivy/ant/antlib.xml"
uri="pastrami:with.mustard">
<classpath>
<fileset dir="${ivy.home}">
<include name="*.jar"/>
</fileset>
</classpath>
</taskdef>
The point is now you can prefix your task names with ivy:. Instead of this:
<cachepath pathid="main.classpath" conf="compile"/>
You can now do this:
<ivy:cachepath pathid="main.classpath" conf="compile"/>
And that's how you gain access to your Ivy Ant tasks.
Now, you have access to your Ivy Ant tasks, you need to define an ivysettings.xml file and use the <ivy:settings/> task to point there:
<ivy:settings file="${ivy.home}/ivysettings.xml"/>
There is a default ivysettings.xml file embedded in Ivy that will point you to the world wide Maven repository system. If you don't have a company wide Maven repository, then you can use the default ivysettings.xml file:
<ivy:settings/>
That's pretty simple.
Once you've done that, you need to read in and resolve your ivy.xml file which usually sits in the root of your project in the same directory as your build.xml file.
Basically, your ivy.xml file contains references to the third party jars you want to bring into your project. For example:
<dependencies>
<dependency org="log4j" name="log4j" rev="1.2.17" conf="compile->default"/>
<dependency org="junit" name="junit" rev="4.10" conf="test->default"/>
</dependencies>
What this is saying is that I need the log4j.jar (revision 1.2.17) for compilation (and for compiling tests too) and I need junit.jar (revision.4.10) for compilation of my test code.
The compile->default is a mapping of my compile configuration to Maven's default configuration (which says I just want the Jar and any other jars that it might depend upon.
Where's does my compile configuration come from? I define it in my ivy.xml. There are ten standard configurations. This also goes into your ivy.xml file:
<configurations>
<conf name="default" visibility="public" description="runtime dependencies and master artifact can be used with this conf" extends="runtime,master"/>
<conf name="master" visibility="public" description="contains only the artifact published by this module itself, with no transitive dependencies"/>
<conf name="compile" visibility="public" description="this is the default scope, used if none is specified. Compile dependencies are available in all classpaths."/>
<conf name="provided" visibility="public" description="this is much like compile, but indicates you expect the JDK or a container to provide it. It is only available on the compilation classpath, and is not transitive."/>
<conf name="runtime" visibility="public" description="this scope indicates that the dependency is not required for compilation, but is for execution. It is in the runtime and test classpaths, but not the compile classpath." extends="compile"/>
<conf name="test" visibility="private" description="this scope indicates that the dependency is not required for normal use of the application, and is only available for the test compilation and execution phases." extends="runtime"/>
<conf name="system" visibility="public" description="this scope is similar to provided except that you have to provide the JAR which contains it explicitly. The artifact is always available and is not looked up in a repository."/>
<conf name="sources" visibility="public" description="this configuration contains the source artifact of this module, if any."/>
<conf name="javadoc" visibility="public" description="this configuration contains the javadoc artifact of this module, if any."/>
<conf name="optional" visibility="public" description="contains all optional dependencies"/>
</configurations>
You can use any configuration name you want, but these map to the default Maven configurations and are widely used.
Once you have your ivy.xml file defined, you can use <ivy.resolve> to resolve your dependencies:
<ivy:resolve/>
So, we have the following:
How to use <taskdef> in your build.xml to incorporate the Ivy Ant tasks into your build.
How to use the Ivy Ant task <ivy:settings> to configure Ivy.
How to use <ivy:resolve/> to read in your ivy.xml file and resolve your third party jar dependencies.
Now, you probably want to actually use those jar files. There are three ways to do this:
<ivy:cachepath pathid="main.classpath" conf="compile"/>
The <ivy:cachepath/> task will create a classpath (in this case called main.classpath) that points to the jars you have in your ivy.xml file's compile configuration. This is used most of the time.
If you need a fileset, you can use this:
<ivy:cachefileset setid="compile.fileset" conf="compile"/>
In this case, it will create a fileset with a refid of compile.fileset.
Sometimes you have to bring the jars into your project. For example, if you create a war or ear file, you want to enclose your jars. In that case, you can use this:
<property name="lib.dir" value="${target.dir}/lib"/>
<ivy:retrieve pattern="${lib.dir}/[artifact].[ext]"
conf="runtime"/>
That will fetch your jars into the ${lib.dir} directory, so you can include them in wars or ears.
Sorry for the long answer, but there are a lot of steps to cover. I highly recommend Manning's book Ant in Action which has a whole chapter on Ivy.
David gave a very fine answer, but I'd like to point out that the taskdef is not required.
Provided the ivy.jar is in the expected location the namespace declaration at the top of the ANT file is enough:
<project ..... xmlns:ivy="antlib:org.apache.ivy.ant">
For more detail I'd recommend reading about how ANT libs work.
The following answer provides some more "setting up ivy" advice:
Ivy fails to resolve a dependency, unable to find cause