i have a project that uses ivy to manage its dependencies. i'm implementing a feature to the project that requires me to include tools.jar. however, because tools.jar is platform dependent, i'm trying to use ivy to resolve to a local file for the artifact. i'm doing the following:
<dependency org="com.sun" names="tools" rev="1.6.0">
<artifact name="tools" type="jar" url="file:///${java.home}/../lib/tools.jar"/>
</dependency>
that should retrieve the file from the local ${java.home}/../lib/tools.jar. (note: java.home points to the JRE installation).
however, there are problems resolving the location. on my windows machine, it seems to think "c" is the protocol (c is coming from ${java.home}. and i'm sure that my url is defined correctly because "file:///C:/foo" is the correct way to specify a url to a file (3 slashes). the problem i see is that it strips off 2 slashes and tries "file:/C:..." instead of "file:///C:.." as i specify above. i also tried specifying the path to the file directly w/o the ${java.home}
i would like to keep this approach retrieving through ivy, but i can't get it to work. any ideas?
JAVA_HOME needs to point at the location of your JDK, not your JRE. Once you change this, ANT will stop complaining about a missing tools jar.
Looking at the path you provide above, I suspect that you already have a JDK installed....
Analysis
On my system the tools jar is located here:
$ find $JAVA_HOME -name tools.jar
/usr/lib/jvm/java-6-openjdk/lib/tools.jar
Oddly, and confusing, the Java JDK ships with a JRE inside
$ find $JAVA_HOME -name java
/usr/lib/jvm/java-6-openjdk/bin/java
/usr/lib/jvm/java-6-openjdk/jre/bin/java
I was able to get this to work using a dedicated resolver
ivysettings.xml
<resolvers>
<!-- your other resolvers here -->
<filesystem name="JDK" local="true">
<artifact pattern="${java.home}/lib/[artifact].[type]" />
<artifact pattern="${java.home}/../lib/[artifact].[type]" />
<!-- You can add more patterns to fit your needs for MacOSX etc -->
</filesystem>
</resolvers>
<modules>
<module organisation="com.sun" name="tools" resolver="JDK"/>
</modules>
ivy.xml
<dependency org="com.sun" name="tools"/>
Works for me...
Related
I have a tack in build.xml that downloads all dependencies to the cache:
<target name="init" depends="init-ivy">
...
<ivy:cachepath
inline="true"
module="jersey-container-servlet"
organisation="org.glassfish.jersey.containers"
pathid="jersey.classpath"
revision="2.23.2"/>
<ivy:cachepath
inline="true"
module="javax.json"
organisation="org.glassfish"
pathid="json.classpath"
revision="1.0.4"/>
...
</target>
The code compiles successfully and a war file is created. Now I need write a task that would deploy the app to tomcat. I need to copy all dependencies to the app's WEB-INF/lib. How is this done? Maybe there is a way to include the dependencies' JARs to the WAR file? I am new to Java development, please help.
The following answer outlines the comprehensive solution using an ivy file.
Ivy dependecy as provided
It answers a different question ("provided" dependencies) but one you will eventually face because not all the jars you use in your build will need to shipped with your application (because they already exist on tomcat).
Attempting to apply this answer to your question is not straightforward because you're resolving your dependencies in inline mode (No ivy file). Firstly I'd recommend combining your dependencies into a single path, rather than creating paths around each dependency:
<ivy:cachepath pathid="compile.classpath">
<dependency org="org.glassfish" name="javax.json" rev="1.0.4" />
<dependency org="org.glassfish.jersey.containers" name="jersey-container-servlet" rev="2.23.2" />
</ivy:cachepath>
Secondly (and to answer your question), it's the alternate ivy retrieve task that's used to place ivy files on the file system. It too can support an inline resolution as follows:
<ivy:retrieve pattern="${build.dir}/lib/[artifact].[ext]">
<dependency org="org.glassfish" name="javax.json" rev="1.0.4" />
<dependency org="org.glassfish.jersey.containers" name="jersey-container-servlet" rev="2.23.2" />
</ivy:retrieve>
<war destfile="${war.file}" webxml="${resources.dir}/web.xml">
<fileset dir="${resources.dir}" excludes="web.xml"/>
<lib dir="${build.dir}/lib"/>
</war>
So in conclusion, while this suggested answer will work, I would recommend investigating how configurations work in concert with an external ivy file to manage your dependencies. Configurations may appear challenging, but they're also very powerful.
Your other question is related. Using ivy's inline mode is convenient but not the most efficient way to use ivy. A single call to the resolve task can be used to determine all a project's dependencies and using configurations to partition these up into various classpath or filesets, etc.
how to get ivy:cachepath location without checking if dependencies downloaded
I add a dependency(let's name it as A) to ivy.xml which has a pom file in maven central. Ivy uses ibiblio for resolving the maven dependencies. The dependency(A) which is added to ivy.xml has a transitive dependency(B). So far so good till here. The dependency(C) of transitive dependency(B) can not be resolved by ivy.
I defined A in ivy.xml like this:
<dependency org="Z" name="A" rev="0.6-SNAPSHOT" conf="*->default"/>
In pom file of B, C is defined both in compile and test scopes like below:
<dependency>
<groupId>X</groupId>
<artifactId>C</artifactId>
</dependency>
<dependency>
<groupId>X</groupId>
<artifactId>C</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
When I look the xml file of B which is resolved by ivy in ivy's cache file(~/.ivy2/cache/X/C/ivy-0.98.8-hadoop2.xml), it looks like this:
<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)"/>
<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)">
<artifact name="C" type="test-jar" ext="jar" conf="" m:classifier="tests"/>
</dependency>
For this reason, ivy can not define C scopes correctly. For the record, I don't have permissions to modify the pom files as they are third party projects. How can I fix it ?
I reviewed the ivy usage of the nutch project and apologies but my conclusion is that it's overly complex for the following reasons:
"compile" and "test" targets are issuing separate calls to the resolve task
Each plugin is also calling an ivy resolve task
Complex logic for maintaining classpaths. Could be simplified using the cachepath task and ivy configurations.
Build plugins are not managed by ivy (Sonar, eclipse, rat)
I started to refactor the build, but had to stop when I realised that I didn't understand the relationship between the main nutch artifact and the plugins... (I discovered NUTCH-1515 the hard way... big time-waster The feed plugin has missing dependencies).
I also noticed issue NUTCH-1371 calling for the removal of ivy. This would be a tricky refactoring without significant change to the current codebase. I suspect it would have to be a multi-module build with each plugin listing its own dependencies.
In conclusion, this work does not answer your question, but thought I needed to at least document the result of a few hours analysis :-) In light of NUTCH-1371 I don't know if your project will tolerant major ivy refactoring?
Refactoring ivy
Here follows what I achieved so far:
Private "development" fork of the nutch project
Diff with trunk
Benefits:
Single ivy report showing all configurations (New ivy-resolve target)
New mechanism for installing ivy (New ivy-install target)
Classpaths are managed using ivy configurations (See use of ivy cachepath task and configurations in ivy file)
Eclipse, sonar and rat ANT tasks automatically installed using ivy (The Eclipse plugin is noteworthy as it uses a packager resolver to download and extract jar from a tar archive).
Impacts the following Nutch issues
NUTCH-1881 : This new approach removes resolve-test and resolve-default targets and manages the classpaths using ivy instead of the ${build.lib.dir}
NUTCH-1805 : Can easily setup a separate configuration for the job target with it's own dependencies.
NUTCH-1755 : I think this one is fixed by assigning a name to the the build.xml (see: diff)
I've been doing static analysis on Java projects, which usually boils down to running javac2 #a_list_of_all_the_java_files_in_the_project, where javac2 is my modified compiler. Except, finding the right libraries to make everything compile is difficult.
I'm working with a project now (incidentally, eclipse SDK 3.7.1) that has a file artifacts.xml in the root folder. This file looks useful. My understanding so far is that it tells eclipse which libraries to use when opening the folder as an eclipse project. If so, I'd like to download these libraries locally and reference them in my custom compilation command.
Can someone explain the purpose of artifacts.xml, and optionally offer feedback on my approach? Ultimately all I want is to be able to compile the project on the command line using a nonstandard compiler.
First few lines or artifacts.xml
<?xml version='1.0' encoding='UTF-8'?>
<?artifactRepository version='1.1.0'?>
<repository name='Bundle pool' type='org.eclipse.equinox.p2.artifact.repository.simpleRepository' version='1'>
<properties size='2'>
<property name='p2.system' value='true'/>
<property name='p2.timestamp' value='1315600353875'/>
</properties>
<mappings size='3'>
<rule filter='(& (classifier=osgi.bundle))' output='${repoUrl}/plugins/${id}_${version}.jar'/>
<rule filter='(& (classifier=binary))' output='${repoUrl}/binary/${id}_${version}'/>
<rule filter='(& (classifier=org.eclipse.update.feature))' output='${repoUrl}/features/${id}_${version}.jar'/>
</mappings>
<artifacts size='405'>
<artifact classifier='osgi.bundle' id='org.eclipse.ecf.provider.filetransfer.ssl' version='1.0.0.v20110531-2218'>
<properties size='1'>
<property name='download.size' value='8460'/>
</properties>
</artifact>
This is the file that the Eclipse 'p2' install system uses to describe a repository of installable artifacts. The file is sometimes compressed in to an artifacts.jar file.
Eclipse p2 is described here: http://wiki.eclipse.org/Equinox/p2
A maven artifact, in general, is a file that gets deployed to a maven repo.
artifacts.xml is the canonical way of listing everything that needs to be sent to said repository.
Check this previous post for more information:
What is a Maven artifact?
I have tried setting different properties and attributes (debug="true"), but it didn't work.
This is from our build.xml (just showing the parts relating to the build step):
<!-- Environment holen -->
<property environment="env" />
<!-- Target: all -->
<target name="all" depends="build, test, export">
</target>
<!-- Target: build -->
<target name="build">
<ant4eclipse:executeProjectSet workspaceDirectory="${env.WORKSPACE}" teamprojectset="${env.WORKSPACE}\${env.JOB_NAME}\projectSet.psf">
<ant4eclipse:forEachProject filter="(executeProjectSet.org.eclipse.jdt.core.javanature=*)">
<buildJdtProject workspaceDirectory="${env.WORKSPACE}" projectName="${executeProjectSet.project.name}" targetLevel="1.6" />
</ant4eclipse:forEachProject>
</ant4eclipse:executeProjectSet>
</target>
Detailed description:
An internal project consists of a large number of classes and some applications, all written in Java.Everything runs just fine when started from within Eclipse.
After each commit to our SVN repository, the project is built using ant4eclipse on our Hudson installation and if tests pass, a zip is automatically created and copied to a file server to be used by simply unpacking and starting the supplied startup batch script.
Now last week a colleague informed me that the version from the file server doesn't work for him. I checked and am able to reproduce the problem - loading data from a database doesn't work. No exception is shown in the log/console and I have no idea what goes wrong. Everything works when started from within eclipse (same vmargs, same JVM etc.).
When trying to connect the debugger, it seems like no debug info is present ("line numbers missing" etc.). So I now need to find out how to convince ant4eclipse to include debug infos.
In the meantime, I found out how to do this myself: I added a default compiler options file like this (attribute defaultCompilerOptionsFile):
<!-- Target: build -->
<target name="build">
<ant4eclipse:executeProjectSet workspaceDirectory="${env.WORKSPACE}" teamprojectset="${env.WORKSPACE}\${env.JOB_NAME}\projectSet.psf">
<ant4eclipse:forEachProject filter="(executeProjectSet.org.eclipse.jdt.core.javanature=*)">
<buildJdtProject
workspaceDirectory="${env.WORKSPACE}"
projectName="${executeProjectSet.project.name}"
targetLevel="1.6"
defaultCompilerOptionsFile="compilerOptions.prefs"/>
</ant4eclipse:forEachProject>
</ant4eclipse:executeProjectSet>
</target>
compiler options file is just a copy of .metadata\.plugins\org.eclipse.core.runtime\.settings\org.eclipse.jdt.core.prefs inside the workspace. Make sure to set the desired options inside the file:
org.eclipse.jdt.core.compiler.debug.lineNumber=generate
org.eclipse.jdt.core.compiler.debug.localVariable=generate
org.eclipse.jdt.core.compiler.debug.sourceFile=generate
I haven't tested if it works if you create a compiler options file that just contains the 3 lines above.
I can't find any thorough explanation of the Ivy dependency tag's conf attribute:
<dependency org="hibernate" name="hibernate" rev="3.1.3" conf="runtime, standalone -> runtime(*)"/>
See that conf attribute? I can't find any explanation (that I can understand) about the right hand side of the -> symbol. PLEASE keep in mind I don't know the first thing about Maven so please explain this attribute with that consideration.
Yes, I've already looked at this: http://ant.apache.org/ivy/history/latest-milestone/ivyfile/dependency.html
Thanks,
Dan
First of all, Ivy is not Maven ;)
Maven2 is a software project management and comprehension tool, whereas Ivy is only a dependency management tool.
Ivy heavily relies on a unique concept called configuration.
In Ivy, a module configuration is a way to use or to see the module.
For instance, you can have a test and runtime configuration in your module. But you can also have a MySQL and an Oracle configuration. Or an Hibernate and a JDBC configuration.
In each configuration, you can declare:
what artifacts (jar, war, ...) are required.
your dependencies on other modules, and describe which configuration of the dependency you need. This is called configuration mapping.
So the conf attribute does precisely that: Describes a configuration mapping for a dependency.
The mapped child element is your "right hand side of the -> symbol" and represents the name of the dependency configuration mapped. '*' wildcard can be used to designate all configurations of this module.
See more at "Simplest Explanation of Ivy Configuration" from Charlie Hubbard
The important part of that is Ivy downloads dependencies and organizes them.
An ivy-module (ie ivy.xml file) has two main parts:
What dependencies do you need?
How do you want them organized?
The first part is configured under the <dependencies> element.
The 2nd is controlled by the <configurations> element
When Ivy is downloading these dependencies it needs to know what scopes to use when pulling these transitive dependencies (are we pulling this for testing, runtime, compilation, etc?). We have to tell Ivy how to map our configurations to Maven scopes so it knows what to pull.
Maven2 on its side has something called the scope.
You can declare a dependency as being part of the test scope, or the buildtime scope.
Then depending on this scope you will get the dependency artifact (only one artifact per module in maven2) with its dependencies depending on their scope. Scopes are predefined in maven2 and you can't change that.
That means :
There are a lot of unnecessary dependencies downloaded for many libraries.
For example, Hibernate downloads a bunch of JBoss JARs and the Display Tag downloads all the various web framework JARs. I found myself excluding almost as many dependencies as I added.
The problem is that hibernate can be used with several cache implementations, several connection pool implementation, ... And this can't be managed with scopes, wheres Ivy configurations offers an elegant solution to this kind of problem.
For instance, in Ivy, assuming hibernate has an Ivy file like this one, then you can declare a dependency like that:
<dependency org="hibernate" name="hibernate" rev="2.1.8" conf="default->proxool,oscache"/>
to get hibernate with its proxool and oscache implementations, and like that:
<dependency org="hibernate" name="hibernate" rev="2.1.8" conf="default->dbcp,swarmcache"/>
to get hibernate with dbcp and swarmcache.
By mapping your default master configuration to "proxool,oscache" or to "dbcp,swarmcache", you specify what you need exactly from the module "hibernate".
You can find those "proxool,..." arguments by listing the Ivy configuration defined for each modules associate with the library. For instance:
<ivy-module version="2.0">
<info organisation="ssn-src" module="pc"/>
<configurations defaultconfmapping="default->default">
<conf name="default" />
<conf name="provided" description="they are provided by the env." />
<conf name="compile" extends="default,provided" />
<conf name="war" extends="default"/>
</configurations>
<dependencies>
Example:
let's suppose modA has two configurations, default and test.
As a practical matter, it's going to be highly unusual to want to leave out the conf attribute of the dependency element.
The ivy.xml for modA might have a dependency:
<dependency org="theteam" name="modB" rev="1.0" conf="default->*" />
You're starting from default, rather than from both default and test.
The above example makes modA's default depend on modB's conf1, conf2, and conf3.
Or you might want to say that modA's default only depends on modB's conf1:
<dependency org="theteam" name="modB" rev="1.0" conf="default->*conf1*" />
I've read these answers and quite frankly I don't find them very helpful. I think they could be improved so I wrote down how I use and understand configurations by showing a practical example:
http://wrongnotes.blogspot.com/2014/02/simplest-explanation-of-ivy.html
Unfortunately, you have to understand a little about maven, and its dependencies because Ivy is using Maven repositories to download those jar files. Therefore, Ivy has to understand Maven and it passes that back to you. But, I think I kept it real simple without going into too much detail about maven.
Thanks VonC!
It helped me alot further.
When it comes to options (configurations) tieTYT, you can find them in the ivy-[revision number].xml file in your Ivy repository under: organization name --> module name.
An example configurations element from the JUnit 4.6 revision downloaded from http://www.springsource.com/repository/app/.
<configurations>
<conf name="compile" visibility="public" description="Compile dependencies"/>
<conf name="optional" visibility="public" extends="compile" description="Optional dependencies"/>
<conf name="provided" visibility="public" description="Provided dependencies"/>
<conf name="runtime" visibility="public" extends="compile" description="Runtime dependencies"/>
</configurations>
In my project's ivy.xml file, I have a configuration compile-test. In the dependencies element I have the following dependency:
<dependency org="org.junit" name="com.springsource.org.junit"
rev="4.6.0" conf="compile-test->compile" />
As you can see, my compile-test configuration depends on the compile configuration in the JUnit's ivy.xml file.
It helped me once to understand things this way:
An ivy configuration is simply a name for some subset of the module's artifacts.
Dependencies between modules are specified in terms of configuration names.