Following is my ivy file :
<configurations>
<conf name="default" description="Default configuration"/>
<conf name="runtime" description="The configuration needed for runtime" extends="default"/>
<conf name="compile" description="The configuration needed to compile" extends="default"/>
<conf name="test" description="The configuration needed to run the tests" extends="compile"/>
<conf name="sources" description="Source files configuration"/>
</configurations>
<dependencies>
<dependency org="hibernate" name="hibernate" rev="3.5.3" conf="compile,runtime -> default"/>
<dependency org="org.apache.hive" name="hive-jdbc" rev="2.0.0" conf="compile,runtime -> default">
<artifact name="hive-jdbc" ext="jar"/>
<exclude module="mail" />
<exclude module="libthrift" />
</dependency>
</dependencies>
When connecting to hive I am getting the following error :
java.lang.IncompatibleClassChangeError: class org.apache.hive.service.cli.thrift.TCLIService$Client has interface org.apache.thrift.TServiceClient as super class
Can anyone please let me know what is the problem?
Related
My dependencies.xml contains a lot of packages required for my web application to run
Following are a few notable fragments
<configurations>
<conf name="test" visibility="public" extends="compile" />
<conf name="compile" visibility="public" extends="runtime" />
<conf name="runtime" visibility="public" />
<conf name="provided" visibility="public" />
<conf name="junit" visibility="public" extends="test" />
</configurations>
<publications>
<artifact name="${project.name}" type="jar" ext="jar" conf="compile" />
<artifact name="${project.name}" type="zip" ext="zip" conf="compile"/>
</publications>
<dependencies>
<dependency org="org.hibernate" name="hibernate-core" rev="5.1.3.Final" transitive="false" conf="runtime->*"/>
<dependency org="org.hibernate" name="hibernate-ehcache" rev="5.1.3.Final" transitive="false" conf="runtime->*"/>
<dependency org="org.hibernate.common" name="hibernate-commons-annotations" rev="5.0.1.Final" transitive="false" conf="runtime->*"/>
<dependency org="org.hibernate.javax.persistence" name="hibernate-jpa-2.1-api" rev="1.0.0.Final" transitive="false" conf="runtime->*"/>
<dependency org="org.javassist" name="javassist" rev="3.21.0-GA" transitive="false" conf="runtime->*"/>
<dependency org="org.jboss.logging" name="jboss-logging" rev="3.3.0.Final" transitive="false" conf="runtime->*"/>
<dependency org="javax.transaction" name="jta" rev="1.1" transitive="false" conf="runtime->*"/>
<dependency org="net.sf.ehcache" name="ehcache-core" rev="2.6.11" transitive="false" conf="runtime->*"/>
<dependency org="antlr" name="antlr" rev="2.7.7" transitive="false" conf="runtime->*"/>
<dependency org="org.antlr" name="antlr4-runtime" rev="4.5.2-1" transitive="false" conf="runtime->*"/>
</dependencies>
I have package JTA from javax.transaction that is required for the application to run under Tomcat and forbidden for the application to run under WebSphere.
I need to know how to make two different WAR files depending on the target platform. I don't exactly know how to use configurations, if that is the way.
Ant will do an ivy-retrieve for configuration runtime and build a WAR archive using the jars downloaded from Artifactory.
I could exclude thos jar(s) manually by doing a delete after Ivy has resolved the artifacts, but hey, we are cool developers and we like to do the things the cleaner way.
How would you suggest me to do an ivy-retrieve that targets Tomcat including JTA an another that targets Websphere excluding it?
build.xml : Building war files
The following fragment builds two war files:
../demo.war
../demo-websphere.war
The magic is that the tomcat retrieve task includes two configurations:
<ivy:retrieve pattern="${lib.dir}/tomcat/[artifact].[ext]" conf="runtime,tomcat_only"/>
<war destfile="${dist.dir}/demo.war" webxml="${resources.dir}/web.xml">
<fileset dir="${resources.dir}" excludes="web.xml"/>
<lib dir="${lib.dir}/tomcat"/>
</war>
<ivy:retrieve pattern="${lib.dir}/websphere/[artifact].[ext]" conf="runtime"/>
<war destfile="${dist.dir}/demo-websphere.war" webxml="${resources.dir}/web.xml">
<fileset dir="${resources.dir}" excludes="web.xml"/>
<lib dir="${lib.dir}/websphere"/>
</war>
build.xml : Publishing
The following answer contains more details on publishing multiple module artifacts to a Maven repository
how to publish 3rdparty artifacts with ivy and nexus
So we need a target to generate the POM file:
<target name="prepare" description="Generate POM">
<!-- Optional: Intermediate file containing resolved version numbers -->
<ivy:deliver deliverpattern="${build.dir}/ivy.xml" pubrevision="${publish.revision}" status="release"/>
<!-- Generate the Maven POM -->
<ivy:makepom ivyfile="${build.dir}/ivy.xml" pomfile="${build.dir}/demo.pom"/>
</target>
And a second target that publish the built files:
<target name="publish" depends="init,prepare" description="Upload to Nexus">
<ivy:publish resolver="nexus-deploy" pubrevision="${publish.revision}" overwrite="true" publishivy="false" >
<artifacts pattern="${build.dir}/[artifact](-[classifier]).[ext]"/>
</ivy:publish>
</target>
Take careful notice of the optional "classifier" attribute and note how the ivy file is structured next
ivy.xml
<ivy-module version="2.0" xmlns:e="http://ant.apache.org/ivy/extra">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="master"/>
<conf name="default" extends="master,runtime"/>
<conf name="compile"/>
<conf name="provided"/>
<conf name="runtime" extends="compile"/>
<conf name="test" extends="runtime"/>
<conf name="tomcat_only" description="A special configuration for special tomcat only dependencies"/>
</configurations>
<publications>
<artifact name="demo" type="war" conf="master"/>
<artifact name="demo" type="pom" conf="master"/>
<artifact name="demo" type="war" conf="master" e:classifier="websphere"/>
</publications>
<dependencies>
<!-- Compile dependencies -->
<dependency org="org.hibernate" name="hibernate-core" rev="5.1.3.Final" conf="compile->default"/>
<dependency org="org.api" name="slf4j-api" rev="1.7.22" conf="compile->default"/>
<!-- Runtime dependencies -->
<dependency org="org.slf4j" name="slf4j-log4j12" rev="1.7.22" conf="runtime->default"/>
<!-- Tomcat dependencies -->
<dependency org="javax.transaction" name="jta" rev="1.1" conf="tomcat_only->master"/>
</dependencies>
</ivy-module>
Few things going on here.
1)
Make configuration mappings work for you
config1->default # Remote artifact plus transitive dependencies
config2->master # Remote artifact only (Same as setting transitive=false)
2)
The "extends" attribute is a set operation, meaning a compile dependency is automatically included as a runtime configuration.
An example is the SLF4J libary. The slf4j-api jar is required when compile code, where as the slf4j-log4j12 jar contained bindings and dependencies on log4j, a runtime (and changeable) dependency.
3)
The "master" configuration in your module is special and by convention in Maven world corresponds to the files published by this module.
4)
The "classifier" attribute is an example of a extra attribute in ivy.
I wrote some Nutch plugin with using different extension points such as Protocol, Parser and etc. These plugins work perfectly inside eclipse. But in order to use them on hadoop cluster it should be built by using ANT. My problem is, I wrote some classes in some new packages inside core folder(src). These classes are shared cross different developed plugins. My problem is at build time of developed plugins, ANT can not find the mentioned shared classes, so I am unable to complete the build process successfully. For better understanding of my problem Here is build.xml of one of my plugins:
<project name="filter-news" default="jar-core">
<import file="../build-plugin.xml"/>
<!-- Build compilation dependencies -->
<target name="deps-jar">
<ant target="jar" inheritall="false" dir="../lib-xml"/>
</target>
<!-- Add compilation dependencies to classpath -->
<path id="plugin.deps">
<fileset dir="${nutch.root}/build">
<include name="**/lib-xml/*.jar" />
</fileset>
</path>
<!-- Deploy Unit test dependencies -->
<!-- Deploy Unit test dependencies -->
<!-- for junit test -->
</project>
ivy.xml:
<ivy-module version="1.0">
<info organisation="org.apache.nutch" module="${ant.project.name}">
<license name="Apache 2.0"/>
<ivyauthor name="Apache Nutch Team" url="http://nutch.apache.org"/>
<description>
Apache Nutch
</description>
</info>
<configurations>
<include file="../../..//ivy/ivy-configurations.xml"/>
</configurations>
<publications>
<!--get the artifact from our module name-->
<artifact conf="master"/>
</publications>
<dependencies>
<dependency org="mysql" name="mysql-connector-java" rev="5.1.31"/>
<dependency org="net.sourceforge.htmlcleaner" name="htmlcleaner" rev="2.2"/>
<dependency org="commons-jxpath" name="commons-jxpath" rev="1.3"/>
</dependencies>
</ivy-module>
plugin.xml:
<plugin id="filter-news" name="Apache Nutch XML/HTML Parser/Indexing Filter" version="1.4" provider-name="nutch.org">
<runtime>
<library name="filter-news.jar">
<export name="*"/>
</library>
<library name="ant-1.7.0.jar"/>
<library name="ant-launcher-1.7.0.jar"/>
<library name="jdom-1.1.jar"/>
<library name="commons-jxpath-1.3.jar"/>
<library name="htmlcleaner-2.2.jar"/>
<library name="mysql-connector-java-5.1.31.jar"/>
</runtime>
<requires>
<import plugin="nutch-extensionpoints"/>
</requires>
<extension id="org.apache.nutch.parse" name="Nutch XML/HTML Html parser filter" point="org.apache.nutch.parse.HtmlParseFilter">
<implementation id="com.ictcert.nutch.filter.news.NewsHtmlFilter" class="com.ictcert.nutch.filter.news.NewsHtmlFilter" />
</extension>
<extension id="org.apache.nutch.indexer" name="Nutch XML/HTML Indexing Filter" point="org.apache.nutch.indexer.IndexingFilter">
<implementation id="com.ictcert.nutch.filter.news.NewsIndexingFilter" class="com.ictcert.nutch.filter.news.NewsIndexingFilter"/>
</extension>
</plugin>
When I try to build this plugin ant can not find all of the class dependencies related to com.ictcert.nutch package which is located in core part of nutch (src). While for other classes located in org.apache.nutch I have not such problem. Would you please tell me what is wrong with my configuration that the default packages could be found by ANT but the new ones could not.
As far as I know from my experience the implementation id should have the same package structure as your extension point. Can you try that and see if it solves your problem.
From my build.xml:
<?xml version="1.0" encoding="utf-8" ?>
<project name="myapp" default="package-core" basedir=".."
xmlns:ivy="antlib:org.apache.ivy.ant"
xmlns:antcontrib="antlib:net.sf.antcontrib">
<target name="bootstrap">
<taskdef resource="org/apache/ivy/ant/antlib.xml"
uri="antlib:org.apache.ivy.ant" classpathref="build.path"/>
</target>
<target name="resolve" depends="bootstrap">
<ivy:settings url="${ivy.settings.home}"/>
<ivy:cleancache/>
<ivy:resolve file="build/${ivy.xml}"/>
<ivy:retrieve pattern="${gen.lib.main.dir}/[artifact]-[revision].[ext]" conf="main"/>
<ivy:report todir="${gen.staging.dir}" />
</target>
...omitted for brevity
<target name="publish" depends="compile">
<ivy:publish resolver="default-resolver" pubrevision="0.2.0" overwrite="true" update="true">
<artifacts pattern="${gen.dist.pub.dir}/[artifact].[ext]" />
</ivy:publish>
</target>
My ivy.xml:
<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="2.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd">
<info organisation="myapp" module="myapp"/>
<configurations>
<conf name="main" description="provides main dependencies for the client JAR"/>
</configurations>
<dependencies>
<!-- Main dependencies. -->
<dependency org="google" name="guava" rev="14.0" conf="main->*"/>
<dependency org="google" name="guice" rev="3.0" conf="main->*"/>
</dependencies>
</ivy-module>
My ivy-settings.xml:
<ivysettings>
<properties file="ivy-settings.properties"/>
<settings defaultResolver="default-resolver"/>
<latest-strategies>
<latest-lexico/>
</latest-strategies>
<credentials host="${ivy.repo.root}" realm="${ivy.std.repo.realm}"
username="${ivy.std.repo.username}" password="${ivy.std.repo.password}"/>
<resolvers>
<chain name="default-resolver" returnFirst="true">
<url name="std-repo">
<ivy pattern="${ivy.repo.root}/${ivy.module.pattern}"/>
<artifact pattern="${ivy.repo.root}/${ivy.artifact.pattern}"/>
</url>
</chain>
</resolvers>
<modules>
<module organisation="myapp" name="*" resolver="default-resolver"/>
</modules>
</ivysettings>
Question:
When I run the resolve target, I successfully pull down the Guava and Guice JARs (and all their dependencies). So I know I have this set up, at least partially correct.
But when I run the publish target, I get the following error:
[jar] Building jar: /home/myuser/sandbox/dsi/workbench/eclipse/workspace/myapp/gen/dist/myapp-server
publish:
[ivy:publish] :: loading settings :: url = jar:file:/home/myuser/sandbox/dsi/workbench/eclipse/4.2/eclipse/plugins/org.apache.ivy.eclipse.ant_2.3.0.final_20130110142753/ivy.jar!/org/apache/ivy/core/settings/ivysettings.xml
BUILD FAILED
/home/myuser/sandbox/dsi/workbench/eclipse/workspace/myapp/build/build-core.xml:289: no organisation provided for ivy publish task: It can either be set explicitely via the attribute 'organisation' or via 'ivy.organisation' property or a prior call to <resolve/>
Any ideas? Thanks in advance!
You have to call 'publish' and 'resolve' in the same Ant session. If you call them separately it won't work and you'll get this error message.
For instance, you could let the 'publish' target depend on the 'resolve' target:
<target name="publish" depends="resolve,compile">
...
</target>
Hope this helps,
Maarten
Not sure if I'm going about this the right way, but I have some artifacts that I'm trying to convert to maven using ivy ant tasks and push into my maven repo.
the component in question is mystuff.services.common.
First I make the pom...
<ivy:makepom ivyfile="${ivy.lib.dir}/ivy/cache/myorg/mystuff.services.common/ivy-mystuff.services.common.xml" pomfile="${ivy.lib.dir}/ivy/cache/myorg/mystuff.services.common/poms/mystuff.services.common.pom">
<mapping conf="default" scope="compile"/>
<mapping conf="runtime" scope="runtime"/>
</ivy:makepom>
Then a little hackery - I insert an artifact element in the ivy file using xml task. This works ok...
<xmltask source="${ivy.lib.dir}/ivy/cache/myorg/mystuff.services.common/ivy-${resolved.revision}.xml" dest="${ivy.lib.dir}/ivy/cache/myorg/mystuff.services.common/ivy-${resolved.revision}.xml">
<insert path="/ivy-module/publications" >
<![CDATA[
<artifact name="mystuff.services.common" type="pom"/>
]]>
</insert>
</xmltask>
Then I resolve/deliver/publish, as per various docs I've seen on how to do this.
<ivy:resolve file="${ivy.lib.dir}/ivy/cache/myorg/mystuff.services.common/ivy-${resolved.revision}.xml"/>
<!--<echoproperties/>-->
<ivy:deliver conf="*" delivertarget="recursive-deliver"/>
<ivy:publish resolver="myrepo-publish" publishivy="false" overwrite="true">
<artifacts pattern="lib/myorg/[module]/[type]s/[artifact].[ext]"/>
</ivy:publish>
And the error I get:
build.xml:235: impossible to publish artifacts for
myorg#mystuff.services.common;1.0.1: java.io.IOException: missing artifact
myorg#mystuff.services.common;1.0.1!mystuff.services.common.pom
If I leave out the pom from the artifacts in the ivy file, the other artifacts just publish fine.
What am I doing wrong?
This is what the ivy file looks like after inserting the pom entry for artifacts
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="../../ivy-doc.xsl"?>
<ivy-module version="1.0">
<info organisation="myorg" module="mystuff.services.common" revision="1.0.1" status="integration" publication="20130206204156"/>
<configurations>
<conf name="default"/>
<conf name="compile" extends="default"/>
</configurations>
<publications>
<artifact name="services.common" type="jar" conf="compile"/>
<artifact name="services.common~test" type="jar" conf="compile"/>
<artifact name="services.common" type="javadoc-zip" ext="zip" conf="compile"/>
<artifact name="services.common~test" type="javadoc-zip" ext="zip" conf="compile"/>
<artifact name="services.common" type="src-zip" ext="zip" conf="compile"/>
<artifact name="services.common~test" type="src-zip" ext="zip" conf="compile"/>
<artifact name="com.myorg.mystuffservices.common" type="osgi-module" ext="jar" conf="compile"/>
<artifact name="services.common" type="pom"/>
</publications>
<dependencies>
<dependency org="org.testng" name="testng" rev="5.11" conf="compile->compile-15"/>
</dependencies>
</ivy-module>
Your publish does not have an artifact pattern that finds the pom generated by your "makepom" task.
Either change the location or alternatively add an extra artifacts tag to your publish task:
<ivy:publish resolver="myrepo-publish" publishivy="false" overwrite="true">
<artifacts pattern="lib/myorg/[module]/[type]s/[artifact].[ext]"/>
<artifacts pattern="${ivy.lib.dir}/ivy/cache/myorg/mystuff.services.common/poms/mystuff.services.common.pom"/>
</ivy:publish>
I also don't understand why you're inserting a POM entry into you ivy file. Why don't you just list in your publications section?
For a detailed example see:
how to publish 3rdparty artifacts with ivy and nexus
I'm using Ivy on my project, with the Ivy Eclipse plugin.
It appears that certain jars which are downloaded and added to my project are the javadoc jars, not the jars with the actual code. Note - this doesn't happen with all jars.
For example, adding this to my ivy.xml file:
<dependency org="junit" name="junit" rev="4.8.2"/>
caused the javadocs for junit to be downloaded and added to my classpath:
This breaks compilation for my project, as none of the unit tests are working.
This was working fine until I added a reference to Spring, and everything broke. I've tried removing the reference, and deleting junit from my local cache to force ivy to fetch it again, but the problem persists.
Here's my total dependency block (with spring removed):
<dependencies>
<dependency org="org.hamcrest" name="hamcrest-library" rev="1.3.RC2"/>
<dependency org="junit" name="junit" rev="4.8.2"/>
<dependency org="org.mockito" name="mockito-core" rev="1.8.5"/>
<dependency org="javax.persistence" name="persistence-api" rev="1.0"/>
</dependencies>
Here's my ivysettings.xml for the project:
<ivysettings>
<caches artifactPattern="[organisation]/[module]/[revision]/[artifact].[ext]" />
<settings defaultResolver="local.ibiblio.jboss.java-net.springsource" checkUpToDate="true" />
<resolvers>
<chain name="local.ibiblio.jboss.java-net.springsource">
<filesystem name="libraries">
<artifact pattern="${basedir}/ivy-repo/[artifact]-[revision].[type]" />
</filesystem>
<ibiblio name="ibiblio" m2compatible="true" />
<ibiblio name="jboss" m2compatible="true"
root="https://repository.jboss.org/nexus/content/groups/public-jboss" />
<ibiblio name="java.net" m2compatible="true"
root="https://repository.jboss.org/nexus/content/repositories/java.net-m2/" />
<ibiblio name="java.net" m2compatible="true"
root="http://repository.codehaus.org/" />
<url name="com.springsource.repository.libraries.release">
<ivy pattern="http://repository.springsource.com/ivy/libraries/release/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
<artifact pattern="http://repository.springsource.com/ivy/libraries/release/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
</url>
<url name="com.springsource.repository.libraries.external">
<ivy pattern="http://repository.springsource.com/ivy/libraries/external/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
<artifact pattern="http://repository.springsource.com/ivy/libraries/external/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
</url>
<url name="com.springsource.repository.bundles.release">
<ivy pattern="http://repository.springsource.com/ivy/bundles/release/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
<artifact pattern="http://repository.springsource.com/ivy/bundles/release/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
</url>
<url name="com.springsource.repository.bundles.external">
<ivy pattern="http://repository.springsource.com/ivy/bundles/external/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
<artifact pattern="http://repository.springsource.com/ivy/bundles/external/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
</url>
</chain>
</resolvers>
</ivysettings>
Some open source modules include optional java doc jars. To remove them add a configuration mapping to each of your dependencies:
<dependency org="junit" name="junit" rev="4.8.2" conf="default"/>
The default configuration in ivy is equivalent to the the compile scope in a maven module. This is how the optional libraries can be automatically omitted. (Check their POMs).
A better approach is to declare your own configurations and the default mapping as follows:
<configurations defaultconfmapping="compile->default">
<conf name="compile" description="Required to compile code"/>
<conf name="test" description="Additional test dependencies" extends="compile" />
</configurations>
Then in your ivy file you only need to declare the non-standard configurations:
<dependencies>
<dependency org="org.hamcrest" name="hamcrest-library" rev="1.3.RC2" conf="test->default"/>
<dependency org="junit" name="junit" rev="4.8.2" conf="test->default"/>
<dependency org="org.mockito" name="mockito-core" rev="1.8.5" conf="test->default"/>
<dependency org="javax.persistence" name="persistence-api" rev="1.0"/>
</dependencies>
In this case we only want the 3 test libraries to appear on the test configuration.
Still confused? The magic of ivy configurations is when you use them to manage your build's class path
<target name='dependencies' description='Resolve project dependencies and set classpaths'>
<ivy:resolve/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="test.path" conf="test"/>
</target>
This is what Maven is doing when you declare a scope tag on a dependency, for example:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.2</version>
<scope>test</scope>
</dependency>
The scopes in Maven are fixed. In ivy you can have as many as you need.