I'll present a simplified version of our problem, but know that we have way more than 2 projects, and so on.
So, at our job, we have 2 projects -
A
B
where B depends on A through ivy.
Recently we added configurations to the mix, mainly default and test.
The problem we ran into is that in A we have test utilities, which B needs in order to run it's own tests, so we need B to get these utilities from A.
We've thought of 2 ways to solve this:
make the test configuration public
create a new conf - test-utils - that will be public, and A will publish it's own tests under that configuration
The problem is that both solutions seem somewhat forced, and I wanted to get an idea how people do this world-wide.
Any ideas?
Note:
This answer assumes you're using a Maven repository manager. (Something
like Nexus or Artifactory).
I suggest that you upload an additional "test-utils" artifact when publishing Project A. This means the primary Maven artifact would be:
<dependency>
<groupId>com.mycompany</groupId>
<artifactId>ProjectA</artifactId>
<version>1.0.1</version>
</dependency>
and the second artifact is:
<dependency>
<groupId>com.mycompany</groupId>
<artifactId>ProjectA</artifactId>
<version>1.0.1</version>
<classifier>test-utils</classifier>
</dependency>
For more details on how ivy publishes and retrieves additional artifacts from a Maven repository I would suggest reading the following:
how to publish 3rdparty artifacts with ivy and nexus
ivy:install from maven with classifiers
Related
It's kind of hard to explain my problem with words, so I took a pic, which shows exactly what my problem is:
As you can see, I have 3 relevant modules, a global, genui and web.
genui depends on global and web depends on genui.
Directly, web doesn't depend on global, but through genui it obviously does.
As you can see, if I take a look at the hibernate version of web->genui->global, it is 5.2.14, but if I look at it from genui->global, it's 5.3.0.
There is not, and there have never been any versions of either of these modules other than 1.0-SNAPSHOT.
I tried cleaning, deleting the files from .m2/repository, and even tried purging the local repo completely, nothing worked. I have no idea where maven gets the 5.2.14 number, I don't remember ever putting that in. The version is received from a property in parent module.
I figured out that the problem is caused by the pom.xml of web module, more specifically this part:
<dependencyManagement>
...
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${spring-boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
...
</dependencyManagement>
You probably need to exclude that repeated dependency from one of the spring ones... verify all your dependencies and look which are the ones that include hibernate. Check that you also have different versions for servlet-api.
I add a dependency(let's name it as A) to ivy.xml which has a pom file in maven central. Ivy uses ibiblio for resolving the maven dependencies. The dependency(A) which is added to ivy.xml has a transitive dependency(B). So far so good till here. The dependency(C) of transitive dependency(B) can not be resolved by ivy.
I defined A in ivy.xml like this:
<dependency org="Z" name="A" rev="0.6-SNAPSHOT" conf="*->default"/>
In pom file of B, C is defined both in compile and test scopes like below:
<dependency>
<groupId>X</groupId>
<artifactId>C</artifactId>
</dependency>
<dependency>
<groupId>X</groupId>
<artifactId>C</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
When I look the xml file of B which is resolved by ivy in ivy's cache file(~/.ivy2/cache/X/C/ivy-0.98.8-hadoop2.xml), it looks like this:
<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)"/>
<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)">
<artifact name="C" type="test-jar" ext="jar" conf="" m:classifier="tests"/>
</dependency>
For this reason, ivy can not define C scopes correctly. For the record, I don't have permissions to modify the pom files as they are third party projects. How can I fix it ?
I reviewed the ivy usage of the nutch project and apologies but my conclusion is that it's overly complex for the following reasons:
"compile" and "test" targets are issuing separate calls to the resolve task
Each plugin is also calling an ivy resolve task
Complex logic for maintaining classpaths. Could be simplified using the cachepath task and ivy configurations.
Build plugins are not managed by ivy (Sonar, eclipse, rat)
I started to refactor the build, but had to stop when I realised that I didn't understand the relationship between the main nutch artifact and the plugins... (I discovered NUTCH-1515 the hard way... big time-waster The feed plugin has missing dependencies).
I also noticed issue NUTCH-1371 calling for the removal of ivy. This would be a tricky refactoring without significant change to the current codebase. I suspect it would have to be a multi-module build with each plugin listing its own dependencies.
In conclusion, this work does not answer your question, but thought I needed to at least document the result of a few hours analysis :-) In light of NUTCH-1371 I don't know if your project will tolerant major ivy refactoring?
Refactoring ivy
Here follows what I achieved so far:
Private "development" fork of the nutch project
Diff with trunk
Benefits:
Single ivy report showing all configurations (New ivy-resolve target)
New mechanism for installing ivy (New ivy-install target)
Classpaths are managed using ivy configurations (See use of ivy cachepath task and configurations in ivy file)
Eclipse, sonar and rat ANT tasks automatically installed using ivy (The Eclipse plugin is noteworthy as it uses a packager resolver to download and extract jar from a tar archive).
Impacts the following Nutch issues
NUTCH-1881 : This new approach removes resolve-test and resolve-default targets and manages the classpaths using ivy instead of the ${build.lib.dir}
NUTCH-1805 : Can easily setup a separate configuration for the job target with it's own dependencies.
NUTCH-1755 : I think this one is fixed by assigning a name to the the build.xml (see: diff)
I have a new library, called Codelet. It depends on two other libraries I've created, called Template Featherweight and XBN-Java. But the primary library--the one I'm trying to get other people to use!--is Codelet. The other two are secondary. Both are required, but they are only directly used when advanced features are needed.
I build all aspects of all three of these projects with Ant. I use Maven for one reason only: To sign the jars
codelet-0.1.0.jar
codelet-0.1.0-sources.jar
codelet-0.1.0-javadoc.jar
and push them to Maven Central. After shedding significant blood, sweat, tears, and soul in the past week, as evidenced by these four questions
How to use Maven to only sign three jars and push them to Maven Central?
Followup questions: Using Maven to only sign and deploy jars to Maven Central. Build and compilation is done entirely with Ant
Followup part 2 -- Using Maven to only sign and deploy jars to Maven Central. Build and compilation is done entirely with Ant
Why am I getting a "401 Unauthorized" error in Maven?
I have finally gotten the jars to upload to Maven Central via mvn deploy, at least once anyway, so that's big progress.
I'm realizing, however, that in order for other people to actually use Codelet, they need the entire dependency tree to be mapped out in Codelet's POM. I believe this is correct.
I am concerned that mapping this three-project dependency tree, in their three POMs, will essentially require that I duplicate much of my Ant build process in Maven. I am hoping with all hope that I don't need to do this, as Ant works well for me, and Maven and I do not get along.
Here are the dependency trees:
Dependencies for compilation of core library classes and example code, only
All items are listed by Maven groupId / artifactId / version.
XBN-Java 0.1.3 depends on
org.apache.commons / commons-collections4 / 4.0
org.apache.commons / commons-io / 2.4
org.apache.commons / commons-lang3 / 3.3.2
com.google.guava / guava / 16.0
Template Featherweight 0.1.0 depends on
com.github.aliteralmind / xbnjava / 0.1.3 (and its dependencies)
Codelet 0.1.0 depends on
com.github.aliteralmind / templatefeather / 0.1.0 (and its dependencies)
${java.home}/../lib/tools.jar (This is considered "provided", since it's part of the JDK and not on Maven Central)
Dependencies for compilation and execution of unit tests, only
These are in addition to those needed for core-compilation.
For all projects: junit / junit / 4.11 (and its dependency: hamcrest core)
For compilation of "Codelets" (which are used only by javadoc.exe), and execution of javadoc.exe
These are in addition to those needed for core-compilation.
For all projects: com.github.aliteralmind / codelet / 0.1.0 (and all its "core-compilation" dependencies)
(Some background: Codelet automates the insertion of example code into JavaDoc, using inline taglets like
{#.codelet.and.out com.github.mylibrary.examples.AGoodExample}
These optional "Codelet classes", called "customizers", are compiled before running javadoc.exe. They are used to customize how example code is displayed. Once compiled, Codelet is executed automatically, as are all inline taglets, via javadoc.exe.)
For core-compilation it's pretty much linear:
XBN-Java is the root
Template Feather depends on XBN-Java, and
Codelet depends on Template Feather
But for "javadoc", all three projects depend on Codelet. So even XBN-Java depends on Codelet...which depends on Template Feather...which depends on XBN-Java.
The POMs in all three projects are working, although this is only as far as it concerns signing and pushing the jars to Maven Central.
After reading Maven's dependency Mechanism documentation, it seems that all three projects could have the same flat dependency tree
<dependencies>
<dependency>
<groupId>com.github.aliteralmind</groupId>
<artifactId>templatefeather</artifactId>
<version>0.1.0</version>
</dependency>
<dependency>
<groupId>com.github.aliteralmind</groupId>
<artifactId>codelet</artifactId>
<version>0.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>4.0</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.3.2</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>16.0</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
</dependencies>
But the right way is to have a parent project, which is inherited by the other two. It seems that XBN-Java should be the parent, but given the recursive nature of the dependencies, I'm not sure.
I am not getting the difference between dependencies and dependencyManagement (why some dependencies blocks can go right into the project proper, and others are sub-blocks in dependencyManagement...although it seems related to parent-child), and I also don't understand how "javadoc" fits into the "scope" attribute. While compile and test are explicitely listed, the word "doc" doesn't even exist on the page.
I would appreciate some advice. Thank you.
If your code needs something, but not for compile-time (just for run-time) then declare the dependency, but add a runtime scope.
Now if you want to provide a "starter parent" pom.xml, then release a "starter parent" pom.xml but don't use it in your actual build chain.
I have a multi-module project built with maven. I need to run the project's integration tests daily. It is not possible to do this during the standard maven build cycle, because on runtime the integration tests defined within the modules have circular dependencies, which are illegal for me to declare on their poms.
Instead, I have created a separate project named Global that lists all modules jars and test-jars as its dependencies. Global has the same parent as all the modules. The idea is that using maven-ant-tasks I will be able to get a classpath of all modules jars and test-jars and go on from there. Global's pom.xml dependency section is as follows:
<dependency>
<groupId>mygroup</groupId>
<artifactId>A</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>mygroup</groupId>
<artifactId>A</artifactId>
<version>${project.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>mygroup</groupId>
<artifactId>B</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>mygroup</groupId>
<artifactId>B</artifactId>
<version>${project.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
...etc
The problem is that I cannot seem to get a classpath that contains all jars and test-jars declared on Global's pom.xml (and their runtime dependencies) using the ant tasks available. I have tried (among other things):
<dependencies pathId="cp1" type="jar" usescope="runtime">
<pom file="${basedir}/pom.xml">
<profile id="DEV" />
</pom>
</dependencies>
[1] This one fetches all runtime dependencies. Nothing wrong with that.
<dependencies pathId="cp2">
<dependency groupId="mygroup" artifactId="Global" version="myVersion" scope="test" type="test-jar"/>
</dependencies>
[2] This one fetches all runtime dependencies along with Global-myversion-tests.jar, but no other test-jar.
<dependencies pathId="cp3" type="test-jar" usescope="test">
<pom file="${basedir}/pom.xml">
<profile id="DEV" />
</pom>
</dependencies>
[3] This one fetches nothing.
Obviously, declaring something like [2] once for each module will do the trick, but I am looking to create a setup that will not need to edit a gazillion files each time a new module is added or removed. BTW I am using maven-ant-task-2.1.3.
Thanks for any input.
---Edits for #yannisf accepted answer---
You should not ever have cyclic dependencies
I assume you mean for maven builds. Having cyclic dependencies on runtime is pretty common, for example:
Module A declares interface: UploadToDocumentManagementSystem
Module B implements it in : UploadToCoolDms (that way in the future, when the DMS system changes to CoolerDms module B can be replaced by a new implementation with no side-effects to the rest of the app).
Module B depends on A compile time (and, by definition, runtime as well)
Module A depends on B on runtime
Maven does not allow to declare this. The reason, that I can sympathize with, is that maven needs to complete build cycles (including tests) of multi-module projects in a specific order. Thing is, it is not really necessary to declare it if you get rid of any runtime dependecy to B for the tests of A (which is good practice and should happen anyway).
You should do things the maven way instead of resorting to ant-tasks
Fair enough, I can see how maven-ant-tasks was not made for this use.
In your global pom you are declaring dual types for the same artifact (jar, test-jar)
Is that a problem in general? For example module A contains some samples for its tests that I would like to use in the tests of module B as well. Is it wrong (by maven best practices standards) to declare that B depends on A jar (compile scope) and on A test-jar (test scope)? Won't an integration tests project justify to depend on a module as well as the same module's samples and resources used for its unit tests?
tl;dr version: I will attempt to rearrange the tests declared on the modules and create separate module(s) for integration tests (assuming I can get 20 developers to play ball). Thanks for the answer and for making me admit defeat and stop trying to make maven work with the project instead of making the project work with maven :).
You are trying to break the maven conventions in many ways. 1. You should not ever have cyclic dependencies, 2. You should do things the maven way instead of resorting to ant-tasks 3. In your global pom you are declaring dual types for the same artifact (jar, test-jar).
Although at first this might not seem to answer your question, you should take a step back and rethink your layout. Integration tests need all the dependencies and are much more demanding than unit tests. So, instead of trying to fit them into the existing projects, create a separate maven project in the same group, that will only host integration tests (under src/java/test, main will be blank) and will have as dependencies all the other projects.
So after reading about DDD and all it benefits and glory it seems like Java EE does not make it easy for you to do so. What I thought was to make a structure like this:
Domain
Repository
Application
View
However in the comment of this answer DDD and application layer it seems like the Application layer which I thought was going to be the layer with all the services annotated with #Stateful, #WebService etc is not the place it really should be in. It seems like the domain models should have these annotations.
So now the question is: How do people structure their applications? Where do you put the different annotations and how do they use each other. Could somebody please help me understand how I can structure an java ee 6 web application? Please help and say not how I do it in a specific tool or anything like that but where the actual classes goes and what the different layers are intended to do.
I am frustrated on where to start and how to organize.
There are no specific rules on how you should structure your application. Best would be to use common sense as well as observe how others are doing it.
You can generate a simple maven project provided by weld team to see how a basic Java EE application can be structured:
mvn archetype:generate -DarchetypeArtifactId=jboss-javaee6-webapp -DarchetypeGroupId=org.jboss.weld.archetypes -DarchetypeVersion=1.0.1.CR1 -DarchetypeRepository=central
For sure you will find many other examples on github or java.net
Here's an example that might be helpful -> EAR Testing
It's called "EAR Testing", but can just as easily apply to building war files. For purposes of this answer I'll change the eartesting directory mentioned in the example to wartesting
EAR files and WAR files are nearly identical since at the Java EE spec level we decided to allow war files to contain EJBs, CDI beans, and more.
That example uses the Maven build system and has two modules, one for the "data obects" and one for "business logic". Seems to fit with how you think of thinks and might be a helpful starting point. It contains a tiny sample application with unit tests for the EJBs.
You might not have read yet, but often people refer to EJBs as hard to test. There're not anymore and that example shows the latest spec compliant solution, so you can kill a few birds with one stone starting from that setup.
What that doesn't include is a module to create the final WAR file that you would deploy in production. To create that you'd just add a third module
wartesting/business-model
wartesting/business-logic
wartesting/business-war (added)
In the busines-war you'd have a pom.xml like the following:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<parent>
<groupId>org.superbiz</groupId>
<artifactId>myear</artifactId>
<version>1.1-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>business-war</artifactId>
<packaging>war</packaging>
<dependencies>
<dependency>
<groupId>org.superbiz</groupId>
<artifactId>business-model</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.superbiz</groupId>
<artifactId>business-logic</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.openejb</groupId>
<artifactId>javaee-api</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Then create the following directories:
wartesting/business-war/src/main/java
wartesting/business-war/src/main/webapp
And we'll say for example you add the following files to each:
wartesting/business-war/src/main/java/org/superbiz/Foo.java
wartesting/business-war/src/main/webapp/WEB-INF/web.xml
wartesting/business-war/src/main/webapp/index.html
Once built, you should get a war file under wartesting/business-war/target/ containing:
WEB-INF/web.xml
WEB-INF/classes/org/superbiz/Foo.class
WEB-INF/lib/business-model-1.1-SNAPSHOT.jar
index.html