Develop Reference

Resolving conflict between transitive dependencies

I have a pom.xml where i've got hadoop-core dependency as provided
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>${hadoop.version}</version>
<scope>provided</scope>
</dependency>
When I add cfg4j as compile time dependency
<dependency>
<groupId>org.cfg4j</groupId>
<artifactId>cfg4j-core</artifactId>
<version>4.4.0</version>
</dependency>
<dependency>
<groupId>org.cfg4j</groupId>
<artifactId>cfg4j-consul</artifactId>
<version>4.4.0</version>
</dependency>
I've got an exception "java.lang.NoSuchMethodError: javax.ws.rs.core.Response.someMethod". I've investigated the problem and find out that the problem is from hadoop and cfg4j-consul. Hadoop core depends to jersey-core and cfg4j depends on cxf. Both declared javax.ws.rs as dependecy so the problem is that jersey has version 1.1 and cxf has 2.0.2. Hadoop dependency is provided, cause it's needed by Flink (framework) and it's in the lib folder. I can't just upgrade it or remove it, nor add it as compile time and exclude the lib. Even I was able to do it, I have no guarantees that hadoop will work as expected. I guess shading doesn't fix the problem cause it's not with cfg4j but one of the dependency of his dependency.
Is there way to resolve the conflict? Does gradle has it's onw ways to fix such issue?

Two approaches:
Shading: A bit more difficult as you say because this a transitive dependency, but I would have a look a Maven shade plugin and it would still be possible to declare the dependency directly if necessary.
Don't use the dependency and try to find some other library or solution for your problem.

Try the following steps, here is the source: https://reflectoring.io/nosuchmethod/
Your issue has nothing to do with the choice between Mavern and Gradle, switching therefor will not help.
Fixing a NoSuchMethodError
There are a lot of different flavors of NoSuchMethodErrors, but they all boil down to the fact that the compile time classpath differs from the runtime classpath.
The following steps will help to pinpoint the problem:
Step 1: Find Out Where the Class Comes From
First, we need to find out where the class containing the method in question comes from. We find this information in the error message of the NoSuchMethodError:
Exception in thread "main" java.lang.NoSuchMethodError:
io.reflectoring.nosuchmethod.Service.sayHello(Ljava/lang/String;)Ljava/lang/String;
Now, we can search the web or within the IDE to find out which JAR file contains this class. In the case above, we can see that it’s the Service class from our own codebase and not a class from another library.
If we have trouble finding the JAR file of the class, we can add the Java option -verbose:class when running our application. This will cause Java to print out all classes and the JARs they have been loaded from:
[Loaded io.reflectoring.nosuchmethod.Service from file:
/C:/daten/workspaces/code-examples2/patterns/build/libs/java-1.0.jar]
Step 2: Find Out Who Calls the Class
Next, we want find out where the method is being called. This information is available in the first element of the stack trace:
Exception in thread "main" java.lang.NoSuchMethodError:
io.reflectoring.nosuchmethod.Service.sayHello(Ljava/lang/String;)Ljava/lang/String;
at io.reflectoring.nosuchmethod.ProvokeNoSuchMethodError.main(ProvokeNoSuchMethodError.java:7)
Here, the class ProvokeNoSuchMethodError tries to call a method that does not exist at runtime. We should now find out which library this file belongs to.
Step 3: Check the Versions
Now that we know where the NoSuchMethodError is provoked and what method is missing, we can act.
We should now list all of our project dependencies.
In Gradle, we can call:
./gradlew dependencies > dependencies.txt
If we’re using Maven, a similiar result can be achieved with:
mvn dependency:list > dependencies.txt`
In this file, we can search for the libraries that contain the class with the missing method and the class that tries to call this method.
Usually we’ll find an output like this somewhere:
\--- org.springframework.retry:spring-retry:1.2.2.RELEASE
| \--- org.springframework:spring-core:4.3.13.RELEASE -> 5.0.8.RELEASE
The above means that the spring-retry library depends on spring-core in version 4.3.13, but some other library also depends on spring-core in version 5.0.8 and overrules the dependency version.
We can now search our dependencies.txt file for 5.0.8.RELEASE to find out which library introduces the dependency to this version.
Finally, we need to decide which of the two versions we actually need to satisfy both dependencies. Usually, this is the newer version since most frameworks are backwards compatible to some point. However, it can be the other way around or we might even not be able to resolve the conflict at all.

AEM OSGI 3rd party dependency, bundle not Active

I'm working off an AEM project, I'm new to AEM btw. I'm trying to use a couple of 3rd party non-OSGI libraries, like Dozer and Spring. I'm really struggling with this, it is a pain in the ass, maybe it is because OSGI is kind of old school. My bundles get installed, but not Active in the OSGI console, as they complain about missing packages.
I want to do step by step. First I want to make work my bundles with just Dozer and nothing else. This is my pom dependencies:
<dependency>
<groupId>net.sf.dozer</groupId>
<artifactId>dozer-osgi</artifactId>
<version>5.5.1</version>
</dependency>
Here is my maven-bundle-plugin configuration:
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<version>4.1.0</version>
<configuration>
<instructions>
<Export-Package>somepackages*</Export-Package>
<Private-Package/>
<Import-Package>*</Import-Package>
<Sling-Bundle-Resources>/META-INF</Sling-Bundle-Resources>
</instructions>
</configuration>
</plugin>
As you can see, I'm using dozer-osgi already. I'm not sure why it doesn't pick it up. It complains about this:
org.dozer,version=[5.5,6) -- Cannot be resolved
org.dozer.loader.api,version=[5.5,6) -- Cannot be resolved
First, I don't understand why it says [5.5,6), because I'm telling it to use 5.5.1. Second, I'm using dozer-osgi already, I believe it should pick it up automatically.
I tried also using:
<Embed-Dependency>dozer-osgi</Embed-Dependency>
Whit dozer-osgi added, things start to get better, and more complex at the same time. It seems like now it loads dozer, but starts complaining about transitive dependencies:
android.dalvik -- Cannot be resolved
dalvik.system -- Cannot be resolved
javassist.util.proxy -- Cannot be resolved
org.apache.commons.beanutils,version=[1.9,2) -- Cannot be resolved
org.apache.commons.beanutils.converters,version=[1.9,2) -- Cannot be resolved
The beanutils utils is easy to fix. I just need to add the regarding dependencies like this:
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
<version>1.9.2</version>
</dependency>
and add "commons-beanutils" here:
<Embed-Dependency>dozer-osgi,commons-beanutils</Embed-Dependency>
Now, it complains about:
android.dalvik -- Cannot be resolved
dalvik.system -- Cannot be resolved
javassist.util.proxy -- Cannot be resolved
org.hibernate.proxy -- Cannot be resolved
The hibernate dependency error is new. I thought beanutils depended on hibernate, but not, is Dozer the one depending on it.
I tried adding hibernate, like this:
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>3.6.9.Final</version>
</dependency>
<Embed-Dependency>dozer-osgi,commons-beanutils,hibernate-core</Embed-Dependency>
And so on, it is like a look, it gets worse and worse everytime I try to add a transitive dependency. I also tried this, only keeping dozer-osgi in my dependencies:
<Embed-Dependency>dozer-osgi</Embed-Dependency>
<Embed-Transitive>true</Embed-Transitive>
Result:
android.dalvik -- Cannot be resolved
dalvik.system -- Cannot be resolved
javassist.util.proxy -- Cannot be resolved
org.hibernate.proxy -- Cannot be resolved
org.apache.commons.beanutils -- Cannot be resolved
org.apache.commons.beanutils.converters -- Cannot be resolved
It is like if Embed-Transitive doesn't work at all.
What is the best way to work with 3rd party libraries when using Maven and AEM? In my case, Maven is using to install the bundles in my AEM instance.

In OSGi you have to distinguish between the build and the runtime. At build time your initial approach was totally fine.
You use dozer and the maven-bundle-plugin creates suitable Import-Package statements. You can check these by looking into the Manifest of the jar.
The rules for package imports result in a range of version that should work with your code. So [5.5,6) is exactly the expected import range.
Now to runtime. Here you have to supply all dependencies of your code (including the transitive ones) as bundles in AEM (or more generally in the OSGi runtime).
So you also need to install dozer OSGi as bundle. If this complains again then you also need to install its dependencies.
This is the default approach and normally totally fine.
Now if you want to make your bundle standalone in respect to having no additional runtime depenencies then you can try to embed all dependencies. Then you only need to install your bundle.
Be aware though that this is not easy. If your code uses some of the embedded classes in its own API then you get into lots of problems with embedding. So if your are not very experienced then better go the way of installing all dependencies as bundles.
To simplify the install process you can create a content package that contains all needed bundles.
You also have to check that all dependencies actually are bundles. In some cases the normal maven dependencies are not suitable. In this case have a look at apache servicemix bundles. It is a project that creates OSGi bundles for popular dependencies and deploys these to maven central.

Excluding Lombok classes from Sonar coverage report

Latest Jacoco plugin (still in snapshot version, 0.7.10-SNAPSHOT), has a nice new feature to filter out the Lombok generated code (https://github.com/jacoco/jacoco/wiki/FilteringOptions).
All we need to do is add a lombok.config file at the root of the repository with:
lombok.addLombokGeneratedAnnotation=true
When I generate the Jacoco report internally, I see the difference.
However, when my regular quality job executes and publishes the result to Sonar, I get different (ie worse) results.
How come I don't have the same results in my local report and in Sonar? Is there any workaround?

As mentionned here : https://github.com/jacoco/jacoco/pull/513#issuecomment-293176354
filtering is performed at a time of report generation (creation of html, xml, etc), not at a time of collection of execution information (creation of exec file). So that tools that read execution data directly instead of reading of xml (which is a kind of mistake on their side to rely on purely internal intermediate format, but what's done is done) and create their own report (such as SonarQube, Jenkins, etc) will need to update their dependency on JaCoCo once it will be released in order to get filtering for reports. We will notify explicitly downstream projects (in particular all mentioned above) about this when our release will be done. So once again - please be patient. Thank you for your understanding.
I didn't find a way for Sonar to read the end report instead of the exec file, so I guess we need to be patient and wait for the official 0.7.10 jacoco plugin release and then an update on Sonar side !
------ UPDATE May 9th 2018
New versions have been released, and I can confirm it works for me.
Using :
Sonar 6.7
SonarJava plugin 5.1.1.13214
jacoco maven plugin 0.8.1
lombok.addLombokGeneratedAnnotation=true in lombok.config
I now get much better coverage results reported to Sonar, as Lombok generated code is now ignored. It really helps identifying what the "real" uncovered areas are, and whether it's risky or not.

First you have to check your lombok version is at least 1.16.14
pom.xml:
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<scope>provided</scope>
<version>1.16.14</version>
</dependency>
Then you have to check that your Jacoco version is at least 0.8.0
pom.xml:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.0</version>
<!-- // -->
</plugin>
Then you have to add a lombok.config file in the src folder of your project (not in the resources folder)
lombok.config:
# tells Lombok that this is the root directory and that it shouldn’t search parent directories for more configuration files
config.stopBubbling = true
# tells Lombok to add #lombok.Generated annotation to all generated methods
lombok.addLombokGeneratedAnnotation = true

Maven-dependency-plugin and annotations with SOURCE RetentionPolicy

In a mvn project where I am utilizing maven-dependency-plugin to detect unused dependencies, there is seemingly no dependency scope I can specify for Google's AutoValue (com.google.auto.value:auto-value) to that will convince the plugin that the dependency is being used in spite of the fact that annotations from the package are being used (e.g. #AutoValue) and the project won't build if auto-value is excluded.
Now one solution is simply adding a configuration entry to my plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<configuration>
<usedDependencies>
<usedDependency>com.google.auto.value:auto-value</usedDependency>
</usedDependencies>
</configuration>
</plugin>
But I would be curious to know whether it's possible to configure either the maven-dependency-plugin or the dependency entry for auto-value in a way that would detect usage of the dependency per its annotations?
My suspicion is that this isn't possible because the RetentionPolicy of the annotations I'm using from auto-value are of RetentionPolicy.SOURCE and are discarded by the compiler. Is this correct?

Unfortunately, your suspicion is correct. The maven-dependency-plugin documentation specifically lists this as a concern here for source level annotations: http://maven.apache.org/shared/maven-dependency-analyzer/
Warning: Analysis is not done at source but bytecode level, then some cases are not detected (constants, annotations with source-only retention, links in javadoc) which can lead to wrong result if they are the only use of a dependency.
You can force AutoValue as used with usedDependencies as you have in your example or use the ignoredUnusedDeclaredDependencies configuration instead (which is what I did recently).
I don't believe it is possible to configure the dependency section to avoid this because maven doesn't provide a scope level that is compile only. I mark AutoValue with the provided scope to keep it out of any shaded jars I might make.
Lastly, you could write (or find if it exists) a custom dependency analyzer that takes source level annotations into account. See the documentation here http://maven.apache.org/plugins/maven-dependency-plugin/analyze-mojo.html#analyzer. Probably not worth the effort.

How to configure a longer version Number in artifactory

The version-numbers for our jars have to be longer them x.x.x.
We would rather need x.x.x.x to integrate some old-fashioned self-made mechanism.
This is, because we tag our software with x.x.x and as soon as we have a delivery to a customer one specific jar has to be build exactly at this point of time to fit to another backend,
which communicates with our program. For that reason this one jar has the version 2.3.4.1,
when generated and in next delivery of the same Version it is build and named 2.3.4.2.
Now artifactory cannot handle this an doesn't save more than x.x.x.2 in some cases.
So we thought of maybe edit the regular expression in the maven repository layout (see attached Screenshot) Because testing the path in the field below shows,
that it cannot handle the version number. Of course for the rest of our jars still x.x.x has to work..
For Example here is the maven-metadata.xml
<?xml version="1.0" encoding="UTF-8"?>
<metadata>
<groupId>com.firm</groupId>
<artifactId>someid</artifactId>
<version>1.5.1</version>
<versioning>
<latest>1.5.1</latest>
<release>1.5.1</release>
<versions>
<version>1.4.62</version>
</versions>
<lastUpdated>20120926073942</lastUpdated>
</versioning>
</metadata>
The folder structure looks like:
someid
- 1.4.62
- 1.4.62.1
- 1.4.62.2
- 1.4.62.3
If we deploy an new artifact version (1.4.62.1), the maven-metadata.xml contains the 1.4.62.1 version.
But the artifactory overrides the version number (1.4.62.x) to (1.4.62) after an unspecified time. It seems that the artifactory only support major, minor and revision numbers, and deletes the buildnumber.
Now we looking for a solution do disable this behavior.
We use the JFrog Artifactory version 2.5.0 (rev. 13086).

I apologize in advance that this will be an unsatisfactory answer for you.
Your version format is incompatible with the version format used by Maven. Maven uses a .. format for version numbers. Anything after this needs to be prefaced with a dash for it to be properly used by Maven. For more information about how version numbers are used and formatted in Maven, please refer to this section of the Maven book: http://www.sonatype.com/books/mvnref-book/reference/pom-relationships-sect-pom-syntax.html