Develop Reference

Resolving conflict between transitive dependencies

I have a pom.xml where i've got hadoop-core dependency as provided
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>${hadoop.version}</version>
<scope>provided</scope>
</dependency>
When I add cfg4j as compile time dependency
<dependency>
<groupId>org.cfg4j</groupId>
<artifactId>cfg4j-core</artifactId>
<version>4.4.0</version>
</dependency>
<dependency>
<groupId>org.cfg4j</groupId>
<artifactId>cfg4j-consul</artifactId>
<version>4.4.0</version>
</dependency>
I've got an exception "java.lang.NoSuchMethodError: javax.ws.rs.core.Response.someMethod". I've investigated the problem and find out that the problem is from hadoop and cfg4j-consul. Hadoop core depends to jersey-core and cfg4j depends on cxf. Both declared javax.ws.rs as dependecy so the problem is that jersey has version 1.1 and cxf has 2.0.2. Hadoop dependency is provided, cause it's needed by Flink (framework) and it's in the lib folder. I can't just upgrade it or remove it, nor add it as compile time and exclude the lib. Even I was able to do it, I have no guarantees that hadoop will work as expected. I guess shading doesn't fix the problem cause it's not with cfg4j but one of the dependency of his dependency.
Is there way to resolve the conflict? Does gradle has it's onw ways to fix such issue?

Two approaches:
Shading: A bit more difficult as you say because this a transitive dependency, but I would have a look a Maven shade plugin and it would still be possible to declare the dependency directly if necessary.
Don't use the dependency and try to find some other library or solution for your problem.

Try the following steps, here is the source: https://reflectoring.io/nosuchmethod/
Your issue has nothing to do with the choice between Mavern and Gradle, switching therefor will not help.
Fixing a NoSuchMethodError
There are a lot of different flavors of NoSuchMethodErrors, but they all boil down to the fact that the compile time classpath differs from the runtime classpath.
The following steps will help to pinpoint the problem:
Step 1: Find Out Where the Class Comes From
First, we need to find out where the class containing the method in question comes from. We find this information in the error message of the NoSuchMethodError:
Exception in thread "main" java.lang.NoSuchMethodError:
io.reflectoring.nosuchmethod.Service.sayHello(Ljava/lang/String;)Ljava/lang/String;
Now, we can search the web or within the IDE to find out which JAR file contains this class. In the case above, we can see that it’s the Service class from our own codebase and not a class from another library.
If we have trouble finding the JAR file of the class, we can add the Java option -verbose:class when running our application. This will cause Java to print out all classes and the JARs they have been loaded from:
[Loaded io.reflectoring.nosuchmethod.Service from file:
/C:/daten/workspaces/code-examples2/patterns/build/libs/java-1.0.jar]
Step 2: Find Out Who Calls the Class
Next, we want find out where the method is being called. This information is available in the first element of the stack trace:
Exception in thread "main" java.lang.NoSuchMethodError:
io.reflectoring.nosuchmethod.Service.sayHello(Ljava/lang/String;)Ljava/lang/String;
at io.reflectoring.nosuchmethod.ProvokeNoSuchMethodError.main(ProvokeNoSuchMethodError.java:7)
Here, the class ProvokeNoSuchMethodError tries to call a method that does not exist at runtime. We should now find out which library this file belongs to.
Step 3: Check the Versions
Now that we know where the NoSuchMethodError is provoked and what method is missing, we can act.
We should now list all of our project dependencies.
In Gradle, we can call:
./gradlew dependencies > dependencies.txt
If we’re using Maven, a similiar result can be achieved with:
mvn dependency:list > dependencies.txt`
In this file, we can search for the libraries that contain the class with the missing method and the class that tries to call this method.
Usually we’ll find an output like this somewhere:
\--- org.springframework.retry:spring-retry:1.2.2.RELEASE
| \--- org.springframework:spring-core:4.3.13.RELEASE -> 5.0.8.RELEASE
The above means that the spring-retry library depends on spring-core in version 4.3.13, but some other library also depends on spring-core in version 5.0.8 and overrules the dependency version.
We can now search our dependencies.txt file for 5.0.8.RELEASE to find out which library introduces the dependency to this version.
Finally, we need to decide which of the two versions we actually need to satisfy both dependencies. Usually, this is the newer version since most frameworks are backwards compatible to some point. However, it can be the other way around or we might even not be able to resolve the conflict at all.

Maven dependencies exclusions not working

I'm using enforcer plugin of Maven and I see a behavior that I dont quite understand and it's dangerous.
Let's say that I have a conflict since dependency A has bla.jar:1.0 and is in conflict with my dependnecy B which has bla.jar:2.0
Then to fix the conflict, I make an exclude of bla.jar:1.0 from A
<dependency>
<groupId>com.foo</groupId>
<artifactId>A</artifactId>
<version>a.version.bla</version>
<exclusions>
<exclusion>
<groupId>com.omg</groupId>
<artifactId>bla</artifactId>
</exclusion>
</exclusions>
</dependency>
expecting the application will get the bla.jar:2.0 fron classpath. But then I see when I run some unit test that the java proce3ss cannot find bla.jar ion the classpath at all and is giving me ClassNotFound in runtime.
Any idea what's wrong here?
I have in my pom defined from top to bottom B and then A

Please note that exclusions are not the best way to resolve dependency version conflicts.
The best approach is to use <dependencyManagement>. It allows you to set a version that replaces all transitive versions of that dependency.
In your case, I would first change the exclusion to <dependencyManagement>. Then I would proceed in the following way:
Check mvn dependency:list which version of the dependency is on the classpath. It should be the one specified in <dependencyManagement> unless there is no version of that dependency in your dependency tree. If you find more than one, then probably the groupId changed at some point. Then you need exclusions.
Check the scope of the dependency and verify that it is indeed compile.
Then open the dependency jar and see whether this jar really contains the class for which you get ClassNotFound. Often classes change from version to version.

opendj-core 6.5.0 maven dependency

On OpenDJ 2.6.4, i’m using the dependency “opendj-ldap-sdk” in order to use the
following classes:
org.forgerock.opendj.asn1.ASN1;
org.forgerock.opendj.asn1.ASN1Writer;
org.forgerock.opendj.ldap.ByteStringBuilder;
I checked that the same classes exist on Directory Services 6.5 on lib\opendj-core.jar but i cannot find the maven dependency to use it (in a context of migration from OpenDJ 2.6.4 to Directory Services 6.5).
I’ve found this dependency:
<dependency>
<groupId>org.codice.org.forgerock.opendj</groupId>
<artifactId>opendj-core</artifactId>
<version>3.0.0.ALPHA1</version>
</dependency>
but the version doesn’t match..
Where can I find the dependency?

You can unzip DS-6.5.0.zip and then add the <systemPath> element to your maven dependency. See https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#System_Dependencies .
For a more robust mechanism, if you intend to use Directory Services 6.5 in production, then I would recommend that you contact your ForgeRock support. They will explain to you how to compile your plugin(s) against this version.

Maven: Resolving Duplicate Dependencies

I'm developing an application that will be used internally at our company. In order for it to interop with our other internal systems I have to use some maven dependencies that we use internally, but this is causing some issues with using some external 3rd party dependencies that I also need.
So essentially my pom looks like this:
<dependencies>
<dependency>
internal-framework-artifact
</dependency>
<dependency>
necessary-third-party-artifact
</dependency>
</dependencies>
I've come to find that both of these dependencies have the apache's commons-collections as one of their own dependencies (among a large number of others, but we'll just keep it at one for this question's simplicity).
If I place exclusion rules on both of them for the commons-collections pom I can compile the project, but my resulting jar won't have access to either version of commons-collections and will just result in a java.lang.NoClassDefFoundError exception. Removing the exclusion rule on either of them just results in a mvn compiler error:
[WARNING] Rule 2: org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message:
Duplicate classes found:
I've been looking through various so q/a's and I can't really seem to find something that's 100% relevant to my situation. I'm really at a loss as to how to resolve this. Am I missing something really obvious?

I've never actually used the maven-shade-plugin for shading, but I think this is the exact use case it was designed for.
Create a new project that uses the maven-shade-plugin (see: http://maven.apache.org/plugins/maven-shade-plugin/) to produce an uber-jar version of internal-framework-artifact which contains that classes in internal-framework-artifact and all its dependencies. Configure the plugin so that it relocates all the classes that are also dependencies of necessary-third-party-artifact to some non-conflicting package names. This new project should produce a .jar with a different name, something like internal-framework-artifact-with-dependencies.
Now modify your original pom so that it is dependent on internal-framework-artifact-with-dependencies instead, and it should work.

Grails NoSuchField exception because of wrong denpendecy version of apache http core

I have Grails 2.0.4 application that depends on a jar that depends on org.apache.httpcomponents:httpcore:4.3.1
I'm getting a NoSuchField exception which I managed to track down to org.apache.http.impl.conn.BasicLineParser.INSTANCE.
This field is only present from httpcore 4.3, before this version the field was BasicLineParser.DEFAULT instead
My BuildConfig.groovy is like:
dependencies {
runtime 'mysql:mysql-connector-java:5.1.16'
runtime 'org.apache.httpcomponents:httpclient:4.3.1'
runtime 'org.apache.httpcomponents:httpcore:4.3.1'
}
Running the dependecy report I found out the there is a signpost-commonshttp4 by oauth.signpost that depends on httpcore:4.0.1 which is probably the root cause of my issue, because it doesn't have the INSTANCE property.
I've tried to exclude this dependency adding
runtime('oauth.signpost:signpost-commonshttp4:1.2.1.1'){
excludes 'httpcore'
}
with no success.
Is there any other way to prevent the code to use the older library version?
Regards
RESOLVED
I managed to resolve this issue adding an extra dependecies configuration
build 'org.apache.httpcomponents:httpclient:4.3.1'
build 'org.apache.httpcomponents:httpcore:4.3.1'
along with the runtime
Thank you all