Develop Reference

Resolving conflict between transitive dependencies

I have a pom.xml where i've got hadoop-core dependency as provided
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>${hadoop.version}</version>
<scope>provided</scope>
</dependency>
When I add cfg4j as compile time dependency
<dependency>
<groupId>org.cfg4j</groupId>
<artifactId>cfg4j-core</artifactId>
<version>4.4.0</version>
</dependency>
<dependency>
<groupId>org.cfg4j</groupId>
<artifactId>cfg4j-consul</artifactId>
<version>4.4.0</version>
</dependency>
I've got an exception "java.lang.NoSuchMethodError: javax.ws.rs.core.Response.someMethod". I've investigated the problem and find out that the problem is from hadoop and cfg4j-consul. Hadoop core depends to jersey-core and cfg4j depends on cxf. Both declared javax.ws.rs as dependecy so the problem is that jersey has version 1.1 and cxf has 2.0.2. Hadoop dependency is provided, cause it's needed by Flink (framework) and it's in the lib folder. I can't just upgrade it or remove it, nor add it as compile time and exclude the lib. Even I was able to do it, I have no guarantees that hadoop will work as expected. I guess shading doesn't fix the problem cause it's not with cfg4j but one of the dependency of his dependency.
Is there way to resolve the conflict? Does gradle has it's onw ways to fix such issue?

Two approaches:
Shading: A bit more difficult as you say because this a transitive dependency, but I would have a look a Maven shade plugin and it would still be possible to declare the dependency directly if necessary.
Don't use the dependency and try to find some other library or solution for your problem.

Try the following steps, here is the source: https://reflectoring.io/nosuchmethod/
Your issue has nothing to do with the choice between Mavern and Gradle, switching therefor will not help.
Fixing a NoSuchMethodError
There are a lot of different flavors of NoSuchMethodErrors, but they all boil down to the fact that the compile time classpath differs from the runtime classpath.
The following steps will help to pinpoint the problem:
Step 1: Find Out Where the Class Comes From
First, we need to find out where the class containing the method in question comes from. We find this information in the error message of the NoSuchMethodError:
Exception in thread "main" java.lang.NoSuchMethodError:
io.reflectoring.nosuchmethod.Service.sayHello(Ljava/lang/String;)Ljava/lang/String;
Now, we can search the web or within the IDE to find out which JAR file contains this class. In the case above, we can see that it’s the Service class from our own codebase and not a class from another library.
If we have trouble finding the JAR file of the class, we can add the Java option -verbose:class when running our application. This will cause Java to print out all classes and the JARs they have been loaded from:
[Loaded io.reflectoring.nosuchmethod.Service from file:
/C:/daten/workspaces/code-examples2/patterns/build/libs/java-1.0.jar]
Step 2: Find Out Who Calls the Class
Next, we want find out where the method is being called. This information is available in the first element of the stack trace:
Exception in thread "main" java.lang.NoSuchMethodError:
io.reflectoring.nosuchmethod.Service.sayHello(Ljava/lang/String;)Ljava/lang/String;
at io.reflectoring.nosuchmethod.ProvokeNoSuchMethodError.main(ProvokeNoSuchMethodError.java:7)
Here, the class ProvokeNoSuchMethodError tries to call a method that does not exist at runtime. We should now find out which library this file belongs to.
Step 3: Check the Versions
Now that we know where the NoSuchMethodError is provoked and what method is missing, we can act.
We should now list all of our project dependencies.
In Gradle, we can call:
./gradlew dependencies > dependencies.txt
If we’re using Maven, a similiar result can be achieved with:
mvn dependency:list > dependencies.txt`
In this file, we can search for the libraries that contain the class with the missing method and the class that tries to call this method.
Usually we’ll find an output like this somewhere:
\--- org.springframework.retry:spring-retry:1.2.2.RELEASE
| \--- org.springframework:spring-core:4.3.13.RELEASE -> 5.0.8.RELEASE
The above means that the spring-retry library depends on spring-core in version 4.3.13, but some other library also depends on spring-core in version 5.0.8 and overrules the dependency version.
We can now search our dependencies.txt file for 5.0.8.RELEASE to find out which library introduces the dependency to this version.
Finally, we need to decide which of the two versions we actually need to satisfy both dependencies. Usually, this is the newer version since most frameworks are backwards compatible to some point. However, it can be the other way around or we might even not be able to resolve the conflict at all.

AEM OSGI 3rd party dependency, bundle not Active

I'm working off an AEM project, I'm new to AEM btw. I'm trying to use a couple of 3rd party non-OSGI libraries, like Dozer and Spring. I'm really struggling with this, it is a pain in the ass, maybe it is because OSGI is kind of old school. My bundles get installed, but not Active in the OSGI console, as they complain about missing packages.
I want to do step by step. First I want to make work my bundles with just Dozer and nothing else. This is my pom dependencies:
<dependency>
<groupId>net.sf.dozer</groupId>
<artifactId>dozer-osgi</artifactId>
<version>5.5.1</version>
</dependency>
Here is my maven-bundle-plugin configuration:
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<version>4.1.0</version>
<configuration>
<instructions>
<Export-Package>somepackages*</Export-Package>
<Private-Package/>
<Import-Package>*</Import-Package>
<Sling-Bundle-Resources>/META-INF</Sling-Bundle-Resources>
</instructions>
</configuration>
</plugin>
As you can see, I'm using dozer-osgi already. I'm not sure why it doesn't pick it up. It complains about this:
org.dozer,version=[5.5,6) -- Cannot be resolved
org.dozer.loader.api,version=[5.5,6) -- Cannot be resolved
First, I don't understand why it says [5.5,6), because I'm telling it to use 5.5.1. Second, I'm using dozer-osgi already, I believe it should pick it up automatically.
I tried also using:
<Embed-Dependency>dozer-osgi</Embed-Dependency>
Whit dozer-osgi added, things start to get better, and more complex at the same time. It seems like now it loads dozer, but starts complaining about transitive dependencies:
android.dalvik -- Cannot be resolved
dalvik.system -- Cannot be resolved
javassist.util.proxy -- Cannot be resolved
org.apache.commons.beanutils,version=[1.9,2) -- Cannot be resolved
org.apache.commons.beanutils.converters,version=[1.9,2) -- Cannot be resolved
The beanutils utils is easy to fix. I just need to add the regarding dependencies like this:
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
<version>1.9.2</version>
</dependency>
and add "commons-beanutils" here:
<Embed-Dependency>dozer-osgi,commons-beanutils</Embed-Dependency>
Now, it complains about:
android.dalvik -- Cannot be resolved
dalvik.system -- Cannot be resolved
javassist.util.proxy -- Cannot be resolved
org.hibernate.proxy -- Cannot be resolved
The hibernate dependency error is new. I thought beanutils depended on hibernate, but not, is Dozer the one depending on it.
I tried adding hibernate, like this:
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>3.6.9.Final</version>
</dependency>
<Embed-Dependency>dozer-osgi,commons-beanutils,hibernate-core</Embed-Dependency>
And so on, it is like a look, it gets worse and worse everytime I try to add a transitive dependency. I also tried this, only keeping dozer-osgi in my dependencies:
<Embed-Dependency>dozer-osgi</Embed-Dependency>
<Embed-Transitive>true</Embed-Transitive>
Result:
android.dalvik -- Cannot be resolved
dalvik.system -- Cannot be resolved
javassist.util.proxy -- Cannot be resolved
org.hibernate.proxy -- Cannot be resolved
org.apache.commons.beanutils -- Cannot be resolved
org.apache.commons.beanutils.converters -- Cannot be resolved
It is like if Embed-Transitive doesn't work at all.
What is the best way to work with 3rd party libraries when using Maven and AEM? In my case, Maven is using to install the bundles in my AEM instance.

In OSGi you have to distinguish between the build and the runtime. At build time your initial approach was totally fine.
You use dozer and the maven-bundle-plugin creates suitable Import-Package statements. You can check these by looking into the Manifest of the jar.
The rules for package imports result in a range of version that should work with your code. So [5.5,6) is exactly the expected import range.
Now to runtime. Here you have to supply all dependencies of your code (including the transitive ones) as bundles in AEM (or more generally in the OSGi runtime).
So you also need to install dozer OSGi as bundle. If this complains again then you also need to install its dependencies.
This is the default approach and normally totally fine.
Now if you want to make your bundle standalone in respect to having no additional runtime depenencies then you can try to embed all dependencies. Then you only need to install your bundle.
Be aware though that this is not easy. If your code uses some of the embedded classes in its own API then you get into lots of problems with embedding. So if your are not very experienced then better go the way of installing all dependencies as bundles.
To simplify the install process you can create a content package that contains all needed bundles.
You also have to check that all dependencies actually are bundles. In some cases the normal maven dependencies are not suitable. In this case have a look at apache servicemix bundles. It is a project that creates OSGi bundles for popular dependencies and deploys these to maven central.

Cannot resolve symbol 'Immutable'

I'm new to threading in java and now read the "java concurrency in practice". As you possibly understand I'm trying to do any examples but can't. when I'm trying to use #Immutable annotation IDE(Idea) underlines it red.
What is the possible reason of this?

The reason is that you do not have the correct jar in your classpath.
The maven artifact for this jar is:
<dependency>
<groupId>net.jcip</groupId>
<artifactId>jcip-annotations</artifactId>
<version>1.0</version>
<dependency>
If you do not have this jar in your classpath, you don't have the annotation.
Note however that this is superseded by JSR 305. There is also a Maven artifact for an implementation of that JSR:
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
<version>2.0.1</version>
</dependency>
(side note: excellent choice of a read! This is one must-have book for all Java devs out there)

Download the jar file providing the jcip annotations and add it to the module dependencies.

As far as working with that book's examples, so all those annotations, like #ThreadSafe, #Immutable/#Threadsafe, they all are just symbolic (though fge above is correct, you can use those jars as well, but from learning perspective, just ignore them)
So, simply ignore them. Anywhere, if you see #Immutable, it simply means that code written is IMMUTABLE, by the virtue of code implementation.
You don't need any jar at all. Just remove those annotations from your examples :)

java compilation error using findbugs. com.sun.tools.javac.code.Symbol$CompletionFailure: class file for javax.annotation.meta.When not found

I am trying to use the annotations of findbugs 1.3.2.
I used the edu.umd.cs.findbugs.annotations.NonNull annotation in a simple test,
and it works fine.
However, now I have a large project, composed of sub-modules, using maven,
and I get the following compilation error by just importing that annotation in some java file:
com.sun.tools.javac.code.Symbol$CompletionFailure: class file for javax.annotation.meta.When not found
what can be the problem?
i tried adding the findbugs dependency in all sub-modules.
maybe it is a conflict with jsr305? I see that one of our dependencies uses jsr305 1.3.9.

In my experience, this error always occurs when you use #NonNull and jsr305.jar is not on the classpath. In order to use the findbugs annotations, you must add both annotations.jar and jsr305.jar to the classpath.
Some annotations (I am thinking of #SuppressWarnings) can be used without jsr305.jar, but #NonNull requires it for sure.
(You mentioned jsr305.jar in your question, but you didn't say explicitly that you've checked. Also, people are going to land here when searching for the error message, and they will need to add jsr305.jar.) Only one version of jsr305.jar should be on the classpath.

You can also use this dependency
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>annotations</artifactId>
<version>3.0.1</version>
</dependency>

What are unused/undeclared dependencies in Maven? What to do with them?

Maven dependency:analyze complains about the dependencies in my project. How does it determine which are unused and which are undeclared? What should I do about them?
Example:
$ mvn dependency:analyze
...
[WARNING] Used undeclared dependencies found:
[WARNING] org.slf4j:slf4j-api:jar:1.5.0:provided
[WARNING] commons-logging:commons-logging:jar:1.1.1:compile
[WARNING] commons-dbutils:commons-dbutils:jar:1.1-osgi:provided
[WARNING] org.codehaus.jackson:jackson-core-asl:jar:1.6.1:compile
...
[WARNING] Unused declared dependencies found:
[WARNING] commons-cli:commons-cli:jar:1.0:compile
[WARNING] org.mortbay.jetty:servlet-api:jar:2.5-20081211:test
[WARNING] org.apache.httpcomponents:httpclient:jar:4.0-alpha4:compile
[WARNING] commons-collections:commons-collections:jar:3.2:provided
[WARNING] javax.mail:mail:jar:1.4:provided
Note:
A lot of these dependencies are used in my runtime container and I declared them as provided to avoid having the same library on the classpath twice with different versions.

Not sure how Maven determines this. It is not required to address all the items reported by this, but this information can be used as appropriate.
Used undeclared dependencies are those which are required, but have not been explicitly declared as dependencies in your project. They are however available thanks to transitive dependency of other dependencies in your project. It is a good idea to explicitly declare these dependencies. This also allows you to control the version of these dependencies (perhaps matching the version provided by your runtime).
As for unused declared dependencies, it is a good idea to remove them. Why add unnecessary dependency to your project? But then transitivity can bring these in anyway, perhaps, conflicting with your runtime versions. In this case, you will need to specify them — essentially to control the version.
By the way, mvn dependency:tree gives the dependency tree of the project, which gives you a better perspective of how each dependency fits in in your project.

The answer to:
"How does it determine which are unused and which are undeclared?".
Maven uses Object WebASM framework that analyzes your raw bytecode. It goes through all your classes and then builds a list of all classes that these reference. That is the how.
As to what to do, I would not recommend removing the "unused, declared dependecies" unless you are absolutely sure they actually unused.

Used undeclared dependencies
Simply, they are the transitive dependencies which you are using them but WITHOUT declaring them explicitly inside your POM file.
In the below digram, the orange colored one.
Hint:
It is good idea to declare them inside your POM file to be loosly coupled against your first level dependencies, so in the future if they planned to change their implementation and not to use this transitive dependency anymore, your application will be safe!
Unused declared dependencies
Simply, they are the dependencies which you are declearing them inside your POM file WITHOUT using them in your application code.
In the below digram, the red colored one.
Hint:
It is good idea to remove them from your POM file, because they are not used and to save the final size of the application artifact also to avoid any developer from using wrong classes by mistake!

This can be easily fixed with by adding ignoredUnusedDeclaredDependencies in pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<configuration>
<ignoredUnusedDeclaredDependencies>
<ignoredUnusedDeclaredDependency>org.slf4j:slf4j-api</ignoredUnusedDeclaredDependency>
</ignoredUnusedDeclaredDependencies>
</configuration>
</execution>
</executions>
</plugin>