I have a Java application built with Maven with a lot of dependencies. When performing my test cases they sometimes pass fine, sometimes they fail because of some incompatible class combinations. So it seems to that there must be some classes twice in classpath which are taken randomly. The one is fine the other not.
How can I find out which classes / jars are incompatible in my classpath?
What is the right approach using Maven not to fall in that compatibility-traps?
I think a better solution would be to use the maven-duplicate-finder-plugin.
Note: The new version is the duplicate-finder-maven-plugin.
You can try using this tool Tattletale.
You can detect duplicate classfile definitions in the classpath or module path using ClassGraph (disclaimer, I am the author of ClassGraph):
for (Entry<String, ResourceList> dup :
new ClassGraph().scan().getAllResources().classFilesOnly().findDuplicatePaths()) {
System.out.println(dup.getKey()); // Classfile path
for (Resource res : dup.getValue()) {
System.out.println(" -> " + res.getURI()); // Resource URI, showing classpath element
}
}
There is a plugin in eclipse to check for duplicate classes in the build path (ClasspathChecker http://classpathchecker.free.fr/)
This problem is basically an application of the more general problem to "somehow scan the classpath (CP) and collect all class files and other resources", and then find duplicates in that...
There are a number of existing libraries for CP scanning (and it's not trivial to do this right in all environments, especially since the application class loader in Java 9 is no longer an URLClassLoader), notably Classgraph, using which it's relatively trivial to do this.
PS: For Java versions <9, JHades (jhades.github.io) is nice (but NOK on Java 9/10/11).
This is a another simple Open Source Duplicate Classpath Finder tool - Classpath Inspector
which gives pretty decent report of duplicate classes in the classpath.
You can use the maven dependency:tree to see the maven hierarchy of your project and
maven exclusion to exclude the jars you don't want
Related
I want to migrate my old java code to the java9 modules. E.g. in the classpath there is a jar-file named org.eclipse.jface.3.7.0.v20110928.jar. In the classpath it is referenced as org.eclipse.jface_3.7.0.v20110928.jar. The point in the filename after jface is replaced with an underscore in the classpath. Don't know how it works. Maybe it is because .3.7.0. is not a legal java identifier.
But now I want to use it as a module. I get an error for the modulename. The part '.7.0.' is not allowed, because a number can not be a java identifier. The underscore is a reserved word in java9.
First I used the same name for the module as it saw it in the classpath (org.eclipse.jface_3.7.0.v20110928.jar). But it is an error. The I tried to use the name of the file (org.eclipse.jface.3.7.0.v20110928.jar), same error.
module iDEpdf.src
{
exports org.idepdf.ri.common.utility.annotation;
...
requires org.eclipse.jface.3.7.0.v20110928;
}
'.3.7.0' is marked and the error is 'illegal token'. When I use org.eclipse.jface_3.7.0.v20110928 the marked substring is '.7.0'. The error is the same.
If it is possible I don't want to rename the jar-file. I don't understand how it works for the classpath and I don't understand why it does not work for the module. How should I handle this?
I think you are using a very old library of JFace. You probably need to update or add Automatic-Module-Name.
Since the library doesn’t have a module descriptor yet, the module isn’t recognized as valid.
My question is also: why migrate to the module system before updating libraries to recent versions?
Please also see this question or this one.
While executing tests in Maven Surefire I see ClassNotFoundExceptions from time to time.
This really gives me a headache, since:
the missing classes vary. Only around 5 classes are affected, but which one it is varys from build to build. However, I see no unique similarities between these classes, which they wouldn't share with 20 other classes of the same kind.
These missing classes come from 2 different dependencies. These are managed by Maven, of course.
When a CNFE is raised I had a look at the class path (during runtime!) and it looks fine!
How I analysed the class path
I took the code of the "class path scanner" from Arno Haase:
public List<URL> getRootUrls () {
List<URL> result = new ArrayList<> ();
ClassLoader cl = Thread.currentThread().getContextClassLoader();
while (cl != null) {
if (cl instanceof URLClassLoader) {
URL[] urls = ((URLClassLoader) cl).getURLs();
result.addAll (Arrays.asList (urls));
}
cl = cl.getParent();
}
return result;
}
The list of URLs is quite short:
a few JRE libs
a "surefire booter jar"
The latter jar bundles all my Maven dependencies in its Manifest file, as described in the Surefire docs.
So I dug further and analysed the "Class-Path" attribute of the manifest. There I found the dependent jar listed, where the missing class should have come from.
When browsing through the jar's entries, I also found the missing class there. The fully qualified path also matches.
So in principle everything seems to be correct and in place.
Where should I continue to investigate now?
There are several things to check for problems like these.
Does this happen from command line or via CI build only? If using Jenkins or Hudson, is this a Maven project or a FreeStyle project with a Maven build step? If this is a Maven project, switch it to a FreeStyle project with a Maven build step, and that just may solve the issue. Stephen Connolly of the Maven team considers the Jenkins Maven build type evil.
Ensure there is only one version of each dependency and that related dependencies (Spring, ASM, Hibernate, etc.) have the same/compatible versions. Pay particular attention to artifacts where the group ID or artifact ID has changed, for example spring.jar vs. spring-core.jar. The old Tattletale plugin might be useful to get started.
Replace any dependencies ending in -all with their component parts. -all jars may contain every class needed to run the library - repackaged into the jar file where Maven's dependency resolution process can't get at them - instead of referencing them as dependencies. mockito-all, hamcrest-all, powermock-all, cglib are examples.
If using coverage tools (Jacoco, Clover) does the build work if you turn off the coverage? If yes, the tool may be introducing classpath jars that conflict with your app's. (Different version of CGLIB for example.) Run in debug mode and compare dependencies with/without coverage to identify the differences.
If using JUnit, make sure Maven surefire is using the right JUnit provider for your version of JUnit. Run the build in debug mode with -X (redirect output to a file if using command line). Grep the output for surefire-junit. You should find something like this:
[DEBUG] org.apache.maven.surefire:surefire-junit4:jar:2.16:test (selected for test)
Now, make sure the version of the provider matches the version of JUnit used. Check out the Maven docs for information on which provider to use and how to configure.
A project runs on Google App Engine. The project has dependency that uses a class that can't be invoked on App Engine due to security constraints (it's not on the whitelist). My (very hacky) solution was to just copy a modified version of that class into my project (matching the original Class's name and package) that doesn't need the restricted class. This works on both dev and live, I assume because my source appears in the classpath before my external dependencies.
To make it a bit cleaner, I decided to put my modified version of that class into it's own project that can be packaged up in a jar and published for anyone else to use should they face this problem.
Here's my build.gradle:
// my jar that has 'fixed' version of Class.
compile files('path/to/my-hack-0.0.1.jar')
// dependency that includes class that won't run on appengine
compile 'org.elasticsearch:elasticsearch:1.4.4'
On my local dev server, this works fine, the code finds my hacked version of the class first at runtime. On live, for some unknown reason, the version in the elasticsearch dependency is loaded first.
I know having two versions of the same class in the classpath isn't ideal but I was hoping I could reliably force my version to be at the start of the classpath. Any ideas? Alternatively, is there a better way to solve this problem?
Not really sure if this is what people visiting this question were looking for, but this was what my problem and a solution that I reached at.
Jar A: contains class XYZ
Jar B: also contains class XYZ
My Project needs Jar B on the classpath before Jar A to be able to get compiled.
Problem is Gradle sorts the dependencies based on alphabetical order post resolving them which meant Jar B will be coming after Jar A in the generated classpath leading to error while compiling.
Solution:
Declare a custom configuration and patch the compileClasspath. This is how the relevant portion of build.gradle might look like.
configurations {
priority
sourceSets.main.compileClasspath = configurations.priority + sourceSets.main.compileClasspath
}
dependencies {
priority 'org.blah:JarB:2.3'
compile 'org.blah:JarA:2.4'
...
}
It's the app engine classloader I should have been investigating, not gradle...
App Engine allows you to customise the class loader JAR ordering with a little bit of xml in your appengine-web.xml. In my case:
<class-loader-config>
<priority-specifier filename="my-hack-0.0.1.jar"/>
</class-loader-config>
This places my-hack-0.0.1.jar as the first JAR file to be searched for classes, barring those in the directory war/WEB-INF/classes/.
...Thanks to a nudge in the right direction from #Danilo Tommasina :)
UPDATE 2020:
I just hit the same problem again and came across my own question... This time, live appengine was loading a different version of org.json than was being loaded in dev. Very frustrating and no amount of fiddling the build script would fix it. For future searchers, if you're getting this:
java.lang.NoSuchMethodError: org.json.JSONObject.keySet()Ljava/util/Set;
It's because it's loading an old org.json dependency from god-knows-where. I fixed it by adding this to my appengine-web.xml:
<class-loader-config>
<priority-specifier filename="json-20180130.jar"/>
</class-loader-config>
You'll also need a matching dependency in build.gradle if you don't already have one:
compile 'org.json:json:20180130'
According to gradle dependencies documentation, the order of dependencies defines the order in the classpath. So, we can simply put the libraries in the correct order in "dependencies".
But beware! here are two rules with higher priorities:
For a dynamic version, a 'higher' static version is preferred over a 'lower' version.
Modules declared by a module descriptor file (Ivy or POM file) are preferred over modules that have an artifact file only.
I'm currently getting this error:
java.lang.NoSuchMethodError: org.json.JSONObject.keySet()Ljava/util/Set;
at ee.ut.cs.Parser.accessLint(Parser.java:39)
I have tried cleaning the project to no awail.
I suspect I have an error in the src/plugin/parse-htmlraw/build.xml while creating the jar file but I'm not certain. I understand that this error is because the function does not exist at runtime, but the object is created which means that the class is there, just not that function. I decompiled the .class file in created jar and it has the necessary functions.
Code is available at https://github.com/jaansusi/WCAGgrader
Q: What is wrong with the build that produces this error?
The problem is that even if I put the necessary class files in the jar I create, they are not linked correctly and the class that's called in the jar can't locate functions inside the other classes. The class object JSONObject is created but the functions inside the JSONObject class can't be found.
If you do not find the problematic version, there is a possibility you get it (especially if you are using Spring) from the following dependency -
<artifactId>android-json</artifactId>
<groupId>com.vaadin.external.google</groupId>
excluding it worked for me,
An easy way of analyzing dependencies is the maven-helper plugin in Intellij, see here
Check for the version you have used.
There might be a case where 2 different versions are being used which in turn causes this error.
To their own maven local repository com\Google\code\gson\gson, see if there are two or more version about json, will have to do is to delete the old, and remember to look at any other place in the project is introduced into the old version of the dependence, if any, change the old version of the dependence to the new version is perfectly solved this problem
I'm working on a large legacy project that I'm trying to componentize, starting with SonarQube. I'm configuring a multi-module project in sonar-project.properties. This works fine. However, I have an issue precisely identifying source folders.
Unfortunately, our modules aren't neatly separated in the file system. The project is separated into many Eclipse projects, and several Eclipse projects together form one module. I can, of course, enumerate all the projects, but this is very cumbersome as there are a lot of them. Here's a (simplified) version of our directory structure:
projects/
moduleAsubmodule1/
src/
com/mycompany/moduleA/submodule1/
moduleAsubmodule2/
src/
com/mycompany/moduleA/submodule2/
moduleBsubmodule1/
src/
com/mycompany/moduleB/submodule1/
moduleBsubmodule2/
src/
com/mycompany/moduleB/submodule2/
moduleBsubmodule3/
src/
com/mycompany/moduleB/submodule3/
Imagine many more modules and submodules, where the project name is concatenated, but the package names are nicely divided, making it much easier to differentiate on those.
moduleA.sonar.projectBaseDir=.
moduleA.sonar.sources=projects/**/src/com/mycompany/moduleA/**/*
moduleA.sonar.test=projects/**/*.test/src/com/mycompany/moduleA/**/*
According to the documentation, this should be possible for exclusions. However, I get the following error message:
16:10:44 ERROR: Unable to execute Sonar
16:10:44 ERROR: Caused by: The folder 'projects/**/src/com/mycompany/mymodule/**/*' does not exist for 'XXX:XXX:mymodule' (base directory = D:\XxxSonar\.)
So I guess globs don't work for sources? If that's indeed the case, what can I do?
We use SonarQube 4.1.2.
I had the same issue, but I solved it by doing like this:
sonar.sources=.
sonar.inclusions=projects/*/src/**/*
The inclusions/exclusions properties support wildcards. Same for your tests:
sonar.test.inclusions=projects/*/*.test/src/**/*
Wildcards are not allowed when specifying "sonar.sources". Instead, you should play with the properties that allow to narrow your source and test files. See the documentation page on how to include or exclude files to narrow the focus.