While executing tests in Maven Surefire I see ClassNotFoundExceptions from time to time.
This really gives me a headache, since:
the missing classes vary. Only around 5 classes are affected, but which one it is varys from build to build. However, I see no unique similarities between these classes, which they wouldn't share with 20 other classes of the same kind.
These missing classes come from 2 different dependencies. These are managed by Maven, of course.
When a CNFE is raised I had a look at the class path (during runtime!) and it looks fine!
How I analysed the class path
I took the code of the "class path scanner" from Arno Haase:
public List<URL> getRootUrls () {
List<URL> result = new ArrayList<> ();
ClassLoader cl = Thread.currentThread().getContextClassLoader();
while (cl != null) {
if (cl instanceof URLClassLoader) {
URL[] urls = ((URLClassLoader) cl).getURLs();
result.addAll (Arrays.asList (urls));
}
cl = cl.getParent();
}
return result;
}
The list of URLs is quite short:
a few JRE libs
a "surefire booter jar"
The latter jar bundles all my Maven dependencies in its Manifest file, as described in the Surefire docs.
So I dug further and analysed the "Class-Path" attribute of the manifest. There I found the dependent jar listed, where the missing class should have come from.
When browsing through the jar's entries, I also found the missing class there. The fully qualified path also matches.
So in principle everything seems to be correct and in place.
Where should I continue to investigate now?
There are several things to check for problems like these.
Does this happen from command line or via CI build only? If using Jenkins or Hudson, is this a Maven project or a FreeStyle project with a Maven build step? If this is a Maven project, switch it to a FreeStyle project with a Maven build step, and that just may solve the issue. Stephen Connolly of the Maven team considers the Jenkins Maven build type evil.
Ensure there is only one version of each dependency and that related dependencies (Spring, ASM, Hibernate, etc.) have the same/compatible versions. Pay particular attention to artifacts where the group ID or artifact ID has changed, for example spring.jar vs. spring-core.jar. The old Tattletale plugin might be useful to get started.
Replace any dependencies ending in -all with their component parts. -all jars may contain every class needed to run the library - repackaged into the jar file where Maven's dependency resolution process can't get at them - instead of referencing them as dependencies. mockito-all, hamcrest-all, powermock-all, cglib are examples.
If using coverage tools (Jacoco, Clover) does the build work if you turn off the coverage? If yes, the tool may be introducing classpath jars that conflict with your app's. (Different version of CGLIB for example.) Run in debug mode and compare dependencies with/without coverage to identify the differences.
If using JUnit, make sure Maven surefire is using the right JUnit provider for your version of JUnit. Run the build in debug mode with -X (redirect output to a file if using command line). Grep the output for surefire-junit. You should find something like this:
[DEBUG] org.apache.maven.surefire:surefire-junit4:jar:2.16:test (selected for test)
Now, make sure the version of the provider matches the version of JUnit used. Check out the Maven docs for information on which provider to use and how to configure.
Related
I have a huge project for which I am testing mutation testing with Pitest. The project is in an OSGi form and having all modules separated. I have this structure:
|-1.myProgramm-parent
|-pom.xml
|-2.myProgramm.module1
|-pom.xml
|-2.myProgramm.module1.Test
|-pom.xml
|-3.myProgramm.module2
|-pom.xml
|-3.myProgramm.module2.Test
|-pom.xml
... and so on.
Now I put into the pom.xml from my 1.myProgramm-parent all the Pitest configurations I need (taken from the official site of pitest.org). The targetClasses and targetTests are in the pom.xml of 2.myProgramm.module1.Test, which I need to use.
Pitest finds all 7 test classes to minion. And sends them. Then gathering for test description is also fine. Coverage generator Minion excited ok.
Then: created 0 mutation test units.
And a build failure is shown. No mutations found.
I tried already all the possible annotatons shown on pitest.org, like: targetClasses, targetTests and additionalClasspathElements.
How can I say that the testClasses are in this folder 2.myProgramm.module1.Test, where I am setting the targetClasses, targetTests in the pom.xml. BUT the normal javaClasses to be minioned are in this package: 2.myProgramm.module1
How I can tell, go out of your test-folder and get into the folder up?
I also gave the pure path to the folder with the normal javaClasses, but NO reaction.
Do you have an idea?
Ps. It is not my program. I didn't wrote it. I am just working on it, to test. I have already 11 other programs with Maven and Gradle. I get all to minion. But this is such a pain in the butt! ARG!
If you are working with multi-module projects, you will need to use the pitmp plugin (https://github.com/STAMP-project/pitmp-maven-plugin).
This is because PIT itself only mutates classes that are defined in the same module as the tests. In contrast, pitmp will execute the tests for all classes of the modules. More details are provided in the link above.
I try to work with JMeter 3.2 source but it's not compiling, It was missing mainly Bouncy Castle jars I added jars but still getting an error:
SMIMEAssertion.java include constructor SMIMESignedParser(BcDigestCalculatorProvider, MimeMultipart) which doesn't exists in JMeter's lib, Code failing:
s = new SMIMESignedParser(new BcDigestCalculatorProvider(), multipart);
s = new SMIMESignedParser(new BcDigestCalculatorProvider(), msg);
I added different jars of Bouncy Castle as latest bcprov-jdk15on-158.jar to older versions bcmail-jdk14-1.46.jar or others as bcpkix-jdk15on-1.55 or bcmail-jdk15-140.jar but still failed to compile
Does anyone knows the correct jars to add?
JMeter uses "ant" to build.
Based on https://github.com/apache/jmeter it looks like they use Travis to build and test (probably on Github).
Based on
before_script:
- test "x$RUN_CHECKSTYLE" != 'x' || ant -Djava.awt.headless=true download_jars install
(https://github.com/apache/jmeter/blob/trunk/.travis.yml)
it appears that you should run
ant download_jars
to get the jars you need. Locate them and add them to your build path.
(And this stuff is exactly why projects are typically migrating to Maven)
Edited in by other user after I wrote it:
Full explanation for building is here:
http://jmeter.apache.org/building.html
A PR of migration to maven is available and should be merged after 3.3 release.
A project runs on Google App Engine. The project has dependency that uses a class that can't be invoked on App Engine due to security constraints (it's not on the whitelist). My (very hacky) solution was to just copy a modified version of that class into my project (matching the original Class's name and package) that doesn't need the restricted class. This works on both dev and live, I assume because my source appears in the classpath before my external dependencies.
To make it a bit cleaner, I decided to put my modified version of that class into it's own project that can be packaged up in a jar and published for anyone else to use should they face this problem.
Here's my build.gradle:
// my jar that has 'fixed' version of Class.
compile files('path/to/my-hack-0.0.1.jar')
// dependency that includes class that won't run on appengine
compile 'org.elasticsearch:elasticsearch:1.4.4'
On my local dev server, this works fine, the code finds my hacked version of the class first at runtime. On live, for some unknown reason, the version in the elasticsearch dependency is loaded first.
I know having two versions of the same class in the classpath isn't ideal but I was hoping I could reliably force my version to be at the start of the classpath. Any ideas? Alternatively, is there a better way to solve this problem?
Not really sure if this is what people visiting this question were looking for, but this was what my problem and a solution that I reached at.
Jar A: contains class XYZ
Jar B: also contains class XYZ
My Project needs Jar B on the classpath before Jar A to be able to get compiled.
Problem is Gradle sorts the dependencies based on alphabetical order post resolving them which meant Jar B will be coming after Jar A in the generated classpath leading to error while compiling.
Solution:
Declare a custom configuration and patch the compileClasspath. This is how the relevant portion of build.gradle might look like.
configurations {
priority
sourceSets.main.compileClasspath = configurations.priority + sourceSets.main.compileClasspath
}
dependencies {
priority 'org.blah:JarB:2.3'
compile 'org.blah:JarA:2.4'
...
}
It's the app engine classloader I should have been investigating, not gradle...
App Engine allows you to customise the class loader JAR ordering with a little bit of xml in your appengine-web.xml. In my case:
<class-loader-config>
<priority-specifier filename="my-hack-0.0.1.jar"/>
</class-loader-config>
This places my-hack-0.0.1.jar as the first JAR file to be searched for classes, barring those in the directory war/WEB-INF/classes/.
...Thanks to a nudge in the right direction from #Danilo Tommasina :)
UPDATE 2020:
I just hit the same problem again and came across my own question... This time, live appengine was loading a different version of org.json than was being loaded in dev. Very frustrating and no amount of fiddling the build script would fix it. For future searchers, if you're getting this:
java.lang.NoSuchMethodError: org.json.JSONObject.keySet()Ljava/util/Set;
It's because it's loading an old org.json dependency from god-knows-where. I fixed it by adding this to my appengine-web.xml:
<class-loader-config>
<priority-specifier filename="json-20180130.jar"/>
</class-loader-config>
You'll also need a matching dependency in build.gradle if you don't already have one:
compile 'org.json:json:20180130'
According to gradle dependencies documentation, the order of dependencies defines the order in the classpath. So, we can simply put the libraries in the correct order in "dependencies".
But beware! here are two rules with higher priorities:
For a dynamic version, a 'higher' static version is preferred over a 'lower' version.
Modules declared by a module descriptor file (Ivy or POM file) are preferred over modules that have an artifact file only.
I'm the author of one of the Maven plugins (not Apache/Codehaus, completely indie). Sometimes I get support requests or test cases where I'd really need to debug the execution of my plugin with an existing pom.xml. Basically the test cases I get are sample/test project (pom.xml with src/main/resoures, src/main/java and so on).
What I need is a way to:
Load an existing pom.xml.
Find a specific execution of my plugin there (usually it's the only one).
Get an instance of MyMojo - fully initialized/condigured, with all the components and parameters corectly injected.
Execute MyMojo.
What's important is that test projects are separate projects, I don't want to copy them into the Maven module of my plugin.
I'd like to be able to do this without remote debugging.
By debugging I mean to be able to set and halt on breakpoints (also conditional), step in/out/over on the source code.
Ideally I'd like to be able to executeMyMojoFrom(new File("pom.xml")) - for instance in a JUnit test or a main method of some class. (I can supply groupId, artifactId etc. All other definitions should just be loaded from that pom.xml.)
How can I achieve this?
What I've tried so far:
Debug As... on pom.xml in Eclipse - does not work well enough (source code not found, breakpoint don't work as its not a Java project context)
Maven Embedder/Invoker solutions - spawn things in separate processes via CLI. Forget breakpoints, no debugging.
Remote debugging with mvnDebug and then remote debugging from Eclipse as suggested by Pascal Thivent here. This is so far the best option. However, remote debugging means starting mvnDebug separately, and there's also not guarantee that the JARs I have in Eclipse are exactly the same that mvnDebug is using. So there's a certain distance here.
maven-plugin-testing-harness - I actually thought this this will do the task. But first I was jumping through hoops for a few hours just to make it start. All of the important dependencies are "provided" so I first had to figure out the right combination of versions of these artifacts. And then - only to discover that AbstractMojoTestCase only works within the plugin module you want to test. Probably I was mistaken when I thought that maven-plugin-testing-harness was a testing harness for Maven plugins. It seems that it's a testing harness for the plugin from that plugin's module. Which is not illogical but does not help my case. I'd like to test my plugin in other modules.
So right now I've got the best results with the remote debugging solution. But what I'm looking for is really something like maven-plugin-testing-harness but not hardwired to the plugin module. Does anyone happen to have a hint, if such a method exists somewhere in Maven artifacts?
To be even more specific, I'd like to write something like:
public void testSomething()
throws Exception
{
File pom = getTestFile( "pom.xml" );
assertNotNull( pom );
assertTrue( pom.exists() );
MyMojo myMojo = (MyMojo) lookupMojo( "myGroupId", "myArtifactid", ...,
"myGoal", pom );
assertNotNull( myMojo );
myMojo.execute();
...
}
Compare it to the MyMojoTest here - it's almost there. Should just not be hardwired into the mymojo Maven module (as it is in maven-plugin-testing-harness).
Update
Few answers to the questions in comments:
You mean you don't want such a test class, i.e MyMojoTest to reside inside the same project as the MyMojo, i.e your plugin project? Why is that?
Exactly. I want to debug the plugin execution in an existing Maven project, I don't want to move that project to my plugin project first to be able to run a test. I want to be able to test/debug an existing project. Ideally, I'd just need to add my-maven-plugin-testing dependency and subclass MyMojoTest in the project's src/test/jaca. This would be a good instrument to debug executions. Dragging the target project into my Mojo project ist just too much overhead - and mostly these aren't really the test cases I want to keep long-term. I hope, this answers, why.
Anyway, it's merely a convention to keep the project-to-test/pom.xml inside the src/test/resources of your plugin module, not a rule...
My problem is not the location of the pom.xml of the project-to-test, that is easily configurable. My difficulty is that maven-plugin-testing-harness is is somehow hardcoded to be in the Mojo's project. It uses the pom.xml of the Mojo, looks for other special files/descriptors in the containing project. So I somehow can't use it in a non-Mojo project, or can I? This is my question.
And I'm not sure why Debug as... didn't help you...
Not sure either, but (1) breakpoints did not work and (2) the source code was not "attached" for some reason.
If the Debug as didn't work for you as well as it should, you can try to use the mojo-executor with a bit of work.
https://github.com/TimMoore/mojo-executor
This is how you would execute the copy-dependencies goal of the Maven Dependency Plugin programmatically:
executeMojo(
plugin(
groupId("org.apache.maven.plugins"),
artifactId("maven-dependency-plugin"),
version("2.0")
),
goal("copy-dependencies"),
configuration(
element(name("outputDirectory"), "${project.build.directory}/foo")
),
executionEnvironment(
mavenProject,
mavenSession,
pluginManager
)
);
The project, session, and pluginManager variables should be injected via the normal Mojo injection. Yes, that means this should be executed from the context of another maven plugin. Now that I think about it, whether this would help you in any way is still a question because this still relies on injection of such components by the underlying plexus container.
My original idea was though to have you build a maven plugin that would invoke your jaxb2 plugin thru the mojo-executor like above, then serialize the mavenProject, mavenSession, pluginManager, i.e, all the plexus injected components and then use those objects to invoke your jaxb2 plugin in future from a standalone class without the plugin that you built.
I have a Java application built with Maven with a lot of dependencies. When performing my test cases they sometimes pass fine, sometimes they fail because of some incompatible class combinations. So it seems to that there must be some classes twice in classpath which are taken randomly. The one is fine the other not.
How can I find out which classes / jars are incompatible in my classpath?
What is the right approach using Maven not to fall in that compatibility-traps?
I think a better solution would be to use the maven-duplicate-finder-plugin.
Note: The new version is the duplicate-finder-maven-plugin.
You can try using this tool Tattletale.
You can detect duplicate classfile definitions in the classpath or module path using ClassGraph (disclaimer, I am the author of ClassGraph):
for (Entry<String, ResourceList> dup :
new ClassGraph().scan().getAllResources().classFilesOnly().findDuplicatePaths()) {
System.out.println(dup.getKey()); // Classfile path
for (Resource res : dup.getValue()) {
System.out.println(" -> " + res.getURI()); // Resource URI, showing classpath element
}
}
There is a plugin in eclipse to check for duplicate classes in the build path (ClasspathChecker http://classpathchecker.free.fr/)
This problem is basically an application of the more general problem to "somehow scan the classpath (CP) and collect all class files and other resources", and then find duplicates in that...
There are a number of existing libraries for CP scanning (and it's not trivial to do this right in all environments, especially since the application class loader in Java 9 is no longer an URLClassLoader), notably Classgraph, using which it's relatively trivial to do this.
PS: For Java versions <9, JHades (jhades.github.io) is nice (but NOK on Java 9/10/11).
This is a another simple Open Source Duplicate Classpath Finder tool - Classpath Inspector
which gives pretty decent report of duplicate classes in the classpath.
You can use the maven dependency:tree to see the maven hierarchy of your project and
maven exclusion to exclude the jars you don't want