Develop Reference

Detaching default functionality from gradle lifecycle tasks

Tl;dr
Is there a way to detach specific plugin behavior (such as checkstyle's check behavior) from existing gradle lifecycle tasks (gradle check, in this particular case)?
Longer version
In our current gradle Java project setup, we've included checkstyle as one of our plugins for static code checking. It currently runs as a part of Jenkins pipeline through gradle's build task. While this has mostly worked out for what we've needed - namely running our tests and making sure we're sticking to code standards - I've also noticed that we could make our feedback loop a little faster if we could run just the checkstyle's plugin's checks before build kicks in the tests.
To do so, as far as I understand, we'd have to create a custom task that runs only the checkstyle functions checkstyleMain and checkstyleTest and decouple the default checkstyle behavior from gradle's build lifecycle task. I've been looking through both gradle and the checkstyle plugin's docs, but quickly found I'm out of my depth.
Code:
plugins {
id "checkstyle"
}
checkstyle {
toolVersion "8.24"
configFile file("config/checkstyle/checkstyle.xml")
}
checkstyleMain {
source = "src/main/java"
}
checkstyleTest {
source = "src/test/java"
}
That is everything checkstyle related inside of build.gradle, the check task itself isn't customized.

In Buildship: how can I substitute a Gradle project with a built jar?

I would like to be a able to get Eclipse to ignore one Gradle project, and instead use a pre-built version of it.
Background
I have a project "parser" written in Scala, and a dozen others written in Java. The weakest link in my tool-set is Scala IDE. I use this plugin to edit & compile Scala code, but unfortunately it breaks the Java (JDT) tooling quite badly in mixed-language projects*.
Specifically: Call-hierarchy is missing results, searches crash and so on. Also Scala IDE appears to have lost funding and the issues sound fairly fundamental, so I'm not holding my breath for these issues to be fixed.
With Maven (m2e) I had a workaround I was quite happy with:
Build as a .jar put into my local .m2 repository:
cd parser; mvn install
In Eclipse, close the "parser" project
"Like magic", m2e simply picked up the most recent 'installed' .jar and used it in place of the closed project.
An awesome answer would be how to get Gradle to do that!
However all I wish for is any solution that meets these...
Requirements
That I can open Project parser when necessary (which is seldom),
to edit and build changes via the Gradle command-line.
I will close it when done.
Other projects use the built .jar from my local .m2 repo.
(It's fine if they always do so.)
The change must not affect others who don't use Eclipse
(ideally) the change can be used by other Eclipse users
Approaches
A similar question had this good answer by #lance-java with a number of general suggestions. I think I can rule out these ideas:
composite build support / multiple repos. Other team members wouldn't think it makes sense to build this project separately, as it is quite closely integrated with the others.
dependency substitution rules - doesn't appear to meet requirement 3.
Something along the lines of lance-java's idea #4 sounds viable. Paraphrasing...
"use the eclipse plugin [in conjunction with] Buildship, e.g. using the whenMerged hook to tweak the generated .classpath [of all the Java projects]."
UPDATE: [18 Apr]: I had hit a brick wall in this approach. Buildship was not putting the built .jar onto the runtime classpath. (UPDATE 2: Now resolved - see my answer.)
Questions
The main question: How can I structure a solution to this, that will actually work & avoid any major pitfalls?
Note that the project itself has a few dependencies, specifically:
dependencies {
compile 'org.scala-lang:scala-library:2.12.4'
compileOnly 'com.google.code.findbugs:jsr305:1.3.9'
antlr 'org.antlr:antlr4:4.5.3'
}
So a sub-question may be: How to pull these in into the other projects without duplicating the definition? (If that doesn't work automatically.)

So the solution was a bit involved. After adding 'maven-publish' to create the library, I then implemented the following to force Eclipse to use the prebuilt library:
subprojects {
// Additional configuration to manipulate the Eclipse classpaths
configurations {
parserSubstitution
}
dependencies {
parserSubstitution module("com.example:parser:${project.version}")
}
apply plugin: 'eclipse'
eclipse {
classpath {
plusConfigurations += [ configurations.pseLangSubstitution ]
file {
whenMerged { cp ->
// Get Gradle to add the depedency upon
// parser-xxx.jar via 'plusConfigurations' above.
// Then this here if we have a dependency on Project(':parser')
// - If so, remove it (completing the project -> jar substitution).
// - If not, remove the .jar dependency: it wasn't needed.
def usesParser = entries.removeAll {
it instanceof ProjectDependency && it.path.startsWith('/parser')
}
def parserJar =
cp.entries.find { it instanceof Library && it.path.contains('parser-') }
if (usesParser) {
// This trick stops Buildship deleting it from the runtime classpath
parserJar ?. entryAttributes ?. remove("gradle_used_by_scope")
} else {
cp.entries.remove { parserJar }
}
}
}
}
So there are 2 parts to this:
Using 'plusConfigurations' felt a bit round-about. I ended up doing this because I could not see how to construct class Library classpath entries directly. However it could well be that this is required to implement the 'transient dependencies' correctly anyway. (See the end of the question.)
The trick to stop Buildship removing the .jar from the runtime classpath (thus deviating from a Gradle command-line launch) was provided to me by a Gradle developer in this discussion.
Usage
The solution works just as I hoped. Every time some code in this library is modified, I execute the following task of mine on the command line (which also does some other code & resource generation steps, in addition to building the parser jar):
./gradlew generateEclipse
Then in Eclipse I press keyboard shortcuts for "Gradle -> Refresh Gradle Projects", Build.
And harmony is restored. :-)
Navigating to the (prebuilt) source of parser works.
If I need to edit the source, I can open the parser project and edit it. Scala-IDE still does a good job for this.
When I'm done I execute the command, close the project and my Java tools are happy.

In parser project
You shoud use the maven-publish plugin with the publishToMavenLocal task
apply plugin: 'maven-publish'
group = 'your.company'
version = '1.0.0'
publishing {
publications {
mavenJava(MavenPublication) {
from components.java
pom.withXml {
def root = asNode()
root.appendNode('name', 'Your parser project name')
root.appendNode('description', 'Your parser project description')
}
}
}
}
Everytime you make a modification, just change the version number if necessary and go with gradle publishToMavenLocal
In other java project using parser
Just use parser as a regular dependency :
repositories {
mavenLocal()
...
}
compile 'your.company:parser:1.0.0'
If my understanding of your situation is good, it should do the trick.

ClassNotFoundExceptions in Maven Surefire tests

While executing tests in Maven Surefire I see ClassNotFoundExceptions from time to time.
This really gives me a headache, since:
the missing classes vary. Only around 5 classes are affected, but which one it is varys from build to build. However, I see no unique similarities between these classes, which they wouldn't share with 20 other classes of the same kind.
These missing classes come from 2 different dependencies. These are managed by Maven, of course.
When a CNFE is raised I had a look at the class path (during runtime!) and it looks fine!
How I analysed the class path
I took the code of the "class path scanner" from Arno Haase:
public List<URL> getRootUrls () {
List<URL> result = new ArrayList<> ();
ClassLoader cl = Thread.currentThread().getContextClassLoader();
while (cl != null) {
if (cl instanceof URLClassLoader) {
URL[] urls = ((URLClassLoader) cl).getURLs();
result.addAll (Arrays.asList (urls));
}
cl = cl.getParent();
}
return result;
}
The list of URLs is quite short:
a few JRE libs
a "surefire booter jar"
The latter jar bundles all my Maven dependencies in its Manifest file, as described in the Surefire docs.
So I dug further and analysed the "Class-Path" attribute of the manifest. There I found the dependent jar listed, where the missing class should have come from.
When browsing through the jar's entries, I also found the missing class there. The fully qualified path also matches.
So in principle everything seems to be correct and in place.
Where should I continue to investigate now?

There are several things to check for problems like these.
Does this happen from command line or via CI build only? If using Jenkins or Hudson, is this a Maven project or a FreeStyle project with a Maven build step? If this is a Maven project, switch it to a FreeStyle project with a Maven build step, and that just may solve the issue. Stephen Connolly of the Maven team considers the Jenkins Maven build type evil.
Ensure there is only one version of each dependency and that related dependencies (Spring, ASM, Hibernate, etc.) have the same/compatible versions. Pay particular attention to artifacts where the group ID or artifact ID has changed, for example spring.jar vs. spring-core.jar. The old Tattletale plugin might be useful to get started.
Replace any dependencies ending in -all with their component parts. -all jars may contain every class needed to run the library - repackaged into the jar file where Maven's dependency resolution process can't get at them - instead of referencing them as dependencies. mockito-all, hamcrest-all, powermock-all, cglib are examples.
If using coverage tools (Jacoco, Clover) does the build work if you turn off the coverage? If yes, the tool may be introducing classpath jars that conflict with your app's. (Different version of CGLIB for example.) Run in debug mode and compare dependencies with/without coverage to identify the differences.
If using JUnit, make sure Maven surefire is using the right JUnit provider for your version of JUnit. Run the build in debug mode with -X (redirect output to a file if using command line). Grep the output for surefire-junit. You should find something like this:
[DEBUG] org.apache.maven.surefire:surefire-junit4:jar:2.16:test (selected for test)
Now, make sure the version of the provider matches the version of JUnit used. Check out the Maven docs for information on which provider to use and how to configure.

How do I specify different (PMD, Checkstyle, Findbugs) rulesets for test and production code using Gradle?

I am Building a Java project with Gradle. I would like to separate tasks:
I would like to use strict rules (i.e. PMD, Checkstyle, Findbugs) for production code
as well as more relaxed rules (i.e. allowing duplicate Strings and magic numbers) for tests .
I have done this previously with ant (quite simple), and even though I know I could just call the ant tasks from Gradle, I would rather use the respective plugins.
How do I go about this?
my current build script for PMD is as follows:
apply plugin: 'pmd'
pmd {
ignoreFailures = true
ruleSetFiles = files("$staticAnalysisCfgDir/pmd/pmdruleset.xml")
toolVersion = '5.1.3'
sourceSets = [sourceSets.main, sourceSets.test]
}

Instead of configuring rule sets on the pmd extension, configure them on the pmdMain and pmdTest tasks (same syntax). Likewise for Checkstyle and FindBugs.

Exclusions not working with Jacoco Gradle plugin

I'm working on setting up the Jacoco Gradle plugin for my project. The plugin is executing fine and generating the coverage report but it's not excluding packages that I would excluded.
Classes I would like to include are in com/xxx/ws/hn/** and classes I would like excluded are in com/xxx/ws/enterprise/**.
Here's my Gradle script:
apply plugin: "jacoco"
jacoco {
toolVersion = "0.6.2.201302030002"
reportsDir = file("$buildDir/reports/jacoco")
}
test {
jacoco{
excludes = ["com/xx/ws/enterprise/**/*"]
includes = ["com/xxx/ws/hn/**/*"]
append = false
}
}
jacocoTestReport {
dependsOn test
description = "Generate Jacoco coverage reports after running tests."
executionData = files('build/jacoco/test.exec')
reports {
xml.enabled true
html.enabled true
}
}
Am I missing anything here? I've tried various patterns of exclusions including a '.' delimiter for packages instead of a '/' but nothing seems to work. Any help would be greatly appreciated.

I have not verified this but when I read the JaCoCo documentation, it states that wildcards can be used but all examples only have ONE *, not two as your example shows.
http://www.eclemma.org/jacoco/trunk/doc/agent.html

To solidify (and verify) Joachim's answer, I came across this SO post and found that specifying canonical class names, classpath style (with slashes, not periods), with single wildcard characters actually worked well. For example, here is my exclude list that works in my build, as defined as a list in a separate gradle file:
def testExcl = [
'android/*',
'com/android/*',
'com/nativelibs4java/*',
'com/ochafik/*',
'com/fasterxml/*',
'com/esotericsoftware/*',
'com/google/*',
'com/lmax/*',
'com/sun/*',
'jdk/*',
'mockit/*',
'org/apache/*',
'org/bridj/*',
'org/gradle/*',
'org/hamcrest/*',
'org/junit/*',
'org/slf4j/*',
'sun/*',
'worker/*',
'*Test',
'*TestIntegration',
'*AllTests',
'*Suite'
]
I tried probably 10 or 12 other ways of formatting the class names, and this is the only way that actually worked.
I don't know why something like this that is so simple/fundamental is so ungoogleable and difficult to find a definitive answer on in the context of Gradle (i.e. outside JaCoCo's official documentation, where no documentation exists about Gradle integration, but Ant and Maven integration docs are there).
Hopefully this helps if others are flailing trying to get JaCoCo to exclude classes.
UPDATE:
This changed in Gradle 7 (I went from 5.6.1 to 7.1.1, so not sure about Gradle 6) and no longer works.
For Gradle 7 you need to replace / with .; here are my updated exclusions:
def testExcl = [
'android.*',
'com.android.*',
'com.nativelibs4java.*',
'com.ochafik.*',
'com.fasterxml.*',
'com.esotericsoftware.*',
'com.google.*',
'com.lmax.*',
'com.sun.*',
'jdk.*',
'mockit.*',
'org.apache.*',
'org.bridj.*',
'org.gcpud.common.*',
'org.gcpud.testutil.*',
'org.gradle.*',
'org.hamcrest.*',
'org.junit.*',
'org.slf4j.*',
'sun.*',
'worker.*',
'*Test',
'*Test$*',
'*TestIntegration',
'*AllTests',
'*Suite'
]
Also see the following SO post for other means of excluding classes from reporting (exclude from reporting, not coverage itself):
Filter JaCoCo coverage reports with Gradle