Develop Reference

gradle: compileJava? Found module. compileTestJava? Module foo not found?

I'm building a project with quite a few sub projects. Each one is a module.
With one of those subprojects in particular, I'm getting:
Task :foo:compileTestJava FAILED
error: module not found: foo
1 error
Note that both compileJava and classes (and an empty processResources if we must be pedantic) both completed successfully.
The build.gradle for this subproject is trivial:
ext.someVar = 'someValue'
dependencies {
compile project(':some.other.subproject.used.in.lots.of.other.places1')
compile project(':some.other.subproject.used.in.lots.of.other.places2')
}
That's it. Yes, these subprojects have been used together in some of those lots.of.other.places.
The main project has a highly non-trival build.gradle. It has an extensive subprojects {}, much of which is a test {}. This section of the build works just fine for the other 10+ subprojects. What it does not have is anything that treats individual subprojects differently. It does not have a custom compileTest, and ext.someVar is used in path construction, not conditions.
I get the same output when I run gradle (5.2.1) either from within IDEA or from the command line.
gradlew --console verbose just made the output prettier.
gradlew -d added quite a bit of noise, but no new signal:
I've run gradlew clean compileTestJava (many times). No change.
I've manually deleted the .gradle folder, then run. No change.
I ran with -s, and it gave me this:
* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':foo:compileTestJava'.
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$2.accept(ExecuteActionsTaskExecuter.java:121)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$2.accept(ExecuteActionsTaskExecuter.java:117)
at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:184)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:110)
at org.gradle.api.internal.tasks.execution.ResolveIncrementalChangesTaskExecuter.execute(ResolveIncrementalChangesTaskExecuter.java:84)
at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:91)
at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionStateTaskExecuter.execute(ResolveBeforeExecutionStateTaskExecuter.java:74)
at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:109)
at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionOutputsTaskExecuter.execute(ResolveBeforeExecutionOutputsTaskExecuter.java:67)
at org.gradle.api.internal.tasks.execution.ResolveAfterPreviousExecutionStateTaskExecuter.execute(ResolveAfterPreviousExecutionStateTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:93)
at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:45)
at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:94)
What could have gone wrong here? Where else can I look for a cause?

How to make gradle download dependencies without actually building things

On a new environment gradle build takes quite a while because all dependencies have to be downloaded.
Is there a way to only download dependencies in order to speed up the following build?
That way we could for example already prefill a CI build environment.

Edit: Updated for Gradle 6+.
Some notes:
This new approach downloads jars into a folder, and then deletes the folder. So the result of having the jars in the Gradle cache is a side-effect.
It currently uses jars configured for the main source-set but could be generalized.
Even though it is neither efficient nor elegant, it can be useful if you actually want the jars (and transitive dependencies): simply comment-out the deletion of the runtime folder.
This solution can be handy when you want the jars (and transitive dependencies), as you simply have to comment-out deleting the folder.
Consider this build.gradle (as an arbitrary, concrete example):
apply plugin: 'java'
dependencies {
implementation 'org.apache.commons:commons-io:1.3.2'
implementation 'org.kie.modules:org-apache-commons-lang3:6.2.0.Beta2'
}
repositories {
jcenter()
}
task getDeps(type: Copy) {
from sourceSets.main.runtimeClasspath
into 'runtime/'
doFirst {
ant.delete(dir: 'runtime')
ant.mkdir(dir: 'runtime')
}
doLast {
ant.delete(dir: 'runtime')
}
}
Example run:
$ find /Users/measter/.gradle/caches -name "commons-io*1.3.2.jar"
$ gradle getDeps
$ find /Users/measter/.gradle/caches -name "commons-io*1.3.2.jar"
/Users/measter/.gradle/caches/modules-2/files-2.1/commons-io/commons-io/1.3.2/[snip]/commons-io-1.3.2.jar

I've found ./gradlew dependencies (as suggested by this user) to be very handy for Docker builds.

You can create a custom task that resolves all the configurations( in doing so, it will also download the dependencies without building the project)
task downloadDependencies {
doLast {
configurations.findAll{it.canBeResolved}.each{it.resolve()}
}
}
Run command ./gradlew downloadDependencies

My answer will favor the gradle plugins and built-in tasks.
I would use "gradle assemble" in the command-line.
It is a minor version of "gradle build".
This way, you may reduce the time of your preparations before running or building anything.
Check the link bellow for the documentation:
https://docs.gradle.org/current/userguide/java_plugin.html#lifecycle_tasks
In general, what is my recipe when I clone a new repository:
-gradle assemble
-do some coding
-gradle run (and basically test until done)
-gradle build (to make distributable files)
note: this last step may have adicional configurations for .jar files as outputs (depends on you).

JAR built by Gradle can't find Gson

The problem is solved, read the last section.
The "background"
I'm making a ToDoApp just for a side project, as an excuse to learn Gradle. The initial build worked fine and the JAR worked as expected.[a]. But that version had no external dependencies; it was self-contained.
I was encouraged to add persistence to the project and I decided to use the GSON library (version 2.6.2) to work with JSON. I use IntelliJ IDEA (2016.1.2) to write code.
Now, I added the gson-2.6.2 artifact via the Maven Repository option in IntelliJ (in Project Structure). While writing the code, I added the library to the classpath of the project when prompted by IntelliJ. The code compiles and works fine when I run it in my IDE. Here's a sample run:
========ToDo App========
The following commands are recognised:
► help
Display this help message.
► add <name>
Add a new Todo task with the given name and also displays its corresponding ID.
► get <id>
Displays the task with the given id.
► mark <id>
Toggles the given task as completed or incomplete.
► print
Displays all tasks in order of their creation.
► update <id> <new text>
Updates the item with the given id to store the new text.
► del <id>
Deletes the task with the given id.
► exit
Exit the program.
The tasks are :
>> add Get the Gradle build to work.
New item added with id = 1
>> add Push the stable system to GitHub and wait for Travis to give the green signal.
New item added with id = 2
>> add Proceed
New item added with id = 3
>> print
The tasks are :
• Get the Gradle build to work. [ID: 1 Completed: false]
• Push the stable system to GitHub and wait for Travis to give the green signal. [ID: 2 Completed: false]
• Proceed [ID: 3 Completed: false]
>> exit
It works fine. The JSON data gets saved as I expect:
{"currentId":3,"toDos":{"1":{"id":1,"name":" Get the Gradle build to work. ","completed":false},"2":{"id":2,"name":" Push the stable system to GitHub and wait for Travis to give the green signal.","completed":false},"3":{"id":3,"name":" Proceed","completed":false}}}
When I rerun the code, the data is read back properly. I'm very happy with my code.
The "situation"
Here's my build.gradle:
group 'ml.cristatus'
version = '0.2'
apply plugin: 'java'
sourceCompatibility = 1.7
repositories {
mavenCentral()
}
dependencies {
compile 'com.google.code.gson:gson:2.6.2'
}
jar {
manifest {
attributes 'Main-Class': 'ml.cristatus.todo.ToDoApp'
}
}
When I run gradle build on my Ubuntu 16.04 terminal, the JAR file is built with the following output:
:compileJava
warning: [options] bootstrap class path not set in conjunction with -source 1.7
1 warning
:processResources UP-TO-DATE
:classes
:jar
:assemble
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE
:check UP-TO-DATE
:build
BUILD SUCCESSFUL
Total time: 7.036 secs
This build could be faster, please consider using the Gradle Daemon: https://docs.gradle.org/2.13/userguide/gradle_daemon.html
So I somehow managed to make the thing compile. But when I try to run it, using java -jar build/libs/ToDoApp-0.2.jar, the code doesn't work:
========ToDo App========
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/gson/Gson
at ml.cristatus.todo.repository.ToDoRepositoryWithJSON.<init>(ToDoRepositoryWithJSON.java:25)
at ml.cristatus.todo.ToDoApp.REPL(ToDoApp.java:38)
at ml.cristatus.todo.ToDoApp.main(ToDoApp.java:17)
Caused by: java.lang.ClassNotFoundException: com.google.gson.Gson
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 3 more
I'm probably making some newbie mistake, but I can't seem to be able to put my finger on it. What am I doing wrong? It is also interesting to note that the code compiles yet it can't find the gson artifact. I'm guessing it has something to do with the classpath? I don't know. I'm not sure.
Please help me in this regard.
Solution
I just added this line to the jar {} block:
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
Courtesy of this tutorial.
[a]Here's the version 0.1 binary.

You have to add jar file with gson jar to classpath. When you are starting application ("java -jar build/libs/ToDoApp-0.2.jar"). There is multiple ways it can be achiveved.
One possible way is:
Add task into your gradle, which copy dependencies (gson.jar in your case) into "lib" directory. And when you are starting your application, add it to classpath. For example in this way "java -cp build/lib/gson.jar -jar build/libs/ToDoApp-0.2.jar"
Maybe better for you will be:
Add dependencies into your manifest. It is discussed in this answer how to copy the dependencies libraries JARs in gradle
It should work for you.
Next Option is:
Create an "uberjar" it means add all dependencies into one big jar file. Personally I don't like it. But it will work in your case. How to create uberjar in gradle is here discussed here: Building a uberjar with Gradle

Gradle builds, then ignores sub-project

I have a gradle project in big-project/ and a sub-project in big-project/lib/.
In big-project/'s big.project.Main class I import big.project.lib.Utils, which is defined in the big-project/lib/ sub-project.
When I try to run gradle build, this is what happens:
:lib:
compileJava UP-TO-DATE
:lib:processResources UP-TO-DATE
:lib:classes UP-TO-DATE
:lib:jar
:compileJava
big-project/src/main/java/big/project/Main.java:3: error: package big.project.lib not exist
import big.project.lib.Utils;
^
1 error
:compileJava FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':compileJava'.
> Compilation failed; see the compiler error output for details.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 6.243 secs
For some reason, gradle compiles the code of the subproject, and then decides to ignore it. How do I fix that?
These are my gradle files:
big-project/settings.gradle:
include 'lib'
big-project/build.gradle:
evaluationDependsOnChildren()
allprojects {
apply plugin: 'java'
sourceCompatibility = '1.8'
version = '1.0'
group = 'big.project'
}
apply plugin: 'war'
dependencies {
compile project(':lib')
}
big-project/lib/build.gradle is empty
I did study the gradle manual: Chapter 56. Multi-project Builds, but nothing useful came from this.

TL;DR: My big-project/lib/src folder wasn't set up correctly (source code was in the wrong sub directory). As this answer states, Java sources need to be in src/main/java.
Thanks to comments on the question I was able to look in the right place for a solution. As suggested, I looked at a working sample project. As I was using Eclipse, I generated a new Gradle project based on the flat-java-multiproject sample. This enabled me to eliminate my build.gradle files as the source of the error.
Next, I decided to inspect the build/ artifacts of the lib/ project and discovered that the .jar file did not contain any .class files. That explains the compiler error I reported in the question. This caused my to see if all .java sources were in the right place, and as mentioned above, they weren't. Fixing this, fixed the compiler error.
The whole crux was that while I was writing my code in eclipse, all references to the lib project's classes seemed to be found.

Incremental Gradle Build not saying UP-TO-DATE

Gradle, being an incremental build system, is supposed to detect when no changes to source or outputs have been made, and skip tasks when appropriate to save on build time.
However, in my build, subsequent executions of a gradle task, with no changes in between, this incremental feature is not working. compileJava, jar, etc are executing with every build rather than only when changes have been made.
Our build is pretty complex (switching to gradle from a very old, very messy ant build), so I'm just going to show a small snippet:
buildDir = 'build-server'
jar {
sourceSets.main.java.srcDirs = ['src',
'../otherProject/src']
sourceSets.main.java {
include 'com/ourCompany/pkgone/allocation/**'
include 'com/ourCompany/pkgone/authenticationmngr/**'
...
//Excludes from all VOBs
exclude 'com/ourCompany/pkgtwo/polling/**'
}
sourceSets.main.resources.srcDirs = ['cotsConfig/ejbconfig']
sourceSets.main.resources {
include 'META-INF/**'
}
}
dependencies {
compile project(':common')
}
Running gradle jar on this project twice in a row results in the following output:
P:\Project>gradlew jar
:common:compileJava
:common:processResources UP-TO-DATE
:common:classes
:common:jar
:clnt:compileJava
:clnt:processResources UP-TO-DATE
:clnt:classes
:clnt:jar
BUILD SUCCESSFUL
Total time: 1 mins 46.802 secs
P:\Project>gradlew jar
:common:compileJava
:common:processResources UP-TO-DATE
:common:classes
:common:jar
:clnt:compileJava
:clnt:processResources UP-TO-DATE
:clnt:classes
:clnt:jar
BUILD SUCCESSFUL
My question is, what might I be doing that would prevent the up-to-date detection to not work properly? Could it be related to my complicated build path?

When you run with --info, Gradle will tell you why the task isn't up-to-date.

Two things you need to do
Check if your gradle version is above 2.1
There is a way setting incremental build:
tasks.withType(JavaCompile) {
options.incremental = true // one flag, and things will get MUCH faster
}
Reference is here: https://blog.gradle.org/incremental-compiler-avoidance