Develop Reference

Create production configuration with additional compile dependency in Gradle

I am trying to define build script for my production versions.
Below is project structure, all projects are java plugin.
wrapper (parent)
|--frontend (child)
| |--src
| | |--js (raw ES6 modules)
| | |--sass (raw)
| |--build
| |--lib
| | |--production-front.jar
| |--dist
| |--js (bundled)
| |--css (compiled production)
|--backend (child) (spring boot)
|--build
|--lib
|--RELEASE.jar
Now what happens here is that by default (sourceSets.main.resources.srcDirs) of backend is linked directly to:
raw :frontent/src/js
generated :frontent/build/dist/css.
This way when you run it, by default it will be in dev mode. Here it means that it will:
use generated scss->css files (which are resource so for example if you run background gulp-sass that compiles it every time you change scss, css will update and boom, fast dev cycle).
use raw JS which is compiled directly in browser (JSPM, SystemJS, Babel) - so you only need to edit :frontent/src/js and refresh page.
Okay, so while dev is love, I also need to compile for production. Project structure mentioned above also shows where :frontend generates production-front.jar.
Here is default java build tree with my notes.
EDIT
I need to make dependency that will compile production-front.jar into RELEASE.jar and ommit mentioned attached resources.
Note that I need to only ommit those resources, not any others in main.resources.srcDirs.
What is proper way to solve this (one that is not making tasks that e.g remove dev resources from .jar and then throw in other production-front.jar instead)? I cannot grasp how would i make multiple sourceSets or configurations that work here.

After very in-depth Gradle learning for over past week (this topic was created because I was close to my demise) I've finally found very satisfying solution.
I'd like to share what were my goals and final solution:
Have smallest possible build.gradle
Have possibility to develop with build --continuous
Make it so that eclipse plugin (probably also other IDEs) can perfectly mirror pure command line Gradle development with their own build system.
This is a multi-project with one of them being Spring Boot application with DevTools.
wrapper (parent)
|--frontend (child)
| |--src-client
| | |--static
| | | |--img (images)
| | | \--js (raw ES6 modules)
| | \--sass (raw, note not in static folder)
| \--build
| |--lib
| | \--front.jar
| |--dist
| | |--js (bundled, minimized)
| | \--css (compiled production, minimized)
| \--dev
| \--css (compiled dev, compact readable format)
\--backend (child) (spring boot)
|--src
| |--main/java
| |--main/resources
| \--test/java
\--build
\--lib
\--application.jar
As I described, goals were to:
Have bootRun run with raw sources for js and compiled css, but also with every resource that is included in backend's main.
Have bootJar compile into production with dependency on compiled front.jar instead of everything that is used in dev (previous point).
I've used combination of configurations, sourceSets and bootRun properties (+ a lot of time).
Here are files (stripped down):
wrapper/build.gradle
wrapper.gradleVersion '5.0-milestone-1'
allprojects {
apply plugin: 'eclipse'
}
wrapper/front/build.gradle
plugins {
id 'java' // possibly use java-base or just custom zip task, since client doesn't actually compile java
}
jar {
dependsOn buildProduction // task that compiles my stuff into build/dist
baseName 'front'
classifier 'SNAPSHOT-' + new Date().format('yyyyMMddHHmmss')
from(buildDir.absolutePath + '/dist') {
into 'static'
}
}
// Note there is a lot of other tasks here that actually compile my stuff, like gulp-sass and JSPM bundling with babel transpiler.
wrapper/backend/build.gradle
buildscript {
repositories {
mavenCentral()
maven { url 'https://repo.spring.io/milestone' }
}
dependencies {
classpath 'org.springframework.boot:spring-boot-gradle-plugin:2.1.0.RC1'
}
}
apply plugin: 'java'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'
sourceCompatibility = 10 // eclipse has (or had) problems with Java 11
targetCompatibility = 10
sourceSets {
// 'java' plugin adds default main sourceSet
dev {
resources.srcDirs = [
project(':front').projectDir.absolutePath + '/src-client',
project(':front').buildDir.absolutePath + '/dev'
]
}
}
bootJar {
baseName 'application'
classifier 'SNAPSHOT-' + new Date().format('yyyyMMddHHmmssSSS')
// I used bootArchives since it was already there and my stuff fits description, you can also define your own configuration and extend runtime one.
classpath configurations.bootArchives
}
bootRun {
sourceResources sourceSets.dev // I make bootRun (dev) use dev sourceSet
}
dependencies {
runtime 'org.springframework.boot:spring-boot-devtools'
// Since bootArchives configuration is used only by bootJar (not bootRun), this will be only in final fat .jar
bootArchives project(':front')
...
}
repositories {
mavenCentral()
maven { url 'https://repo.spring.io/milestone' }
}
Some link that helped:
http://mrhaki.blogspot.com/2014/09/gradle-goodness-adding-dependencies.html
Note that client's source folder is called src-client: This is key for making "perfect mirror" in eclipse. If we would name it src which is already being used by main in backend eclipse would vomit with name clash (which probably can be fixed by configurating eclipse plugin, but then again - we wanted to keep it simple, without IDE configurations).
As final result we get very nice command line development with gradle's continuous building and also EXACTLY mirrored behaviour in eclipse which defaults its configuration to work in the same way (but use eclipse's builder instead of gradle's). We also get nice linked folders in backend project to sources used from front. Same would probably apply to IDEA :).

Building with Intellij 2017.2 /out directory duplicates files in /build directory

After updating to Intellij 2017.2, building my project creates an /out directory that contains generated source files and resource files. These files duplicate files that are already contained in /build and result in duplicate class compiler errors for the generated classes. Any ideas on a fix I need in Gradle or IntelliJ?

IntelliJ IDEA is no longer sharing the output with Gradle, please see this ticket for details.
You can either override it via the following configuration:
allprojects {
apply plugin: 'idea'
idea {
module {
outputDir file('build/classes/main')
testOutputDir file('build/classes/test')
}
}
if(project.convention.findPlugin(JavaPluginConvention)) {
// Change the output directory for the main and test source sets back to the old path
sourceSets.main.output.classesDir = new File(buildDir, "classes/main")
sourceSets.test.output.classesDir = new File(buildDir, "classes/test")
}
}
or delegate the build to Gradle: File | Settings | Build, Execution, Deployment | Build Tools | Gradle | Runner => Delegate IDE build/run actions to gradle.

File | Project Structure | Project Settings | Modules | Paths tab | Compiler output
Select 'Inherit project compile output path' to continue using /build for build artifacts

Here is my understanding:
Basically, this is a work-around for an incompatibility issue between
Gradle build path and IDEA output path.
the issue is - https://github.com/gradle/gradle/issues/2315
the solution is - keep these two directories seperate, therefore you have two (out/ and build/) https://youtrack.jetbrains.com/issue/IDEA-189063

Gradle multi project, how to deal with different rootDirs

My company is currently switching from Ant to Gradle for its Java projects, but I am stuck with a nice clean setup. Let's say I work for a company that builds websites for clients which generally use the same root libraries (core project), but have some specific code, which is put in the sub-project. Per client, we build a new sub-project, that depends on the core project. Number of clients will increase in the future.
Currently we have three projects:
A core project. This should run individually, as we want to be able to do the unit testing seperately for this.
Two sub-projects that depend on the core project, but have some own projects that are specific to the project.
I was sucessful in converting the whole ant build into a gradle build for the core project. My idea is to have all functionality and project structure in the core, and only the extra for what is actually needed in the sub-projects.
Here is a short sample of our folder structure:
-- core
- build.gradle
- settings.gradle
-- repository (our external jars used)
-- Implementation
-- source_code
-- all the core project folders
-- Projects
-- Client A
- build.gradle
- settings.gradle
-- more project specific folders
-- Client B
- build.gradle
- settings.gradle
-- more project specific folders
I use the $rootDir variable a lot. A fraction of the core's settings.gradle looks as such:
project(':CoreProjectA').projectDir = new File(rootDir, 'Implementation/Source_code/Core/coreA')
project(':CoreProjectB').projectDir = new File(rootDir, 'Implementation/Source_code/Core/CoreB')
But with many more. Also, in the core build.gradle, I refer to our repository as such:
repositories {
//All sub-projects will now refer to the same 'libs' directory
flatDir {
dirs "$rootDir/repository/lib/jaxb"
//many more dirs here
}
}
Now this all works great when I do a gradle build from the core project.
I was planning to put the next piece of code in every client's subproject build.gradle:
apply from: '../../../build.gradle'
When I run a gradle build from Client A folder, my rootDir obviously has changed, and now, all my paths cannot be found anywhere.
Is there any way to set this up in a nice clean way? So that every future sub project added can always use the same structure? Or will I have to give each sub-project its own build.gradle and settings.gradle entirely?
I know the last option could work, but it is a lot of overhead, and just doesn't seem nice and clean to me at all..
Thanks in advance!

I recently worked on a similar configuration, so let me explain me how I build the Gradle infrastructure. Since you mentioned a lot of requirements, I hope that I'll miss any of them and you can apply my scheme to your problem.
General
We actually use build systems (like Gradle) to let them take care of any dependencies (projects, modules, tasks). So why should we define a depedency in something like a filesystem hierarchy, if we can simply define it in Gradle?
I would avoid using paths as much as possible (convention over configuration) and try to stick to Gradle projects for both the build scripts and the dependencies.
Also, if you define dependencies in your core gradle.build, you should just call this gradle file, even if you only want to build a subproject. Your apply from: '../../../build.gradle' destroys the whole Gradle logic. Instead you should use something like gradle :sub1:build to only build the first subproject.
First approach (with core as root project)
Filesystem structure:
core/
build.gradle
settings.gradle
src/
...
sub1/
src/
...
build.gradle [optional]
sub2/
src/
...
build.gradle [optional]
Core settings.gradle:
include 'sub1'
include 'sub2'
Core build.gradle:
allprojects {
apply plugin: 'java'
repositories {
// define repos for all projects
}
}
subprojects {
dependencies {
// let subprojects depend on core
compile rootProject
}
}
project(':sub1') {
// define anything you want (e.g. dependencies) just for this subproject
// alternative: use build.gradle in subproject folder
}
Second approach (all projects independent)
Filesystem structure:
core/
src/
...
build.gradle [optional]
sub1/
src/
...
build.gradle [optional]
sub2/
src/
...
build.gradle [optional]
build.gradle
settings.gradle
Root settings.gradle:
include 'core'
include 'sub1'
include 'sub2'
Root 'build.gradle'
subprojects {
apply plugin: 'java'
repositories {
// define repos for all projects
}
}
configure(subprojects.findAll {it.name != 'core'}) {
dependencies {
// Add dependency on core for sub1 and sub2
compile project(':core')
}
}
project(':sub1') {
// define anything you want (e.g. dependencies) just for this subproject
// alternative: use build.gradle in subproject folder
}
This approach provides great flexibility, since every dependency logic is handled by Gradle and you'll never have to copy anything to another position. Simply change the dependency and you are fine.
Sources
Gradle Tutorial on Multi-project Builds
Question in Gradle Forum

It looks like you have extra settings.gradle inside subprojects. That makes Gradle think the sub-project is a standalone one. If you remove settings.gradle from subprojects, Gradle will look for it up the filesystem hierarchy, will find one in core project, will create correct multimodule project and all the paths should work properly.
So, just remove extra settings.gradle files and your build will work fine. Having build.gradle in subprojects is perfectly fine.

I want to thank Nikita and especially lukegv for his detailed answer, but I went with a different approach ultimately.
I didn't want to have one main gradle build and extend it each time we have a new project. Also, I wanted to keep it simple and logical for my colleagues if they want to create a build for one of the projects.
Therefor, I kept the structure I had as described above. In the root gradle.settings but changed the
project(':CoreProjectA').projectDir = new File(rootDir, 'Implementation/Source_code/Core/coreA')
project(':CoreProjectB').projectDir = new File(rootDir, 'Implementation/Source_code/Core/CoreB')
into:
project(':CoreProjectA').projectDir = new File(customRootDir, 'Implementation/Source_code/Core/coreA')
project(':CoreProjectB').projectDir = new File(customRootDir, 'Implementation/Source_code/Core/CoreB')
In each project I have a properties.gradle with the customRootDir filled in (relatively to the projects directory).
It all works like a charm. I can go into any project folder and produce builds, while using the functionality of the root's build.gradle.

Gradle- Multi-project with outside projects?

Is it not possible in a Gradle multi-project setup to use external dependencies outside the main project folder?
Like in the settings.gradle file, can I not have something like
include 'C:\some\path\to\dependent\project\ChildA','ChildB'
or do I have to always include the dependent projects in the parent project folder?

Assuming the following filesystem hierarchy:
|
\ workspace
|
\
MyProject
|
\
DependencyA
Add the following into your settings.gradle in MyProject:
include '..:DependencyA'
and inside your inner build.gradle of MyProject
dependencies {
compile project(':..:DependencyA')
}
Repeat for as many projects as you have that depend on DependencyA.

Build jar and put in resources that are built into another jar with gradle

So I have two gradle projects: P1 and P2. P2 does not depend on P1 to build, however, during the build phase of P2, I want to build P1, add it to my resources folder and then add that folder to P2's jar.
I am still new to the whole gradle thing, so I am looking for an example build.gradle that would do something like this.
Thanks!
EDIT: It should be noted that P1 and P2 are both modules within the same project.
My project structure looks like this:
Root Project
Root Project
|
|__P1
| |
| |__build.grdale
|
|__P2
| |
| |__build.gradle
|
|__settings.gradle
My settings.gradle looks like this:
include 'P1', 'P2'

So I ended up finding a similar question here: Gradle: how to copy subproject jar to another subproject when task is run?
The build gradle should end up looking something like this:
task fix(dependsOn: ':P1:jar', type: Copy) {
from tasks.getByPath(':P1:jar')
into 'path/to/resources'
}
build.dependsOn fix