I am trying to define build script for my production versions.
Below is project structure, all projects are java plugin.
wrapper (parent)
|--frontend (child)
| |--src
| | |--js (raw ES6 modules)
| | |--sass (raw)
| |--build
| |--lib
| | |--production-front.jar
| |--dist
| |--js (bundled)
| |--css (compiled production)
|--backend (child) (spring boot)
|--build
|--lib
|--RELEASE.jar
Now what happens here is that by default (sourceSets.main.resources.srcDirs) of backend is linked directly to:
raw :frontent/src/js
generated :frontent/build/dist/css.
This way when you run it, by default it will be in dev mode. Here it means that it will:
use generated scss->css files (which are resource so for example if you run background gulp-sass that compiles it every time you change scss, css will update and boom, fast dev cycle).
use raw JS which is compiled directly in browser (JSPM, SystemJS, Babel) - so you only need to edit :frontent/src/js and refresh page.
Okay, so while dev is love, I also need to compile for production. Project structure mentioned above also shows where :frontend generates production-front.jar.
Here is default java build tree with my notes.
EDIT
I need to make dependency that will compile production-front.jar into RELEASE.jar and ommit mentioned attached resources.
Note that I need to only ommit those resources, not any others in main.resources.srcDirs.
What is proper way to solve this (one that is not making tasks that e.g remove dev resources from .jar and then throw in other production-front.jar instead)? I cannot grasp how would i make multiple sourceSets or configurations that work here.
After very in-depth Gradle learning for over past week (this topic was created because I was close to my demise) I've finally found very satisfying solution.
I'd like to share what were my goals and final solution:
Have smallest possible build.gradle
Have possibility to develop with build --continuous
Make it so that eclipse plugin (probably also other IDEs) can perfectly mirror pure command line Gradle development with their own build system.
This is a multi-project with one of them being Spring Boot application with DevTools.
wrapper (parent)
|--frontend (child)
| |--src-client
| | |--static
| | | |--img (images)
| | | \--js (raw ES6 modules)
| | \--sass (raw, note not in static folder)
| \--build
| |--lib
| | \--front.jar
| |--dist
| | |--js (bundled, minimized)
| | \--css (compiled production, minimized)
| \--dev
| \--css (compiled dev, compact readable format)
\--backend (child) (spring boot)
|--src
| |--main/java
| |--main/resources
| \--test/java
\--build
\--lib
\--application.jar
As I described, goals were to:
Have bootRun run with raw sources for js and compiled css, but also with every resource that is included in backend's main.
Have bootJar compile into production with dependency on compiled front.jar instead of everything that is used in dev (previous point).
I've used combination of configurations, sourceSets and bootRun properties (+ a lot of time).
Here are files (stripped down):
wrapper/build.gradle
wrapper.gradleVersion '5.0-milestone-1'
allprojects {
apply plugin: 'eclipse'
}
wrapper/front/build.gradle
plugins {
id 'java' // possibly use java-base or just custom zip task, since client doesn't actually compile java
}
jar {
dependsOn buildProduction // task that compiles my stuff into build/dist
baseName 'front'
classifier 'SNAPSHOT-' + new Date().format('yyyyMMddHHmmss')
from(buildDir.absolutePath + '/dist') {
into 'static'
}
}
// Note there is a lot of other tasks here that actually compile my stuff, like gulp-sass and JSPM bundling with babel transpiler.
wrapper/backend/build.gradle
buildscript {
repositories {
mavenCentral()
maven { url 'https://repo.spring.io/milestone' }
}
dependencies {
classpath 'org.springframework.boot:spring-boot-gradle-plugin:2.1.0.RC1'
}
}
apply plugin: 'java'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'
sourceCompatibility = 10 // eclipse has (or had) problems with Java 11
targetCompatibility = 10
sourceSets {
// 'java' plugin adds default main sourceSet
dev {
resources.srcDirs = [
project(':front').projectDir.absolutePath + '/src-client',
project(':front').buildDir.absolutePath + '/dev'
]
}
}
bootJar {
baseName 'application'
classifier 'SNAPSHOT-' + new Date().format('yyyyMMddHHmmssSSS')
// I used bootArchives since it was already there and my stuff fits description, you can also define your own configuration and extend runtime one.
classpath configurations.bootArchives
}
bootRun {
sourceResources sourceSets.dev // I make bootRun (dev) use dev sourceSet
}
dependencies {
runtime 'org.springframework.boot:spring-boot-devtools'
// Since bootArchives configuration is used only by bootJar (not bootRun), this will be only in final fat .jar
bootArchives project(':front')
...
}
repositories {
mavenCentral()
maven { url 'https://repo.spring.io/milestone' }
}
Some link that helped:
http://mrhaki.blogspot.com/2014/09/gradle-goodness-adding-dependencies.html
Note that client's source folder is called src-client: This is key for making "perfect mirror" in eclipse. If we would name it src which is already being used by main in backend eclipse would vomit with name clash (which probably can be fixed by configurating eclipse plugin, but then again - we wanted to keep it simple, without IDE configurations).
As final result we get very nice command line development with gradle's continuous building and also EXACTLY mirrored behaviour in eclipse which defaults its configuration to work in the same way (but use eclipse's builder instead of gradle's). We also get nice linked folders in backend project to sources used from front. Same would probably apply to IDEA :).
Related
I have my project structured in this way:
projectRoot
+-src
+-main
| +-java
| | +-package/java files go here
| +-resources
| +-config
| +-files go here
+-test
+-java
| +-package/java files go here
+-resources
+-config
| +-files go here
+-features
+-files go here
When this project is built, it produces the following output structure:
projectRoot
+-out
+-production
| +-classes
| | +-package/classes in here
| +-resources
| +-config
| +-files in here
+-test
+-classes
| +-package/classes in here
+-resources
+-config
| +-files in here
+-features
+-files in here
This is all as expected. I have a task defined to run cucumber tests, which looks like this:
task runCucumber() {
doLast {
javaexec {
main = "cucumber.api.cli.Main"
args += ['--plugin', 'pretty', '--plugin', 'html:out/reports/cucumber/', '--plugin', 'json:out/reports/cucumber/report.json',
'--glue', 'com.example.mypackage.cucumber', 'classpath:features'
, '--tags', 'not #ignore']
systemProperties['http.keepAlive'] = 'false'
systemProperties['http.maxRedirects'] = '20'
systemProperties['env'] = System.getProperty("env")
systemProperties['envType'] = System.getProperty("envType")
classpath = configurations.cucumber + configurations.compile + configurations.testCompile + sourceSets.main.runtimeClasspath + sourceSets.test.runtimeClasspath
}
}
}
When I execute this task, my classes are not found and neither are features. Clearly, this is due to classpath not being set correctly. If I run the task with --info, I can see that the classpath is set using the parameter I indicated, however instead of out directories, it contains build directories. I expected the classpath to be:
<all dependencies>:/projectRoot/out/production/classses:/projectRoot/out/production/resources:/projectRoot/out/test/classes:/projectRoot/out/test/resources
Yet, the classpath contains the following:
<all dependencies>:/projectRoot/build/classes/java/main:/projectRoot/build/classes/java/test:
Interestingly, directory build is not produced at all under the project root. How can I set the classpath so that classes and resources can be found?
Ok, turned out the issue was not with Gradle but with IntelliJ IDEA. IDEA was overriding standard gradle paths and saving output in a different path from the one that gradle expected. Then, when the next gradle task ran, the output wasn't there. Compiling directly using gradle instead of IDEA solved the problem.
Very simple use case, I am using Eclipse Oxygen 4.7.3a that includes support from Java 9. I have two projects that are Java 9 projects:
- projectOne
| - src
| - module-info.java
| - com
| - package1
| - first
| Classificator.java
- projectTwo
| - src
| - module-info.java
| - com
| - package2
| - second
| Classifiable.java
I want to use the Classifiable class inside the Classificator, so I try to import the second module into the first project.
module-info.java Project 1:
module projectOne {
requires projectTwo;
}
module-info.java Project 2:
module projectTwo {
}
Eclipse is giving me the error:
projectTwo cannot be resolved to a module
Do I have to gather all my Java projects under one "main project" in order to let Eclipse know that all those modules are usable inside it? Or have I missed something else?
No, you don't need them to be in the same directory. Ideally, your project-one module defines some uses, which are implements by your project-two module (or vice-versa). Get the runtime implementation of your used interfaces. For this, both jars/classes must be on the module path.
For further information on module build, multi module builds,... you can refer to this link. Even if you do not use gradle, its tutorial on java 9 module build is quite interesting and gives some insight.
While wiring as a service is certainly a viable approach (as described in the other answer), let me add that requires can be made to work, too, of course.
The thing that was probably missing from the first approach is: Eclipse still needs to know where to look for modules. In real world projects this will typically be mediated by your build tool of choice plus a plug-in like m2e or buildship.
If no such plug-in is used, a normal project dependency should be added to the Build Path of project projectOne in order to make project projectTwo visible. Once the project is on the Build Path the module defined by the project can be resolved in references in module-info.java.
After updating to Intellij 2017.2, building my project creates an /out directory that contains generated source files and resource files. These files duplicate files that are already contained in /build and result in duplicate class compiler errors for the generated classes. Any ideas on a fix I need in Gradle or IntelliJ?
IntelliJ IDEA is no longer sharing the output with Gradle, please see this ticket for details.
You can either override it via the following configuration:
allprojects {
apply plugin: 'idea'
idea {
module {
outputDir file('build/classes/main')
testOutputDir file('build/classes/test')
}
}
if(project.convention.findPlugin(JavaPluginConvention)) {
// Change the output directory for the main and test source sets back to the old path
sourceSets.main.output.classesDir = new File(buildDir, "classes/main")
sourceSets.test.output.classesDir = new File(buildDir, "classes/test")
}
}
or delegate the build to Gradle: File | Settings | Build, Execution, Deployment | Build Tools | Gradle | Runner => Delegate IDE build/run actions to gradle.
File | Project Structure | Project Settings | Modules | Paths tab | Compiler output
Select 'Inherit project compile output path' to continue using /build for build artifacts
Here is my understanding:
Basically, this is a work-around for an incompatibility issue between
Gradle build path and IDEA output path.
the issue is - https://github.com/gradle/gradle/issues/2315
the solution is - keep these two directories seperate, therefore you have two (out/ and build/) https://youtrack.jetbrains.com/issue/IDEA-189063
I want to set up a large-ish project, and I'm told that gradle is the way to do it.
I'm very confused by Gradle, and the entire system seems like a lot of magic, hand-waving, and knowledge that I don't want to read all 60 chapters of the Gradle guide to grok.
I'm going to end up with the following components:
ProtoBuf files defining a bunch of messages
Java Library 1 and tests (dependent on the ProtoBufs)
Java Library 2 and tests (dependent on Java Library 1)
Java Applications (dependent on Java Library 2)
Android Application (dependent on Java Library 2)
iOS Application
Duplicates of all of the Java stuff, but for Python, C++, and Objective-C)
I want to be able to build and test everything in one large shot. So normally I'd build a tree like this:
project/ proto/
lib1/ java/ src/
test/
python/ src/
test/
...
lib2/ java/ src/
test/
python/ src/
test/
...
app1/ java/ src/
test/
python/ src/
test/
...
app2/ java/ src/
test/
python/ src/
test/
...
android/ src/
test/
iOS/ src/
test/
I get that build iOS from gradle might not be possible, so I'm happy to ignore it for now.
Is this an appropriate structure to use? How do I structure and place my gradle.build files so that libraries can be used by other teams properly? How do I make sure my dependencies are tight so that libraries include only the minimum set of what they need to include?
Gradle build files seem to leave a bunch of potentially unused tasks littered around. Do I just try to ignore these?
The structure I resided on was based on grouping similar projects into subproject structures.
project/client/core/common-android
project/client/core/features-android
project/client/core/ui-android
project/client/app/client-android
project/server/admin
project/server/base
project/server/util
project/server/api-deployment
project/bundles/bundle1
project/bundles/bundle2
project/bundles/bundle3
project/bundles/bundle4
This structure make using subproject much more intuitive then flat directory structures. Now we can apply configurations to the subproject that are similar. My final project structure looks like this
------------------------------------------------------------
Root project
------------------------------------------------------------
Root project 'platform'
+--- Project ':client' - Client: Android core projects
| +--- Project ':client:common-android' - Client: Common library for Android aar
| +--- Project ':client:features-android' - Client: Features library for Android aar
| +--- Project ':client:ui-android' - Client: UI library for Android aar
| +--- Project ':client:app-android' - Client: Apk client Android apk
+--- Project ':bundles' - bundles: OSGi bundles container project
| +--- Project ':bundles:bundle1' - bundles: OSGi bundle jar
| +--- Project ':bundles:bundle2' - bundles: OSGi bundle jar
| +--- Project ':bundles:bundle3' - bundles: OSGi bundle jar
| +--- Project ':bundles:bundle4' - bundles: OSGi bundle jar
\--- Project ':server' - Server: Coriolis root project
+--- Project ':server:admin' - Server: admin jar
+--- Project ':server:base' - Server: jar base
+--- Project ':server:apiDeployment' - Server: platform deployment war
\--- Project ':server:util' - Server: utils jar
Since we don't want the parent projects to know as little as possible about their children we can let each grouping of projects configure itself during configuration via the settings.gradle
settings.gradle:
rootProject.name = 'platform'
Map<String, String> projectProperties = startParameter.getProjectProperties()
projectProperties.put('platform', true.toString())
// TODO: Make project imports smarter by removing hardcoding of paths
def subprojects = settingsDir.listFiles(new FileFilter() {
#Override
boolean accept(File file) {
return file.isDirectory() && (file.name == 'client' || file.name == 'server')
}
})
for (File file : subprojects) {
println "Found subproject directory: $file.absolutePath"
switch (file.name) {
case 'client':
def androidHome = 'ANDROID_HOME'
// any non-null value will add android modules to the build.
// Assumption is only a valid SDK location will be set.
if (System.getenv(androidHome)) {
def clientFile = new File("$file.absolutePath/core/childProjectSettings.gradle")
if (clientFile.exists()) {
println "Adding android client"
include ':client'
project(":client").projectDir = clientFile.parentFile
apply from: clientFile.absolutePath
}
} else {
println "WARNING: Environment variable {$androidHome} not set. Not adding Android modules as they are " +
"impossible to build without the Android SDK being installed. To install the Android SDK " +
"please see: http://developer.android.com/sdk/installing/index.html"
}
break
case 'server':
def serverFile = new File("$file.absolutePath/server/childProjectSettings.gradle")
if (serverFile.exists()) {
println "Adding server"
include ':server'
project(':server').projectDir = serverFile.parentFile
apply from: serverFile.absolutePath
}
def bundlesFile = new File("$file.absolutePath/bundles/childProjectSettings.gradle")
if (bundlesFile.exists() && !projectProperties.containsKey('noBundles')) {
println "Adding osgi bundles"
include ':bundles'
project(':bundles').projectDir = bundlesFile.parentFile
apply from: bundlesFile.absolutePath
}
break
default:
println "Unknown subproject found: $file.absolutePath"
}
}
Now only subprojects that exist on the disk will be included, we can remove the remaining hardcoding for a more dynamic example but this is simpler. Then we create a file (in this example) childProjectSettings.gradle for each of our project groupings (client, server, bundles). Your childProjectSettings.gradle should specify it's subprojects in a way that it doesn't need to be updated every time a new subproject is added.
childProjectSettings.gradle:
File moduleSettingsDir = new File("$settingsDir.absolutePath/server", "bundles")
println "Bundles sees settings dir as: $moduleSettingsDir.absolutePath"
def bundleDirectories = moduleSettingsDir.listFiles(new FileFilter() {
#Override
boolean accept(File pathname) {
return pathname.isDirectory()
}
})
// get a reference to this project's descriptor so we can add subprojects
ProjectDescriptor bundles = project(':bundles')
bundleDirectories.each { File bundleDir ->
if (new File(bundleDir, "build.gradle").exists()) {
// normalize project names (blah-blah -> blahBlah)
def bundleName = bundleDir.name
if (bundleName.contains("-")) {
def names = bundleDir.name.split("-")
bundleName = names[0] + names[1].capitalize()
}
// include a subproject in the build
include ":bundles:$bundleName"
// default location will be wrong lets update the project's directory
project(":bundles:$bundleName").projectDir = bundleDir
// add the project as a subproject giving us better grouping
bundles.children.add(project(":bundles:$bundleName"))
}
}
project(':bundles').children.each {
println "Parent {$it.parent} found child {$it} in path {$it.path} using buildScript {$it.buildFile $it.path}"
}
Gradle build files seem to leave a bunch of potentially unused tasks littered around. Do I just try to ignore these?
It's not the build files that create the tasks it's the plugins that are applied in the build.gradle files. To keep the tasks as tight as possible then only declare plugins in the build.gradle that it's actually used. Tasks are not inherited from dependency projects but declared dependencies are inherited from dependency projects.
*There is an important note here all dependencies are transitive by default so if ui depends on core which declares the gson dependency then ui will by default have gson in it's classpath.
For your question about the source folders a more gradle structure would probably look more like the below. Where your groups of apps would be lib1:java|lib1:python|lib2:java|lib2:python|app1:java|app1:python|app2:java|app2:python each of theses would be a subproject of their containing group.
Then lib1 project contains two subproject lib1:java and lib1:python each compiled with the plugins in their own build.gradle files. Common code can be placed in a custom plugin in the buildSrc if needed.
project/ proto/
lib1/ java/ src/main/java/
src/main/javaTest/
python/ src/main/python/
src/main/pythonTest/
...
lib2/ java/ src/main/java/
src/main/javaTest/
python/ src/main/python/
src/main/pythonTest/
...
app1/ java/ src/main/java/
src/main/javaTest/
python/ src/main/python/
src/main/pythonTest/
...
app2/ java/ src/main/java/
src/main/javaTest/
python/ src/main/python/
src/main/pythonTest/
...
android/ src/
test/
iOS/ src/
test/
I have the following project tree:
root
|
|--MP
| |
| |---build.gradle
|
|--API
|
|---build.gradle
|
|---settings.gradle
MP::buiild.gradle:
dependencies {
compile project(':API')
}
root:build.gradle:
subprojects{
apply plugin : 'java'
repositories{
mavenCentral()
}
version = '1.0'
jar{
manifest{
attributes 'Gradle': 'Multiproject'
}
}
}
root::settings.gradle:
include 'API', 'MP'
The thing is if we delete one of these three files gradle build task will fail. So it's not clear to me how java plugin builds the project. I run gradle build for MP::build.gradle, the following output was produced:
:API:compileJava
:API:processResources UP-TO-DATE
:API:classes
:API:jar
:MP:compileJava
:MP:processResources UP-TO-DATE
:MP:classes
:MP:jar
:MP:assemble
:MP:compileTestJava UP-TO-DATE
:MP:processTestResources UP-TO-DATE
:MP:testClasses UP-TO-DATE
:MP:test UP-TO-DATE
:MP:check UP-TO-DATE
:MP:build
So, the first what we need to do when we run gradle build for MP::build.gradle is to resolve all dependecies. As far as I understand it means to load jars from an external repositories and, if need, to compile jar-files from a separate projects. In my case it's just to get API project jar-file.
So my question is what is the subsequnce of actions to compile that jar. What will happens when gradle came across the compie project(':API'). It's looking for the gradle.settings file and report an error if there isn't or it's looking for build.gradle in the root directory first?
To have quick look of what is going on in a java multiproject:
http://www.gradle.org/docs/current/userguide/tutorial_java_projects.html
"For the subsequence of actions to compile that jar." Look at the diagram
http://www.gradle.org/docs/current/userguide/tutorial_java_projects.html
And for crossproject dependencies
http://www.gradle.org/docs/current/userguide/multi_project_builds.html#sec:cross_project_configuration
Quote: "By default, the configuration of all projects happens before any task is executed"
I hope you have already figured it out .
So the gradle build life cycle explains it ... the life cycle is as below
1) initialisation
2) configuation
3) execuion .
for your particular case of a multi project build , what happens is
1) initialisation ::
here the settings.gradle is searched for no matter from which project you run it (it always tries to find settings.gradle file when you run a task and includes those projects defined in its include directive.)
2) configures
it creates the task tree based on the task you have tried to run and its dependencies.
3)execution ::
runs the task tree.
Please let me know if this is helpful.
You can read this page for more clarity.
http://www.gradle.org/docs/current/userguide/build_lifecycle.html