Develop Reference

How can you add dependency source code to sourceJar task output?

If you got a multi-project gradle build. And one module depends on another.
How could you add the dependency module source code to the output jar
Now i am using this:
java {
withSourcesJar()
}
I am new to gradle builds and i don't know any kotlin.
And if you have the source code of a dependency as a .jar file. Could you also add that
to the output?
So I have a project module:
dependencies:
project module
local .jar
What i want:
One .jar of the project (including other modules and dependencies) compiled code:
project-0.5.0.jar
..and one .jar of the source code (including other modules and dependencies)
project-0.5.0-sources.jar
I have all source code of dependencies stored locally as .jar files
Edit
My project conventions (global for all modules):
plugins {
`java-library`
}
java {
withSourcesJar()
}
How I am currently creating the project "fat".jar with compiled code:
(inside the build script)
tasks.jar {
//manifest.attributes["Main-Class"] = "com.example.MyMainClass"
val dependencies = configurations
.runtimeClasspath
.get()
.map(::zipTree) // OR .map { zipTree(it) }
from(dependencies)
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
I have figured out how to add a project moduleA to another moduleB output sources .jar like so (inside moduleB's build-script):
tasks.sourcesJar {
from(project(":moduleA").sourceSets.main.get().allSource)
}
Now I need to figure out how to include source code from a dependency .jar
from(file("../path/dependency-1.0.0-sources.jar"))
This packs the .jar as it is. I need it's files.

I figured it out. And it was easier than i thought. Keep in mind i am using Kotlin.
(All code snippets are inside the build.gradle.kts file of the project / module you are creating the sources .jar for)
First off you need to include either the java or java-library plugin:
plugins {
`java-library`
}
And as far as i know, also this plugin extension:
java {
withSourcesJar()
}
This makes the sourcesJar task available (task used to create the sources jar), and you can modify it like so:
tasks.sourcesJar {
from(project(":common").sourceSets.main.get().allSource)
from(zipTree("../libs/tinylog-2.5.0/tinylog-api-2.5.0-sources.jar"))
}
The first line inside the brackets includes my "common" module source code to the output .jar.
The second line adds the .java files inside the tinylog sources .jar to the output .jar.

Gradle multi project, how to deal with different rootDirs

My company is currently switching from Ant to Gradle for its Java projects, but I am stuck with a nice clean setup. Let's say I work for a company that builds websites for clients which generally use the same root libraries (core project), but have some specific code, which is put in the sub-project. Per client, we build a new sub-project, that depends on the core project. Number of clients will increase in the future.
Currently we have three projects:
A core project. This should run individually, as we want to be able to do the unit testing seperately for this.
Two sub-projects that depend on the core project, but have some own projects that are specific to the project.
I was sucessful in converting the whole ant build into a gradle build for the core project. My idea is to have all functionality and project structure in the core, and only the extra for what is actually needed in the sub-projects.
Here is a short sample of our folder structure:
-- core
- build.gradle
- settings.gradle
-- repository (our external jars used)
-- Implementation
-- source_code
-- all the core project folders
-- Projects
-- Client A
- build.gradle
- settings.gradle
-- more project specific folders
-- Client B
- build.gradle
- settings.gradle
-- more project specific folders
I use the $rootDir variable a lot. A fraction of the core's settings.gradle looks as such:
project(':CoreProjectA').projectDir = new File(rootDir, 'Implementation/Source_code/Core/coreA')
project(':CoreProjectB').projectDir = new File(rootDir, 'Implementation/Source_code/Core/CoreB')
But with many more. Also, in the core build.gradle, I refer to our repository as such:
repositories {
//All sub-projects will now refer to the same 'libs' directory
flatDir {
dirs "$rootDir/repository/lib/jaxb"
//many more dirs here
}
}
Now this all works great when I do a gradle build from the core project.
I was planning to put the next piece of code in every client's subproject build.gradle:
apply from: '../../../build.gradle'
When I run a gradle build from Client A folder, my rootDir obviously has changed, and now, all my paths cannot be found anywhere.
Is there any way to set this up in a nice clean way? So that every future sub project added can always use the same structure? Or will I have to give each sub-project its own build.gradle and settings.gradle entirely?
I know the last option could work, but it is a lot of overhead, and just doesn't seem nice and clean to me at all..
Thanks in advance!

I recently worked on a similar configuration, so let me explain me how I build the Gradle infrastructure. Since you mentioned a lot of requirements, I hope that I'll miss any of them and you can apply my scheme to your problem.
General
We actually use build systems (like Gradle) to let them take care of any dependencies (projects, modules, tasks). So why should we define a depedency in something like a filesystem hierarchy, if we can simply define it in Gradle?
I would avoid using paths as much as possible (convention over configuration) and try to stick to Gradle projects for both the build scripts and the dependencies.
Also, if you define dependencies in your core gradle.build, you should just call this gradle file, even if you only want to build a subproject. Your apply from: '../../../build.gradle' destroys the whole Gradle logic. Instead you should use something like gradle :sub1:build to only build the first subproject.
First approach (with core as root project)
Filesystem structure:
core/
build.gradle
settings.gradle
src/
...
sub1/
src/
...
build.gradle [optional]
sub2/
src/
...
build.gradle [optional]
Core settings.gradle:
include 'sub1'
include 'sub2'
Core build.gradle:
allprojects {
apply plugin: 'java'
repositories {
// define repos for all projects
}
}
subprojects {
dependencies {
// let subprojects depend on core
compile rootProject
}
}
project(':sub1') {
// define anything you want (e.g. dependencies) just for this subproject
// alternative: use build.gradle in subproject folder
}
Second approach (all projects independent)
Filesystem structure:
core/
src/
...
build.gradle [optional]
sub1/
src/
...
build.gradle [optional]
sub2/
src/
...
build.gradle [optional]
build.gradle
settings.gradle
Root settings.gradle:
include 'core'
include 'sub1'
include 'sub2'
Root 'build.gradle'
subprojects {
apply plugin: 'java'
repositories {
// define repos for all projects
}
}
configure(subprojects.findAll {it.name != 'core'}) {
dependencies {
// Add dependency on core for sub1 and sub2
compile project(':core')
}
}
project(':sub1') {
// define anything you want (e.g. dependencies) just for this subproject
// alternative: use build.gradle in subproject folder
}
This approach provides great flexibility, since every dependency logic is handled by Gradle and you'll never have to copy anything to another position. Simply change the dependency and you are fine.
Sources
Gradle Tutorial on Multi-project Builds
Question in Gradle Forum

It looks like you have extra settings.gradle inside subprojects. That makes Gradle think the sub-project is a standalone one. If you remove settings.gradle from subprojects, Gradle will look for it up the filesystem hierarchy, will find one in core project, will create correct multimodule project and all the paths should work properly.
So, just remove extra settings.gradle files and your build will work fine. Having build.gradle in subprojects is perfectly fine.

I want to thank Nikita and especially lukegv for his detailed answer, but I went with a different approach ultimately.
I didn't want to have one main gradle build and extend it each time we have a new project. Also, I wanted to keep it simple and logical for my colleagues if they want to create a build for one of the projects.
Therefor, I kept the structure I had as described above. In the root gradle.settings but changed the
project(':CoreProjectA').projectDir = new File(rootDir, 'Implementation/Source_code/Core/coreA')
project(':CoreProjectB').projectDir = new File(rootDir, 'Implementation/Source_code/Core/CoreB')
into:
project(':CoreProjectA').projectDir = new File(customRootDir, 'Implementation/Source_code/Core/coreA')
project(':CoreProjectB').projectDir = new File(customRootDir, 'Implementation/Source_code/Core/CoreB')
In each project I have a properties.gradle with the customRootDir filled in (relatively to the projects directory).
It all works like a charm. I can go into any project folder and produce builds, while using the functionality of the root's build.gradle.

Gradle: multiple JARs without multiple projects

I have a library which is used to build a number of CLI tools using Gradle. Each CLI tool is a separate JAR. At the moment every tool requires a separate Gradle project, with an associated set of directories, like this:
Having all of this structure is resulting in the whole collection of tools becoming very unwieldy and difficult to work with. Is there any way to collect all of the different Mains into a single folder (suitably renamed) and configure Gradle to turn each one into a separate JAR?
FWIW, the JARs are currently created using https://github.com/johnrengelman/shadow . JAR size doesn't matter.
Thanks in advance.

Jars are just zip files with META-INF folder inside. Use Zip tasks to create them and dependsOn to run tasks as part of your build sequence.
I had the code like below for changing jar files:
task changeJar (type: Zip) {
baseName project.name
extension 'jar'
destinationDir new File('build')
entryCompression ZipEntryCompression.STORED
from { zipTree(new File(core.libsDir, core.name + '.jar')) }
from ( <somewhere else> ) {
exclude 'META-INF/'
}
}

I'm not sure if it's a good fit but you might be interested in my gradle-java-flavours plugin.
eg:
apply plugin: 'com.lazan.javaflavours'
javaFlavours {
flavour 'tool1'
flavour 'tool2'
}
dependencies {
compile 'a:a:1.0' // common to all tools
compileTool1 'b:b:2.0' // compile deps for tool1 only
runtimeTool2 'c:c:2.0' // runtime deps for tool2 only
}
Directories
src/main/java, src/test/java, src/main/resources, src/test/resources - common code & tests
src/tool1/java, src/testTool1/java, src/tool1/resources, src/testTool1/resources - tool1 only sources
src/tool2/java, src/testTool2/java, src/tool2/resources, src/testTool2/resources - tool2 only sources
Jars
projectName.jar
projectName-tool1.jar
projectName-tool2.jar

filter gradle configuration for specific libraries

I am currently about to migrate the build system for my project from ANT to gradle. So I am pretty new to gradle My project features a plugin mechanism that dynamically loads jars configured in an XML file. Each plugin is in its own subproject. The entries of the XML configuration look as follows
<?xml version="1.0" encoding="UTF-8"?>
<implementation ...>
<repository>text</repository>
<classpath>library1.jar</classpath>
<classpath>library2.jar</classpath>
...
</implementation>
I added an XSL transformation to prefix each entry with the correct lib folder in the build folder to make it usable in a run task of my main project.
This already works OK for the libraries that are created within my project. However, some of them rely on third party libraries, e.g., protobuf.
For now, I solved this by copying all referenced jars to the lib folder. My build.gradle looks as follows
dependencies {
compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.1.0'
...
compile project(':main')
}
task copyReferencedLibraries(type: Copy) {
into "$buildDir/libs"
from configurations.compile
}
jar {
dependsOn xslt
dependsOn copyReferencedLibraries
...
}
However, this copies all libraries, including those inherited from the main project to the lib folder. More than I actually need.
How could I filter the libraries to those defined in this subproject or at least manually filter the ones that I want to be copied?

You could just filter in your Copy task
task copyReferencedLibraries(type: Copy) {
into "$buildDir/libs"
from(configurations.compile) {
exclude 'some.jar', 'some-other.jar'
include 'some-required.jar', 'some-other-required.jar'
}
}
Reference: https://docs.gradle.org/current/dsl/org.gradle.api.tasks.Copy.html
Just be careful you don't exclude second or third tier dependency (ie a jar required by a jar your using). If you do then you won't get compile errors you will get runtime errors when the code is executed that touches the missing dependency.

Using Gradle with native dependencies

I am trying to use Sigar in a Gradle project. Sigar distribution is by default provided with 2 types of files:
a JAR that contains classes
some native files (.so, dylib, .dll)
My purpose is to repackage these files so that I can use them as dependencies deployed and downloaded on-demand from a personal Maven repository.
My first try was to define dependencies as files in order to check that my application is working as expected before to repackage. Below is the Gradle code I used for my first test that works:
dependencies {
compile files("${rootDir}/lib/sigar/sigar.jar")
runtime fileTree(dir: "${rootDir}/lib/sigar/", exclude: "*.jar")
}
Then, I have repackaged Sigar native files into a JAR and renamed the other one to match rules for maven artifacts since I want to deploy them in a Maven repository. Below is what I get:
sigar-1.6.4.jar (contains .class files)
sigar-1.6.4-native.jar (contains .dylib, .so, and .dll files at the root)
The next step was to deploy these files in my custom repository. Then, I have updated my build.gradle as follows:
dependencies {
compile 'sigar:sigar:1.6.4'
runtime 'sigar:sigar:1.6.4:native'
}
Unfortunately, when I do a gradle clean build, new dependencies are fetched but native libraries can no longer be found at runtime since now I get the following exception:
Error thrown in postRegister method: rethrowing <java.lang.UnsatisfiedLinkError: org.hyperic.sigar.Sigar.getCpuInfoList()[Lorg/hyperic/sigar/CpuInfo;>
Consequently, I am looking for a solution to fetch and to link native files to my Java app like for other dependencies. Any advice, comment, suggestion, help, solution, etc. are welcome ;)

A solution is to define a new gradle configuration that unzips JAR files at the desired location:
project.ext.set('nativeLibsDir', "$buildDir/libs/natives")
configurations {
nativeBundle
}
dependencies {
nativeBundle 'sigar:sigar:1.6.4:native'
}
task extractNativeBundle(type: Sync) {
from {
configurations.nativeBundle.collect { zipTree(it) }
}
into file(project.nativeLibsDir)
}
dist.dependsOn extractNativeBundle
Then, this location must be put in java.library.path for tasks that depend on native libraries:
systemProperty "java.library.path", project.nativeLibsDir