I have a gradle monolithic project with too many dependencies.
I'd like to explode it into many sub-projects and publish all sub-projects (build + sources + javadoc) + an extra project being the merge of all sub-projects.
This extra project should be like a virtual artifact with all my projects in a single jar like it is today because I don't want a too big change for my users.
The jar must not include dependencies (it is not an uber-jar) but the resulted pom.xml must contain the dependencies of all sub-projects (the generated pom.xml of the maven artifact must contain all dependencies).
The virtual artifact will include the merge of javadoc and sources too in order to respect Maven Central conventions.
Current state:
Project Main, generate
pom.xml
main.jar
main-sources.jar
main-javadoc.jar
Expected state:
Subproject A, generate
A-pom.xml
A.jar
A-sources.jar
A-javadoc.jar
Subproject B, generate
B-pom.xml
B.jar
B-sources.jar
B-javadoc.jar
virtal-Project Main, generate
pom.xml=A-pom.xml+B-pom.xml
main.jar=A.jar+B.jar
main-sources.jar=A-sources.jar+B-sources.jar
main-javadoc.jar=A-javadoc.jar+B-javadoc.jar
How can I manage it?
We have been in exactly the same situation for some time now. We want to publish a single artifact for our clients to depend on, although internally the product is developed through a few separate component projects. I got it done eventually (with compromises), and here is what I learned:
Merging jars is not as straightforward as it looks like because there could be things like resource files within a jar that are not
always namespace-ed. It is possible that two of your jars have a
resource file with the same name, in which case you will have to
merge the content of those files.
Javadoc is very hard to merge without accessing the original source
files because it has summary pages (index pages).
So my advice would be:
Think twice, maybe what you really want is NOT a single jar, but a single dependency for your clients? These are different. You can easily have a pom only artifact. Depending on this pom only artifact will simply translates transitively into depending on individual artifacts of your component sub projects. To your client, practically, nothing is changed. Spring Boot takes this approach. To do it, you can create an empty java-library project, make all your component projects its api dependency. You don't even need any source code in this project.
If you really want to merge into a single jar, you can try building a fat jar with customization. The customization is not to pull in 3rd party dependencies.
We use the Gradle Shadow plugin for merging jars. Its original purpose was to build a fat jar, which will include all the transitive dependencies. But it also has a special "shadow" configuration, to which you can add dependencies if you want the dependencies to be exported into POM rather than bundled. So what you need to do:
Define a non-transitive configuration (say bundler) to which you will add your sub-project as dependencies. This is going to be the target configuration for the Gradle Shadow plugin.
Define a transitive configuration (bundlerTransitive) that extends from your non-transitive one. This will be manually resolved in order to find the 3rd party dependencies
in your build.gradle, register an afterEvaluate closure, where you find the level two dependencies of the resolved transitive configuration, add them to the shadow configuration. The reason for level-two is that level one dependencies will be your sub-project artifacts.
After all the above, the artifact produced by shadowJar task is the one to be uploaded to maven. You will need to configure the shadowJar task to remove the classifier (which is shadow by default)
Here is a complete example (build.gradle) of bundling vertx-web and all its dependencies within the io.vertx group:
plugins {
id 'java'
id 'maven-publish'
id 'com.github.johnrengelman.shadow' version '5.2.0'
}
group 'org.example'
version '1.0-SNAPSHOT'
repositories {
mavenCentral()
}
configurations {
bundler {
transitive = false
}
bundlerTansitive {
extendsFrom bundler
transitive = true
}
}
dependencies {
bundler "io.vertx:vertx-web:4.0.0"
bundler "io.vertx:vertx-web-common:4.0.0"
bundler "io.vertx:vertx-core:4.0.0"
bundler "io.vertx:vertx-auth-common:4.0.0"
bundler "io.vertx:vertx-bridge-common:4.0.0"
}
shadowJar {
configurations = [project.configurations.bundler]
classifier ''
}
publishing {
publications {
shadow(MavenPublication) { publication ->
project.shadow.component(publication)
}
}
}
project.afterEvaluate {
// this is needed because your sub-projects might have inter-dependencies
def isBundled = { ResolvedDependency dep ->
return configurations.bundler.dependencies.any {
dep.moduleGroup == it.group && dep.moduleName == it.name
}
}
logger.lifecycle '\nBundled artifacts and their 1st level dependencies:'
// level one dependencies
configurations.bundlerTansitive.resolvedConfiguration.firstLevelModuleDependencies.forEach {
logger.lifecycle "+--- ${it.getName()}"
// level two dependencies
it.children.findAll({ ResolvedDependency dep -> !isBundled(dep) })
.forEach { ResolvedDependency dep ->
logger.lifecycle "| +--- ${dep.name}"
project.dependencies.add('shadow', [group: dep.moduleGroup, name: dep.moduleName, version: dep.moduleVersion])
}
}
logger.lifecycle '\nExported Dependencies:'
configurations.shadow.getResolvedConfiguration().getFirstLevelModuleDependencies().forEach {
project.logger.lifecycle "+--- ${it.getName()}"
}
}
For javadoc if you don't care about the index (compromise, as I said), then it is just a jar task with a copy spec:
configurations {
javadoc {
transitive = false
}
}
dependencies {
javadoc 'com.my:component-a:1.1.0:javadoc'
javadoc 'com.my:component-b:1.1.0:javadoc'
javadoc 'com.my:component-c:1.1.0:javadoc'
javadoc 'com.my:component-d:1.1.0:javadoc'
}
task javadocFatJar(type: Jar) {
archiveClassifier.set('javadoc')
from {
configurations.javadoc.collect { it.isDirectory() ? it : zipTree(it) }
}
with jar
}
This cannot be done with maven-publish directly, but one can add individual java-library modules and package each of them with sources and docs. With Gradle this would be a simple jar task, but when the artifacts are publicly available ...such transitive dependencies should better be provided by a meta package; nothing but Maven (Local/Central) dependencies, instead of embedded JARS. In this case, this would be just another module (which obviously would only build after having published the others).
And concerning the concept, that it would require any "merged" JavaDocs ...
https://central.sonatype.org/pages/requirements.html#supply-javadoc-and-sources
While they're referenced (Maven Central) in *.pom, Gradle will be able to find them.
Just use repository mavenLocal() instead of mavenCentral() for testing purposes.
Related
I have a gradle project with 8 child projects and a configured shadowjar task to create an "all" jar. The toplevel project is setup to have dependencies to all its children, this tells shadowjar what to include:
project(':') {
dependencies {
compile project(':jfxtras-agenda')
compile project(':jfxtras-common')
compile project(':jfxtras-controls')
compile project(':jfxtras-icalendarfx')
compile project(':jfxtras-icalendaragenda')
compile project(':jfxtras-menu')
compile project(':jfxtras-gauge-linear')
compile project(':jfxtras-font-roboto')
}
}
shadowJar {
classifier = null // do not append "-all", so the generated shadow jar replaces the existing jfxtras-all.jar (instead of generating jfxtras-all-all.jar)
}
This works fine, but maven central is refusing the all jar, because it does not have an associated sources and javadocs jar.
How do I tell gradle to also generate the sources and javadoc? ShadowJar's documentation says it should do this by default.
The shadow plugin doesn't seem to have a feature of building a fat sources/javadocs jars.
Below, I provide a few short tasks (javadocJar and sourcesJar) that will build fat javadoc and source jars. They are linked to be always executed after shadowJar. But it has no dependency on the shadow jar plugin.
subprojects {
apply plugin: 'java'
}
// Must be BELOW subprojects{}
task alljavadoc(type: Javadoc) {
source subprojects.collect { it.sourceSets.main.allJava }
classpath = files(subprojects.collect { it.sourceSets.main.compileClasspath })
destinationDir = file("${buildDir}/docs/javadoc")
}
task javadocJar(type: Jar, dependsOn: alljavadoc) {
classifier = 'javadoc'
from alljavadoc.destinationDir
}
task sourcesJar(type: Jar) {
classifier = 'sources'
from subprojects.collect { it.sourceSets.main.allSource }
}
shadowJar.finalizedBy javadocJar
shadowJar.finalizedBy sourcesJar
Note, the subprojects section is required, even if you already apply the java plugin inside your subprojects.
Also note, it doesn't include javadocs of the third party libraries your subprojects might depend on. But usually you wouldn't want to do it anyway, probably.
This is an old thread, but I'm posting my solution, as it's a lot easier than the above, and I've confirmed it works:
plugins {
id 'com.github.johnrengelman.shadow' version '7.1.0'
id 'signing'
id 'maven-publish'
}
// If using Spring Boot, this is needed
jar.enabled = true
jar.dependsOn shadowJar
java {
withJavadocJar()
withSourcesJar()
}
// Remove the -all extension from the "fat" Jar, or it can't be used
// when published to Maven Central.
shadowJar {
archiveClassifier.set('')
}
// The contents of this section are described here:
// https://docs.gradle.org/current/userguide/publishing_maven.html
publishing {
publications {
jwtopaLibrary(MavenPublication) {
artifactId = 'jwt-opa'
artifacts = [ shadowJar, javadocJar, sourcesJar ]
pom {
// etc. ...
}
// Signs the `publication` generated above with the name `jwtopaLibrary`
// Signing plugin, see: https://docs.gradle.org/current/userguide/signing_plugin.html#signing_plugin
signing {
sign publishing.publications.jwtopaLibrary
}
It's not made clear anywhere, and the information needs to be collected in several places, but for the signing plugin to work, you need the short form hex key ID:
# gradle.properties
# The `signing` plugin documentation is less than helpful;
# however, this is the magic incantation to find the `keyId`:
#
# gpg --list-signatures --keyid-format 0xshort
#
# The key also needs to be distributed to public GPG servers:
#
# gpg --keyserver keyserver.ubuntu.com --send-keys 123...fed
#
# In all cases, we need to use the values from the `pub` key.
signing.keyId=0x1234abcde
Then, it's just a matter of running ./gradlew publish and magic happens (well, not really, you still have to go to Sonatype repository, do the "close & release dance", but you know, whatever).
Dear StackOverflow users
I have a gradle project of which I want to turn the artifact into an osgi bundle. In this bundle I have:
packages that I don't want to export (may not appear in manifest's Export-Package entry)
dependencies that I want to embed (may not appear in manifest's Import-Package entry)
After a bit of tinkering I have come up with the following gradle.build file which does what I intend but maybe not in the cleanest way possible, leveraging bnd...
group 'com.mycompany'
version '1.0.0'
apply plugin: 'java'
apply plugin: 'osgi'
repositories {
jcenter()
}
dependencies{
compile 'org.osgi:org.osgi.framework:1.8.0' //provided
compile 'com.google.code.gson:gson:2.8.0' //embedded
}
jar {
//embedding the gson dependency
from({
def x = configurations.compile.find({
return it.getName().contains('gson')
})
def tree = zipTree(x)
return tree
})
//explicitly building manifest entries
manifest {
instruction 'Bundle-Vendor',
'My Company'
instruction 'Bundle-Activator',
'com.mycompany.mybundle.Activator'
instruction 'Import-Package',
'!com.google.gson',
'*'
instruction 'Export-Package',
/com.mycompany.mybundle;version="${version}"/
}
}
Is it possible to accomplish this in a cleaner way? I mainly want to avoid two things:
having to manually write the import and export-package entries
having to manually copy the contents of the embedded dependencies (gson) into my jar
I thought bnd (underlying the osgi plugin) could do that for me, but with what I have tried so far (even if I add them as private package) bnd still exports everything and imports the gson package as well as it won't add the gson classes to the jar
You would be better off using the Bnd Gradle plugin for OSGi. It is written and supported by the developers of bnd who know a thing or two about OSGi.
I have an Android closed source module that will be be used as a library in other projects. It contains external dependencies.
To publish it, I'm creating a Maven artifact with the following gradle task:
apply plugin: 'maven'
def coreAarFile = file('...build\\outputs\\aar\\android-sdk-release.aar')
artifacts {
archives coreAarFile
}
uploadArchives {
repositories.mavenDeployer {
repository(url: "file://.\\mvn-repo")
pom.project {
groupId 'a.blabla'
artifactId 'blabla-sdk'
version "1.0.0"
}
}
}
It generates the .aar file, the pom.xml, etc without problems.
Then I create a project that have a dependeny to my library declared. It works until it needs to access to the external dependencies, when throws a class not found exception.
How can I edit my gradle task to include external dependencies or at least a reference to them? They are published in mvnrepository.com and github.com.
I moved uploadArchives to the build.gradle of the module and removed the artifacts element. It works!
Thanks to CommonsWare for pointing to the right direction.
To specify a Maven dependency in my project, I provide a name, a group id, and a version. This has been enough for every dependency in my project, save one. Pig has multiple jars in the same artifact (not sure if I have the proper nomenclature; I'm still rather new to Maven), but I only need one.
Specifically, I need pig-0.13.0-h2.jar. However, when I provide the dependency
compile "org.apache.pig:pig:0.13.0"
in my build.gradle, only pig-0.13.0.jar, pig-0.13.0-sources.jar, and pig-0.13.0.pom are downloaded. I need the "*-h2.jar", because that's the correct one to work with my version of Hadoop.
Is there a way to tell Gradle (and, generally, Maven or whatever) that my compile dependency requires this exact jar, and that only this one should be included in the classpath?
What you need is to specify the classifier. The following script will do the job:
apply plugin: 'java'
repositories {
mavenCentral()
}
dependencies {
compile "org.apache.pig:pig:0.13.0:h2"
}
task copyDeps(type: Copy) {
from configurations.compile
into 'deps'
}
Gradle 2.3; shadow plugin 1.2.1.
In my build.gradle, I use the shadow plugin in order to repackage a dependency, like such:
shadowJar {
relocate("com.google.common", "r.com.google.common");
}
I also add the shadow jar to the list of artifacts to publish:
artifacts {
archives jar;
archives sourcesJar;
archives javadocJar;
archives shadowJar;
}
However the list of dependencies of the shadow jar still contains all the dependencies of the "normal" jar, even though it has every dependency builtin.
Is this the intended behavior? How can I make the shadow jar exclude this or that dependency?
Here at work we had the same problem and we just put this in a build.gradle of one of our projects:
def installer = install.repositories.mavenInstaller
def deployer = uploadArchives.repositories.mavenDeployer
[installer, deployer]*.pom*.whenConfigured { pom ->
pom.dependencies.retainAll {
it.groupId == 'our.group.id' && it.artifactId == 'some-api'
}
}
This removes all dependencies from the pom.xml except for the dependency on one of our API projects.
(And it is a pretty verbatim copy of an example from the official Gradle documentation.)