If you're looking for sample gradle protobuf project look here.
I'm having hard time with gradle and protobuf,
i want to create a simple gradle project that will take any proto files from default src/main/proto, src/test/proto and compile them to src/main/java, src/test/java accordingly, then pack that into a jar and publish to local repo.
Unfortunately i'm new to gradle and cant figure out how the original project is composed.
Here is my unfinished build.gradle file
apply plugin: 'java'
apply plugin: "com.google.protobuf"
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'com.google.protobuf:protobuf-gradle-plugin:0.7.0'
}
}
repositories {
mavenCentral()
}
dependencies {
compile 'com.google.protobuf:protobuf-java:3.0.0-beta-1'
}
sourceSets {
main {
proto {
srcDir 'src/main/proto'
}
java {
srcDir 'src/main/java'
}
}
test {
proto {
srcDir 'src/test/proto'
}
proto {
srcDir 'src/test/java'
}
}
}
protobuf {
// Configure the protoc executable
protoc {
// Download from repositories
artifact = 'com.google.protobuf:protoc:3.0.0-alpha-3'
}
generateProtoTasks {
// all() returns the collection of all protoc tasks
all().each { task ->
// Here you can configure the task
}
// In addition to all(), you may get the task collection by various
// criteria:
// (Java only) returns tasks for a sourceSet
ofSourceSet('main')
}
}
After runing jar task we have this :
as you can see gradle builds both test and main protos to the same classes directory (red arrows), in the jar i can see both generated classes included (while tests should be skipped).
but the main problem is that I want to make compile proto files directly to appropriate source directories (blue arrows), after that ordinary build will do the correct thing... After all we need those classes in src to use them in business logic...
So we only need one task that compiles proto to appropriate src directory... nothing more.
src/main/proto to src/main/java
src/test/proto to src/test/java
The current project as it is is located here. Please help to configure this, i'm pretty sure lot of people will need it later...
If I don't misunderstand your question it's quite simple to solve. If you don't want to distinguish between your own and the generated sources you just have to add set the generatedFileBaseDir like this generateProtoTasks.generatedFilesBaseDir = 'src'
So the entire build file looks like:
// ...
protobuf {
// Configure the protoc executable
protoc {
// Download from repositories
artifact = 'com.google.protobuf:protoc:3.0.0-alpha-3'
}
generateProtoTasks.generatedFilesBaseDir = 'src' // <- that line
generateProtoTasks {
// all() returns the collection of all protoc tasks
all().each { task ->
// Here you can configure the task
}
Than your folder looks like:
src/main/java/com/vach/tryout/AddressBookProtos.java
src/main/java/com/vach/tryout/protobuf/Main.java
BUT:
That might not be the best idea to mix generate with handcrafted source code. So my suggestion would be to generate the source code into an own directory like generatedSources and add this directory to the java sourceSet. The build file would look like this:
sourceSets {
main {
proto {
srcDir 'src/main/proto'
}
java {
// include self written and generated code
srcDirs 'src/main/java', 'generated-sources/main/java'
}
}
// remove the test configuration - at least in your example you don't have a special test proto file
}
protobuf {
// Configure the protoc executable
protoc {
// Download from repositories
artifact = 'com.google.protobuf:protoc:3.0.0-alpha-3'
}
generateProtoTasks.generatedFilesBaseDir = 'generated-sources'
generateProtoTasks {
// all() returns the collection of all protoc tasks
all().each { task ->
// Here you can configure the task
}
// In addition to all(), you may get the task collection by various
// criteria:
// (Java only) returns tasks for a sourceSet
ofSourceSet('main')
}
}
Your directory will look like this
src/main/proto/dtos.proto
src/main/java/com/vach/tryout/protobuf/Main.java
generated-sources/main/java/com/vach/tryout/AddressBookProtos.java
A nice side effect is that you can ignore this generated-sources dir in your git configuration. That's always a good idea not to publish generated source code.
Related
Basically what I'm trying to do is publish a jar file to GitHub Packages with a certain name. What I have now is:
shadowJar {
archiveFileName = "Some-Name-${parent.version}.${extension}"
}
publishing {
...
publications {
shadow(MavenPublication) { publication ->
project.shadow.component(publication)
artifactId = 'me.project'
groupId = 'some-project'
version = 1.1.0
}
}
}
But from this I get some-project-1.1.0-all.jar, I would like to get some-project-1.1.0.jar but cant seem to find the way how. Changing the archiveFileName in the shadowJar task doesn't seem to affect the publishing jar only the build jar.
I believe you need to change the archiveClassifier of the shadowJar task. By default, this is configured as all.
Point 5: https://github.com/johnrengelman/shadow/blob/master/src/docs/getting-started/README.md#default-javagroovy-tasks
https://github.com/johnrengelman/shadow/blob/7.0.0/src/main/groovy/com/github/jengelman/gradle/plugins/shadow/ShadowJavaPlugin.groovy#L66
Something like:
tasks.shadowJar {
archiveClassifier = ""
}
The shadowJar task extends of the Jar task type. By default, with the Java plugin, archiveClassifier is configured as an empty String "". The Shadow plugin reconfigures its shadowJar task with all.
I'm attempting to include a generated pom.xml in the jar that I'm creating with gradle.
So far, in my parent project, I have
subprojects {
apply plugin: 'maven-publish'
publishing {
publications {
maven(MavenPublication) {
from(components.java)
}
}
}
}
and in the sub-project I have:
tasks.build.dependsOn install
sourceSets {
main {
resources {
srcDirs = [ "src/main/resources", "build/poms" ]
}
}
}
This will generate ./build/poms/pom-default.xml, but it will not add it to the JAR.
Creating a dependency on an earlier phase than build creates circular dependencies (and I don't know whether this is the problem anyway).
Also, I'd like the pom.xml to show up inside META-INF with name pom.xml (not pom-default.xml), so this may not be the right approach anyway.
Somehow I'm thinking it can't be as complicated as this looks?
You should be able to include the POM in your JAR by adding the following to your subprojects closure:
jar {
into("META-INF/maven/${project.group}/${project.name}") {
from generatePomFileForMavenPublication
rename { it.replace('pom-default.xml', 'pom.xml') }
}
}
If you already have a jar closure, you can add it there. This automatically creates a task dependency on the generatePomFileForMavenPublication task, so that the POM file is there when the JAR is created.
The sourceSets part from your question would not be required for this.
(Side note: It would not be strictly necessary to do this at all, because the Maven publish process will publish the POM as an individual artifact anyway.)
I'm using gradle. I only have one project, broken up into source files, then test files. My test code cannot see the source code. When the test code tries to compile, it just says that the source code packages are not present. What gives?
When running "gradle build", or "gradle test", I get the following:
What went wrong:
Execution failed for task ':compileTestJava'.
Compilation failed; see the compiler error output for details.
Sample compilation error (because there's hundreds of these)
/tst/ContactsTests.java:1: error: package HoaryGuts does not exist
import HoaryGuts.AlertsHandlers;
^
Gradle build.gradle file:
apply plugin: 'java'
sourceCompatibility = 1.8
targetCompatibility = 1.8
version = '1.0'
repositories {
mavenCentral()
}
task getHomeDir << {
println gradle.gradleHomeDir
}
sourceSets {
source {
java {
srcDir "src"
}
resources {
srcDir "resources"
}
}
test {
java {
srcDir "tst"
}
compileClasspath = compileClasspath + configurations.compileOnly
}
}
test {
println "In the test function"
useTestNG()
testLogging {
exceptionFormat = 'full'
}
afterTest { desc, result ->
println "${desc.className} ${desc.name} ${result.resultType}"
}
outputs.upToDateWhen {false}
}
dependencies {
compile group: 'org.testng', name: 'testng', version: '6.9.10'
}
Just like maven, gradle uses main for production sources. These are normally defined under src/main/java for Java and test includes them just like it includes compile dependencies alongside of testCompile.
There is a reason for having src followed by a named configuration, followed by language: in your setup you do not provide a location for test resources which are very common and you make it more cumbersome to include other JVM languages such as Kotlin or Scala into the project. Maybe you do not need those, but there are rarely no good reasons to strongly deviate from the industry standard.
In any case, you did not move your main to a different location, you introduced a new source set source in addition to main (actually preserving main where they originally were). If you just specified new srcDir for main instead of introducing source (as you did for test) you would not need to extend compileClasspath and your main sources would be found in test.
If you do really want to stick to source rather than using main at a new location, in test you should add source compileClasspath and output to the classpath for test:
compileClasspath += sourceSets.source.compileClasspath + source.output + [whatever else you want to add]
But well, it is a poor choice.
I have a gradle project with 8 child projects and a configured shadowjar task to create an "all" jar. The toplevel project is setup to have dependencies to all its children, this tells shadowjar what to include:
project(':') {
dependencies {
compile project(':jfxtras-agenda')
compile project(':jfxtras-common')
compile project(':jfxtras-controls')
compile project(':jfxtras-icalendarfx')
compile project(':jfxtras-icalendaragenda')
compile project(':jfxtras-menu')
compile project(':jfxtras-gauge-linear')
compile project(':jfxtras-font-roboto')
}
}
shadowJar {
classifier = null // do not append "-all", so the generated shadow jar replaces the existing jfxtras-all.jar (instead of generating jfxtras-all-all.jar)
}
This works fine, but maven central is refusing the all jar, because it does not have an associated sources and javadocs jar.
How do I tell gradle to also generate the sources and javadoc? ShadowJar's documentation says it should do this by default.
The shadow plugin doesn't seem to have a feature of building a fat sources/javadocs jars.
Below, I provide a few short tasks (javadocJar and sourcesJar) that will build fat javadoc and source jars. They are linked to be always executed after shadowJar. But it has no dependency on the shadow jar plugin.
subprojects {
apply plugin: 'java'
}
// Must be BELOW subprojects{}
task alljavadoc(type: Javadoc) {
source subprojects.collect { it.sourceSets.main.allJava }
classpath = files(subprojects.collect { it.sourceSets.main.compileClasspath })
destinationDir = file("${buildDir}/docs/javadoc")
}
task javadocJar(type: Jar, dependsOn: alljavadoc) {
classifier = 'javadoc'
from alljavadoc.destinationDir
}
task sourcesJar(type: Jar) {
classifier = 'sources'
from subprojects.collect { it.sourceSets.main.allSource }
}
shadowJar.finalizedBy javadocJar
shadowJar.finalizedBy sourcesJar
Note, the subprojects section is required, even if you already apply the java plugin inside your subprojects.
Also note, it doesn't include javadocs of the third party libraries your subprojects might depend on. But usually you wouldn't want to do it anyway, probably.
This is an old thread, but I'm posting my solution, as it's a lot easier than the above, and I've confirmed it works:
plugins {
id 'com.github.johnrengelman.shadow' version '7.1.0'
id 'signing'
id 'maven-publish'
}
// If using Spring Boot, this is needed
jar.enabled = true
jar.dependsOn shadowJar
java {
withJavadocJar()
withSourcesJar()
}
// Remove the -all extension from the "fat" Jar, or it can't be used
// when published to Maven Central.
shadowJar {
archiveClassifier.set('')
}
// The contents of this section are described here:
// https://docs.gradle.org/current/userguide/publishing_maven.html
publishing {
publications {
jwtopaLibrary(MavenPublication) {
artifactId = 'jwt-opa'
artifacts = [ shadowJar, javadocJar, sourcesJar ]
pom {
// etc. ...
}
// Signs the `publication` generated above with the name `jwtopaLibrary`
// Signing plugin, see: https://docs.gradle.org/current/userguide/signing_plugin.html#signing_plugin
signing {
sign publishing.publications.jwtopaLibrary
}
It's not made clear anywhere, and the information needs to be collected in several places, but for the signing plugin to work, you need the short form hex key ID:
# gradle.properties
# The `signing` plugin documentation is less than helpful;
# however, this is the magic incantation to find the `keyId`:
#
# gpg --list-signatures --keyid-format 0xshort
#
# The key also needs to be distributed to public GPG servers:
#
# gpg --keyserver keyserver.ubuntu.com --send-keys 123...fed
#
# In all cases, we need to use the values from the `pub` key.
signing.keyId=0x1234abcde
Then, it's just a matter of running ./gradlew publish and magic happens (well, not really, you still have to go to Sonatype repository, do the "close & release dance", but you know, whatever).
It is written in manual:
I.e. that src/sourceSet/java is a default path for "Java source for the given source set".
How to utilize this? Suppose I wish to create source set demo.
Can I write
sourceSets {
demo {
java {
srcDir 'src/demo/java'
}
resources {
srcDir 'src/demo/resources'
}
}
}
Can I write somehow without explicit paths?
May be I not required to write anything, just put files into demo subfolder?
UPDATE
I have tested
sourceSets {
demo {
java {
srcDir 'src/demo/java'
}
resources {
srcDir 'src/demo/resources'
}
}
}
and
sourceSets {
demo {
java
resources
}
}
and
sourceSets {
demo
}
In all cases running gradle build does not cause sources compiled.
build.gradle file is follows:
group 'net.inthemoon.tests'
version '1.0-SNAPSHOT'
apply plugin: 'java'
sourceCompatibility = 1.5
repositories {
mavenCentral()
}
dependencies {
testCompile group: 'junit', name: 'junit', version: '4.11'
}
sourceSets {
demo {
java {
srcDir 'src/demo/java'
}
resources {
srcDir 'src/demo/resources'
}
}
}
Sample project: https://github.com/dims12/MultipleSourceRoots
Gradle default tasks will not build non-default source sets unless there is an explicit dependency on them in the main chain. In order to compile your demo classes you'd have to call gradle demoClasses as per cricket_007's answer - or better yet, just do gradle build demo which will also put the generated classes in the target jar.
All you need is this :
sourceSets {
demo
}
... and the build demo task will indeed create any missing directory as expected, unless you have some weird permission scheme in effect.
Now I have looked at your git project and it seems your demo source set depends on the main classes. You need to make this dependency clear (i.e. add them to the demo classpaths) to avoid compilation errors :
sourceSets {
main
demo {
compileClasspath += main.output
runtimeClasspath += main.output
}
}
This should be all you need:
sourceSets {
demo
}
More configuration may be needed to define the dependencies of these sources and where they should be used.
Brand new IntelliJ Java Gradle project. (Disclaimer: there was a setting to auto-create empty source directories automatically that I checked)
Upon adding this, it created that demo directory and the resources under it.
sourceSets {
demo
}
To compile your additional sourceSet you can run
gradlew demoClasses
As described in gradle's documentation and already mentioned in the other answers, to add a new source set it should be sufficient to write:
sourceSets {
demo {
compileClasspath += main.output
}
}
The Java plugin will automatically add corresponding compileDemoJava and processDemoResources tasks to your build.
To automatically execute these tasks during your regular build, you can add a dependency to the default tasks:
tasks.compileJava.dependsOn compileDemoJava
or the other way around:
tasks.compileJava.finalizedBy compileDemoJava
To add the output of your new source set to some distribution file, you will however need to define a corresponding task and declare this task's output to be an artifact of your build:
task demoJar(type: Jar) {
from sourceSets.demo.output
}
artifacts {
archives demoJar
}
See here for further reading on the publishing stuff
UPDATE:
Added main.output to compile classpath of demo source set as already suggested by #Amine