I'm trying to generate Protobufs in a Java project that are defined in another Git repository that I'd like to add as a Git submodule. My build.gradle contains
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:4.0.0-rc-2"
}
plugins {
grpc {
artifact = "io.grpc:protoc-gen-grpc-java:${grpcVersion}"
}
}
generateProtoTasks {
all()*.plugins {
grpc {}
}
}
}
// Inform IDEs like IntelliJ IDEA, Eclipse or NetBeans about the generated code.
sourceSets {
main {
java {
srcDirs 'build/generated/source/proto/main/grpc'
srcDirs 'build/generated/source/proto/main/java'
}
}
}
and I've included the protobufs repository (called my-protobufs) in the src/main/proto directory. The Protobufs are in turn located in a proto subdirectory of my-protobufs. A partial directory structure looks like this:
src/main/proto/edm-grpc-protobufs/proto
├── mypackage
│ └── v1
│ ├── bar.proto
│ └── foo.proto
The foo.proto file has an import statement like this:
import "mypackage/v1/bar.proto";
That is because in that repository, the Protobuf path is the proto directory. The problem is that when I try to ./gradlew build, I get an error like the following:
> Task :generateProto FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':generateProto'.
> protoc: stdout: . stderr: mypackage/v1/bar.proto: File not found.
my-protobufs/proto/mypackage/v1/foo.proto:5:1: Import "axmorg/v1/bar.proto" was not found or had errors.
my-protobufs/proto/mypackage/v1/foo.proto:10:5: "SourceType" is not defined.
The problem is basically that the --proto_path (in the parlance of protoc) or the directory in which to search for imports is not correctly defined, so protobuf-gradle-plugin doesn't know where to find them. Is is possible to update the build.gradle to specify this path?
I found this in die documentation:
https://github.com/google/protobuf-gradle-plugin#customizing-source-directories
sourceSets {
main {
proto {
// In addition to the default 'src/main/proto'
srcDir 'src/main/protobuf'
srcDir 'src/main/protocolbuffers'
// In addition to the default '**/*.proto' (use with caution).
// Using an extension other than 'proto' is NOT recommended,
// because when proto files are published along with class files, we can
// only tell the type of a file from its extension.
include '**/*.protodevel'
}
java {
...
}
}
test {
proto {
// In addition to the default 'src/test/proto'
srcDir 'src/test/protocolbuffers'
}
}
}
I ended up working around this problem: the packages containing relative Protobuf imports actually weren't needed for the Java project whereas the ones that were needed did not contain relative imports, so I modified the sourceSets in build.gradle to be like
sourceSets {
main {
java {
srcDirs 'build/generated/source/proto/main/grpc'
srcDirs 'build/generated/source/proto/main/java'
}
proto {
exclude '**/*.proto'
include 'my-protobufs/proto/mypackage/**/*.proto'
}
}
}
which circumvented the issue with the Protobuf path as there are no longer any imports. I'm still curious how I would specify the Protobuf path, though.
Related
If you got a multi-project gradle build. And one module depends on another.
How could you add the dependency module source code to the output jar
Now i am using this:
java {
withSourcesJar()
}
I am new to gradle builds and i don't know any kotlin.
And if you have the source code of a dependency as a .jar file. Could you also add that
to the output?
So I have a project module:
dependencies:
project module
local .jar
What i want:
One .jar of the project (including other modules and dependencies) compiled code:
project-0.5.0.jar
..and one .jar of the source code (including other modules and dependencies)
project-0.5.0-sources.jar
I have all source code of dependencies stored locally as .jar files
Edit
My project conventions (global for all modules):
plugins {
`java-library`
}
java {
withSourcesJar()
}
How I am currently creating the project "fat".jar with compiled code:
(inside the build script)
tasks.jar {
//manifest.attributes["Main-Class"] = "com.example.MyMainClass"
val dependencies = configurations
.runtimeClasspath
.get()
.map(::zipTree) // OR .map { zipTree(it) }
from(dependencies)
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
I have figured out how to add a project moduleA to another moduleB output sources .jar like so (inside moduleB's build-script):
tasks.sourcesJar {
from(project(":moduleA").sourceSets.main.get().allSource)
}
Now I need to figure out how to include source code from a dependency .jar
from(file("../path/dependency-1.0.0-sources.jar"))
This packs the .jar as it is. I need it's files.
I figured it out. And it was easier than i thought. Keep in mind i am using Kotlin.
(All code snippets are inside the build.gradle.kts file of the project / module you are creating the sources .jar for)
First off you need to include either the java or java-library plugin:
plugins {
`java-library`
}
And as far as i know, also this plugin extension:
java {
withSourcesJar()
}
This makes the sourcesJar task available (task used to create the sources jar), and you can modify it like so:
tasks.sourcesJar {
from(project(":common").sourceSets.main.get().allSource)
from(zipTree("../libs/tinylog-2.5.0/tinylog-api-2.5.0-sources.jar"))
}
The first line inside the brackets includes my "common" module source code to the output .jar.
The second line adds the .java files inside the tinylog sources .jar to the output .jar.
I know this should be pretty simple, but there's obviously something that I'm just not getting.
I have a gradle application setup in the following in build.gradle:
plugins {
id 'application'
}
application {
mainClass = 'com.my.Main'
}
dependencies {
implementation 'other project 1'
implementation 'other project 2'
}
The source is set up
src/
main/
java/
resources/
I want to have a different distribution (for local development) that has different files for the resources directory, but for this 'dev' distribution, I want to keep the mainClass and the dependencies from the main distribution.
If I add a 'dev' distribution, I can package up files from the src/dev/... directory, but I don't have the application dependencies.
distributions {
dev {
contents {
// something here to add the compiled java and dependencies from main
// but not the resources?
}
}
main {
}
}
Also, gradle only seems to have one 'run' task that runs the main distribution. Is it possible to define a run for the dev distribution?
You need to define a SourceSet called dev:
sourceSets {
main {
}
dev {
resources {
exclude ...
}
}
}
And also the corresponding directory (that's not even required, when only excluding files):
mkdir -p ./modulename/src/dev/resources
I am trying to run a Java class as a gradle task.
I have added this to my build.gradle:
task(downloadKeystore, dependsOn: 'classes', type: JavaExec) {
main = 'com.orbitbenefits.keystore.KeystoreDownloader'
}
However, when I run on the command line gradle downloadKeystore, it fails with the following error:
:Noa:downloadKeystoreError: Could not find or load main class com.orbitbenefits.keystore.KeystoreDownloader
So I have added a classpath to my task as specified in this question:
task(downloadKeystore, dependsOn: 'classes', type: JavaExec) {
main = 'com.orbitbenefits.keystore.KeystoreDownloader'
classpath = sourceSets.main.runtimeClasspath
}
However, this is large legacy project with extremely long classpath, so when I run gradle downloadKeystore I get another error:
Caused by: java.io.IOException: Cannot run program "C:\Program Files\Java\jdk1.8.0_77\bin\java.exe" (in directory "C:\Users\pawlakj\IdeaProjects\noa\Noa"): CreateProcess error=206, The filename or extension is too long
So I have modified my sourceSets in build.gradle so it now looks like this:
sourceSets {
main {
java {
srcDirs(...)
}
resources {
srcDirs(...)
}
}
keystore {
java {
srcDirs = ['src/test/java/com/orbitbenefits/keystore']
}
}
test {
java {
srcDirs(...)
}
resources {
srcDirs(...)
}
}
}
...
task(downloadKeystore, dependsOn: 'classes', type: JavaExec) {
main = 'com.orbitbenefits.keystore.KeystoreDownloader'
classpath = sourceSets.keystore.runtimeClasspath
}
This works on the command line, however when I run gradle refresh in IntelliJ, it generally breaks the project. It looks like this:
But it should look like this:
I have tried manually setting test/src root directories but it doesn't really work and also I don't want other developers to have to do this.
I have also tried setting classpath manually using something like:
classpath = classpath('src/test/java/com/orbitbenefits/keystore')
But I couldn't make it work (gradle doesn't like it).
Question
I need to solve only one of these two problems:
How can I define classpath manually?
OR
How can I make IntelliJ to not mess up project structure when using its gradle refresh button?
Your IntelliJ is having a problem with srcDirs = ['src/test/java/com/orbitbenefits/keystore'] because src/test/java is already a folder containing some source.
One solution could be to define a new folder sibling to src where you have your KeystoreDownloader class and then import the keystore as follows:
keystore {
java {
srcDirs = ['keystore']
}
}
The big picture is that I have a protobuf (think of it like a class) datatype that refers to another protobuf that is within another Jar file that is a dependency in my POM file. This works perfectly for .java files but unfortunately doesn't work for protobuf files. The best solution I can think of is to tell Maven to extract this other dependency Jar (that contain proto files) file in a location and then tell Maven to do a protoc compile of all these proto files in that location. Alas, I dont know how to tell Maven to do this. Any help would be greatly appreciated as we use standard Jar files to capture our proto files.
import "X.proto"; // refers to a proto file in another jar
import "Y.proto";
message A {
repeated X o = 1; // This cant be done
repeated Y e = 2;
}
The above will not work since X is not in the same path as this file.
I found the solution for this in Gradle. Fill in the blanks below and point your repo correctly, and you should be able to get the below to work. Now you can hierarchically compose protobufs across multiple projects through proto files within other jar files !
In gradle, you can use the following:
// these are your protobuf plugin repositories
buildscript {
repositories {
maven {
url 'WHERE YOUR PROTOBUF PLUGIN EXISTS. e.g. http://maven or MavenCentral()'
}
}
dependencies {
classpath 'com.google.protobuf:protobuf-gradle-plugin:0.7.0'
classpath 'com.google.protobuf:protoc:2.6.1'
}
}
apply plugin: 'com.google.protobuf'
group = "YOUR PACKAGE NAME. e.g. com.x.y"
version = "1.0" // release version
// these are your regular dependencies repositories. This could be
// very similar to the ones in buildscript above.
repositories {
mavenLocal()
}
// this is needed by the plugin and is where teh magic happens. It
// tells protobuf where to extract all the proto files that it finds
// in all the dependency jar files you have
protobuf {
generatedFilesBaseDir="$projectDir/src/"
}
// also magic you need for this protobuf plugin to work
sourceSets {
main {
// this tells the plugin where your project specific
// protofiles are located
proto {
srcDir 'src/main/resources/proto/'
}
java {
srcDir 'src/main/java'
}
}
}
dependencies {
compile 'com.google.protobuf:protobuf-java:2.6.1'
}
I'm using Dagger 2 to generate some source code in my Gradle project. Right now those sources are being generated and added in the ./build/classes/main folder along with all the class files.
How do I choose a folder to separate all the generated .java files to?
How do I include that folder in my gradle Java project, and have IntelliJ view those as sources so I can use them in my project?
It looks like the application plugin only uses a certain set of directories by default, mixing in flavours of build to decide what files to compile.
However, I did find an example build script that creates a dagger configuration and manipulates gradle into using it for the generated output and adds it to the classpath. It uses dagger-compiler.
The core of it is:
sourceSets {
dagger {
java {
srcDirs = ['src/dagger/java']
}
}
}
configurations {
compileDagger
}
compileJava {
description = "dagger annotation processor is loaded automatically from classpath"
sourceSets.dagger.java.srcDirs*.mkdirs()
classpath += configurations.compileDagger
options.compilerArgs += [
'-s', sourceSets.dagger.java.srcDirs.iterator().next()
]
}
clean {
description = "delete files in generated source directory tree"
delete fileTree(dir: sourceSets.dagger.java.srcDirs.iterator().next())
}
dependencies {
ext.daggerVersion = "2.0.1"
compile(
"com.google.dagger:dagger:${daggerVersion}",
"com.google.guava:guava:18.0")
compileDagger(
"com.google.dagger:dagger-compiler:${daggerVersion}")
}
Regarding IntelliJ, the plugin should automatically add any srcSets via the normal building of the idea project, so there should be no additional configuration needed, just regenerate it.