Develop Reference

Duplicate handling strategy error with gradle while using protobuf for java

I am using the below configuration build.gradle
plugins {
id "com.google.protobuf" version "0.8.17"
id "java"
}
group "de.prerna.aws.tests"
version "1.0-SNAPSHOT"
repositories {
mavenCentral()
}
ext {
protobufVersion = "3.18.1"
}
dependencies {
implementation "com.google.protobuf:protobuf-java:$protobufVersion"
sourceSets {
main {
proto {
srcDir 'src/main/proto'
}
java {
// include self written and generated code
srcDirs 'src/main/java'
}
}
}
protobuf {
protoc {
artifact = 'com.google.protobuf:protoc:4.0.0-rc-2'
}
plugins {
grpc {
artifact = "io.grpc:protoc-gen-grpc-java:1.39.0"
}
}
generateProtoTasks.generatedFilesBaseDir = 'generated-sources'
generateProtoTasks {
all().each { task ->
task.plugins { grpc{} }
}
ofSourceSet('main')
}
}
Error
* What went wrong:
Execution failed for task ':processResources'.
> Entry Person.proto is a duplicate but no duplicate handling strategy has been set. Please refer to https://docs.gradle.org/7.2/dsl/org.gradle.api.tasks.Copy.html#org.gradle.api.tasks.Copy:duplicatesStrategy for details.

A variant of BParolini for build.gradle (Groovy DSL)
tasks.withType(Copy) {
filesMatching("**/*.proto") {
duplicatesStrategy = DuplicatesStrategy.INCLUDE
}
}

I could fix this problem by adding the following code to my build.gradle.kts:
tasks {
withType<Copy> {
filesMatching("**/*.proto") {
duplicatesStrategy = DuplicatesStrategy.INCLUDE
}
}
}
Extra info: I'm using Gradle 7.3-rc-3 and Java 17.

Unfortunately nobody explains reasons for this problem, so here is some of my explorations and guesses. Please correct me if you know more.
If found that following build script code causes this error:
proto { srcDir 'src/main/proto' }
If look inside "build/extracted-include-protos" directory, there are original .proto files copied into "build/extracted-include-protos/test" (but not into main).
My guess is that those auto-copied .proto files are originally uses as the only sources, but when adding "src/main/proto" source set we give some compiler tool second set of same files.
Removing this srcDir is not a good idea, because it required for IDEA to correctly open included .proto on Ctrl+click (otherwise it is opened extracted copies which is useless).

Constant path replacement for different environments using gradle

This is a Apache Storm based project. I have a Constants file which looks something like this
public class Constant {
public static final String CONTEXT_PATH ="<some path to a context.xml file>";
public static final String APP_PROPERTIES_PATH = "<path to the properties file>";
//...More static properties
}
This CONTEXT_PATH variable is different for different environments (dev, test, prod).
I have a gradle task which generates the JAR file for deployment
task stormJar(type: Jar) {
baseName = 'diagnostic'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}
I was looking to dynamically change or refer the CONTEXT_PATH variable so that we can create builds for different environments without making any changes to this file.
I believe there are solutions to do so in the Android territory (BuildConfig), but not able to find a solution for a plain Java project.
The solution should be such, it should work for IDE (IntelliJ and Eclipse) as well as create environment specific build.
Doing something like below should get the work done
gradlew build -pEnvironment=prod
Not much experience with gradle. Please point me in the right direction.

Note there is a working example here.
One technique is to generate Constants.java as part of the build. Consider this template (stored as a resource, not as Java code):
public class Constants {
public static final String CONTEXT_PATH = "__CONTEXT_PATH";
public static final String APP_PROPERTIES_PATH = "__APP_PROPERTIES_PATH";
}
and the following generates Constants.java early in the compileJava task:
compileJava.doFirst {
def newConstantsFile = new File("${projectDir}/src/main/java/net/codetojoy/util/Constants.java")
def templateConstantsFile = new File("${projectDir}/resources/TemplateConstants.java")
newConstantsFile.withWriter { def writer ->
templateConstantsFile.eachLine { def line ->
def newLine = line.replace("__PACKAGE", "net.codetojoy.util")
.replace("__CONTEXT_PATH", getContextPath())
.replace("__APP_PROPERTIES_PATH", getAppPropertiesPath())
writer.write(newLine + "\n");
}
}
}
and then the crucial env-specific predicates:
def getContextPath = { ->
def result = "default"
if (project.Environment == "prod") {
result = "PROD context path here"
} else if (project.Environment == "uat") {
result = "UAT context path here"
} else if (project.Environment == "dev") {
result = "DEV context path here"
}
return result
}
def getAppPropertiesPath = { ->
def result = "default"
if (project.Environment == "prod") {
result = "PROD app properties path here"
} else if (project.Environment == "uat") {
result = "UAT app properties path here"
} else if (project.Environment == "dev") {
result = "DEV app properties path here"
}
return result
}
Note that the ENV-specific values could easily be abstracted into config files, ENV vars, etc.
Also note that the example addresses: Java package used, project version, and build timestamp as well. I write something like this for most projects.

How to combine multiple Javadoc into one using Gradle

This question was answered before but the chosen answer doesn't explain a lot for me on how this is doable on Gradle.
That and the fact that I can't comment on the solution to ask for more info forced me to make this question.
I have a Gradle project that has several modules available and I now want to set up the Javadoc task to combine the Javadoc comments of all the modules into a single location where I could browse it.
How would I now be able to do this using Gradle? I run Gradle 5.5 if the version is of any importance and I have the following things set in the build.gradle file:
allprojects {
ext {
// Convenience method to configure Javadoc
configureJavadoc = { Object jDocConfig ->
jDocConfig.options {
it.author()
it.encoding = 'UTF-8'
it.memberLevel = JavadocMemberLevel.PROTECTED
if (it instanceof StandardJavadocDocletOptions) {
def opt = it as StandardJavadocDocletOptions
opt.links(
"https://docs.example.com/java/"
)
if (JavaVersion.current().isJava9Compatible()) {
opt.addBooleanOption("html5", true)
opt.addStringOption("-release", "8")
}
if (JavaVersion.current().isJava11Compatible()) {
opt.addBooleanOption("-no-module-directories", true)
}
}
}
}
}
}
subprojects {
javadoc {
destinationDir = file("$rootDir/docs/")
configureJavadoc(it)
}
}

I was able to do it with:
def exportedProjects = [
":",
":module-a",
":module-b",
":module-c"
]
task allJavadoc(type: Javadoc) {
source exportedProjects.collect { project(it).sourceSets.main.allJava }
classpath = files(exportedProjects.collect { project(it).sourceSets.main.compileClasspath })
destinationDir = file("${buildDir}/docs/javadoc-all")
}

Is it possible to check which gradle dependencies contains given class?

Recently we had a version mismatch problem with a class org.apache.commons.beanutils.PropertyUtilsBean. We thought that mismatch is just between some dependency that brings commons-beanutils in versions 1.8 and 1.9.3 but after tracking and excluding each transitive dependency we were still facing an issue.
It turns out that the the PropertyUtilsBean was also packed inside the commons-digester3-3.2-with-deps instead declared as dependency to commons-beanutils.
Is it possible in gradle to search all dependencies (including transitive ones) for specific fully qualified classname? That way we could resolve such problems on the spot.

I tried it and it is possible using some custom gradle build logic:
Kotlin DSL
tasks {
val searchClass by creating {
doLast {
configurations.forEach { // check all configurations
if (it.isCanBeResolved) {
try {
val classLoader = configToClassloader(it)
// replace here class you are looking for
val cl = Class.forName("arrow.core.Either", false, classLoader)
println("found in Configuration $it")
println(cl.protectionDomain.codeSource.location)
} catch (e: Exception) {}
}
}
}
}
}
// Helper function: convert a gradle configuration to ClassLoader
fun configToClassloader(config: Configuration) =
URLClassLoader(
config.files.map {
it.toURI().toURL()
}.toTypedArray())
This could be further enhanced by replacing the hard coded classname with some parameter mechanism.
Sample output:
> Task :searchClass
Configuration configuration ':domain:apiDependenciesMetadata'
file:/Users/abendt/.gradle/caches/modules-2/files-2.1/io.arrow-kt/arrow-core-data/0.9.0/a5b0228eebd5ee2f233f9aa9b9b624a32f84f328/arrow-core-data-0.9.0.jar
Groovy DSL
def configToClassloader(config) {
return new URLClassLoader(
*config.files.collect {
it.toURI().toURL()
}.toArray())
}
task searchClass {
doLast {
configurations.forEach { // check all configurations
if (it.canBeResolved) {
try {
def classLoader = configToClassloader(it)
// replace here class you are looking for
def cl = Class.forName("arrow.core.Either", false, classLoader)
println("found in Configuration $it")
println(cl.protectionDomain.codeSource.location)
} catch (e) {}
}
}
}
}
Edit: I have recently created a Gradle Plugin that provides the described tasks: https://plugins.gradle.org/plugin/io.github.redgreencoding.findclass

You could do this
task findJarsForClass {
doLast {
def findMe = 'org/apache/commons/beanutils/PropertyUtilsBean.class'
def matches = configurations.runtime.findAll { f ->
f.name.endsWith('.jar') &&
!(zipTree(f).matching { include findMe }.empty)
}
println "Found $findMe in ${matches*.name}"
}
}

Just ctrl+left click class name which was imported, then you can see the jar on your ide(eclipse has that feature, probably IntelliJ has as well)

Try using the task dependencyInsight :
gradle -q dependencyInsight --configuration compile --dependency commons-beanutils
Every Gradle project provides the task dependencyInsight to render the
so-called dependency insight report from the command line. Given a
dependency in the dependency graph you can identify the selection
reason and track down the origin of the dependency selection.

Getting location of cross-project resources

I have read many similar questions where the reply is that the project structure is not ideal so my questions based on the following:
I have a main project (ProjA) which needs to include a second project (ProjB) which is not a child project. ProjB has various resource files which need to be copied in the distribution of ProjA.
build.gradle of ProjA
dependencies {
compile project(":ProjB")
}
distributions {
main {
baseName = "Something"
contents {
into('bin') { from jar.archivePath }
into('lib') { from configurations.runtime }
into('etc') {
from ('../../projb/src/main/webapp') // Fix me!
}
}
}
}
1.) Ideally ProjB should expose the location of the resource files through a property used by ProjA, how can this be done?
2.) Is this the correct way to do it as I have read alot about cross-project properties not being ideal - or should I be doing something completely different?

Don't know if it helps but it seems that the best way is to do it in the following way:
distributions {
main {
baseName = "Something"
contents {
into('bin') { from jar.archivePath }
into('lib') { from configurations.runtime }
into('etc') {
from project(':projB').file('src/main/webapp')
}
}
}
}
The path must be hardcoded in that case.
Second option might be specifying a project property - in general not a very good idea - and use in another project - there must be also evaluation order defined.
In projB
ext.resourcesDir = project.file('src/main/webapp2')
and in projA
evaluationDependsOn(':projB')
and:
distributions {
main {
baseName = "Something"
contents {
into('bin') { from jar.archivePath }
into('lib') { from configurations.runtime }
into('etc') {
from project(':projB').file('src/main/webapp')
from project(':projB').resourcesDir
}
}
}
}
Here's complete example.