Develop Reference

Error while running Fortify on Gradle 7.2 project

I am getting an error while running Fortify 20 on a Gradle-Java Project. The project compiles smoothly with "gradle build" command, but when running Fortify I get this error:
Must not use executable property on ForkOptions together with javaCompiler property
The only clue I have is that this functionality was introduced since version 6.7 but the project was built on gradle 7.
I wonder if it is possible to inhibit one of the 2 things causing the error?

Looks like it might be related to Gradle.
https://github.com/gradle/gradle/blob/master/subprojects/language-java/src/main/java/org/gradle/api/tasks/compile/JavaCompile.java
If you look at the github code, you will see this method:
private void validateConfiguration() {
if (javaCompiler.isPresent()) {
checkState(getOptions().getForkOptions().getJavaHome() == null, "Must not use `javaHome` property on `ForkOptions` together with `javaCompiler` property");
checkState(getOptions().getForkOptions().getExecutable() == null, "Must not use `executable` property on `ForkOptions` together with `javaCompiler` property");
}
}
validateConfiguration() is called from createSpec() which is run during compile().
Please see example build.gradle below which uses java plugin and compileJava task, and options.forkOptions.executable:
https://docs.gradle.org/current/userguide/java_plugin.html#java_plugin
import com.nr.builder.JarUtil
apply plugin: 'java'
subprojects {
dependencies {
// introspector classes for testing externals
testImplementation(project(":instrumentation-test"))
}
}
ext.moduleName = "com.greetings"
dependencies {
implementation("junit:junit:4.13")
}
compileJava {
inputs.property("moduleName", "com.greetings")
doFirst {
options.compilerArgs = [
'--module-path', classpath.asPath,
'--patch-module', "$moduleName=" + files(sourceSets.main.java.srcDirs).asPath,
]
classpath = files()
}
}
compileJava.options.encoding = 'UTF-8'
compileJava.options.fork = true
// Compile with Java 11 to test module support
compileJava.options.forkOptions.executable = jdk11 + '/bin/javac'
compileJava.options.forkOptions.javaHome = new File(jdk11)
compileTestJava.options.encoding = 'UTF-8'
compileTestJava.options.fork = true
// Compile with Java 11 to test module support
compileTestJava.options.forkOptions.executable = jdk11 + '/bin/javac'
compileTestJava.options.forkOptions.javaHome = new File(jdk11)
// Boot classpath no longer works in JDK 9+ so we should ignore it here
compileJava.options.bootstrapClasspath = null
sourceCompatibility = JavaVersion.VERSION_11
targetCompatibility = JavaVersion.VERSION_11
def module_test_args = [
"-javaagent:${project.jar.archivePath.absolutePath}",
"-javaagent:${JarUtil.getNewRelicJar(project(":newrelic-agent")).absolutePath}",
"-Dnewrelic.config.file=${project(':newrelic-agent').projectDir}/src/test/resources/com/newrelic/agent/config/newrelic.yml",
"-Dnewrelic.unittest=true",
"-Dnewrelic.config.startup_log_level=warn",
"-Dnewrelic.debug=$newrelicDebug",
"--module-path=lib:out/production/classes",
"--add-modules com.greetings",
"--module junit/org.junit.runner.JUnitCore"
]
test {
dependsOn(project(":newrelic-agent").getTasksByName("newrelicJar", false))
forkEvery = 1
maxParallelForks = Runtime.runtime.availableProcessors()
executable = jdk11 + '/bin/java'
minHeapSize = "256m"
maxHeapSize = "256m"
beforeSuite {
descriptor ->
// We get two notifications per Test class. One of them is simply the Gradle executor used to run the test
// We filter that one out and only log the Test class name with this null check.
if (descriptor.getClassName() != null) {
logger.lifecycle("Running test suite: " + descriptor.getClassName())
}
}
}
javadoc {
onlyIf { JavaVersion.current().isJava11Compatible() }
}

If you want to keep using the Gradle toolchain feature, add the following to your build.gradle:
// Circumvents issues with Fortify scan without disabling Gradle toolchain feature
tasks.withType(JavaCompile).configureEach {
doFirst {
configure(options) {
configure(forkOptions) {
executable = null
javaHome = null
}
}
}
}

Writing a pipelineplugin in Jenkins

I am writing a Jenkinsplugin. I have set up a pipeline script, when I execute the script its calling some shell scripts and setting up a pipeline. Thats working fine.
Example of my code:
node('master') {
try {
def appWorkspace = './app/'
def testWorkspace = './tests/'
stage('Clean up') {
// cleanWs()
}
stage('Build') {
parallel (
app: {
dir(appWorkspace) {
git changelog: false, credentialsId: 'jenkins.git', poll: false, url: 'https://src.url/to/our/repo'
dir('./App') {
sh "#!/bin/bash -lx \n ./gradlew assembleRelease"
}
}
},
tests: {
dir(testWorkspace) {
git changelog: false, credentialsId: 'jenkins.git', poll: false, url: 'https://src.url/to/our/repo'
sh "#!/bin/bash -lx \n nuget restore ./Tests/MyProject/MyProject.sln"
sh "#!/bin/bash -lx \n msbuild ./Tests/MyProject/MyProject.Core/ /p:Configuration=Debug"
}
}
)
}
stage('Prepare') {
parallel (
'install-apk': {
sh '''#!/bin/bash -lx
result="$(adbExtendedVersion shell pm list packages packagename.app)"
if [ ! -z "$result" ]
then
adbExtendedVersion uninstall packagename.app
fi
adbExtendedVersion install ''' + appWorkspace + '''/path/to/app-release.apk'''
},
'start-appium': {
sh "#!/bin/bash -lx \n GetAllAttachedDevices.sh"
sh "sleep 20s"
}
)
}
stage('Test') {
// Reading content of the file
def portsFileContent = readFile 'file.txt'
// Split the file by next line
def ports = portsFileContent.split('\n')
// Getting device IDs to get properties of device
def deviceIDFileContent = readFile 'IDs.txt'
def deviceIDs = deviceIDFileContent.split('\n')
// Define port and id as an pair
def pairs = (0..<Math.min(ports.size(), deviceIDs.size())).collect { i -> [id: deviceIDs[i], port: ports[i]] }
def steps = pairs.collectEntries { pair ->
["UI Test on ${pair.id}", {
sh "#!/bin/bash -lx \n mono $testWorkspace/Tests/packages/NUnit.ConsoleRunner.3.7.0/tools/nunit3-console.exe $testWorkspace/Tests/bin/Debug/MyProject.Core.dll --params=port=${pair.port}"
}]
}
parallel steps
}
}
catch (Exception e) {
println(e);
}
finally {
stage('Clean') {
archiveArtifacts 'TestResult.xml'
sh "#!/bin/bash -lx \n KillInstance.sh"
}
}
}
This is a groovy script defining my pipeline. What I am trying to achieve with my plugin is, that the user who uses this plugin just inserts some pathvariables eg. path to his solution, or path to his github source. My Plugin then executes the above listed script automatically with the given parameters.
My problem is, that I cant find any documentation how to write such a pipeline construct in Java. If someone could point me in the right direction I would appreciate that.

Gradle task order for creating dynamic Global List for a Copy task exclude

Ok, I am still having an issue and would greatly appreciate someone shedding some light on what I am obviously missing.
I thought I understood the Gradle task I wanted to accomplish, but I cannot seem to get one last bit working.
Here is what I want to do, in numbered order:
Create a dynamic, global 'copyExcludes' list; in one task
Copy entire directory tree, excluding the 'copyExcludes' list; in another task
Run the maven command 'mvn verify' on every directory in the file tree from the Copy task
Create a ZIP archive for the directory tree after all the Maven commands complete successfully.
Below is the code, I have been hacking at, and cannot get the 'copyExcludes' list, to update, BEFORE the 'copy' task executes, thus ALL modules (sub directories) are copied over, not just the few that I want.
/**
* Copy all files to ${assemblyDir}, then run 'mvn verify' on each and every included module.
* Then create an assembly ZIP
*/
def copyFromDir = rootProject.file('.')
def assemblyDir = "target/gradle_assembly"
def assemblyDestination = "target/assembly"
def assemblyName = "assembly.zip"
def List includedModules = []
//---------------------------------------------------------------------------//
def allModules = [
'common',
'spring-hello-world',
'spring-configuration',
'spring-advanced-configuration',
'spring-jdbc',
'spring-jms',
'spring-batch',
'spring-batch-configuration'
]
def copyExcludes = []
//---------------------------------------------------------------------------//
/**
* Copy all files from root (./)
but exclude the module sub directories that are from 'getCopyExcludes(allModules)'
*/
task copyFiles(type: Copy) {
def cpExcludes = []
ext {
println "********** copyFiles ext{} **********"
println "1 ********** (${copyExcludes.size()}) copyExcludes: '${copyExcludes}' **********"
cpExcludes = copyExcludes
cpExcludes = getCopyExcludes(allModules)
}
doFirst {
println "********** copyFiles doFirst{} **********"
println "2 ********** (${copyExcludes.size()}) copyExcludes: '${copyExcludes}' **********"
cpExcludes = copyExcludes
cpExcludes = getCopyExcludes(allModules)
println "********** (${cpExcludes.size()}) cpExcludes: '${cpExcludes}' **********"
}
// doLast{
if (file(buildDir).exists()) {
delete "./target"
}
from("${copyFromDir}") {
into '.'
include '*'
include '**/*' //to include contents of a folder present inside Reports directory
include '**/*.*' //to include contents of a folder present inside Reports directory
exclude (getCopyExcludes(allModules))
}
into {
def outputDir = assemblyDir;
outputDir
}
// }
}
//---------------------------------------------------------------------------//
//task mavenVerify() {
task mavenVerify(dependsOn:copyFiles) {
println "\n\n********** mavenVerify **********\n"
def moduleDir = "${assemblyDir}/code"
doLast {
// For each module that is to be included,
allModules.each { module ->
if (file("${assemblyDir}/code/${module}").exists()) {
moduleDir = "${assemblyDir}/code/${module}"
exec {
args "-D${mavenRepo} -D${mavenOpt}"
workingDir "${moduleDir}"
def command = commandLine 'mvn', 'verify'
}
println "----------> END ${moduleDir} ----------\n"
}
}
}
return "success"
}
//---------------------------------------------------------------------------//
/**
* Create a ZIP file for the previoulsy validate code copied and verified.
*/
task assembleZip(type: Zip, dependsOn:mavenVerify){
println "********** assembleZip **********"
from "${assemblyDir}"
include '*'
include '**/*' //to include contents of a folder present inside Reports directory
include '**/*.*' //to include contents of a folder present inside Reports directory
archiveName "${assemblyName}"
destinationDir(file("${assemblyDestination}"))
}
//---------------------------------------------------------------------------//
task basic_jdbc_aop_mvc_rest << {
println "********** basic_jdbc_aop_mvc_rest **********"
def modulesToKeep = []
// doFirst{
ext {
// Create List
modulesToKeep.addAll(modules_basic())
modulesToKeep.addAll(modules_jdbc())
// println "********** (${modulesToKeep.size}) modulesToKeep: '${modulesToKeep}' **********"
// allModules.removeAll(modulesToKeep)
// includedModules = allModules
// println "********** (${includedModules.size()}) allModules (pre): '${includedModules/**/}' **********"
// copyExcludes = getCopyExcludes(allModules)
// println "********** (${copyExcludes.size()}) copyExcludes: '${copyExcludes}' **********"
}
// doFirst{
// println "********** (${allModules.size()}) allModules (pre): '${allModules}' **********"
// println "********** (${modulesToKeep.size}) modulesToKeep: '${modulesToKeep}' **********"
// includedModules = allModules.minus(modulesToKeep)
allModules.removeAll(modulesToKeep)
includedModules = allModules
println "********** (${includedModules.size()}) allModules (pre): '${includedModules/**/}' **********"
copyExcludes = getCopyExcludes(allModules)
// println "********** (${copyExcludes.size()}) copyExcludes: '${copyExcludes}' **********"
// }
}
def List getCopyExcludes(mtr) {
def newList = mtr.collect{ "**/${it}" }
newList.addAll('**/target',
'**/.git',
'**/*.iml',
'**/.idea',
'**/gradle', // #5
'**/.gradle',
)
// newList.addAll(modulesToRemoveExclude(mtr))
println "********** getCopyExcludes newList (${newList.size}): '${newList}' **********"
return newList
}
[copyFiles]*.shouldRunAfter basic_jdbc_aop_mvc_rest
//---------------------------------------------------------------------------//
def modules_basic(){
return [
// Basic
'common',
'spring-hello-world',
'spring-configuration',
'spring-advanced-configuration'
]
}
def modules_jdbc(){
return [
// JDBC
'spring-jdbc'
]
}
//--- THE END ---------------------------------------------------------------//

I think you may be confusing yourself with all the debug statements. Perhaps trimming it down to the bare minimum will help you see the problem better?
For example, it's hard to see problem areas like an "into ." inside a from clause and a strange looking "def List getCopyExcludes(mtr)" function declaration.
For example, here is a similar smaller script that does what you want it to do.
def copyFromDir = "source/gradle_assembly"
def assemblyDir = "target/gradle_assembly"
task copyFiles(type: Copy) {
from "${copyFromDir}"
into "${assemblyDir}"
exclude getCopyExcludes()
}
def getCopyExcludes() {
// exclude these 2 log files
return ["tomcat.log", "cargo.log"]
}
Perhaps from here, you can expand it out to your current version by adding each task back in one at a time while verifying it still works? If it breaks something, you'll know what caused it.

method of one task is automatically called by another task in gradle

I am facing the problem in build.gradle file. I am using gradle version 2.14.1. In my gradle build there is a function named ask(),I am using in task grun(type:JavaExec) {...}, which is automatically called by other tasks. I don't want this to happen. ask() function invokes gradle console to get input from user while I run gradle grun --no-daemon. But I am getting console in all gradle tasks like gradle compile --no-daemon , gradle grunGui --no-daemon or gradle grunTree --no-daemon and no other task is depended on this task except grun. This is my gradle code:
apply plugin: 'antlr'
repositories {
mavenCentral()
}
dependencies {
antlr 'org.antlr:antlr4:4.5.2' // using ANTLR v4
}
generateGrammarSource {
/*arguments += ["-visitor", "-long-messages"]*/
arguments += ["-visitor", "-no-listener"]
}
task compile(type: JavaCompile){
classpath = project.getConfigurations().getByName(AntlrPlugin.ANTLR_CONFIGURATION_NAME) + sourceSets.main.runtimeClasspath
source = fileTree(dir: 'src/main/', include: '*.java')
destinationDir = file('build/classes/main')
}
task grunGui (type:JavaExec){
classpath = project.getConfigurations().getByName(AntlrPlugin.ANTLR_CONFIGURATION_NAME) + sourceSets.main.runtimeClasspath
main = 'org.antlr.v4.gui.TestRig'
def grammarName = "Wolf"
args = [grammarName, 'init', '-gui', "src/main/Wolf/first.wlf"]
}
task grunTokens (type:JavaExec){
classpath = project.getConfigurations().getByName(AntlrPlugin.ANTLR_CONFIGURATION_NAME) + sourceSets.main.runtimeClasspath
main = 'org.antlr.v4.gui.TestRig'
def grammarName = "Wolf"
args = [grammarName, 'init', '-tokens', "src/main/Wolf/first.wlf"]
}
task grunTree (type:JavaExec){
classpath = project.getConfigurations().getByName(AntlrPlugin.ANTLR_CONFIGURATION_NAME) + sourceSets.main.runtimeClasspath
main = 'org.antlr.v4.gui.TestRig'
def grammarName = "Wolf"
args = [grammarName, 'init', '-tree', "src/main/Wolf/first.wlf"]
}
def ask() {
def console = System.console()
if (console)
return console.readLine('\n> Please enter input string or filename to parse OR press enter⏎ to skip:\n')
else
logger.error "Cannot get console. Using default inputFile src/main/Wolf/first.wlf"
return ''
}
task grun(type:JavaExec) {
classpath = project.getConfigurations().getByName(AntlrPlugin.ANTLR_CONFIGURATION_NAME) + sourceSets.main.runtimeClasspath
main = 'Wolf'
def input = ask()
args = input.length() > 0 ? [input] : ["src/main/Wolf/first.wlf"]
}
Console is coming while calling compile task

Maven-publish cannot not apply withXML on pom file (No signature of method)

Ok, so I am having the exact issue as described here:
Android library dependencies missing from POM with Gradle
I copied the provided answer to my gradle file as follows:
publishing {
publications {
mavenAar(MavenPublication) {
groupId group
artifactId 'exampleId'
version version
artifact source: file('build/outputs/aar/example-release.aar')
//The publication doesn't know about our dependencies, so we have to manually add them to the pom
pom.withXml {
// for dependencies and exclusions
def dependenciesNode = asNode().appendNode('dependencies')
configurations.compile.allDependencies.each { ModuleDependency dp ->
def dependencyNode = dependenciesNode.appendNode('dependency')
dependencyNode.appendNode('groupId', dp.group)
dependencyNode.appendNode('artifactId', dp.name)
dependencyNode.appendNode('version', dp.version)
// for exclusions
if (dp.excludeRules.size() > 0) {
def exclusions = dependencyNode.appendNode('exclusions')
dp.excludeRules.each { ExcludeRule ex ->
def exclusion = exclusions.appendNode('exclusion')
exclusion.appendNode('groupId', ex.group)
exclusion.appendNode('artifactId', ex.module)
}
}
}
}
}
}
repositories {
mavenLocal()
maven {
url selectDeploymentURL()
if ( project.hasProperty( 'nexusUser' ) ) {
credentials {
username project.getProperty('nexusUser')
password project.getProperty('password')
}
}
}
}
}
However I am getting an error when attempting to publishToMavenLocal, and I'm not sure why?
FAILURE: Build failed with an exception.
* Where:
Build file 'path/build.gradle' line: 136
* What went wrong:
Execution failed for task ':example:generatePomFileForMavenAarPublication'.
> Could not apply withXml() to generated POM
> No signature of method: build_42xq5bsii69isvukzktk1oy51$_run_closure5_closure19_closure21_closure23_closure24.doCall() is applicable for argument types: (org.gradle.api.internal.artifacts.dependencies.DefaultSelfResolvingDependency_Decorated) values: [org.gradle.api.internal.artifacts.dependencies.DefaultSelfResolvingDependency_Decorated#433edba9]
Possible solutions: doCall(org.gradle.api.artifacts.ModuleDependency), findAll(), findAll(), isCase(java.lang.Object), isCase(java.lang.Object)
Note: Line 136 is this line:
configurations.compile.allDependencies.each { ModuleDependency dp ->

So as Karma has it, I found the answer shortly after posting
I just removed the ModuleDependency dp and referred to everything from it, and it got past this error, I see the new nodes in the pom file now.
configurations.compile.allDependencies.each {
def dependencyNode = dependenciesNode.appendNode('dependency')
dependencyNode.appendNode('groupId', it.group)
dependencyNode.appendNode('artifactId', it.name)
dependencyNode.appendNode('version', it.version)

The root cause of the crash is that not all dependencies are ModuleDependency objects. Something like the following should fix the problem and lets you still add exclusions (which aren't on the base class).
configurations.compile.allDependencies.each { dp ->
if (dp instanceof ModuleDependency) {
// do stuff
}
}

Compile is deprecated we have to use implementation.
for Example
configurations.implementation.allDependencies.each {
if(it.group != null && (it.name != null || "unspecified".equals(it.name)) && it.version != null)
{
def dependencyNode = dependenciesNode.appendNode('dependency')
dependencyNode.appendNode('groupId', it.group)
dependencyNode.appendNode('artifactId', it.name)
dependencyNode.appendNode('version', it.version)
}
}