Develop Reference

How to run a Gradle task on a compiled jar without having its sources?

I have a compiled jar with JUnit tests that I want to run from a docker container.
I want to do it with a Gradle task.
First, I will compile the jar and copy it with all its dependencies to a Gradle-based image.
(Or I can create a fat jar which will contain all the third party compiled to a .class).
Then I want to run the task - this task will only run tests according to a test name, JUnit tag, etc.
Is it possible to run a Gradle task on a compiled jar without having its sources?
What should I include in this image beside the gradle.build file for it to work?
Thank you

Gradle fetch transitive dependencies and run its tests in test process by default, so you can use this feature in your case.
Note that abstract classes are not executed. In addition, be aware that Gradle scans up the inheritance tree into jar files on the test classpath. So if those JARs contain test classes, they will also be run.
Gradle tests detection document.
Here is how to do it in your case :
Create an empty Gradle project and apply the java plugin.
import the test dependencies tools with the needed scopes in the dependencies section.
import myApp.jar as a local dependency.
Configure the test task (add needed properties and args).
Run Gradle test with the specific properties.
build.gradle example :
apply plugin: 'java'
repositories {
mavenCentral()
}
dependencies {
// just examples change with needed unit tests dependencies
testImplementation('org.junit.jupiter:junit-jupiter-api:5.4.2')
testRuntime('org.junit.jupiter:junit-jupiter-engine:5.4.2')
// the target jar file
compile file("/myApp.jar")
}
project.ext.testName = project.hasProperty("testName") ?
project.property("testName") : "*"
test {
useJUnitPlatform()
filter {
//include specific method in any of the tests
includeTestsMatching "$testName"
}
}
Now the Gradle test command will run the target tests in myApp.jar.
For more information about it check the Testing in Java & JVM projects official Gradle documents

How to include a custom .jar file in a java gradle build?

I have a gradle project to build a .jar file from Java sources. Some of the classes it needs are in a separate .jar file (let's called it helper.jar). To build helper.jar there is an external build system that I can invoke via a shell script.
If I manually run the shell script first to create helper.jar, and then add the file as a top-level dependency:
dependencies {
....
implementation files('/path/to/helper.jar')
Then everything compiles ok.
I am trying to convert this to automatically run the shell script to build helper.jar if necessary. I removed the file from the dependencies above and tried adding this task:
tasks.register('buildHelperJar', Exec) {
workingDir '/path/to/helper/root'
commandLine 'build_helper.sh'
outputs.file file('/path/to/helper.jar')
}
And then add it as a dependency:
project.afterEvaluate { proj ->
tasks.findByName('compileReleaseJavaWithJavac').dependsOn('buildHelperJar')
}
This seems to invoke the buildHelperJar task ok (and helper.jar gets built ok) but the subsequent compileReleaseJavaWithJavac task fails with missing symbol errors for everything that should be in helper.jar. So it's like I'm not actually adding the jar file itself into the compile step. What should I be doing to achieve this? Thanks a lot

Migrating from ant to gradle - macrodef alternatives?

We are migrating from Ant to Gradle. In the starting Gradle looked promising, when I started migrating targets from Ant to Gradle, facing similar issues, earlier having with Ant.
The Ant Build Flow looks like -
We have a Java application - Ant used to clean, compile and build jar of the project.
Next we have few targets to copy built jar and some other resources files to windows server.Its just a windows server not a nexus like repo.
These copy targets are release based(internally using macrodefs) - if alpha, it has to copy to different folder and for stable release different folder.
Another macrodef was to just backup the existing files present in server to a backup folder before copying new files there.
Migrating to Gradle-
I liked the way Gradle does the tasks of clean, compile and build. I am using Java plugin for this.
I am facing issues in next tasks to copy files to windows server and taking backup of old files before copying new files.
In Gradle I created few custom tasks like copyJarAlpha, copyJarStable,backupJarAlpha,backupJarStable etc..
Internally they are doing the same thing just copying files from source to destination and some print statements. But it looks repetitive. I was looking for ways like macrodef in ant.
task deployAlpha {
doFirst {
isJenkinsAvailable(deployDirAlpha)
}
doFirst {
if(isDeployLibsTaskRequired(outputDir,deployDirAlpha)) {
tasks.findByName('deployLibsAlpha').execute()
}
}
doLast {
println "Deploying corejar jar to : ${deployDirAlpha}"
copy {
from "${outputDir}/${corejar}"
into "${deployDirAlpha}"
}
println "Deploying evan-core jar to : ${deployDirAlpha}/lib"
copy {
from "${externalLibsDir}/core2.jar"
into "${deployDirAlpha}/lib"
}
println "Deploying test resource files to : ${deployDirAlpha}"
copy {
from "./test-resources/logback-test.xml", "./test-resources/JUnit-TestSuiteResults.xsl"
into "${deployDirAlpha}"
}
}
}
I have similar snippets at few places. Its not looking nice.
Is there any way we can reuse methods/functions in gradle? or I am assuming wrong things here.. Or I am trying to use gradle for things its not inteded for? What could be the best way to accomplish these tasks of copying and backing up files if not using custom tasks in gradle?

Here's a couple of options.
Create a plugin (See writing custom plugins)
public class MyPlugin implements Plugin<Project> {
void apply(Project project) {
project.with {
apply plugin: 'x'
dependencies {...}
task deployAlpha{...}
}
}
}
Usage
apply plugin: MyPlugin
Or a script plugin
apply from: 'path/to/script.gradle'
You can also attach functions to domain objects via the ext of a domain object (see ExtraPropertiesExtension)
allprojects {
ext.doProjectStuff = { String arg -> return "project${arg}"}
dependencies.ext.doSependenciesStuff = { String arg -> return "dependencies${arg}"}
}
Usage
def foo = doProjectStuff('x')
dependencies {
def bar = doDependenciesStuff('y')
}

How to make gradle download dependencies without actually building things

On a new environment gradle build takes quite a while because all dependencies have to be downloaded.
Is there a way to only download dependencies in order to speed up the following build?
That way we could for example already prefill a CI build environment.

Edit: Updated for Gradle 6+.
Some notes:
This new approach downloads jars into a folder, and then deletes the folder. So the result of having the jars in the Gradle cache is a side-effect.
It currently uses jars configured for the main source-set but could be generalized.
Even though it is neither efficient nor elegant, it can be useful if you actually want the jars (and transitive dependencies): simply comment-out the deletion of the runtime folder.
This solution can be handy when you want the jars (and transitive dependencies), as you simply have to comment-out deleting the folder.
Consider this build.gradle (as an arbitrary, concrete example):
apply plugin: 'java'
dependencies {
implementation 'org.apache.commons:commons-io:1.3.2'
implementation 'org.kie.modules:org-apache-commons-lang3:6.2.0.Beta2'
}
repositories {
jcenter()
}
task getDeps(type: Copy) {
from sourceSets.main.runtimeClasspath
into 'runtime/'
doFirst {
ant.delete(dir: 'runtime')
ant.mkdir(dir: 'runtime')
}
doLast {
ant.delete(dir: 'runtime')
}
}
Example run:
$ find /Users/measter/.gradle/caches -name "commons-io*1.3.2.jar"
$ gradle getDeps
$ find /Users/measter/.gradle/caches -name "commons-io*1.3.2.jar"
/Users/measter/.gradle/caches/modules-2/files-2.1/commons-io/commons-io/1.3.2/[snip]/commons-io-1.3.2.jar

I've found ./gradlew dependencies (as suggested by this user) to be very handy for Docker builds.

You can create a custom task that resolves all the configurations( in doing so, it will also download the dependencies without building the project)
task downloadDependencies {
doLast {
configurations.findAll{it.canBeResolved}.each{it.resolve()}
}
}
Run command ./gradlew downloadDependencies

My answer will favor the gradle plugins and built-in tasks.
I would use "gradle assemble" in the command-line.
It is a minor version of "gradle build".
This way, you may reduce the time of your preparations before running or building anything.
Check the link bellow for the documentation:
https://docs.gradle.org/current/userguide/java_plugin.html#lifecycle_tasks
In general, what is my recipe when I clone a new repository:
-gradle assemble
-do some coding
-gradle run (and basically test until done)
-gradle build (to make distributable files)
note: this last step may have adicional configurations for .jar files as outputs (depends on you).

Gradle exclude java class from lib replaced by own class to avoid duplicate

In Android Studio, there is a specific file (src/org/luaj/vm2/lib/jse/JavaMethod.java) that I need to overwrite from a package that is pulled in via Gradle (dependencies {compile 'org.luaj:luaj-jse:3.0.1'}).
I copied the file into my source directory with the exact same path and made my changes to it. This was working fine for an individual JUnit test case that was using it. It also looks like it is working for a normal compile of my project (unable to easily confirm at the moment).
However, when I try to run all my tests at once via a configuration of ProjectType="Android Tests", I get Error:Error converting bytecode to dex:
Cause: com.android.dex.DexException: Multiple dex files define Lorg/luaj/vm2/lib/jse/JavaMethod$Overload;.
Is there a specific task or command that I need to add to my Gradle file to make sure the project selects the file in my local source directory? I tried the Copy task and the sourceSets->main->java->exclude command, but neither seemed to work (I may have done them wrong). I also tried the "exclude module/group" directive under "compile" from this post.
The non-default settings for the Run/Debug Confirmation:
Type=Android Tests
Module=My module
Test: All in package
Package: "test"
All my JUnit test cases are in the "test" package.
Any answer that gets this to work is fine. If not Gradle, perhaps something in the android manifest or the local source file itself.
[Edit on 2016-07-24]
The error is also happening on a normal compile when my android emulator is running lower APIs. API 16 and 19 error out, but API 23 does not.

issue: when linking your app the linker finds two versions
org.luaj:luaj-jse:3.0.1:org.luaj.vm2.lib.jse.JavaMethod and
{localProject}:org.luaj.vm2.lib.jse.JavaMethod
howto fix: tell gradle to exclude org.luaj:luaj-jse:3.0.1:org.luaj.vm2.lib.jse.JavaMethod from building
android {
packagingOptions {
exclude '**/JavaMethod.class'
}
}
I have not tried this with "exclude class" but it works for removing duplicate gpl license files a la "COPYING".
If this "exclude" does not work you can
download the lib org.luaj:luaj-jse:3.0.1 to the local libs folder,
open jar/aar with a zip-app and manually remove the duplicate class.
remove org.luaj:luaj-jse:3.0.1 from dependencies since this is now loaded from lib folder

I am not completely sure I understand your problem; however, it sounds like a classpath ordering issue, not really a file overwrite one.
AFAIK, gradle does not make a 'guarantee' on the ordering from a 'dependencies' section, save for that it will be repeatable. As you are compiling a version of file that you want to customize, to make your test/system use that file, it must come earlier in the classpath than the jar file it is duplicated from.
Fortunately, gradle does allow a fairly easy method of 'prepending' to the classpath:
sourceSets.main.compileClasspath = file("path/to/builddir/named/classes") + sourceSets.main.compileClasspath
I don't know enough about your system to define that better. However, you should be able to easily customize to your needs. That is, you can change the 'compile' to one of the other classpath (runtime, testRuntime, etc) if needed. Also, you can specify the jarfile you build rather than the classes directory if that is better solution. Just remember, it may not be optimal, but it is fairly harmless to have something specified twice in the classpath definition.

This is rather convoluted but it is technically feasible. However it's not a single task as asked by the poster:
Exclude said dependency from build.gradle and make sure it's not indirectly included by another jar (hint: use ./gradlew dependencies to check it)
create a gradle task that downloads said dependency in a known folder
unpack such jar, remove offending .class file
include folder as compile dependency
If it's safe to assume that you're using Linux/Mac you can run a simple command line on item 3, it's only using widely available commands:
mkdir newFolder ; cd newFolder ; jar xf $filename ; rm $offendingFilePath
If you don't care about automatic dependency management you can download the jar file with curl, which I believe to be widely available on both linux and mac.
curl http://somehost.com/some.jar -o some.jar
For a more robust implementation you can substitute such simple command lines with groovy/java code. It's interesting to know that gradle can be seen as a superset of groovy, which is arguable a superset of java in most ways. That means you can put java/groovy code pretty much anywhere into a gradle.build file. It's not clean but it's effective, and it's just another option.
For 4 you can have something along either
sourceSets.main.java.srcDirs += ["newFolder/class"]
at the root level of build.gradle, or
dependencies {
. . .
compile fileTree(dir: 'newFolder', include: ['*.class'])
. . .

This is what I ended up adding after Fabio's suggestion:
//Get LUAJ
buildscript { dependencies { classpath 'de.undercouch:gradle-download-task:3.1.1' }}
apply plugin: 'de.undercouch.download'
task GetLuaJ {
//Configure
def JARDownloadURL='http://central.maven.org/maven2/org/luaj/luaj-jse/3.0.1/luaj-jse-3.0.1.jar' //compile 'org.luaj:luaj-jse:3.0.1'
def BaseDir="$projectDir/luaj"
def ExtractToDir='class'
def ConfirmAlreadyDownloadedFile="$BaseDir/$ExtractToDir/lua.class"
def JarFileName=JARDownloadURL.substring(JARDownloadURL.lastIndexOf('/')+1)
def ClassesToDeleteDir="$BaseDir/$ExtractToDir/org/luaj/vm2/lib/jse"
def ClassNamesToDelete=["JavaMethod", "LuajavaLib"]
//Only run if LuaJ does not already exist
if (!file(ConfirmAlreadyDownloadedFile).exists()) {
//Download and extract the source files to /luaj
println 'Setting up LuaJ' //TODO: For some reason, print statements are not working when the "copy" directive is included below
mkdir BaseDir
download {
src JARDownloadURL
dest BaseDir
}
copy {
from(zipTree("$BaseDir/$JarFileName"))
into("$BaseDir/$ExtractToDir")
}
//Remove the unneeded class files
ClassNamesToDelete=ClassNamesToDelete.join("|")
file(ClassesToDeleteDir).listFiles().each {
if(it.getPath().replace('\\', '/').matches('^.*?/(?:'+ClassNamesToDelete+')[^/]*\\.class$')) {
println "Deleting: $it"
it.delete()
}
}
}
}
I'll upload a version that works directly with the jar later.

Another solution if we got then source jar:
task downloadAndCopy {
def downloadDir = "${buildDir}/downloads"
def generatedSrcDir = "${buildDir}/depSrc"
copy {
from(configurations.detachedConfiguration(dependencies.add('implementation', 'xxx:source')))
file(downloadDir).mkdirs()
into(downloadDir)
}
println("downloading file into ${downloadDir}")
fileTree(downloadDir).visit { FileVisitDetails details ->
if (!details.file.name.endsWith("jar")) {
println("ignore ${details.file.name}")
return
}
println("downloaded ${details.file.name}")
def srcFiles = zipTree(details.file).matching {
include "**/*.java"
exclude "**/NeedEclude*java"
}
srcFiles.visit {FileVisitDetails sourceFile ->
println("include ${sourceFile}")
}
copy {
from(srcFiles)
into(generatedSrcDir)
}
}
}
and remember to add depSrc to srcDirs
android {
sourceSets {
`main.java.srcDirs = ['src/main/java', "${buildDir}/depSrc"]
}
}