I am encountering an issue when I try to build a fat jar file with gradle. Is there a way to configure gradle to support multiple JDBC drivers in the JAR file? I noticed in my META-INF/services/java.sqlDriver that it will take whatever JDBC library comes first. So when running the jar file I would often see this error.
java.sql.SQLException: No suitable driver
at java.sql/java.sql.DriverManager.getDriver(DriverManager.java:299)
I see that the solution to this issue revolves around adding Class.forName('jdbcDriver') before I initialize my db connections, but I see that for JTDS libraries you do not need that anymore because of JDBC type 4 can automatically detect which driver you need based on the connection string.
My build.gradle file
plugins {
id("application")
id("com.github.johnrengelman.shadow") version "7.1.0"
}
dependencies {
implementation group: 'net.sourceforge.jtds', name: 'jtds', version: '1.3.1'
implementation group: 'mysql', name: 'mysql-connector-java', version: '8.0.26'
}
My connection string
jdbc:jtds:sqlserver://localhost:1111/db;user=TEST;password=DUMMY
I am using Java 17 and Gradle 7.3.3
You need to configure the Gradle Shadow Plugin to merge the provider-configuration files. How to do that is documented here:
Merging Service Descriptor Files
Java libraries often contain service descriptors files in the META-INF/services directory of the JAR. A service descriptor typically contains a line delimited list of classes that are supported for a particular service. At runtime, this file is read and used to configure library or application behavior.
Multiple dependencies may use the same service descriptor file name. In this case, it is generally desired to merge the content of each instance of the file into a single output file. The ServiceFileTransformer class is used to perform this merging. By default, it will merge each copy of a file under META-INF/services into a single file in the output JAR.
// Merging Service Files
shadowJar {
mergeServiceFiles()
}
The above code snippet is a convenience syntax for calling transform(ServiceFileTransformer.class)
If using the Kotlin DSL, it would look like:
tasks {
shadowJar {
mergeServiceFiles()
}
}
Or:
tasks.shadowJar {
mergeServiceFiles()
}
Related
i am new to gradle and i am trying to create several applications/distributions of a java application, to be used in different operating systems.
I need one application/distro for each: Windows64,Windows32,Linux64,Linux32.
The difference between these are that there are different dependencies (jar files from external projects) for each distro, and also some dll files must be included in some distros.
Finally, windows distros do not need unix build scripts and vice versa.
I've tried reading the gradle user guide and searching for answers online, but i do not have any experience in this and i don't know what to look for. Any help?
You can use different build script (windows64.gradle, windows32.gradle, linux64.gradle, linux32.gradle etc...) and load them thanks to a specific project property
def defaultPlatform = 'windows64'
ext.platform = project.hasProperty('pf') ? pf : defaultPlatform
apply from: "${ext.platform}.gradle"
And then just run gradle -Ppf=linux64 build or gradle -Ppf=linux64 yourCustomTask
In each .gradle file, add specific dependencies etc...
You could likely use my java-flavours plugin
Eg:
ext {
someApiVersion = '1.0'
}
plugins {
id "com.lazan.javaflavours" version "1.2"
}
javaFlavours {
flavour 'windows32'
flavour 'windows64'
flavour 'linux32'
flavour 'linux64'
}
dependencies {
compile "log4j:log4j:1.2.17"
compile "foo:xxx-interfaces:$someApiVersion"
compileWindows32 "foo:xxx-windows32:$someApiVersion"
compileWindows64 "foo:xxx-windows64:$someApiVersion"
// etc
}
Each flavour will produce a separate jar by joining common sources (src/main/java and src/main/resources) with flavour specific sources (src/$flavour/java and src/$flavour/resources). There's also support for flavour specific tests.
In my projects, I need to import third party jar file and Facebook SDK.
compile files('libs/SkinSDK.jar')
compile 'com.facebook.android:facebook-android-sdk:4.14.0'
Both include same BundleJSONConverter class. So, I cannot do generate signed APK. It always shows duplicate entry com/facebook/internal/BundleJSONConverter.
So, I want to exclude in Facebook or SkinSDK.jar. I tried like
compile ('com.facebook.android:facebook-android-sdk:4.14.0') {
exclude group: 'com.facebook.internal', module: 'BundleJSONConverter'
}
It's not working and showing same error.
The exclude method of the configuration closure for a dependency excludes transitive dependencies. So, if your module dependency depends on other modules, you can exclude them from your build. You can check out the transitive dependencies of the 'com.facebook.android:facebook-android-sdk:4.14.0' module on its Maven repository info page.
If the BundleJSONConverter class exists in a transitive dependency, you can exclude the specific module in the same way you are trying now. Just specify the group, the module and the version, like you do for dependencies.
If you just want to exclude one class for a dependency jar, take a look at the jar jar links tool and its Gradle plugin. It allows you to alter included jars, e.g. to change packages or remove classes.
The following (shortened) example shows the usage of the plugin and some methods to alter the dependency jar:
compile jarjar.repackage {
from 'org.apache.hive:hive-exec:0.13.0.2.1.5.0-695'
archiveBypass "commons*.jar"
archiveExclude "slf4j*.jar"
classDelete "org.apache.thrift.**"
classRename 'org.json.**', 'org.anarres.hive.json.#1'
}
Bumped into similar situation. This is what I did, not elegant as I hoped, but it works:
Rename the jar file (SkinSDK.jar in your case): .zip instead of .jar
Go "inside" the zip file (I'm using DoubleCommander, there are many other utilities for that), or extract it to a temporary folder.
Delete the duplicate class that causes the problem. Go "outside" the zip file.
Rename (or re-pack) the file from .zip to .jar . Compile.
Hope it works...
I had a similar problem with duplicated classes after importing a jar. In my case, the conflict was between a class in that jar and a class in my own project.
Below I share the solution you can use to discard classes that you have available in your own source tree, assuming the one in the jar is the right one to use:
android {
sourceSets {
main {
java {
filter.excludes = [
"com/package/Duplicated.java",
]
}
}
}
}
I am currently about to migrate the build system for my project from ANT to gradle. So I am pretty new to gradle My project features a plugin mechanism that dynamically loads jars configured in an XML file. Each plugin is in its own subproject. The entries of the XML configuration look as follows
<?xml version="1.0" encoding="UTF-8"?>
<implementation ...>
<repository>text</repository>
<classpath>library1.jar</classpath>
<classpath>library2.jar</classpath>
...
</implementation>
I added an XSL transformation to prefix each entry with the correct lib folder in the build folder to make it usable in a run task of my main project.
This already works OK for the libraries that are created within my project. However, some of them rely on third party libraries, e.g., protobuf.
For now, I solved this by copying all referenced jars to the lib folder. My build.gradle looks as follows
dependencies {
compile group: 'com.google.protobuf', name: 'protobuf-java', version: '3.1.0'
...
compile project(':main')
}
task copyReferencedLibraries(type: Copy) {
into "$buildDir/libs"
from configurations.compile
}
jar {
dependsOn xslt
dependsOn copyReferencedLibraries
...
}
However, this copies all libraries, including those inherited from the main project to the lib folder. More than I actually need.
How could I filter the libraries to those defined in this subproject or at least manually filter the ones that I want to be copied?
You could just filter in your Copy task
task copyReferencedLibraries(type: Copy) {
into "$buildDir/libs"
from(configurations.compile) {
exclude 'some.jar', 'some-other.jar'
include 'some-required.jar', 'some-other-required.jar'
}
}
Reference: https://docs.gradle.org/current/dsl/org.gradle.api.tasks.Copy.html
Just be careful you don't exclude second or third tier dependency (ie a jar required by a jar your using). If you do then you won't get compile errors you will get runtime errors when the code is executed that touches the missing dependency.
In Android Studio, there is a specific file (src/org/luaj/vm2/lib/jse/JavaMethod.java) that I need to overwrite from a package that is pulled in via Gradle (dependencies {compile 'org.luaj:luaj-jse:3.0.1'}).
I copied the file into my source directory with the exact same path and made my changes to it. This was working fine for an individual JUnit test case that was using it. It also looks like it is working for a normal compile of my project (unable to easily confirm at the moment).
However, when I try to run all my tests at once via a configuration of ProjectType="Android Tests", I get Error:Error converting bytecode to dex:
Cause: com.android.dex.DexException: Multiple dex files define Lorg/luaj/vm2/lib/jse/JavaMethod$Overload;.
Is there a specific task or command that I need to add to my Gradle file to make sure the project selects the file in my local source directory? I tried the Copy task and the sourceSets->main->java->exclude command, but neither seemed to work (I may have done them wrong). I also tried the "exclude module/group" directive under "compile" from this post.
The non-default settings for the Run/Debug Confirmation:
Type=Android Tests
Module=My module
Test: All in package
Package: "test"
All my JUnit test cases are in the "test" package.
Any answer that gets this to work is fine. If not Gradle, perhaps something in the android manifest or the local source file itself.
[Edit on 2016-07-24]
The error is also happening on a normal compile when my android emulator is running lower APIs. API 16 and 19 error out, but API 23 does not.
issue: when linking your app the linker finds two versions
org.luaj:luaj-jse:3.0.1:org.luaj.vm2.lib.jse.JavaMethod and
{localProject}:org.luaj.vm2.lib.jse.JavaMethod
howto fix: tell gradle to exclude org.luaj:luaj-jse:3.0.1:org.luaj.vm2.lib.jse.JavaMethod from building
android {
packagingOptions {
exclude '**/JavaMethod.class'
}
}
I have not tried this with "exclude class" but it works for removing duplicate gpl license files a la "COPYING".
If this "exclude" does not work you can
download the lib org.luaj:luaj-jse:3.0.1 to the local libs folder,
open jar/aar with a zip-app and manually remove the duplicate class.
remove org.luaj:luaj-jse:3.0.1 from dependencies since this is now loaded from lib folder
I am not completely sure I understand your problem; however, it sounds like a classpath ordering issue, not really a file overwrite one.
AFAIK, gradle does not make a 'guarantee' on the ordering from a 'dependencies' section, save for that it will be repeatable. As you are compiling a version of file that you want to customize, to make your test/system use that file, it must come earlier in the classpath than the jar file it is duplicated from.
Fortunately, gradle does allow a fairly easy method of 'prepending' to the classpath:
sourceSets.main.compileClasspath = file("path/to/builddir/named/classes") + sourceSets.main.compileClasspath
I don't know enough about your system to define that better. However, you should be able to easily customize to your needs. That is, you can change the 'compile' to one of the other classpath (runtime, testRuntime, etc) if needed. Also, you can specify the jarfile you build rather than the classes directory if that is better solution. Just remember, it may not be optimal, but it is fairly harmless to have something specified twice in the classpath definition.
This is rather convoluted but it is technically feasible. However it's not a single task as asked by the poster:
Exclude said dependency from build.gradle and make sure it's not indirectly included by another jar (hint: use ./gradlew dependencies to check it)
create a gradle task that downloads said dependency in a known folder
unpack such jar, remove offending .class file
include folder as compile dependency
If it's safe to assume that you're using Linux/Mac you can run a simple command line on item 3, it's only using widely available commands:
mkdir newFolder ; cd newFolder ; jar xf $filename ; rm $offendingFilePath
If you don't care about automatic dependency management you can download the jar file with curl, which I believe to be widely available on both linux and mac.
curl http://somehost.com/some.jar -o some.jar
For a more robust implementation you can substitute such simple command lines with groovy/java code. It's interesting to know that gradle can be seen as a superset of groovy, which is arguable a superset of java in most ways. That means you can put java/groovy code pretty much anywhere into a gradle.build file. It's not clean but it's effective, and it's just another option.
For 4 you can have something along either
sourceSets.main.java.srcDirs += ["newFolder/class"]
at the root level of build.gradle, or
dependencies {
. . .
compile fileTree(dir: 'newFolder', include: ['*.class'])
. . .
This is what I ended up adding after Fabio's suggestion:
//Get LUAJ
buildscript { dependencies { classpath 'de.undercouch:gradle-download-task:3.1.1' }}
apply plugin: 'de.undercouch.download'
task GetLuaJ {
//Configure
def JARDownloadURL='http://central.maven.org/maven2/org/luaj/luaj-jse/3.0.1/luaj-jse-3.0.1.jar' //compile 'org.luaj:luaj-jse:3.0.1'
def BaseDir="$projectDir/luaj"
def ExtractToDir='class'
def ConfirmAlreadyDownloadedFile="$BaseDir/$ExtractToDir/lua.class"
def JarFileName=JARDownloadURL.substring(JARDownloadURL.lastIndexOf('/')+1)
def ClassesToDeleteDir="$BaseDir/$ExtractToDir/org/luaj/vm2/lib/jse"
def ClassNamesToDelete=["JavaMethod", "LuajavaLib"]
//Only run if LuaJ does not already exist
if (!file(ConfirmAlreadyDownloadedFile).exists()) {
//Download and extract the source files to /luaj
println 'Setting up LuaJ' //TODO: For some reason, print statements are not working when the "copy" directive is included below
mkdir BaseDir
download {
src JARDownloadURL
dest BaseDir
}
copy {
from(zipTree("$BaseDir/$JarFileName"))
into("$BaseDir/$ExtractToDir")
}
//Remove the unneeded class files
ClassNamesToDelete=ClassNamesToDelete.join("|")
file(ClassesToDeleteDir).listFiles().each {
if(it.getPath().replace('\\', '/').matches('^.*?/(?:'+ClassNamesToDelete+')[^/]*\\.class$')) {
println "Deleting: $it"
it.delete()
}
}
}
}
I'll upload a version that works directly with the jar later.
Another solution if we got then source jar:
task downloadAndCopy {
def downloadDir = "${buildDir}/downloads"
def generatedSrcDir = "${buildDir}/depSrc"
copy {
from(configurations.detachedConfiguration(dependencies.add('implementation', 'xxx:source')))
file(downloadDir).mkdirs()
into(downloadDir)
}
println("downloading file into ${downloadDir}")
fileTree(downloadDir).visit { FileVisitDetails details ->
if (!details.file.name.endsWith("jar")) {
println("ignore ${details.file.name}")
return
}
println("downloaded ${details.file.name}")
def srcFiles = zipTree(details.file).matching {
include "**/*.java"
exclude "**/NeedEclude*java"
}
srcFiles.visit {FileVisitDetails sourceFile ->
println("include ${sourceFile}")
}
copy {
from(srcFiles)
into(generatedSrcDir)
}
}
}
and remember to add depSrc to srcDirs
android {
sourceSets {
`main.java.srcDirs = ['src/main/java', "${buildDir}/depSrc"]
}
}
I am trying to use Sigar in a Gradle project. Sigar distribution is by default provided with 2 types of files:
a JAR that contains classes
some native files (.so, dylib, .dll)
My purpose is to repackage these files so that I can use them as dependencies deployed and downloaded on-demand from a personal Maven repository.
My first try was to define dependencies as files in order to check that my application is working as expected before to repackage. Below is the Gradle code I used for my first test that works:
dependencies {
compile files("${rootDir}/lib/sigar/sigar.jar")
runtime fileTree(dir: "${rootDir}/lib/sigar/", exclude: "*.jar")
}
Then, I have repackaged Sigar native files into a JAR and renamed the other one to match rules for maven artifacts since I want to deploy them in a Maven repository. Below is what I get:
sigar-1.6.4.jar (contains .class files)
sigar-1.6.4-native.jar (contains .dylib, .so, and .dll files at the root)
The next step was to deploy these files in my custom repository. Then, I have updated my build.gradle as follows:
dependencies {
compile 'sigar:sigar:1.6.4'
runtime 'sigar:sigar:1.6.4:native'
}
Unfortunately, when I do a gradle clean build, new dependencies are fetched but native libraries can no longer be found at runtime since now I get the following exception:
Error thrown in postRegister method: rethrowing <java.lang.UnsatisfiedLinkError: org.hyperic.sigar.Sigar.getCpuInfoList()[Lorg/hyperic/sigar/CpuInfo;>
Consequently, I am looking for a solution to fetch and to link native files to my Java app like for other dependencies. Any advice, comment, suggestion, help, solution, etc. are welcome ;)
A solution is to define a new gradle configuration that unzips JAR files at the desired location:
project.ext.set('nativeLibsDir', "$buildDir/libs/natives")
configurations {
nativeBundle
}
dependencies {
nativeBundle 'sigar:sigar:1.6.4:native'
}
task extractNativeBundle(type: Sync) {
from {
configurations.nativeBundle.collect { zipTree(it) }
}
into file(project.nativeLibsDir)
}
dist.dependsOn extractNativeBundle
Then, this location must be put in java.library.path for tasks that depend on native libraries:
systemProperty "java.library.path", project.nativeLibsDir