Develop Reference

Java & Gradle & log4j NoClassDefFoundError: org/apache/logging/log4j/LogManager

I'm trying to use log4j in my java core console application. In the documentation there is sample code on how to add Gradle dependency for log4j:
https://logging.apache.org/log4j/2.x/maven-artifacts.html
dependencies {
compile group: 'org.apache.logging.log4j', name: 'log4j-api', version: '2.14.1'
compile group: 'org.apache.logging.log4j', name: 'log4j-core', version: '2.14.1'
}
I pasted it to my build.gradle file and wrote sample code for running the code.
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
public class Program {
private static final Logger logger = LogManager.getLogger("HelloWorld");
public static void main(String[] args) {
logger.error("Just a test error entry");
}
}
along with that I also added manifest in order to run the code from the console.
jar {
manifest {
attributes "Main-Class": "Program"
}
}
If I build it with Gradle - everything is built successfully. But if I try to run java -jar from my build directory then I got
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/logging/log4j/LogManager
at Program.<clinit>(Program.java:5)
Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.LogManager
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
... 1 more
I found 1 workaround: modify to jar{} section:
jar {
manifest {
attributes "Main-Class": "Program"
}
from {
configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
}
}
and it started working, I even try to zip .jar file and saw that there is log4j dependency added.
But what is I don't want transitive dependency on log4j? If I set implementation group instead of compile group it doesn't work again. Is there any complete example of using log4j?
On top of that, one thing to point, while it is not working with java -jar <.jar>, for some reason it still works in Intellij idea. Could someone explain me why?

The problem you are encountering is caused by a problem in the application's classpath, when you launch the application (cf. What is a classpath and how do I set it?).
Gradle has many options to help you with setting the classpath right. Since you don't mention any Gradle version, I'll use version 7.1 in the examples. Assume you have this build.gradle file in a project named appName:
plugins {
id 'application'
}
dependencies {
implementation group: 'org.apache.logging.log4j', name: 'log4j-api', version: '2.14.1'
runtimeOnly group: 'org.apache.logging.log4j', name: 'log4j-core', version: '2.14.1'
}
application {
mainClass = 'com.example.App'
}
(log4j-core is not necessary to compile your code, so it is in the runtimeOnly dependency configuration, cf. dependency configurations)
Manually setting the classpath
If you generate a plain JAR file (execute the jar task and look in build/libs) you can execute the application with:
java -cp log4j-api-2.14.1.jar:log4j-core-2.14.1.jar:appName.jar com.example.App
(I assume all jars are in the current directory; the path separator is system specific, on Windows it is ;)
This becomes easily difficult to get right, therefore the application Gradle plugin generates a shell and a batch script to help you: execute the installDist task and look into build/install/appName (or you can use the distZip or distTar tasks for a compressed version of this folder). Setting up the correct command is reduced to calling:
bin/appName
Hardcoding the classpath in the Manifest
You can also hardcode the dependencies needed in the JARs manifest file. In Gradle you can do it with:
jar {
manifest {
attributes(
'Main-Class': 'com.example.App',
'Class-Path': configurations.runtimeClasspath.collect { it.name }.join(' ')
)
}
}
to obtain a manifest file with all the information needed to start the application:
Main-Class: pl.copernik.gradle.App
Class-Path: log4j-core-2.14.1.jar log4j-api-2.14.1.jar
Notice the usage of runtimeClasspath, which is a configuration that contains both the runtimeOnly and implementation dependency configurations needed to start the application.
If the dependency jars are in the same folder as appName.jar, you can run the application with:
java -jar appName.jar
Spring Boot loader
If you want to have all dependencies in a single JAR, you can use the Spring Boot Gradle plugin instead of the application plugin:
plugins {
id 'java'
id 'org.springframework.boot' version '2.5.3'
}
The bootJar task will produce an appName.jar resembling in its structure the appName.zip file produced by the distZip file. However the shell/batch scripts are replaced with Java code, so you can call:
java -jar appName.jar
Fat Jar
What you did in your question with:
jar {
manifest {
attributes "Main-Class": "com.example.App"
}
from {
configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) }
}
}
is usually called fat JAR. Basically it is produced by unzipping your jar file and all dependencies into a single folder and zipping it back together.
This works, but has some disadvantages: e.g. you might lose licence information on the dependencies (which might be a licence violation) and you will not be able to replace the dependencies with new versions without recompiling. See this question for an example of an absurdly large fat jar.

How to Obfuscate a fatJar with Klassmaster

I have a fat jar which is generated by using gradle script. Post the gradle script when I run the following command :-
java -jar fileName.jar
it is running the main method and things are fine. Nevertheless when I try to obfuscate this jar, the resulting jar is complaining that :-
Error: Invalid or corrupt jarfile ObfusactedTest.jar
My code is as follows:-
build.gradle:-
buildscript {
repositories {
mavenLocal()
mavenCentral()
}
dependencies {
classpath files("E:\\softs\\ZKM\\ZKMEval\\ZKM.jar") //ZKM_JAR_PATH must be set to point to your ZKM.jar
classpath 'com.zelix.gradle:plugin:1.0.0'
}
}
apply plugin: 'java'
apply plugin: 'maven'
apply plugin: 'com.zelix.gradle.plugin'
group = 'com.github.jitpack'
sourceCompatibility = 1.8 // java 8
targetCompatibility = 1.8
repositories {
mavenLocal()
mavenCentral()
}
dependencies {
compile group: 'org.apache.commons', name: 'commons-lang3', version: '3.11'
compile group: 'org.apache.commons', name: 'commons-collections4', version: '4.4'
}
jar {
manifest {
attributes "Main-Class": "com.github.jitpack.Hello"
}
zip64 = true
from {
configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
}
}
zkmSetting {
scriptName = "Obfuscate.txt" //Must be set to point to the ZKM Script to execute.
}
Obfuscate.txt:-
print "Obfuscating fatJar.....";
classpath
"C:\Program Files\Java\jdk-10.0.2\lib\jrt-fs.jar"
".\obfuscateFatJar.jar";
open ".\obfuscateFatJar.jar" {"*.class"};
exclude org.apache.commons.*.*;
exclude com.github.jitpack.Hello.*;
obfuscate keepInnerClassInfo=false
keepGenericsInfo=true
exceptionObfuscation=heavy
encryptStringLiterals=flowObfuscate;
saveAll archiveCompression=asIs
deleteEmptyDirectories=true
deleteXMLComments=false
"ObfusactedTest.jar";
By the way Hello.java has got the main method.

Your ZKM Script "open" statement specifies the {"*.class"} file filter. So you are filtering out ALL non-class files including your MANIFEST.MF. See https://www.zelix.com/klassmaster/docs/openStatement.html#filter.
A missing MANIFEST.MF will give you a "Invalid or corrupt jarfile" error. Note that your Zelix KlassMaster log file will contain messages like the following.
MESSAGE: Filtering out path 'obfuscateFatJar.jar!META-INF/MANIFEST.MF' because it does not match specified filter '{".class"}>' (D)*
You can work around this by not using a file filter (the safest option in this case) or by broadening your file filter to include other file types. E.g. {".class" || ".MF"}

Update the filter in the class path. The code looks like this now. Works like a charm.
execute "del ObfusactedTest.jar";
classpath
"C:\Program Files\Java\jdk-10.0.2\lib\jrt-fs.jar"
".\obfuscateFatJar.jar";
open ".\obfuscateFatJar.jar" {"*.class" || "*.MF"};
exclude org.apache.commons.*.*;
obfuscate keepInnerClassInfo=false
keepGenericsInfo=true
exceptionObfuscation=heavy
encryptStringLiterals=flowObfuscate;
saveAll archiveCompression=asIs
deleteEmptyDirectories=true
deleteXMLComments=false
"ObfusactedTest.jar";

Java doesn't load classes from manifest-defined classpath

I'm using Gradle to build a project. This is my build.gradle:
plugins { id 'application' }
group 'my.group'
version '1.0'
sourceCompatibility = 1.11
repositories { mavenCentral() }
dependencies {
testCompile group: 'junit', name: 'junit', version: '4.12'
implementation group: 'org.xerial', name: 'sqlite-jdbc', version: '3.7.2'
}
mainClassName = 'my.group.App'
jar {
manifest {
attributes(
'Main-Class': mainClassName,
'Class-Path': configurations.runtimeClasspath.files.collect { it.name }.join(' ')
)
}
from configurations.runtimeClasspath
into ''
}
This correctly generates a jar file with the following META-INF/MANIFEST.MF:
Manifest-Version: 1.0
Main-Class: my.group.App
Class-Path: sqlite-jdbc-3.7.2.jar
[newline]
And in the root of this jar file, there is indeed the sqlite-jdbc-3.7.2.jar file referenced in the manifest; I manually verified that inside this jar file there is a class called org.sqlite.JDBC.
However, running the generated jar with java -jar jarfile.jar results in:
Exception in thread "main" java.lang.ExceptionInInitializerError
at my.group.LEManagerSqlite.<init>(LEManagerSqlite.java:64)
at my.group.LEManager.createLEManager(LEManager.java:80)
at my.group.GuiFrame.<init>(GuiFrame.java:60)
at my.group.App.openFromFile(App.java:23)
at my.group.App.main(App.java:19)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.sqlite.JDBC
at my.group.SqliteConnectionManager.<clinit>(SqliteConnectionManager.java:17)
... 5 more
Caused by: java.lang.ClassNotFoundException: org.sqlite.JDBC
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at my.group.SqliteConnectionManager.<clinit>(SqliteConnectionManager.java:14)
... 5 more
For reference, SqliteConnectionManager has a static initializer which loads org.sqlite.JDBC, which is what is referenced as SqliteConnectionManager.java:14 in the stack trace:
Class.forName("org.sqlite.JDBC");
I tested this with OpenJDK 11 and OpenJ9 11, with identical results. I conclude that I'm doing something wrong, but I cannot understand what.

I found out what I was doing wrong: the documentation clearly says (doh!) that the Class-Path directive looks for additional classpath resources in the local filesystem or the network.
Loading additional classes from jar files included in the main jar file is not supported by the JVM, and thus requires custom loading code. A simpler alternative is to unpack the additional classes from the jar files and include them directly. A jar file built this way is referred to as a "fat jar" (or "uber jar").
There is a Gradle plugin called Shadow which can build such jars without any further configuration.

Gradle: JDBC driver not compiled into JAR

I have a module build.gradle file that looks like this:
apply plugin: 'java'
sourceCompatibility = 1.5
version = '1.0'
repositories {
mavenCentral()
}
dependencies {
testCompile group: 'junit', name: 'junit', version: '4.11'
compile 'mysql:mysql-connector-java:5.1.6'
}
I also have an artefact set up like this:
When I build this artifact, my classes are included but MySQL driver is not. When executed, MySQL driver class is not found. How do I build so that my JAR artifact (used as a library)
ccontains MySQL JDBC driver?

Configuring this in Gradle and IntelliJ is two different things. The most direct way to create a fat Jar in Gradle is:
jar {
from { configurations.runtime.collect { it.directory ? it : zipTree(it) } }
}
PS: Merging Jars can cause problems. In most cases it should not be done, in particular for a library.

Add to subproject to gradle jar

How can I add a subproject referenced using project(':api') to the jar gradle builds?
This is the build.gradle of my main project. The subproject is includes as git submodule and has a similar buildscript.
apply plugin: 'java'
sourceCompatibility = 1.5
version = '1.0'
jar {
manifest {
attributes('Main-Class': '..........')
}
}
repositories {
mavenCentral()
}
dependencies {
compile files('libs/jfxrt.jar')
compile project(':api')
testCompile group: 'junit', name: 'junit', version: '4.11'
}

I figured it out on my own.
Include the source of a subproject in the main jar:
sourceSets {
main {
java {
srcDir project(':api').file('src/main/java')
}
}
}
Including the classes of a jar in the main jar:
jar {
from zipTree('libs/abc.jar')
}

Try to add classpath to your manifest file. You need to have directory (example below uses "lib") to keep jar files on which your project depends.
Try modifying your "jar" block in gradle build to something like this. I have some addition properties just for demonstration. But the important one is Class-Path
jar {
manifest.attributes(
'Class-Path': lib/api.jar
'Built-By': System.getProperty('user.name'),
'Built-JDK': System.getProperty('java.version'),
'Built-OS': System.getProperty('os.name'),
'Built-DATE': buildDate,
)
}
I hope it helps to fix your issue.

In the simplest case, a fat Jar can be created as follows:
main/build.gradle:
jar {
from configurations.runtime
}
There are other, more robust solutions, such as the gradle-one-jar plugin for "main" method style applications.