How to deal with fat jar dependency - java

I have a proprietary dependency that I use in my project that I can not refuse. It was built in one big fat jar with all dependent packages collected inside. By all I mean even common ones like slf4j-api, apache-commons, javax packages, etc.
Using it together with my own list of declared dependecies is risky because there's always a race in classloader on which class will be loaded first - mine or outdated class inside fat jar.
I was wondering is there a way around this problem? How to treat such fat jars? I'm using maven for dependency management.


In Maven, the order in which you define dependencies in your POM is significant.
If you list them in the correct order, they should be added to the jar in that order,
and whichever class is higher in the file, that's the one that will get loaded first.
If you will compose your runtime classpath out of multiple jars,
then again, it's a matter of putting the jars in the right order.


If you know that some functionalities of the fat jar will work when you exclude some fo their dependencies or you want to include them yourself, you can try this:
Make a maven project that depends on the fat jar only
Use maven-shade-plugin, in particular it's relocation feature to exclude packages you don't want, or to just relocate all the jar's classes into another package, and thus move them out of the way.
Use the project's artifact instead of fat propietary jar in your other project.


I think that there's another way. I can wrap this library jar in its own custom classloader
URLClassLoader c1 = new URLClassLoader(new Url[] { new URL("file:lib/fatJarDep.jar"});
and create a factory that will instantiate classes of this library using this isolated classloader
Class.forName("className", true, c1);


If the jar has all dependency class extracted inside you can't exclude them from classloading, so same dependecy in your project may conflict.
You should edit the fat jar and manually remove the classes to make a light version, then install it in your repo and refer to it in your pom.

Related

Is fat jar equal to runnable/executable jar? [duplicate]

I am reading Maven documentation and came across the name uber-jar.
What does an uber-jar mean and what are its features/advantages?

Über is the German word for above or over (it's actually cognate with the English over).
Hence, in this context, an uber-jar is an "over-jar", one level up from a simple JAR (a), defined as one that contains both your package and all its dependencies in one single JAR file. The name can be thought to come from the same stable as ultrageek, superman, hyperspace, and metadata, which all have similar meanings of "beyond the normal".
The advantage is that you can distribute your uber-jar and not care at all whether or not dependencies are installed at the destination, as your uber-jar actually has no dependencies.
All the dependencies of your own stuff within the uber-jar are also within that uber-jar. As are all dependencies of those dependencies. And so on.
(a) I probably shouldn't have to explain what a JAR is to a Java developer but I'll include it for completeness. It's a Java archive, basically a single file that typically contains a number of Java class files along with associated metadata and resources.

ubar jar is also known as fat jar i.e. jar with dependencies.
There are three common methods for constructing an uber jar:
Unshaded: Unpack all JAR files, then repack them into a single JAR.
Works with Java's default class loader. Tools maven-assembly-plugin
Shaded: Same as unshaded, but rename (i.e., "shade") all packages of all dependencies. Works with Java's default class loader. Avoids some (not all) dependency version clashes. Tools maven-shade-plugin
JAR of JARs: The final JAR file contains the other JAR files embedded within. Avoids dependency version clashes. All resource files are preserved. Tools: Eclipse JAR File Exporter
for more

Paxdiablo's definition is really good.
In addition, please consider delivering an uber-jar is sometimes quite useful, if you really want to distribute a software and don't want customer to download dependencies by themselves. As a draw back, if their own policy don't allow usage of some library, or if they have to bind some extra-components (slf4j, system compliant libs, arch specialiez libs, ...) this will probably increase difficulties for them.
You can perform that :
basically with maven-assembly-plugin
a bit more further with maven-shade-plugin
A cleaner solution is to provide their library separately; maven-shade-plugin has preconfigured descriptor for that. This is not more complicated to do (with maven and its plugin).
Finally, a really good solution is to use an OSGI Bundle. There is plenty of good tutorials on that :)
For further configuration, please read those topics :
Should you provide dependent libraries in client jar?
Best practices in building and deploying Clojure applications: good tutorials?

The different names are just ways of packaging java apps.
Skinny – Contains ONLY the bits you literally type into your code editor, and NOTHING else.
Thin – Contains all of the above PLUS the app’s direct dependencies of your app (db drivers, utility libraries, etc).
Hollow – The inverse of Thin – Contains only the bits needed to run your app but does NOT contain the app itself. Basically a pre-packaged “app server” to which you can later deploy your app, in the same style as traditional Java EE app servers, but with important differences.
Fat/Uber – Contains the bit you literally write yourself PLUS the direct dependencies of your app PLUS the bits needed to run your app “on its own”.
Source: Article from Dzone
Reposted from: https://stackoverflow.com/a/57592130/9470346

A self-contained, executable Java archive. In the case of WildFly Swarm uberjars, it is a single .jar file containing your application, the portions of WildFly required to support it, an internal Maven repository of dependencies, plus a shim to bootstrap it all. see this

According to uber-JAR Documentation Approaches:
There are three common methods for constructing an uber-JAR:
Unshaded Unpack all JAR files, then repack them into a single JAR.
Tools: Maven Assembly Plugin, Classworlds Uberjar
Shaded Same as unshaded, but rename (i.e., "shade") all packages of all dependencies.
Tools: Maven Shade Plugin
JAR of JARs The final JAR file contains the other JAR files embedded within.
Tools: Eclipse JAR File Exporter, One-JAR.

For Java Developers who use SpringBoot, ÜBER/FAT JAR is normally the final result of the package phase of maven (or build task if you use gradle).
Inside the Fat JAR one can find a META-INF directory inside which the MANIFEST.MF file lives with all the info regarding the Main class. More importantly, at the same level of META-INF directory you find the BOOT-INF directory inside which the directory lib lives and contains all the .jar files that are the dependencies of your application.

Is there a way to bundle up multiple jars within the same jar file and make sure only one main jar inside it becomes part of classpath of the app

Is there a way to bundle up multiple jars within the same jar file and make sure only one main jar inside it becomes part of classpath of the application which adds this full jar into their classpath.
Code inside my main jar will use a custom class loader to load classes present in other Jars in the full jar.
I also want to create directory structure for other jars so that I can segregate jars based on use case and load only jars from a within directory inside the full jar.
Any help with any of the requirement above is appreciated. Thanks.

you can use Maven Shade plugin because it is a better option to create jar within jar based on custom business logic, if compare with the maven assembly plugin, it provides a class relocating feature, to avoid the issues in the complex structure of the classpath.
sample shade plugin configuration
you can also use many transformations in Shade plugin from the below list as per your requirement.

For an external jar, is it necessary to add a dependency in pom.xml file with a system path if jar already present/copied to lib folder

I have downloaded ojdbc14 jar from the internet and copied it to the lib folder of my maven project. Is it necessary to add dependency in pom.xml as well. Currently working without adding.

You can use tricks to manually upload a jar into the lib folder, but it does not make sense. And it would work cause the build process will just look for that jar into the lib and if found everything will compile nicely. BUT....
Maven is a useful tool that helps you handle dependencies, internal, external, third parties, any kind, it's one of his benefits, don't need anymore to look around for jars, and put them manually into the lib dir, but why? You would override one of the basic behaviour of Maven.
Maven set lots of rules to give you the ability to manage them the way you want, you have options about how to handle every single dependency of your project, you can point to a local jar within a dependency, you can set the scope of the dependency, the type you can exclude some of the inherited transitives, and so on...
But this is the standard approach for standard situation
You should simply define the dependency, maven will downloaded from the configured repo or the default one, maven central, and retrieved from your local repo if there are no updates on that artifacts all the other time you will build that artifact.
If you have issues with licenses for ojdbc14 then the solution is configure the oracle repo where you can easily download it.

What are the differences between uberJar, fatJar and shadowJar in Gradle?

I'm trying to understand when should I use which. I understand that they all assemble all the dependent classes, but how are they different from each other?

The terms are sometimes used interchangeably, but usually refers to:
Fat jar (also named Uber jar) - used to describe a jar that has all classes from dependent jars zipped directly inside it in the correct directory structure, and not in other jars. There is a good explanation here.
Shaded jar (or shaded classes) - usually refers to a process of changing classes bytecode to change packages names of the classes, and also modify were it is used in the jar. It is used to link classes to a specific version of other classes and avoid versions collisions. It can be created by Maven Shade Plugin. Shaded jar do not necessary contain all the dependencies needed.
The gradle shadow plugin can generate both fat jars and shaded jars, and same for the maven shaded plugin. I guess that is why people mix those terms.
Note: I have seen cases that refers to fat jar as a jar contains dependencies as a packed jars inside it.

There is no difference whatsoever. These terms are all synonyms of each other.
The term "uber-jar" may be more commonly used in documentations (take the maven-shade-plugin documentation for example) but "fat-jar" is also widely used.

Uber Jar, Fat Jar and Shadow Jar are synonyms.
All three tasks do the same thing - you can choose the term you personally prefer the most.

taking care of dependencies while creating a jar file

I have created a package that is to be used by other programmers by importing in their code.
my programs use other jar files for XML parsing and I don't want others to worry about the dependencies
what is the best way to make sure that my jar files always gets its dependencies?
Should i include the dependencies in my original jar?
Is there any alternative way?

I would say cleanest solution is to use bulid scripts like using Ant or Maven. In Maven you could create a local repository with the name of mayank. Now, all your team members just need to include dependency mayank; all other dependencies will automatically be downloaded. They dont have to worry about anything else.

If you want to release your source as a zip archive, I would keep the dependencies outside the project jar. For example in a folder name lib.
I would use a build tool like Maven (http://maven.apache.org) to manage my dependencies. It's pretty easy to set up a repository like Nexus (http://www.sonatype.org/nexus) where your team members can get your jar and all the required dependencies.

Use jarjar, seems doing exactly that you want, does not force your potential users to use exactly Maven (some may use old Ant scripts or IDE features to add .jar file directly).

Categories

Resources