Develop Reference

How Maven loads classes using ClassLoader

My knowledge of ClassLoader in Java at the moment, is a little obscure. That's because I have not found any good documentation geared towards beginner of CLassLoader, thus far. And what I am looking for exactly is in relation to Maven. With disclaimer stated, let me get into my question.
I am writing a Spring MVC application, and I decided to look into how the Dependencies - Jars and classes are loaded using ClassLoader. And what I found is according to the documentation of ClassLoader, classes in Jar are loaded from CLASSPATH, and I can see them under the .m2/repository directory, but CLASSPATH does not yield anything, it's practically empty.
Can somebody please explain to me, how the classes from JAR are loaded into JVM Memory using ClassLoader using Maven, if the CLASSPATH is empty.
Thanks

You're confusing a few things.
The ClassLoader
The ClassLoader is a runtime concept. Maven is a compile time concept. Therefore, one has nothing whatsoever to do with the other. Maven and ClassLoaders do not interact. At all.
When you start a basic java app (java -jar foo.jar or java com.foo.MainClass), you get 2 classloaders. One loader will load system stuff: java.lang.String, for example. the executable itself 'just knows' how to do this (you don't need to configure PATH, CLASSPATH or JAVA_HOME - it just works); up to java 8, it finds rt.jar automatically, which contains String.class and other core classes. Starting from java 11, it finds the jmod files in your java distro.
Then, once the VM has 'booted up', the VM makes another classloader, also based on its built-in stuff: The app classloader.
This one uses 'the classpath'. The source of this depends on how you ran your java app:
java -jar somejar.jar
The source of the classpath in this case is the Class-Path: line in the jar's manifest (the file at META-INF/MANIFEST.MF). And nothing else - the CLASSPATH environment variable, and any -cp or -classpath options are entirely ignored.
java -cp a.jar:b.jar:. com.foo.ClassName
Note that -cp is short for -classpath (they mean the same thing): Here, the classpath is taken to be all the files and directories listed (on windows, use ; as separator instead), and classes are loaded from there. Including com.foo.ClassName itself.
java com.foo.ClassName
If you don't explicitly specify a -cp param, then the environment var CLASSPATH is used. You don't want this though. Always specify classpath.
That's runtime - maven has nothing whatsoever to do with this.
Make your own
You can make your own ClassLoader; the abstraction is such that all it needs to be able to do, is turn a resource name into bytes. That's it. You can make a ClassLoader (literally! public class MyLoader extends java.lang.ClassLoader { ... }) that.. loads data from a network, or generates it on the fly, or fetches it from an encrypted data store. Whatever you like.
Using custom classloaders like this is a solution for finding classes in 'weird' places (not jar files or directories), as well as a mechanism to allow java to 'reload classes' on the fly - very useful when developing, say, web apps, without the use of a hot-code-replacing debugger like eclipse has. ClassLoaders is a mechanism whereby a web server can have the following feature: "I load jars or wars from a certain preconfigured directory.. and if you replace the jar, I will see it and start using the new one".
Writing your own ClassLoader is bordering on rocket science and not usually required unless you're, say, writing an app server. Not a common job.
Maven
To compile source code, the compiler must know the methods and fields and such of all the types you refer. You can't compile "Hello".toLowerCase(); if the compiler doesn't know what String contains.
Thus, compilers also have this notion of 'I need to find classes'. But This is not called class loading, and notably, maven never loads any classes. If it did, any static initializers in any class would run, and mess up your compile. Maven instead just inspects the class file, never letting the VM actually load it, to know what kind of methods and fields and the like are on offer.
java.lang.ClassLoader plays no part in this.
javac itself has a -classpath option as well. So does maven, really.
Maven constructs the classpath automatically, by putting the stuff it already compiled for you (e.g. when compiling the stuff in src/test/java, the compiled stuff from src/main/java is on the classpath), as well as all the dependencies. How? Well, does it matter? Maven does. It constructs a large list of dirs and jars and passes it to javac via the -classpath parameter.

Compiling a single class file edited from existing jar with no dependencies

I need to edit a single class file from within a jar.
I have successfully extracted the class file from the jar, and I have decompiled it and found the logic I need to change.
However, I'm unable to recompile this class file, because it imports libraries I don't have and don't know where to get (netbeans and iharder).
The needed files should all be within the jar, right? Can I use the jar for this purpose?
I do not understand much of Java's overarching syntax, so anything related to packages or jar file structure might go over my head...

The JVM just needs to be able to find the dependencies at runtime. Often, they'll be installed in a standard location (the classpath), rather than being bundled with the jar that uses them. However, you could theoretically even do something like download dependencies at runtime and load them via a classloader.
Apart from that, decompiling and recompiling is often not a good idea, since decompilation is a lossy and error prone process. It generally only works in simple cases, and has limitations, as you've discovered.
If you understand Java bytecode, you can edit the class by disassembling it with Krakatau, editing the .j file, and then reassembling. This allows you to edit any classfile without needing to compile, meaning you don't need the dependencies. It also works no matter how complicated the class is, and even works on obfuscated classes.

jar built with jwrapper doesn't work

jwrapper manipulates application jars somehow, and is resulting in a non-functioning jar: at runtime it throws a "MyClass cannot be cast to MyClass" type error. I believe this is caused by re-evaluating code that creates a class loader, leading to multiple instances of class MyClass being loaded.
The jwrapper docs don't describe the changes made to the jar, except for the use of pack200. I've tested pack200 in isolation, and it does not cause this problem.
I've also tested the jar built by jwrapper without using the wrapper executable, by passing it to "java -jar". So it's not jvm transmuting, or anything else that the wrapper is doing: the jar itself is broken.
UPDATE:
jwrapper allows skipping pack200, but then the install file is huge. Since pack200 works when run standalone, I could work around this if there were some way to tell jwrapper that the file is already packed. Using <Pack200Exceptions> doesn't help, because then it doesn't know the file is packed.

The underlying problem is that jwrapper sets the pack200 option "modification_time" to "latest", which changes the modification times of all the class files. At run-time this causes the clojure compiler to attempt to recompile the classes from source.
A work-around is to remove the .clj files from the jar prior to packaging, preventing the compiler from running. The lein ":omit-source" option is not sufficient here, because it leaves in .clj files from any dependencies. Instead you must use a pattern in :uberjar-exclusions, e.g.
:uberjar-exclusions [#".(clj|java)"]
as detailed here:
https://github.com/technomancy/leiningen/issues/1357

NoClassDefFoundError when referencing library

ok I have been search the internet all day and I have tried everything I have seen so i am wondering if someone could help.
I have a class that references a jar file which i have copied into my workspace in the lib folder the jar is: Classifier4J.jar, My class run perfectly on the console no error at all. When I try to package the .jar together and run the .jar from another program it gives me this error:
Exception in thread "pool-2-thread-1" java.lang.NoClassDefFoundError: net/sf/classifier4J/bayesian/IWordsDataSource...(10 more)
So clearly when i create the .jar its not able to reference the classifier4j library that it needs.
Things I have tried:
-Configure my build path
-Change the manifest file
-packaged the .jar with my .jar
-in eclipse went to file>export>java>runnable .jar then references the libraries
and many other things and nothing worked.
If anyway has had a similar issue or knows why this is happening could you please help me its really wrecking my head. Is this a problem with eclipse? can it be done through eclipse?
Thanks in advance
Jay

Ok after hours of looking at the problem i finally found the solution, When I output my classes as a .jar file I pointed it to my manifest file. I couldn't find what i was doing wrong because it didn't work. I decided to look at what was actually put into the jar and i saw that eclipse wasn't putting the correct manifest file i asked it to into the jar. It was putting a new one that looked like:
Manifest-Version: 1.0
where it should have looked like:
Manifest-Version: 1.0
Class-Path: ../lib/Classifier4J.jar
where the lib folder was back a directory from where my jar was. Everything is working perfectly now. Thank you for all your help.
Jay

It is hard to understand what you are actually trying to do, what you have actually tried, and what you expect to work, but I think the clue is here:
packaged the .jar with my .jar
This seems to imply that you are trying to create an "executable JAR" that contains all of the dependent JARs ... as-is. That won't work, an "executable JAR" cannot contain other JARs. (Well is can ... but you can't put them on your application's classpath without doing seriously complicated things.)
Basically, you have two options:
Don't try to include the dependent JARs in your (executable or not) application JAR. Keep them separate, and configure the execution classpath to include them. (And beware that for an executable JAR, the execution classpath must be configured as a Manifest entry. If you use java -jar ..., the -cp argument and $CLASS_PATH are ignored!)
Create a so-called "uber-JAR" by exploding the application JAR and all of the dependent JARs into a single directory and then creating a single JAR (with a suitable Manifest) from the lot. Your build tool or IDE may have support for this via some plugin.
I personally prefer the former approach ... combined with an "installation directory" for the application and a wrapper script. With the latter approach you make it hard for the user (or deployer) to mix-and-match versions of dependent JARs. Furthermore, the "uber-JAR" approach could conflict with a 3rd-party library's license.
At last, maybe we can get somewhere
.... the .jar file is ran from a tomcat application its a simple adapter for log files that all.
OK. You should have mentioned that before, because it is a critical piece of information. In order for a JAR file to usable within in a web container (i.e. in the same JVM as Tomcat), there must be a copy of the JAR file and all of its dependent JAR files in the web container's directory tree. There are two choices. Either you put them in the webapp's lib directory (i.e. webapp/WEB-INFO/lib) or you put them in the shared library directory ... which depends on which version of Tomcat you are using.
(The "executable JAR" approach won't work here. The classpath stuff you put in the Manifest is irrelevant. And nesting JAR files won't work either.)

The particular library is not included in your jar. You could either try to fix your eclipses build configuration to include that library, or add the library's jar to the classpath when you run the program.
The later may be easier. Just add the following to the command when you execute your program.
java -cp yourjar.jar;thelibrarysjar.jar com.your.Mainclass

You need to understand that NoClassDefFoundError hardly ever means that the identified class can't be found. Far more often, the class was found, but something prevented it from being successfully loaded.
The two most likely problems are -
Some other missing class that prevents the named class from being "verified".
Some incompatibility due to a different version of a class (usually another class) being used in compilation vs execution.
In your case it's most likely that when you have the problem you're picking up a different (and incompatible) version of some other jar, vs the environment during compilation.

How to combine library with my jar?

Ok so i wrote a program that makes use of a 3rd party open source library and i want to package it with my program in a single jar. I'm using netbeans 6.8 and everything I've tried java always spit back the error:
java.lang.NoClassDefFoundError: libraryname;
off topic:also i would like to know how to make an executable-jar(exe) through netbeans if it is possible. (ive seen programs that were written in java but were an .exe)
EDIT discovered a plugin for eclipse called FatJar which can do what i want, but i cant find something similar for netbeans, is there such thing?

I'll start off with the obligatory disclaimer: Java executable JARs do not work this way. An executable JAR has a main class defined in the JAR's MANIFEST.MF file, and the MANIFEST also allows the definition of a class path to include libraries that the code in the executable JAR will need. The class path definition in the MANIFEST must enumerate every JAR or folder to put on the class path, relative paths are relative to the location of the executable JAR - not to paths contained inside the executable JAR. Executable JARs are launched with the "-jar" argument to the java executable, and both the java "-cp" flag and the CLASSPATH environment variable are ignored. As for why executable JARs were designed this way, you should be aware of the primary disadvantage of loading classes from JARs contained within JARs, even though the rest of this reply will focus on doing just that.
NOTE: I lost the original sun forum topic that explained it fully, but essentially it is because entries in the top level JAR can be read in a random access manner, but the entire embedded JAR must be read before any entries can be accessed, because the top level JAR might have compressed its entries.
I have used One-Jar successfully in the past, but the structure of the final resulting jar may not be what you expect. Essentially the One-Jar classes are the only non-JARd classes in the final jar; all other code (your code and any dependent library code) is included in the resulting as JAR as JAR files. Your application is JARed as a regular JAR file named "main.jar" in the final JAR's "main" folder. Any libraries your code needs is placed, as JAR files, in the final JAR's "lib" folder. And last but not least the final JAR's MANIFEST.MF file tells One-Jar what your main class is. Execution is a dead simple "java -jar final.jar [args your app uses]". I don't know how to take the next step of converting to an OS-native EXE regarding your off-topic question, but it would probably be best to use a different packaging mechanism than One-Jar anyway. I'm not sure how to go about this with NetBeans, my advice there is to use a build tool to package the final jar. Fortunately One-Jar provides instructions on generating the final jar with Ant, and that should be easily integratable into NetBeans.
I believe the Eclipse FatJar plugin creates a One-Jar executable JAR, so if that plugin seems to do what you want, then One-Jar is the way to do it. Personally, I used a Maven assembly.
There is a caveat - any signed libraries that require (or desire) to take advantage of Java's signed JAR verification may not work this way - Java Cryptographic Extension (JCE) implementations like BouncyCastle are a notable example. I think the reason is that the signature verification runs against the final JAR, not the signed library. Fortunately One-Jar allows the end user to add additional libraries to the classpath, something that is explicitly precluded when running an executable JAR; to workaround this you might be better off delivering the problematic JARs with the final JAR and an OS dependent launch script (.bat, .sh, etc).

I realize that this doesn't achieve exactly what you want, but I'll describe the customary method of distributing a standalone application. If it does meet your needs, you'll find that it's better supported by tools and more readily understood by users, because it follows established conventions.
Put your code in a jar (I'll call it app.jar) along with a META-INF/MANIFEST.MF file with entries like this:
Main-Class: com.y.app.AppMain
Class-path: third-party.jar blort.jar foo.jar
Then, you can throw all of the jars into a directory and run AppMain like this:
java -jar app.jar
If you want, you can put the third-party libraries in a single directory like lib and refer to them in the Class-path attribute using a path relative to the main jar: lib/third-party.jar That helps keep your distribution tidy.

My generic answer to your off-topic question is a (rather lengthy) article: Convert Java to EXE - Why, When, When Not and How. It has lots of links to free and commercial tools, but I have never seen a Netbeans plugin with such functionality, sorry.

To include another jar in your jar, you might find jarjar useful.
Executable jars just have a class defined as 'Main', if I'm not mistaken. This may be useful.

If there's not any concern of repackaging 3rd party jars into your final big jar, then this should be the easiest method.

If there are no licencing issues then the most preffered way is to unjar the actual jar and rejar it with your class files in it, to a new jar.
You can simply use the jar cmd itself for this, no big deal!!

if you use MAVEN, use "maven-shade-plugin" plugin. It will compile jar with all dependencies(3rd party and etc.)