Develop Reference

What is the expected way to consume binary dependencies from Java packages at runtime?

For context, I don't work a lot with Java, but need to create a Java maven package to provide a simple java wrapper around a cdylib (dll and so files). The dll size is relatively large, and I've seen some posts that there's issues loading DLLs from and to the running JAR if it's just a file that's included in the JAR archive.
So I'm curious if the Java devs out there expect Java maven packages to behave a certain way where DLLs/large-resources are concerned? Is there an expectation it would handle attempting to download the larger dependencies at runtime and just provide settings to override/control that behavior? Or should I just document it as needing those files and where to find them for package consumers?
If there's an expectation they would come bundled in a JAR, I also have yet to find any examples of how to create platform specific package builds from a shared codebase, only changing resources in Gradle, and would welcome any help on how to do that.

What are the differences of putting JARs into JVM lib than the project lib?

Background
I recently came across a Java project that requires me to put a JAR file IngrianNAE-5.0.0.jar into /usr/lib/jvm/java-8-oracle/jre/lib/ext/ directory. (The instruction was made by a former developer who has left.)
The build tool does not pick up this IngrianNAE-5.0.0.jar during the project building process, and the project JAR file cannot run with a JVM that does not have this IngrianNAE-5.0.0.jar in its lib/ext/ directory.
Also, even if I include this IngrianNAE-5.0.0.jar into my build path, e.g. claiming it in dependencies section in Maven pom.xml, the project still requires the JVM to have IngrianNAE-5.0.0.jar in its lib/ext/ directory. (At this point, Maven can produce the project rich JAR file, which contains IngrianNAE-5.0.0.jar. However, JVM still needs to have IngrianNAE-5.0.0.jar in its lib/ext/ directory. To me, it seems that this IngrianNAE-5.0.0.jar is not used by the project, but by JVM.)
By the way, as IngrianNAE-5.0.0.jar being an example, it is just one of many JARs to be added into /usr/lib/jvm/java-8-oracle/jre/lib/ext/ directory along with few properties files, such as IngrianNAE.properties, etc.
Question Elaboration
Why do I have to have such JARs being added, not into the project, but into the JVM.
How does it works?
What documentations I can look into to get more information (or tutorials) about it?

Documentation from Oracle
Installed extensions are JAR files in the lib/ext directory[...]
installed extensions extend the platform's core API[...]
the symbols defined by installed extensions will be visible in all Java processes
See also this question

Since the IngrianNAE jar file is extending the JVM capabilities through JCE, I believe (unfortunately) that the extension must be applied at the JVM level.

Difference between a jar file and a library in Java

NetBeans allows the programmer to add a library and a jar file.
What is the difference between a jar file and a library? Is library similar to GAC assembly as in Windows.
There are similar questions, but they are way too specific and I was not able to understand the difference.

to put things very simple : library is a collection of jars
You could like create a global library java-ee which contains all Java EE related jar files. Then you could use this global library in your different projects. It will be simpler to manage them; and adding in new projects.

A JAR serves the same function an an Assembly in the C#/.net world. It's a collection of java classes, a manifest, and optionally other resources, such as properties files.
A library is a more abstract concept, in java, a library is usually packaged as a JAR (Java ARchive), or a collection of JARs.

A jar file is zip archive containing among other files, the java class files. A Netbeans library contains resources required by the project, including jar files.

If well understood: A library is simply a folder that groups classes. For example in JDK, a library present there is a group of classes stored together.
If not mistaken a .jar file is a group of compiled classes in .class format and was created by Java creators so a program will be OS independent; which means within a JVM you will run your app in .jar format on a Linux, Windows, etc without re-coding tour app for various OSs.

This article explains it all..
It states
Java's libraries are commonly known as class libraries. However, Java
refers to class libraries as packages.

How do I build a single bundled JAR with all the needed classes to run a Java application?

I'm going to deploy a Java application with a custom launcher, and I need to have all the classes needed for my app in a single jar file so I don't have to deploy the entire Java SE libraries with it.
I was thinking of using some pre-existent ant tasks to create a target that recursively searches all my compiled classes files for its dependencies. After all the dependencies have been determined it would extract the needed class files from their JAR's, copy it along with my classes to an output directory and make a single jar from it.
If there's no such thing avaliable out of box, I can create one myself, but I need to know how references to other classes are stored in .class files. It could be much easier if there's some kind of java library like .NET's Mono.Cecil/System.Reflection that exposes an high level API for inspecting/manipulating Java.
Since I'm new to Java, I'm having some trouble in finding what is needed to acomplish those things. Can someone give me some direction?

Unfortunately, you cannot ship only part of Java SE - that will breach the license agreement. If you use Java SE, then all of Java SE must be available.
The simplest way to achieve this is to use an Ahead Of Time compiler. These take care of packaging only the classes you need, and adhere to the JDK license agreement by making the "unused" parts of the JDK available via optional download.
For example, Excelsior JET is a good AOT compiler and will package just the classes you need. It's not free for commercial use, although open source projects can apply for a free license grant.
Alternatively, you may simply assume that the user has the JRE already since it's installed on over 90% of desktops, and in cases where the JRE is not available, have your installer download one for the user. AdvancedInstaller has a free edition that will accomplish this.

After all the dependencies have been
determined it would extract the needed
class files from their JAR's, copy it
along with my classes to an output
directory and make a single jar from
it.
As an easy solution, if you use Eclipse IDE you use the following solution:
Under the Java project properties (right click):
Export... => Export as Runnable JAR
The exported JAR will have all its dependencies packed into it.
alt text http://www.ubuntu-pics.de/bild/97131/selection_016_mg6IDm.png

Here is one suggestion I found on the web:
<jar destfile="${build-abc}/abc.jar" duplicate="fail" index="true">
<zipfileset src="${compile-lib}/demo.jar" includes="**/*.class"/>
</jar>
This should add the dependencies to the jar (as shown with demo.jar). You still have to adapt the manifest file so that the added jars appear on the classpath.
This doesn't solve the 'Java SE' classes problem - you'll have to bundle a jre in the installer package or depend on an existing one on the target system.

You won't have to deply Java SE classes, as they already are in the customer JRE.
If your dependencies are expressed ion a common way (through Maven or Ivy) i guess it's quite easy to find an equivalent of maven uberjar task ... which will do what you want to do, but in a more simple way (as it simply repacks all jars in one big jar).

Eclipse Java project folder organization

I am coming to Java and Eclipse from a C#/Visual Studio background. In the latter, I would normally organize a solution like so:
\MyProjects\MyApp\MyAppsUtilities\LowerLevelStuff
where MyApp would contain a project to build a .exe, MyAppsUtilities would make an assembly DLL called by the .exe, and LowerLevelStuff would probably build an assembly containing classes used by the higher-level utilities DLL.
In Eclipse (Ganymede, but could be convinced to switch to Galileo) I have:
\MyProjects\workspace\MyApp
When I create my initial project. There is an option to put source and build files in same folder, but I have .java files created on a path that is reflective of my package hierarchy:
\MyProjects\workspace\MyApp\src\com\mycompany\myapp\MyApp.java
My question is this: when I create subprojects (is that the right Java/Eclipse term?) for .jar files that will be analogous to the above MyAppsUtilities and LowerLevelStuff assembly DLLs in .NET, can (should) I organize the folders equivalently? E.g.:
\MyProjects\workspace\MyApp\src\com\mycompany\myapp\myapputilities\MyAppsUtilities.java
What is the standard/right way to organize this stuff, and how is it specifcally done in the IDE?

Think of Java source code packages as one big hierarchical namespace. Commercial applications typically live under 'com.mycompany.myapp' (the website for this application might be 'http://myapp.mycompany.com' although this is obviously not always the case).
How you organize stuff under your myapp package is largely up to you. The distinction you make for C# between executable (.exe), DLL's and low-level classes does not exist in the same form in Java. All Java source code is compiled into .class files (the contents of which is called 'bytecode') which can be executed by a Java Virtual Machine (JVM) on many platforms. So there is no inherent distinction in high-level/low-level classes, unless you attribute such levels via your packaging. A common way of packaging is:
com.mycompany.myapp: main class; MyApp (with a main method)
com.mycompany.myapp.model: domain model classes; Customer, Order, etc.
com.mycompany.myapp.ui: user interface (presentation or view) code
com.mycompany.myapp.service: services within your application, i.e. 'business logic'
com.mycompany.myapp.util: helper classes used in several places
this suggests a standalone Java app, it might be different if it is a webapp using one of the many frameworks.
These packages correspond to a directory hierarchy in your project. When using Eclipse, the root of such a hierarchy is called a 'source directory'. A project can define multiple source directories, commonly a 'main' and a 'test' source directory.
Example of files in your project:
src/test/java/com/acme/foo/BarTest.java
src/main/java/com/acme/foo/Bar.java
lib/utilities_1_0.jar
And inside utilities_1_0.jar:
com/acme/foo/BarUtils.class
BarUtils.class this is a compiled java class, so in platform independent bytecode form that can be run on any JVM. Usually jarfiles only contain the compiled classes although you can sometimes download a version of the jar that also contains the source (.java) files. This is useful if you want to be able to read the original source code of a jar file you are using.
In the example above Bar, BarTest and BarUtils are all in the same package com.acme.foo but physically reside in different locations on your harddisk.
Classes that reside directly in a source directory are in the 'default package', it is usually not a good idea to keep classes there because it is not clear to which company and application the class belongs and you can get name conflicts if any jar file you add to your classpath contains a class with the same name in the default package.
Now if you deploy this application, it would normally be compiled into .class files and bundled in a .jar (which is basically a fancy name for a .zip file plus some manifest info).
Making a .jar is not necessary to run the application, but handy when deploying/distributing your application. Using the manifest info you can make a .jar file 'executable', so that a user can easily run it, see [a].
Usually you will also be using several libraries, i.e. existing .jar files you obtained from the Internet. Very common examples are log4j (a logging framework) or JDBC libraries for accessing a database etc. Also you might have your own sub-modules that are deployed in separate jarfiles (like 'utilities_1_0.jar' above). How things are split over jarfiles is a deployment/distribution matter, they still all share the universal namespace for Java source code. So in effect, you could unzip all the jarfiles and put the contents in one big directory structure if you wanted to (but you generally don't).
When running a Java application which uses/consists of multiple libraries, you run into what is commonly referred to as 'Classpath hell'. One of the biggest drawbacks of Java as we know it. (note: help is supposedly on the way). To run a Java application on the command line (i.e. not from Eclipse) you have to specify every single .jar file location on the classpath. When you are using one of Java's many frameworks (Maven, Spring, OSGi, Gradle) there is usually some form of support to alleviate this pain. If you are building a web application you would generally just have to adhere to its layering/deployment conventions to be able to easily deploy the thing in the web container of your choice (Tomcat, Jetty, Glassfish).
I hope this gives some general insight in how things work in Java!
[a] To make an executable jar of the MyApp application you need a JDK on your path. Then use the following command line in your compile (bin or target) directory:
jar cvfe myapp.jar com.mycompany.myapp.MyApp com\mycompany\myapp
You can then execute it from the command line with:
java -jar myapp.jar
or by double-clicking the jar file. Note you won't see the Java console in that case so this is only useful for applications that have their own GUI (like a Swing app) or that may run in the background (like a socket server).

Maven has a well thought out standard directory layout. Even if you are not using it Maven directly, you can think of this as a defacto standard. Maven "multi module" projects are a fair analogy to the .net multiple assembly layout that you described.

Typically you would create related/sub projects as different Projects in Eclipse.

There are two things you need to clarify before this question can be answered:
Which source code repository will you use?
Which build system will you use to automatically build artifacts outside of Eclipse?
The answers will strongly influence your options.
We have opted for "one Eclipse project pr component" which may be either a library or a finished runnable/executable jar. This has made it easy to automate with Hudson. Our usage of CVS is also easier, since single projects do not have multiple responsibilities.
Note, each project may contain several source folders separating e.g. test code from configuration from Java source. That is not as important as simplifying your structure.