I expected it's possible to use i.e. Guava-19 in myModuleA and guava-20 in myModuleB, since jigsaw modules have their own classpath.
Let's say myModuleA uses Iterators.emptyIterator(); - which is removed in guava-20 and myModuleB uses the new static method FluentIterable.of(); - which wasn't available in guava-19. Unfortunately, my test is negative. At compile-time, it looks fine. In contrast to runtime the result is a NoSuchMethodError. Means that, the class which was the first on the classloader decides which one fails.
The encapsulation with the underlying coupling? I found a reason for myself. It couldn't be supported because of transitive dependencies would have the same problem as before. If a guava class which has version conflicts occurred in the signature in ModuleA and ModuleB depends on it. Which class should be used?
But why all over the internet we can read "jigsaw - the module system stops the classpath hell"? We have now multiple smaller "similar-to-classpaths" with the same problems. It's more an uncertainty than a question.
Version Conflicts
First a correction: You say that modules have their own class path, which is not correct. The application's class path remains as it is. Parallel to it the module path was introduced but it essentially works in the same way. Particularly, all application classes are loaded by the same class loader (by default at least).
That there is only a single class loader for all application classes also explains why there can't be two versions of the same class: The entire class loading infrastructure is built on the assumption that a fully qualified class name suffices to identify a class with a class loader.
This also opens the path to the solution for multiple versions. Like before you can achieve that by using different class loaders. The module system native way to do that would be to create additional layers (each layer has its own loader).
Module Hell?
So does the module system replace class path hell with module hell? Well, multiple versions of the same library are still not possible without creating new class loaders, so this fundamental problem remains.
On the other hand, now you at least get an error at compile or launch due to split packages. This prevents the program from subtly misbehaving, which is not that bad, either.
Theoretically it is possible to use different versions of the same library within your application. The concept that enables this: layering!
When you study Jigsaw under the hood you find a whole section dedicated to this topic.
The idea is basically that you can further group modules using these layers. Layers are constructed at runtime; and they have their own classloader. Meaning: it should be absolutely possible to use modules in different versions within one application - they just need to go into different layers. And as shown - this kind of "multiple version support" is actively discussed by the people working on java/jigsaw. It is not an obscure feature - it is meant to support different module versions under one hood.
The only disclaimer at this point: unfortunately there are no "complete" source code examples out there (of which I know), thus I can only link to that Oracle presentation.
In other words: there is some sort of solution to this versioning problem on the horizon - but it will take more time until to make experiences in real world code with this new idea. And to be precise: you can have different layers that are isolated by different class loaders. There is no support that would allow you that "the same object" uses modV1 and modV2 at the same time. You can only have two objects, one using modV1 and the other modV2.
( German readers might want to have a look here - that publication contain another introduction to the topic of layers ).
Java 9 doesn't solve such problems. In a nutshell what was done in java 9 is to extend classic access modifiers (public, protected, package-private, private) to the jar levels.
Prior to java 9, if a module A depends on module B, then all public classes from B will be visible for A.
With Java 9, visibility could be configured, so it could be limited only to a subset of classes, each module could define which packages exports and which packages requires.
Most of those checks are done by the compiler.
From a run time perspective(classloader architecture), there is no big change, all application modules are loaded by the same classloader, so it's not possible to have the same class with different versions in the same jvm unless you use a modular framework like OSGI or manipulate classloaders by yourself.
As others have hinted, JPMS layers can help with that. You can use them just manually, but Layrry might be helpful to you, which is a fluent API and configuration-based launcher for running layered applications. It allows you to define the layer structure by means of configuration and it will fire up the layer graph for you. It also supports the dynamic addition/removal of layers at runtime.
Disclaimer: I'm the initial creator of Layrry
Related
Is there a way to automatically find out which Java classes are actually loaded (either during compile time, as far as that's possible, or during the runtime of an application), and to throw out all other classes from a JAR to create a smaller JAR? Does that actually make sense in practice?
I am talking about the application classes for an application JAR. Usually there are lots of libraries in an application, and an application rarely needs all features of those libraries. So I suspect that would make a considerably smaller application. In theory that might be done for example via an Java agent that logs which classes and resources are read by one or several runs of an application (or even just by java -verbose:class), and a maven plugin that throws out all other classes from a jar-with-dependencies. Is there already something like that?
Clarification: I am not talking about unused dependencies (JARs that are not used at all), but about removing unused parts of each included JAR.
Well, the Maven Shade Plugin has an option minimizeJar when creating an Uber-JAR for your application:
https://maven.apache.org/plugins/maven-shade-plugin/
But, as others already pointed out, this is quite dangerous, as it regularly fails to detect class accesses which are done via Reflection or other dynamic references.
It may not be a good approach automate, as application can use reflection to initialise objects or one JAR is dependent on another JAR.
Only way that I can think of is to remove each JARs one by one and check if application runs as expected. Then again in this approach all modules of the application has to be tested, since one module can work without particular dependency and other may not.
Better solution is to take care while developing. The application developer must be careful in adding a dependency and removing unwanted dependency after his/her piece of code is done.
Global strategy.
1) Find all the classes that are loaded during runtime.
2) List of all the classes available in the classpath.
3) Reduce your class path by creating copies of jars containing only classes you need.
I have done 1 and 2 part so I can help you.
1) Find out all the classes that are loaded. You need 100 % code coverage (I am not talking about tests, but production). So run all possible scenarios, so all the classes your app needs will be loaded and logged.
To log loaded classes try several approaches. Reflection, –verbose:class flag, also you can learn about java agent. It allows to modify methods during runtime. This is an example of some java agent code or another java agent example
2) To find all the classes available in jar, you can write a program. You need to know all places where application jars are placed. Loop throw these jars (You can use ZipFile), loop through ZipFileEntry entries, and collect all classes.
3) After that write a script or program that reassembles your application. For example, now you can create a new jar file for each library and put there only needed classes.
Also you may use a tool (again, you are a programmer, so write a program), which checks code for classes dependence. You do not want to remove classes if they are used for compilation. When I was a student, I wrote code alanyzer, which builds an oriented graph for classes dependencies.
As #Gokul Nath KP notes, I did this before. I manually change gradle and maven dependencies, removing one by one, and then full regression test. It took me a week (our application was small comparing to modern world enterprise systems created by hundreds of developers).
So, be creative, and in case of success, your project will be used by millions!
In clean architecture the structure is like that:
CORE:
CoreClass.java
SomeDAOInterface.java
IO
SomeDAOInterfaceImpl.java (implement SomeDAOInterface)
If I was supposed to split Core and IO in different .jar files, different projects, how am I supposed to handle "SomeDAOInterface" dependency in IO part? It is only contained in Core part, so I cannot really implement it without compiler error (no class SomeDAOInterface found).
What you describe is far from an unusual design, and there are plenty of examples around. For example Java EE declares a number of interfaces which are to be implemented by various containers. Or Jdbc also declares interfaces which will be implemented by database engines.
There are 2 possible designs depending on whether the binding will occur at build time or at run time.
When binding occurs at build time (common for jdbc for example), you must have an implementation available at build time, for example you declare a MySQL database driver in your project. In your example, it means the the IO project will depend on the Core one.
When binding occurs at run time (Java EE for example), you use a dummy project that only contains the interface classes (SomeDAOInterface in your example) and not the implementations for compilation and declare to the builder not to link it in the final jar but that it will be provided at run time. And at run-time you do provide in classpath a full implementation, containing both the interface classes (SomeDAOInterface) and the implementation ones (SomeDAOInterfaceImpl in your example). You will just have to read your build system documentation to know how to declare that.
Alternately, you can link the dummy project in the core jar, and declare that it will be provided in the implementation one.
If u r talking about Clean Architecture from Uncle Bob then I wonder what the CORE project is?
In case u refer to the "entities circle" then having the interface defined there is fine IF this is really part of ur core business rules.
u would then create a dependency from ur IO project (which is in frameworks or interface adapters layer) to the CORE project which is correct according to the dependency rule.
For a more detailed discussion on project structures in Clean Architecture pls refer to my post: https://plainionist.github.io/Implementing-Clean-Architecture-Scream/
I am beginner in JPMS and can't understand its dynamism. For example, in current JVM instance moduleA.jar is running. moduleA requires only java.base module. Now, I want
to load dynamically moduleB.jar that needs java.sql module and moduleC.jar
execute some code from moduleB
unload moduleB, java.sql, moduleC from JVM and release all resources.
Can it be done in Java 9 module system?
This is an advanced topic. At a high-level, the module system is not dynamic in the sense that individual modules can not be unloaded or replaced in a running VM. However, you can use the API, specifically java.lang.module.Configuration and java.lang.ModuleLayer to create dynamic configurations of modules and instantiate them in a running VM as a layer of modules. In your scenario, then you may create a layer of modules with modules B and C. This layer of modules will be GC'ed/unloaded once there are no references to them.
As I said, this is an advanced topic and it's probably better to spend time mastering the basics, including services, before getting into dynamic configurations and module layers.
The other answer is fully correct, but please note that "in the end" these things didn't really change.
Before Java 9 you could use custom class loader instances to achieve something like this. That is for example how application servers such as Tomcat allow you to re-deploy an application - by basically throwing away a whole "context" that was initially "built" using a specific class loader instance.
With Java9, this concept is described using that layers abstraction - but in the end it still means that custom code needs to provide all the implementation of actually creating layers with different class loaders.
And for some further read on layers see this answer that I gave some time back on a similar question (which focused on how to use different versions of the same module within a single application).
In trying to learn about Java class loaders from Wikipedia, I think I can see why they have the three major class loaders:
1) Bootstrap class loader
2) Extensions class loader
3) System class loader
They go on to say you can define your own classloader. I'm not sure I see the value in defining your own, but the following quote from Wikipedia really makes me wonder:
The most complex JAR hell problems arise in circumstances that take
advantage of the full complexity of the classloading system. A Java
program is not required to use only a single "flat" classloader, but
instead may be composed of several (potentially very many) nested,
cooperating classloaders. Classes loaded by different classloaders may
interact in complex ways not fully comprehended by a developer,
leading to errors or bugs that are difficult to analyze, explain, and
resolve.
If it's so complex, why bother with it? Shouldn't the three already-defined classloaders be enough?
(And yes, for those curious, I did run into a ClassCastException that I didn't think should have happened, much like the graphic labelled Figure 2. Class identity crisis. I'm trying to understand the background is all.)
Certain use cases require custom classloaders.
A few examples:
Dynamically adding new folders/jars to be loadable. (Without restarting the whole application).
Dynamically removing folder/jars from being loadable.
Runtime bytecode generation with javassist.
Multiple (actually used at the same time) versions of the same classes in the same application/jvm
I always doubt when creating packages, I want to take advantage of the package limited access but at the same time I want to have similar classes divided into packages.
The problem comes when you understand that packages are not hierarchical in Java:
At first, packages appear to be
hierarchical, but they are not.
source
Imagine I have an API defined with its classes at foo.bar, only the classes the API client needs are set public. Then I have another package with some internal objects I need in the API defined at foo.bar.pojos, this classes need to be public so they can be accessed by foo.bar but this means the API client could also access them if the package foo.bar.pojos is imported.
What is the common package politic that should be followed?
I've seen two ways of doing.
The first one consists in separating the public API and internal classes into two different artefacts (jars). The documentation is separated as well, and it's thus easy for the end user to make the distinction between what is internal and what is not. But it sometimes make things more complex to have two jars, two source trees, etc.
The second one consists in delivering a single jar, but have a good documentation allowing to know what's internal and what's not. The textual documentation can explain how to use the API (and thus avoids talking about the internals). And the javadoc can specify that a class is for internal use and is thus subject to changes.
Yes, Java packages don't give you enough control over your dependencies. The classic way to deal with this is to put external APIs in one package and internal implementation classes in another, and rely on people's good sense to avoid creating dependencies on the latter.
With Maven and OSGI, you have an additional mechanism for managing dependencies between modules / bundles of packages. In the case of OSGI, you can explicitly declare some packages as not exported, and an OSGI aware development environment will prevent people creating harmful dependencies. Maven's module support is weaker, but at least it controls dependency cycles.
Finally, you could use custom PMD rules to enforce your project's modularization conventions ... in the same way that there are rules to discourage dependencies on Java's "com.sun.*" package tree.
It is a mess.
Using only what Java itself offers, you have to put everything in the same package. You end up with a single (or a few) packages with lots of classes, and no good way to group them for yourself (but at least that problem does not leak outside). Most people don't do that, though, and as a result, your (as a developer on top of these libraries) public classpath is littered with stuff you should never need to see.
You might like OSGi, which has (and enforces) the concept of bundle-private packages. Those are not exported to the outside world.