From the viewpoint of a Gradle java library author, I understand that a dependency specified in the implementation configuration will be marked with the runtime scope in the resulting POM file that gets published (using the maven-publish Gradle plugin). This makes sense, as anyone consuming my published library doesn't need the dependency for compilation (it is an internal dependency), but instead only for runtime. If I specify a dependency in the api configuration, it will be marked with the compile scope in the resulting POM file, which again makes sense, as anyone consuming my library needs this for compilation (and runtime).
This makes me believe that the meaning of the Maven dependency scope is relative to anyone consuming the component, and not relative to the component itself. Consider a published Maven library (containing Java class files) a dependency marked with compile should mean:
If you compile against me, then use this dependency on the compilation classpath too!
However, according to the Maven docs, it seems that it means:
I was compiled with that dependency on my compilation classpath, and if you want to compile me again, do the same!
If this were true, then one could not distinguish between API-dependencies and implementation-dependencies like Gradle does. Also, it would only make sense if the published component actually contains the sources, not only the class files.
Did Gradle actually "misuse" the meaning of these scopes to make some improvements, or did I fundamentally misunderstand something?
Gradle cleverly "misuses" the scopes.
Maven has the design flaw that the build POM is published 1:1 as consumer POM (this will change with the upcoming Maven 4.x). So Maven does not have the chance to use something for compilation in the project, but for runtime when consumed by another project (at least not without applying tricks). The Maven docs therefore do not discuss the possibility of "implementation/api".
Related
In my multi-module project, I created module-info.java only for few modules. And during compilation with maven-compiler-plugin:3.7.0 I'm getting next warning:
[WARNING]
* Required filename-based automodules detected. Please don't
publish this project to a public artifact repository! *
What does it mean? Is that because I have only a few modules with module-info.java and not the whole project?
Automatic module recap
An explicit module (i.e. one with a module-info.java) can only access code of modules that it requires (ignoring implied readability for a moment). That's great if all dependencies are modularized, but what if they are not? How to refer to a JAR that isn't modular?
Automatic modules are the answer: Any JAR that ends up on the module path gets turned into a module. If a JAR contains no module declaration, the module system creates an automatic module with the following properties:
inferred name (this is the important bit here)
reads all other modules
exports all packages
Maven relies on that mechanism and once you create a module-info.jar it places all dependencies on the module path.
Automatic names
There are two ways to infer an automatic module's name:
entry in the manifest
guess from the JAR file name
In the first case, the name was deliberately picked by the maintainer, so it can be assumed to be stable (for example it doesn't change when the project gets modularized). The second one is obviously unstable across the ecosystem - not all project setups lead to the exact same file names of their dependencies.
What does it mean?
The reason for the warnings is that some of your dependencies are automatic modules and do not define their future module name in the manifest. Instead, their name is derived from the file name, which makes them unstable.
Stable names
So why are unstable names such a problem? Assume your library gets published with requires guava and my framework gets published with requires com.google.guava. Now somebody uses your library with my framework and suddenly they need the modules guava and com.google.guava on their module path. There is no painless solution to that problem, so it needs to be prevented!
How? For example by discouraging developers from publishing artifacts that depend on filename-based automatic modules. 😉
[WARNING] * Required filename-based automodules detected. Please don't
publish this project to a public artifact repository! *
Is that because I have only a few modules with module-info.java and not the whole project?
No, its not because of a few module listed on the module-info.java but generated by the maven-compiler-plugin for all the automatic modules found in the module graph.
What does it mean?
Not to publish the current project is insisted probably since the automatic modules are expected to be converted to named or explicit modules by their owners and then published to repositories which might result in change to their module name as well. Additionally a precautionary point to note here is that according to the progress document of Maven ~> Java+9+-+Jigsaw, they are still not completely ready with the JDK9 compatible plugin versions.
Just to portray an example for such a use case. Think over these lines -
I've published an artifact com-foo-bar:1.0.0-SNAPSHOT:jar.
Another project of mine com-xyz:1.0.0 depends on it.
Eventually a project of yours relies on com-foo-bar transitively via com-xyz
You plan to modularize your code and make use of something like
module your.module {
requires com.foo.bar;
requires com.xyz;
}
(you need to specify the transitive dependencies in the module declarations separately)
All worked fine, but until the time I decided to modularize my libraries.
Now, the first thing I did was name my modules!
And I did something fantastic to explicitly call out my efforts like this:-
module modular.com.foo.bar {}
I end up breaking the code of any dependent library and eventually any that depends on yours in a modularized way.
Note: I agree over not practicing to use SNAPSHOTs in production, but there could be cases when eventually you rely on an artifact which is still in development phase.
Edit: From the comments by #khmarbaise
Its understood that people would wish to publish to artifactories but
if they are not aware of the consequences you will be beaten in the
future by this.
Maven would like to make it clear that the WARNING in this case is very
serious which could have instead been a FAILURE but that would had been a
bad user experience to deal with.
The ideal way to deal with this is that the library owners plan to migrate their artifacts to JDK9 and the tree is traversed bottom-up in which case the named/explicit module would be the only aspect prevailing without the need of the automatic module names and such warnings.
With the maven-compiler-plugin v3.7.0 it's an informational message. Not sure why you see it as a warning...
Here's what I get when I build my Java 10 based project with a module-info.java:
[INFO] Required filename-based automodules detected. Please don't publish this project to a public artifact repository!
We are building an ear that is going to run on a Websphere where j2ee.jar is provided.
Now we have the situation that an ejb (call it ejb.jar) depends on another jar (call it util.jar) which depends on j2ee.jar.
If mark j2ee.jar in the pom of util.jar as "provided", the ejb.jar won't build because provided is not transitive. If we mark it as "compile", it may become a compile dependency of the ear, unless we overwrite the scope.
What is the best approach? Should util.jar have provided dependencies, even if it is just a humble jar? Or should jars only have compile dependencies?
JARs can have provided dependencies... but the user having a dependency on it needs to make sure that this dependency is actually going to be provided at run-time. Since provided dependencies are not transitive, they also need to make sure that they do not depend on it for compilation; but if they do, the best practice would be to declare it explicitly with the compile (or provided) scope, and not rely on some form of transitivity (look at the analyze goal of the Dependency Plugin, which, for example, lists used, but undeclared, dependencies).
Provided dependencies in JARs can be useful when creating executable JARs. Consider the building of an uber-jar (a JAR with the classes all of its dependencies included in it): you may want to say that a specific dependency shouldn't end up in the uber-jar, because the container launching it will provide it at run-time.
Also, a JAR may need a dependency to compile its code, but does not actually need it to run; as example, consider Maven plugins which declares maven-plugin-annotations as a provided dependency because they only need the annotations to be built.
Final point, there are JARs that have a good idea in which context they are going to be used: Spring WebMVC, for example, certainly depends on the Servlet API to compile, but at run-time, it knows it's going to be used in a Java EE context, and that the Servlet API will be provided by the Java EE server.
As a rule of thumb though, apart from the cases above, you probably don't want to have provided JAR dependencies inside of a JAR project: it should be up the client to decide whether some compile-time dependencies of yours are going to be provided for their specific case, and let the client override the scope. As a library writer, you don't really know how your library is going to be used.
In your specific case, since ejb.jar actually needs j2ee.jar to compile, it would be best to declare that dependency with the compile, or even with the provided scope in your case, regardless of what scope util.jar has set for j2ee.jar. (I'll note that it's weird for an utility JAR to have a dependency on what appears to be a JAR from Java EE web application classes.)
With maven we can exclude some transitive dependencies.
But what if we need them at runtime, and still as architect I don't want them to be used and become API dependencies.
Is there tool to define and check for unwanted used dependency (i.e. imported in some Java class)?
A search here gives me hint for maven
In maven, can you disallow usage of transitive dependency in your code but still keep it in the classpath?
But that may be to laborious to define. Maybe IDE tools should be used?
How to disallow import and use of some transitive maven dependencies?
So that code will not be accessing different layers of our stack.
Yes, I understand that some educational work should go as well.
When dealing with maven there is a concept of dependency scope. Typically for Java EE applications you will see 3 scopes leveraged the most:
Compile - This is the default scope used if none is specified. These will be available in all of a project's classpaths.
Provided - This scope is used when the dependency is not needed for compilation, but is expected to be in the container at runtime.
Test - This is a dependency needed for testing, but not required for the normal use of the application
Based off of your use case I believe you are looking to leverage the provided scope for a dependency where the dependency is needed at runtime, but should not be available to the application during compilation. You can read more about dependency scopes at: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
I am coming from .NET background and I need to do some JAVA work these days. One thing I don't quite understand is how JAvA runtime resolve its jar dependencies. For example, I want to use javax.jcr to do some node adding. So I know I need to add these two dependencies because I need to use javax.jcr.Node and org.apache.jackrabbit.commons.JcrUtils.
<dependency>
<groupId>javax.jcr</groupId>
<artifactId>jcr</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>org.apache.jackrabbit</groupId>
<artifactId>jackrabbit-jcr-commons</artifactId>
<version>2.8.0</version>
</dependency>
</dependency>
Now I passed the compilation but I get an exception in runtime. Then someone told me to add one more dependency which solves the problem.
<dependency>
<groupId>org.apache.jackrabbit</groupId>
<artifactId>jackrabbit-jcr2dav</artifactId>
<version>2.6.0</version>
</dependency>
From my understanding, jackrabbit-jcr-commons needs jackrabbit-jcr2dav to run. If the jar misses a dependecy, how can it pass the compilation? And also how do I know I miss this particular dependency from jcr-common? This is a general question, it doesn't have to be specific to java jcr.
Java doesn't have any built-in way to declare dependencies between libraries. At runtime, when a class is needed, the Java ClassLoader tries to load it from all the jars in the classpath, and if the class is missing, then you get an exception. All the jars you need must be explicitly listed in the classpath. You can't just add one jar, and hope for Java to transitively load classes from this jar dependencies, because jar dependencies are a Maven concept, and not a Java concept. Nothing, BTW, forbids a library writer to compile 1000 interdependant classes at once, but put the compiled classes in 3 several different jars.
So what's left is Maven. I know nothing about JCR. But if a jar A published on Maven depends on a jar B published on Maven, then it should list B in its list of dependencies, and Maven should download B when it downloads A (and put both jars in the classpath).
The problem, however, is that some libraries have a loose dependency on other libraries. For example, Spring has native support for Hibernate. If you choose to use Spring with Hibernate, then you will need to explicitly declare Hibernate in your dependencies. But you could also choose to use Spring without Hibernate, and in that case you don't need to put Hibernate in the dependencies. Spring thus chooses to not declare Hibernate as one of its own dependencies, because Hibernate is not always necessary when using Spring.
In the end, it boils down to reading the documentation of the libraries you're using, to know which dependencies you need to add based on the features you use from these libraries.
Maven calculates transitive dependencies during compile-time, so compilation passes ok. The issue here is that, by default, maven won't build a proper java -cp command line to launch your application with all of its' dependencies (direct and transitive).
Two options to solve it:
Adjust your Maven project to build a "fat jar" -- jar which will include all needed classes from all dependencies. See SO answer with pom.xml snippet to do this: https://stackoverflow.com/a/16222971/162634. Then you can launch by just java -cp myfatjar.jar my.app.MainClass
For multi-module project, with several result artifacts (that is, usually, different java programs) it makes sense to build custom assembly.xml which will tell Maven how to package your artifacts and which dependencies to include. You'll need to provide some kind of script in resulting package which will contain proper java -cp ... command. As far as I know, there's no "official" Maven plugin to build such a script during compilation/packaging.
There's free Maven book which more or less explains how dependencies and assemblies work.
Your question mixes Maven (a java-centric dependency resolution tool) and Java compile-time and run-time class-resolution. Both are quite different.
A Java .jar is, in simplified terms, a .zip file of Java .class files. During compilation, each Java source file, say MyClass.java, results in a Java bytecode file with the same name (MyClass.class). For compilation to be successful, all classes mentioned in a Java file must be available in the class-path at compile-time (but note that use of reflection and run-time class-name resolution, ala Class.forName("MyOtherClass") can avoid this entirely; also, you can use several class-loaders, which may be scoped independently of each other...).
However, after compilation, you do not need to place all your .class files together into the same Jar. Developers can split up their .class files between jars however they see fit. As long as a program that uses those jars only compile-time refers to and run-time loads classes that have all their dependencies compile-time and run-time available, you will not see any runtime errors. Classes in a .jar file are not recompiled when you compile a program that uses them; but, if any of their dependencies fails at run-time, you will get a run-time exception.
When using Maven, each maven artifact (typically a jar file) declares (in its pom.xml manifest file) the artifacts that it depends on. If it makes any sense to use my-company:my-library-core without needing my-company:my-library-random-extension, it is best practice to not make -core depend on -random-extension, although typically -random-extension will depend on -core. Any dependencies of an artifact that you depend on will be resolved and "brought in" when maven runs.
Also, from your question, a word of warning -- it is highly probable that jackrabit-jcr2dav version 2.6.0 expects to run alongside jackrabbit-jcr-commons version 2.6.0, and not 2.8.0.
If I had to guess (without spending too much time delving into the Maven hierarchies of this particular project), I believe your problem is caused by the fact that jackrabbit-jcr-commons has an optional dependency on jackrabbit-api. That means that you will not automatically get that dependency (and it's dependencies) unless you re-declare it in your POM.
Generally speaking, optional dependencies are a band-aid solution to structural problems within a project. To quote the maven documentation on the subject (http://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html):
Optional dependencies are used when it's not really possible (for
whatever reason) to split a project up into sub-modules. The idea is
that some of the dependencies are only used for certain features in
the project, and will not be needed if that feature isn't used.
Ideally, such a feature would be split into a sub-module that depended
on the core functionality project...this new subproject would have
only non-optional dependencies, since you'd need them all if you
decided to use the subproject's functionality.
However, since the project cannot be split up (again, for whatever
reason), these dependencies are declared optional. If a user wants to
use functionality related to an optional dependency, they will have to
redeclare that optional dependency in their own project. This is not
the most clear way to handle this situation, but then again both
optional dependencies and dependency exclusions are stop-gap
solutions.
Generally speaking, exploring the POMs of your dependencies will reveal this kind of problem, though that process can be quite painful.
I am working on a big project that consists of about 40 sub-projects with very not optimized dependencies. There are declared dependencies that are not in use as well as used but undeclared dependencies. The second case is possible when dependency is added via other dependency.
I want to remove redundant and add required dependencies. I ran mvn dependency:analyze and got a long list of warnings I have to fix now.
I wonder whether there is maven plugin or any other utility that can update my pom.xml files automatically. I tried to do it manually but it takes a lot of time. It seems it will take a couple of days of copy/paste to complete the task.
In worse case I can write such script myself but probably ready stuff exists?
Here is how mvn dependency:analyze reports dependency warnings:
[WARNING] Used undeclared dependencies found:
[WARNING] org.apache.httpcomponents:httpcore:jar:4.1:compile
[WARNING] Unused declared dependencies found:
[WARNING] commons-lang:commons-lang:jar:2.4:compile
[WARNING] org.json:json:jar:20090211:compile
I would not say: with very not optimized dependencies. it's simply someone has not done his job well, cause defining dependencies which are not used shows someone didn't understand what a build tools is and how its working. That can be compared with a Java file which contains many unused imports. In the case of the unused imports in a Java sources this can simply be handle by the IDE but for dependencies in Maven there does not exist such a simple way as already been expressed the problem are kinds of DI etc. which makes this job hard. You can try to output the result of dependency:analyze into a script (there exist an option for that goal) and afterwards test the resulting build after cleaning up the dependencies.
It might be a good idea to run
mvn dependency:analyze -DscriptableOutput=true
which produces output which can be very simple extracted from the output and can be used for further processing like using as input for the versions-maven-plugin (with some pre conversion).
I would not recommend to clean-up dependencies automatically.
Adding of all 'Used undeclared...' lead to duplication of most of transitive dependencies that lead to spending of more time to reading and managing them.
Removing of all 'Unused declared...' can lead to errors in run-time because they are: called by reflection or specially declared to override version of the same artifact that already used in 3rd party dependencies (changing their compile scope to runtime is preferable, while test scope should be untouched to avoid leak them to production package) or added to declare usage of an optional transitive dependency of some 3rd party library etc.