I'm currently in the process of removing the Spring dependency from Flyway. In the future though other types of dependencies might be needed to support a subset of users (such as JBoss VFS support).
Which is the best way to support optional dependencies (optional=true in Maven POM)?
Qualities of the solution would be:
Ease of use for end-users (minimum work required to use functionality if dependency is present)
Ease of use for developers (code dealing with optional dependency should be as readable and as straightforward as possible)
No unnecessary required dependencies (if some end-user don't need this functionality, no need to pull in the dependency)
I think Maven's optional dependency functionality is quite limited.
http://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html
Optional dependencies will not get pulled down (as transitive dependencies) by default. However, if your users need to use these optional features the missing dependencies must be explicitly declared, in their POM.
Personally, it's not clear to me how this is helpful to users.... I suppose the optional dependencies in your POM do document which versions your code depends on. Not all users however will read the POM, all they'll see is the "NoClassDef Found" error :-(
My final observation is that this is one of those rare scenarios where a dependency manager like ivy offers more flexibility. Ivy has a concept called "configurations". Module authors can assemble different combinations of dependencies, for example "with-spring" or "without-spring".
There's a choice of:
keep the project in a single module; and use optional dependencies.
split the project into multiple modules; where each module has a (non-optional) dependency on any libraries;
I think the first makes more sense in most cases: users need to figure out their way around fewer artifacts. Typically, they'll have to add fewer new dependencies to their pom. Unless the code to support third-party projects is large, this will help improve maven download times too (fewer round-trips). With the latter approach, you can find yourself in awkward situations where the user has defined their own set of versions, but only for some of the third-party dependencies.
I prefer to see the optional dependencies in the pom (I sometimes look to see which version it's built against). It's true that some people might not look. I think copy-and-pasteable pom snippets on the website is the best solution for that. For example, if you have a page about Spring integration, you could put the relevant pom snippet on that page.
I'd suggest that non-free dependencies (or anything not easily resolvable) be kept in a separate maven module, so that contributors are always able to build the primary artifact. (I had that problem with Quartz, which IIRC has an optional dependency on an Oracle JDBC jar).
Edit: If you're worried about users seeing NoClassDefFoundErrors, it wouldn't do any harm to check that the class can be resolved before trying to use it. For example, you could can an exception, and throw a more meaningful error message pointing the user to documentation. SLF4J is a good example of this.
Related
How do I know which dependencies I need to suppress? Basically, what makes the difference in the dependencies that are not relevant to those that are?
an example would be extremely helpful.
I would strongly discourage you from asking security related questions here: you are not looking for financial advices on SO, the same applies for security - hire security expert and route all such questions to him.
In regards to dependency-check-maven from OWASP:
It is a bad idea to embed it into build lifecycle the way that it fails the build if vulnerable dependencies have been found (failBuildOnAnyVulnerability, failBuildOnCVSS, etc), otherwise it may suddenly stop the entire SDLC, but developers are not responsible for SEC stuff. Reporting is OK though - just give SEC guys a chance to participate in SDLC via reviewing reports and giving suggestions
If dependency-check-maven did find nothing it does not mean the application is not vulnerable at the current moment and won't be vulnerable in the future - unfortunately, security is not a state but a process, that in turn means you need to perform such checks periodically, i.e. either maintain CMDB with detailed information about installed applications and perform checks against that CMDB or use dependabot or other tools.
dependency-check-maven uses NVD database as a source of information about vulnerabilities, unfortunately neither NVD nor MITRE are actually performing security analysis - they just maintain database, that causes some dumb cases when researcher reports "vulnerability" which is actually not a vulnerability, some examples: CVE-2021-31684, CVE-2022-26520, however, investigating such cases is not a developer responsibility
If you have the time, you can remove all the dependencies from your pom, keeping them aside in a .bak file.
Of course, the compilation will fail,
but then you add them back, one by one, and only the one Maven build is complaining about their lack.
Then you execute and check that everything in your application is still working well.
That way, you seldom remove two or three dependencies when the project is very old.
Each Maven dependency can provide some of transitive dependencies.
Each used dependency in your project is relevant, simply project will not compile without it. Some of dependencies can be needed in runtime by your project.
You can read about Maven Dependency Mechanism and how to exclude dependencies.
You can also examine output of:
mvn dependency:analyze
mvn dependency:tree
Please familiarize yourself with documentation of Maven Dependency Plugin
I am a library maintainer and I came to a point where I discovered that one of my libraries can result into a dependency issue for the end user as it provides a transitive dependency to the end users. The end-user might be using a different version of the transitive one and therefore it can result into unexpected behaviour. I don't want to enforce the end user to use a specific version by providing the transitive one however I still want my library to be functional and therefore I don't know what is the best way to solve this issue. Should I use the default dependency scope or should I switch to the scope provided?
I also want to provide some context to make this question more clear. I created sslcontext-kickstart which is just a high-level library for configuring ssl for a server or client. The library has additional separate dependencies which the end-user can use to make it more easy to use for their use case. So the core library has only a dependency on slf4j-api. However there are separate libraries which contain mappers which I also created for apache4, apache5, netty and jetty which relay on the core library. Apache4, apache5, netty and jetty are currently a compile scoped dependency and therefor the end-user will also get the version which is specified in my pom. Let's assume someone is using the apache4 version. So should the end-user exclude the dependency manually when they are using my library and don't want the specific transitive apache4 dependency? Or should I mark apache4 as provided scope type. In that way there will be no transitive dependency, however the end-user should have the apache4 dependency present on their classpath or else they will get a runtime exception when they use my library.
What do you guys think regarding this topic? What are my options and which should I choose in your opinion?
Two options:
Mark the seldomly needed dependencies as optional (see e.g. https://stackoverflow.com/a/40398649/927493).
Create different JARs for the different target platforms like sslcontenxt-kickstart-for-apache4. Then the users of the library can choose whichever fits best for their needs without getting the unwanted dependencies.
We have a project which depends on Aspose Words' com.aspose:aspose-words:16.10.0:jdk16.
The POM for aspose-words declares no dependencies, but this turns out to be a lie. It actually uses jai-core, latest version of which is at javax.media:jai-core:1.1.3.
The POM for jai-core, though, also lies - it declares no dependencies, but actually depends on jai-codec, which is at com.sun.media:jai-codec:1.1.3.
Getting these projects to fix things seems impractical. JAI is basically a dead project and Maven Central have no idea who added that POM so there is nobody responsible for fixing the metadata. Aspose refuse to fix things without a test reproducing it, even if you can show them their own code doing it wrong, and even if they fixed it, they would then add their dependency on jai-core:1.1.3, which only fixes half the problem anyway.
If I look at our entire tree of dependencies, this is only one example of the problem. Others are lurking, masked out by other dependency chains coincidentally pulling in the missing dependency. In some cases, we have even reported POM issues to projects, only for them to say that the dependency "isn't real", despite their classes clearly referring to a class in the other library.
I can think of a few equally awkward options:
Create jai-core:1.1.3.1 and aspose-words:16.10.0.1 and fix their POMs to include the missing dependencies, but whoever updates them in the future will have to do the same thing. Plus, any other library I don't know about which happens to depend on jai-core would also have to be updated.
Add a dependency from our own project, even though it really isn't one.
Edit the POM for the versions which are there now to fix the problem directly, only caveat being that people might have cached the wrong one.
So I guess I have two related questions about this:
Is there no proper way to resolve this? It seems like any non-toy project would eventually hit this problem, so there not being an obviously correct way to deal with it is worrying.
Is there a way to stop incorrect dependency metadata getting into the artifact server in the first place? It's getting kind of out of hand, because other devs on the team are adding the dependencies without checking things properly, and then I'm left to clean up their error when something breaks a year later.
Tunaki has already given many good approaches. Let me add the following:
We had to deal with a lot of legacy jars which are some old or strange versions of already existing jars on MavenCentral. We gave them a special kind of version number (like 1.2.3-companyname) and created a POM for them that fitted our purposes. This is - more or less - your first "awkward option". This is what I would go for in your case; additionally, I would define the version in the dependencyManagement, so that Maven dependency mediation will not set it to some other version.
If a new version of your jar comes around, you can check if it still has the same problems (if they did a correct Maven build, they should have all dependencies inside the POM). If so, you need to fix it again.
I wouldn't change poms for already existing versions because it confuses people and may lead to inconsistency problems because Maven will not grab the new POM if an old version is already in the local repository. Adding the dependency to your own project is an option if you have very few projects to manage so that you still see what is going on (a proper comment on the dependencies in the POM could make it clearer).
JAI is optional for Aspose.Words for Java. Aspose.Words for Java uses JAI image encoders and decoders only if they available. And it will work okay without JAI.
The codecs complement standard java ImageIO encoders/decoders. The most notable addition is support of Tiff.
JAI (Java Advanced Imaging) is not usual library. First of all - it is native library. I.e. it has separate distributives for different platforms. It has also "portable" pure-java distributive, but if you want full power of JAI - you should stick to native option.
Another thing: usually you should run installation of JAI native distributive on the host system. I.e. it installed like desktop application, not like usual java library. Again, JAI codec acts not like usual library: if it installed on system - it will plug into ImageIO, irrelevant to classpath.
So, i don't know good way to install JAI using Maven - it is like using Maven to install Skype or any other desktop application. But it is IMHO, I am not great specialist on Maven:)
We have a multi-module POM, which also serves as a parent POM for all sub-modules involved. Call it MultiModulePOM. We have about 70 modules, say numbered Module1 to Module70.
Now: The first 30 of these modules require a set of JAR files at compile-time only. That is - scope=provided. Since we're talking about a set of JAR files, it is quite tedious to keep those 30 modules in sync and in general, I am not a huge fan of copying definitions around.
So, I fell into the pitfall of dependency grouping. Seemed like a good idea, however it doesn't work for provided dependencies. In other words: if I group the dependent JARs in a module called ExtDependencies, and make Module1 depend on ExtDependencies, the JARs referred-to by ExtDependencies won't be transitively-added to Module1, because their scope is provided.
(If the last paragraph is not true, please let me know as it could really get me out of a jam)
The only other option that I could see was to create a parent POM called (for example) IntermediaryPOM. IntermediaryPOM extends MultiModulePOM and enlists the set of dependent JAR files with scope=provided. Modules Module1-Module30 then extend IntermediaryPOM.
That seemed to do the trick but I have three problems with it:
It adds another layer of POM that I'm not sure is really needed.
Later, during distribution time, I find myself having to install/deploy the intermediary POM's as well.
Consider the general case: the intermediary POM may have other siblings used for other sets of JARs (for modules 31-50). Therefore, this solution doesn't seem to scale well.
So my question is - according to your experience, what is the best way to approach this? any known best practices for such a use case?
I'm afraid there is no easy solution here.
You're right saying that if you declare common dependencies in ExtDependencies as provided they won't be added to the classpath of any other module that is dependent on ExtDependencies. That's how provided works.
But you could declare these common dependencies without scope (e.g. with default compile scope) and add provided dependency on ExtDependencies. In this case all of the ExtDependencies dependencies will be added to classpath. God, that's a lot of "dependencies" :)
You've also mentioned other possible option -- introduce another level of abstraction (which, as you might know, is a way to solve almost any problem). But such multi-level hierarchy is less elegant and more difficult to maintain (I have it in our projects, so I've been there).
In general, I haven't come across this problem in such a scale but if I were to solve it I'd go with the first option taking into account scoping suggestion.
For my previous employer I've worked with Hibernate, and now that I'm in a small startup I would like to use it again. However, downloading both the Hibernate core and the Hibernate annotations distributions is rather painful, as it requires putting a lot of JAR files together. Because the JARs are split up into categories such as "required" and "optional" I would assume that every developer ends up with a different contents of his lib folder.
What is the common way to handle this problem? Basically I want to have a formal way to get all the JARs for Hibernate, so that (in theory) I would end up with exactly the same stuff if I would need again for another project next month.
Edit: I know roughly what Maven does, but I was wondering if there was another way to manage this sort of thing.
As Aaron has already mentioned, Maven is an option.
If you want something a bit more flexible you could use Apache Ant with Ivy.
Ivy is a dependency resolution tool which works in a similar way to Maven, you just define what libraries your project needs and it will go off and download all the dependencies for you.
Maybe this is not much of an answer, but I really don't see any problem with Hibernate dependencies. Along with hibernate3.jar, you need to have:
6 required jars, out of which commons-collections, dom4j and slf4j are more often used in other open-source projects
1 of either javassist or CGLIB jars
depending on cache and connection pooling, up to 2 jar files, which are pretty much Hibernate specific
So, at the very worst, you will have a maximum of 10 jars, Hibernate's own jar included. And out of those, only commons-collections, dom4j and slf4j will probably be used by some other library in your project. That is hardly a zillion, it can be managed easily, and surely does not warrant using an "elephant" like Maven.
I use Maven 2 and have it manage my dependencies for me.
One word of caution when considering using Maven or Ivy for managing dependencies is that the quality of the repository directly affects your build experience. If the repo is unavailable or the meta-data for the artifacts (pom.xml or ivy.xml) is incorrect you might not be able to build. Building your own local repository takes some work but is probably worth the effort. Ivy, for example, has an ANT task that will import artifacts from a Maven repository and publish them to you own Ivy repository. Once you have a local copy of the Maven repo, you can adjust the meta-data to fit what ever scheme you see fit to use. Sometimes the latest and greatest release is not in the public repository which can sometimes be an issue.
I assume you use the Hibernate APIs explicitly? Is it an option to use a standard API, let's say JPA, and let a J2EE container manage the implementation for you?
Otherwise, go with Maven or Ivy, depending on your current build system of choice.