As I have worked with npm which looks for dependencies in package.json file and download it for you. Similarly, I see a pom.xml file in Java project. Does maven looks in this file and download dependencies for me. Can I pass around this pom.xml file like package.json, rather than giving the dependency jars ? Are these tools similar and just build for different platforms ?
Same tool, different language?
Maven is the most popular build and dependency resolution tool for Java, just like NPM is for JS. But it's not just the same tool for a different language. There are obviously huge differences between Java and JS builds, and these differences are directly visible in the way Maven operates. For example, while many JS tools rely on Git to do some heavy-lifting, Maven works with custom filesystem-based Maven repositories, as Maven predates Git and needs to handle binary artifacts, which Git historically didn't handle well. In Maven there's a clear separation between sources and binaries, while they are often the same thing in JS world.
Maven basics
Maven in its purest form follows a declarative model, where pom.xml (similar to package.json) defines different properties of the build, but contains no scripts. The disadvantage is it can be a challenge to fine-tune some aspects of the build without using scripts as you have to rely on plugins. The advantage is it can be easier to understand other builds just by looking at pom.xml, as they usually follow the same approach without too much customization. Gradle is a popular Groovy-based tool built on top of Maven standards and conventions, and is specifically designed to simplify pom.xml and break this "no script" barrier.
Referencing your dependencies
Similarly to package.json, you don't work with pom.xml of your dependency directly, but rather define dependency coordinates and let your build tool handle the rest. In Maven the basic form of these coordinates is GAV (groupId, artifactId, version).
Flat dependency tree?
Based on comments in the other answer, Maven provides "flat dependency tree", not "nested dependency tree" that NPM provides by default. Maven does not allow multiple versions of the same dependency. If it happens that different versions are requested, Maven uses dependency resolution to pick a single version. This means that sometimes your transitive dependencies will get a different version than they require, but there are ways to manage this. However, this limitation comes from Java, not Maven, as (normally) in Java a class loader will only provide access to a single class definition even if multiple definitions are found on the classpath. Since Java is not particularly good at handling this, Maven tries to avoid this scenario in the first place.
Note: since npm v3 the dependencies are flatten. The alternative package manager yarn also does the same.
Maturity
Furthermore, Maven is considerably older than NPM, has a larger user base, huge number of custom plugins, and so far could probably be considered more mature overall. Sometimes Maven is used for non-Java or even polyglot projects, as there are plugins for handling other languages or specific environments, such as Android. There are plugins that bridge Maven and other build tools, such as frontend-maven-plugin that actually handles multiple JS build tools.
Yes they are similar in the context that their main purpose is to provide a way describing the project dependencies, instead of keeping them within the project code, and their secondary purpose is to provide developers with an easy way to perform, define and share dev-time/build-time tasks. Both of the above are expressed inside a descriptor file.
Now deciding which one to use is, most of the times, straightforward because it depends on the primary language you are working on. A rough grouping is:
java: maven
javascript/typescript: npm
Below I provide a detailed explanation of the common features and differences. I use | to separate between maven | npm terms respectively:
Common features:
Both tools support dynamic fetch of dependencies ( artifacts | packages ) based on a descriptor file pom.xml|package.json, and also allow you to deploy | publish your own artifacts | packages.
They both have a default public repository | registry ( http://repo.maven.apache.org/maven2/ | https://registry.npmjs.org), but 3rd-party can also be used (via settings.xml|.npmrc ).
They both support the concept of build-level dependencies (plugins | devDependencies used in scripts). *Maven supports provided dependencies also but this does not seem to apply to npm, since javascript is rarely deployed into containers.
They both support dependency namespacing: groupId|scope
Differrences:
maven has an additional local repository(cache):
No need to fetch again the same dependency for differrent projects.
Artifacts that are installed locally, are automatically accessible by other local projects.
dependencies from a project build in maven are downloaded in <homedir>/.m2. With npm they are downloaded in <projectdir>/node_modules.
Building in maven is commonly a one-step process: mvn package (fetch deps , build). In npm it is a 2-step process: npm install (fetch deps) , npm build (build)
maven defines build lifecycles (for building,testing,deploying) consisted of phases, to which default operations(plugin goals) attach, based on differrent packaging options(.jar,.war,.ear e.t.c). You can then overwrite these operations, or inject new ones (via the plugin system). This provides kind of an out-of-the box solution for build,docgen,test,deploy e.t.c.
npm approach is more simplistic ( see: scripts)
Due to the above, npm is labeled as a package-management tool for javascript while maven is labeled as a build-automation and dependency-management tool for java.
In maven setting-up the build process more commonly involves editing the pom.xml.
In npm it involves writing code or configuring complementary build tools like gulp,webpack e.t.c
For some reason version ranges defined by users in npm modules, are much more loose than in maven. This can cause issues with transitive dependencies, that is why an additional file was recently added: package-lock.json
With npm it is much more straightforward to start a new project: npm init. With maven, you need to know how to write a minimal pom.xml, or read about archetypes.
In general it is much more common to edit pom.xml than package.json. E.g. adding dependencies in maven is done manually (or via IDE) while in npm via command line.
As with all build tools, you can call one tool from inside the other, but I think its much more common to call npm from inside maven, than the opposite.
npm supports dev,production builds. In maven this needs to be defined through profiles.
yes. it's a similar packaging tool for java. look for gradle also which gives you more liberty with groovy language, but for start you can use maven to organize your dependencies. you include them as tags there and maven does the job for you.
it traverses the dependency tree and downloads all the appropriate jars.
Yes, same with gradle, but they are not user friendly as npm.
Related
I just started developing a project in Spring MVC and i want to know how important Maven is.
The following are the key features of Maven :
Simple project setup that follows best practices - get a new project or module started in seconds
Consistent usage across all projects - means no ramp-up time for new developers coming onto a project
Superior dependency management including automatic updating, dependency closures (also known as transitive dependencies)
Able to easily work with multiple projects at the same time
A large and growing repository of libraries and metadata to use out of the box, and arrangements in place with the largest Open Source projects for real-time availability of their latest releases
6.Extensible, with the ability to easily write plugins in Java or scripting languages
Instant access to new features with little or no extra configuration
Ant tasks for dependency management and deployment outside of Maven
Model based builds: Maven is able to build any number of projects into predefined output types such as a JAR, WAR, or distribution based on metadata about the project, without the need to do any scripting in most cases.
Coherent site of project information: Using the same metadata as for the build process, Maven is able to generate a web site or PDF including any documentation you care to add, and adds to that standard reports about the state of development of the project. Examples of this information can be seen at the bottom of the left-hand navigation of this site under the "Project Information" and "Project Reports" submenus.
Release management and distribution publication: Without much additional configuration, Maven will integrate with your source control system (such as Subversion or Git) and manage the release of a project based on a certain tag. It can also publish this to a distribution location for use by other projects. Maven is able to publish individual outputs such as a JAR, an archive including other dependencies and documentation, or as a source distribution.
Dependency management: Maven encourages the use of a central repository of JARs and other dependencies. Maven comes with a mechanism that your project's clients can use to download any JARs required for building your project from a central JAR repository much like Perl's CPAN. This allows users of Maven to reuse JARs across projects and encourages communication between projects to ensure that backward compatibility issues are dealt with.
Getting started with maven only takes about 10 minutes. Reasons why you should learn maven:
It helps you manage your dependencies very easily so you don't need to add jars to your project classpath manually
You can run unit tests
Has over 20 useful plugins which you can use. Plugins make up lifecycles like test, package which make your work more efficient
You can use it to build your project
The most important thing about it at the beginning is that you don't need to worry about setting up your project by adding dependencies, maven does it for you automatically.
Read this spring guide for building with maven
Any other guides in this section for spring boot has the same mechanism
Is there a way to fail a build in Jenkins if a certain jar is used in a Java Maven Project?
For example I know org.example:badartifact:1.0.1 has a security vulnerability. I told everyone about that, and they fixed their projects..., but maybe some third-party artifacts bring this with them as a transitive and nobody realizes that.
Or maybe someone down the line forgets this old bug...
So I would like to have a last check in Jenkins preferably, so that we don't end up with projects that have that special artifact included.
How do you handle situations like that, what tools do you use? (Whitelisting libs? Blacklisting libs?, etc)
Any suggestions are appreciated.
Possible Maven solution
You could have a company super POM (parent POM of all Maven projects within the company/department/team) and in that super POM configure the Maven Enforcer Plugin, its bannedDependencies rule to ban any library, version or even scope. I have personally used this option even for trivial mistakes (i.e. junit not in test scope would make the build fail).
This solution is a centralized one and as such easier to maintain, however requires all the projects to have the same parent POM and developers could at any time change the parent pom and as such skip this governance. On the other hand, a centralized parent POM is really useful for dependencies Management, common profiles, reporting and so on.
Note: you cannot configure it in the Maven settings of the Jenkins server via an active by default profile, for instance, in order to have it applied to all running Maven build, because Maven limits customization of builds in profiles provided by the settings (it's a design choice, to limit external impact and as such have an easier troubleshooting). I've tried it in the past and hit the wall.
Profiles in external files
Profiles specified in external files (i.e in settings.xml or profiles.xml) are not portable in the strictest sense. Anything that seems to stand a high chance of changing the result of the build is restricted to the inline profiles in the POM. Things like repository lists could simply be a proprietary repository of approved artifacts, and won't change the outcome of the build. Therefore, you will only be able to modify the and sections, plus an extra section
Possible Jenkins solution
If you want to have governance centralized in Jenkins directly, hence independently than Maven builds, I have applied these solutions in the past (and they perfectly work):
Jenkins Text Finder Plugin: you can make the build fail in case a regex or a matching text was found as part of the build output. In your case, you could have a Jenkins build step executing always mvn dependency:tree and as such have as part of the build output the list of dependencies (even transitive). A Text Finder rule matching your banned dependency will then match it and fail the build.
Fail The Build Jenkins Plugin: similar to the one above, but with a centralize management of configured Failure Causes. Again, failures are based on matching text, but no build configuration is required: it will be applied by default to all builds.
Here is one solution to do the job :)
With the Maven License plugin, you can scan the 3rd party dependencies for your Maven project and produce a THIRD_PARTY.txt report (in the target/generated-sources/license folder).
Maven command line:
mvn license:aggregate-add-third-party
Next, you can use the TextFinder plugin to search the "unsafe" dependencies in the THIRD_PARTY.txt file (ex: org.example:badartifact:1.0.1) and change the status of the build if needed.
Another solution is to use a 3rd party tool to do that.
I'm doing some investigation with this one: http://www.whitesourcesoftware.com/
This tool can provide a list of 3rd party dependencies with vulnerability issues.
I have multiple java projects in a folder. Also there is a second folder with libraries, that might be used as build dependencies from the projects. The projects may also have dependencies to other Projects. What's the best approach to build all projects ?
In other words I want to build the projects without explicit telling their dependencies.I think the biggest problem is the dependecy between the projects.
There are multiple build systems that are available that you may use. Maven has a complete dependency system built into it. Almost all third party open source jars are directly accessible via the World Wide Maven repository system. Basically, you describe the jar you need (groupId, artifactId, and version) and Maven will automatically fetch it for you. Not only that, but Maven also will build your project without having to create a build file. Instead, you have to describe your project in a project object model (a pom.xml file) and Maven will download everything you need, including all compilers, etc.
Almost all new projects use Maven, but Maven has a few downsides:
Since you don't control a build process, it can sometimes feel like poking a prodding a black box to get the build to work the way you want.
Documentation can be scant -- especially if you're moving beyond basic Java compiles.
You usually have to arrange your project in a specific layout. For example, source files should go under src/main/java while JUnit tests are under src/test/java. You don't have to follow the recommended layout, but then you'd have to modify the pom.xml file this way and that to get your build to work. That defeats the whole purpose of the pom.xml in the first place.
If you already have another build system setup (like Ant), you lose everything. There's no easy way to move from Ant to Maven.
The other is called Ant with Ivy. Ivy uses Ant for building, but can access Maven's world wide repository system for third party dependencies. It's a great compromise if you already are heavily invested in Ant. I also find Ant with Ivy to be better documented than Maven (although that's not too difficult). There's an excellent chapter going over the basics of Ivy in Manning Publication's Ant in Action.
With either process, I would recommend that you build a company wide Maven repository using either Nexus or Artifactory. This way, any proprietary third party jars (like Oracle jars) can also be stored in your company wide Maven repository since they won't be in the standard World Wide Maven repository.
By the way, if this is a company wide effort, and you are moving multiple Ant projects into Ivy, I have an Ivy project I use in Github that makes things easier.
Oh, there's a third possibility called Gradle which I know nothing about. I also believe it can use the World Wide Maven repository. It's based on Groovy which is based on Java syntax, and that's about all I can say. Maybe others can fill you in on the details. The Gradle group contends it solves a lot of problems of both Ant/Ivy and Maven.
Whatever tool you use, if you have various projects interdependent, you need to be clear on the independent ones which will be built first before building the dependent projects. You need to have a clear dependency structure for your projects.
You can do this with Apache Ivy. You can lay out the locations for you common libraries, define published artifacts and inter-dependencies in an ivy.xml document in each project, and let a top-level Ant build with the Ivy tasks figure out what the build order should be based on those dependencies.
Our software is written in Java and comprise many (7) projects.
These projects are Netbeans ant projects.
I'm considering to converting them to maven2.
Where can I find some hints for doing such thing?
Don't read that book. It will only make you confused. Read this book instead: "Maven - The definitive guide" http://www.sonatype.com/books/maven-book/reference/ .
Also, the maven site has a lot of information, but the structure is terrible so you'll need to use google to navigate in it.
Here is my suggestion:
Do this by hand, not with "automagic" "help" from the IDE. Maven integration doesn't work that good yet, not in any IDE.
Make sure you program project is divided into modules under a common umbrella module, so that each module produces a single binary artifact (jar, war,...) possibly accompanied by the javadoc of the source code behind that artifact, a zip with the source code etc. The basic principle is that each module produces a single artifact, containing all the non-test-code under that module. You can do this while the project is still built by ant.
Each module should conform to the standard maven directory layout. The build destination is under [module]/target/[output-type, e.g. "classes"]. The source code is under [module]/src/main/[src-type e.g. "java"] and [module]/test/[src-type]. The artifact consists of all the code under src/main, and none of the code under src/test, as it built to the target directories. You can do this while the is still built by ant.
Start by transforming the sub-module that has no dependencies on other modules in the project.
Now you can create the parent maven module pom.xml with artifact type "pom", consisting of one of the modules below. Make a child module for the first submodule (the one with only external dependencies), using the umbrella module as "parent". Remember that you need to specify version for the parent. Remember to add the child module as a "module" in the parent too. Always use ${project.version} as version in the child modules when you create multi-module projects like this. All modules under a parent must be released simultaneously in a single operation, and if you use this setting maven will make sure the version fields stay the same across all modules and gets updated everywhere during the release. This may make it difficult to re-use the existing numbering scheme, but that doesn't matter. You are never going to run out of version numbers anyway.
Add the necessary dependencies, and make sure you can build the parent and the child module together using the command "mvn clean install" from the parent module.
Proceed with the rest of the modules the same way. Dependencies to other modules under the same parent project should also use ${project.version} as the "version" they are depending on, meaning "the same version as this". NOTE THAT in order to build, the module you are depending on must be built using "mvn install", so that it gets deployed to you local (computer) repository. Otherwise the depending module will not be able to find the classes. There are NO source-code dependencies between modules in maven, only dependencies to built and packed versions installed in local and remote repositories. This can be very confusing if you come from ant-projects. Build from the root module until you get comfortable with this. It takes two days.
Don't use maven integration in IDEs. It is a bad idea. Use "mvn idea:idea" or "mvn eclipse:eclipse" to set up your workspace as a non-maven ordinary IDE project. The inter-module dependencies mechanisms in maven and the IDE aren't identical and will never be. Also, if you have several mavenized projects with dependencies in between, you want to have several of these in your workspace with dependencies set up between. You can do this with mvn idea:idea / eclipse:eclipse if you create a separate maven project file called "workspace.xml" (or whatever) in the same directory as parent module, set up as a multi-module project containing modules "." and "../otherproject" (only one-way reference here, no parent ref back). If you run "mvn idea:idea / eclipse:eclipse -f workspace.xml" you get a workspace with all these modules linked together. No IDE integration lets you do that. This sound like a lot of extra work, but the workspace.xml-file is really small. It doesn't have to contain all that dependency stuff and all that, only the reference to the modules you want to bind together in your IDE.
I did a succeful migration of NetBeans Ant project to Maven project using the instruccions by Joseph Mocker here: http://forums.netbeans.org/ptopic55953.html
I cite the important part:
close the project
rename the build.xml, nbproject files/folders to something so NB won't recognize them.
close and restart NB (so any memory cache knowledge of the project is gone)
copy in an empty pom from some other project.
open the project back up in NB (NB should now identify it as a maven project)
rearrange the files to follow the maven way (™)
This won't be an easy task since Maven2 expects the files to be organized in a specific way. Anyway Better Builds with Maven is a free book that should get you started. It will help you understand Maven and it also has a chapter on migration.
I discovered that the migration is not necessary. The real requirements that I need was automatic download of dependencies (libraries).
This is also achieved by Ivy which nonetheless uses maven repositories.
I solved converting project from ant to ant+ivy with IvyBeans.
I have built a script to migrate Ant builds to Maven. You can find more information here:
https://github.com/ewhauser/ant2maven
It won't help you with fixing your directory structure and or any additional Ant tasks, but it removes a lot of the tedious steps to get started.
What are the possibilities to enforce restrictions on the package dependencies in a Java build system? For example, the myapp.server.bl.Customer class should not be allowed to refer to the myapp.client.ui.customlayout package.
I'm interested in either Ant-based or IDE-specific solutions.
I'd like to get an error message in the build indicating that a (custom) package dependency rule has been violated and the build aborted. I also would like to maintain the dependencies in a list, preferably in a text file, outside of the Ant scripts or IDE project files.
(I don't know Maven but I've read it here it has better support for module dependency management)
I believe Checkstyle has a check for that.
It's called Import Control
You can configure Eclipse projects to specify Access Rules. Access rules can specify "Forbidden", "Discouraged", and "Accessible" levels all with wildcard rules. You can then configure violations of either Discouraged or Forbidden to be flagged as either warnings or errors during builds.
Kind of an old article on the idea (details may be out of date):
http://www.eclipsezone.com/eclipse/forums/t53736.html
If you're using Eclipse (or OSGi) plugins, then the "public" parts of the plugin/module are explicitly defined and this is part of the model.
ivy seems like a good solution for your problem (if you are using ant). Ivy is the offical dependency management component of Ant and thus integrates nicely with ant. It is capable of resolving dependencies, handle conflicts, create exclusions and so on.
It uses a simple xml structure to describe the dependencies and is easier to use than Maven, because it only tries to address dependency resolution problems.
From the Ivy homepage:
Ivy is a tool for managing (recording, tracking, resolving and reporting) project dependencies. It is characterized by the following:
flexibility and configurability - Ivy is essentially process agnostic and is not tied to any methodology or structure. Instead it provides the necessary flexibility and configurability to be adapted to a broad range of dependency management and build processes.
tight integration with Apache Ant - while available as a standalone tool, Ivy works particularly well with Apache Ant providing a number of powerful Ant tasks ranging from dependency resolution to dependency reporting and publication.
For the IDE specific solutions, IntelliJ IDEA has a dependency analysis tool that allows one to define invalid dependencies as well.
http://www.jetbrains.com/idea/webhelp2/dependency-validation-dialog.html
The dependency violation will be shown both when compiling and live, while editing the dependent class (as error/warning stripes in the right side error bar).
Even more automation can be obtained with JetBrains' TeamCity build server, that can run inspection builds and report the above configured checks.
For another IDE independent solution, AspectJ can be used to declare invalid dependencies (and integrate the step in the build process, in order to obtain warning/error info for the issues).
Eclipse has support for this via Build Path properties / jar properties. I think it may only work across jar / project boundaries.
Maybe Classsycle can be used:
http://classycle.sourceforge.net/ddf.html
You can use multiple modules in IDEA or Maven or multiple projects in Eclipse and Gradle. The concept is the same in all cases.
A trivial interpretation would be a module for myapp.server.bl and another for myapp.client.ui.customlayout with no compile time dependencies between either of them. Now any attempt to compile code or code-complete against the opposite module/project will fail as desired.
To audit how extensive the problem already is, a useful starting point for IntelliJ IDEA is Analyzing Dependencies:
http://www.jetbrains.com/idea/webhelp/analyzing-dependencies.html
From that article you can see how to run and act on dependency analysis for your project.