I have multiple projects, some are single module, and others are multi-module. Some of these modules are libraries intended to be used in multiple projects.
How do I share these modules between projects, and what would I store in version control?
If I store each module in it's own repository, then the .idea folders will not be available, and as such information about the module such as its dependencies, copyright notices, artefact config etc will be missing. Therefore, I would like to store each project in a repository, but then how would I reference one module from another project as a dependency?
Note: I am using git for version control.
How do I share these modules between projects.
I would either have a project for common modules, or add them to each project.
and what would I store in version control?
I wouldn't store IntelliJ specific files if you are working with another developer. If not, you can just store everything. What ever you do, you can swap later.
If I store each module in it's own repository,
Or in its own sub-directory. Git does encourage creating more repositories, either way works.
then the .idea folders will not be available,
I would check them in anyway, but if you check them in, they are as available as any other file.
and as such information about the module such as its dependencies, copyright notices, artefact config etc will be missing.
I would use maven for dependencies and build.
Therefore, I would like to store each project in a repository, but then how would I reference one module from another project as a dependency?
You can import a module from one project to another in IntelliJ, but I would use maven which supports this.
Related
My team uses a GitHub.com organization to keep all of our source code in private repos. (Prior, our old workflow was emailing Dropbox links). Most of the time each repo is one separate project with no dependancy of any other (the only dependancies are on third-party open source libraries). Or if there is some dependancy, then the .java files have just been copy pasted into the other project.
I've recently been splitting up some of my code into reusable modules, but I don't know any way to do the dependancy management when I use the libraries I'm creating in another project.
I know with Gradle you can add a git repo like this:
gitRepository('https://github.com/user/project.git') {
producesModule('user:project')
}
but I don't know if there's a way to make it work with private repos, and I don't know if there's a way to specify versions.
My currently solution is to just build the library JAR, and keep track of the binary version with GitHub release tagging, and when I need to use the library in another project, I download the desired version of the JAR (typically the most recent) and add it to a local /lib/ folder in the other project and import the JAR into the module path as a local JAR. Of course I need to go through the whole process again manually if I want to make a change to the library.
I also heard you can set up private Gradle or Maven servers and some companies do that, but I guess that would mean migrating away from GitHub.com?
Is there any way to make this work (either Gradle or Maven, it doesn't matter) to manage dependancies between GitHub private repos?
Can someone tell me, what is the most sensible way (or ways) to solve this?
Thanks.
What you need is a very typical maven/gradle based setup where
each of your projects will be producing an artifact with a coordinate
of the form group:name:version
your projects do not have to be explicitly aware of each other. They depend on the artifacts produced by other projects. This is called binary dependency
for a project to locate a binary dependency, you will need a central registry where you can publish all your artifacts to. GitHub has a product called GitHub Package for precisely this purpose.
If you don't want to use GitHub Package yet, or your setup (number of projects, size of each projects, size your team) is small enough, you can locally checkout all the projects and include them into a gradle composite build so that binary dependencies will be substituted with local project dependencies. The good thing about the composite build is that when you decide to invest in a package registry, your build.gradle requires no change at all.
BTW, where you run your private package registry does not really matter. You can use the GitHub Package, or some other hosted services, or even run e.g. jfrog artifactory on your own server. It is completely unrelated to where you host your source code, so you dont need to migrate away from GitHub in any case.
I'm setting up a new Java project on GitHub, and I'll have some Apache Commons libraries as dependencies.
What are the best practices to establish that my project needs those jar files? Should I upload the dependencies to my GitHub repository (ugly)? Or use a Maven-like tool for that?
Or is there a way to link a file in another git repository? Apache provides git repositories for they libraries. They are read-only, but I'm o.k. with that, since I just want to use the jars. The bad thing is that they contain all the sources, and I just care about the compiled jar. It seems we can't git submodule just a file.
The two approaches are:
declarative and component-based, where you declare (describe) what components (jars, exe, other binaries) you need for your project to (compile, execute, deploy, etc.), and you use a third-party tool (like Maven/Nexus) to bring those components on demand.
So they aren't versioned in your repo. They are declared/described (like in a pom.xml, if you were to use Nexus)
See also "Difference between Git and Nexus?".
inclusive and system-based, where you complete your project with other project sources/binaries, in order to get in your repo everything you need right after the clone step (no need to call a third-party tool or to do anything: every other part of your system in there).
With Git, especially if those "other parts" are in a git repo (like the apache libs one), then you would declare those sub-repos as submodules of your main repo.
That way, all you keep in your main repo is a special entry (gitlink, mode 160000) referencing a specific SHA1 of another repo (but you can make that submodule follow a branch too, a bit like svn external).
And with sparse checkouts in submodules (as in this example), you even can update those modules for them to checkout only the part of the repo you want (like just the jars, not the sources).
Note that you aren't supposed to store any delivery that you would produce (like jars of your own) in your GitHub repo.
You can associate those deliveries to GitHub releases though.
I hope I can keep this question specific enough, my team at work is currently debating the best way to manage our dependencies for a huge project (150+ dependencies ~300mb).
We have two main problems
Keeping all the developers dependencies the same so we are compiling against the same files
Ensure the project (once compiled) is comliped against the same dependencies
The two ideas that have been suggested are using a BirJar (all dependencies in one file) and just adding a version number to it and using a shared folder and pointing everyone's machines at the same place.
Or making including all the dependencies in the jar when we compile it (a jar, of jars, of jars) and just have a project that "has no dependencies"
Someone also mentioned setting up an internal version of Ivy and pointing all the code to pull dependencies from there.
What are the best practices regarding massive dependency management?
Why don't you use Maven and its dependency management ?
You can specify each dependency, its particular version and its scope (compile-time, for testing, for deployment etc.). You can provide a master pom.xml (the config file) that specifies these, and developers can override if they need (say, to evaluate new versions).
e.g. I specify a pom.xml that details the particular jars I require and their versions (or range). Dependent jars are determined/downloaded automatically. I can nominate which of these jars are used for compilation vs. deployment etc. If I use a centralised repository such as Nexus I can then build my artefact (e.g. a library) and deploy that into Nexus, and it'll become available for other developers to download in exactly the same manner as 3rd party libs etc.
Incase you dont like/want to follow the Maven project structure...
If you already use Ant, then your best bet is to use Ivy for dependency management.
http://ant.apache.org/ivy/
It provides a rich set of ant tasks for dependency manipulation.
from : Ant dependency management
What is an artifact and why does Maven need it?
An artifact is a file, usually a JAR, that gets deployed to a Maven repository.
A Maven build produces one or more artifacts, such as a compiled JAR and a "sources" JAR.
Each artifact has a group ID (usually a reversed domain name, like com.example.foo), an artifact ID (just a name), and a version string. The three together uniquely identify the artifact.
A project's dependencies are specified as artifacts.
In general software terms, an "artifact" is something produced by the software development process, whether it be software related documentation or an executable file.
In Maven terminology, the artifact is the resulting output of the maven build, generally a jar or war or other executable file. Artifacts in maven are identified by a coordinate system of groupId, artifactId, and version. Maven uses the groupId, artifactId, and version to identify dependencies (usually other jar files) needed to build and run your code.
I know this is an ancient thread but I wanted to add a few nuances.
There are Maven artifacts, repository manager artifacts and then there are Maven Artifacts.
A Maven artifact is just as other commenters/responders say: it is a thing that is spat out by building a Maven project. That could be a .jar file, or a .war file, or a .zip file, or a .dll, or what have you.
A repository manager artifact is a thing that is, well, managed by a repository manager. A repository manager is basically a highly performant naming service for software executables and libraries. A repository manager doesn't care where its artifacts come from (maybe they came from a Maven build, or a local file, or an Ant build, or a by-hand compilation...).
A Maven Artifact is a Java class that represents the kind of "name" that gets dereferenced by a repository manager into a repository manager artifact. When used in this sense, an Artifact is just a glorified name made up of such parts as groupId, artifactId, version, scope, classifier and so on.
To put it all together:
Your Maven project probably depends on several Artifacts by way of its <dependency> elements.
Maven interacts with a repository manager to resolve those Artifacts into files by instructing the repository manager to send it some repository manager artifacts that correspond to the internal Artifacts.
Finally, after resolution, Maven builds your project and produces a Maven artifact. You may choose to "turn this into" a repository manager artifact by, in turn, using whatever tool you like, sending it to the repository manager with enough coordinating information that other people can find it when they ask the repository manager for it.
Hope that helps.
Maven organizes its build in projects.
An artifact in maven is a resource generated by a maven project. Each maven project can have exactly one artifact like a jar, war, ear, etc.
The project's configuration file "pom.xml" describes how the artifact is build, how unit tests are run, etc.
Commonly a software project build with maven consists of many maven-projects that build artifacts (e.g. jars) that constitute the product.
E.g.
Root-Project // produces no artifact, simply triggers the build of the other projects
App-Project // The application, that uses the libraries
Lib1-Project // A project that creates a library (jar)
Lib2-Project // Another library
Doc-Project // A project that generates the user documentation from some resources
Maven artifacts are not limited to java resources. You can generate whatever resource you need. E.g. documentation, project-site, zip-archives, native-libraries, etc.
Each maven project has a unique identifier consiting of [groupId, artifactId, version]. When a maven project requires resources of another project a dependency is configured in it's pom.xml using the above-mentioned identifier. Maven then automatically resolves the dependencies when a build is triggered. The artifacts of the required projects are then loaded either from the local repository, which is a simple directory in your user's home, or from other (remote) repositories specified in you pom.xml.
Q. What is Artifact in maven?
ANS: ARTIFACT is a JAR,(WAR or EAR), but it could be also something else. Each artifact has,
a group ID (like com.your.package),
an artifact ID (just a name), and
a version string. The three together uniquely identify the artifact.
Q.Why does Maven need them?
Ans: Maven is used to make them available for our applications.
An artifact is a JAR or something that you store in a repository. Maven gets them out and builds your code.
To maven, the build process is arranged as a set of artifacts. Artifacts include:
The plugins that make up Maven itself.
Dependencies that your code depends on.
Anything that your build produces that can, in turn be consumed by something else.
Artifacts live in repositories.
usually we talking Maven Terminology about Group Id , Artifact Id and Snapshot Version
Group Id:identity of the group of the project
Artifact Id:identity of the project
Snapshot version:the version used by the project.
Artifact is nothing but some resulting file like Jar, War, Ear....
simply says Artifacts are nothing but packages.
Usually, when you create a Java project you want to use functionalities made in another Java projects.
For example, if your project wants to send one email you dont need to create all the necessary code for doing that. You can bring a java library that does the most part of the work.
Maven is a building tool that will help you in several tasks. One of those tasks is to bring these external dependencies or artifacts to your project in an automatic way ( only with some configuration in a XML file ).
Of course Maven has more details but, for your question this is enough.
And, of course too, Maven can build your project as an artifact (usually a jar file ) that can be used or imported in other projects.
This website has several articles talking about Maven :
https://connected2know.com/programming/what-is-maven/
https://connected2know.com/programming/maven-configuration/
An artifact is an element that a project can either use or produce. In Maven terminology, an artifact is an output generated after a Maven project build. It can be, for example, a jar, war, or any other executable file.
Also, Maven artifacts include five key elements, groupId, artifactId, version, packaging, and classifier. Those are the elements we use to identify the artifact and are known as Maven coordinates.
read it from here
Our software is written in Java and comprise many (7) projects.
These projects are Netbeans ant projects.
I'm considering to converting them to maven2.
Where can I find some hints for doing such thing?
Don't read that book. It will only make you confused. Read this book instead: "Maven - The definitive guide" http://www.sonatype.com/books/maven-book/reference/ .
Also, the maven site has a lot of information, but the structure is terrible so you'll need to use google to navigate in it.
Here is my suggestion:
Do this by hand, not with "automagic" "help" from the IDE. Maven integration doesn't work that good yet, not in any IDE.
Make sure you program project is divided into modules under a common umbrella module, so that each module produces a single binary artifact (jar, war,...) possibly accompanied by the javadoc of the source code behind that artifact, a zip with the source code etc. The basic principle is that each module produces a single artifact, containing all the non-test-code under that module. You can do this while the project is still built by ant.
Each module should conform to the standard maven directory layout. The build destination is under [module]/target/[output-type, e.g. "classes"]. The source code is under [module]/src/main/[src-type e.g. "java"] and [module]/test/[src-type]. The artifact consists of all the code under src/main, and none of the code under src/test, as it built to the target directories. You can do this while the is still built by ant.
Start by transforming the sub-module that has no dependencies on other modules in the project.
Now you can create the parent maven module pom.xml with artifact type "pom", consisting of one of the modules below. Make a child module for the first submodule (the one with only external dependencies), using the umbrella module as "parent". Remember that you need to specify version for the parent. Remember to add the child module as a "module" in the parent too. Always use ${project.version} as version in the child modules when you create multi-module projects like this. All modules under a parent must be released simultaneously in a single operation, and if you use this setting maven will make sure the version fields stay the same across all modules and gets updated everywhere during the release. This may make it difficult to re-use the existing numbering scheme, but that doesn't matter. You are never going to run out of version numbers anyway.
Add the necessary dependencies, and make sure you can build the parent and the child module together using the command "mvn clean install" from the parent module.
Proceed with the rest of the modules the same way. Dependencies to other modules under the same parent project should also use ${project.version} as the "version" they are depending on, meaning "the same version as this". NOTE THAT in order to build, the module you are depending on must be built using "mvn install", so that it gets deployed to you local (computer) repository. Otherwise the depending module will not be able to find the classes. There are NO source-code dependencies between modules in maven, only dependencies to built and packed versions installed in local and remote repositories. This can be very confusing if you come from ant-projects. Build from the root module until you get comfortable with this. It takes two days.
Don't use maven integration in IDEs. It is a bad idea. Use "mvn idea:idea" or "mvn eclipse:eclipse" to set up your workspace as a non-maven ordinary IDE project. The inter-module dependencies mechanisms in maven and the IDE aren't identical and will never be. Also, if you have several mavenized projects with dependencies in between, you want to have several of these in your workspace with dependencies set up between. You can do this with mvn idea:idea / eclipse:eclipse if you create a separate maven project file called "workspace.xml" (or whatever) in the same directory as parent module, set up as a multi-module project containing modules "." and "../otherproject" (only one-way reference here, no parent ref back). If you run "mvn idea:idea / eclipse:eclipse -f workspace.xml" you get a workspace with all these modules linked together. No IDE integration lets you do that. This sound like a lot of extra work, but the workspace.xml-file is really small. It doesn't have to contain all that dependency stuff and all that, only the reference to the modules you want to bind together in your IDE.
I did a succeful migration of NetBeans Ant project to Maven project using the instruccions by Joseph Mocker here: http://forums.netbeans.org/ptopic55953.html
I cite the important part:
close the project
rename the build.xml, nbproject files/folders to something so NB won't recognize them.
close and restart NB (so any memory cache knowledge of the project is gone)
copy in an empty pom from some other project.
open the project back up in NB (NB should now identify it as a maven project)
rearrange the files to follow the maven way (™)
This won't be an easy task since Maven2 expects the files to be organized in a specific way. Anyway Better Builds with Maven is a free book that should get you started. It will help you understand Maven and it also has a chapter on migration.
I discovered that the migration is not necessary. The real requirements that I need was automatic download of dependencies (libraries).
This is also achieved by Ivy which nonetheless uses maven repositories.
I solved converting project from ant to ant+ivy with IvyBeans.
I have built a script to migrate Ant builds to Maven. You can find more information here:
https://github.com/ewhauser/ant2maven
It won't help you with fixing your directory structure and or any additional Ant tasks, but it removes a lot of the tedious steps to get started.