Any best practices or tools for Ivy repositories? - java

we use Gradle for building Java projects and at the moment we have Ivy repositories to store third-party artifacts and also to publish our own artifacts into (repo is build using Gant scripts and the Ivy ANT tasks). but repo management is basic.
Gradle is able to work with a maven repo as well, so switching to a Maven artifact manager like Archiva or Nexus is an option, but perhaps unnecessary. do you know any tools or best practices than can help us in building and maintaining Ivy repos?
just to be clear: we have already read the tutorials and more and understand how to do it, but it's still basic to maintain.

In the past, I've only used an ivy repository for small private repositories publishing artifacts using simple low level protocols like an FTP site. (All the site needs is a versioned directory layout and an ivy.xml file describing the arifacts)
The maven based repository infrastructure is now so pervasive, with some many projects using it, it's almost pointless to promote an alternative repository management standard.
Sonatype (company behind Maven) make their repository product, Nexus, available to all, because it's in everyone's interest to keep the band-width requirements to Maven central under control.
Thankfully, ivy plays nice with Maven meaning you can take advantage of the best of both worlds.

I've created a blog entry about my Ivy repository layout and choices. You might have different requirements, but I think it is always good to check other's solutions to get some ideas...

There's a community project called Ivy Roundup that aims to build a consistent, up-to-date ivy repo of common third party libraries. It may be a good idea to match the naming conventions used there, or even better, just get modules from there using the <ivy:install> task.

In my opinion there isn't much in Ivy's repository to work with because it just works. What you can't do with Ivy's Ant tasks you can do directly from the file system, simple as that.
Admittedly something like changing the artefact name can be difficult but then again that's something you shouldn't do anyway.

What I am typically doing in practice when I need a third-party library is to search for it in the MVN repository and then click on the "Ivy" tab to get the Ivy dependency for my ivy.xml.

Related

How to publish a regular, non-Android Java library so that it becomes available for inclusion in Java projects via Maven & Gradle build scripts?

I recently authored a 3rd party library for Java. Currently, I'm distributing the library via Github and a website I own where I make it possible to download the JAR file. I want to make it so developers can use Gradle and Maven (and possibly other build automation tools) to easily include my 3rd party library in their projects.
I'm new to Maven, Gradle, Bintray, JCenter, MavenCentral, Ivy, etc. However, I do have experience with Ant, Java, and Linux.
I've been trying to learn how to publish my 3rd party library using the tools/platforms above, but I've become confused.
Based on what I've read, Gradle, Maven, and Ant are competing technologies. So, why then, in the context of actual use-case scenarios, does Gradle seem to be so intertwined with Maven?
Bintray advertises itself as a software publishing and distribution platform. So, why was there a need to create JCenter as something distinct from the rest of Bintray? What can JCenter do that Bintray can't?
I've created a test package and version on Bintray and I uploaded a JAR file to the version. Is it possible to write a Gradle script that will be able to include that JAR file as a dependency in a Java project? Or does the package need to be "linked" with JCenter?
Basically, can anyone tell me what's going on with all of the above technologies? Don't feel obligated to provide answers to all of the sentences in this post that end in a question mark. Those questions are primarily intended to demonstrate my current level of understanding. A simple overview of how the technologies work together would be great, and if possible, an answer to the question in the title of this post.
I will try answer to all the sentences ending with question marks :)
Based on what I've read, Gradle, Maven, and Ant are competing technologies. So, why then, in the context of actual use-case scenarios, does Gradle seem to be so intertwined with Maven?
Besides being a build tool and a dependency manager (in which Maven compete with Gradle), Maven introduced a standard artifact descriptor (the POM file), a standard project layout (src/main/java etc) and a standard artifact layout (groupId/version/artifactId-version.ext) that due to the popularity of Maven became a standard de-facto in the industry. They aren't particularly bad, so they stuck for now and Gradle works with them as well.
Also, Gradle appeared when Maven already been exceptionally popular. To overtake such a popular technology Gradle had to provide a clean and easy migration path, which means supporting Maven-structured projects, Maven-structured local caches, etc.
Bintray advertises itself as a software publishing and distribution
platform. So, why was there a need to create JCenter as something
distinct from the rest of Bintray? What can JCenter do that Bintray
can't?
JCenter is not "something distinct" from Bintray. JCenter is a repository inside Bintray. This repository is special (we call this type of repository a "Central Repository"), since it is maintained by Bintray team themselves, and include (or link) the biggest collection of Java libraries in the world. For a lot of Java (or Android for that matter) developers, who don't intend to publish their work through Bintray and don't care about other people's personal repositories the only thing of interest on Bintray is JCenter. As such, people refer to JCenter as a thing of its own.
I've created a test package and version on Bintray and I uploaded a
JAR file to the version. Is it possible to write a Gradle script that
will be able to include that JAR file as a dependency in a Java
project?
Absolutely. The "Set Me Up" button opens a window in which you'll find the instructions on how to add your personal repository in Gradle. Once done, the artifacts from your repository will be available for resolution in your Gradle project.
Or does the package need to be "linked" with JCenter?
The benefit of including your package in JCenter is that other developers who are familiar with JCenter as a source for 3rd party dependencies, but not familiar with your repository on Bintray will be able to find and use your library.
I hope it helped.
I am with JFrog, the company behind Bintray and [artifactory], see my profile for details and links.
It is really hard (next to impossible) to give an answer, because you are asking multiple questions (I count 5 question marks), plus you want an explanation of how all of the mentioned technologies play together (or not). This really exceeds the scope and usefulness of stackexchange.
However, if it's maven you could live with, then what you probably want is:
1) deploy your artifact to maven central repo ==> see https://maven.apache.org/guides/mini/guide-central-repository-upload.html
2) deploy to some other well known public maven repo (like sonatype)
3) host your own public maven repo (that can be used by others) and upload there, still keep "advertising" through web and github pages

Is there anything like pkg-config for Java?

I have a project where our CLASSPATH is getting out of hand. We have jars that depend on other jars, which in turn depend on even more jars, as well as a large collection of third-party jars. So each time we use one of these jars we need to construct a rather large CLASSPATH, and if downstream dependencies change it is no longer in sync.
One thought is to maybe write a tool a little like pkg-config to help with all of this. I did some searching and didn't find anything. Are there any tools like this that I've missed or do people have alternate solutions?
No, I don't want to use Maven. We already have a different build system in place.
If you aren't already using a build tool, I recommend starting with Maven. The main thing that turns people off Maven is that it has a standard project structure. Personally, I think that's a good thing.
If you want to be a little more free-form with your projects, there are the Maven Ant Tasks for Ant. I much prefer these to Ivy, but part of that is because I'm used to the Maven artifact naming conventions.
In either case (or if you use Ivy), look into a repository manager. Nexus is written by the people who maintain Maven, and I find it easier to use than Artifactory, which is another widely used system. One benefit for Artifactory is that they provide hosting (for a fee).
You may want to use maven with an IDE plugin like m2eclipse. All you have to do is type the class in your code and it will help you search for and bring in the right packages with their dependancies. It will also automatically configure the classpath for you. Here is a quick tutorial http://www.sonatype.com/people/2010/03/adding-dependencies-using-m2eclipse/

Build multiple java projects with dynamic dependencies

I have multiple java projects in a folder. Also there is a second folder with libraries, that might be used as build dependencies from the projects. The projects may also have dependencies to other Projects. What's the best approach to build all projects ?
In other words I want to build the projects without explicit telling their dependencies.I think the biggest problem is the dependecy between the projects.
There are multiple build systems that are available that you may use. Maven has a complete dependency system built into it. Almost all third party open source jars are directly accessible via the World Wide Maven repository system. Basically, you describe the jar you need (groupId, artifactId, and version) and Maven will automatically fetch it for you. Not only that, but Maven also will build your project without having to create a build file. Instead, you have to describe your project in a project object model (a pom.xml file) and Maven will download everything you need, including all compilers, etc.
Almost all new projects use Maven, but Maven has a few downsides:
Since you don't control a build process, it can sometimes feel like poking a prodding a black box to get the build to work the way you want.
Documentation can be scant -- especially if you're moving beyond basic Java compiles.
You usually have to arrange your project in a specific layout. For example, source files should go under src/main/java while JUnit tests are under src/test/java. You don't have to follow the recommended layout, but then you'd have to modify the pom.xml file this way and that to get your build to work. That defeats the whole purpose of the pom.xml in the first place.
If you already have another build system setup (like Ant), you lose everything. There's no easy way to move from Ant to Maven.
The other is called Ant with Ivy. Ivy uses Ant for building, but can access Maven's world wide repository system for third party dependencies. It's a great compromise if you already are heavily invested in Ant. I also find Ant with Ivy to be better documented than Maven (although that's not too difficult). There's an excellent chapter going over the basics of Ivy in Manning Publication's Ant in Action.
With either process, I would recommend that you build a company wide Maven repository using either Nexus or Artifactory. This way, any proprietary third party jars (like Oracle jars) can also be stored in your company wide Maven repository since they won't be in the standard World Wide Maven repository.
By the way, if this is a company wide effort, and you are moving multiple Ant projects into Ivy, I have an Ivy project I use in Github that makes things easier.
Oh, there's a third possibility called Gradle which I know nothing about. I also believe it can use the World Wide Maven repository. It's based on Groovy which is based on Java syntax, and that's about all I can say. Maybe others can fill you in on the details. The Gradle group contends it solves a lot of problems of both Ant/Ivy and Maven.
Whatever tool you use, if you have various projects interdependent, you need to be clear on the independent ones which will be built first before building the dependent projects. You need to have a clear dependency structure for your projects.
You can do this with Apache Ivy. You can lay out the locations for you common libraries, define published artifacts and inter-dependencies in an ivy.xml document in each project, and let a top-level Ant build with the Ivy tasks figure out what the build order should be based on those dependencies.

Creating/Accessing Maven repository in SVN through NetBeans 6.7 *Revised*

EDIT: Ok... so I've gathered that SVN shouldn't really be used for this... which makes sense, I suppose (why version individual files when the version should be a separate jar?).
So we should use an internal server to host a repository management tool like Nexus (etc), and access that over http to pull down and put out dependencies. We are keeping our projects in SVN now. What is the standard for deployments? Dependencies go into Maven. Projects go into SVN. Should we ignore the dist and build folders? Where would our WAR files get deployed from?
OLD QUESTION (for posterity)
I'm brand new to Maven and don't know jack about it. I'm trying to evaluate it to see how it will do with our Java development.
I would like to have a Maven repository in our SVN server so that dependencies can be pulled down from there using NetBeans 6.7. I have not been able to find how to do this throughout many google and stackoverflow searches.
What are the best practices here? I'm thinking that we'd want to download dependencies using svn+ssh, but most things online seem to point to using http.
Fill my brain with great things!
I'd strongly recommend against doing this. Maven artifacts don't belong on an SCM server. You should consider using a repository manager like Nexus to store your artifacts. See here for a comparison of the main repository managers.
Having said that. If you are determined to use Subversion to host your artifacts. See this question on using the wagon scm to deploy to a Subversion repository.
If you want to find out more about Maven, check out Maven: the definitive guide.
There is a Maven plugin for Netbeans that will manage dependencies. This article lists some best practices for Maven and Netbeans.
Update based on your updated question. What to do with your own jars:
Maven has a deploy phase that will publish your artifacts to the remote repository. You need to configure the distributionManagement section of the pom, and provide appropriate credentials in your settings.xml to allow the deployment to happen. Typically you would set up a discrete logical repository on the server for your own artifacts to keep them isolated from third party artifacts. The Nexus book gives some good guidance on configuring repositories on Nexus. In particular see the Adopting a Repository Manager section.
If you have configured your project correctly, run mvn deploy and all phases up to and including the deploy phase will run, and your artifact will be published to the repository, available for use by the rest of your team/company.
If you need to restrict access to repositories, you can configure access controls to your repository so only authenticated users can access those artifacts (for Nexus see the Managing Security section of the book for guidance).
It's worth noting you can do largely the same things (more or less) with Artifactory or Archiva as Nexus, I've included Nexus references because I prefer it, and the documentation is really good.
Don't store them in SVN.
I would do two things to make sure you're not getting too many headaches:
Mirror a repository closer to your box someplace that you and your workmates can share. This will eliminate extra downloading and allow you to fix any problems that may come up (and they will) with the mirrored pom/jar files so your mates don't have to share in the headache. There are several repo managers out there that help with this.
Do your best to work with your machine repository and push changes/modifications to any pom files that you may make to the local shared repo.

what is ivy? and how it is related to ant?

I have seen many ivy files in my application's build projects. What is Ivy, and its relation with ant?
Ivy is an Ant extension for managing external libraries during the development process, giving you a way of adding them to your classpath, bundling them into your application build, etc.
There's some overlap with what Maven does, but Ivy is much more lightweight (and doesn't do as much).
Ivy is a dependency manager -- it manages and controls the JAR files that your project depends on. If you don't have the JARs, it will pull them down for you by default (from the Maven 2 repository), which can make project setup a lot easier.
Ivy is originally a Jayasoft.fr product which at that time was under BSD license, however the core of Ivy has always been the same: It's a dependency management tool.
Ivy's relation to Ant is that it is an Ant extension in the sense that it is through Ant, however it doesn't really depend on it (beyond the obvious parts). Ivy was merged into an Apache project under Ant as subproject through incubation process on October 11, 2007.
Very common misconception people have is that they compare Ivy with Maven as a whole. However that doesn't really work since Ivy only handles dependencies in an agile manner and is very good at it with simplistic configuration and wide automated support with various build systems etc. through Ant while Maven really is a build system in itself. Personal opinions may and do differ in which one does what best and that is worth several Stackoverflow/Google searches on its own.
I believe these are significant and useful points to add:
Ivy can be used standalone from command line without Ant. If your CI environment does not include Ant it will not preclude you from using Ivy. Yes, integration with Ant is nice and, by far, better documented. One could apply Ant integration documentation to standalone usage though.
Ivy is not just for Java. My team successfully uses it to define dependencies for a fairly large .NET project (10 interdependent packages with several versions of each in production with multiple third party upstream dependencies). Works quite well. I wish documentation were a bit clearer and fuller.
There are additional perks that Ivy offers for Build Time: upstream and downstream builds triggering.

Categories

Resources