I'm looking at options to separate our "mono-repo" so that each deployable service will have its own repo. One disadvantage of this is making changes to any common modules/libraries becomes tedious as it would require publishing the library first and then updating the version in gradle to use the changes.
npm appears to get around this through npm link which would allow you to modify the library and test it in your application before pushing your changes. I was wondering if gradle or any other build tool provides something similar for java applications.
It sounds like composite builds are what you want. The feature allows you to replace a project dependency with another build.
For example, imagine you have an application project called "Acme", which has a dependency on library "MyLib". Acme normally downloads MyLib from Maven Central or your company's artifact repository. But you want to make some changes to MyLib and test them with Acme, so you grab the source for MyLib, make the changes and then use the composite builds feature to run Acme's build with your modified MyLib.
Note that you don't have to manually build MyLib or install the generated artifact anywhere. Gradle will automatically execute MyLib's build and include its artifact in Acme's build.
Hope that makes sense.
Related
My team uses a GitHub.com organization to keep all of our source code in private repos. (Prior, our old workflow was emailing Dropbox links). Most of the time each repo is one separate project with no dependancy of any other (the only dependancies are on third-party open source libraries). Or if there is some dependancy, then the .java files have just been copy pasted into the other project.
I've recently been splitting up some of my code into reusable modules, but I don't know any way to do the dependancy management when I use the libraries I'm creating in another project.
I know with Gradle you can add a git repo like this:
gitRepository('https://github.com/user/project.git') {
producesModule('user:project')
}
but I don't know if there's a way to make it work with private repos, and I don't know if there's a way to specify versions.
My currently solution is to just build the library JAR, and keep track of the binary version with GitHub release tagging, and when I need to use the library in another project, I download the desired version of the JAR (typically the most recent) and add it to a local /lib/ folder in the other project and import the JAR into the module path as a local JAR. Of course I need to go through the whole process again manually if I want to make a change to the library.
I also heard you can set up private Gradle or Maven servers and some companies do that, but I guess that would mean migrating away from GitHub.com?
Is there any way to make this work (either Gradle or Maven, it doesn't matter) to manage dependancies between GitHub private repos?
Can someone tell me, what is the most sensible way (or ways) to solve this?
Thanks.
What you need is a very typical maven/gradle based setup where
each of your projects will be producing an artifact with a coordinate
of the form group:name:version
your projects do not have to be explicitly aware of each other. They depend on the artifacts produced by other projects. This is called binary dependency
for a project to locate a binary dependency, you will need a central registry where you can publish all your artifacts to. GitHub has a product called GitHub Package for precisely this purpose.
If you don't want to use GitHub Package yet, or your setup (number of projects, size of each projects, size your team) is small enough, you can locally checkout all the projects and include them into a gradle composite build so that binary dependencies will be substituted with local project dependencies. The good thing about the composite build is that when you decide to invest in a package registry, your build.gradle requires no change at all.
BTW, where you run your private package registry does not really matter. You can use the GitHub Package, or some other hosted services, or even run e.g. jfrog artifactory on your own server. It is completely unrelated to where you host your source code, so you dont need to migrate away from GitHub in any case.
I have a library and a program, both under my control and built using Gradle. What's the best way to develop these two at the same time?
I have set up a private maven repository to distribute the library and that's working, but I don't want to release to that repository every little experiment I make during development. It's slow and disruptive to users of the library.
I tried installing the jar to the local maven repository as explained here: Gradle alternate to mvn install but the project that's using the library is not picking up that newly installed version.
I think, you can try to use multi-project builds for that if it's possible. But you will likely need to restructure both your current projects to become modules of the same new project.
What's the best way to develop these two at the same time?
It depends by how the team is organized and what are your policies.
Of course if the team can use a git repo and access to the source code you can just use git without pushing a new version on the maven server for each commit or push.
Otherwise if other users can only use the final library, you have to push the version on the maven server.
I have set up a private repository to distribute the library and that's working, but I don't want to release to that repository every little experiment I make during development. It's slow and disruptive to users of the library.
Every maven repo has 2 different repositories:
release
snapshot
Usually release repo is used only for stable releases and the snapshot repo is used to publish little change, beta release and so on.
In any case it is not required that every changes in the code is pushed in the maven repo (it is a your choice)
It's slow
The time to upload artifacts usually is not so big, in any case you can evaluate to push the release in the maven repo with a CI server.
The best method seems to be to make one project include the other one when present by adding:
if (file("../libraryproject").exists()) {
includeBuild "../libraryproject"
}
to the settings.gradle file of the project that uses the library. That can be committed to the source code repo because when that directory doesn't exist, the dependency will be included in the traditional way.
This is called Composite Build in the Gradle world, IntelliJ seems to handle properly and theres'a recorded webcast showing the whole setup: https://www.youtube.com/watch?v=grPJanXfRPg
Currently, my built structure for a plugin in is a bit messy: I'm using the normal IDEA project file to build the plugin locally. When I push it to the repo and travis-ci is building it, it uses the maven pom.xml because for travis to work, it always has to download the complete IDEA sources.
Although this works, this has several drawbacks:
I need to keep two built mechanisms up to date. This is
When a new IDEA version is out (every few weeks), I need to change the SDK in maven and in my IDEA settings
When I add a new library, change resources, etc. I need to do this for two the two settings as well
I ran into problems when I kept the IDEA Maven plugin turned on because it saw the pom.xml and interfered with my local built. Turning it off means, I cannot download libraries with Maven which has the feature of tracking dependencies.
I saw that Gradle has an 'idea' plugin and after googling, I got the impression that Gradle is the preferred choice these days. I have seen Best way to add Gradle support to IntelliJ IDEA and I'm sure I can use the answers there to turn my pom.xml into a valid build.gradle.
However, maybe someone else has already done this or can provide a better approach. What I'm looking for is a unified way to build my plugin locally and on Travis-CI.
Some Details
For compiling an IDEA plugin, you need its SDK which you can access through an installation of IDEA or a download of the complete package. Locally, I'm using my installation for the SDK. With Travis, my maven built has the rule to download the tar.gz and extract it.
It turns out that in particular for building an IntelliJ plugin, Gradle seems to have many advantages. This is mainly due to the great IntelliJ plugin for Gradle which makes compiling plugins so much easier. With Gradle, I could turn my >220 lines of Maven build into a few lines of easily readable Gradle code. The main advantages are that
It takes care of downloading and using the correct IDEA SDK while you only have to specify the IDEA version.
It can publish your plugin to your Jetbrains repository and make it instantly available to all users
It fixes items in your plugin.xml, e.g. you can use one central version number in gradle.build and it will keep plugin.xml up-to-date or it can include change-notes
It seamlessly integrates with Travis-CI
How to use Gradle with an existing IDEA plugin
Do it manually. It's much easier.
Create an empty build.gradle file
Look at an example and read through the README (there are many build.gradle of projects at the end) to see what each intellij property does.
Adapt it to your plugin by
Setting the intellij.version you want to build against
Setting your intellij.pluginName
Define where your sources and resources are
Define your plugin version
Define a Gradle wrapper that enables people (and Travis) to build your plugin without having Gradle
Create the gradle wrapper scripts with gradle wrapper
Test and fix your build process locally with ./gradlew assemble
If everything works well, you can push build.gradle, gradlew, gradlew.bat and the gradle-folder to your repo.
Building with Travis-CI
For Travis you want to use the gradlew script for building. To do so, you need to make it executable in the travis run. An example can be found here.
Recently, a project I need as a dependency for some of my programs has switched to providing patch files instead of actual jars (legal reasons).
They included a small tool to automatically patch your existing jars with the new update you downloaded.
I could write a small program that automatically downloads and patches the file, all I need is a way to tell maven not to download the file but to run a command and then use the file from my local repo.
Is there a way to do this (maybe using a custom plugin)?
You could write a small ant task run the tool and call this task from your maven build using maven-antrun plugin.
You could also write your own maven plugin but for such a simle task I would not recommend it.
The reason why your IDE does not find the dependency is that it is not published anymore (not the version you seek apparently).
Building the artifact and installing it locally as #Joe suggested is an elegant solution to solve your issue from my perspective.
The advantage is twofold:
Your IDE can seamlessly find the missing artifact
You can keep multiple version of that dependency (one for every new patch) and thus reproduce previous (working) build in case a new patch breaks something.
if you have a local repository, it is even better as this would maven the dependency available to all your team.
Of course you would still need a tool to download new patch, apply them and deploy the produced artifact to the repository automatically. But doing this as a separate task give you more flexibility concerning the tools you could use.
Our software is written in Java and comprise many (7) projects.
These projects are Netbeans ant projects.
I'm considering to converting them to maven2.
Where can I find some hints for doing such thing?
Don't read that book. It will only make you confused. Read this book instead: "Maven - The definitive guide" http://www.sonatype.com/books/maven-book/reference/ .
Also, the maven site has a lot of information, but the structure is terrible so you'll need to use google to navigate in it.
Here is my suggestion:
Do this by hand, not with "automagic" "help" from the IDE. Maven integration doesn't work that good yet, not in any IDE.
Make sure you program project is divided into modules under a common umbrella module, so that each module produces a single binary artifact (jar, war,...) possibly accompanied by the javadoc of the source code behind that artifact, a zip with the source code etc. The basic principle is that each module produces a single artifact, containing all the non-test-code under that module. You can do this while the project is still built by ant.
Each module should conform to the standard maven directory layout. The build destination is under [module]/target/[output-type, e.g. "classes"]. The source code is under [module]/src/main/[src-type e.g. "java"] and [module]/test/[src-type]. The artifact consists of all the code under src/main, and none of the code under src/test, as it built to the target directories. You can do this while the is still built by ant.
Start by transforming the sub-module that has no dependencies on other modules in the project.
Now you can create the parent maven module pom.xml with artifact type "pom", consisting of one of the modules below. Make a child module for the first submodule (the one with only external dependencies), using the umbrella module as "parent". Remember that you need to specify version for the parent. Remember to add the child module as a "module" in the parent too. Always use ${project.version} as version in the child modules when you create multi-module projects like this. All modules under a parent must be released simultaneously in a single operation, and if you use this setting maven will make sure the version fields stay the same across all modules and gets updated everywhere during the release. This may make it difficult to re-use the existing numbering scheme, but that doesn't matter. You are never going to run out of version numbers anyway.
Add the necessary dependencies, and make sure you can build the parent and the child module together using the command "mvn clean install" from the parent module.
Proceed with the rest of the modules the same way. Dependencies to other modules under the same parent project should also use ${project.version} as the "version" they are depending on, meaning "the same version as this". NOTE THAT in order to build, the module you are depending on must be built using "mvn install", so that it gets deployed to you local (computer) repository. Otherwise the depending module will not be able to find the classes. There are NO source-code dependencies between modules in maven, only dependencies to built and packed versions installed in local and remote repositories. This can be very confusing if you come from ant-projects. Build from the root module until you get comfortable with this. It takes two days.
Don't use maven integration in IDEs. It is a bad idea. Use "mvn idea:idea" or "mvn eclipse:eclipse" to set up your workspace as a non-maven ordinary IDE project. The inter-module dependencies mechanisms in maven and the IDE aren't identical and will never be. Also, if you have several mavenized projects with dependencies in between, you want to have several of these in your workspace with dependencies set up between. You can do this with mvn idea:idea / eclipse:eclipse if you create a separate maven project file called "workspace.xml" (or whatever) in the same directory as parent module, set up as a multi-module project containing modules "." and "../otherproject" (only one-way reference here, no parent ref back). If you run "mvn idea:idea / eclipse:eclipse -f workspace.xml" you get a workspace with all these modules linked together. No IDE integration lets you do that. This sound like a lot of extra work, but the workspace.xml-file is really small. It doesn't have to contain all that dependency stuff and all that, only the reference to the modules you want to bind together in your IDE.
I did a succeful migration of NetBeans Ant project to Maven project using the instruccions by Joseph Mocker here: http://forums.netbeans.org/ptopic55953.html
I cite the important part:
close the project
rename the build.xml, nbproject files/folders to something so NB won't recognize them.
close and restart NB (so any memory cache knowledge of the project is gone)
copy in an empty pom from some other project.
open the project back up in NB (NB should now identify it as a maven project)
rearrange the files to follow the maven way (™)
This won't be an easy task since Maven2 expects the files to be organized in a specific way. Anyway Better Builds with Maven is a free book that should get you started. It will help you understand Maven and it also has a chapter on migration.
I discovered that the migration is not necessary. The real requirements that I need was automatic download of dependencies (libraries).
This is also achieved by Ivy which nonetheless uses maven repositories.
I solved converting project from ant to ant+ivy with IvyBeans.
I have built a script to migrate Ant builds to Maven. You can find more information here:
https://github.com/ewhauser/ant2maven
It won't help you with fixing your directory structure and or any additional Ant tasks, but it removes a lot of the tedious steps to get started.