I have several projects which share some locally developed libraries; these libraries vary from being fairly static to being flexibly updated as the applications that use them grow.
For debugging/update purposes it is useful to have them as part of my project (when a new library version is released I can test/update the library as necessary if bugs are uncovered while running the main application projects).
For building purposes it's a pain that when rebuilding a project it also rebuilds the (sometimes large) libraries.
From my perspective, it would be ideal if some of the libraries could be set to not rebuild or clean unless it is explicitly done. Or if I could switch them between being JAR/AAR libraries vs. full modules.
At this point I wonder if there is a better way.
So, if you want to include a library (for edit/update purposes) in a project but do not want it rebuilding all the time, is there a way and what is the best practice?
FWIW, the basic structure I currently have is:
MainApp
platform specific code
Library for app (usually the platform independant code specific to the app)
Local Utils lib (can be fairly dynamic, but often goes for days/weeks without changes)
Other Local Utils (fairly static, occasionally updated)
You can make the library as a separate project then install it as a local artifact using maven plugin in the library. Read more at Deploying an Artifact to the Local Cache in Gradle
In your project, you need to add mavenLocal() to the repositories in your root build.gradle:
repositories {
mavenCentral()
mavenLocal()
}
And add the dependencies to your project build.gradle:
dependencies {
...
compile 'com.your.library:x.y.z'
...
}
But if your project is shared in your local network, you need to use Repository management like Artifactory by JFrog or Nexus Repository Manager by Sonatype.
Related
I have implemented a springboot application that can be extended by loading plugins into it. The plugins are created by extending from a common interface specified in my library (LibA) and loaded into the springboot application using Java SPI. Because of this, LibA is a dependency to the springboot application.
There are some common libraries (LibB, LibC) that are used by the plugins which are not part of the springboot application.
The current implementation requires each plugin to be a fat jar containing LibA, LibB, LibC and because of this the versions of the libs need to be the same otherwise there will be issues (such as method not found) while loading these plugins.
I would like to have these common libraries as part of the springboot application and have the plugins refer to the versions in the application. Doing this could result in smaller jars and also ensure the consistency of the version of the libraries used by the plugins.
I could specify LibA, LibB and LibC as dependencies of the springboot application and have the plugins remove them from it's jar and things might work but this would still require the versions to be specified in the plugin's build.gradle.
What I want to know is how to have something similar to the way springboot specifies and provides it's dependencies. Other ways to manage this is also appreciated. I'm using gradle as the build tool.
Current
App (LibA), plugins (LibA, LibB, LibC)
What i need
App (LibA, LibB, LibC), plugins (just refer to the versions in app)
I'm unsure if this is the correct channel to post this question, let me know if it's not suitable.
My team uses a GitHub.com organization to keep all of our source code in private repos. (Prior, our old workflow was emailing Dropbox links). Most of the time each repo is one separate project with no dependancy of any other (the only dependancies are on third-party open source libraries). Or if there is some dependancy, then the .java files have just been copy pasted into the other project.
I've recently been splitting up some of my code into reusable modules, but I don't know any way to do the dependancy management when I use the libraries I'm creating in another project.
I know with Gradle you can add a git repo like this:
gitRepository('https://github.com/user/project.git') {
producesModule('user:project')
}
but I don't know if there's a way to make it work with private repos, and I don't know if there's a way to specify versions.
My currently solution is to just build the library JAR, and keep track of the binary version with GitHub release tagging, and when I need to use the library in another project, I download the desired version of the JAR (typically the most recent) and add it to a local /lib/ folder in the other project and import the JAR into the module path as a local JAR. Of course I need to go through the whole process again manually if I want to make a change to the library.
I also heard you can set up private Gradle or Maven servers and some companies do that, but I guess that would mean migrating away from GitHub.com?
Is there any way to make this work (either Gradle or Maven, it doesn't matter) to manage dependancies between GitHub private repos?
Can someone tell me, what is the most sensible way (or ways) to solve this?
Thanks.
What you need is a very typical maven/gradle based setup where
each of your projects will be producing an artifact with a coordinate
of the form group:name:version
your projects do not have to be explicitly aware of each other. They depend on the artifacts produced by other projects. This is called binary dependency
for a project to locate a binary dependency, you will need a central registry where you can publish all your artifacts to. GitHub has a product called GitHub Package for precisely this purpose.
If you don't want to use GitHub Package yet, or your setup (number of projects, size of each projects, size your team) is small enough, you can locally checkout all the projects and include them into a gradle composite build so that binary dependencies will be substituted with local project dependencies. The good thing about the composite build is that when you decide to invest in a package registry, your build.gradle requires no change at all.
BTW, where you run your private package registry does not really matter. You can use the GitHub Package, or some other hosted services, or even run e.g. jfrog artifactory on your own server. It is completely unrelated to where you host your source code, so you dont need to migrate away from GitHub in any case.
I have a library and a program, both under my control and built using Gradle. What's the best way to develop these two at the same time?
I have set up a private maven repository to distribute the library and that's working, but I don't want to release to that repository every little experiment I make during development. It's slow and disruptive to users of the library.
I tried installing the jar to the local maven repository as explained here: Gradle alternate to mvn install but the project that's using the library is not picking up that newly installed version.
I think, you can try to use multi-project builds for that if it's possible. But you will likely need to restructure both your current projects to become modules of the same new project.
What's the best way to develop these two at the same time?
It depends by how the team is organized and what are your policies.
Of course if the team can use a git repo and access to the source code you can just use git without pushing a new version on the maven server for each commit or push.
Otherwise if other users can only use the final library, you have to push the version on the maven server.
I have set up a private repository to distribute the library and that's working, but I don't want to release to that repository every little experiment I make during development. It's slow and disruptive to users of the library.
Every maven repo has 2 different repositories:
release
snapshot
Usually release repo is used only for stable releases and the snapshot repo is used to publish little change, beta release and so on.
In any case it is not required that every changes in the code is pushed in the maven repo (it is a your choice)
It's slow
The time to upload artifacts usually is not so big, in any case you can evaluate to push the release in the maven repo with a CI server.
The best method seems to be to make one project include the other one when present by adding:
if (file("../libraryproject").exists()) {
includeBuild "../libraryproject"
}
to the settings.gradle file of the project that uses the library. That can be committed to the source code repo because when that directory doesn't exist, the dependency will be included in the traditional way.
This is called Composite Build in the Gradle world, IntelliJ seems to handle properly and theres'a recorded webcast showing the whole setup: https://www.youtube.com/watch?v=grPJanXfRPg
I'm new on a web project where we have to develop plugins (called "extensions" in that specific project). The main application runs in a modified Tomcat web server and we have to add our plugins' .jars in a common lib folder. I'm still not very used to the application and how it works, but I'm pretty sure there is a common classloader for the application and all its plugins. All libraries in that lib folder are shared.
My question is how to deal with the plugins' dependencies and potentially conflictual versions, in that environment.
Should we have shared libraries for example some-common-lib-1.3.4 as jars in the lib folder and plugins have to use those versions when they need to use a library?
Or should a plugin contain its own dependencies (using Maven Shade Plugin for example) so different versions of a same dependency are not an issue?
The problem I see with having shared libraries with a specific version to use for all plugins, is about transitive dependencies. If a common library has a dependency to some-transitive-dependency-1.0.0 and we have a specific plugin which requires a new library which itself have a transitive dependency on some-transitive-dependency-2.0.0 then we're screwed... We would then need both some-transitive-dependency-1.0.0 and some-transitive-dependency-2.0.0 in the lib folder and who knows what will happen.
Also, if for one specific plugin we need to update a dependency to a new major version, we may have to update all plugins since that library is shared by all.
Any real world experience with such situation? Any tips?
Since OSGI is not an option, and potentially everyone could create new plugins, the only feasible way to separate them is, as you already suggested, using the shade plugin or some similar technique.
Since you cannot separate classloaders and recompiling all plugins (for which you might not even have the source code) is really not an option and sometimes you might even have non-resolvable conflicts (asm 1.x and 2.x are totally incompatible), you have to use your own "poor-man's OSGI" and use shade.
Note, however, that this does reduce the option of plugins working together or sharing common data not defined in the main application.
I hope I can keep this question specific enough, my team at work is currently debating the best way to manage our dependencies for a huge project (150+ dependencies ~300mb).
We have two main problems
Keeping all the developers dependencies the same so we are compiling against the same files
Ensure the project (once compiled) is comliped against the same dependencies
The two ideas that have been suggested are using a BirJar (all dependencies in one file) and just adding a version number to it and using a shared folder and pointing everyone's machines at the same place.
Or making including all the dependencies in the jar when we compile it (a jar, of jars, of jars) and just have a project that "has no dependencies"
Someone also mentioned setting up an internal version of Ivy and pointing all the code to pull dependencies from there.
What are the best practices regarding massive dependency management?
Why don't you use Maven and its dependency management ?
You can specify each dependency, its particular version and its scope (compile-time, for testing, for deployment etc.). You can provide a master pom.xml (the config file) that specifies these, and developers can override if they need (say, to evaluate new versions).
e.g. I specify a pom.xml that details the particular jars I require and their versions (or range). Dependent jars are determined/downloaded automatically. I can nominate which of these jars are used for compilation vs. deployment etc. If I use a centralised repository such as Nexus I can then build my artefact (e.g. a library) and deploy that into Nexus, and it'll become available for other developers to download in exactly the same manner as 3rd party libs etc.
Incase you dont like/want to follow the Maven project structure...
If you already use Ant, then your best bet is to use Ivy for dependency management.
http://ant.apache.org/ivy/
It provides a rich set of ant tasks for dependency manipulation.
from : Ant dependency management