My team uses a GitHub.com organization to keep all of our source code in private repos. (Prior, our old workflow was emailing Dropbox links). Most of the time each repo is one separate project with no dependancy of any other (the only dependancies are on third-party open source libraries). Or if there is some dependancy, then the .java files have just been copy pasted into the other project.
I've recently been splitting up some of my code into reusable modules, but I don't know any way to do the dependancy management when I use the libraries I'm creating in another project.
I know with Gradle you can add a git repo like this:
gitRepository('https://github.com/user/project.git') {
producesModule('user:project')
}
but I don't know if there's a way to make it work with private repos, and I don't know if there's a way to specify versions.
My currently solution is to just build the library JAR, and keep track of the binary version with GitHub release tagging, and when I need to use the library in another project, I download the desired version of the JAR (typically the most recent) and add it to a local /lib/ folder in the other project and import the JAR into the module path as a local JAR. Of course I need to go through the whole process again manually if I want to make a change to the library.
I also heard you can set up private Gradle or Maven servers and some companies do that, but I guess that would mean migrating away from GitHub.com?
Is there any way to make this work (either Gradle or Maven, it doesn't matter) to manage dependancies between GitHub private repos?
Can someone tell me, what is the most sensible way (or ways) to solve this?
Thanks.
What you need is a very typical maven/gradle based setup where
each of your projects will be producing an artifact with a coordinate
of the form group:name:version
your projects do not have to be explicitly aware of each other. They depend on the artifacts produced by other projects. This is called binary dependency
for a project to locate a binary dependency, you will need a central registry where you can publish all your artifacts to. GitHub has a product called GitHub Package for precisely this purpose.
If you don't want to use GitHub Package yet, or your setup (number of projects, size of each projects, size your team) is small enough, you can locally checkout all the projects and include them into a gradle composite build so that binary dependencies will be substituted with local project dependencies. The good thing about the composite build is that when you decide to invest in a package registry, your build.gradle requires no change at all.
BTW, where you run your private package registry does not really matter. You can use the GitHub Package, or some other hosted services, or even run e.g. jfrog artifactory on your own server. It is completely unrelated to where you host your source code, so you dont need to migrate away from GitHub in any case.
Related
I am creating a java project in IntelliJ (without maven or grandle). The project uses an external library, whose .jar file I’ve put into a /lib directory. After that I had to select at the /lib folder “add as library” to use it.
Now I want to push the project to GitHub, so that some people (who are using IntelliJ as well, but in different versions) can use the project.
Now my question:
Is there a way, that they do not have to do the step “add as a library” themselves?
My first idea was to push also some parts of the .idea folder to GitHub, but I am not sure which ones to push and if that could actually work (especially with different versions of IntelliJ).
Do you have any idea how to solve this issue?
If you are using only IntelliJ for building the project, then yes, you should push the .iml files from the .idea folder (or where they happen to be), since they contain the dependencies configured in IntelliJ.
Note, that projects with multiple contributors typically use a build tool like Maven or Gradle.
This is a special build requirement, which I would use Gradle for. With Gradle you can look up a given folder, like /lib and use all .jar files as dependency.
See Gradle example about exactly what you want.
IntelliJ is handy when you do something simple mostly for learning, but if you want to be a professional one day I highly suggest looking into Gradle. It has a learning curve for sure, but you can achieve such simple tasks like this in your question relatively simply. And as you seem to know, pushing .idea to the repository is really not the nicest thing one can do :)
Just a small additional note: Gradle solves the "different version" problem by including a "Gradle wrapper" inside the repository, so everyone cloning the repository will have the same copy of Gradle as well, so the same build process is guaranteed for all contributors.
Also, when I started programming I downloaded the dependencies and used them as jars. But if you learn at least Maven, and your dependency is uploaded to a repository like Maven Central, you can just paste a line of code into your pom.xml (Maven) or build.gradle (Gradle) and you are good to go :)
I want to use the Google Firebase Messaging library in my Android Project. But my laptop just works offline and has not any access to the internet. I want to download FirebsaeMessaging library with all it's dependencies then add those files as aar library files to my project. First I want to get list of all dependencies of that library. How can i do this without internet? Thanks.
There are two approaches in general:
You download the complete dependencies with your online system, copy them into a libs folder and then create a project where you dependent on every jar in that libs folder.
You create a maven based project, build the project so that every dependency is copied to your local repository and copy the .m2 (local repository) to your offline machine. In case you want a gradle based project, #lance-java already posted a solution to that.
The first approach has the advantage that you have an isolated project with no dependency to the internet. Everything is in your project and explicit. You would have to check in your dependencies into your source control system like in good old times.
The second approach is good, if you are working in a team on that project. The export of your dependencies hat nothing to do with the project setup itself. Every programmer would find a "normal" project setup and dependency management.
If guess, that you want to go for the 1. way. If so, I would suggest to use ivy to download the complete dependency tree. Put the ivy-X.Y.Z.jar into your project beside a script that lists all dependencies - some sort of poor mans dependency management.
Example:
$ java -jar ivy-2.5.0.jar -dependency com.google.firebase firebase-admin 6.8.1 -retrieve "libs/[artifact](-[classifier]).[ext]"
This command would download the complete dependency tree for com.google.firebase:firebase-admin:6.8.1 and store the jars without version into the libs folder.
You could use my dependency-export plugin to export dependencies to a directory using maven directory conventions. You can then use the directory as a local repository for building offline.
As of Gradle version 6.1 the dependency cache is portable meaning that you can copy the dependency cache from one machine to another and the second machine should be able build offline. Perhaps this is what you want?
See this issue
I am a new IntelliJ user (I've used Eclipse for many years). I'm having a hard time wrapping my head around the-project-is-a-module concept in IntelliJ, since this does not hold true in Eclipse. The main issue I'm having is that I am using my top level package as the project in IntelliJ. I would like this top level package to be in a git repo. I would also like all the dependencies of this package to be in their own respetive git repos. When I check these packages out into my project, a do git status on the top level package, all of the dependencies show up in the untracked files. This behavior seems incorrect to me. How can I fix it?
Thanks!
Edit:
To summarize the clarifications in the comments:
I would like to support hundreds of libraries any of which could change at a time. The dependency graph will also be frequently changing. For this reason, having one git repo or constantly updating .gitignore files is not maintainable.
Currently, I'm using Maven to manage dependencies but I'm open to using whatever is best suited for this job.
Finally, I would like to check out any library into my workspace and modify it and, if possible, have Intellij reflect my local changes when running code as if my local code were already built into the dependency graph. A type of local override if you will.
IntelliJ's directory structure places all of the modules in their parent project's directory. If you are developing libraries which are shared between several other projects, importing the library as a module is probably not the correct solution. Instead, you should treat each library as its own independent project and make "releases" using a build tool such as gradle or maven. Then your projects can treat the libraries the same way they do third-party libraries and use the build tool to import the library.
I'm setting up a new Java project on GitHub, and I'll have some Apache Commons libraries as dependencies.
What are the best practices to establish that my project needs those jar files? Should I upload the dependencies to my GitHub repository (ugly)? Or use a Maven-like tool for that?
Or is there a way to link a file in another git repository? Apache provides git repositories for they libraries. They are read-only, but I'm o.k. with that, since I just want to use the jars. The bad thing is that they contain all the sources, and I just care about the compiled jar. It seems we can't git submodule just a file.
The two approaches are:
declarative and component-based, where you declare (describe) what components (jars, exe, other binaries) you need for your project to (compile, execute, deploy, etc.), and you use a third-party tool (like Maven/Nexus) to bring those components on demand.
So they aren't versioned in your repo. They are declared/described (like in a pom.xml, if you were to use Nexus)
See also "Difference between Git and Nexus?".
inclusive and system-based, where you complete your project with other project sources/binaries, in order to get in your repo everything you need right after the clone step (no need to call a third-party tool or to do anything: every other part of your system in there).
With Git, especially if those "other parts" are in a git repo (like the apache libs one), then you would declare those sub-repos as submodules of your main repo.
That way, all you keep in your main repo is a special entry (gitlink, mode 160000) referencing a specific SHA1 of another repo (but you can make that submodule follow a branch too, a bit like svn external).
And with sparse checkouts in submodules (as in this example), you even can update those modules for them to checkout only the part of the repo you want (like just the jars, not the sources).
Note that you aren't supposed to store any delivery that you would produce (like jars of your own) in your GitHub repo.
You can associate those deliveries to GitHub releases though.
I have multiple java projects in a folder. Also there is a second folder with libraries, that might be used as build dependencies from the projects. The projects may also have dependencies to other Projects. What's the best approach to build all projects ?
In other words I want to build the projects without explicit telling their dependencies.I think the biggest problem is the dependecy between the projects.
There are multiple build systems that are available that you may use. Maven has a complete dependency system built into it. Almost all third party open source jars are directly accessible via the World Wide Maven repository system. Basically, you describe the jar you need (groupId, artifactId, and version) and Maven will automatically fetch it for you. Not only that, but Maven also will build your project without having to create a build file. Instead, you have to describe your project in a project object model (a pom.xml file) and Maven will download everything you need, including all compilers, etc.
Almost all new projects use Maven, but Maven has a few downsides:
Since you don't control a build process, it can sometimes feel like poking a prodding a black box to get the build to work the way you want.
Documentation can be scant -- especially if you're moving beyond basic Java compiles.
You usually have to arrange your project in a specific layout. For example, source files should go under src/main/java while JUnit tests are under src/test/java. You don't have to follow the recommended layout, but then you'd have to modify the pom.xml file this way and that to get your build to work. That defeats the whole purpose of the pom.xml in the first place.
If you already have another build system setup (like Ant), you lose everything. There's no easy way to move from Ant to Maven.
The other is called Ant with Ivy. Ivy uses Ant for building, but can access Maven's world wide repository system for third party dependencies. It's a great compromise if you already are heavily invested in Ant. I also find Ant with Ivy to be better documented than Maven (although that's not too difficult). There's an excellent chapter going over the basics of Ivy in Manning Publication's Ant in Action.
With either process, I would recommend that you build a company wide Maven repository using either Nexus or Artifactory. This way, any proprietary third party jars (like Oracle jars) can also be stored in your company wide Maven repository since they won't be in the standard World Wide Maven repository.
By the way, if this is a company wide effort, and you are moving multiple Ant projects into Ivy, I have an Ivy project I use in Github that makes things easier.
Oh, there's a third possibility called Gradle which I know nothing about. I also believe it can use the World Wide Maven repository. It's based on Groovy which is based on Java syntax, and that's about all I can say. Maybe others can fill you in on the details. The Gradle group contends it solves a lot of problems of both Ant/Ivy and Maven.
Whatever tool you use, if you have various projects interdependent, you need to be clear on the independent ones which will be built first before building the dependent projects. You need to have a clear dependency structure for your projects.
You can do this with Apache Ivy. You can lay out the locations for you common libraries, define published artifacts and inter-dependencies in an ivy.xml document in each project, and let a top-level Ant build with the Ivy tasks figure out what the build order should be based on those dependencies.