I have seen many interesting (and duplicated) questions here about "sharing or using classes between projects".
I see this as quite practical but the proposed solutions I have read about definitely assume certain prerequisites such as:
shared eclipse workspaces
projects that can be made as dependencies of oneanother
common servers such that classpaths can be added with local urls
While likely acceptable solutions, I am looking for an alternative with perhaps greater flexibility and portability.
I am thinking of learning how to use gradle (or maybe maven, I haven't fully committed to one or the other yet). And from what I understand it may be possible to manage shared classes with one of these dedicated dependency management programs.
Theoretically is this possible? Can I setup a gradle or maven enabled java project to handle and keep uptodate personal classes on a local server or folder on a portable drive or cloud mirror?
The way I understand dependency management at the moment (on a superficial level, I know the devil is in the details) is that for a configured dependency management enabled project, gradle/maven will handle classpath additions and the actual version specific comparison, retrieval and storage (and maybe even compilation is possible but I don't know about this) of JARS from external sources.
Rather than go through the steps to setup classpaths to jars I have to keep current and compiled myself as proposed in many other answers, I am considering creating a dummy project on a server that I can put generic classes which I could then point numerous individual gradle/maven enabled java projects to use. (I think most people would be able to keep them as stand alone classes, but I think I might need to keep them in a dummy project to be developed and debugged in context from a main class. I am somewhat new to java architecture so if the only thing that would make this solution impossible is pointing to a "project" instead of a "library" I can definitely adjust from there. (Assuming I am even applying the concept of the "library appropriately).)
Other info:
I would like this to simplify personal dependency using both Netbeans and eclipse IDE's and work cross platform (but Linux and Windows is what I plan to test it on)
So you're looking for portability, and you don't want to compile your java class that you want to share between projects. And you don't mind a local deployment.
The first thing that comes to mind for me is Git - I'm not sure if Gradle/Maven deal in the gritty underworld of the uncompiled. Composer will pull in git repos for php, so that got me thinking.
If you're happy with one-way sharing of code among projects, Git has submodules that let you do that.
But searching around, apparently there's a git script that goes one step further - Git Subtrees. I also found an intriguing tutorial that will allow you to make changes to common code that you change while working on any particular project that shares it - so obviously be careful - but check it out and see if the Subtrees script might suit your needs.
Actually, I don't see too much sense for dependency management on a "class level". Typically you would bundle your classes in a jar file, which in turn can be considered as a unit with a particular functional range. Such a jar is suitable to be put in a dependency management.
If you are new to such tools, I'd recommend Maven. It is widely used in the Java world and well-integrated in common IDEs. If you stick to its conventions, it will take care of your whole build process from compiling, testing to packaging. There are a lot of plugins available that let you customize practically everything in a simple XML based configuration. You'll have your first project running in 30min and your current project migrated in another 30min.
To share your code with others, you still need a repository where you can upload your Maven-built artifacts to. Depending on your preference there are many possibilities. Shove it to Amazon S3, Maven Central or install your own Sonatype Nexus in your private network.
Related
In a java application that uses plugins (such as Minecraft server, an IDE ...etc) many times it's necessary to use some library and shadow it in your .jar.
The problem comes across when another plugin also uses that library and also includes it in it's .jar but with a different version, both plugins will work fine until they meet together under the same classloader, then, the classloader will load just one version of the library and the other plugin will be running with a different version than the one it was designed for, leading to NoSuchMethodExceptions and/or different runtime behavior if the library is not backward compatible.
Note: This question is not about how to compile with maven when 2 dependencies have the same transitory dependency but in a different version. The problem is way different in this case because you cannot resolve or even detect dependency issues at compile time since the different versions of the library aren't going to be in the same .jar, neither in the same project, only under the same classloader at a given time by 2 plugins probably made by different authors with different purposes.
What is the general strategy to avoid such conflicts?
If I'm making my own library and it cannot be backward compatible, what can I do to avoid this problem?
This is a classic java problem, and there are some major redesigns of java technologies underway to change this, though these are a while off (Java 9 or Java 10).
For now, the best thing you can do is explicit and regular dependency analysis of your projects.
To start, run this: mvn dependency:tree
You will be able to see multiple versions of the same dependency. Unfortunately when using external libraries, you may need to conform to the same dependency version as they use, or look for an updated build of the external dependency.
I have used a lot of time in dependency analysis, and it mostly has to be done manually (one of the bugbears of working with java...)
If you are making your own software, us explicit API versioning and make a significant change to the version whenever you make a breaking change.
For example, use "version 1.1", "version 1.2" and so on. Let people know that version 1.1.1 and 1.1.4 are backwards compatible with 1.1, but as soon as you introduce an API breaking change (this happens), go to version 1.2 and force people to update.
Unfortunately this is an issue faced in my application especially when you can't manage or maintain any third party pr external system. To minimize such situations you should create parent poms and put external dependencies there. Also regularly check if any mismatch or conflicts are there and if can be removed at development stage. Sometimes u reach dead lock when your application need a specific dependency but rest of the architecture is build on different one. Roll out new changes with custom build packs and huge changes and sometimes you can't help it. Precaution is better than cure in this case.
#vikingsteve's answer explains how to design your APIs to avoid the problem. Unfortunately, sometimes practical considerations will get in the way. For example, when your code depends on two large libraries that (in turn) have dependencies on different versions of a third library. And those versions are incompatible.
If you are in this situation, and you can't solve it by rewriting the libraries, bugging the library developers / vendors to fix it, etc., there are still a couple of options:
You could restructure your software do that the libraries with conflicting dependencies are in separate applications; e.g. split them into a "client" and a "server", or "peers" communicating over a message bus, or a parent and child process talking over a pipe.
You could restructure your codebase so that the two libraries are loaded in two separate child classloaders of your main classloader. This would allow the two versions of the common dependency to be loaded by the two child classloaders .... and co-exist within the JVM.
Obviously, neither of these approaches is a magic bullet. And both will entail some rework of your application. But they may solve your "dependency hell" problems.
My solution was to create another independent running jar application with the external dependencies I needed and perform the tasks there.
What exactly do Gradle and Maven "do" that eclipse or sts doesn't? From what I've read it builds, runs, etc which can all be done in eclipse easily.
If I have an existing project I've created, built, and currently runs in eclipse via tomcat, what would I use gradle for?
There's not a lot of benefit to using Maven or Gradle on a small project that you never share with anyone; an IDE can do the build just as well. But as the number of developers increases and the complexity of the build increases, it becomes very useful to separate the build instructions from the IDE. Let's drill into these a little bit.
With the increase in developers, you don't want everyone to have to come by and use your IDE to get a build done. That would be really annoying! So they're on their own machines, but then they tend to have different setups (how dare they have different user account names!) and probably have their IDE installations set up a little differently too. So we need some kind of build instructions that people can use to get things going, and it helps if everyone can use the same build instructions repeatably so that you don't get too many instances of “but it works on my machine!”. It's also very helpful if those instructions are simple enough to use that a new programmer to the team can get up to speed rapidly.
But the other thing that often happens as projects grow in scope is that their builds become more complicated. They very often gain additional dependencies (they didn't start out needing a high-performance date parser and MIME-type identifier, buit they've become required since and you don't want to have to write all those from scratch) and that means you've got to make sure that when the build is done, the right version of those dependencies is used. But that's not the only way that complexity increases. It's also very often the case that you find you're using more automatically-generated code. You might find yourself working with XML schemas or WSDL a lot, or maybe your using Hibernate, or Spring, or … well, there's lots of ways in which things can get complicated, OK? Getting the various steps to do all the build right, reliably, in these sorts of scenarios can be a bit tricky, but encoding them as instructions to something like Maven makes life a lot easier once you've taken the jump in the first place. (It gets even more important when you start trying to deal with projects which need many different sub-programs that work in concert; some of those are plain hard to build even with Maven or Gradle or any other tool.)
And then there's the possibility of offloading work to a build server, running tests automatically, managing dependencies cleanly, etc. IDEs don't handle these all that well by themselves; where they do a reasonable job of it, it's usually because they're using a tool like Maven under the covers to do the heavy lifting.
tl;dr
You don't have to make your code work with a build system, but it helps if you do and in many ways.
Maven and Gradle can do many things that Eclipse doesn't. However, the most important thing they do, is to decouple the bulding and testing processes from the IDE you choose (i.e. Eclipse). When you work on a large environment, with many programmers, usually you can not control the IDE they use. So, it's better to use a tool like Maven and Gradle to standardize these tasks. The same happens with the code examples of a book: instead of the authors having to provide the instructions for configuring any IDE to execute them, they provide the Maven or Gradle files, so the reader can build and test them on any IDE he's using.
Another very important feature that Maven and Gradle give you, is the fact that dependencies are managed without the need of having the executable code under source version control. Instead of having the executable code you depend on as part of the project, you declare the dependencies on a text file (which is under source version control), and then get them from a repository.
However, you may only see the real advantages of using tools like Maven or Gradle (and even Jenkins or Hudson), when you think in large scale projects, developed along many months by teams composed of many developers).
Gradle and Maven are build tools. Maven was first and is a bit older, Gradle is newer and has redefined a way of how projects are built and maintained. In my opinion it's also much easier to use, more readable and easier to maintain. I prefer Gradle ;)
You use eclipse or STS (any other IDE) for development. And while You finish this process You need to provide a configured artifact (war, ear, whatever...) to production and deploy it there. These artifacts have well defined format and the application won't be run from eclipse or STS at the production environment. It's tiresome and error prone to prepare such artifacts by hand.
Gradle or Maven can take responsbility of building and preparing these artifacts (in fact such tools can do much more) off Your shoulders, they make this process automated.
I recently joined a new team. This team has been using Subversion for their source control. They asked me to investigate the possibility of moving the source code to Git.
This seemed like a trivial exercise until I realized they have a large number of svn:externals. The project is a large Java EE project that consists of several web projects. The team uses svn:externals to share some fixed resources (like css, javascript, etc.) amongst the various projects.
My question is twofold:
Is there a better strategy for sharing resources within a set of Java projects that we could consider, allowing us to remove the necessity for svn:externals and making the switch to git simpler?
If no such better strategy exists, then what are my alternatives?
This is quite a large codebase (lots of revisions and files) so I'd like to take advantage of the performance of git.
Here is the relevant software stack they are using:
Subversion
Eclipse
Maven (2.2.1 IIRC)
Windows (for development), AIX (for production)
Thanks
Sounds like a good case for Git submodules. They should be pretty similar to svn:externals.
Excerpt from the Git Book:
It often happens that while working on one project, you need to use another project from within it. Perhaps it’s a library that a third party developed or that you’re developing separately and using in multiple parent projects. A common issue arises in these scenarios: you want to be able to treat the two projects as separate yet still be able to use one from within the other.
Read more on Git submodules on the Git book: http://book.git-scm.com/5_submodules.html
On large-scale Java/.Net Enterprise projects, does every developer need to have all the components/libraries/dependencies in their classpath/local development environment, in order to make it build?
Or are they divided up into smaller sections can be built in isolation (so that they don't need to reference all the dependencies)?
In other words: if they want to run the whole application, they need all the components; but if they are only running a subset of the app, they'll only need the corresponding subset of components.
Are large enterprise projects usually organized in the first way or the second way?
A possible organization is if you are working on a module of the whole project that is self-contained, but referenced by other modules (in other words, a leaf-node in the dependency tree).
Another organization is if you dynamically load classes that you use, you can build without having any of them in your classpath. To run it, your classpath only needs to access the ones that you actually load (there might be many others that form different parts of the project, that you don't load).
These are theoretical possibilities; but what's standard practice for enterprise projects, in... well, in practice?
I've expanded this to include .Net, because I think the same issues would arise there (DLL hell?)
There's a different answer to this question for every project out there. A few general points:
"running a subset of the app" is often not possible, as very few apps are modular enough so that each part of them can actually run independantly.
What you sometimes have is an app core that is always required, and modules built on that core that are more or less independant of each other.
The big difference is usually not between having vs. not having all components, but between having them as source code vs. having them as JAR files.
On large apps, developers typically have only the parts they're working on in source code and the rest as JAR files
If you need runtime modularization (i.e. components are loaded and unloaded on demand at runtime), that's what OSGi is intended for.
They may need only a subset to build, and another subset to run their tests, but because all dependencies of less-than-trivially-sized Java projects can very quickly become a nightmare to keep track of, Java developers have come to love developed a love/hate-relationship with their elaborate build systems, such as Maven, which manage their development environment for them.
For projects that do not use such a system, it is generally easiest to just include everything all the time. The trade-off is unnecessarily bloated development environments versus having to spend time to track down missing dependencies.
A good project structure will break down things so that you can run independent modules.
But in real life, most projects I've seen don't do this until someone gets fed up and takes initiative to break them down.
If you use a good dependency management infrastructure like Maven or Ivy properly, you can store compiled modules on a server and download these dependencies on an as-needed basis.
You can also get away with having many mock objects and services to help break down the testing dependencies on other product components.
I certainly agree with the comments that it would be "good" to separate things. But in practice, that's very rare.
Assuming that you must work in an environment which has not been separated, there's another organizational strategy, and it's what I've seen used. Since your question refers to both build and run dependencies, you don't appear to be talking about processes, but about classes and jars.
The simple solution for that is to have the complete set of built, integration-tested (or integration-test-ready, for that matter) dependencies up on a shared server.
Then developers build in their local environments the portions of the system on which they're working, using a classpath which references first their development and then the appropriate shared server.
Your question isn't very clear, but I think the answer is that every class your application needs has to be in the CLASSPATH or the class loader with throw a ClassNotFoundException.
That's true whether you're a solo developer or working on a larger, distributed team.
In my experience, applications are packaged one way. If you only want a subset, you have to package it as such.
If you mean test cases as something separate, those usually aren't packaged with production code.
In my opinion, it's not whether developers can work on a subset of the application, but rather managing the dependencies between the projects (think Eclipse projects) that make up the app. Often you might have a tree of such projects where one or more project can depend on other projects. In such cases it's usually the role of the upstream/common project to make sure downstream projects are not broken due to changes in this upstream project.
Think of it like this - let's assume you have a utils project where you put all the common/utility functionality for your application - this could be validation logic, string utilities, logging, etc. And you have a bunch of other projects that use classes from this utils.
utils
/ \
proja projb
In this case, the person working on utils should also have proja and projb on their development environment as any change to utils will break them. However if you're only working on projb then you might not have to include proja as you have no dependency to that project.
I have a large, growing OSGi application with a number of bundles. I am curious to know the best way to manage this type of application. Currently, I am using Eclipse and Maven, but although this is great for building bundles (via maven-bundle-plugin), as of now it has not been easy to manage the entire application.
What I would like to do is either have ONE run configuration or ONE pom.xml that can be launched and the entire application/project be built and launched. Also, I would like to have something that would be good for debugging.
I have heard of PAX Construct and have it installed in Eclipse, but so far it has been of little help (maybe I'm not using it correctly).
I am sure there are people out there with large OSGi applications that are being managed correctly. Any advice that could be shared would help tremendously.
Thank you,
Stephen
A run configuration is possible via Pax Runner. It lets you choose OSGi platform implementation, specify profiles (pre-packaged sets of bundles for some role, e.g. web, log, ds, etc.) and has good provisioning support, for instance it can load bundles from Maven repository. As a result, you can have a run configuration like
--platform=felix
--log=INFO
--profiles=scalamodules,ds,config,log
mvn:com.my/bundle/1.0.1-SNAPSHOT#update
# other bundles
In case your application is very large or you have different applications, there a way to create own profiles as well.
Well...
It all deopends on what do You mean by "managing" the application.
For dev time launching, building and debugging - Eclipse IDE should fit the bill just perfectly.
Maven... I can't speak for it, as I've never used it myself.
We have a pretty large eclipse based application (several, actually) and on the dev side of things we are not using anything special besides the Eclipse and it's integrated SCM.
In the cc build server, we also use headless eclipse to do the building and packaging.
Now the setup of the workspace has gone a bit out of hand of late with all the dependencies and intermediate build steps, so we are investigating Buckminster for managing the materialization of target platform and workspace resources.
If that works out, we'll probably move to building with Bucky as well - it sure looks promising.
(I do not have any experience with PAX, but at a glance, it looks promising as well...)
i'm quite new to OSGi but,
wouldn't it be possible to use OBR-service in such a way that
you would have one OBR repository file which needs the bundles
and let the OBR-service figure out the dependencies and populate your OSGIhost for you?
This area I think has very poor support at the moment. OSGI doesn't really define anything about deployment or packaging so its up to other frameworks (e.g. Eclipse) to come up with their own way of doing it.
If you are building an RCP (Eclipse base) application, then the eclipse systems do all this stuff, right down to creating exes etc. However builds are mainly done on the Eclipse workspace, headless builds are trickier. The Tycho project is trying to make this more sensible by joining the Maven and Eclipse build cycles, however it is still focussed on RCP applications rather than generic OSGI.
If you not doing RCP, which is my situation as well, then you probably have to roll your own solution, as I haven't found any general solution. Here's an outline of what we do:
We define one POM project that lists all the bundles that are contained in your application. All this project does is list the references - lets call it the 'bundle-list' project.
Then, we use pax provision to run the project in development mode. This is achieved by making the 'bundle-list' pom the parent of the provisioning pom of the pax project (usually in the 'provision' folder). Then, when you start pax, it uses the list of bundles from that project to start OSGI. The bundle references in the 'bundle-list' project have to be marked as 'provided' scope for this to work.
Then, to create a distribution, we have another project. This project also has the 'bundle-list' project as its parent. This project uses various plugins to create a distribution, including downloading the bundle jars. The distribution includes scripts that start up OSGI, but these are hand written, there's no pax systems here.
This works well for us to keep the list of bundles in one place, but there's still a lot of hand written scripts, and there are issues sharing configuration between the two systems - e.g. config files, bundle start levels etc.