Managing a Large OSGi Application - java

I have a large, growing OSGi application with a number of bundles. I am curious to know the best way to manage this type of application. Currently, I am using Eclipse and Maven, but although this is great for building bundles (via maven-bundle-plugin), as of now it has not been easy to manage the entire application.
What I would like to do is either have ONE run configuration or ONE pom.xml that can be launched and the entire application/project be built and launched. Also, I would like to have something that would be good for debugging.
I have heard of PAX Construct and have it installed in Eclipse, but so far it has been of little help (maybe I'm not using it correctly).
I am sure there are people out there with large OSGi applications that are being managed correctly. Any advice that could be shared would help tremendously.
Thank you,
Stephen

A run configuration is possible via Pax Runner. It lets you choose OSGi platform implementation, specify profiles (pre-packaged sets of bundles for some role, e.g. web, log, ds, etc.) and has good provisioning support, for instance it can load bundles from Maven repository. As a result, you can have a run configuration like
--platform=felix
--log=INFO
--profiles=scalamodules,ds,config,log
mvn:com.my/bundle/1.0.1-SNAPSHOT#update
# other bundles
In case your application is very large or you have different applications, there a way to create own profiles as well.

Well...
It all deopends on what do You mean by "managing" the application.
For dev time launching, building and debugging - Eclipse IDE should fit the bill just perfectly.
Maven... I can't speak for it, as I've never used it myself.
We have a pretty large eclipse based application (several, actually) and on the dev side of things we are not using anything special besides the Eclipse and it's integrated SCM.
In the cc build server, we also use headless eclipse to do the building and packaging.
Now the setup of the workspace has gone a bit out of hand of late with all the dependencies and intermediate build steps, so we are investigating Buckminster for managing the materialization of target platform and workspace resources.
If that works out, we'll probably move to building with Bucky as well - it sure looks promising.
(I do not have any experience with PAX, but at a glance, it looks promising as well...)

i'm quite new to OSGi but,
wouldn't it be possible to use OBR-service in such a way that
you would have one OBR repository file which needs the bundles
and let the OBR-service figure out the dependencies and populate your OSGIhost for you?

This area I think has very poor support at the moment. OSGI doesn't really define anything about deployment or packaging so its up to other frameworks (e.g. Eclipse) to come up with their own way of doing it.
If you are building an RCP (Eclipse base) application, then the eclipse systems do all this stuff, right down to creating exes etc. However builds are mainly done on the Eclipse workspace, headless builds are trickier. The Tycho project is trying to make this more sensible by joining the Maven and Eclipse build cycles, however it is still focussed on RCP applications rather than generic OSGI.
If you not doing RCP, which is my situation as well, then you probably have to roll your own solution, as I haven't found any general solution. Here's an outline of what we do:
We define one POM project that lists all the bundles that are contained in your application. All this project does is list the references - lets call it the 'bundle-list' project.
Then, we use pax provision to run the project in development mode. This is achieved by making the 'bundle-list' pom the parent of the provisioning pom of the pax project (usually in the 'provision' folder). Then, when you start pax, it uses the list of bundles from that project to start OSGI. The bundle references in the 'bundle-list' project have to be marked as 'provided' scope for this to work.
Then, to create a distribution, we have another project. This project also has the 'bundle-list' project as its parent. This project uses various plugins to create a distribution, including downloading the bundle jars. The distribution includes scripts that start up OSGI, but these are hand written, there's no pax systems here.
This works well for us to keep the list of bundles in one place, but there's still a lot of hand written scripts, and there are issues sharing configuration between the two systems - e.g. config files, bundle start levels etc.

Related

Java dependancy management from a personal dependancy storage/maintenance destination

I have seen many interesting (and duplicated) questions here about "sharing or using classes between projects".
I see this as quite practical but the proposed solutions I have read about definitely assume certain prerequisites such as:
shared eclipse workspaces
projects that can be made as dependencies of oneanother
common servers such that classpaths can be added with local urls
While likely acceptable solutions, I am looking for an alternative with perhaps greater flexibility and portability.
I am thinking of learning how to use gradle (or maybe maven, I haven't fully committed to one or the other yet). And from what I understand it may be possible to manage shared classes with one of these dedicated dependency management programs.
Theoretically is this possible? Can I setup a gradle or maven enabled java project to handle and keep uptodate personal classes on a local server or folder on a portable drive or cloud mirror?
The way I understand dependency management at the moment (on a superficial level, I know the devil is in the details) is that for a configured dependency management enabled project, gradle/maven will handle classpath additions and the actual version specific comparison, retrieval and storage (and maybe even compilation is possible but I don't know about this) of JARS from external sources.
Rather than go through the steps to setup classpaths to jars I have to keep current and compiled myself as proposed in many other answers, I am considering creating a dummy project on a server that I can put generic classes which I could then point numerous individual gradle/maven enabled java projects to use. (I think most people would be able to keep them as stand alone classes, but I think I might need to keep them in a dummy project to be developed and debugged in context from a main class. I am somewhat new to java architecture so if the only thing that would make this solution impossible is pointing to a "project" instead of a "library" I can definitely adjust from there. (Assuming I am even applying the concept of the "library appropriately).)
Other info:
I would like this to simplify personal dependency using both Netbeans and eclipse IDE's and work cross platform (but Linux and Windows is what I plan to test it on)
So you're looking for portability, and you don't want to compile your java class that you want to share between projects. And you don't mind a local deployment.
The first thing that comes to mind for me is Git - I'm not sure if Gradle/Maven deal in the gritty underworld of the uncompiled. Composer will pull in git repos for php, so that got me thinking.
If you're happy with one-way sharing of code among projects, Git has submodules that let you do that.
But searching around, apparently there's a git script that goes one step further - Git Subtrees. I also found an intriguing tutorial that will allow you to make changes to common code that you change while working on any particular project that shares it - so obviously be careful - but check it out and see if the Subtrees script might suit your needs.
Actually, I don't see too much sense for dependency management on a "class level". Typically you would bundle your classes in a jar file, which in turn can be considered as a unit with a particular functional range. Such a jar is suitable to be put in a dependency management.
If you are new to such tools, I'd recommend Maven. It is widely used in the Java world and well-integrated in common IDEs. If you stick to its conventions, it will take care of your whole build process from compiling, testing to packaging. There are a lot of plugins available that let you customize practically everything in a simple XML based configuration. You'll have your first project running in 30min and your current project migrated in another 30min.
To share your code with others, you still need a repository where you can upload your Maven-built artifacts to. Depending on your preference there are many possibilities. Shove it to Amazon S3, Maven Central or install your own Sonatype Nexus in your private network.

How to get started with Java modules?

I'm reading about OSGi and JBoss modules. I understand the purpose of modules -- to avoid Jar hell and conflicting classes -- but I don't know where to start if I want to write a new application that uses them. Is there a kernel somewhere that I can start with, that then loads the other modules? I don't want to run Eclipse or Wildfly; I want to start fresh with my own app. Hello world for modules.
Edit: I'm using IntelliJ, and for several reasons I really don't want to switch to Eclipse. So I can't do much with answers that require I install Eclipse plugins.
OSGi would be the first choice. JBoss Modules is still much more an internal JBoss thing than a general purpose modularity solution.
To start with an OSGi application, take a look at Bndtools. This is an Eclipse plugin to make OSGi development easy. From there you can create a "run configuration" which is the definition of your application. From there you can also export the application to a single executable JAR to distribute your app.
This is a nice video to get you started.
Also, for more high level components (REST, databases, scheduling etc.) take a look at Amdatu.
Try starting with OSGi enRoute: http://enroute.osgi.org/. This is a project, still in development, to demonstrate using OSGi.

Which .JAR do I use to embed Jetty?

I'm at the point in my application where I would like to have an HTTP Server embedded into my project that updates the page in real-time using AJAJ(Similar to AJAX). However, I have no idea where to begin and the amount of tutorials on this subject are fairly limited, so I decided to go with a name that I've heard quite a few times before, Jetty
So, I downloaded Jetty and read through some documentation, and I'm staring at their beginner tutorial asking myself, "Which one of these f*kin jars do I use?" There's like 9,001 of them. Not to mention that there's like 1200 folders that all contain 1500 more jar files each.
Okay, I'm over exaggerating, but take a look.
It's fairly, uhm... confusing. This is much different than most libraries that are a single jar file, this is just... insane.
Anyway, I'm trying to figure out what all I need to be able to use JQuery, AJAX(AJAJ), and basic HTML features.
I'd suggest you to start with this simple tutorial and jetty-all jar
Embedding Jetty Webinar recording
Embedding Jetty docs
jetty-all different versions downlad
To followup on Gas's answer.
jetty-all doesn't have 100% of Jetty.
It used to, hence the name.
However, today its impossible to have 100% of jetty, as many components can conflict with each other.
If you use maven, or gradle, or ant+ivy, then you'll likely want to depend on:
org.eclipse.jetty:jetty-webapp
org.eclipse.jetty.websocket:javax-websocket-server-impl
let the transitive nature of those build tools pull in the rest.
This would get you "started" easily enough.
There are also plenty of example projects that use embedded jetty.
See:
Embedded Jetty: with JSP enabled
Embedded Jetty: with various WebSocket configurations
Embedded Jetty: using Servlet 3.0 features
Embedded Jetty: using Servlet 3.1 features
Embedded Jetty: various Logging configurations
Some use 100% embedded jetty (without a war file, or WEB-INF, or web.xml), some use a war file built elsewhere.
Jetty uses maven so it can participate in the global central artifact repository, and that we have 2 developers on Jetty that are also developers on Maven.
If you want to manage the dependencies yourself, then you will need to know intimately the purpose and role/purpose/relationship/requirements of every jar file that you are going to add into your project. (and answering that is way out of scope for stackoverflow)
You have many build tool options to make managing the dependencies easier:
Apache Maven
Gradle/Grails
Apache Buildr
Apache Ivy (an add-on for Apache ant)
Groovy Grape
Scala SBT (for working with Scala on top of Java)
Leiningen (for working with Clojure on top of Java)
Maven isn't required, you could use any of the above tools.
Tip: Maven and Gradle are the best integrated in various IDEs (like Eclipse IDE and IntelliJ)

Trying to understand what gradle and maven actually do

What exactly do Gradle and Maven "do" that eclipse or sts doesn't? From what I've read it builds, runs, etc which can all be done in eclipse easily.
If I have an existing project I've created, built, and currently runs in eclipse via tomcat, what would I use gradle for?
There's not a lot of benefit to using Maven or Gradle on a small project that you never share with anyone; an IDE can do the build just as well. But as the number of developers increases and the complexity of the build increases, it becomes very useful to separate the build instructions from the IDE. Let's drill into these a little bit.
With the increase in developers, you don't want everyone to have to come by and use your IDE to get a build done. That would be really annoying! So they're on their own machines, but then they tend to have different setups (how dare they have different user account names!) and probably have their IDE installations set up a little differently too. So we need some kind of build instructions that people can use to get things going, and it helps if everyone can use the same build instructions repeatably so that you don't get too many instances of “but it works on my machine!”. It's also very helpful if those instructions are simple enough to use that a new programmer to the team can get up to speed rapidly.
But the other thing that often happens as projects grow in scope is that their builds become more complicated. They very often gain additional dependencies (they didn't start out needing a high-performance date parser and MIME-type identifier, buit they've become required since and you don't want to have to write all those from scratch) and that means you've got to make sure that when the build is done, the right version of those dependencies is used. But that's not the only way that complexity increases. It's also very often the case that you find you're using more automatically-generated code. You might find yourself working with XML schemas or WSDL a lot, or maybe your using Hibernate, or Spring, or … well, there's lots of ways in which things can get complicated, OK? Getting the various steps to do all the build right, reliably, in these sorts of scenarios can be a bit tricky, but encoding them as instructions to something like Maven makes life a lot easier once you've taken the jump in the first place. (It gets even more important when you start trying to deal with projects which need many different sub-programs that work in concert; some of those are plain hard to build even with Maven or Gradle or any other tool.)
And then there's the possibility of offloading work to a build server, running tests automatically, managing dependencies cleanly, etc. IDEs don't handle these all that well by themselves; where they do a reasonable job of it, it's usually because they're using a tool like Maven under the covers to do the heavy lifting.
tl;dr
You don't have to make your code work with a build system, but it helps if you do and in many ways.
Maven and Gradle can do many things that Eclipse doesn't. However, the most important thing they do, is to decouple the bulding and testing processes from the IDE you choose (i.e. Eclipse). When you work on a large environment, with many programmers, usually you can not control the IDE they use. So, it's better to use a tool like Maven and Gradle to standardize these tasks. The same happens with the code examples of a book: instead of the authors having to provide the instructions for configuring any IDE to execute them, they provide the Maven or Gradle files, so the reader can build and test them on any IDE he's using.
Another very important feature that Maven and Gradle give you, is the fact that dependencies are managed without the need of having the executable code under source version control. Instead of having the executable code you depend on as part of the project, you declare the dependencies on a text file (which is under source version control), and then get them from a repository.
However, you may only see the real advantages of using tools like Maven or Gradle (and even Jenkins or Hudson), when you think in large scale projects, developed along many months by teams composed of many developers).
Gradle and Maven are build tools. Maven was first and is a bit older, Gradle is newer and has redefined a way of how projects are built and maintained. In my opinion it's also much easier to use, more readable and easier to maintain. I prefer Gradle ;)
You use eclipse or STS (any other IDE) for development. And while You finish this process You need to provide a configured artifact (war, ear, whatever...) to production and deploy it there. These artifacts have well defined format and the application won't be run from eclipse or STS at the production environment. It's tiresome and error prone to prepare such artifacts by hand.
Gradle or Maven can take responsbility of building and preparing these artifacts (in fact such tools can do much more) off Your shoulders, they make this process automated.

What is the best way to handle Java web application versioning?

I have a standard Java application that handles both REST and UI calls. What is the best way for me to create and manage an application version (major.minor.release.build)? I'm using Subversion, Maven, Bamboo (continuous build) and Spring in the stack. I would like the version to be tied together with SVN, Bamboo and Maven. And, would like to be able to log version on start-up -- likely using some Spring bean.
There must be a framework/pattern out there to help with this. I'd rather not roll my own.
Thank you!
Why not use Semantic Versioning? It is what most people expect nowadays, it is pretty well defined and it is out there. Good enough for me.
Maven has a release plugin. This is a bear to setup first but once it is working it works well. It does all the nitty gritty of making sure everything is cleanly checked in, tagged properly, and does the magic with the version numbers. It is not a ask to look forward to, but at least now it is properly done. It pays to setup some maven repository. We use Nexus and can recommend that, but I heard good things of artifactory too.
During testing we do not rely on the maven version too much but on the build number, which we put in a discrete place on the web pages and similar artifacts so we can quickly determine which exact build we're talking about. We use hudson which provides the build number in an environment variable, but Bamboo must provide that too. The filter copy functionality makes that pretty straightforward.
Hudson tags the VCS (we use git, but that does not matter) with the build number and the maven release plugin tags the releases.
You can include the SCM revision number in your artifact using the maven build-number plugin (http://mojo.codehaus.org/buildnumber-maven-plugin/), e.g. in a filtered resource, such as a properties file.
If you are using Artifactory as your binary repo then it can also tag your binary artifacts with a build number and have full traceability from your artifact to the CI server build that created it. Currently this is supported with Hudson, TeamCity and Bamboo.

Categories

Resources