In my working place, they asked to me to learn the OSGi framework and to decide what is the best approach to work with it.
In the last two weeks, I surfed the web and I discovered a lot of different approaches to work with OSGi, for example, I found the OSGi enRoute approaches, and an Eclipse plug-in called BndTools. I discovered that I can use simply Declarative Services or framework like AIOLOS.
I'm a little bit confused about all these different approaches and technologies... What do you think is the best approach to get started with OSGi for a beginner? Is there an implementation that is better than the others (for instance Equinox)? Do you have a preferred approach to work with this framework?
Thank you very much in advance!
update To address this question better, I've written a booklet & videos that allows you to get inside OSGi quickly. It is very interactive, using Bndtools and the OSGi Gogo shell. You can find it here.
updated text for readability & current state
I can understand the confusion ... there are a plethora of build tools and they nowadays all support OSGi. Since you sometimes need combinations of tools, the space is complex.
bnd
To use OSGi you need to build bundles. A bundle is a JAR file, the default format of Java libraries/executables. They are bundles when the manifest in the JAR file contains OSGi metadata. This metadata provides information to tooling and the OSGi framework what capabilities it requires from the runtime and what capabilities it provides to the runtime. This information is used to assemble runtimes as well as verify during runtime that things are compatible.
Maintaining this information by hand is a lot of nauseating work. For this reason bnd was developed by me, already about 19 years ago. bnd is currently the primary library in the industry to create this metadata, simplify decorating the metadata, and verifying the validity of the metadata. It does extensive analysis and annotations of the bundle to minimize the manual work.
In the bnd case, it also supports the OSGi standard build annotations for Declarative Services, Manifest annotations, and more. (Many OSGi standards originated in bnd.)
IDE & Continuous Integration
IDEs are the preferred to tools to read, write and debug code. However, without a continuous integration solution that runs on a remote server, you cannot rely on the results since your IDE might depend on the information that you only have on your laptop. For professional development, it is, therefore, a must to have some server that builds your software from scratch without using caches.
Clearly, it is paramount that when you develop software on your laptop the results are identical when you build on the server. For this reason, bnd provides a library that can be used in the IDEs and different build tools. Although there are a myriad of combinations feasible with bnd, there are few most popular ones.
Models
Maven Only
Maven is a popular build tool for Java applications. It defines all build information in POM (Project Object Model) files, which are XML files. POMs can inherit from other POMs. Each POM (and artifact) is identified by a group id, an artifact id, and an opaque version. Maven has a pretty fixed structure for a project. All build work is done via plugins that get their configuration from the POM.
There are two Maven plugins, based on bnd, that provide the necessary OSGi metadata generation.
Apache Felix Maven Plugin
Bnd Maven Plugins
In this model, bnd is only used for providing the metadata in the bundle. All dependencies must be in Maven repositories.
There is a third plugin, Tycho, that builds bundles using the Eclipse PDE model. I hear few people recommending going to PDE/Tycho today, it has not been a painless development and many PDE users are looking at alternatives. This plugin must bridge a very large semantic gap between PDE and Maven.
Gradle Only
Although Gradle is also relying on plugins for all the low-level work it heavily relies on Groovy to provide the build actions.
The bndtools group provides a plugin that makes it trivial to generate the OSGi metadata in the normal Java build:
bnd Gradle plugin
In this model, all dependencies must be stored in repositories accessible by Gradle, which are generally Maven repositories.
Eclipse, M2E, Maven, and Bndtools
In this quartet, Eclipse is the basic IDE, M2E is a plugin that teaches Eclipse how to build the bundles according to a maven specification (pom file). In this quartet, bnd runs inside Maven as a plugin. Bndtools provides some bonus OSGi IDE functionality that M2E is lacking. This is mainly focused on creating assemblies for OSGi runtimes and viewing bundles.
In this model, all build information is stored in Maven POMs. This is the model that BJ Hargrave in another answer to this question posted.
In this model, bnd is only used for providing the metadata in the bundle. All dependencies must be in Maven repositories.
Eclipse, Bndtools, Gradle
The other model, which was developed specifically for OSGi/bnd, is the bnd workspace model. In this model, a workspace is a directory with the OSGi bundle projects, a special directory (cnf) holds workspace wide information. All build information is stored in bnd files which are good old Java property files.
Eclipse/Bndtools and Gradle (and for that matter Ant!) have plugins that read the information in the workspace and project directories and teach their respective build tools where the sources are, where the binaries should be stored, etc. The plugins for these tools that use bnd go out of their way to ensure the build results are identical.
The original archived OSGi enRoute was based on this model. Although it is archived, it is still the primary model for Bndtools and used by many companies. At EclipseCon 2018 there are several presentations about this model:
How OSGi drives cross-sector energy management – Fully based on the v2Archive OSGi enRoute model
How we used OSGi to build open liberty – Build with Bndtools and Gradle
Migrating from PDE to Bndtools in Practice – Migrated recently from PDE (the original Eclipse bundle build model) to Bndtools
This is also the model that OSGi uses itself to build the specification JARs, Reference Implementations, and the Compliance Test suites.
The bnd workspace model is the only model that supports all repository standards. This includes Maven repositories (Maven Central!), OSGi repository standard, Eclipse P2 repositories, directory-based repositories, and even POM based repositories. The support for external repositories in the bnd workspace model is extraordinarily flexible.
Where to Start?
Generally, developers that start in OSGi already have Java experience. This generally drives their choice for a tool since there is already a legacy.
If you can start with a blank slate then my personal preference is the bnd workspace model. It puts the priority on the IDE, which is where you spent most of your time while having an extremely good fidelity with the continuous integration build. In the last 2 years, I've helped two companies, one to start with OSGi from scratch, the other having 8 years experience with PDE. Both are now based on this workspace model and I am quite impressed by how they've been able to reap the benefits of OSGi without any prior experience much faster than I'd ever seen before.
An interactive tutorial, using Bndtools and the OSGi Gogo shell can be found here. With videos!
Start with OSGi enRoute. It will discuss using Bndtools as an IDE. It already uses the Bnd maven plugins to build bundles and demonstrates using Declarative Services to code providing and using services.
Related
I just started developing a project in Spring MVC and i want to know how important Maven is.
The following are the key features of Maven :
Simple project setup that follows best practices - get a new project or module started in seconds
Consistent usage across all projects - means no ramp-up time for new developers coming onto a project
Superior dependency management including automatic updating, dependency closures (also known as transitive dependencies)
Able to easily work with multiple projects at the same time
A large and growing repository of libraries and metadata to use out of the box, and arrangements in place with the largest Open Source projects for real-time availability of their latest releases
6.Extensible, with the ability to easily write plugins in Java or scripting languages
Instant access to new features with little or no extra configuration
Ant tasks for dependency management and deployment outside of Maven
Model based builds: Maven is able to build any number of projects into predefined output types such as a JAR, WAR, or distribution based on metadata about the project, without the need to do any scripting in most cases.
Coherent site of project information: Using the same metadata as for the build process, Maven is able to generate a web site or PDF including any documentation you care to add, and adds to that standard reports about the state of development of the project. Examples of this information can be seen at the bottom of the left-hand navigation of this site under the "Project Information" and "Project Reports" submenus.
Release management and distribution publication: Without much additional configuration, Maven will integrate with your source control system (such as Subversion or Git) and manage the release of a project based on a certain tag. It can also publish this to a distribution location for use by other projects. Maven is able to publish individual outputs such as a JAR, an archive including other dependencies and documentation, or as a source distribution.
Dependency management: Maven encourages the use of a central repository of JARs and other dependencies. Maven comes with a mechanism that your project's clients can use to download any JARs required for building your project from a central JAR repository much like Perl's CPAN. This allows users of Maven to reuse JARs across projects and encourages communication between projects to ensure that backward compatibility issues are dealt with.
Getting started with maven only takes about 10 minutes. Reasons why you should learn maven:
It helps you manage your dependencies very easily so you don't need to add jars to your project classpath manually
You can run unit tests
Has over 20 useful plugins which you can use. Plugins make up lifecycles like test, package which make your work more efficient
You can use it to build your project
The most important thing about it at the beginning is that you don't need to worry about setting up your project by adding dependencies, maven does it for you automatically.
Read this spring guide for building with maven
Any other guides in this section for spring boot has the same mechanism
I am in process to create a maven repository of a JAVA module which is part of Eclipse (probably OSGI). I am trying to get transitive dependencies of the JAR files I need using this way (http://wiki.eclipse.org/JFace).
It use Plugin dependency analyser feature of the Eclipse to create a tree. I am wondering how does it work in background. I can find Imported namespace in manifest file of a JAR. But how does it find the relevent JAR file using that information?
My end goal is to export all these transitive depenedencies JAR and convert them to maven repository. I want to automate it ideally so I don't have to do it manually whenever there is an update.
In general Eclipse PDE projects have a target platform. This target platform holds the dependencies.
In OSGi bundles, the dependencies are specified in the manifest. There are many types of dependencies, the OSGi dependency model is magnitudes more powerful than the simplistic 'require' model of other module systems. Not only has it many types of dependencies (require bundle (the classic one), import package, require execution environment, require an implementation for an API, require a service, etc.) specified in namespaces, it also supports a powerful filter that is asserted on the properties of a capability. A capability is the opposite of a requirement.
To handle these capabilities and requirements, OSGi has a resolver. It takes a set of initial requirements and finds a solution where all requirements are satisfied and all the rules are followed. The result of the resolver is a set of wires that connect bundles together.
By far the best environment to play with this is bndtools. In bndtools you can use a P2 repository directly (Normally the target platform is a P2 repository). (You can also directly use a Maven repository or an OSGi standardized repository aka OBR.) In the bndtools bndrun file you can then select one or more root bundles of your graph. The resolver then calculates the set of bundles that are a happy closure. This is all done on GUI level.
There is also a command line called 'bnd' that you can install via brew or download its jar from https://bndtools.ci.cloudbees.com/job/bnd.master/lastSuccessfulBuild/artifact/biz.aQute.bnd/generated/
That is a miraculous little tool that can do wonders but very badly documented although it does have a help command. :-( (I am the author)
I have multiple java projects in a folder. Also there is a second folder with libraries, that might be used as build dependencies from the projects. The projects may also have dependencies to other Projects. What's the best approach to build all projects ?
In other words I want to build the projects without explicit telling their dependencies.I think the biggest problem is the dependecy between the projects.
There are multiple build systems that are available that you may use. Maven has a complete dependency system built into it. Almost all third party open source jars are directly accessible via the World Wide Maven repository system. Basically, you describe the jar you need (groupId, artifactId, and version) and Maven will automatically fetch it for you. Not only that, but Maven also will build your project without having to create a build file. Instead, you have to describe your project in a project object model (a pom.xml file) and Maven will download everything you need, including all compilers, etc.
Almost all new projects use Maven, but Maven has a few downsides:
Since you don't control a build process, it can sometimes feel like poking a prodding a black box to get the build to work the way you want.
Documentation can be scant -- especially if you're moving beyond basic Java compiles.
You usually have to arrange your project in a specific layout. For example, source files should go under src/main/java while JUnit tests are under src/test/java. You don't have to follow the recommended layout, but then you'd have to modify the pom.xml file this way and that to get your build to work. That defeats the whole purpose of the pom.xml in the first place.
If you already have another build system setup (like Ant), you lose everything. There's no easy way to move from Ant to Maven.
The other is called Ant with Ivy. Ivy uses Ant for building, but can access Maven's world wide repository system for third party dependencies. It's a great compromise if you already are heavily invested in Ant. I also find Ant with Ivy to be better documented than Maven (although that's not too difficult). There's an excellent chapter going over the basics of Ivy in Manning Publication's Ant in Action.
With either process, I would recommend that you build a company wide Maven repository using either Nexus or Artifactory. This way, any proprietary third party jars (like Oracle jars) can also be stored in your company wide Maven repository since they won't be in the standard World Wide Maven repository.
By the way, if this is a company wide effort, and you are moving multiple Ant projects into Ivy, I have an Ivy project I use in Github that makes things easier.
Oh, there's a third possibility called Gradle which I know nothing about. I also believe it can use the World Wide Maven repository. It's based on Groovy which is based on Java syntax, and that's about all I can say. Maybe others can fill you in on the details. The Gradle group contends it solves a lot of problems of both Ant/Ivy and Maven.
Whatever tool you use, if you have various projects interdependent, you need to be clear on the independent ones which will be built first before building the dependent projects. You need to have a clear dependency structure for your projects.
You can do this with Apache Ivy. You can lay out the locations for you common libraries, define published artifacts and inter-dependencies in an ivy.xml document in each project, and let a top-level Ant build with the Ivy tasks figure out what the build order should be based on those dependencies.
Currently when I am writting a bundle in that depends on a package, I have to "import" or "depend" on a whole other bundle in Maven that contains that package.
This seems like it is counter-productive to what OSGi gives me.
For example let's say I have two bundles: BundleAPI and BundleImpl.
BundleAPI provides the API interfaces:
// BundleAPI's manifest
export-package: com.service.api
BundleImpl provides the implementation:
//BundleImpl's manifest
import-package com.service.api
However, when I am coding BundleImpl in Eclipse, I am forced to "depend" in maven POM on BundleAPI itself - so that eclipse does not complain.
//BundleImpl's POM
<dependency>
<groupId>com.service</groupId>
<artifactId>com.service.api</artifactId>
[...]
</dependency>
So - on one hand, I am depending only on the package com.service.api, while on the other - I need to have the whole bundle - BundleAPI.
Is there a way to make maven or eclipse smart enough to just find the packages somewhere, instead of whole bundles?
I am very much confused as to how this works - any type of clarity here would be great. Maybe I am missing something fundamentally simple?
The key is to distinguish between build-time dependencies and runtime dependencies.
At build time you have to depend on a whole artifact, i.e. a JAR file or bundle. That's pretty much unavoidable because of the way Java compilers work. However at runtime you depend only on the packages you use in your bundle, and this is how OSGi manages runtime substitution. This is the Import-Package statement in your final bundle.
Of course as a developer you don't want to list two parallel sets of dependencies, that would be crazy. Fortunately maven-bundle-plugin is based on a tool called bnd that calculates the Import-Package statement for you based on analysing your code and discovering the actual packages used. Other tools such as bndtools (an Eclipse-based IDE for OSGi development) also use bnd in this way. Incidentally bnd is much more reliable and accurate than any human at doing this job!
So, you define only the module-level dependencies that you need at build time, and the tool generates the runtime package-level dependencies.
I would recommend against using Tycho because it forces you to use Eclipse PDE, which in turn forces you to manually manage imported packages (for the sake of full disclosure, I am the author of bndtools which competes against PDE).
You cannot develop bundles like regular Java projects with Maven and eclipse. You basically have 2 options.
Apache Felix Bundle Plugin: Basically you develop the project as a regular Java project and use Maven as you normally would. This plugin will be used to add all the OSGi specifics to the jar manifest at deployment time to OSGi enable it. The disadvantage of this aproach is that you are using a Java project in your workspace instead of a bundle, which makes running your project in the OSGi container a little extra work since Eclipse doesn't recognize it as a plugin project. Thus you have to add the jar from the Maven build as part of the target platform manually.
Tycho: This is another Maven plugin that attempts to actually bring theses two environments together and does a pretty good job of it. In this scenario, you actually create an Eclipse bundle/plugin project, which obviously makes for seamless integration in Eclipse. The pom then marks the project as being an eclipse-plugin type, which effectively makes Maven resolve the project dependencies (defined in the manifest) via the target platform instead of Maven itself.
I would take the Tycho approach as it gives a much more integrated approach with Eclipse.
Having the whole jar as a dependency shouldn't be a problem, that's how you have to do it with Maven anyway.
I have seen many ivy files in my application's build projects. What is Ivy, and its relation with ant?
Ivy is an Ant extension for managing external libraries during the development process, giving you a way of adding them to your classpath, bundling them into your application build, etc.
There's some overlap with what Maven does, but Ivy is much more lightweight (and doesn't do as much).
Ivy is a dependency manager -- it manages and controls the JAR files that your project depends on. If you don't have the JARs, it will pull them down for you by default (from the Maven 2 repository), which can make project setup a lot easier.
Ivy is originally a Jayasoft.fr product which at that time was under BSD license, however the core of Ivy has always been the same: It's a dependency management tool.
Ivy's relation to Ant is that it is an Ant extension in the sense that it is through Ant, however it doesn't really depend on it (beyond the obvious parts). Ivy was merged into an Apache project under Ant as subproject through incubation process on October 11, 2007.
Very common misconception people have is that they compare Ivy with Maven as a whole. However that doesn't really work since Ivy only handles dependencies in an agile manner and is very good at it with simplistic configuration and wide automated support with various build systems etc. through Ant while Maven really is a build system in itself. Personal opinions may and do differ in which one does what best and that is worth several Stackoverflow/Google searches on its own.
I believe these are significant and useful points to add:
Ivy can be used standalone from command line without Ant. If your CI environment does not include Ant it will not preclude you from using Ivy. Yes, integration with Ant is nice and, by far, better documented. One could apply Ant integration documentation to standalone usage though.
Ivy is not just for Java. My team successfully uses it to define dependencies for a fairly large .NET project (10 interdependent packages with several versions of each in production with multiple third party upstream dependencies). Works quite well. I wish documentation were a bit clearer and fuller.
There are additional perks that Ivy offers for Build Time: upstream and downstream builds triggering.