Develop Reference

Trouble getting 3rd party jar as OSGI bundle in Karaf

I have a 3rd party jar, no source, that is used to connect to a proprietary database system. I believe the jar is actually an Eclipse plugin; it was taken from one of the lib folders of the Eclipse-based editor for that system.
Anyway, I'm trying to develop a bundle that exposes the functionality of this jar, using Netbeans 7.4 and Karaf 3.0.1. Just trying to install the jar into Karaf isn't working - if I drop it into the deploy folder it doesn't even appear on the list, like it doesn't recognize it as an OSGI bundle. I created a new OSGI Maven bundle in Netbeans, added the jar as a dependency, and have tried all manner of shading, including in Bundle-ClassPath, Export-Package, with no luck. At one point I thought I had it working but Karaf was complaining about needing some Eclipse requirements.
I did more digging, unzipped the jar and the manifest for the 3rd party jar has this:
Require-Bundle: org.eclipse.core.runtime
So then I started down the rabbit hole of fulfilling those dependencies. Based on the maven pages I put together this features.xml file:
<features xmlns="http://karaf.apache.org/xmlns/features/v1.0.0">
<feature name='custom_deps' version='1.0'>
<bundle>mvn:org.eclipse.equinox/log/1.0.100-v20070226</bundle>
<bundle>mvn:org.eclipse/osgi/3.5.0.v20090520</bundle>
<bundle>mvn:org.eclipse.core/contenttype/3.2.100-v20070319</bundle>
<bundle>mvn:org.eclipse.core/jobs/3.3.0-v20070423</bundle>
<bundle>mvn:org.eclipse.core.runtime.compatibility/auth/3.2.100-v20070502</bundle>
<bundle>mvn:org.eclipse.equinox/app/1.3.100-v20130327-1442</bundle>
<bundle>mvn:org.eclipse.equinox/common/3.6.200-v20130402-1505</bundle>
<bundle>mvn:org.eclipse.equinox/preferences/3.5.100-v20130422-1538</bundle>
<bundle>mvn:org.eclipse.equinox/registry/3.5.301-v20130717-1549</bundle>
<bundle>mvn:org.eclipse.core/runtime/3.3.100-v20070530</bundle>
</feature>
</features>
Now when I try to install that I get:
karaf#root()> feature:install custom_deps
Error executing command: Could not start bundle mvn:org.eclipse/osgi/3.5.0.v2009
0520 in feature(s) custom_deps-1.0: Activator start error in bundle org.eclips
e.osgi [256].
which I think might have something to do with equinox vs felix? I'm just using the default Karaf setup and creating bundles using the built-in Netbeans Maven project and am still relatively new to the OSGI world. For the most part I can get bundles up and running and have put together a basic collection for an app, but this one bundle depends on being able to use this library and I'm struggling to find a way to get it working. I'm wondering if there's a way to either resolve all the linked dependencies of the jar, or somehow strip out whatever is depending on eclipse? I'm wondering if it's just plugin-related parts that aren't necessary for the connectivity I'm looking for.
Thanks for any ideas.
Edit: adding more info about the 3rd party jar
The jar is vendor-provided, but from a now defunct vendor. I'm not sure whether I can give vendor details or not so I'll mask the specific company info here. So the jar is named "com.bigcompany.product.productbeans_4.3.1.jar". The full manifest of the jar looks like this:
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: Productbeans Plug-in
Bundle-SymbolicName: com.bigcompany.product.productbeans
Bundle-Version: 4.3.1
Bundle-Activator: com.bigcompany.product.productbeans.ProductbeansPlugin
Bundle-Vendor: BIGCOMPANY
Bundle-Localization: plugin
Require-Bundle: org.eclipse.core.runtime
Eclipse-AutoStart: true
Export-Package: com.bigcompany.product.productbeans
The switch to Equinox in Karaf was relatively painless, but dropping the jar into Karaf still it still didn't appear in the list. I tried unzipping the jar, modifying the manifest to remove the Require-Bundle and Eclipse-Autostart lines, repacked it up and tried installing that - at that point it showed up but my bundles that want this library still complain about missing requirements 'com.bigcompany.product.productbeans'.
From there I'm tried redeploying my bundle that tries to import the 3rd party package, and when I try installing my features file above that complains about missing constraints - is there a required order of the bundles in the features file?
I think ultimately what I'm hoping is is there an authoritative way to just embed or include a jar (or it's contained .class files?) into a bundle that requires them, and to have it just work? I've tried Embed-Dependency, Bundle-Classpath (although I'm not sure if I have the syntax or locations correct?), using the maven shade plugin, and either I get to where my bundle says it's missing the 3rd party package, or if it doesn't give that it says it can't find the eclipse dependencies. If I have a non-OSGI project, the maven dependencies don't list or include any sort of eclipse-related dependencies. I can include snippets of my POM or something else if needed.

Basically the Require-Bundle says that it needs the equinox bundle. Karaf by standard uses Apache felix as OSGi runtime. As equinox is also a runtime you can not simply install it inside felix.
Instead you simply switch your karaf to use equinox instead of felix.
Edit etc/config.properties and set
karaf.framework=equinox
Then try to deploy your bundle again (without additional dependencies first).

How to manage dependencies when developing OSGI bundle in Netbeans and Karaf?

I'm struggling to wrap my head around developing OSGI bundles to publish to Karaf. I'm using Netbeans 7.4 and the Create Maven OSGI bundle when creating a new project. I think what I'm wrestling with is dealing with dependencies between different projects and working with them in Netbeans and then publishing to Karaf.
What I'm trying to do is set up a REST web service; I found this tutorial to be a good starting point in getting something basic up and running. So I started setting up a parent project with sub projects (all of these are from the Maven OSGI Bundle template). But after I started deploying bundles to my Karaf instance I started wrestling with the "missing requirement" errors within Karaf. I'd start trying to list each dependency in <Export-Package> or <Embed-Dependency> but it seems like I just keep going farther down and always getting new "missing requirement" messages listing yet another dependency. I then started trying the maven copy-dependencies plugin to export whatever dependencies are used in my project and copying whatever comes out into the Karaf deploy folder, too. That worked for the most part but I'm wrestling with what I think are downstream dependencies of a 3rd party jar, but that's probably a separate question anyway.
So, what I'd like to know is how should I be developing a coherent "application" with this setup? I understand I should separate everything into standalone bundles; i.e., I'll have my REST interface as one, the implementation as another, business logic as another. As a maven project, if I have a dependency for datetime handling or string utilities, maven will handle resolving all of those. But these dependencies won't automatically be included in my OSGI bundle, is that the case? What is the workflow to make sure that everything my project depends on is available to the bundles in Karaf? What about my projects that have other projects as dependencies, i.e., an interface implementation? I would include that project as a dependency so it will compile, but how do I make sure that that dependency is met after it is published?

The mismatch between dependencies at compile time and at deployment time is indeed a problem when working with OSGi.
What you need to know is how these relate to each other. At compile time you have the maven dependencies. When you build your project the maven bundle plugin creates the necessary Import-Package and Export-Package statements for you. Most of the time this simply works and you should avoid tuning too much there by hand.
So after the build the Manifest specifies what the resulting bundle needs but this does not automatically make sure the dependencies are met. So when you deploy the bundle to karaf you get the errors about missing dependencies. This is normal.
Now you need to install other bundles that fulfil these dependencies. Often you can simply install the jars of your maven dependencies (if they are bundles). This does not always work though. So basically you install the bundles and check if the requirements are met now.
The nice thing with karaf is that many bigger dependencies like cxf are already available as features. So it is a good idea to first try to get the dependencies by installing available features before you try to install them bundle by bundle.
So this helps you in getting your dependencies right. Then for "production" use the best solution is to create your own feature file where you refer to the bundles you found.
While installing bundles using the deploy dir seems nice at first it is not a good solution. Better install the bundles directly from maven using the mvn: url syntax karaf provides.
Regarding embedding dependencies. Sometimes it is a good solution but often it only makes things worse as it can lead to package use conflicts which are hard to solve. So better do not embed anything if it is possible.

Dependency management with OSGi

I want to develop a multi-module application according to OSGi specification. Let's assume that one of my modules uses Apache Commons Logging 1.1.1. Spring provides a bundled version of Apache Commons Logging 1.1.1 in their repository, so I can add the corresponding dependency in my POM.
If I install my bundle in Apache Felix for example, is it right that the dependency to Apache Commons Logging 1.1.1 will not be resolved until I install the bundle of Apache Commons Logging 1.1.1 as well? My bundle will try to import a package that hasn't been exported.
I don't really understand how dependency management works in the OSGi world. Should I install every bundle that my application needs? Also, I don't understand how it integrates with Maven?
Thanks in advance for your explanations
EDIT: I've seen there is a subproject of Apache Felix called OBR that seems to facilitate bundles management (e.g. deployment). But, we already have a Maven architecture with local repositories, private repositories... How is OBR integrated to Maven?

If you want to use OSGi together with maven then I recommend Apache Karaf as a server which can use the Felix OSGi framework. The advantage is that you can install bundles directly from maven repositories using mvn: urls.
If you just use Apache Karaf with a maven repo then you have no transitive resolution at runtime. You have to install all bundles you need. What helps a lot is that karaf has the concept of features. So you can use the features as coarse grained building blocks. You can also create your own features where you refer to other features and bundles. This allows to install your whole app with one command.
Felix and Karaf also support OBR but you would have to create your own OBR Repo. There are currently no public OBR repos. The advantage of OBR is that it can resolve most of the transitive dependencies. The Karaf features even work together with OBR so you can just list some top level bundles in the feature and let OBR resolve the rest.
In practice I have good experiences with simple Karaf features without OBR. It is some manual work but less than you would expect and works quite well.

OSGi dependencies are based on Java packages: when a bundle states that it needs to import a given package (indicating a range of versions that are acceptable), the framework will attempt to "wire" that import to a suitable version of that package, that must be exported by another bundle.
If the package cannot be found, the bundle won't be resolved and cannot be started - so yes you'll need to install all bundles that your application requires, and as Christian indicates there are various tools that can help you with that.
A simple way to get the required bundles is to use the maven-dependencies-plugin to grab the bundles from Maven's list of dependencies and copy them somewhere where your app can find them at startup to install them, as done in [2] (in launcher/pom.xml), a small example app that I wrote.
You could also use Sling's maven-launchpad-plugin [3] to generate a runnable jar file that embeds the OSGi framework and all bundles that you need, and sets up everything at startup.
About the import and exports - without going into details, assuming you use the maven-bundle-plugin [1] to build your bundles, you'll specify what packages you want your bundle to export (others will be invisible to other bundles), and the maven-bundle-plugin will generate (in most cases automatically, but you can override that as needed) the list of packages to import, so there's usually not much work if your own code is cleanly split between packages that you want to export and internal implementation packages.
[1] http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html
[2] https://github.com/bdelacretaz/OSGi-for-mere-mortals
[3] http://sling.apache.org/site/maven-launchpad-plugin.html

automated osgi bundle development with netbeans 7.1

I'm currently developing a maven osgi bundle using Netbeans 7.1. While its easy to create new maven osgi bundle project from netbeans, I've been struggling about how I would run it. Simply running it from netbeans gives me an error that tells me there is missing requirement (missing slf4j for example).
Then I tried a different way. I ran equinox in a terminal and then manually install my project jar to it. But then I must also resolve all the dependencies manually.
Is there any way to automatically download all the required dependencies from an OSGI bundle and install it to a running OSGI framework?
thanx before

AFAIK there's no automation for OSGi in Netbeans (though I may well be wrong). The support you may have seen in Eclipse is only for Eclipse plugins not vanilla OSGi.
Your best bet is to go for some integrating testing or launch via a maven goal, pax-exam or bndtools or even pax-runner will allow you to launch from IDE/maven.
However I don't know of anything that will automatically resolve all dependencies (transitive dependencies and implementations of APIs you depend on would be problematic)

Its a big pain, no question about it. What I used to do was to do a full build (and if you've configured your manifests correctly should include all necessary dependencies) which will generate the necessary jar. I then wired my Tomcat to pick up the jar from my target repository and configured it to hotswap automatically.
Its a matter of preference if you want to run your app server from within your netbeans but I preferred to execute a separate instance of tomcat outside of my IDE. It'll work either way tho.

Avoid duplicating OSGi imports in maven dependancies?

Currently when I am writting a bundle in that depends on a package, I have to "import" or "depend" on a whole other bundle in Maven that contains that package.
This seems like it is counter-productive to what OSGi gives me.
For example let's say I have two bundles: BundleAPI and BundleImpl.
BundleAPI provides the API interfaces:
// BundleAPI's manifest
export-package: com.service.api
BundleImpl provides the implementation:
//BundleImpl's manifest
import-package com.service.api
However, when I am coding BundleImpl in Eclipse, I am forced to "depend" in maven POM on BundleAPI itself - so that eclipse does not complain.
//BundleImpl's POM
<dependency>
<groupId>com.service</groupId>
<artifactId>com.service.api</artifactId>
[...]
</dependency>
So - on one hand, I am depending only on the package com.service.api, while on the other - I need to have the whole bundle - BundleAPI.
Is there a way to make maven or eclipse smart enough to just find the packages somewhere, instead of whole bundles?
I am very much confused as to how this works - any type of clarity here would be great. Maybe I am missing something fundamentally simple?

The key is to distinguish between build-time dependencies and runtime dependencies.
At build time you have to depend on a whole artifact, i.e. a JAR file or bundle. That's pretty much unavoidable because of the way Java compilers work. However at runtime you depend only on the packages you use in your bundle, and this is how OSGi manages runtime substitution. This is the Import-Package statement in your final bundle.
Of course as a developer you don't want to list two parallel sets of dependencies, that would be crazy. Fortunately maven-bundle-plugin is based on a tool called bnd that calculates the Import-Package statement for you based on analysing your code and discovering the actual packages used. Other tools such as bndtools (an Eclipse-based IDE for OSGi development) also use bnd in this way. Incidentally bnd is much more reliable and accurate than any human at doing this job!
So, you define only the module-level dependencies that you need at build time, and the tool generates the runtime package-level dependencies.
I would recommend against using Tycho because it forces you to use Eclipse PDE, which in turn forces you to manually manage imported packages (for the sake of full disclosure, I am the author of bndtools which competes against PDE).

You cannot develop bundles like regular Java projects with Maven and eclipse. You basically have 2 options.
Apache Felix Bundle Plugin: Basically you develop the project as a regular Java project and use Maven as you normally would. This plugin will be used to add all the OSGi specifics to the jar manifest at deployment time to OSGi enable it. The disadvantage of this aproach is that you are using a Java project in your workspace instead of a bundle, which makes running your project in the OSGi container a little extra work since Eclipse doesn't recognize it as a plugin project. Thus you have to add the jar from the Maven build as part of the target platform manually.
Tycho: This is another Maven plugin that attempts to actually bring theses two environments together and does a pretty good job of it. In this scenario, you actually create an Eclipse bundle/plugin project, which obviously makes for seamless integration in Eclipse. The pom then marks the project as being an eclipse-plugin type, which effectively makes Maven resolve the project dependencies (defined in the manifest) via the target platform instead of Maven itself.
I would take the Tycho approach as it gives a much more integrated approach with Eclipse.

Having the whole jar as a dependency shouldn't be a problem, that's how you have to do it with Maven anyway.