Continuous deployment of OSGi-based application on jenkins

Question

After spending a couple of hours trying to understand how to make continuous deployment work in the case of an OSGi-based application, I am finally posing my first question on stackoverflow, hoping for some indications on what I might have done wrong or missed - somehow I feel being on the wrong track...

This is what I want to achieve:

1. build some bundles and install them to the maven repository (no problem here, using bnd)

2. now having all the bundles making up my application (passing all tests and so on), I want to deploy and run the application, that is, start some OSGi framework using those bundles.

3. Starting is not the issue - "mvn pax:provision -Dframework=equinox" does the trick. My application starts jetty, so it is easy to verify via a browser to see if things look ok (additionally to all the tests)

4. But, now, trying being "continuous", the next time I want to apply this procedure, I really should make sure to shut down the running instance of my application first (releasing at least the port being used). So, to rerun everything, I somehow have to shut down the old installation first.

And this is where my question starts: Is there anything helping me with this? I understand there is the maven-deploy-plugin, but this only seems to be useful when deploying some WAR/EAR file into some standard application container (without the need of restarting it seems).

I really only need to run some script to start up the OSGi enviroment - and then, next time, to shut it down gracefully before I start it again. With tomcat, jetty, jboss and the like, there are some shutdown.sh scripts or mvn jetty:stop instructions, but do I really have to write those kind of scripts myself? This is where I think I am starting to be on the wrong track, somebody must have had those issues before me and solved them I guess ;)

I understood that I could somehow try to use JMX or use the telnet console to access the running instance to issue an "stop 0" command but this feels wrong.

Processes started from jenkins should compile/build/deploy projects, but not start long-running threads I guess, so I somehow have to start some process "outside" which I want to kill first the next time I try it again.

Any ideas? Maybe I am missing something? Thanks in advance for any input on this!

Answer 1

The telnet way seems to be the cleanest in my opinion.

However, if you want to be creative you can create a simple shutdown bundle that you install before you get ready to redeploy. Make sure you have auto-deploy on so the bundle is activated when installed. When this bundle activates it's job is to cleanly shutdown the current running Equinox container.

I would still suggest the telnet approach as you need to make sure your container is shutdown before you attempt to redeploy.

If you don't like any of these approaches take a look at Apache Karaf. You can send a running container commands. You can even stop, uninstall then reinstall all your bundles without ever stopping Karaf.

Karaf can run on top of Apache Felix or Eclipse Equinox.

Answer 2

Maybe I'm missing something, but why do you want to shutdown the whole OSGi framework when you want to redeploy/update your application? The whole point of OSGi is that you can update bundles without having to restart the whole system (remember the loved "You have to restart Windows for the changes to take effect" stuff?). What's more, restarting the whole framework will hide any errors regarding releasing resources in the stop method of your bundle, so I definitely recommend to test by updating only the bundle (and checking the logs and the resource consumption after several start/stop cycles too!)

If I was doing it, I would start an OSGi framework only once completely independently from maven, and then use one of many possible methods for deploying and updating the bundle in it without restarting the framework.

For example: * you could configure the system so that the updated version of the bundle is always put in a given location on the disk, and then use the OSGi console/telnet/whatever admin tool comes with the OSGi framework you have chosen to call "update ".

• or you can use some tool which can connect to the running framework instance and update the bundle. For example for Eclipse there is a plugin from ProSyst which does exactly that.

• or you can use some real remote management software, which could help you test the remote deployment on several targets at the same time and also monitor the deployment status directly. One such system is for example mPower Remote Manager

Regarding releasing the port - if you have problems with this, the reason is most probably the bundle itself, and restarting the OSGi framework won't solve it, because in a real-life scenario the fw is not supposed to be restarted upon update of a single bundle. Check whether you are stopping all threads and releasing all resources correctly in the stop() method of the BundleActivator of your bundle.

Answer 3

After some new insights and checking out quite a few approaches, I want to document what I found working best for my own initial question regarding continuous deployment of a OSGi-based application.

The main problem here was not to be able to update bundles in a running OSGi application just by dropping them in some directory (that could be done with org.apache.felix.fileinstall for example), but to be able to continuously deploy the current state of the software right from the SCM - automatically, via jenkins, exactly as it would look like when being deployed the very first time.

The solution for me was rather simple, really: There is a daemon version of pax-runner, which I can utilize as follows:

In jenkins project configuration, Post-steps, execute shell I put something like:

export BUILD_ID=dontKillMe
myproject/etc/scripts/postDeploy.sh &

with a postDeploy script looking like

#!/bin/bash

cd <myproject>/workspace/skysail.server.ext.osgimonitor/target
cp project.zip /somepath/pax-runner-1.8.5/

cd /somepath/pax-runner-1.8.5/

unzip project.zip

cd project
./rund.sh

with the final rund.sh script similar to

java $JAVA_OPTS -cp .:bin/pax-runner-1.8.5.jar org.ops4j.pax.runner.daemon.DaemonLauncher \
--startd --clean scan-composite:file:rund.composite \
--log=DEBUG

rund.composite is just a file referencing some provisioning files like this

scan-file:bundledefs/core.setup
scan-file:bundledefs/logging.setup
scan-file:bundledefs/restlet.setup

And, finally as an example for those files:

mvn:commons-lang/commons-lang/2.6
mvn:org.codehaus.jackson/jackson-core-lgpl/1.9.5
mvn:org.codehaus.jackson/jackson-mapper-lgpl/1.9.5
mvn:javax.xml.stream/com.springsource.javax.xml.stream/1.0.1
mvn:org.xmlpull/com.springsource.org.xmlpull/1.1.4.c
mvn:com.thoughtworks.xstream/com.springsource.com.thoughtworks.xstream/1.3.1
mvn:org.codehaus.jettison/com.springsource.org.codehaus.jettison/1.0.1
...

With this setup, whenever the project is built by jenkins, the post deploy script is triggered and the application is restarted in a clean, inital state.

Answer 4

In bndtools this is all automated ... Once you save a source file, it builds the jar and tells the framework which bundles to update or install when they are new. Try it, amazing short edit-compile-build-run-debug cycle of about 3 secs.