Tuesday, July 28, 2009

Start Over: JSR 223 JRuby engine on OSGi container

I got a comment from Neil Bartlett about my previous post. Yes, it was hard for OSGi people to understand what's wrong with it. Originally, my blog entry was to answer the question, "how can I create an osgi bundle using maven which uses jruby-engine to execute a jruby script?" So, I pasted entire pom.xml on it. As Neil commented, I should have pasted MANIFEST.MFs that all bundles used. This is hopefully for OSGi people to figure out culprits and help me out.


  • What I want to do is ...

    I'm a committer of JSR 223 JRuby engine. I want to provide a painless OSGi bundle of JSR 223 JRuby engine to users. JRuby engine works on top of JRuby, consequently, users need at least three bundles, JRuby, JRuby engine, and their own bundle, to get it work on OSGi containers. As far as I tested, current MANIFEST.MF of JRuby engine or JRuby, or both might have a flaw, but not sure. I want to fix JRuby engine's flaw if it exists as well as JRuby's.

  • Initial Problems were ...

    There were two basic problems. The first one was JSR 223's discovery mechanism didn't work on OSGi. The mechanism is officially introduced in JDK 1.6, but work on JDK 1.5, too. The mechanism works like this:
    1. looks for META-INF/services/javax.script.ScriptEngineFactory file in every jar file found from classpath.
    2. instantiate JSR 223 engine class specified in the javax.script.ScriptEngineFactory file.

    Probably, because of a classloading issue, this mechanism doesn't work on Apache Felix. However, we can avoid this problem by instatiating a JRuby engine factory directly bypassing discovery mechanism.

    The second problem is the one I'm seeking the best solution. While instantiating JRuby engine factory, com.sun.script.jruby.JRubyScriptEngineFactory (line 13 in the snippet of Take One), JRuby engine, com.sun.script.jruby.JRubyScriptEngine, is also instatiated. These two are in the same, JRuby engine's bundle. While instantiating JRuby engine, Ruby runtime is instantiated, too. Ruby runtime is in a different, JRuby's bundle. Up to here, no problem exists. At the same time, JRuby engine tries to load the instance of org.jruby.javasupport.Java on to Ruby runtime using JRuby's custom classloader. The class, org.jruby.javasupport.Java is in JRuby's bundle. This ends up in raising exception.
    org.jruby.exceptions.RaiseException: library `java' could not be loaded: java.lang.ClassNotFoundException: org.jruby.javasupport.Java
    I don't think I have a choice to use another classloader to load org.jruby.javasupport.Java since it is JRubish way to use Java classes in Ruby scripts.

    JRuby's MANIFEST.MF used for this sample code is here. (This is so long to paste.)

    JRuby engine's MANIFEST.MF

    Manifest-Version: 1.0
    Built-By: yoko
    Created-By: Apache Maven Bundle Plugin
    Import-Package: com.sun.script.jruby,javax.script,org.jruby,org.jruby.
    exceptions,org.jruby.internal.runtime,org.jruby.javasupport,org.jruby
    .runtime,org.jruby.runtime.builtin,org.jruby.runtime.load,org.jruby.u
    til,org.jruby.util.io
    Bnd-LastModified: 1247081259404
    Export-Package: com.sun.script.jruby;uses:="javax.script,org.jruby.run
    time.builtin,org.jruby.runtime,org.jruby,org.jruby.internal.runtime,o
    rg.jruby.exceptions,org.jruby.javasupport,org.jruby.util,org.jruby.ru
    ntime.load,org.jruby.util.io"
    Bundle-Version: 1.0
    Bundle-Name: JRuby JSR223 Engine
    Build-Jdk: 1.5.0_19
    Private-Package: com.sun.script.jruby,
    Bundle-ManifestVersion: 2
    Bundle-SymbolicName: com.sun.script.jruby
    Tool: Bnd-0.0.311

    And the MANIFEST.MF of the snippet:

    Manifest-Version: 1.0
    Built-By: yoko
    Created-By: Apache Maven Bundle Plugin
    Bundle-Activator: hickory.example.Activator
    Import-Package: com.sun.script.jruby,hickory.example,javax.script,org.
    osgi.framework;version="1.4"
    Bnd-LastModified: 1248816762637
    Export-Package: hickory.example;uses:="javax.script,com.sun.script.jru
    by,org.osgi.framework"
    Bundle-Version: 1.0.0.SNAPSHOT
    Bundle-Name: Hickory
    Build-Jdk: 1.5.0_19
    Private-Package: .
    Bundle-ManifestVersion: 2
    Bundle-SymbolicName: hickory.example.Hickory
    Tool: Bnd-0.0.311


  • The Workaound is ...

    Hasan found the workaround of the problem (see Using JRuby in OSGi).
    Using Hasan's workaround, I wrote the second snippet.

    MANIFEST.MFs of JRuby and JRuby engine are the same as the first try. The differences of the MANIFEST.MF of the second snippet are just Bundle-Activator and Bnd-LastModified lines.

    Manifest-Version: 1.0
    Built-By: yoko
    Created-By: Apache Maven Bundle Plugin
    Bundle-Activator: hickory.example.Activator1
    Import-Package: com.sun.script.jruby,hickory.example,javax.script,org.
    osgi.framework;version="1.4"
    Bnd-LastModified: 1248818750858
    Export-Package: hickory.example;uses:="javax.script,com.sun.script.jru
    by,org.osgi.framework"
    Bundle-Version: 1.0.0.SNAPSHOT
    Bundle-Name: Hickory
    Build-Jdk: 1.5.0_19
    Private-Package: .
    Bundle-ManifestVersion: 2
    Bundle-SymbolicName: hickory.example.Hickory
    Tool: Bnd-0.0.311

    This worked well although I'm not sure this is the best. Then, another problem came.

  • Further problem is ...

    JRuby users use Java classes in thier Ruby scripts very often. Thoses classes are usually in differenct jar archives or in classpath that JRuby knows. Here's a further problem happened.

    The third snippet raised an exception when I was to instantiate my Java class in Ruby script.
    org.jruby.exceptions.RaiseException: cannot load Java class hickory.example.YellOut
    In the program, Ruby script, "include Java\nputs Java::hickory.example.YellOut.new.whats," is passed to Ruby runtime to be evaluated. Typically, JRuby finds hickory.example.YellOut class out from classpath and instantiates it using JRuby's classloader. But, this process failed on the OSGi container.

    Again, the only differences in MANIFEST.MF used here are just Bundle-Activator and Bnd-LastModified lines.

    Manifest-Version: 1.0
    Built-By: yoko
    Created-By: Apache Maven Bundle Plugin
    Bundle-Activator: hickory.example.Activator2
    Import-Package: com.sun.script.jruby,hickory.example,javax.script,org.
    osgi.framework;version="1.4"
    Bnd-LastModified: 1248820140956
    Export-Package: hickory.example;uses:="javax.script,com.sun.script.jru
    by,org.osgi.framework"
    Bundle-Version: 1.0.0.SNAPSHOT
    Bundle-Name: Hickory
    Build-Jdk: 1.5.0_19
    Private-Package: .
    Bundle-ManifestVersion: 2
    Bundle-SymbolicName: hickory.example.Hickory
    Tool: Bnd-0.0.311


  • One more workaround might be ...

    I tried the failed example after adding "DynamicImport-Package: *" to JRuby bundle. Now, JRuby's new MANIFEST.MF had "DynamicImport-Package: *" and the third snippet worked.

    However, Tommy strongly opposed to add "DynamicImport-Package: *" to JRuby's bundle, and added the comment to http://jira.codehaus.org/browse/JRUBY-3792.

    According to Neil, Tommy's workaround works only on SpringSource's dm Server. Then, what is the best way to get these code work on other OSGi containers, for example on Apache Felix?

  • If JSR 223 engine has a flaw in its MANIFEST.MF, I'll fix it to provide a painless API.
    I wrote this entry because I couldn't get any relevant information by googling.

4 comments:

Unknown said...

I'm also curious about generic OSGI way, until then Equinox's buddy policy is working for me:

JRuby's bundle:
--------------
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: JRuby Plug-in
Bundle-SymbolicName: org.jruby
Bundle-Version: 1.2.0
Bundle-Vendor: NG
Bundle-ClassPath: jruby-complete_1.2.0.jar
Bundle-Localization: plugin
Export-Package: .,
...snip...
xsd.xmlparser
Eclipse-BuddyPolicy: registered


JRubyEngine's bundle:
------------------------
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: com.sun.script.jruby Library Plug-in 1.1.6
Bundle-SymbolicName: com.sun.script.jruby
Bundle-Version: 1.1.6
Bundle-Vendor: NG
Bundle-ClassPath: jruby-engine_1.1.6.jar
Bundle-Localization: plugin
Require-Bundle: com.sun.script,
org.jruby
Export-Package: .,
com.sun.script.jruby
Eclipse-RegisterBuddy: com.sun.script

JSR223's bundle (I'm using Java 5..)
----------------------
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: com.sun.script Library Plug-in 1.0
Bundle-SymbolicName: com.sun.script
Bundle-Version: 1.0
Bundle-Vendor: NG
Bundle-ClassPath: script-api_1.0.jar
Bundle-Localization: plugin
Export-Package: .,
javax.script
Eclipse-BuddyPolicy: registered

yokolet said...

Thanks for pasting theses. I'll try my snippet on Equinox.

Unknown said...

Oh, I forgot to add the MANIFEST for the bundle that's running its scripts with the help of the above:


Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: My Script Library
Bundle-SymbolicName: my.scriptlib; singleton:=true
Bundle-Version: 1.0
Bundle-Vendor: NG
Require-Bundle: org.jruby,
com.sun.script,
com.sun.script.jruby,
...
Export-Package: my.scriptlib
Eclipse-RegisterBuddy: org.jruby
Eclipse-BuddyPolicy: registered
Bundle-ClassPath: .,
script/
Eclipse-LazyStart: true
Bundle-Activator: my.scriptlib.ScriptingLibPlugin

Chris said...

I'm interested in discovering your final solution to this problem as I am having the exact same issue with my own JSR-223 scripting language.

Will the Equinox buddy loader work on Felix now or has it entered the standard?

Chris