RedBridge, what are new and improved in JRuby 1.5.0RC1

As you may know, Tom Enebo announced the release of JRuby 1.5.0RC1 on Apr. 15 saying "aged like a fine wine." @headius tweated "Over 1250 commits for JRuby 1.5, our largest amount of work ever for any individual release." Also, RedBridge is. RedBridge has been improved since last release based on user inputs. It's API had many changes to become more useful and organized API. Although I've already written about all changes in this blog, I'm going to put them together here for convenience.

New and Deprected Configuration API
RedBridge in JRuby 1.5.0 has a lot of Ruby runtime configuration methods. Before, those were available through getProvider().getRubyInstanceConfig() method, however, this was not a good idea. Since the method exposes JRuby's internal API, users' code might be affected by internal API changes. This fact is against to the purpose of RedBridge. RedBrdige should cover JRuby's internal API and absorb internal changes so that users don't need to fix their code by themselves. Avoid using getProvider().getRubyInstanceConfig() method as much as possible. If you want more runtime configuration methods, please request us.

New runtime configuration methods of ScriptingContainer:

• get/setInput
  
• get/setOutput
  
• get/setError
  
• get/setCompileMode
  
• get/setRunRubyInProcess
  
• get/setCompatVersion
  
• get/setObjectSpaceEnabled
  
• get/setEnvironment
  
• get/setCurrentDirectory
  
• get/setHomeDirectory
  
• get/setClassCache
  
• get/setClassLoader
  
• get/setProfile
  
• get/setLoadServiceCreator
  
• get/setArgv
  
• get/setScriptFileName
  
• get/setRecordSeparator
  
• get/setKCode
  
• get/setJITLogEvery
  
• get/setJITThreshold
  
• get/setJITMax
  
• get/setJITMaxSize
  

Deprecated configuration methods:

• getRuntime()
  
• getProvider().setLoadPaths()
  
• getProvider().setClassCache()
  

Usage example:

[JRuby 1.4.0]
ScriptingContainer container = new ScriptingContainer();
container.getProvider().getRubyInstanceConfig().setJRubyHome(jrubyhome);

[JRuby 1.5.0]
ScriptingContainer container = new ScriptingContainer();
container.setHomeDirectory(jrubyhome);

New Options
Also, RedBridge got two new options: SHARING_VARIABLES and TERMINATION. The first, SHARING_VARIABLES, option turns on/off a sharing variables feature.This is an essential feature for some users while useless for other users. For those people, sharing variables is just a source of performance degradation. When the feature is turned off, the performance will be a bit better.
Usage example:

[Embed Core]
container.setAttribute(AttributeName.SHARING_VARIABLES, false);

[JSR223]
engine.getContext().setAttribute("org.jruby.embed.sharing.variables", false, ScriptContext.ENGINE_SCOPE);

The second, TERMINATION, option is for JSR223 users to call terminate. This option was added in light of RedBridge's behavior change. This is a big change, so I'll discuss more about this.

Changed Behaviors

* No termination in each evaluation and method call

RedBridge in JRuby 1.4.0 always called Ruby runtime's terminate method at the end of each evaluation and method call. This was to execute at_exit blocks and release resources automatically. The idea came from JSR223 API, which doesn't have a terminate method defined. However, I eliminated this behavior in JRuby 1.5.0 for three reasons. The first one is for performance. The terminate() method is so slow and was the biggest culprit of bad performance. The more Ruby code uses ruby files, instance variables, etc, the more it takes time. The second one is to make RedBridge's behavior more natural. Since the terminate() method fires at_exit blocks automatically, users might have unexpected results when they use third party libraries. Users should have a chance to fire at_exit blocks by themselves. The third one is that JRuby's memory leak was fixed. Thus, RedBridge doesn't need to invoke the terminate method just for releasing resources. Make sure, you have the terminate method in the right place.

Usage examples:

[Embed Core]

ScriptingContainer container = null;
try {
    container = new ScriptingContainer();
    container.runScriptlet(PathType.CLASSPATH, testname);
} catch (Throwable t) {
    t.printStackTrace();
} finally {
    container.terminate();
}

[JSR223]

ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("jruby");
engine.eval("$x='GVar'");
engine.eval("at_exit { puts \"#{$x} in an at_exit block\" }");  // nothing is printed here
engine.getContext().setAttribute(AttributeName.TERMINATION.toString(), true, ScriptContext.ENGINE_SCOPE);
engine.eval("");  // prints "GVar in an at_exit block"

* Global runtime

When a global ruby runtime exists on a single JVM, a singleton model of RedBridge uses the global runtime in JRuby 1.5.0. This works behind the scene and seems not so attractive, but is interesting. Here's a bit tricky usage:

$ pwd
/Users/yoko/Tools/jruby-1.5.0.RC1
$ jruby --1.9 -Ctest -S irb
irb(main):001:0> require 'java'
=> true
irb(main):002:0> container=org.jruby.embed.ScriptingContainer.new
=> org.jruby.embed.ScriptingContainer@771eb1
irb(main):003:0> container.compat_version                <---- --1.9 option
=> RUBY1_9
irb(main):004:0> container.current_directory             <---- -Ctest option
=> "/Users/yoko/Tools/jruby-1.5.0.RC1/test"
irb(main):009:0> container.home_directory                <---- jruby home
=> "/Users/yoko/Tools/jruby-1.5.0.RC1"
irb(main):010:0> container.run_scriptlet "at_exit { puts \"see you, later\" }" 
=> #<Proc:0x88a1b@<script>:1>
irb(main):011:0> container.terminate
see you, later
=> nil

This new behavior might be useful in a complicated application.
However, you should be aware that setting a runtime configuration doesn't work if the global runtime is there already. This is because the runtime configuration is read only when the runtime is instantiated. You should be careful not miss the timing to set configuration.

* Lazy Runtime Initialization

RedBridge (in this case, I mean Embed Core) delays ruby runtime initialization as much as possible. This is to improve ScriptingContainer's start up time. You may know loading Ruby runtime is a huge job and takes pretty much time. This might cause frustration if it happens right after the ScriptinContainer gets started. The question is when runtime is up and running. Some of ScriptingContainer's methods will kick ruby runtime to wake up. Here's a list:

• put()
  
• runScriptlet()
  
• setWriter()
  
• resetWriter()
  
• setErrorStream()
  
• resetErrorStream()
  
• setReader()
  

Thus, when you want configuration settings to work, you need to set them before these methods.
Meanwhile, JSR223 implementation doesn't delay ruby runtime initialization. It was not easy without breaking JSR223's requirement.

* Lazy Java Library Loading

Red Bridge doesn't load a java library while ruby runtime is initialized in JRuby 1.5.0. This is also for performance improvement. Loading libraries on to ruby runtime is quite a cumbersome job. Checking loaded library tables up to see whether a specified library has not yet loaded, judging how to load the library, then loading, caching... Even though Java library is not loaded while initialization, it will be loaded internally if necessary. Or you can load Java library explicitly:

container.runScriptlet("require 'java'");

Performance Tuning Tips
RedBridge's performance has been improved compared to older version, but you can tweak a bit more. For example, you can remove variables for sharing or clear sharing variable table at some point:

org.jruby.embed.ScriptingContainer#remove(String key)
org.jruby.embed.ScriptingContainer#clear()

RedBridge retrieves instance variables and constants as much as possible at the end of each evaluation and method call. All retrieved values are injected to runtime when the next script or method is evaluated. You can cut down the time for injection by removing unnecessary values.

Remaining jobs
RedBridge couldn't resolve all issues and has remaining jobs. Among them, OSGi and configuration on JSR223 impl would be two big issues. By the final release of JRuby 1.5.0, I want to improve these.


Finally, your input will help us to make RedBridge more perfect API. Give it a try and report us!

JRuby Embed (Red Bridge) Update: termination and skipping sharing variables

Hers' recent update of RedBridge. Performance got much better, but the change on termination might affect your code. If you expect at_exit blocks to be executed automatically, you need to add termination.

By the recent change, Embed Core, JSR223, and BSF, all three implementations had changes in their behaviors of evaluation and method invocation. Termination is no longer executed automatically. This means, at_exit blocks are not executed at the end of every runScriplet, run and callMethod (eval, invokeMethod and invoekeFunction in JSR223). It is effective since commit 673df9f.

This rather big changes was made for two reasons. One is to avoid possible unexpected behavior caused by at_exit blocks to be executed too early. For example, a gem might have a class that has an at_exit block, which should run after other code have finished. The second reason is a performance improvement. Terminate method takes much time to complete. Because of this, evaluation and method invocation of embedding API were very slow. Now, embedding API got much better performance than before.

Then, how to do that? To terminate explicitly, call terminate method on Embed Core and BSF. It's simple. However, JSR223 doesn't have terminate method defined by the specification. So, use newly added attribute, AttributeName.TERMINATION or org.jruby.embed.termination to trun termination on. Next, evaluate blank code.

Usage examples:

[Embed Core]

ScriptingContainer container = null;
try {
    container = new ScriptingContainer();
    conatiner.runScriptlet(PathType.CLASSPATH, testname);
} catch (Throwable t) {
    t.printStackTrace();
} finally {
    container.terminate();
}

[JSR223]

ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("jruby");
engine.eval("$x='GVar'");
engine.eval("at_exit { puts \"#{$x} in an at_exit block\" }");  // nothing is printed here
engine.getContext().setAttribute(AttributeName.TERMINATION.toString(), true, ScriptContext.ENGINE_SCOPE);
engine.eval("");  // prints "GVar in an at_exit block"

Let's move on to the second change. A new option has been added to skip sharing variables. Sharing variables is a useful or required feature for users from JSR223 and BSF background. But, it is not a necessary for others especially who have directly used JRuby's internal API. Sharing variables just slowed down the evaluations and method invocations. When sharing variables is skipped, the performance will be a bit better.

Usage example:

[Embed Core]
container.setAttribute(AttributeName.SHARING_VARIABLES, false);

[JSR223]
engine.getContext().setAttribute(AttributeName.SHARING_VARIABLES.toString(), false, ScriptContext.ENGINE_SCOPE);

Have fun with RedBridge!

Rava - pure Ruby JavaVM

Since JRuby 1.4.0, become_java! method has been available to use to create a real Java class from Ruby class. This new feature always reminds me "Rava," which was written by Koichi Sasada(Ko1) back in 2002. Ko1 is, of course, famous Ruby committer and the author of YARV. When the days that Ruby was infamous while Java was thriving, Ko1 wrote Rava. Although I could not make it work, I think the code itself is still worth glancing at.

- Rava / JavaVM on Ruby (2) (Rava version 0.0.2)
- Rava / JavaVM on Ruby (Rava version 0.0.1)

According to Ko1, Rava is pure Ruby JavaVM and joke software. He said in the article for a Japanese Magazine that Rava could load and interpret a Java class. Rava was not perfect but had basic features, for example:

• interprets most of bytecode


• invokes static/non-static methods


• reads/writes static/non-static fields


• handles exceptions


• runs threads

Also, Ko1 created a prototype of a JIT compiler. He made all of those in a week or so. Ko1 explained that's Ruby.

Here're excerpts from the article about Rava in depth.

• Operand Stack : Rava used Ruby Array to handle Java's operand stack since Ruby Array has enough feature to manage the stack.


• Types and data: Rava mapped Java's primitive types to Ruby's Number or its descendant. Java's reference type was converted into a field and Ruby object that has a reference to the original object. Java's field was mapped to Hash with keys of field names.


• Method invocation : Rava had its method frame as in below. JVM stack was in a single array, which includes operand stack.



[Rava Method Frame]
                         +----------+ --
       operand stack --->|          |  |
       stack pointer --->+----------+  |
  invoker frame info --->|          |  | method fame
                         +----------+  |
  local variable are --->|          |  |
      frame pointer ---> +----------+ --
invoker method frame --> |          |
                         +----------+
                           JVM stack



• JIT compiler : Rava converted bytecode into an equvalent Ruby script. To choose what bytecode should be compiled, Rava had a profiler to count a number of method invocation.

JRuby interprets Ruby on Java, while Rava interprets Java on Ruby. JRuby's become_java! converts Ruby class into Java bytecode, while Rava's JIT compiler converts Java bytecode into Ruby class. Unlike JRuby, Rava was outdated, which is a big difference; however, exploring Rava code might be fun.

Hacking JRuby - add all Hash methods to Map

I recently filed JRUBY-4528, whose patch adds all Ruby's Hash methods to a java.util.Map type object. Applying the patch, I confirmed that I could use "add_ruby_methods" method on Map type object, then, Hash methods for Map object. This would be useful especially for embedding API users since they often want to share Map object between Java and Ruby, back and forth.

What's the problem of current JRuby? When an instance of java.util.HashMap is sent into Ruby code, the object is converted into a "usable" Java object in Ruby world. This is what org.jruby.javasupport.JavaEmbedUtils.javaToRuby() method does, and we can't get Java Map converted into Ruby Hash automatically. Why? People might want to use that object as it is, HashMap type object itself, for other Java APIs used in Ruby. However, no built-in method converts Map to Hash so far although some of methods are added to.

My patch is attempt to add all Hash methods to Map type object by "add_ruby_methods" method.
For example:

irb(main):004:0> require 'java'
=> true
irb(main):005:0> jhash = java.util.HashMap.new
=> {}
irb(main):006:0> jhash.put("1", 100)
=> nil
irb(main):007:0> jhash.put("2", 200)
=> nil
irb(main):008:0> jhash.inspect
=> "{2=200, 1=100}"
irb(main):009:0> rhash = jhash.add_ruby_methods
=> {"2"=>200, "1"=>100}
irb(main):010:0> rhash.inspect
=> "{\"2\"=>200, \"1\"=>100}"
irb(main):011:0> p rhash.values
[200, 100]
=> nil
irb(main):012:0> rhash.merge!({"2"=>222, "3"=>333}) 
=> {"3"=>333, "2"=>222, "1"=>100}
irb(main):013:0> jhash.inspect
=> "{3=333, 2=222, 1=100}"

On jirb, I created java.util.HashMap object and put two key-value pairs using Java API, which was inspected by automatically added "inspect" method while converting. Then, I used add_ruby_methods method. After that, I could use Hash's inspect, values and merge! methods. Operations for "rhash" object above is also operations to "jhash," so when I inspected jhash, key-value pairs were also updated.

What if I create a Map object in Java code and give it to Ruby? Key-value pairs in a Java Map object was completely manipulated by Ruby. For example, see code below:

ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
ConcurrentHashMap map1 = new ConcurrentHashMap();
map1.put("a", 100);
map1.put("b", 200);
Map map2 = new HashMap();
map2.put("b", 254);
map2.put("c", 300);
container.put("h1", map1);
container.put("h2", map2);
container.put("num", 0);
String script = 
    "rh = h1.add_ruby_methods\n" +
    "puts \"num: #{num}\"\n" +
    "rh.merge!(h2.add_ruby_methods) {|k,o,n| num += 1; o+n }";
container.runScriptlet(script);
Set entries = map1.entrySet();
for (Map.Entry entry : entries) {
    System.out.print(entry.getKey() + ": " + entry.getValue() + ", ");
}

outputs:

b: 454, a: 100, c: 300,

As you see, merge! method worked. All java.util.Map type such as ConcurrentHashMap or TreeMap are available to apply the method.

Possible problem of this attempt is that contents of an object after add_ruby_methods applied are Java objects. For this reason, a direct comparison to a Ruby Hash object fails. In this case, to_hash method would work since to_hash method returns real Ruby Hash of converted Java Map.

This is just an attempt, and I'm not sure this patch will be applied or not. If you think this is useful, leave a comment on JIRA.

JRuby Embed (Red Bridge) Gotchas: on jirb

I haven't used like that before, but there are people who want to use JRuby Embed API on jirb. I fixed a bug, http://jira.codehaus.org/browse/JRUBY-4521, and tried what I could do on jirb.

At first, I instantiated ScriptingContainer and checked what initial parameters were set.

irb(main):001:0> require 'java'
=> true
irb(main):002:0> container = org.jruby.embed.ScriptingContainer.new
=> org.jruby.embed.ScriptingContainer@ea7549
irb(main):003:0> p container.get_home_directory                    
"/Users/yoko/DevSpace/jruby~main"
=> nil
irb(main):004:0> p container.load_paths
[/Users/yoko/DevSpace/jruby~main/lib/profile.jar, /Users/yoko/NetBeansProjects/cirrus/build/classes]
=> nil
irb(main):005:0> p container.class_loader
org.jruby.util.JRubyClassLoader@5e2075
=> nil
irb(main):006:0> p container.current_directory
"/Users/yoko/NetBeansProjects/cirrus"
=> nil
irb(main):007:0> p container.compat_version
RUBY1_8
=> nil
irb(main):008:0> p container.supported_ruby_version
"jruby 1.5.0.dev (ruby 1.8.7 patchlevel 174) (2010-02-02 0505fb1) (Java HotSpot(TM) Client VM 1.5.0_22) [i386-java]"
=> nil

Hmmm.... interesting. Of course, no compilation at all. Perhaps, ScriptingContainer's API is useful to see jirb internal settings.

Then, how evaluations go?

irb(main):009:0> script = "puts \"Hello World\""
=> "puts \"Hello World\""
irb(main):010:0> container.run_scriptlet(script)
Hello World
=> nil
irb(main):011:0> message = "Hi, there!"
=> "Hi, there!"
irb(main):012:0> container.put("message", message)
=> nil
irb(main):013:0> container.run_scriptlet("puts \"message: #{message}\"")
message: Hi, there!
=> nil

OK, evaluations as well as sharing variables between Java(?) (or jirb?) and Ruby seem to work.
How about method call?

irb(main):014:0> script = "def say\nputs \"oh!\"\nend"
=> "def say\nputs \"oh!\"\nend"
irb(main):015:0> recv = container.run_scriptlet(script)
=> nil
irb(main):016:0> container.call_method(recv, "say", java.lang.Object.class)
:1:in `say': wrong # of arguments(1 for 0) (ArgumentError)
 from :1
NativeException: org.jruby.embed.InvokeFailedException: wrong # of arguments(1 for 0)
 from org/jruby/embed/internal/EmbedRubyObjectAdapterImpl.java:387:in `call'
 from org/jruby/embed/internal/EmbedRubyObjectAdapterImpl.java:326:in `callMethod'
 from org/jruby/embed/ScriptingContainer.java:1268:in `callMethod'
 from :1

No, it failed. This is because jirb chose "public Object callMethod(Object receiver, String methodName, Object... args)" for callMethod. Unfortunately, in this case, Ruby doesn't know the difference of several callMethod methods.
Ok, then, no argument for "say" method. Will it work?

irb(main):017:0> container.call_method(recv, "say")                        
CallableSelector.java:196:in `assignableOrDuckable': java.lang.ArrayIndexOutOfBoundsException: 2
 from CallableSelector.java:22:in `access$200'
 from CallableSelector.java:163:in `accept'
 from CallableSelector.java:101:in `findCallable'
 from CallableSelector.java:86:in `findMatchingCallableForArgs'
 from CallableSelector.java:39:in `matchingCallableArityN'
 from RubyToJavaInvoker.java:170:in `findCallable'
 from InstanceMethodInvoker.java:29:in `call'
 from InstanceMethodInvoker.java:67:in `call'
 from AliasMethod.java:66:in `call'
 from CachingCallSite.java:329:in `cacheAndCall'
...
...

Oh dear, jirb was blown up.

So far, ScriptingContainer on jirb doesn't work enough but might be fun for quick hack.

........

Wrap this up.

We have java_send method and can specify exact Java method by its argument, but this also didn't work well. When I tried to run "volume" method of this Ruby code:

def volume(r)
  4.0 / 3.0 * Math::PI * r ** 3.0
end

irb(main):008:0> ret = container.run_scriptlet(org.jruby.embed.PathType::CLASSPATH, "ruby/sphere.rb")
=> nil
irb(main):009:0> container.java_send :callMethod, [java.lang.Object, java.lang.String, java.lang.Class], self, "volume", java.lang.Integer.new(3)
TypeError: for method ScriptingContainer.callMethod expected [class java.lang.Object, class java.lang.String, class java.lang.Class]; got: [org.jruby.RubyObject,java.lang.String,java.lang.Integer]; error: argument type mismatch
 from :1

like in the above, I got TypeError. A receiver object was the problem. Java program could cast java.lang.RubyObject to java.lang.Object, but jirb could not.

........

Wrap up, part 2.

While testing ScriptingContainer on jirb, I found a bug. Singleton model didn't see the same RubyInstanceConfig when Ruby runtime had already instantiated preceding ScriptingContainer. After the fix, some of runtime configurations can be changed through ScriptingContainer's methods. For example, I could change Ruby version to be used. The Ruby code below uses a block local variable introduced in Ruby 1.9.

# This snippet is borrowed from http://gihyo.jp/dev/serial/01/ruby/0003
# defines local variable x
x = "bear"

# A block local variable x is used in this block. (Two "x"s work together)
["dog", "cat", "panda"].each do |x|
  # This x is a block local variable.
  p x
  break if x == "cat"
end

# This x is a local variable since it is used outside of the block.
p x

When I tried this code by setting both Ruby 1.8 and 1.9, I got appropriate outputs for both mode on jirb.

irb(main):001:0> require 'java'
=> true
irb(main):002:0> container = org.jruby.embed.ScriptingContainer.new
=> org.jruby.embed.ScriptingContainer@ea7549
irb(main):003:0> container.compat_version
=> RUBY1_8
irb(main):004:0> container.run_scriptlet(org.jruby.embed.PathType::CLASSPATH, "ruby/block-param-scope.rb")
"dog"
"cat"
"cat"
=> nil
irb(main):005:0> container.set_compat_version(org.jruby.CompatVersion::RUBY1_9)
=> nil
irb(main):006:0> container.compat_version
=> RUBY1_9
irb(main):007:0> container.run_scriptlet(org.jruby.embed.PathType::CLASSPATH, "ruby/block-param-scope.rb")
"dog"
"cat"
"bear"
=> nil

So, again, SciprtingContainer on jirb is interesting. :)

JRuby Embed (Red Bridge) Update: configuration and global runtime

Here're recent updates of JRuby Embed (Red Bridge). Since I wrote about updates last time, Red Bridge's API has been changed a lot around configuration. Although this would directly affect to users' code, new API is easier to use and more stable. For example, setting jruby home directory has been changed from the fist to second one below:

[JRuby 1.4.0]
ScriptingContainer container = new ScriptingContainer();
container.getProvider().getRubyInstanceConfig().setJRubyHome(jrubyhome);

[JRuby 1.5.0.dev]
ScriptingContainer container = new ScriptingContainer();
container.setHomeDirectory(jrubyhome);

As you see clearly, getProvider().getRubyInstanceConfig() was gone. This is to hide JRuby's internal API so that internal API changes won't force users to adjust thier code. This also fixes the problem that current Red Bridge API exposes internal API too much and doesn't absorb internal changes well enough. To absorb internal changes is a Red Bridge's job, and by this update, more jobs have been covered. Other than get/setHomeDirectory(), we had following configuration methods in ScriptingContainer.

• get/setInput
  
• get/setOutput
  
• get/setError
  
• get/setCompileMode
  
• get/setRunRubyInProcess
  
• get/setCompatVersion
  
• get/setObjectSpaceEnabled
  
• get/setEnvironment
  
• get/setCurrentDirectory
  
• get/setHomeDirectory
  
• get/setClassCache
  
• get/setClassLoader
  
• get/setProfile
  
• get/setLoadServiceCreator
  
• get/setArgv
  
• get/setScriptFileName
  
• get/setRecordSeparator
  
• get/setKCode
  
• get/setJITLogEvery
  
• get/setJITThreshold
  
• get/setJITMax
  
• get/setJITMaxSize
  

Please go to API documantation and JRuby Options to know what are those.

The second update info is about global runtime. The "global runtime" means a static instance of Ruby runtime (org.jruby.Ruby) and, now, is used in a singleton model. The singleton model is a default local context type in JRuby 1.5.0, so unless users choose threadsafe or singlethread models explicitly, they will use static, JVM global Ruby runtime. Changing it a JVM global instance, we are supposed to get exactly the same Ruby runtime instance everywhere on a single JVM. This would be useful for some cases such as using Red Brdige for DSL. Meanwhile, we should be careful when Ruby code gets run on multi-threaded environment. Since Ruby runtime holds many states, most of them will be shared globally.

Red Bridge is still under the way to the stable API and may have more changes. However, it is surely on the way to more feasible API. Please feel free to suggest new methods or features over Red Bridge.

JRuby Embed (Red Bridge) Gotchas: __FILE__

About a month ago, I wrote __FILE__ didn't work when Ruby code was loaded from classpath in the thread, Load path issues inside jar / external app. Tracking jruby down with a debugger, I found out one solution. It was a combination of setting a feasible current directory and using File.expand_path.

Here's a test code:

# file_check.rb [Birch]

puts "__FILE__: #{__FILE__}"
puts "dirname: #{File.dirname(__FILE__)}"
puts "expanded path: #{File.expand_path(File.dirname(__FILE__))}"
puts "joined path 1: #{File.join(File.dirname(__FILE__), "abc.rb")}"
puts "joined path 2: #{File.join(File.expand_path(File.dirname(__FILE__)), "abc.rb")}"

// FileCheck.java
package vanilla;

import org.jruby.embed.LocalContextScope;
import org.jruby.embed.PathType;
import org.jruby.embed.ScriptingContainer;

public class FileCheck {

    private FileCheck() {
        //String userDir = System.getProperty("user.dir");
        //System.setProperty("user.dir", userDir+"/src/ruby");
        ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
        System.out.println("currentDirectory: " + container.getProvider().getRubyInstanceConfig().getCurrentDirectory());
        container.getProvider().getRubyInstanceConfig().setCurrentDirectory(System.getProperty("user.dir")+"/src/ruby");
        System.out.println("currentDirectory: " + container.getProvider().getRubyInstanceConfig().getCurrentDirectory());
        container.runScriptlet(PathType.CLASSPATH, "file_check.rb");
    }

    public static void main(String[] args) {
        new FileCheck();
    }
}

The absolute path to file_check.rb was /Users/yoko/NetBeansProjects/Birch/src/ruby/file_check.rb. So, I added the path, /Users/yoko/NetBeansProjects/Birch/src/ruby, to "-cp" option of java command. In a Java code, I set /Users/yoko/NetBeansProjects/Birch/src/ruby as the current directory. Then, the result was below:

yoko$ java -cp build/classes:/Users/yoko/DevSpace/jruby~main/lib/jruby.jar:./src/ruby  vanilla.FileCheck
currentDirectory: /Users/yoko/NetBeansProjects/Birch
currentDirectory: /Users/yoko/NetBeansProjects/Birch/src/ruby
__FILE__: file_check.rb
dirname: .
expanded path: /Users/yoko/NetBeansProjects/Birch/src/ruby
joined path 1: ./abc.rb
joined path 2: /Users/yoko/NetBeansProjects/Birch/src/ruby/abc.rb

As you see, a wrapped path by File.expand_path is correct though just File.dirname didn't work. The combination of setting the current directory and using File.expand_path would be the solution of this kind of cases. If you are using JSR223 or BSF, setting user.dir system property works as I commented out in the Java code. This is because JRuby uses the current directory when it expands a path, and the current directory is based on user.dir system property. If it is a web application, perhaps, we can set the current directory using ServletContext#getRealPath().

JRuby Embed (Red Bridge) Update: global vars, loading java, and more

During these weeks, I made a couple of changes on Red Bridge (JRuby Embed), which would improve performance a bit and reduce problems caused by global variables. This change is available from 161d0fe in master (1.5.0.dev).

Firstly, I changed an internal implementation of sharing global variables. Red Bridge injects all variables in a variable map just before the evaluation, and tries to retrieve all local, instance, global variables and constants used in Ruby just after the evaluation. This behavior is really greedy, also ends up in poor performance. However, it is necessary since Red Bridge terminates the all state including variable values right after the evaluation is done, which is to save resources. Unless retrieving all variables and constants, Red Bridge can't return requested variables in a Java program. For example, users can do with Red Bridge:

ScriptingContainer container = new ScriptingContainer();
container.runScriptlet("$theta = Math::PI / 6.0");
container.runScriptlet("$value = Math.sin($theta)");
System.out.println(container.get("$theta") + ", " + container.get("$value"));

Above outputs: 0.5235987755982988, 0.49999999999999994


Local, instance variables and constants (except global constants) need to be saved before those are disappeared by the termination, but global variables are still on Ruby runtime. So, I changed to get global variables lazily. Only when it is requested, Red Bridge takes the requested global variable out from runtime.

This new behavior would also reduce troubles caused by global variables. Before, Red Bridge retrieves global variables as much as possible from Ruby runtime except predefined ones. Then, Red Bridge injects all global variables in its variable map to runtime for successive evaluation with values of previous evaluation. This behavior occasionally causes unexpected results and warnings. After the change, Red Bridge doesn't grab unnecessary global variables, doesn't inject them for the next evaluation. Perhaps, unexpected results related to global variables will be reduced. This new behavior is not available when a global local variable behavior, JSR223's default behavior, is chosen since it is tailored to behave exactly the same as the reference implementation.

Some of you might already know clearing up the variable map before the successive evaluation contributes performance. I added two shortcut methods to ScriptingContainer:

org.jruby.embed.ScriptingContainer#remove(String key)
org.jruby.embed.ScriptingContainer#clear()

The remove method removes a specified key-value pair from the variable map and runtime. The clear method removes all key-value pairs from the variable map and runtime. The smaller the variable map size is, the shorter the time for injection is. Don't forget to remove redundant key-value pairs.


I made one more change. Red Bridge no more loads a java library during the initialization. The process of loading libraries in JRuby is quite a cumbersome job. Looking the loaded library tables up to see it is not already loaded, judging how and from where loads the library, then loading, and caching them to avoid duplication... Nevertheless, not all Ruby scripts need the java library. If people run Fibonacchi written in pure Ruby on Red Bridge, they don't need the java library at all. When people want to use the java library, adding the line "require 'java'" in a Ruby code works fine. Moreover, people add "require 'java'" when they run scripts using jruby command if the scripts need the java library. The advantage of pre-loading the java library seems to be less. So, I stopped loading the java library during the initialization. Perhaps, the time for initialization got shortened a bit.

A Japanese Teenage Boy Improved Ruby 1.9 Performance Up to 63%

Japanese online magazine, @IT Jibun Senryaku Lab. (information site for IT engineers to educate and/or develop oneself), published an interview with a Japanese teenage boy, Masahiro Kanai, who improved the performance of several methods in Ruby 1.9. He is the age of high school freshman (the third grade of junior high school in Japanese school system). The article (written in Japanese) is here.

According to the article, Masahiro Kanai joined “the Security and Programming Camp 2009” this summer and chose the subject of Ruby’s performance improvement. His mentor was Koichi Sasada (ko1). The performances of the methods he worked have been bumped up 63% in maximum, 8% in average. His patches were applied to Ruby trunk in Oct. 5 this year.

What Masahiro Kanai did was fundamental for performance tuning. He took unnecessary macro references out from a loop. Masahiro spotted macros below in array.c, string.c, and struct.c were referred every time Ruby checked whether data was hold in a structure or not. Even though data were constants, Ruby saw the macros to judge data’s presence in every loop.

-RARRAY_PTR, RARRAY_LEN
-RSTRING_PTR, RSTRING_LEN
-RSTRUCT_PTR, RSTRUCT_LEN

He optimized the loop by eliminating macro references when data were constants.

The interviewer acclaimed that he made it in his age.

JRuby Embed (Red Bridge) Update

Since my last post about JRuby Embed (Red Bridge), it has been vastly changed. JRuby Embed codebase has been merged into JRuby! JRuby 1.5.0 will have Red Bridge inside in its both binary and source archives. Along with this, JRuby Embed wiki pages also have been merged into JRuby's wiki, Embedding JRuby section.

Now, JRuby Embed project is almost in end-of-life period. I'll soon close jruby-embed users ml since it is natural to talk at jruby-users/jruby-dev. Besides, most of discussions have done on jruby-users ml. Jira is also going to be merged into JRuby, but this will be done after JRuby's jira is completed moving from codehaus to kenai. Anyway, JRuby Embed users, please use jruby's ml and jira. JRuby's embedding seciton of jira would be good for us to file issues. However, I'll keep source code repository for JRuby 1.4. JRuby 1.4 has JRuby Embed binary but doesn't have sources. The binary that JRuby 1.4 has is built from codebase of this project, so it still has a reason to be there.

One of the biggest changes is JRuby Embed 0.1.3 has been released from JRuby Embed Project. It will be included in upcoming JRuby 1.4 release. In this release, default value of local context type has been switched from threadsafe to singleton. See the discussion about it. Please make sure your choice is the best to your case. Walk through Context Instance Type section to know what you should choose.

The version, 0.1.3 is identical to the one in JRuby trunk (1.5.0.dev) and also had a fix of JRUBY_EMBED-10. Give it a try. If you find something, file at "JRuby Jira" and ask about it at JRuby's mailing list.

What's the embedding API of JRuby 1.4.0RC1?

JRuby 1.4.0RC1 has been released on Oct. 2 and was a big release. JRuby had a lot of bug fixes and new features. Among them, JRuby Embed (aka Red Bridge) was there. The name, Red Bridge, means a bridge from Java to Ruby and, of course, the bridge has a color of ruby. However, many people would have thought, “What’s the new embedding API?” when they saw Tom’s announce. In this blog post, I ‘m going to answer such question so that people can have better understandings about Red Bridge.

Red Bridge is a Java API to run Ruby scripts in a Java program, and the project is hosted at http://kenai.com/projects/jruby-embed. Red Bridge has two layers, Embed Core and Core based implementations of scripting API. Currently, JSR223 (javax script: http://jcp.org/en/jsr/detail?id=223) and Jakarta BSF 2.4 (Bean Scripting Framework: http://jakarta.apache.org/bsf/) are implemented on top of Embed Core. Embed Core is totally different API from JRuby’s JavaEmbedUtils, which has similar but much fewer API compared to Embed Core. Embed Core has a lot of useful methods and features for embedders. Users of this new embedding API don’t need to use JavaEmbedUtils anymore. Besides, not like scripting APIs that are common to many languages, Embed Core is focused on leveraging JRuby’s power. For example, Embed Core allows users to configure Ruby runtime easily. For example, Embed Core’s parse method can have a JRuby friendly argument, InputStream, to read scripts from.

Red Bridge was originally my solo project I started in the last winter at Google Code to solve issues that Sun’s JSR223 JRuby engine reference implementation had. I was a contributor of JRuby engine at scripting.dev.java.net but felt reluctant to rewrite the it vastly since I’m not a Sun employee. Especially, the license of the reference implementation was a big issue to distribute with JRuby. JSR223 JRuby engine users wanted the implementation to be bundled in JRuby. So, I tried to get permission from Sun, and if possible, modify the license to fit into JRuby. But, I couldn’t get any answer from Sun at all. Other than the license issue, reference implementation’s bug-prone sharing global variable mechanism was a headache to me. That part was repeatedly affected by JRuby’s internal API changes, and grew to literally patchwork like ugly code. That global variables were only one type for sharing variables between Java and Ruby was also a problem. For JavaScript, PHP or maybe other languages, a variable name should be start with ‘$.” However, the name, $something, means not just a variable to Ruby but a globally referenced variable. Some people were eager to use another variable types to share. The sharing global variable of reference implementation also had a problem when JRuby engine was used on a multi-threaded environment such as a Servlet container (Java based web application server). The reference implementation might have set true to ThreadLocal option of Ruby runtime using a System property. However, relying on JVM wide system properties caused another problem especially on web application servers. A web application server might have multiple web applications (wars) on it and system property settings affect all of them.

In light of these issues Red Bridge has exactly the same license as JRuby and new mechanism for sharing variables, besides enables sharing global, local, and instance variables. Users can choose ThreadLocal model for context local values such as Ruby runtime, or sharing variables and other instances. Embed Core provides users methods to configure Ruby runtime. However, JSR223 and BSF engines still rely on JVM wide system property since those APIs haven’t defined such method. See Wiki, http://kenai.com/projects/jruby-embed/pages/Home, for details.

At kenai.com, you might find the project whose name is “Red Bridge.” When I moved my project to kenai.com right after I got the invitation from Charles Oliver Nutter, the name was Red Bridge, the same one at Google Code. A couple of weeks later, I talked with Charles and Thomas Enebo about Red Bridge. They liked Core part of two layers of Red Bridge and wanted to have just core layer bundled in JRuby. Following their choice, I started “JRuby Embed” project just for Embed Core. After that, JSR223 and BSF were added to the list to be bundled in JRuby, and Red Bridge was merged into JRuby Embed project. Merged into Red Bridge was definitely another choice. However, I chose JRuby Embed because people were interested in Embed Core part more than JSR 223 implementation and more members have been subscribed in JRuby Embed. Besides, the package name is org.jruby.embed, no redbridge in it. Since the name, “Red Bridge,” is easy to memorize and nice compared to banal name, “JRuby Embed,” I’ll keep using Red Bridge. While BSF implementation never had its own project ever. The implementation was added after JSR223 was merged in and took for a week or so.

Having JRuby 1.4.0RC1, users might be confusing JRuby’s JavaEmbedUtils and Red Bridge, and which one they should use. Definitely, new users should use Red Bridge since it is easy to use and powerful. (I’m working hard to update documents, so some of them are old. Sorry!) Right now, JavaEmbedUtils as well as other embed related interfaces are on a discussion to seek how they can be obsolete. API of JavaEmbedUtils and others have been used in many packages including JRuby Rack, so making them obsolete would be influential. Red Bridge will probably need to have bug fixes and improve its performance. Also API of Red Bridge probably needs to be reviewed and modified. I think it takes a time to eliminate JavaEmbedUtils.

Then, what will be next? I want to add a feature to run compiled Ruby scripts on Red Bridge. Currently, JIT and Force compiled modes are supported, but those are different from executing *.class files generated from *.rb. “Rails on Red Bridge” will be my exciting challenge. If people can write Struts’ action by Ruby using Red Bridge, it might be interesting. I don’t have a clear load map right now, but I want to keep going.

Have fun with Red Bridge!

Splitting jruby-complete.jar up for Google App Engine

When we write a web application using JRuby, we need jruby-complete.jar included in a war to use builtin libraries. The builtin libraries are supposed to be located under jruby.home, so the jruby.home system property is expected to be set correctly. However, we need an alternative to set the property since setting jruby.home on the web application doesn't make sense. The answer is jruby-complete.jar, which has builtin libraries under META-INF/jruby.home directory in it.

When we use Google App Engine, another problem pops up. GAE has 10MB limit per each file to upload (http://googleappengine.blogspot.com/2009/02/skys-almost-limit-high-cpu-is-no-more.html). The size of jruby-complete.jar is unfortunately over 10MB. The shell script to split jruby-complete.jar up into two jar archives has been introduced at http://olabini.com/blog/2009/04/jruby-on-rails-on-google-app-engine/. However, I learned the way in the blog was already obsolete when I filed JRUBY-3949. The smart way of doing that was already out there. The latest JRuby, I mean, JRuby 1.4.0dev in git HEAD has had a Rakefile to create jars.

This is what I acutually did on a terminal:

$ git clone git://kenai.com/jruby~main
$ cd jruby~main
$ export JRUBY_HOME=`pwd`
$ PATH=$JRUBY_HOME/bin:$PATH
$ ant
$ gem install rake
$ gem install hoe
$ cd gem
$ rake update

Then, jruby-core-1.4.0dev.jar and jruby-stdlib-1.4.0dev.jar was built in jruby~main/gem/lib directory.

Finally yaml worked on Google App Engine

In my previous post, I wrote about my struggle over an application on Google App Engine that uses JRuby's builtin library, yaml. I found the reason of the error at JRUBY-3892. Builtin library needs jruby.home environment variable to be set correctly, and it should be done in jruby-complete.jar. But, jruby-complete.jar built from old JRuby 1.4.0dev had a somewhat broken path. Since the issue has been resolved in the end of Auguest, I tried again using the latest JRuby 1.4.0dev cloned out from git repo. It worked. So, the final release of JRuby 1.4.0 won't have this problem.

Now, my sample Servlet, ParsenRunServlet is working at http://servletgarden-in-red.appspot.com/. If you are interested in the code, those are in JRuby Embed API Wiki.

Yaml doesn't work on Google App Engine

While I was testing JRuby Embed API on Google App Engine, I encountered this awkward problem. Yaml never worked on GAE. Exactly the same Servlet successfully worked on GlassFish. Servlet and Ruby codes were:

# yaml_snippet.rb

require 'yaml'

content = YAML::load @text

def format element
  case element
  when String: print "<p>#{element}</p>"
  when Array:
    print "<ul>"
    element.each do |child|
      print "<li>"
      format child
      print "</li>"
    end
    puts "</ul>"
  when Hash:
    element.each do |key, value|
      print "<ul><li>#{key}"
      format value
      print "</li></ul>"
    end
  end
end

content.each do |heading, paragraph|
  puts "<h4>#{heading}</h4>"
  paragraph.each do |element|
    format element
  end
end

package olive.jruby.example;

import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.PrintWriter;
import java.util.Arrays;
import java.util.List;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.jruby.embed.ScriptingContainer;
import org.jruby.javasupport.JavaEmbedUtils.EvalUnit;

public class YamlSampleServlet extends HttpServlet {
    private ScriptingContainer container;
    private EvalUnit yaml_unit;
    private String text =
            "Trees:\n" +
            "- This is a small example to general HTML.\n" +
            "- - Quince\n" +
            "  - flower: Red\n" +
            "- - Apple\n" +
            "  - fruit: Red\n" +
            "- - Maple\n" +
            "  - leaf: Red";

    @Override
    public void init() {
        String classpath = getServletContext().getRealPath("/WEB-INF/classes");
        List<String> loadPaths = Arrays.asList(classpath.split(File.pathSeparator));
        container = new ScriptingContainer();
        container.getProvider().setLoadPaths(loadPaths);
        String filename = "ruby/yaml_snippet.rb";
        InputStream istream = container.getRuntime().getJRubyClassLoader().getResourceAsStream(filename);
        yaml_unit = container.parse(istream, filename);
    }
   
    protected void processRequest(HttpServletRequest request, HttpServletResponse response)
    throws ServletException, IOException {
       response.setContentType("text/html;charset=UTF-8");
        PrintWriter out = response.getWriter();
        container.setWriter(out);
        try {
            out.println("<html>");
            out.println("<head>");
            out.println("<title>Servlet YamlSampleServlet</title>");
            out.println("</head>");
            out.println("<body>");
            out.println("<h3>Servlet YamlSampleServlet at " + request.getContextPath() + "</h3>");
            container.put("@text", text);
            yaml_unit.run();
            out.println("</pre></body>");
            out.println("</html>");
        } finally {
            out.close();
        }
    } 

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response)
    throws ServletException, IOException {
        processRequest(request, response);
    } 

    @Override
    protected void doPost(HttpServletRequest request, HttpServletResponse response)
    throws ServletException, IOException {
        processRequest(request, response);
    }

    @Override
    public String getServletInfo() {
        return "Yaml Sample";
    }
}

And the programs produced:

<html>
<head>
<title>Servlet YamlSampleServlet</title>
</head>
<body>
<h3>Servlet YamlSampleServlet at /Olive</h3>
<h4>Trees</h4>
<p>This is a small example to general HTML.</p><ul><li><p>Quince</p></li><li><ul><li>flower<p>Red</p></li></ul></li></ul>
<ul><li><p>Apple</p></li><li><ul><li>fruit<p>Red</p></li></ul></li></ul>
<ul><li><p>Maple</p></li><li><ul><li>leaf<p>Red</p></li></ul></li></ul>
</pre></body>
</html>

However, this test servlet never worked on GAE because of the exception:

     [java] yaml/constants:15:in `const_missing': uninitialized constant YAML::Yecht::Resolver (NameError)
     [java]         from yaml:88
     [java]         from yaml:2:in `require'
     [java]         from ruby/yaml_snippet.rb:2
     [java]         ...internal jruby stack elided...
     [java]         from Module.const_missing(yaml:88)
     [java]         from (unknown).(unknown)(yaml:2)
     [java]         from (unknown).(unknown)(yaml:2)
     [java]         from Kernel.require(ruby/yaml_snippet.rb:2)
     [java]         from (unknown).(unknown)(:1)

JRuby has yaml library under the "builtin" directory in its source tree, so all of necessary scripts here are in jruby.jar as well as jruby-complete.jar. But, yaml library needs jruby.home system property to be set correctly, so jruby-complete.jar is the only choice on a web application. In case of an application, I could set jruby.home explicitly as in below and got the result as I expected:

package brick;

import java.io.InputStream;
import org.jruby.embed.ScriptingContainer;
import org.jruby.javasupport.JavaEmbedUtils.EvalUnit;

public class YamlSample {

    private String filename = "ruby/yaml_snippet.rb";
    private String text =
            "Trees:\n" +
            "- This is a small example to general HTML.\n" +
            "- - Quince\n" +
            "  - flower: Red\n" +
            "- - Apple\n" +
            "  - fruit: Red\n" +
            "- - Maple\n" +
            "  - leaf: Red";

    private YamlSample() {
        System.setProperty("jruby.home", "/Users/yoko/Works/080909-jruby/jruby~main");
        ScriptingContainer container = new ScriptingContainer();
        InputStream istream = container.getRuntime().getJRubyClassLoader().getResourceAsStream(filename);
        EvalUnit yaml_unit = container.parse(istream, filename);
        container.put("@text", text);
        yaml_unit.run();
    }

    public static void main(String[] args) {
        new YamlSample();
    }
}

I tried to copy all yaml library under the source tree of my web application, and clean & build, redeploy, and request again... with no luck. Sigh... I tried to set the path to the yaml library in jar archive,

String classpath = getServletContext().getRealPath("/WEB-INF/lib/jruby-complete.jar!/builtin");
List loadPaths = Arrays.asList(classpath.split(File.pathSeparator));
container = new ScriptingContainer();
container.getProvider().setLoadPaths(loadPaths);

... no luck. The same exception, as ever.

Since the program worked on GlassFish, I guess the difference in class loading mechanism between GlassFish and GAE might have caused the exception on GAE. But, I haven't figured the culprit out so far. Any idea?

NekoBean Fall Version

NetBeans' mascot, NekoBean, is enjoying cool air in fall surrounded by colored foliage.

More at:

http://nekobean.net/2009/08/post-18.html.

JRuby Embed API Update: Servlet Examples

I added Servlet Examples section in JRuby Embed API Wiki. Right now, just three examples are in that section. (I'll add more examples later.) Those are:

• HelloWorldServlet
  Simple "Hello World" example, but helpful to get started.


• GreetingServlet
  Two methods written in Ruby are called from Servlet.


• SortableServlet
  Java interface is implemented in two ways in Ruby.

I tested these Servlets on Google App Engine and felt relieved since all three Servlets worked well. I've wanted to verify that Embed API works on GAE, which has some restrictions in programming on it. Embed API doesn't use any unsupported API, so there should not be any problem. However, I realized that I had to specify a classpath explicitly, and the classpath to be specified should not include appengine-tools-api.jar. If no classpath is given, Embed API sees java.class.path system property that has a path to appengine-tools-api.jar. This means, JRuby tries to load appengine-tools-api.jar onto Ruby runtime. The result is ... simply getting an exception. Thus, when Embed API is used with Servlet, especially, with Google App Engine, setting classpath is really important.

Embed API has two ways of setting a classpath. One is to use org.jruby.embed.class.path system property. This is easy, but not a preferred way in a web application. Since system property is common on Java VM, so the value is shared by every Servlet in more than one war archives and mutliple web applications on a single web application server. Some servlet might set classpath "A" using org.jruby.embed.class.path. At the same time another servlet might try to set classpath "B" using org.jruby.embed.class.path. We don't know what classpath is actually used.

Another way of setting classpath is to use setLoadPaths method of API. For example,

public class HelloWorldServlet extends HttpServlet {
    private ScriptingContainer container;

    @Override
    public void init() {
        String classpath = getServletContext().getRealPath("/WEB-INF/classes");
        List loadPaths = Arrays.asList(classpath.split(File.pathSeparator));
        container =
                new ScriptingContainer(LocalContextScope.SINGLETHREAD);
        container.getProvider().setLoadPaths(loadPaths);
    }
...

Technically, we don't need to set the classpath to /WEB-INF/classes, since it has been already set by a server. But, some path with no further trouble is needed, so I chose that.

JRuby Embed API has a public method to set classpath, but JSR 223 implementation is unable to have such method. The specification doesn't define such method. The only way to set classpath for JSR 223 implementation is to use system property. It is true also in RedBridge. So, be careful to choose a harmless classpath to all Servlets on a web application server.

RedBridge and JRuby Embed API update

I updated both JRuby Embed API and RedBridge and the latest version is, now, 0.0.1.1. By this update, three types of local variable behaviors were added in light of the discussion, http://www.nabble.com/Call-for-discussion-about-embed-API-tc24528478.html. Before the update of Embed API and RedBridge, Ruby's local variables always survived over the multiple evaluations. Thus, local variables used in the first script evaluation were always reused in the second, third, or fourth evaluation even though scripts has no relation to each other. Of course, users could delete unwanted local variables explicitly before the following evaluations went on, but this didn't happen in default. This feature was useful especially for ex-BSF users; however, it was not semantically correct. So, Tom Enebo concerned about it. During the discussion was going on, a nice idea of a toggle-able local variable was suggested (Thank you, Adam ;) ), and seemed to satisfy conflicting needs. The latest version supported the toggle-able local variable.

New local variable behavior has three options, transient, persistent and global. The first default behavior, transient, is a faithful behavior to Ruby semantics. So, local variables vanish after each evaluation. Java programs can't get local variables used in Ruby scripts. If you want to use the same value or object as a local variable in more than one script, you need to reset it again and again. However, instance and global variables survive over the evaluations as those were in the previous version.

The second variable behavior, persistent, is the behavior that the previous version did. Thus, the same local variables can be used in multiple script evaluations. Also, those can be retrieved from Ruby and used in Java. As well as a local variable, an instance and global variables, and a constant are persistent over multiple evaluations.

Example for JRuby Embed API:

package brick;

import java.util.Map;
import java.util.Set;
import org.jruby.embed.ScriptingContainer;
import org.jruby.embed.LocalVariableBehavior;

public class Sample1 {

    private Sample1() {

        ScriptingContainer container = new ScriptingContainer(LocalVariableBehavior.PERSISTENT);
        container.runScriptlet("p=9.0");
        container.runScriptlet("q = Math.sqrt p");
        container.runScriptlet("puts \"square root of #{p} is #{q}\"");
        Map m = container.getVarMap();
        Set<String> keys = container.getVarMap().keySet();
        for (String key : keys) {
            System.out.println(key + ", " + m.get(key));
        }
        System.out.println("Ruby used: p = " + container.get("p") +
                ", q = " + container.get("q"));
    }

    public static void main(String[] args) {
        new Sample1();
    }
}

Example for RedBridge (JSR223):

package redbridge;

import java.util.Set;
import javax.script.Bindings;
import javax.script.ScriptContext;
import javax.script.ScriptEngine;
import javax.script.ScriptException;
import org.jruby.embed.jsr223.JRubyScriptEngineManager;

public class EvalStringSample {

    private EvalStringSample() throws ScriptException {
        System.out.println("[" + getClass().getName() + "]");
        System.setProperty("org.jruby.embed.localvariable.behavior", "persistent");
        JRubyScriptEngineManager manager = new JRubyScriptEngineManager(Thread.currentThread().getContextClassLoader());
        ScriptEngine engine = manager.getEngineByName("jruby");
        engine.eval("p=9.0");
        engine.eval("q = Math.sqrt p");
        engine.eval("puts \"square root of #{p} is #{q}\"");

        Bindings bindings = engine.getBindings(ScriptContext.ENGINE_SCOPE);
        Set<String> keys = bindings.keySet();
        for (String key : keys) {
            System.out.println(key + ", " + bindings.get(key));
        }
        System.out.println("Ruby used: p = " + engine.get("p") +
                ", q = " + engine.get("q"));
    }

    public static void main(String[] args) throws ScriptException {
        new EvalStringSample();
    }
}

Output:

square root of 9.0 is 3.0
MANT_DIG, 53
MAX_10_EXP, 308
DIG, 15
MIN_EXP, -1021
ROUNDS, 1
MAX, 1.7976931348623157E308
RADIX, 2
EPSILON, 2.220446049250313E-16
MIN, 4.9E-324
q, 3.0
p, 9.0
MIN_10_EXP, -307
MAX_EXP, 1024
Ruby used: p = 9.0, q = 3.0

The third variable behavior, global, is a backwards compatibility option for users who have used JSR223 reference implementation released form scripting.dev.java.net. The reference implementation (RI) uses Ruby's global variable to share variables between Java and Ruby. And the name of variables used in Java has the same form as the one of a local variable in Ruby. I mean, Java sees "message" while Ruby sees "$message." However, Embed API and RedBridge enable not only the global variable but also the instance and local variable and constant sharing. On Redbridge, when people use the name "message" in Java, they also use "message" in Ruby. When it is "$message" in Java, also, "$message" in Ruby. So that RI users can move on to RedBridge easily, I added this local variable behavior.

Example for RedBridge (JSR223):

# greetings_globalvars.rb

def greet
  message = "How are you? #{$who}."
end

def sayhi
  $, = ","
  $\ = "\n"
  print "Hi", $people
  $, = ""
  $\ = nil
end

def count
  $people.size + 1
end

// OldVariableBehaviorSample.java
package redbridge;

import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.Reader;
import java.util.ArrayList;
import java.util.List;
import javax.script.Invocable;
import javax.script.ScriptEngine;
import javax.script.ScriptException;
import org.jruby.embed.jsr223.JRubyScriptEngineManager;

public class OldVariableBehaviorSample {
    private final static String basedir = "/Users/yoko/NetBeansProjects/Birch";

    private OldVariableBehaviorSample()
            throws ScriptException, FileNotFoundException, NoSuchMethodException {
        System.setProperty("org.jruby.embed.localvariable.behavior", "old");
        JRubyScriptEngineManager manager = new JRubyScriptEngineManager();
        ScriptEngine engine = manager.getEngineByName("jruby");
        String filename = basedir + "/src/ruby/greetings_globalvars.rb";
        Reader reader = new FileReader(filename);
        engine.put("who", "Anakin");
        List people = new ArrayList();
        people.add("Obi-Wan");
        people.add("C-3PO");
        people.add("R2-D2");
        engine.put("people", people);
        engine.eval(reader);
        Object[] args = null;
        Object result = ((Invocable)engine).invokeFunction("greet", args);
        System.out.println(result.toString());
        ((Invocable)engine).invokeFunction("sayhi", args);
        result = ((Invocable)engine).invokeFunction("count", args);
        System.out.println("counted: " + result.toString());
    }

    public static void main(String[] args)
            throws ScriptException, FileNotFoundException, NoSuchMethodException {
        new OldVariableBehaviorSample();
    }
}

Output:

How are you? Anakin.
Hi,[Obi-Wan, C-3PO, R2-D2]
counted: 4

See wiki pages for details.
JRuby Embed API: http://kenai.com/projects/jruby-embed/pages/Home
RedBridge: http://kenai.com/projects/redbridge/pages/Home

RedBridge Update: JRubyScriptEngineManager

Today, I added two classes, JRubyScriptEngineManager and ServiceFinder, to RedBridge (JSR 223 JRuby engine). ServiceFinder is used from JRubyScriptEngineManager, and not for users. This update will be helpful especially for OS X users. Now, RedBridge works on both JDK 1.5 and 1.6 on OS X Java Update 4.

Since its first release, RedBridge hasn't had JRubyScriptEngineManager mainly because of copyright. JSR 223 JRuby Engine released from Scripting Project at dev.java.net has the same name and behavior class. Although I wrote that class, I couldn't simply include it in RedBridge since Sun has copyright. Thus, I've tested RedBridge on JDK 1.6 though RedBridge itself was compiled on JDK 1.5. However, after OS X's Java has been updated in last June, JDK 1.6's service discovery failed to locate RedBridge. So, I decided to write it. Like other classes of RedBridge, I totally rewrote JRubyScriptEgineManager, too, so that RedBridge won't suffer from unexpectd legal issues. The new JRubyScriptEngineManager isn't just an modified version of the old one. I wrote it as simple as possible because, I think, JSR 223 is, in many cases, used with frameworks. Keeping it vanilla would be better for users. Less headache.

Now, the snippet will be:

package redbridge;

import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
import javax.script.ScriptException;
import org.jruby.embed.jsr223.JRubyScriptEngineManager;

public class EvalStringSample {

    private EvalStringSample() throws ScriptException {
        System.out.println("[" + getClass().getName() + "]");
        System.setProperty("org.jruby.embed.localcontext.scope", "singlethread");
        //ScriptEngineManager manager = new ScriptEngineManager();
        JRubyScriptEngineManager manager = new JRubyScriptEngineManager();
        ScriptEngine engine = manager.getEngineByName("jruby");
        engine.eval("p=9.0");
        engine.eval("q = Math.sqrt p");
        engine.eval("puts \"square root of #{p} is #{q}\"");
        System.out.println("q = " + engine.get("q"));
    }

    public static void main(String[] args) throws ScriptException {
        new EvalStringSample();
    }
}

Result:

[redbridge.EvalStringSample]
square root of 9.0 is 3.0
q = 3.0

JRubyScriptEngineManager can have a classloader in its constructor argument. When no classloader is given, JRubyScriptEngineManager uses System classloader. For example, to use a current context classloader:

JRubyScriptEngineManager manager =
    new JRubyScriptEngineManager(Thread.currentThread().getContextClassLoader());

See, Wiki at the RedBridge project for other usages.

Start Over: JSR 223 JRuby engine on OSGi container

I got a comment from Neil Bartlett about my previous post. Yes, it was hard for OSGi people to understand what's wrong with it. Originally, my blog entry was to answer the question, "how can I create an osgi bundle using maven which uses jruby-engine to execute a jruby script?" So, I pasted entire pom.xml on it. As Neil commented, I should have pasted MANIFEST.MFs that all bundles used. This is hopefully for OSGi people to figure out culprits and help me out.

• What I want to do is ...
  
  I'm a committer of JSR 223 JRuby engine. I want to provide a painless OSGi bundle of JSR 223 JRuby engine to users. JRuby engine works on top of JRuby, consequently, users need at least three bundles, JRuby, JRuby engine, and their own bundle, to get it work on OSGi containers. As far as I tested, current MANIFEST.MF of JRuby engine or JRuby, or both might have a flaw, but not sure. I want to fix JRuby engine's flaw if it exists as well as JRuby's.


• Initial Problems were ...
  
  There were two basic problems. The first one was JSR 223's discovery mechanism didn't work on OSGi. The mechanism is officially introduced in JDK 1.6, but work on JDK 1.5, too. The mechanism works like this:
  1. looks for META-INF/services/javax.script.ScriptEngineFactory file in every jar file found from classpath.
  2. instantiate JSR 223 engine class specified in the javax.script.ScriptEngineFactory file.
  
  Probably, because of a classloading issue, this mechanism doesn't work on Apache Felix. However, we can avoid this problem by instatiating a JRuby engine factory directly bypassing discovery mechanism.
  
  The second problem is the one I'm seeking the best solution. While instantiating JRuby engine factory, com.sun.script.jruby.JRubyScriptEngineFactory (line 13 in the snippet of Take One), JRuby engine, com.sun.script.jruby.JRubyScriptEngine, is also instatiated. These two are in the same, JRuby engine's bundle. While instantiating JRuby engine, Ruby runtime is instantiated, too. Ruby runtime is in a different, JRuby's bundle. Up to here, no problem exists. At the same time, JRuby engine tries to load the instance of org.jruby.javasupport.Java on to Ruby runtime using JRuby's custom classloader. The class, org.jruby.javasupport.Java is in JRuby's bundle. This ends up in raising exception.

  org.jruby.exceptions.RaiseException: library `java' could not be loaded: java.lang.ClassNotFoundException: org.jruby.javasupport.Java

  I don't think I have a choice to use another classloader to load org.jruby.javasupport.Java since it is JRubish way to use Java classes in Ruby scripts.
  
  JRuby's MANIFEST.MF used for this sample code is here. (This is so long to paste.)
  
  JRuby engine's MANIFEST.MF
  

  
  Manifest-Version: 1.0
  Built-By: yoko
  Created-By: Apache Maven Bundle Plugin
  Import-Package: com.sun.script.jruby,javax.script,org.jruby,org.jruby.
   exceptions,org.jruby.internal.runtime,org.jruby.javasupport,org.jruby
   .runtime,org.jruby.runtime.builtin,org.jruby.runtime.load,org.jruby.u
   til,org.jruby.util.io
  Bnd-LastModified: 1247081259404
  Export-Package: com.sun.script.jruby;uses:="javax.script,org.jruby.run
   time.builtin,org.jruby.runtime,org.jruby,org.jruby.internal.runtime,o
   rg.jruby.exceptions,org.jruby.javasupport,org.jruby.util,org.jruby.ru
   ntime.load,org.jruby.util.io"
  Bundle-Version: 1.0
  Bundle-Name: JRuby JSR223 Engine
  Build-Jdk: 1.5.0_19
  Private-Package: com.sun.script.jruby,
  Bundle-ManifestVersion: 2
  Bundle-SymbolicName: com.sun.script.jruby
  Tool: Bnd-0.0.311
  

  
  And the MANIFEST.MF of the snippet:
  

  
  Manifest-Version: 1.0
  Built-By: yoko
  Created-By: Apache Maven Bundle Plugin
  Bundle-Activator: hickory.example.Activator
  Import-Package: com.sun.script.jruby,hickory.example,javax.script,org.
   osgi.framework;version="1.4"
  Bnd-LastModified: 1248816762637
  Export-Package: hickory.example;uses:="javax.script,com.sun.script.jru
   by,org.osgi.framework"
  Bundle-Version: 1.0.0.SNAPSHOT
  Bundle-Name: Hickory
  Build-Jdk: 1.5.0_19
  Private-Package: .
  Bundle-ManifestVersion: 2
  Bundle-SymbolicName: hickory.example.Hickory
  Tool: Bnd-0.0.311
  

  
  


• The Workaound is ...
  
  Hasan found the workaround of the problem (see Using JRuby in OSGi).
  Using Hasan's workaround, I wrote the second snippet.
  
  MANIFEST.MFs of JRuby and JRuby engine are the same as the first try. The differences of the MANIFEST.MF of the second snippet are just Bundle-Activator and Bnd-LastModified lines.
  

  
  Manifest-Version: 1.0
  Built-By: yoko
  Created-By: Apache Maven Bundle Plugin
  Bundle-Activator: hickory.example.Activator1
  Import-Package: com.sun.script.jruby,hickory.example,javax.script,org.
   osgi.framework;version="1.4"
  Bnd-LastModified: 1248818750858
  Export-Package: hickory.example;uses:="javax.script,com.sun.script.jru
   by,org.osgi.framework"
  Bundle-Version: 1.0.0.SNAPSHOT
  Bundle-Name: Hickory
  Build-Jdk: 1.5.0_19
  Private-Package: .
  Bundle-ManifestVersion: 2
  Bundle-SymbolicName: hickory.example.Hickory
  Tool: Bnd-0.0.311
  

  
  This worked well although I'm not sure this is the best. Then, another problem came.
  


• Further problem is ...
  
  JRuby users use Java classes in thier Ruby scripts very often. Thoses classes are usually in differenct jar archives or in classpath that JRuby knows. Here's a further problem happened.
  
  The third snippet raised an exception when I was to instantiate my Java class in Ruby script.

  org.jruby.exceptions.RaiseException: cannot load Java class hickory.example.YellOut

  In the program, Ruby script, "include Java\nputs Java::hickory.example.YellOut.new.whats," is passed to Ruby runtime to be evaluated. Typically, JRuby finds hickory.example.YellOut class out from classpath and instantiates it using JRuby's classloader. But, this process failed on the OSGi container.
  
  Again, the only differences in MANIFEST.MF used here are just Bundle-Activator and Bnd-LastModified lines.
  

  
  Manifest-Version: 1.0
  Built-By: yoko
  Created-By: Apache Maven Bundle Plugin
  Bundle-Activator: hickory.example.Activator2
  Import-Package: com.sun.script.jruby,hickory.example,javax.script,org.
   osgi.framework;version="1.4"
  Bnd-LastModified: 1248820140956
  Export-Package: hickory.example;uses:="javax.script,com.sun.script.jru
   by,org.osgi.framework"
  Bundle-Version: 1.0.0.SNAPSHOT
  Bundle-Name: Hickory
  Build-Jdk: 1.5.0_19
  Private-Package: .
  Bundle-ManifestVersion: 2
  Bundle-SymbolicName: hickory.example.Hickory
  Tool: Bnd-0.0.311
  

  
  


• One more workaround might be ...
  
  I tried the failed example after adding "DynamicImport-Package: *" to JRuby bundle. Now, JRuby's new MANIFEST.MF had "DynamicImport-Package: *" and the third snippet worked.
  
  However, Tommy strongly opposed to add "DynamicImport-Package: *" to JRuby's bundle, and added the comment to http://jira.codehaus.org/browse/JRUBY-3792.
  
  According to Neil, Tommy's workaround works only on SpringSource's dm Server. Then, what is the best way to get these code work on other OSGi containers, for example on Apache Felix?
  

If JSR 223 engine has a flaw in its MANIFEST.MF, I'll fix it to provide a painless API.
I wrote this entry because I couldn't get any relevant information by googling.

What's the ideal way to get JSR223 work on OSGi?

After I wrote the entry, JSR 223 JRuby Engine won't work on OSGi platform, a workaround and an opposition to the workaround were posted to jruby-users ml, which is archived http://www.nabble.com/running-jruby-in-an-osgi-container-td24379565.html. The workaround Hasan found out worked well. But, Tommy opposed because the workaround would cause a tangle of references on some conainter that has multiple types of applications. To avoid this, Tommy advised me to add "Import-Bundle: org.jruby.jruby" to my bundle configuration. I tried Tommy's advise, but it didn't work for me. What's wrong with it?

Still, I haven't figured out how to get JSR 223 JRuby engine on OSGi platform "ideally." Still, I need a help, suggestion, advise, and whatever I can find the ideal way. For ease of tracking the discussion down, I'm going to write what I did along with it.

Versions:

• Java for Mac OS X 10.5 Update 4


java version "1.5.0_19" for making a bundle


java version "1.6.0_13" for starting the bundles


• JRuby 1.3.1


• JSR 223 JRuby Engine 1.1.7


• Apache Felix 1.8.0

Take one: Original Test Program

The first one is the original test program that raises java.lang.ClassNotFoundException: org.jruby.javasupport.Java. I simplified this from the old one seeing Hasan's sample.

- pom.xml

     1 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     2   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
     3   <modelVersion>4.0.0</modelVersion>
     4   <groupId>hickory.example</groupId>
     5   <artifactId>Hickory</artifactId>
     6   <packaging>bundle</packaging>
     7   <version>1.0-SNAPSHOT</version>
     8   <name>Hickory</name>
     9   <url>http://maven.apache.org</url>
    10   <build>
    11     <plugins>
    12       <plugin>
    13         <groupId>org.apache.felix</groupId>
    14         <artifactId>maven-bundle-plugin</artifactId>
    15         <extensions>true</extensions>
    16         <configuration>
    17           <instructions>
    18             <Bundle-Activator>hickory.example.Activator</Bundle-Activator>
    19           </instructions>
    20         </configuration>
    21       </plugin>
    22       <plugin>
    23         <groupId>org.apache.maven.plugins</groupId>
    24         <artifactId>maven-compiler-plugin</artifactId>
    25         <configuration>
    26           <source>1.5</source>
    27           <target>1.5</target>
    28         </configuration>
    29       </plugin>
    30     </plugins>
    31   </build>
    32   <repositories>
    33     <repository>
    34       <id>maven2-repository.dev.java.net</id>
    35       <name>Java.net Repository for Maven</name>
    36       <url>http://download.java.net/maven/2/</url>
    37       <layout>default</layout>
    38     </repository>
    39   </repositories>
    40   <dependencies>
    41     <dependency>
    42       <groupId>org.apache.felix</groupId>
    43       <artifactId>org.osgi.core</artifactId>
    44       <version>1.3.0-SNAPSHOT</version>
    45     </dependency>
    46     <dependency>
    47       <groupId>org.livetribe</groupId>
    48       <artifactId>livetribe-jsr223</artifactId>
    49       <version>2.0.5</version>
    50     </dependency>
    51     <dependency>
    52       <groupId>com.sun.script.jruby</groupId>
    53       <artifactId>jruby-engine</artifactId>
    54       <version>1.1.7</version>
    55     </dependency>
    56     <dependency>
    57       <groupId>junit</groupId>
    58       <artifactId>junit</artifactId>
    59       <version>3.8.1</version>
    60       <scope>test</scope>
    61     </dependency>
    62   </dependencies>
    63 </project>

- Snippet

     1 package hickory.example;
     2 
     3 import com.sun.script.jruby.JRubyScriptEngineFactory;
     4 import javax.script.ScriptEngine;
     5 import javax.script.ScriptEngineFactory;
     6 import org.osgi.framework.BundleActivator;
     7 import org.osgi.framework.BundleContext;
     8 
     9 public class Activator implements BundleActivator {
    10 
    11   public void start(BundleContext context) throws Exception {
    12     System.out.println("Poor Activator");
    13     ScriptEngineFactory factory = (ScriptEngineFactory) new JRubyScriptEngineFactory();
    14     ScriptEngine engine = factory.getScriptEngine();
    15 
    16     System.out.println("Everything should be ready.");
    17     engine.eval("puts \"Yeaaaaaah! See?\"");
    18   }
    19 
    20   public void stop(BundleContext context) {
    21     System.out.println("Bye!");
    22   }
    23 }

- On Apache Felix

cd felix-1.8.0
java -jar bin/felix.jar

Welcome to Felix.
=================

-> ps
START LEVEL 1
  ID   State         Level  Name
[   0] [Active     ] [    0] System Bundle (1.8.0)
[   1] [Active     ] [    1] Apache Felix Shell Service (1.2.0)
[   2] [Active     ] [    1] Apache Felix Shell TUI (1.2.0)
[   3] [Active     ] [    1] Apache Felix Bundle Repository (1.4.0)
-> start http://repo1.maven.org/maven2/org/jruby/jruby-complete/1.3.1/jruby-complete-1.3.1.jar
-> start http://download.java.net/maven/2/com/sun/script/jruby/jruby-engine/1.1.7/jruby-engine-1.1.7.jar
-> start file:///Users/yoko/NetBeansProjects/Hickory/target/Hickory-1.0-SNAPSHOT.jar
Poor Activator
Warning: JRuby home "/4.0:1/META-INF/jruby.home" does not exist, using /var/folders/xY/xYuRYl0RHjy7p6SeA0nHVU+++TI/-Tmp-/
org.osgi.framework.BundleException: Activator start error in bundle hickory.example.Hickory [6].
 at org.apache.felix.framework.Felix.startBundle(Felix.java:1506)
 at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:779)
 at org.apache.felix.shell.impl.StartCommandImpl.execute(StartCommandImpl.java:105)
 at org.apache.felix.shell.impl.Activator$ShellServiceImpl.executeCommand(Activator.java:291)
 at org.apache.felix.shell.tui.Activator$ShellTuiRunnable.run(Activator.java:177)
 at java.lang.Thread.run(Thread.java:637)
Caused by: org.jruby.exceptions.RaiseException: library `java' could not be loaded: java.lang.ClassNotFoundException: org.jruby.javasupport.Java
 at (unknown).initialize(:1)
 at (unknown).(unknown)(:1)
org.jruby.exceptions.RaiseException: library `java' could not be loaded: java.lang.ClassNotFoundException: org.jruby.javasupport.Java
-> 

Take two: Applying Hasan's workaround

According to Hasan's analysis, the snippet above doesn't work because ...

JRuby could not find the class org.jruby.javasupport.Java if run within OSGi environment. So, this is a class loading problem. Tracing the log "could not be loaded" took us from org/jruby/ext/LateLoadingLibrary.java to org/jruby/RubyInstanceConfig.java. In this class we found:

private ClassLoader contextLoader = Thread.currentThread().getContextClassLoader();
private ClassLoader loader = contextLoader == null ? RubyInstanceConfig.class.getClassLoader() : contextLoader;

In an OSGi environment, the Thread.currentThread().getContextClassLoader() of our bundle cannot find the abovementioned java class of JRuby.

So, my second version became below:

- pom.xml
I changed the Activator's class name form Activator to Activator1.

18             <Bundle-Activator>hickory.example.Activator1</Bundle-Activator>

- Snippet

     1 package hickory.example;
     2 
     3 import com.sun.script.jruby.JRubyScriptEngineFactory;
     4 import javax.script.ScriptEngine;
     5 import javax.script.ScriptEngineFactory;
     6 import org.osgi.framework.BundleActivator;
     7 import org.osgi.framework.BundleContext;
     8 
     9 public class Activator1 implements BundleActivator {
    10 
    11   public void start(BundleContext context) throws Exception {
    12     System.out.println("Activator1");
    13     ScriptEngineFactory factory = (ScriptEngineFactory) new JRubyScriptEngineFactory();
    14     final ClassLoader oldClassLoader = Thread.currentThread().getContextClassLoader();
    15     Thread.currentThread().setContextClassLoader(null);
    16     ScriptEngine engine = factory.getScriptEngine();
    17     Thread.currentThread().setContextClassLoader(oldClassLoader);
    18 
    19      System.out.println("Everything should be ready.");
    20      engine.eval("puts \"Yeaaaaaah! See?\"");
    21   }
    22 
    23   public void stop(BundleContext context) {
    24     System.out.println("Bye!");
    25   }
    26 }

Lines 14, 15, and 17 were added to the first one.

- On Apache Felix
After recreating the bundle, I tried this.

-> shutdown

rm -rf felix-cache
java -jar bin/felix.jar 

Welcome to Felix.
=================

-> start http://repo1.maven.org/maven2/org/jruby/jruby-complete/1.3.1/jruby-complete-1.3.1.jar
-> start http://download.java.net/maven/2/com/sun/script/jruby/jruby-engine/1.1.7/jruby-engine-1.1.7.jar
-> start file:///Users/yoko/NetBeansProjects/Hickory/target/Hickory-1.0-SNAPSHOT.jar
Activator1
Warning: JRuby home "/4.0:1/META-INF/jruby.home" does not exist, using /var/folders/xY/xYuRYl0RHjy7p6SeA0nHVU+++TI/-Tmp-/
Everything should be ready.
Yeaaaaaah! See?
->

It worked!
I dare to remove Apache Felix's cache and restart it every time before I try modified bundles. The cache seems to remember something worked before, so I've gotten a different result before and after I removed the cache. It is a bit annoying, but needs to have accurate results.

Take three: Using a defined Java class in Ruby

Everything seems fine, but Tommy brought another problem that the workaround does not work when a java class is used in Ruby script. To try this, I defined the class, hickory.example.YellOut, in the same package as the Activator. Now, test programs are as in below:

- pom.xml

18             <Bundle-Activator>hickory.example.Activator2</Bundle-Activator>

- Snippet

     1 package hickory.example;
     2 
     3 import com.sun.script.jruby.JRubyScriptEngineFactory;
     4 import javax.script.ScriptEngine;
     5 import javax.script.ScriptEngineFactory;
     6 import org.osgi.framework.BundleActivator;
     7 import org.osgi.framework.BundleContext;
     8 
     9 public class Activator2 implements BundleActivator {
    10 
    11   public void start(BundleContext context) throws Exception {
    12     System.out.println("Activator2");
    13     ScriptEngineFactory factory = (ScriptEngineFactory) new JRubyScriptEngineFactory();
    14     final ClassLoader oldClassLoader = Thread.currentThread().getContextClassLoader();
    15     Thread.currentThread().setContextClassLoader(null);
    16     ScriptEngine engine = factory.getScriptEngine();
    17     Thread.currentThread().setContextClassLoader(oldClassLoader);
    18 
    19     System.out.println("Everything should be ready.");
    20     engine.eval("include Java\nputs Java::hickory.example.YellOut.new.whats");
    21   }
    22 
    23   public void stop(BundleContext context) {
    24     System.out.println("Bye!");
    25   }
    26 }

     1 package hickory.example;
     2 
     3 public class YellOut {
     4     public String whats() {
     5         return "I made it!!!";
     6     }
     7 }

- On Apache Felix

-> shutdown
-> Bye!

rm -rf felix-cache
java -jar bin/felix.jar

Welcome to Felix.
=================

-> start http://repo1.maven.org/maven2/org/jruby/jruby-complete/1.3.1/jruby-complete-1.3.1.jar
-> start http://download.java.net/maven/2/com/sun/script/jruby/jruby-engine/1.1.7/jruby-engine-1.1.7.jar
-> start file:///Users/yoko/NetBeansProjects/Hickory/target/Hickory-1.0-SNAPSHOT.jar
Activator2
Warning: JRuby home "/4.0:1/META-INF/jruby.home" does not exist, using /var/folders/xY/xYuRYl0RHjy7p6SeA0nHVU+++TI/-Tmp-/
Everything should be ready.
org.osgi.framework.BundleException: Activator start error in bundle hickory.example.Hickory [6].
 at org.apache.felix.framework.Felix.startBundle(Felix.java:1506)
 at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:779)
 at org.apache.felix.shell.impl.StartCommandImpl.execute(StartCommandImpl.java:105)
 at org.apache.felix.shell.impl.Activator$ShellServiceImpl.executeCommand(Activator.java:291)
 at org.apache.felix.shell.tui.Activator$ShellTuiRunnable.run(Activator.java:177)
 at java.lang.Thread.run(Thread.java:637)
Caused by: javax.script.ScriptException: org.jruby.exceptions.RaiseException: cannot load Java class hickory.example.YellOut
 at com.sun.script.jruby.JRubyScriptEngine.evalNode(JRubyScriptEngine.java:509)
 at com.sun.script.jruby.JRubyScriptEngine.eval(JRubyScriptEngine.java:184)
 at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:247)
 at hickory.example.Activator2.start(Activator2.java:20)
 at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
 at org.apache.felix.framework.Felix.startBundle(Felix.java:1458)
 ... 5 more
Caused by: org.jruby.exceptions.RaiseException: cannot load Java class hickory.example.YellOut
 at (unknown).(unknown)(/builtin/java/ast.rb:49)
 at (unknown).get_proxy_or_package_under_package(/builtin/javasupport/java.rb:51)
 at #.method_missing(:2)
 at (unknown).(unknown)(:1)
javax.script.ScriptException: org.jruby.exceptions.RaiseException: cannot load Java class hickory.example.YellOut
-> 

JRuby failed to load hickory.example.YellOut even though this class is in the same bundle as the Activator. Class loading issue again. I need to let JRuby know where hickory.example.YellOut.class resides by doing something.

Take four: Applying the fix reported in http://jira.codehaus.org/browse/JRUBY-3792.

The filed issue came up in my mind. I thought this might have fix something this sort of problems. So, I recompiled JRuby.

- JRuby recompilation
I added "DynamicImport-Package: *" at end of jruby.bnd.template. Following is entire jruby.bnd.template file.

Export-Package: org.jruby.*;version="@JRUBY_VERSION@"
Import-Package: !org.jruby.*, *;resolution:=optional
Bundle-Version: @JRUBY_VERSION@
Bundle-Description: JRuby @JRUBY_VERSION@ OSGi bundle
Bundle-Name: JRuby @JRUBY_VERSION@
Bundle-SymbolicName: org.jruby.jruby
DynamicImport-Package: *

Then, recompiled JRuby by running "ant jar-complete."

- On Apache Felix

-> shutdown

rm -rf felix-cache
java -jar bin/felix.jar

Welcome to Felix.
=================

-> start file:///Users/yoko/Tools/jruby-1.3.1/lib/jruby-complete.jar
-> start http://download.java.net/maven/2/com/sun/script/jruby/jruby-engine/1.1.7/jruby-engine-1.1.7.jar
-> start file:///Users/yoko/NetBeansProjects/Hickory/target/Hickory-1.0-SNAPSHOT.jar
Activator2
Warning: JRuby home "/4.0:1/META-INF/jruby.home" does not exist, using /var/folders/xY/xYuRYl0RHjy7p6SeA0nHVU+++TI/-Tmp-/
Everything should be ready.
I made it!!!
-> 

Worked!

However, Tommy posed the problem of this way of fixing bundles becuase "DynamicImport-Package: *" would be a culprit of linkage problems when multiple applications and bundles are deployed on a single OSGi container, especially differenct versions of JRuby bundles exists on it. The suggestion was

The better solution, IMHO, is for the script bundle to actually declare its dependency on the jruby-complete bundle either by using Import-Package to bring in all of the packages it needs, or using Import-Bundle to pull in everything exported from the jruby-complete bundle. The first is pretty clearly a non-starter, since there is no way for me to tell which packages from jruby-complete the script bundle is going to need. The second works fine, though, and even allows the script bundle to as for a particular version of the jruby-complete bundle, which is one of the weaknesses of DynamicImport-Package.

Take five: Trying "Import-Bundle: org.jruby.jruby"

Adding "Import-Bundle: org.jruby.jruby" to application's bundle configuration is Tommy's advice. So, I also tried this.

- pom.xml

The line 19 is added to existing pom.xml. As in line 20, I also tried to give Import-Bundle configuration from a separate file, osgi.bnd. Osgi.bnd file has just a line, Import-Bundle: org.jruby.jruby, in it.

     1 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     2   xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
     3   <modelVersion>4.0.0</modelVersion>
     4   <groupId>hickory.example</groupId>
     5   <artifactId>Hickory</artifactId>
     6   <packaging>bundle</packaging>
     7   <version>1.0-SNAPSHOT</version>
     8   <name>Hickory</name>
     9   <url>http://maven.apache.org</url>
    10   <build>
    11     <plugins>
    12       <plugin>
    13         <groupId>org.apache.felix</groupId>
    14         <artifactId>maven-bundle-plugin</artifactId>
    15         <extensions>true</extensions>
    16         <configuration>
    17           <instructions>
    18             <Bundle-Activator>hickory.example.Activator2</Bundle-Activator>
    19             <Import-Bundle>org.jruby.jruby</Import-Bundle>
    20             <!--<_include>src/main/resources/osgi.bnd</_include>-->
    21           </instructions>
    22         </configuration>
    23       </plugin>
    24       <plugin>
    25         <groupId>org.apache.maven.plugins</groupId>
    26         <artifactId>maven-compiler-plugin</artifactId>
    27         <configuration>
    28           <source>1.5</source>
    29           <target>1.5</target>
    30         </configuration>
    31       </plugin>
    32     </plugins>
    33   </build>
    34   <repositories>
    35     <repository>
    36       <id>maven2-repository.dev.java.net</id>
    37       <name>Java.net Repository for Maven</name>
    38       <url>http://download.java.net/maven/2/</url>
    39       <layout>default</layout>
    40     </repository>
    41   </repositories>
    42   <dependencies>
    43     <dependency>
    44       <groupId>org.apache.felix</groupId>
    45       <artifactId>org.osgi.core</artifactId>
    46       <version>1.3.0-SNAPSHOT</version>
    47     </dependency>
    48     <dependency>
    49       <groupId>org.livetribe</groupId>
    50       <artifactId>livetribe-jsr223</artifactId>
    51       <version>2.0.5</version>
    52     </dependency>
    53     <dependency>
    54       <groupId>com.sun.script.jruby</groupId>
    55       <artifactId>jruby-engine</artifactId>
    56       <version>1.1.7</version>
    57     </dependency>
    58     <dependency>
    59       <groupId>junit</groupId>
    60       <artifactId>junit</artifactId>
    61       <version>3.8.1</version>
    62       <scope>test</scope>
    63     </dependency>
    64   </dependencies>
    65 </project>

- On Apache Felix

-> shutdown
-> Bye!

rm -rf felix-cache
java -jar bin/felix.jar

Welcome to Felix.
=================

-> start http://repo1.maven.org/maven2/org/jruby/jruby-complete/1.3.1/jruby-complete-1.3.1.jar
-> start http://download.java.net/maven/2/com/sun/script/jruby/jruby-engine/1.1.7/jruby-engine-1.1.7.jar
-> start file:///Users/yoko/NetBeansProjects/Hickory/target/Hickory-1.0-SNAPSHOT.jar
Activator2
Warning: JRuby home "/4.0:1/META-INF/jruby.home" does not exist, using /var/folders/xY/xYuRYl0RHjy7p6SeA0nHVU+++TI/-Tmp-/
Everything should be ready.
org.osgi.framework.BundleException: Activator start error in bundle hickory.example.Hickory [6].
 at org.apache.felix.framework.Felix.startBundle(Felix.java:1506)
 at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:779)
 at org.apache.felix.shell.impl.StartCommandImpl.execute(StartCommandImpl.java:105)
 at org.apache.felix.shell.impl.Activator$ShellServiceImpl.executeCommand(Activator.java:291)
 at org.apache.felix.shell.tui.Activator$ShellTuiRunnable.run(Activator.java:177)
 at java.lang.Thread.run(Thread.java:637)
Caused by: javax.script.ScriptException: org.jruby.exceptions.RaiseException: cannot load Java class hickory.example.YellOut
 at com.sun.script.jruby.JRubyScriptEngine.evalNode(JRubyScriptEngine.java:509)
 at com.sun.script.jruby.JRubyScriptEngine.eval(JRubyScriptEngine.java:184)
 at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:247)
 at hickory.example.Activator2.start(Activator2.java:20)
 at org.apache.felix.framework.util.SecureAction.startActivator(SecureAction.java:589)
 at org.apache.felix.framework.Felix.startBundle(Felix.java:1458)
 ... 5 more
Caused by: org.jruby.exceptions.RaiseException: cannot load Java class hickory.example.YellOut
 at (unknown).(unknown)(/builtin/java/ast.rb:49)
 at (unknown).get_proxy_or_package_under_package(/builtin/javasupport/java.rb:51)
 at #.method_missing(:2)
 at (unknown).(unknown)(:1)
javax.script.ScriptException: org.jruby.exceptions.RaiseException: cannot load Java class hickory.example.YellOut
-> 

Unfortunately, it didn't work for me though Tommy said it worked.


Tommy also talked about JSR 223 JRuby engine's bundle.

It might be better for the JSR223 bundle to import all of the jruby-complete packages and then re-export them, so bundles like the script bundle would just have to specify a dependency on the JSR223 engine.

So, what is the ideal solution in terms of OSGi?