JRuby Embed (Red Bridge) Update: termination and skipping sharing variables

Hers' recent update of RedBridge. Performance got much better, but the change on termination might affect your code. If you expect at_exit blocks to be executed automatically, you need to add termination.

By the recent change, Embed Core, JSR223, and BSF, all three implementations had changes in their behaviors of evaluation and method invocation. Termination is no longer executed automatically. This means, at_exit blocks are not executed at the end of every runScriplet, run and callMethod (eval, invokeMethod and invoekeFunction in JSR223). It is effective since commit 673df9f.

This rather big changes was made for two reasons. One is to avoid possible unexpected behavior caused by at_exit blocks to be executed too early. For example, a gem might have a class that has an at_exit block, which should run after other code have finished. The second reason is a performance improvement. Terminate method takes much time to complete. Because of this, evaluation and method invocation of embedding API were very slow. Now, embedding API got much better performance than before.

Then, how to do that? To terminate explicitly, call terminate method on Embed Core and BSF. It's simple. However, JSR223 doesn't have terminate method defined by the specification. So, use newly added attribute, AttributeName.TERMINATION or org.jruby.embed.termination to trun termination on. Next, evaluate blank code.

Usage examples:

[Embed Core]

ScriptingContainer container = null;
try {
    container = new ScriptingContainer();
    conatiner.runScriptlet(PathType.CLASSPATH, testname);
} catch (Throwable t) {
    t.printStackTrace();
} finally {
    container.terminate();
}

[JSR223]

ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("jruby");
engine.eval("$x='GVar'");
engine.eval("at_exit { puts \"#{$x} in an at_exit block\" }");  // nothing is printed here
engine.getContext().setAttribute(AttributeName.TERMINATION.toString(), true, ScriptContext.ENGINE_SCOPE);
engine.eval("");  // prints "GVar in an at_exit block"

Let's move on to the second change. A new option has been added to skip sharing variables. Sharing variables is a useful or required feature for users from JSR223 and BSF background. But, it is not a necessary for others especially who have directly used JRuby's internal API. Sharing variables just slowed down the evaluations and method invocations. When sharing variables is skipped, the performance will be a bit better.

Usage example:

[Embed Core]
container.setAttribute(AttributeName.SHARING_VARIABLES, false);

[JSR223]
engine.getContext().setAttribute(AttributeName.SHARING_VARIABLES.toString(), false, ScriptContext.ENGINE_SCOPE);

Have fun with RedBridge!

Rava - pure Ruby JavaVM

Since JRuby 1.4.0, become_java! method has been available to use to create a real Java class from Ruby class. This new feature always reminds me "Rava," which was written by Koichi Sasada(Ko1) back in 2002. Ko1 is, of course, famous Ruby committer and the author of YARV. When the days that Ruby was infamous while Java was thriving, Ko1 wrote Rava. Although I could not make it work, I think the code itself is still worth glancing at.

- Rava / JavaVM on Ruby (2) (Rava version 0.0.2)
- Rava / JavaVM on Ruby (Rava version 0.0.1)

According to Ko1, Rava is pure Ruby JavaVM and joke software. He said in the article for a Japanese Magazine that Rava could load and interpret a Java class. Rava was not perfect but had basic features, for example:

• interprets most of bytecode


• invokes static/non-static methods


• reads/writes static/non-static fields


• handles exceptions


• runs threads

Also, Ko1 created a prototype of a JIT compiler. He made all of those in a week or so. Ko1 explained that's Ruby.

Here're excerpts from the article about Rava in depth.

• Operand Stack : Rava used Ruby Array to handle Java's operand stack since Ruby Array has enough feature to manage the stack.


• Types and data: Rava mapped Java's primitive types to Ruby's Number or its descendant. Java's reference type was converted into a field and Ruby object that has a reference to the original object. Java's field was mapped to Hash with keys of field names.


• Method invocation : Rava had its method frame as in below. JVM stack was in a single array, which includes operand stack.



[Rava Method Frame]
                         +----------+ --
       operand stack --->|          |  |
       stack pointer --->+----------+  |
  invoker frame info --->|          |  | method fame
                         +----------+  |
  local variable are --->|          |  |
      frame pointer ---> +----------+ --
invoker method frame --> |          |
                         +----------+
                           JVM stack



• JIT compiler : Rava converted bytecode into an equvalent Ruby script. To choose what bytecode should be compiled, Rava had a profiler to count a number of method invocation.

JRuby interprets Ruby on Java, while Rava interprets Java on Ruby. JRuby's become_java! converts Ruby class into Java bytecode, while Rava's JIT compiler converts Java bytecode into Ruby class. Unlike JRuby, Rava was outdated, which is a big difference; however, exploring Rava code might be fun.

Hacking JRuby - add all Hash methods to Map

I recently filed JRUBY-4528, whose patch adds all Ruby's Hash methods to a java.util.Map type object. Applying the patch, I confirmed that I could use "add_ruby_methods" method on Map type object, then, Hash methods for Map object. This would be useful especially for embedding API users since they often want to share Map object between Java and Ruby, back and forth.

What's the problem of current JRuby? When an instance of java.util.HashMap is sent into Ruby code, the object is converted into a "usable" Java object in Ruby world. This is what org.jruby.javasupport.JavaEmbedUtils.javaToRuby() method does, and we can't get Java Map converted into Ruby Hash automatically. Why? People might want to use that object as it is, HashMap type object itself, for other Java APIs used in Ruby. However, no built-in method converts Map to Hash so far although some of methods are added to.

My patch is attempt to add all Hash methods to Map type object by "add_ruby_methods" method.
For example:

irb(main):004:0> require 'java'
=> true
irb(main):005:0> jhash = java.util.HashMap.new
=> {}
irb(main):006:0> jhash.put("1", 100)
=> nil
irb(main):007:0> jhash.put("2", 200)
=> nil
irb(main):008:0> jhash.inspect
=> "{2=200, 1=100}"
irb(main):009:0> rhash = jhash.add_ruby_methods
=> {"2"=>200, "1"=>100}
irb(main):010:0> rhash.inspect
=> "{\"2\"=>200, \"1\"=>100}"
irb(main):011:0> p rhash.values
[200, 100]
=> nil
irb(main):012:0> rhash.merge!({"2"=>222, "3"=>333}) 
=> {"3"=>333, "2"=>222, "1"=>100}
irb(main):013:0> jhash.inspect
=> "{3=333, 2=222, 1=100}"

On jirb, I created java.util.HashMap object and put two key-value pairs using Java API, which was inspected by automatically added "inspect" method while converting. Then, I used add_ruby_methods method. After that, I could use Hash's inspect, values and merge! methods. Operations for "rhash" object above is also operations to "jhash," so when I inspected jhash, key-value pairs were also updated.

What if I create a Map object in Java code and give it to Ruby? Key-value pairs in a Java Map object was completely manipulated by Ruby. For example, see code below:

ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
ConcurrentHashMap map1 = new ConcurrentHashMap();
map1.put("a", 100);
map1.put("b", 200);
Map map2 = new HashMap();
map2.put("b", 254);
map2.put("c", 300);
container.put("h1", map1);
container.put("h2", map2);
container.put("num", 0);
String script = 
    "rh = h1.add_ruby_methods\n" +
    "puts \"num: #{num}\"\n" +
    "rh.merge!(h2.add_ruby_methods) {|k,o,n| num += 1; o+n }";
container.runScriptlet(script);
Set entries = map1.entrySet();
for (Map.Entry entry : entries) {
    System.out.print(entry.getKey() + ": " + entry.getValue() + ", ");
}

outputs:

b: 454, a: 100, c: 300,

As you see, merge! method worked. All java.util.Map type such as ConcurrentHashMap or TreeMap are available to apply the method.

Possible problem of this attempt is that contents of an object after add_ruby_methods applied are Java objects. For this reason, a direct comparison to a Ruby Hash object fails. In this case, to_hash method would work since to_hash method returns real Ruby Hash of converted Java Map.

This is just an attempt, and I'm not sure this patch will be applied or not. If you think this is useful, leave a comment on JIRA.

JRuby Embed (Red Bridge) Gotchas: on jirb

I haven't used like that before, but there are people who want to use JRuby Embed API on jirb. I fixed a bug, http://jira.codehaus.org/browse/JRUBY-4521, and tried what I could do on jirb.

At first, I instantiated ScriptingContainer and checked what initial parameters were set.

irb(main):001:0> require 'java'
=> true
irb(main):002:0> container = org.jruby.embed.ScriptingContainer.new
=> org.jruby.embed.ScriptingContainer@ea7549
irb(main):003:0> p container.get_home_directory                    
"/Users/yoko/DevSpace/jruby~main"
=> nil
irb(main):004:0> p container.load_paths
[/Users/yoko/DevSpace/jruby~main/lib/profile.jar, /Users/yoko/NetBeansProjects/cirrus/build/classes]
=> nil
irb(main):005:0> p container.class_loader
org.jruby.util.JRubyClassLoader@5e2075
=> nil
irb(main):006:0> p container.current_directory
"/Users/yoko/NetBeansProjects/cirrus"
=> nil
irb(main):007:0> p container.compat_version
RUBY1_8
=> nil
irb(main):008:0> p container.supported_ruby_version
"jruby 1.5.0.dev (ruby 1.8.7 patchlevel 174) (2010-02-02 0505fb1) (Java HotSpot(TM) Client VM 1.5.0_22) [i386-java]"
=> nil

Hmmm.... interesting. Of course, no compilation at all. Perhaps, ScriptingContainer's API is useful to see jirb internal settings.

Then, how evaluations go?

irb(main):009:0> script = "puts \"Hello World\""
=> "puts \"Hello World\""
irb(main):010:0> container.run_scriptlet(script)
Hello World
=> nil
irb(main):011:0> message = "Hi, there!"
=> "Hi, there!"
irb(main):012:0> container.put("message", message)
=> nil
irb(main):013:0> container.run_scriptlet("puts \"message: #{message}\"")
message: Hi, there!
=> nil

OK, evaluations as well as sharing variables between Java(?) (or jirb?) and Ruby seem to work.
How about method call?

irb(main):014:0> script = "def say\nputs \"oh!\"\nend"
=> "def say\nputs \"oh!\"\nend"
irb(main):015:0> recv = container.run_scriptlet(script)
=> nil
irb(main):016:0> container.call_method(recv, "say", java.lang.Object.class)
:1:in `say': wrong # of arguments(1 for 0) (ArgumentError)
 from :1
NativeException: org.jruby.embed.InvokeFailedException: wrong # of arguments(1 for 0)
 from org/jruby/embed/internal/EmbedRubyObjectAdapterImpl.java:387:in `call'
 from org/jruby/embed/internal/EmbedRubyObjectAdapterImpl.java:326:in `callMethod'
 from org/jruby/embed/ScriptingContainer.java:1268:in `callMethod'
 from :1

No, it failed. This is because jirb chose "public Object callMethod(Object receiver, String methodName, Object... args)" for callMethod. Unfortunately, in this case, Ruby doesn't know the difference of several callMethod methods.
Ok, then, no argument for "say" method. Will it work?

irb(main):017:0> container.call_method(recv, "say")                        
CallableSelector.java:196:in `assignableOrDuckable': java.lang.ArrayIndexOutOfBoundsException: 2
 from CallableSelector.java:22:in `access$200'
 from CallableSelector.java:163:in `accept'
 from CallableSelector.java:101:in `findCallable'
 from CallableSelector.java:86:in `findMatchingCallableForArgs'
 from CallableSelector.java:39:in `matchingCallableArityN'
 from RubyToJavaInvoker.java:170:in `findCallable'
 from InstanceMethodInvoker.java:29:in `call'
 from InstanceMethodInvoker.java:67:in `call'
 from AliasMethod.java:66:in `call'
 from CachingCallSite.java:329:in `cacheAndCall'
...
...

Oh dear, jirb was blown up.

So far, ScriptingContainer on jirb doesn't work enough but might be fun for quick hack.

........

Wrap this up.

We have java_send method and can specify exact Java method by its argument, but this also didn't work well. When I tried to run "volume" method of this Ruby code:

def volume(r)
  4.0 / 3.0 * Math::PI * r ** 3.0
end

irb(main):008:0> ret = container.run_scriptlet(org.jruby.embed.PathType::CLASSPATH, "ruby/sphere.rb")
=> nil
irb(main):009:0> container.java_send :callMethod, [java.lang.Object, java.lang.String, java.lang.Class], self, "volume", java.lang.Integer.new(3)
TypeError: for method ScriptingContainer.callMethod expected [class java.lang.Object, class java.lang.String, class java.lang.Class]; got: [org.jruby.RubyObject,java.lang.String,java.lang.Integer]; error: argument type mismatch
 from :1

like in the above, I got TypeError. A receiver object was the problem. Java program could cast java.lang.RubyObject to java.lang.Object, but jirb could not.

........

Wrap up, part 2.

While testing ScriptingContainer on jirb, I found a bug. Singleton model didn't see the same RubyInstanceConfig when Ruby runtime had already instantiated preceding ScriptingContainer. After the fix, some of runtime configurations can be changed through ScriptingContainer's methods. For example, I could change Ruby version to be used. The Ruby code below uses a block local variable introduced in Ruby 1.9.

# This snippet is borrowed from http://gihyo.jp/dev/serial/01/ruby/0003
# defines local variable x
x = "bear"

# A block local variable x is used in this block. (Two "x"s work together)
["dog", "cat", "panda"].each do |x|
  # This x is a block local variable.
  p x
  break if x == "cat"
end

# This x is a local variable since it is used outside of the block.
p x

When I tried this code by setting both Ruby 1.8 and 1.9, I got appropriate outputs for both mode on jirb.

irb(main):001:0> require 'java'
=> true
irb(main):002:0> container = org.jruby.embed.ScriptingContainer.new
=> org.jruby.embed.ScriptingContainer@ea7549
irb(main):003:0> container.compat_version
=> RUBY1_8
irb(main):004:0> container.run_scriptlet(org.jruby.embed.PathType::CLASSPATH, "ruby/block-param-scope.rb")
"dog"
"cat"
"cat"
=> nil
irb(main):005:0> container.set_compat_version(org.jruby.CompatVersion::RUBY1_9)
=> nil
irb(main):006:0> container.compat_version
=> RUBY1_9
irb(main):007:0> container.run_scriptlet(org.jruby.embed.PathType::CLASSPATH, "ruby/block-param-scope.rb")
"dog"
"cat"
"bear"
=> nil

So, again, SciprtingContainer on jirb is interesting. :)