Rubinius on JRuby … ?

Of course, compiled Rubinius binary, *.rbc, file doesn’t work on JRuby. This post is about JRuby’s rubinius branch. I’m not sure how may people are aware that, but JRuby does have a rubinius branch. As far as I looked at that branch, it is not to be merged into master, at least, in near future. Maybe it is headius’ pet project at this moment, and is implemented as a JRuby extension. Yes, it is a JRuby extension. The branch attempts “extending JRuby,” like I wrote about in my blog post (http://yokolet.blogspot.com/2011/05/extending-jruby.html). Since this extension is still small, it would be a good practice to figure out how extension works. So, I’m going to write how you can decipher existing JRuby extension. Hopefully, this will help you to write your own extension.


To try the rubinius branch, you have two preparations to be done. The first one is to build JRuby of the rubinius branch. It is easy. Just clone out JRuby, checkout rubinius branch, and run ant command as in below:

git clone git://github.com/jruby/jruby.git
cd jruby
git checkout rubinius

ant clean-all
ant

Next, you need Rubinius source. You don’t need to build rubinius just to see how it works. But, the rubinius extension uses Rubinius’ kernal sources, which is included only in the source archive. That’s why you need the source. Rubinius source archive is available to download from http://rubini.us/. Get the archive and unzip it.


Up until now, you had rubinius branch JRuby and Rubinius source. Set the environment variables. Suppose the JRuby’s home directory is /Users/yoko/Projects/jruby, then, set JRUBY_HOME and PATH like in below. Adjust them to fit in to your system:

export JRUBY_HOME=/Users/yoko/Projects/jruby
PATH=$JRUBY_HOME/bin:$JRUBY

Suppose, Rubinius source are in /Users/yoko/Projects/rubinius-1.2.2, then set RBX_KERNEL as in below:

export RBX_KERNEL=/Users/yoko/Projects/rubinius-1.2.2/kernel

Everything should be ready. Let’s try this out.

bash-3.2 rubinius$ jruby -S irb
irb(main):001:0> require 'rubinius'
=> true

Yay! Rubinus was successfully loaded on JRuby. What’s next? Look at the RubiniusLibrary.java (https://github.com/jruby/jruby/blob/rubinius/src/org/jruby/ext/rubinius/RubiniusLibrary.java). At the line 51, “Rubinius” module is defined.

RubyModule rubinius = runtime.getOrCreateModule("Rubinius");

So, there should be a constant, Rubinius.

irb(main):002:0> Rubinius
=> Rubinius
irb(main):003:0> Rubinius.class
=> Module

So far, so good. Then, at the line 56, you can see

RubyTuple.createTupleClass(runtime);

This means you should go to RubyTuple.java (https://github.com/jruby/jruby/blob/rubinius/src/org/jruby/ext/rubinius/RubyTuple.java). On the line 55-62 of RubyTuple.java, createTupleClass method is defined. In this method, "Tuple" class is defined under the "Rubinius" module. Then, annotated methods, which have Java annotation @JRubyMethod, are defined. Looking at the rest of the code in RubyTuple.java, you can see three annotated methods (new, [], and []=), and one override method (dup) are there. Let’s try these.

irb(main):013:0> Rubinius::Tuple
=> Rubinius::Tuple
irb(main):014:0> tuple = Rubinius::Tuple.new 3
=> #<Rubinius::Tuple:0x3df89785>

This constructor needs one argument because rbNew method is defined as:

public static IRubyObject rbNew(ThreadContext context, IRubyObject tupleCls, IRubyObject cnt) {

Here's the rule. First two arguments are given internally, and the rest of the arguments are given from users. So, I typed “3” as an argument. Let’s keep going on.

irb(main):018:0> tuple[0]
=> nil
irb(main):019:0> tuple[0]=123
=> 123
irb(main):020:0> tuple[0]
=> 123
irb(main):021:0> tuple_dup = tuple.dup
=> #
irb(main):022:0> tuple_dup[0]
=> 123

All right, methods worked.


Next, get back to RubiniusLibrary.java and let’s look at the lines 84-88.

runtime.getObject().deleteConstant("Hash");
runtime.getLoadService().lockAndRequire(rbxHome + "/common/hash.rb");
RubyClass hash = (RubyClass)runtime.getClass("Hash");
hash.defineAnnotatedMethods(RubiniusHash.class);
runtime.setHash(hash);

Soooo interesting! Could you figure out what’s going on here? Hash is redefined using Rubinius code! JRuby’s Hash is entirely written in Java (https://github.com/jruby/jruby/blob/rubinius/src/org/jruby/RubyHash.java). But, once “require rubinius” is done, the Hash is totally replaced by Rubinius’ Hash, which IS written in Ruby. See? JRuby can be extended also in Ruby, not just Java. To do double check this, let’s add one line in initialize method of kernel/common/hash.rb:

def initialize(key, key_hash, value)
  @key      = key
  @key_hash = key_hash
  @value    = value
  @next     = nil
  puts "Rubinius Hash!!"   # this line is added
end

Then, restart irb and re-request rubinius.

bash-3.2 rubinius$ jruby -S irb
irb(main):001:0> require 'rubinius'
=> true
irb(main):002:0> h = Hash.new
Rubinius Hash!!
=> {}

Yay, Hash is really Rubinius’ Hash. What an idea!


As I wrote, you have a lot of options for “extending JRuby.” You can extend using mature Java APIs and, also, cutting edge Ruby code. Why don’t you try this fantastic extension? It should be fun.

Extending JRuby

As pure Java Nokogiri does, we can extend JRuby writing a library backed by Java API. Other than Nokogiri, Weakling (https://github.com/headius/weakling), Warbler(https://github.com/nicksieger/warbler), JSON(https://github.com/flori/json) and more are examples of JRuby extension by Java. If you use google code search with a keyword, "BasicLibraryService," you'll find some more gems. This BasicLibraryService is a sign that the gem is implemented by Java. BasicLibraryService is an interface and has just one method, basicLoad(Ruby runtime). Simple. However, questions might come up in people's mind. What should I write in basicLoad method? How is it called? Not many answers are out there. The helpful answer I could find was a comment on LoadService.java (or LoadService19.java for 1.9 mode). But, it would be still short to write a JRuby extension for JRuby users who want to write their own. So, I wrote a sample code to see how JRuby can be extended. This sample is quite a simple one and far from real JRuby extensions such as pure Java Nokogiri or others. But, the first thing is to understand how it works. This sample will help to get started.


Before going deeper, let's look at a usage of Java API directly from JRuby. I chose Apache Commons Math API (http://commons.apache.org/math/). This API is interesting. It makes many mathematical calculations easy and natural. Among them, I picked up a fraction package. Everybody knows. In Japan, elementary school kids study how to add, subtract or common denominator, etc. It should be easy, but neither Java or Ruby doesn't have such API in a standard library.

Below is a JRuby code that uses fraction Java API directly. This code adds up reciprocals of 1 to 4, Harmonic series of n = 4. The answer is obvious, 1/1 + 1/2 + 1/3 + 1/4 = 25/12.

require 'java'
$: << '/Users/yoko/Tools/commons-math-2.2'
require 'commons-math-2.2'

java_import org.apache.commons.math.fraction.Fraction

f = Fraction.new(1, 1)
(2..4).each do |i|
  f = f.add(Fraction.new(1, i))
end
puts f

My commons-math-2.2.jar is in /Users/yoko/Tools/commons-math-2.2 directory, so I added that path to $LOAD_PATH, then, required that jar archive. The .jar suffix is optional when requiring something on JRuby. JRuby searches from every possible paths adding .class, .rb, .jar or .bundle suffixes. Next, I imported Fraction class and calculated in a straightforward way.

Let's think how this code can be improved to more Ruby like one. Ruby programmer might like f.add!(something) rather than f = f.add(something). So, in this JRuby extension sample, I implemented "add!" method.

Firstly, I wrote FractionService class, which implements BasicLibraryService interface. But, wait. API design should come in before starting it because XXXService class works based on convention over configuration. Java's package name and Ruby's module structure must coincide. In my design, the Fraction class is Commons::Math::Fraction::Fraction in Ruby. This means FractionService class should be in a commons.math.fraction package, and require statement in Ruby should be "require 'commons/math/fraction/fraction.' This is how basicLoad() method is called.

Next would what we should write in basicLoad() method. In general, defining module structures/classes and object allocators are done in this method. Ola Bini's blog, "The JRuby Tutorial #4: Writing Java extensions for JRuby" (http://ola-bini.blogspot.com/2006/10/jruby-tutorial-4-writing-java.html) would be worth to read how to write the method. My simple FractionService became as in below:

package commons.math.fraction;

import java.io.IOException;

import org.jruby.Ruby;
import org.jruby.RubyClass;
import org.jruby.RubyModule;
import org.jruby.runtime.ObjectAllocator;
import org.jruby.runtime.builtin.IRubyObject;
import org.jruby.runtime.load.BasicLibraryService;

public class FractionService implements BasicLibraryService {

   @Override
   public boolean basicLoad(Ruby runtime) throws IOException {
       RubyModule commons = runtime.defineModule("Commons");
       RubyModule math = commons.defineModuleUnder("Math");
       RubyModule fractionModule = math.defineModuleUnder("Fraction");
       RubyClass fraction = fractionModule.defineClassUnder("Fraction", runtime.getObject(), FRACTION_ALLOCATOR);
       fraction.defineAnnotatedMethods(Fraction.class);
       return true;
   }
  
   private static ObjectAllocator FRACTION_ALLOCATOR = new ObjectAllocator() {
       public IRubyObject allocate(Ruby runtime, RubyClass klazz) {
           return new Fraction(runtime, klazz);
       }
   };
}

Then, I wrote commons.math.fraction.Fraction class. In this class, I defined "add!" and "to_s" methods. We can't use "!" in a method name in Java, so the Java method name is add_bang instead. Ruby method name is define in @JRubyMethod annotation. Also, I wrote Ruby's constructor method "new," which is "rbNew" method in Java. The "new" method should be a class method, so it is a static method in Java. Annotations of methods are important. Three annotated *JRubyMethods* in Fraction class get fired up by fraction.defineAnnotatedMethods(Fraction.class); in FractionService class. Because of this, we can use Java methods in Ruby. See my Fraction class below:

package commons.math.fraction;

import org.jruby.Ruby;
import org.jruby.RubyClass;
import org.jruby.RubyObject;
import org.jruby.anno.JRubyClass;
import org.jruby.anno.JRubyMethod;
import org.jruby.runtime.Arity;
import org.jruby.runtime.ThreadContext;
import org.jruby.runtime.builtin.IRubyObject;

@JRubyClass(name="Commons::Math::Fraction")
public class Fraction extends RubyObject {
   private org.apache.commons.math.fraction.Fraction j_fraction = null;
  
   @JRubyMethod(name="new", meta = true, rest = true)
   public static IRubyObject rbNew(ThreadContext context, IRubyObject klazz, IRubyObject[] args) {
       Fraction fraction = (Fraction) ((RubyClass)klazz).allocate();
       fraction.init(context, args);
       return fraction;
   }

   public Fraction(Ruby runtime, RubyClass klass) {
       super(runtime, klass);
   }
  
   void init(ThreadContext context, IRubyObject[] args) {
       Arity.checkArgumentCount(context.getRuntime(), args, 2, 2);
       int numerator = (Integer) args[0].toJava(Integer.class);
       int denominator = (Integer) args[1].toJava(Integer.class);
       j_fraction = new org.apache.commons.math.fraction.Fraction(numerator, denominator);
   }
  
   org.apache.commons.math.fraction.Fraction getJFraction() {
       return j_fraction;
   }
  
   @JRubyMethod(name = "add!")
   public IRubyObject add_bang(ThreadContext context, IRubyObject other) {
       if (other instanceof Fraction) {
           org.apache.commons.math.fraction.Fraction other_fraction = ((Fraction)other).getJFraction();
           j_fraction = j_fraction.add(other_fraction);
           return this;
       } else {
           throw context.getRuntime().newArgumentError("argument should be Commons::Math::Fraction type");
       }
   }
  
   @JRubyMethod
   public IRubyObject to_s(ThreadContext context) {
       return context.getRuntime().newString(j_fraction.toString());
   }
}

I haven't written Rakefile for packaging at this moment, so I manually create jar archive of two Java classes.

jar -J-Duser.language=en -cvf ../lib/commons/math/poplar.jar commons

OK, all Java classes are ready for my simple JRuby extension, so let's work on Ruby code. We might have an option to require Java classes directory, but that is not nice. Since users themselves must require FractionService or other, internal change will affect users code. Besides, it doesn't look like Rubygems. So, I wrote commons_math_fraction.rb to hook up FractionService.

require 'commons-math-2.2'
require 'commons/math/fraction'

Surely, this code needs to be brush up, for example, paths. But, I kept simple since this sample is to understand the idea of extending JRuby.
Then, one more Ruby code, commons/math/fraction.rb:

require 'commons/math/poplar'
require 'commons/math/fraction/fraction'

module Commons
 module Math
   module Fraction
   end
 end
end

The first line requires poplar.jar archive, and the second does FractionService class.


Everything is ready, so let's write Ruby code using this tiny, shiny JRuby extension. The file name is fraction_sample.rb:

require 'java'

$: << '/Users/yoko/Documents/workspace/Poplar/lib'
require 'commons_math_fraction'
f = Commons::Math::Fraction::Fraction.new(1, 1)
(2..4).each do |i|
  f.add!(Commons::Math::Fraction::Fraction.new(1, i))
end
puts f

Since my JRuby extension is not yet gem, GEM_PATH or other gem loading ways don't work. I set a load path to my *Poplar* project. Under the lib directory, I have jars and Ruby files.

jruby fraction_sample.rb

gave me the answer 25 / 12.


Like this, we can extend JRuby using Java API. Using Java API under the hood, we can create Rubygems. Some are re-implementation by Java like pure Java Nokgoiri. Others are Java originated Ruby API. JRuby extension is an example that Java effectively complements Ruby. So, add Ruby API to your favorite Java tools.


All codes of this sample are https://github.com/yokolet/Poplar.