Showing posts with label RedBridge. Show all posts
Showing posts with label RedBridge. Show all posts

Tuesday, September 06, 2011

Sinatra on RedBridge

Trinidad (http://thinkincode.net/trinidad/), Kirk (https://github.com/strobecorp/kirk), TorqueBox (http://torquebox.org/), mizuno (https://github.com/matadon/mizuno ) and perhaps some more are out there. As you know, those hook up Rails and/or Sinatra app on Java web servers. They are all easy to use tools even for people don't have Java background. On the other hand, there are people who want to write Java Servlet and control Rails/Sinatra apps on a Servlet, like me. :)


In the past, to make Rails apps controllable on Servlet, I've attempted Rails on RedBridge a couple of times. For example, The second step to Rails on RedBridge and Rails on RedBridge, Scaffolded App to Work are. In those attempts, I tried to invoke rack middleware's method, call, directly using RedBrdige. It was possible because Rails' controllers have a "call(env)" method, the one of a rack middleware. The idea was simple; however, it revealed that creating "env" argument was not so simple. In my past attempts, I somehow created "env" argument on Servlet and succeeded to make just a simple app to work. But, for more complicated database models or complicated HTTP requests, that was not enough. Thus, in this attempt, I used JRuby-Rack to create "env", of course, on the Servlet. A web framework is now Sinatra. Since Sinatra is much simpler than Rails, it is easy to try my idea out.


My attempt has done in three steps. The first step is really simple and just checks whether it works. I created a Java Web Application project on Eclipse whose name is Walnut. Java web server is Tomcat 7.0.4. Nothing is special to create the project so far. After I created the project, I put jruby-complete.jar in WEB-INF/lib under the web app directory tree. On Eclipse, the directory is located in WebContent/WEB-INF/lib, which varies on IDEs. Then, set the build path on Eclipse so that jruby-complete.jar is used in both compiling and executing. Before writing a Servlet, I installed gems in the Web application directory tree.

cd WebContent/WEB-INF/lib
mkdir -p jruby/1.8
export GEM_HOME=`pwd`/jruby/1.8
java -jar jruby-complete.jar -S gem install bundler --no-ri --no-rdoc -i jruby/1.8
java -jar jruby-complete.jar -S jruby/1.8/bin/bundle init
vi Gemfile
java -jar jruby-complete.jar -S jruby/1.8/bin/bundle install --path=.

Gemfile is below:

# A sample Gemfile
source "http://rubygems.org"

gem "sinatra"
gem "jruby-rack"


The first Servlet I wrote was below, which can be seen at HelloJRuby.java:

1 package walnut;
2
3 import java.io.File;
4 import java.io.IOException;
5 import java.util.ArrayList;
6 import java.util.List;
7 import java.util.Map;
8 import java.util.concurrent.ConcurrentHashMap;
9
10 import javax.servlet.ServletConfig;
11 import javax.servlet.ServletException;
12 import javax.servlet.annotation.WebServlet;
13 import javax.servlet.http.HttpServlet;
14 import javax.servlet.http.HttpServletRequest;
15 import javax.servlet.http.HttpServletResponse;
16
17 import org.jruby.embed.LocalContextScope;
18 import org.jruby.embed.ScriptingContainer;
19
20 /**
21 * Servlet implementation class HelloJRuby
22 */
23 @WebServlet("/HelloJRuby")
24 public class HelloJRuby extends HttpServlet {
25 private static final long serialVersionUID = 1L;
26 private List<String> gemPaths;
27 private ServletConfig config;
28 private ScriptingContainer container;
29
30 /**
31 * @see HttpServlet#HttpServlet()
32 */
33 public HelloJRuby() {
34 super();
35 gemPaths = new ArrayList<String>();
36 container = new ScriptingContainer(LocalContextScope.THREADSAFE);
37 }
38
39 private void addGemPaths(String gem_path) {
40 File gem_dir = new File(gem_path);
41 File[] gems = gem_dir.listFiles();
42 for (File gem : gems) {
43 String path = gem + "/lib";
44 gemPaths.add(path);
45 }
46 }
47
48 public void init(ServletConfig config) {
49 this.config = config;
50 String path = config.getServletContext().getRealPath("/WEB-INF/lib/jruby/1.8/gems");
51 addGemPaths(path);
52 }
53
54 /**
55 * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
56 */
57 protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
58 processHttpRequest(request, response);
59 }
60
61 /**
62 * @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
63 */
64 protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
65 processHttpRequest(request, response);
66 }
67
68 private void processHttpRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
69 container.setLoadPaths(gemPaths);
70 String class_def =
71 "require 'rubygems'\n" +
72 "require 'sinatra/base'\n" +
73 "class MyApp < Sinatra::Base\n" +
74 " get '/' do\n" +
75 " 'Hello from Sinatra'\n" +
76 " end\n" +
77 "end\n" +
78 "MyApp.new";
79 Object myApp = container.runScriptlet(class_def);
80 Map<String, String> minimal_env = new ConcurrentHashMap<String, String>();
81 minimal_env.put("PATH_INFO", "/");
82 minimal_env.put("REQUEST_METHOD", "GET");
83 minimal_env.put("rack.input", "");
84 List rack_response = (List)container.callMethod(myApp, "call", minimal_env, List.class);
85 response.getWriter().print(rack_response.get(2));
86 container.clear();
87 }
88 }

The Sinatra app in HelloJRuby Servlet would be the simplest one with a minimal env argument. What I wanted to test in this Servlet was whether gems were correctly loaded or not. Since a web application must be portable, every path must be relative to a Servlet context. The line 50 gets the path to gems, which is relative to the context, then the paths are saved in a List, "gemPaths" . This gemPaths is set to the ScriptingContainer in line 69. I ran this Servlet on Eclipse and got the string "Hello from Sinatra" on a browser.


The second Servlet uses JRuby-Rack instead of creating rack request directly. This needs a little trick to make JRuby-Rack work. I didn't want to use whole lot of JRuby-Rack but just a part of it to create a rack request on a Servlet. The second Servlet is below, which is also HelloRack.java on Github:

1 package walnut;
2
3 import java.io.File;
4 import java.io.IOException;
5 import java.util.ArrayList;
6 import java.util.List;
7
8 import javax.servlet.ServletConfig;
9 import javax.servlet.ServletException;
10 import javax.servlet.annotation.WebServlet;
11 import javax.servlet.http.HttpServlet;
12 import javax.servlet.http.HttpServletRequest;
13 import javax.servlet.http.HttpServletResponse;
14
15 import org.jruby.embed.LocalContextScope;
16 import org.jruby.embed.ScriptingContainer;
17
18 /**
19 * Servlet implementation class HelloRack
20 */
21 @WebServlet("/HelloRack")
22 public class HelloRack extends HttpServlet {
23 private static final long serialVersionUID = 1L;
24 private List<String> gemPaths;
25 private ServletConfig config;
26 private ScriptingContainer container;
27
28 /**
29 * @see HttpServlet#HttpServlet()
30 */
31 public HelloRack() {
32 super();
33 gemPaths = new ArrayList<String>();
34 container = new ScriptingContainer(LocalContextScope.THREADSAFE);
35 }
36
37 private void addGemPaths(String gem_path) {
38 File gem_dir = new File(gem_path);
39 File[] gems = gem_dir.listFiles();
40 for (File gem : gems) {
41 String path = gem + "/lib";
42 gemPaths.add(path);
43 }
44 }
45
46 public void init(ServletConfig config) {
47 this.config = config;
48 String path = config.getServletContext().getRealPath("/WEB-INF/lib/jruby/1.8/gems");
49 addGemPaths(path);
50 }
51
52 /**
53 * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
54 */
55 protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
56 processHttpRequest(request, response);
57 }
58
59 /**
60 * @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
61 */
62 protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
63 processHttpRequest(request, response);
64 }
65
66 private void processHttpRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
67 container.setLoadPaths(gemPaths);
68 String class_def =
69 "require 'rubygems'\n" +
70 "require 'sinatra/base'\n" +
71 "class MyApp < Sinatra::Base\n" +
72 " get '/' do\n" +
73 " 'Hello from Sinatra over JRuby-Rack'\n" +
74 " end\n" +
75 "end\n" +
76 "MyApp.new";
77 Object myApp = container.runScriptlet(class_def);
78 container.put("rack_app", myApp);
79 String creates_handler =
80 "require 'jruby-rack'\n" +
81 "require 'rack/handler/servlet'\n" +
82 "Rack::Handler::Servlet.new rack_app";
83 Object handler = container.runScriptlet(creates_handler);
84 container.put("handler", handler);
85 container.put("request", request);
86 container.put("config", config);
87 String calls_app =
88 "request.instance_variable_set(:@context, config.getServletContext)\n" +
89 "class << request\n" +
90 " def to_io\n" +
91 " self.getInputStream.to_io\n" +
92 " end\n" +
93 " def getScriptName\n"+
94 " self.getPathTranslated\n" +
95 " end\n" +
96 " def context\n" +
97 " @context\n" +
98 " end\n" +
99 "end\n" +
100 "handler.call(request)";
101 Object rack_response = container.runScriptlet(calls_app);
102 String body = (String)container.callMethod(rack_response, "getBody", String.class);
103 response.getWriter().print(body);
104 container.clear();
105 }
106 }

Line 79-83 creates Rack Servlet handler. This is a part of JRuby-Rack and nothing special. Following part of line 84-101 is a little hack. Basically, JRuby-Rack uses an instance of HttpServletRequest as an argument of "call" method. However, some methods are lacked. The snippet here adds those methods in Ruby way, dynamically to the instance. The return value at line 101 is an instance of JRuby::Rack::Response type (response.rb). So, I called "getBody" method of the returned object. We can call other methods of JRuby::Rack::Response in the same way. When I ran this Servlet on Eclipse, I got the expected result. JRuby-Rack worked.


The third step is the one leads to a real Sinatra app. In HelloRack Servlet, I wrote whole Sinatra app in the Servlet as a string. Nobody does this to create a real app. So, the third Servlet, HelloSinatraApp, uses config.ru file, below:

require 'my_app'
run MyApp

"my_app.rb" is:

require 'sinatra/base'

class MyApp < Sinatra::Base
get '/' do
'Hello from Sinatra App over JRuby-Rack!'
end
end

I put both config.ru and my_app.rb files in WEB-INF/lib/app directory. This means I need to add WEB-INF/lib/app to a load path. The HelloSinatraApp Servlet is below, or HelloSinatraApp.java on Github:

1 package walnut;
2
3 import java.io.File;
4 import java.io.IOException;
5 import java.util.ArrayList;
6 import java.util.List;
7
8 import javax.servlet.ServletConfig;
9 import javax.servlet.ServletException;
10 import javax.servlet.annotation.WebServlet;
11 import javax.servlet.http.HttpServlet;
12 import javax.servlet.http.HttpServletRequest;
13 import javax.servlet.http.HttpServletResponse;
14
15 import org.jruby.embed.LocalContextScope;
16 import org.jruby.embed.ScriptingContainer;
17
18 /**
19 * Servlet implementation class HelloSinatraApp
20 */
21 @WebServlet("/HelloSinatraApp")
22 public class HelloSinatraApp extends HttpServlet {
23 private static final long serialVersionUID = 1L;
24 private List<String> gemPaths;
25 private ServletConfig config;
26 private ScriptingContainer container;
27 private String config_ru_path;
28
29 /**
30 * @see HttpServlet#HttpServlet()
31 */
32 public HelloSinatraApp() {
33 super();
34 gemPaths = new ArrayList<String>();
35 container = new ScriptingContainer(LocalContextScope.THREADSAFE);
36 }
37
38 private void addGemPaths(String gem_path) {
39 File gem_dir = new File(gem_path);
40 File[] gems = gem_dir.listFiles();
41 for (File gem : gems) {
42 String path = gem + "/lib";
43 gemPaths.add(path);
44 }
45 }
46
47 public void init(ServletConfig config) {
48 this.config = config;
49 String path = config.getServletContext().getRealPath("/WEB-INF/lib/jruby/1.8/gems");
50 addGemPaths(path);
51 gemPaths.add(config.getServletContext().getRealPath("/WEB-INF/lib/app"));
52 config_ru_path = config.getServletContext().getRealPath("/WEB-INF/lib/app/config.ru");
53 }
54
55 /**
56 * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
57 */
58 protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
59 processHttpRequest(request, response);
60 }
61
62 /**
63 * @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
64 */
65 protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
66 processHttpRequest(request, response);
67 }
68
69 private void processHttpRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
70 container.setLoadPaths(gemPaths);
71 container.runScriptlet("require 'rubygems'");
72 container.put("config_ru_path", config_ru_path);
73 String creates_handler =
74 "require 'rack'\n" +
75 "rack_app, options = Rack::Builder.parse_file config_ru_path\n" +
76 "require 'jruby-rack'\n" +
77 "require 'rack/handler/servlet'\n" +
78 "Rack::Handler::Servlet.new rack_app";
79 Object handler = container.runScriptlet(creates_handler);
80 container.put("handler", handler);
81 container.put("request", request);
82 container.put("config", config);
83 String calls_app =
84 "request.instance_variable_set(:@context, config.getServletContext)\n" +
85 "class << request\n" +
86 " def to_io\n" +
87 " self.getInputStream.to_io\n" +
88 " end\n" +
89 " def getScriptName\n"+
90 " self.getPathTranslated\n" +
91 " end\n" +
92 " def context\n" +
93 " @context\n" +
94 " end\n" +
95 "end\n" +
96 "handler.call(request)";
97 Object rack_response = container.runScriptlet(calls_app);
98 String body = (String)container.callMethod(rack_response, "getBody", String.class);
99 response.getWriter().print(body);
100 container.clear();
101 }
102
103 }

Let's see. The path to WEB-INF/lib/app is added in line 51. The path to config.ru is set in line 52. Line 73-79 creates Sinatra app instance and Rack Servlet handler. Other part is the same as the second Servlet, HelloRack. Again, I got the result I expected from this Servlet. I may write paths to gems and apps in web.xml to make this Servlet configurable.


Like in the above, Sinatra on RadBridge worked! This needs a knowledge of Servlet and Java web application. But, the good side is the app won't choose web servers. The app is an ordinary Java web application and works on any Servlet based web application. If you want to write a Servlet for some reasons, like me, this would be the choice.

Wednesday, July 13, 2011

JRuby on Heroku via Clojure

The big news about "Matz to Heroku" reminded me another news from Heroku. That's "Clojure on Heroku"!!! Yes, this news was for me, a JVM language lover. JVM has started running on Heroku, which means all JVM languages run on Heroku via Clojure. When I heard the news, I thought I should have tried that. So, today, I actually tried to run JRuby from Clojure. Happily, JRuby easily ran on Heroku.

I read these two, Clojure on Heroku and Getting Started With Clojure on Heroku/Cedar, and tried. These are good documents to get started. Following the documents, I wrote project.clj below:

(defproject hello-world "0.0.1"
:dependencies
[[org.clojure/clojure "1.2.1"]
[org.jruby/jruby-complete "1.6.3"]
[ring/ring-jetty-adapter "0.3.9"]])

This dependency is parsed by Leiningen(https://github.com/technomancy/leiningen). The format is [maven's groupId/ArtifactId version], so I added [org.jruby/jruby-complete "1.6.3"] in project.clj. Next, I edited web.clj as in below:

(comment "filename: web.clj")

(ns demo.web
(:use ring.adapter.jetty))

(import '(org.jruby.embed ScriptingContainer))
(def c (ScriptingContainer.))
(def version (. c runScriptlet "JRUBY_VERSION"))

(defn app [req]
{:status 200
:headers {"Content-Type" "text/plain"}
:body (str "Hello JRuby " version " from Clojure!")})

(defn -main []
(let [port (Integer/parseInt (System/getenv "PORT"))]
(run-jetty app {:port port})))

In this file, I imported org.jruby.embed.ScriptingContainer class. It is RedBridge. Then, I instantiated ScriptingContainer and evaluated JRUBY_VERSION constant which was assigned to "version" so that I could use later.

When I uploaded this project and requested from a browser, I could see the JRuby's version.
OK, JRuby worked!


I tried one more since just showing JRuby version is too simple. The second attempt was somehow proxy like code using Ruby's open-uri standard library. I wrote src/demo/jruby.clj as in below:

(comment "filename: jruby.clj")

(ns demo.jruby
(:use ring.adapter.jetty))

(import '(org.jruby.embed ScriptingContainer))
(def c (ScriptingContainer.))

(defn app [req]
{:status 200
:headers {"Content-Type" "text/html"}
:body (. c runScriptlet "require 'rubygems'; require 'open-uri'; f = open('http://www.ruby-lang.org/').read")})

(defn -main []
(let [port (Integer/parseInt (System/getenv "PORT"))]
(run-jetty app {:port port})))

In jruby.clj, I changed text/plain to text/html of response header and, in response body part, put a result of evaluating a short Ruby code. As you know, this tiny Ruby code reads HTML from http://www.ruby-lang.org/ and returns the contents. Then, changed Procfile as in below because the file is jruby.clj this time.

Procfile

web: lein run -m demo.jruby

Again, upload all to Heroku. Browsing the url, I got the output below:
Yes! It worked!



My next attempt will be Rubygems to Clojure on Heroku. I think Rubygems will possibly cover something missing in Clojure.



You might be interested about using Rubygems from Clojure. My presentation at RubyConf 2010, "Rubygems to All JVM Languages" (slide: http://servletgarden-point.appspot.com/slideshow, sample code: https://github.com/yokolet/rubyconf2010), might help you understand.

Monday, September 20, 2010

New featues of embedding API for JRuby 1.6

This month, a lot of work for JRuby's embedding API (RedBridge) has been done. Mainly, bug fixing. While I was fixing bugs, I eventually landed to add a new feature and change the area of sharing variables. These will be in JRuby 1.6. Currently, snapshot is available at http://ci.jruby.org/snapshots/ if you want to test it. I believe the changes are good to use Ruby more naturally, but those might affect the code already written a little bit. That why I'm writing this. If you are the user of JRuby's embedding API or JSR223, be aware of upcoming changes.


1. Sharing variables becomes a receiver sensitive

Before the changes, embedding API didn't mind the difference of receivers. A receiver means Ruby's receiver, which is returned as a result of evaluation. Variables and constants to be shared are injected to a top level, in other words, runtime's top self. Also, those should be retrieved from the top level variables and constants. However, this logic didn't work perfectly.

Firstly, a trouble happened in sharing instance variables. The reason is that embedding API didn't use consistent receivers to inject and retrieve instance variables. When multiple receivers were involved, multiple values were assigned to the same key. This ended up unwanted results occasionally.

In light of this, I added three methods to ScriptingContainer.

get(receiver, key)
put(receiver, key, value)
remove(receiver, key, value)

These methods explicitly interact with a given receiver. Existing get/put/remove methods will have top self receivers for the argument.

Let me show you example. I'm going to use the Ruby code, tree_sample.rb below:

class Tree
attr_accessor :name, :shape, :type

def initialize name, shape, type
@name = name
@shape = shape
@type = type
end

def name= name
@name = name
end

def to_s
"#{name.capitalize} is a(n) #{shape} shaped, #{type} tree."
end
end

When the code is evaluated by:

ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
container.runScriptlet(PathType.CLASSPATH, "ruby/tree_sample.rb");

The runtime caches the Tree class and returns nil, which is converted to null for Java code. Then, suppose two objects are instantiated:

Object tree1 = container.runScriptlet("Tree.new('any', 'pyramidal', 'evergreen')");
Object tree2 = container.runScriptlet("Tree.new('any', 'oval', 'deciduous')");

Tree1 and tree2 above are receivers. Before the changes, container retrieved instance variables from the receivers and saved in an internal variable table tied to the instance name at the end of runScriptlet method. As you know, there're two objects. Two values of each instance variable were assigned to the single key without any receiver info. The retrieval should be receiver sensitive as well as injecting.

After the change, the instance variable values are tied to both the key and receiver. In JRuby 1.6, you'll get expected results even though multiple receivers of the same class are there. For example, suppose callMethod methods are run for each receiver object:

container.callMethod(tree1, "name=", "pine");
container.callMethod(tree2, "name=", "poplar");
System.out.println(container.callMethod(tree1, "to_s", String.class));
System.out.println(container.callMethod(tree2, "to_s", String.class));

The result is the one expected:

Pine is a(n) pyramidal shaped, evergreen tree.
Poplar is a(n) oval shaped, deciduous tree.

For clarity, let me add more lines of sharing variable related method usages:

container.put(tree1, "@name", "camellia");
container.put(tree2, "@name", "cherry");
container.put(tree1, "@shape", "oval");
container.put(tree2, "@shape", "round");

System.out.println(container.callMethod(tree1, "to_s", String.class));
System.out.println(container.callMethod(tree2, "to_s", String.class));

System.out.println("@type of tree1: " + container.get(tree1, "@type"));
System.out.println("@type of tree2: " + container.get(tree2, "@type"));

Above prints:

Camellia is a(n) oval shaped, evergreen tree.
Cherry is a(n) round shaped, deciduous tree.
@type of tree1: evergreen
@type of tree2: deciduous


So far, I talked about instance variables. Constants are also receiver sensitive in JRuby 1.6. Look at the code below:

ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
String script =
"COLOR = 'pink'\n" +
"class ColorSample\n" +
" COLOR = 'orange'\n" +
"end\n" +
"ColorSample.new";
Object receiver = container.runScriptlet(script);
System.out.println("top level: " + container.get("COLOR"));
System.out.println("class: " + container.get(receiver, "COLOR"));

There're two constants which have the same name, COLOR. While container.get("COLOR")) gets a top level contant, container.get(receiver, "COLOR")) gets a constant from a given receiver. So, the output is:

top level: pink
class: orange


How about global and local variables? Global variables are receiver insensitive because those should be referenced globally. Local variables are always injected to/retrieved from the top level. It might have been possible to tie to the receiver, but that is not a good idea because Ruby code might use gems or third party libraries. The injected local variables from Java might be extraterrestrials for those libraries and cause unexpected results.

For JSR223 user, I added an "org.jruby.embed.receiver" attribute, but this needs more work to make it happen.


2. Variable/constant value retrieval becomes lazy by default


Before the change ScriptingContainer tried to retrieve variable/constant values as much as possible at the end of runScriptlet/callMethod and EvalUnit.run methods. This was convenient to get variables/constants defined in Ruby ready for Java. However, too many values were saved in the internal variable table. Those variables were injected to Ruby code in succeeding evaluations implicitly. Obviously, this behavior made performance and memory usage worse. When multiple gems were used, it would be serious.

In JRuby 1.6, all variables/constants are retrieved lazily except persistent local variables. This means the internal variable table will have minimum key-value pairs. When a get method of ScriptingContainer is called, the requested key-value pair is saved in the variable table. When key-value pairs have been put to the internal variable table before the evaluation, those will be updated right after the evaluation. Others are not. However, persistent local variables are exception. The values are retrieved eagerly as it was done before. This is because of the internal of JRuby and is helped by the policy that local variables to be shared are only top level ones. Probably, I can consider top level local variables are few.

The change will bring a good result; however, JSR223 users might be affected. The existence of JSR223's SimpleBindings and SimpleScriptContext are always headache to me. It's very hard to add a trick to those two. Workaround is to set false in lazy option and to use the method, ScriptEngine#getContext().getAttrubute() method. Otherwise, set a key and dummy value pair to the bindings. Then, JRubyEngine updates the value right after the evaluation. By the release of 1.6, I'll improve JSR223 support.


By this change, users program will be more natural in terms of Ruby coding even on JRuby's embedding API. My work on these has not yet finished. I'll add more tests and some measures to make these on JSR223.

Saturday, September 04, 2010

My Presentation at JRubyKaigi 2010

I spoke at JRubyKaigi 2010 about JRuby embed API (RedBridge). Since attendees were almost all Japanese, I spoke in Japanese. However, my slide was written using simple English words, so that even non-Japanese speanker could understand what's going on. Don't worry about the Language of my slide.

Now, you can see my slide at http://servletgarden-point.appspot.com/slideshow. As the url shows, my slide itself was a demo built on Google App Engine using JRuby on Rails. This app uses jQuery UI and ajax. Click on buttons on the left side, then right side contents will change, and you'll see accordion. Also, you can try examples I showed in my presentation. Cloning the git repo,

git://github.com/yokolet/jrubykaigi_examples.git

and look at README to run examples.

Tuesday, May 04, 2010

A Small Step to Rails on RedBridge

When RedBridge (JRuby Embed) was released included in JRuby 1.4.0RC1 for the first time ever, I wrote“'Rails on Red Bridge' will be my exciting challenge" in my blog, What's the embedding API of JRuby 1.4.0RC1?. Since then, RedBridge had many improvements and bug fixes, so I tackled the issue this week. As a result, it went good. I could successfully made a small step to Rials on RedBridge. However, the process was a bit tricky, so I'm going to write down how I could make it, step by step.

The most difficult part was gem bundling. Since Rails on RedBridge is built on Java web application, all gems should be in a Java based web application structure. Besides, the gems should be looked up by a Rails app on a servlet. After I tried some, I concluded Rails 3 was the best choice because of its built-in feature of gem bundler. So, I created a Java web application project with Apache Tomcat on Eclipse and Rails 3 app within.

First, I built jruby-complete.jar from JRuby 1.5.0 source archive. At this moment, JRuby 1.5.0.RC3 is the latest.

$ tar zxfv /Users/yoko/Downloads/jruby-src-1.5.0.RC3.tar.gz
$ cd jruby-1.5.0.RC3
$ ant jar-complete
$ export JRUBY_HOME=`pwd`
$ PATH=$JRUBY_HOME/bin:$PATH

Then, I installed rails 3 and jdbc adapter gems. In my case, I used sqlite3, so I typed a gem name for jdbc sqlite3 adapter.

$ jruby -S gem install rails --pre --no-rdoc --no-ri
$ jruby -S gem install activerecord-jdbcsqlite3-adapter --no-rdoc --no-ri


Next, I created a Java web application project, Hemlock, on Eclipse. Once the project was built, I had the tree structure below:

Hemlock -+- WebContent -+- META-INF --- MANIFEST.MF
| +- WEB-INF -+- lib
| +- web.xml
+- build -+- classes
+- src

Then, I created a Rails app under WebContent/WEB-INF/lib. This is because a web application adds the path, WEB-INF/lib, into the classpath automatically. WebContent/WEB-INF/classes would have been among the choices for the same reason.

My Rails app is based on a classic tutorial, "Four Days on Rails." This original document got outdated, but Japanese version was updated up to Rails 2.3.2. You can download the PDF file from Ita-san's blog: Four Days on Rails 2.0. Online translation service might help you to read the PDF.

$ cd [path-to-workspace]/Hemlock/WebContent/WEB-INF/lib
$ jruby -S rails todo
$ cd todo

Then, I edited Gemfile and config/database.yml to replace adapter gem and setting from sqlite3 to jdbcsqlite3.

Gemfile

#gem 'sqlite3-ruby', :require => 'sqlite3'
gem 'activerecord-jdbcsqlite3-adapter'

config/database.yml

#adapter: sqlite3
adapter: jdbcsqlite3

Before proceeding further, I did scaffolding to see database connection was set up right.

$ jruby -S rails g scaffold Category category:string
$ jruby -S rake db:migrate
$ jruby -S rails server

Requesting http://localhost:3000/categories on my browser, I added three categories.


I did one more setup on Rails --- Rails metal. Rails metal was originally invented to improve perfomance. But, metal exposes Rack's bare interface, so metal makes it easy to handle Rails app in the servlet.

$ jruby -S rails g metal poller


OK, let's setup gems for a servlet.

$ jruby -S bundle install vendor --disable-shared-gems

This command installed all necessary gems in WebContent/WEB-INF/lib/todo/vendor/gems directory.

Then, I moved on to the web application setting and the servlet. I copied jruby-compelete.jar to WEB-INF/lib, first. Next, after refreshing the Hemlock Eclipse project, I added jruby-complete.jar by Properties -> Java Build Path -> Add External JARS -> [on a JAR Selection window, I chose Hemlock/WebContent/WEB-INF/lib/jruby-complete.jar] -> OK. My servlet became as in below:

package com.servletgarden.hemlock;

import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.jruby.embed.LocalContextScope;
import org.jruby.embed.ScriptingContainer;

public class SimpleMetalServlet extends HttpServlet {
private static final long serialVersionUID = 1L;
private String basepath;
private ScriptingContainer container;
private List loadPaths;

public SimpleMetalServlet() {
super();
}

@Override
public void init() {
basepath = getServletContext().getRealPath("/WEB-INF");
String[] paths = {
basepath + "/lib/todo/vendor/gems/bundler-0.9.25/lib/",
basepath + "/lib/todo"
};
loadPaths = Arrays.asList(paths);
container = new ScriptingContainer(LocalContextScope.THREADSAFE);
}

@Override
protected void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
container.setLoadPaths(loadPaths);
container.runScriptlet("ENV['BUNDLE_GEMFILE'] = \"" + basepath + "/lib/todo/Gemfile\"");
container.runScriptlet("require 'config/environment'");
Map env = getEnvMap(request);
container.put("env", env);
List result = (List) container.runScriptlet("Poller.call env");
returnResult(response, result);
}

private Map getEnvMap(HttpServletRequest request) {
Map map = new HashMap();
map.put("PATH_INFO", "/poller");
return map;
}

private void returnResult(HttpServletResponse response, List result) throws IOException {
response.setStatus(((Long)result.get(0)).intValue());
Map headers = (Map)result.get(1);
Set keys = headers.keySet();
for (String key : keys) {
if ("Content-Type".equals(key)) {
response.setContentType((String)headers.get(key));
}
}
List contents = (List)result.get(2);
for (String content : contents) {
response.getWriter().write(content);
}
}
}

When I right-clicked on the servlet source and chose Run As -> Run on Server, my Eclipse showed that simple "Hello, World!"


OK, let's look at what I did in SimpleMetalServlet. The first one is a load path setting.

Hemlock tree

Hemlock -+- WebContent -+- META-INF --- MANIFEST.MF
| +- WEB-INF -+- lib -+- todo -+- Gemfile
| | | +- app -+- controllers --- ...
| | | | +- metal --- poller.rb
| | | | +- ...
| | | +- config -+- environment.rb
| | | +- vendor -+- gems -+- bundler-0.9.25 -+- lib
| | | -+- ...
| | | +- ...
| | +- jruby-complete.jar
| +- web.xml
+- build -+- classes
+- src -+- com -+- servletgarden -+- hemlock -+- SimpleMetalServlet.java

in init() method:

basepath = getServletContext().getRealPath("/WEB-INF");
String[] paths = {
basepath + "/lib/todo/vendor/gems/bundler-0.9.25/lib/",
basepath + "/lib/todo"
};
loadPaths = Arrays.asList(paths);

in service() method:

container.setLoadPaths(loadPaths);

The path to bundler is to use bundler gem to load bundled gems. Bundler gem is installed in the todo/vendor/gems directory, but before loading bundled gem, SimpleMetalServlet can't load bundler itself. So, the servlet is telling JRuby runtime where the bundler gem is. The path to todo, Rails app top directory, is to tell JRuby where Rails app top directory is located.

The second one is the path to Gemfile.

container.runScriptlet("ENV['BUNDLE_GEMFILE'] = \"" + basepath + "/lib/todo/Gemfile\"");

Gemfile matters to Rails. To make Rails up and running, SimpleMetalServlet needs to give info about Gemfile. In this case, SimpleMetalServlet set the real path to Gemfile tied to the key "BUNDLE_GEMFILE" in ENV hash.

Then,

container.runScriptlet("require 'config/environment'");

Todo Rails app should get started by this.

Everything should be setup now, so SimpleMetalServlet is able to kick the metal method. Before talking about how the servlet kicked it, let's see what code Rails tailored.

poller.rb

# Allow the metal piece to run in isolation
require File.expand_path('../../../config/environment', __FILE__) unless defined?(Rails)

class Poller
def self.call(env)
if env["PATH_INFO"] =~ /^\/poller/
[200, {"Content-Type" => "text/html"}, ["Hello, World!"]]
else
[404, {"Content-Type" => "text/html", "X-Cascade" => "pass"}, ["Not Found"]]
end
end
end

Poller class has a class method, call, which needs an argument, env. The value, env, is a hash, in which PATH_INFO key and the value tied to it is expected. So, SimpleMetalServlet creates java.util.Map type object, puts to ScriptingContainer and evaluates Poller.call method with the argument, env.

Map env = getEnvMap(request);
container.put("env", env);
List result = (List) container.runScriptlet("Poller.call env");

private Map getEnvMap(HttpServletRequest request) {
Map map = new HashMap();
map.put("PATH_INFO", "/poller");
return map;
}


At last, response handling comes in.

returnResult(response, result);

private void returnResult(HttpServletResponse response, List result) throws IOException {
response.setStatus(((Long)result.get(0)).intValue());
Map headers = (Map)result.get(1);
Set keys = headers.keySet();
for (String key : keys) {
if ("Content-Type".equals(key)) {
response.setContentType((String)headers.get(key));
}
}
List contents = (List)result.get(2);
for (String content : contents) {
response.getWriter().write(content);
}
}

Rack based application returns the response as an array of a status code, hash of http response headers, and array of body contents. SimpleMetalServlet parses the response and sends back to a browser. Since Poller class returns so simple reponse, SimpleMetalServlet doesn't do much in this case.

I tweaked poller.rb a bit so that registered categories are showed up.

poller.rb

#[200, {"Content-Type" => "text/html"}, ["Hello, World!"]]
[200, {"Content-Type" => "application/xhtml+xml"}, [Category.all.to_xml]]

Refreshing Hemlock project on Eclipse, restarting Tomcat also on Eclipse, then requesting http://localhost:8080/Hemlock/SimpleMetalServlet gave me the XML below.


It seems Rails on RedBridge worked.

Friday, April 16, 2010

RedBridge, what are new and improved in JRuby 1.5.0RC1

As you may know, Tom Enebo announced the release of JRuby 1.5.0RC1 on Apr. 15 saying "aged like a fine wine." @headius tweated "Over 1250 commits for JRuby 1.5, our largest amount of work ever for any individual release." Also, RedBridge is. RedBridge has been improved since last release based on user inputs. It's API had many changes to become more useful and organized API. Although I've already written about all changes in this blog, I'm going to put them together here for convenience.

New and Deprected Configuration API
RedBridge in JRuby 1.5.0 has a lot of Ruby runtime configuration methods. Before, those were available through getProvider().getRubyInstanceConfig() method, however, this was not a good idea. Since the method exposes JRuby's internal API, users' code might be affected by internal API changes. This fact is against to the purpose of RedBridge. RedBrdige should cover JRuby's internal API and absorb internal changes so that users don't need to fix their code by themselves. Avoid using getProvider().getRubyInstanceConfig() method as much as possible. If you want more runtime configuration methods, please request us.

New runtime configuration methods of ScriptingContainer:

  • get/setInput
  • get/setOutput
  • get/setError
  • get/setCompileMode
  • get/setRunRubyInProcess
  • get/setCompatVersion
  • get/setObjectSpaceEnabled
  • get/setEnvironment
  • get/setCurrentDirectory
  • get/setHomeDirectory
  • get/setClassCache
  • get/setClassLoader
  • get/setProfile
  • get/setLoadServiceCreator
  • get/setArgv
  • get/setScriptFileName
  • get/setRecordSeparator
  • get/setKCode
  • get/setJITLogEvery
  • get/setJITThreshold
  • get/setJITMax
  • get/setJITMaxSize

Deprecated configuration methods:

  • getRuntime()
  • getProvider().setLoadPaths()
  • getProvider().setClassCache()

Usage example:

[JRuby 1.4.0]
ScriptingContainer container = new ScriptingContainer();
container.getProvider().getRubyInstanceConfig().setJRubyHome(jrubyhome);

[JRuby 1.5.0]
ScriptingContainer container = new ScriptingContainer();
container.setHomeDirectory(jrubyhome);


New Options
Also, RedBridge got two new options: SHARING_VARIABLES and TERMINATION. The first, SHARING_VARIABLES, option turns on/off a sharing variables feature.This is an essential feature for some users while useless for other users. For those people, sharing variables is just a source of performance degradation. When the feature is turned off, the performance will be a bit better.
Usage example:

[Embed Core]
container.setAttribute(AttributeName.SHARING_VARIABLES, false);

[JSR223]
engine.getContext().setAttribute("org.jruby.embed.sharing.variables", false, ScriptContext.ENGINE_SCOPE);

The second, TERMINATION, option is for JSR223 users to call terminate. This option was added in light of RedBridge's behavior change. This is a big change, so I'll discuss more about this.

Changed Behaviors

* No termination in each evaluation and method call

RedBridge in JRuby 1.4.0 always called Ruby runtime's terminate method at the end of each evaluation and method call. This was to execute at_exit blocks and release resources automatically. The idea came from JSR223 API, which doesn't have a terminate method defined. However, I eliminated this behavior in JRuby 1.5.0 for three reasons. The first one is for performance. The terminate() method is so slow and was the biggest culprit of bad performance. The more Ruby code uses ruby files, instance variables, etc, the more it takes time. The second one is to make RedBridge's behavior more natural. Since the terminate() method fires at_exit blocks automatically, users might have unexpected results when they use third party libraries. Users should have a chance to fire at_exit blocks by themselves. The third one is that JRuby's memory leak was fixed. Thus, RedBridge doesn't need to invoke the terminate method just for releasing resources. Make sure, you have the terminate method in the right place.

Usage examples:

[Embed Core]

ScriptingContainer container = null;
try {
container = new ScriptingContainer();
container.runScriptlet(PathType.CLASSPATH, testname);
} catch (Throwable t) {
t.printStackTrace();
} finally {
container.terminate();
}

[JSR223]

ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("jruby");
engine.eval("$x='GVar'");
engine.eval("at_exit { puts \"#{$x} in an at_exit block\" }"); // nothing is printed here
engine.getContext().setAttribute(AttributeName.TERMINATION.toString(), true, ScriptContext.ENGINE_SCOPE);
engine.eval(""); // prints "GVar in an at_exit block"


* Global runtime

When a global ruby runtime exists on a single JVM, a singleton model of RedBridge uses the global runtime in JRuby 1.5.0. This works behind the scene and seems not so attractive, but is interesting. Here's a bit tricky usage:

$ pwd
/Users/yoko/Tools/jruby-1.5.0.RC1
$ jruby --1.9 -Ctest -S irb
irb(main):001:0> require 'java'
=> true
irb(main):002:0> container=org.jruby.embed.ScriptingContainer.new
=> org.jruby.embed.ScriptingContainer@771eb1
irb(main):003:0> container.compat_version <---- --1.9 option
=> RUBY1_9
irb(main):004:0> container.current_directory <---- -Ctest option
=> "/Users/yoko/Tools/jruby-1.5.0.RC1/test"
irb(main):009:0> container.home_directory <---- jruby home
=> "/Users/yoko/Tools/jruby-1.5.0.RC1"
irb(main):010:0> container.run_scriptlet "at_exit { puts \"see you, later\" }"
=> #<Proc:0x88a1b@<script>:1>
irb(main):011:0> container.terminate
see you, later
=> nil

This new behavior might be useful in a complicated application.
However, you should be aware that setting a runtime configuration doesn't work if the global runtime is there already. This is because the runtime configuration is read only when the runtime is instantiated. You should be careful not miss the timing to set configuration.

* Lazy Runtime Initialization

RedBridge (in this case, I mean Embed Core) delays ruby runtime initialization as much as possible. This is to improve ScriptingContainer's start up time. You may know loading Ruby runtime is a huge job and takes pretty much time. This might cause frustration if it happens right after the ScriptinContainer gets started. The question is when runtime is up and running. Some of ScriptingContainer's methods will kick ruby runtime to wake up. Here's a list:

  • put()
  • runScriptlet()
  • setWriter()
  • resetWriter()
  • setErrorStream()
  • resetErrorStream()
  • setReader()

Thus, when you want configuration settings to work, you need to set them before these methods.
Meanwhile, JSR223 implementation doesn't delay ruby runtime initialization. It was not easy without breaking JSR223's requirement.

* Lazy Java Library Loading

Red Bridge doesn't load a java library while ruby runtime is initialized in JRuby 1.5.0. This is also for performance improvement. Loading libraries on to ruby runtime is quite a cumbersome job. Checking loaded library tables up to see whether a specified library has not yet loaded, judging how to load the library, then loading, caching... Even though Java library is not loaded while initialization, it will be loaded internally if necessary. Or you can load Java library explicitly:
container.runScriptlet("require 'java'");


Performance Tuning Tips
RedBridge's performance has been improved compared to older version, but you can tweak a bit more. For example, you can remove variables for sharing or clear sharing variable table at some point:
org.jruby.embed.ScriptingContainer#remove(String key)
org.jruby.embed.ScriptingContainer#clear()

RedBridge retrieves instance variables and constants as much as possible at the end of each evaluation and method call. All retrieved values are injected to runtime when the next script or method is evaluated. You can cut down the time for injection by removing unnecessary values.

Remaining jobs
RedBridge couldn't resolve all issues and has remaining jobs. Among them, OSGi and configuration on JSR223 impl would be two big issues. By the final release of JRuby 1.5.0, I want to improve these.


Finally, your input will help us to make RedBridge more perfect API. Give it a try and report us!

Thursday, February 25, 2010

JRuby Embed (Red Bridge) Update: termination and skipping sharing variables

Hers' recent update of RedBridge. Performance got much better, but the change on termination might affect your code. If you expect at_exit blocks to be executed automatically, you need to add termination.

By the recent change, Embed Core, JSR223, and BSF, all three implementations had changes in their behaviors of evaluation and method invocation. Termination is no longer executed automatically. This means, at_exit blocks are not executed at the end of every runScriplet, run and callMethod (eval, invokeMethod and invoekeFunction in JSR223). It is effective since commit 673df9f.

This rather big changes was made for two reasons. One is to avoid possible unexpected behavior caused by at_exit blocks to be executed too early. For example, a gem might have a class that has an at_exit block, which should run after other code have finished. The second reason is a performance improvement. Terminate method takes much time to complete. Because of this, evaluation and method invocation of embedding API were very slow. Now, embedding API got much better performance than before.

Then, how to do that? To terminate explicitly, call terminate method on Embed Core and BSF. It's simple. However, JSR223 doesn't have terminate method defined by the specification. So, use newly added attribute, AttributeName.TERMINATION or org.jruby.embed.termination to trun termination on. Next, evaluate blank code.

Usage examples:

[Embed Core]

ScriptingContainer container = null;
try {
container = new ScriptingContainer();
conatiner.runScriptlet(PathType.CLASSPATH, testname);
} catch (Throwable t) {
t.printStackTrace();
} finally {
container.terminate();
}

[JSR223]

ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("jruby");
engine.eval("$x='GVar'");
engine.eval("at_exit { puts \"#{$x} in an at_exit block\" }"); // nothing is printed here
engine.getContext().setAttribute(AttributeName.TERMINATION.toString(), true, ScriptContext.ENGINE_SCOPE);
engine.eval(""); // prints "GVar in an at_exit block"


Let's move on to the second change. A new option has been added to skip sharing variables. Sharing variables is a useful or required feature for users from JSR223 and BSF background. But, it is not a necessary for others especially who have directly used JRuby's internal API. Sharing variables just slowed down the evaluations and method invocations. When sharing variables is skipped, the performance will be a bit better.

Usage example:

[Embed Core]
container.setAttribute(AttributeName.SHARING_VARIABLES, false);

[JSR223]
engine.getContext().setAttribute(AttributeName.SHARING_VARIABLES.toString(), false, ScriptContext.ENGINE_SCOPE);


Have fun with RedBridge!

Friday, February 05, 2010

Hacking JRuby - add all Hash methods to Map

I recently filed JRUBY-4528, whose patch adds all Ruby's Hash methods to a java.util.Map type object. Applying the patch, I confirmed that I could use "add_ruby_methods" method on Map type object, then, Hash methods for Map object. This would be useful especially for embedding API users since they often want to share Map object between Java and Ruby, back and forth.

What's the problem of current JRuby? When an instance of java.util.HashMap is sent into Ruby code, the object is converted into a "usable" Java object in Ruby world. This is what org.jruby.javasupport.JavaEmbedUtils.javaToRuby() method does, and we can't get Java Map converted into Ruby Hash automatically. Why? People might want to use that object as it is, HashMap type object itself, for other Java APIs used in Ruby. However, no built-in method converts Map to Hash so far although some of methods are added to.

My patch is attempt to add all Hash methods to Map type object by "add_ruby_methods" method.
For example:

irb(main):004:0> require 'java'
=> true
irb(main):005:0> jhash = java.util.HashMap.new
=> {}
irb(main):006:0> jhash.put("1", 100)
=> nil
irb(main):007:0> jhash.put("2", 200)
=> nil
irb(main):008:0> jhash.inspect
=> "{2=200, 1=100}"
irb(main):009:0> rhash = jhash.add_ruby_methods
=> {"2"=>200, "1"=>100}
irb(main):010:0> rhash.inspect
=> "{\"2\"=>200, \"1\"=>100}"
irb(main):011:0> p rhash.values
[200, 100]
=> nil
irb(main):012:0> rhash.merge!({"2"=>222, "3"=>333})
=> {"3"=>333, "2"=>222, "1"=>100}
irb(main):013:0> jhash.inspect
=> "{3=333, 2=222, 1=100}"

On jirb, I created java.util.HashMap object and put two key-value pairs using Java API, which was inspected by automatically added "inspect" method while converting. Then, I used add_ruby_methods method. After that, I could use Hash's inspect, values and merge! methods. Operations for "rhash" object above is also operations to "jhash," so when I inspected jhash, key-value pairs were also updated.

What if I create a Map object in Java code and give it to Ruby? Key-value pairs in a Java Map object was completely manipulated by Ruby. For example, see code below:

ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
ConcurrentHashMap map1 = new ConcurrentHashMap();
map1.put("a", 100);
map1.put("b", 200);
Map map2 = new HashMap();
map2.put("b", 254);
map2.put("c", 300);
container.put("h1", map1);
container.put("h2", map2);
container.put("num", 0);
String script =
"rh = h1.add_ruby_methods\n" +
"puts \"num: #{num}\"\n" +
"rh.merge!(h2.add_ruby_methods) {|k,o,n| num += 1; o+n }";
container.runScriptlet(script);
Set entries = map1.entrySet();
for (Map.Entry entry : entries) {
System.out.print(entry.getKey() + ": " + entry.getValue() + ", ");
}

outputs:

b: 454, a: 100, c: 300,

As you see, merge! method worked. All java.util.Map type such as ConcurrentHashMap or TreeMap are available to apply the method.

Possible problem of this attempt is that contents of an object after add_ruby_methods applied are Java objects. For this reason, a direct comparison to a Ruby Hash object fails. In this case, to_hash method would work since to_hash method returns real Ruby Hash of converted Java Map.

This is just an attempt, and I'm not sure this patch will be applied or not. If you think this is useful, leave a comment on JIRA.

Tuesday, February 02, 2010

JRuby Embed (Red Bridge) Gotchas: on jirb

I haven't used like that before, but there are people who want to use JRuby Embed API on jirb. I fixed a bug, http://jira.codehaus.org/browse/JRUBY-4521, and tried what I could do on jirb.

At first, I instantiated ScriptingContainer and checked what initial parameters were set.

irb(main):001:0> require 'java'
=> true
irb(main):002:0> container = org.jruby.embed.ScriptingContainer.new
=> org.jruby.embed.ScriptingContainer@ea7549
irb(main):003:0> p container.get_home_directory
"/Users/yoko/DevSpace/jruby~main"
=> nil
irb(main):004:0> p container.load_paths
[/Users/yoko/DevSpace/jruby~main/lib/profile.jar, /Users/yoko/NetBeansProjects/cirrus/build/classes]
=> nil
irb(main):005:0> p container.class_loader
org.jruby.util.JRubyClassLoader@5e2075
=> nil
irb(main):006:0> p container.current_directory
"/Users/yoko/NetBeansProjects/cirrus"
=> nil
irb(main):007:0> p container.compat_version
RUBY1_8
=> nil
irb(main):008:0> p container.supported_ruby_version
"jruby 1.5.0.dev (ruby 1.8.7 patchlevel 174) (2010-02-02 0505fb1) (Java HotSpot(TM) Client VM 1.5.0_22) [i386-java]"
=> nil

Hmmm.... interesting. Of course, no compilation at all. Perhaps, ScriptingContainer's API is useful to see jirb internal settings.

Then, how evaluations go?

irb(main):009:0> script = "puts \"Hello World\""
=> "puts \"Hello World\""
irb(main):010:0> container.run_scriptlet(script)
Hello World
=> nil
irb(main):011:0> message = "Hi, there!"
=> "Hi, there!"
irb(main):012:0> container.put("message", message)
=> nil
irb(main):013:0> container.run_scriptlet("puts \"message: #{message}\"")
message: Hi, there!
=> nil

OK, evaluations as well as sharing variables between Java(?) (or jirb?) and Ruby seem to work.
How about method call?

irb(main):014:0> script = "def say\nputs \"oh!\"\nend"
=> "def say\nputs \"oh!\"\nend"
irb(main):015:0> recv = container.run_scriptlet(script)
=> nil
irb(main):016:0> container.call_method(recv, "say", java.lang.Object.class)
:1:in `say': wrong # of arguments(1 for 0) (ArgumentError)
from :1
NativeException: org.jruby.embed.InvokeFailedException: wrong # of arguments(1 for 0)
from org/jruby/embed/internal/EmbedRubyObjectAdapterImpl.java:387:in `call'
from org/jruby/embed/internal/EmbedRubyObjectAdapterImpl.java:326:in `callMethod'
from org/jruby/embed/ScriptingContainer.java:1268:in `callMethod'
from :1

No, it failed. This is because jirb chose "public Object callMethod(Object receiver, String methodName, Object... args)" for callMethod. Unfortunately, in this case, Ruby doesn't know the difference of several callMethod methods.
Ok, then, no argument for "say" method. Will it work?

irb(main):017:0> container.call_method(recv, "say")
CallableSelector.java:196:in `assignableOrDuckable': java.lang.ArrayIndexOutOfBoundsException: 2
from CallableSelector.java:22:in `access$200'
from CallableSelector.java:163:in `accept'
from CallableSelector.java:101:in `findCallable'
from CallableSelector.java:86:in `findMatchingCallableForArgs'
from CallableSelector.java:39:in `matchingCallableArityN'
from RubyToJavaInvoker.java:170:in `findCallable'
from InstanceMethodInvoker.java:29:in `call'
from InstanceMethodInvoker.java:67:in `call'
from AliasMethod.java:66:in `call'
from CachingCallSite.java:329:in `cacheAndCall'
...
...

Oh dear, jirb was blown up.

So far, ScriptingContainer on jirb doesn't work enough but might be fun for quick hack.

........

Wrap this up.

We have java_send method and can specify exact Java method by its argument, but this also didn't work well. When I tried to run "volume" method of this Ruby code:

def volume(r)
4.0 / 3.0 * Math::PI * r ** 3.0
end

irb(main):008:0> ret = container.run_scriptlet(org.jruby.embed.PathType::CLASSPATH, "ruby/sphere.rb")
=> nil
irb(main):009:0> container.java_send :callMethod, [java.lang.Object, java.lang.String, java.lang.Class], self, "volume", java.lang.Integer.new(3)
TypeError: for method ScriptingContainer.callMethod expected [class java.lang.Object, class java.lang.String, class java.lang.Class]; got: [org.jruby.RubyObject,java.lang.String,java.lang.Integer]; error: argument type mismatch
from :1

like in the above, I got TypeError. A receiver object was the problem. Java program could cast java.lang.RubyObject to java.lang.Object, but jirb could not.

........

Wrap up, part 2.

While testing ScriptingContainer on jirb, I found a bug. Singleton model didn't see the same RubyInstanceConfig when Ruby runtime had already instantiated preceding ScriptingContainer. After the fix, some of runtime configurations can be changed through ScriptingContainer's methods. For example, I could change Ruby version to be used. The Ruby code below uses a block local variable introduced in Ruby 1.9.

# This snippet is borrowed from http://gihyo.jp/dev/serial/01/ruby/0003
# defines local variable x
x = "bear"

# A block local variable x is used in this block. (Two "x"s work together)
["dog", "cat", "panda"].each do |x|
# This x is a block local variable.
p x
break if x == "cat"
end

# This x is a local variable since it is used outside of the block.
p x

When I tried this code by setting both Ruby 1.8 and 1.9, I got appropriate outputs for both mode on jirb.

irb(main):001:0> require 'java'
=> true
irb(main):002:0> container = org.jruby.embed.ScriptingContainer.new
=> org.jruby.embed.ScriptingContainer@ea7549
irb(main):003:0> container.compat_version
=> RUBY1_8
irb(main):004:0> container.run_scriptlet(org.jruby.embed.PathType::CLASSPATH, "ruby/block-param-scope.rb")
"dog"
"cat"
"cat"
=> nil
irb(main):005:0> container.set_compat_version(org.jruby.CompatVersion::RUBY1_9)
=> nil
irb(main):006:0> container.compat_version
=> RUBY1_9
irb(main):007:0> container.run_scriptlet(org.jruby.embed.PathType::CLASSPATH, "ruby/block-param-scope.rb")
"dog"
"cat"
"bear"
=> nil

So, again, SciprtingContainer on jirb is interesting. :)

Tuesday, January 19, 2010

JRuby Embed (Red Bridge) Update: configuration and global runtime

Here're recent updates of JRuby Embed (Red Bridge). Since I wrote about updates last time, Red Bridge's API has been changed a lot around configuration. Although this would directly affect to users' code, new API is easier to use and more stable. For example, setting jruby home directory has been changed from the fist to second one below:

[JRuby 1.4.0]
ScriptingContainer container = new ScriptingContainer();
container.getProvider().getRubyInstanceConfig().setJRubyHome(jrubyhome);

[JRuby 1.5.0.dev]
ScriptingContainer container = new ScriptingContainer();
container.setHomeDirectory(jrubyhome);

As you see clearly, getProvider().getRubyInstanceConfig() was gone. This is to hide JRuby's internal API so that internal API changes won't force users to adjust thier code. This also fixes the problem that current Red Bridge API exposes internal API too much and doesn't absorb internal changes well enough. To absorb internal changes is a Red Bridge's job, and by this update, more jobs have been covered. Other than get/setHomeDirectory(), we had following configuration methods in ScriptingContainer.

  • get/setInput
  • get/setOutput
  • get/setError
  • get/setCompileMode
  • get/setRunRubyInProcess
  • get/setCompatVersion
  • get/setObjectSpaceEnabled
  • get/setEnvironment
  • get/setCurrentDirectory
  • get/setHomeDirectory
  • get/setClassCache
  • get/setClassLoader
  • get/setProfile
  • get/setLoadServiceCreator
  • get/setArgv
  • get/setScriptFileName
  • get/setRecordSeparator
  • get/setKCode
  • get/setJITLogEvery
  • get/setJITThreshold
  • get/setJITMax
  • get/setJITMaxSize

Please go to API documantation and JRuby Options to know what are those.

The second update info is about global runtime. The "global runtime" means a static instance of Ruby runtime (org.jruby.Ruby) and, now, is used in a singleton model. The singleton model is a default local context type in JRuby 1.5.0, so unless users choose threadsafe or singlethread models explicitly, they will use static, JVM global Ruby runtime. Changing it a JVM global instance, we are supposed to get exactly the same Ruby runtime instance everywhere on a single JVM. This would be useful for some cases such as using Red Brdige for DSL. Meanwhile, we should be careful when Ruby code gets run on multi-threaded environment. Since Ruby runtime holds many states, most of them will be shared globally.

Red Bridge is still under the way to the stable API and may have more changes. However, it is surely on the way to more feasible API. Please feel free to suggest new methods or features over Red Bridge.

Friday, November 27, 2009

JRuby Embed (Red Bridge) Gotchas: __FILE__

About a month ago, I wrote __FILE__ didn't work when Ruby code was loaded from classpath in the thread, Load path issues inside jar / external app. Tracking jruby down with a debugger, I found out one solution. It was a combination of setting a feasible current directory and using File.expand_path.

Here's a test code:

# file_check.rb [Birch]

puts "__FILE__: #{__FILE__}"
puts "dirname: #{File.dirname(__FILE__)}"
puts "expanded path: #{File.expand_path(File.dirname(__FILE__))}"
puts "joined path 1: #{File.join(File.dirname(__FILE__), "abc.rb")}"
puts "joined path 2: #{File.join(File.expand_path(File.dirname(__FILE__)), "abc.rb")}"

// FileCheck.java
package vanilla;

import org.jruby.embed.LocalContextScope;
import org.jruby.embed.PathType;
import org.jruby.embed.ScriptingContainer;

public class FileCheck {

private FileCheck() {
//String userDir = System.getProperty("user.dir");
//System.setProperty("user.dir", userDir+"/src/ruby");
ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
System.out.println("currentDirectory: " + container.getProvider().getRubyInstanceConfig().getCurrentDirectory());
container.getProvider().getRubyInstanceConfig().setCurrentDirectory(System.getProperty("user.dir")+"/src/ruby");
System.out.println("currentDirectory: " + container.getProvider().getRubyInstanceConfig().getCurrentDirectory());
container.runScriptlet(PathType.CLASSPATH, "file_check.rb");
}

public static void main(String[] args) {
new FileCheck();
}
}

The absolute path to file_check.rb was /Users/yoko/NetBeansProjects/Birch/src/ruby/file_check.rb. So, I added the path, /Users/yoko/NetBeansProjects/Birch/src/ruby, to "-cp" option of java command. In a Java code, I set /Users/yoko/NetBeansProjects/Birch/src/ruby as the current directory. Then, the result was below:

yoko$ java -cp build/classes:/Users/yoko/DevSpace/jruby~main/lib/jruby.jar:./src/ruby vanilla.FileCheck
currentDirectory: /Users/yoko/NetBeansProjects/Birch
currentDirectory: /Users/yoko/NetBeansProjects/Birch/src/ruby
__FILE__: file_check.rb
dirname: .
expanded path: /Users/yoko/NetBeansProjects/Birch/src/ruby
joined path 1: ./abc.rb
joined path 2: /Users/yoko/NetBeansProjects/Birch/src/ruby/abc.rb

As you see, a wrapped path by File.expand_path is correct though just File.dirname didn't work. The combination of setting the current directory and using File.expand_path would be the solution of this kind of cases. If you are using JSR223 or BSF, setting user.dir system property works as I commented out in the Java code. This is because JRuby uses the current directory when it expands a path, and the current directory is based on user.dir system property. If it is a web application, perhaps, we can set the current directory using ServletContext#getRealPath().

Wednesday, November 25, 2009

JRuby Embed (Red Bridge) Update: global vars, loading java, and more

During these weeks, I made a couple of changes on Red Bridge (JRuby Embed), which would improve performance a bit and reduce problems caused by global variables. This change is available from 161d0fe in master (1.5.0.dev).

Firstly, I changed an internal implementation of sharing global variables. Red Bridge injects all variables in a variable map just before the evaluation, and tries to retrieve all local, instance, global variables and constants used in Ruby just after the evaluation. This behavior is really greedy, also ends up in poor performance. However, it is necessary since Red Bridge terminates the all state including variable values right after the evaluation is done, which is to save resources. Unless retrieving all variables and constants, Red Bridge can't return requested variables in a Java program. For example, users can do with Red Bridge:

ScriptingContainer container = new ScriptingContainer();
container.runScriptlet("$theta = Math::PI / 6.0");
container.runScriptlet("$value = Math.sin($theta)");
System.out.println(container.get("$theta") + ", " + container.get("$value"));

Above outputs: 0.5235987755982988, 0.49999999999999994


Local, instance variables and constants (except global constants) need to be saved before those are disappeared by the termination, but global variables are still on Ruby runtime. So, I changed to get global variables lazily. Only when it is requested, Red Bridge takes the requested global variable out from runtime.

This new behavior would also reduce troubles caused by global variables. Before, Red Bridge retrieves global variables as much as possible from Ruby runtime except predefined ones. Then, Red Bridge injects all global variables in its variable map to runtime for successive evaluation with values of previous evaluation. This behavior occasionally causes unexpected results and warnings. After the change, Red Bridge doesn't grab unnecessary global variables, doesn't inject them for the next evaluation. Perhaps, unexpected results related to global variables will be reduced. This new behavior is not available when a global local variable behavior, JSR223's default behavior, is chosen since it is tailored to behave exactly the same as the reference implementation.

Some of you might already know clearing up the variable map before the successive evaluation contributes performance. I added two shortcut methods to ScriptingContainer:

org.jruby.embed.ScriptingContainer#remove(String key)
org.jruby.embed.ScriptingContainer#clear()

The remove method removes a specified key-value pair from the variable map and runtime. The clear method removes all key-value pairs from the variable map and runtime. The smaller the variable map size is, the shorter the time for injection is. Don't forget to remove redundant key-value pairs.


I made one more change. Red Bridge no more loads a java library during the initialization. The process of loading libraries in JRuby is quite a cumbersome job. Looking the loaded library tables up to see it is not already loaded, judging how and from where loads the library, then loading, and caching them to avoid duplication... Nevertheless, not all Ruby scripts need the java library. If people run Fibonacchi written in pure Ruby on Red Bridge, they don't need the java library at all. When people want to use the java library, adding the line "require 'java'" in a Ruby code works fine. Moreover, people add "require 'java'" when they run scripts using jruby command if the scripts need the java library. The advantage of pre-loading the java library seems to be less. So, I stopped loading the java library during the initialization. Perhaps, the time for initialization got shortened a bit.

Monday, November 02, 2009

JRuby Embed (Red Bridge) Update

Since my last post about JRuby Embed (Red Bridge), it has been vastly changed. JRuby Embed codebase has been merged into JRuby! JRuby 1.5.0 will have Red Bridge inside in its both binary and source archives. Along with this, JRuby Embed wiki pages also have been merged into JRuby's wiki, Embedding JRuby section.

Now, JRuby Embed project is almost in end-of-life period. I'll soon close jruby-embed users ml since it is natural to talk at jruby-users/jruby-dev. Besides, most of discussions have done on jruby-users ml. Jira is also going to be merged into JRuby, but this will be done after JRuby's jira is completed moving from codehaus to kenai. Anyway, JRuby Embed users, please use jruby's ml and jira. JRuby's embedding seciton of jira would be good for us to file issues. However, I'll keep source code repository for JRuby 1.4. JRuby 1.4 has JRuby Embed binary but doesn't have sources. The binary that JRuby 1.4 has is built from codebase of this project, so it still has a reason to be there.

One of the biggest changes is JRuby Embed 0.1.3 has been released from JRuby Embed Project. It will be included in upcoming JRuby 1.4 release. In this release, default value of local context type has been switched from threadsafe to singleton. See the discussion about it. Please make sure your choice is the best to your case. Walk through Context Instance Type section to know what you should choose.

The version, 0.1.3 is identical to the one in JRuby trunk (1.5.0.dev) and also had a fix of JRUBY_EMBED-10. Give it a try. If you find something, file at "JRuby Jira" and ask about it at JRuby's mailing list.

Monday, October 05, 2009

What's the embedding API of JRuby 1.4.0RC1?

JRuby 1.4.0RC1 has been released on Oct. 2 and was a big release. JRuby had a lot of bug fixes and new features. Among them, JRuby Embed (aka Red Bridge) was there. The name, Red Bridge, means a bridge from Java to Ruby and, of course, the bridge has a color of ruby. However, many people would have thought, “What’s the new embedding API?” when they saw Tom’s announce. In this blog post, I ‘m going to answer such question so that people can have better understandings about Red Bridge.

Red Bridge is a Java API to run Ruby scripts in a Java program, and the project is hosted at http://kenai.com/projects/jruby-embed. Red Bridge has two layers, Embed Core and Core based implementations of scripting API. Currently, JSR223 (javax script: http://jcp.org/en/jsr/detail?id=223) and Jakarta BSF 2.4 (Bean Scripting Framework: http://jakarta.apache.org/bsf/) are implemented on top of Embed Core. Embed Core is totally different API from JRuby’s JavaEmbedUtils, which has similar but much fewer API compared to Embed Core. Embed Core has a lot of useful methods and features for embedders. Users of this new embedding API don’t need to use JavaEmbedUtils anymore. Besides, not like scripting APIs that are common to many languages, Embed Core is focused on leveraging JRuby’s power. For example, Embed Core allows users to configure Ruby runtime easily. For example, Embed Core’s parse method can have a JRuby friendly argument, InputStream, to read scripts from.

Red Bridge was originally my solo project I started in the last winter at Google Code to solve issues that Sun’s JSR223 JRuby engine reference implementation had. I was a contributor of JRuby engine at scripting.dev.java.net but felt reluctant to rewrite the it vastly since I’m not a Sun employee. Especially, the license of the reference implementation was a big issue to distribute with JRuby. JSR223 JRuby engine users wanted the implementation to be bundled in JRuby. So, I tried to get permission from Sun, and if possible, modify the license to fit into JRuby. But, I couldn’t get any answer from Sun at all. Other than the license issue, reference implementation’s bug-prone sharing global variable mechanism was a headache to me. That part was repeatedly affected by JRuby’s internal API changes, and grew to literally patchwork like ugly code. That global variables were only one type for sharing variables between Java and Ruby was also a problem. For JavaScript, PHP or maybe other languages, a variable name should be start with ‘$.” However, the name, $something, means not just a variable to Ruby but a globally referenced variable. Some people were eager to use another variable types to share. The sharing global variable of reference implementation also had a problem when JRuby engine was used on a multi-threaded environment such as a Servlet container (Java based web application server). The reference implementation might have set true to ThreadLocal option of Ruby runtime using a System property. However, relying on JVM wide system properties caused another problem especially on web application servers. A web application server might have multiple web applications (wars) on it and system property settings affect all of them.

In light of these issues Red Bridge has exactly the same license as JRuby and new mechanism for sharing variables, besides enables sharing global, local, and instance variables. Users can choose ThreadLocal model for context local values such as Ruby runtime, or sharing variables and other instances. Embed Core provides users methods to configure Ruby runtime. However, JSR223 and BSF engines still rely on JVM wide system property since those APIs haven’t defined such method. See Wiki, http://kenai.com/projects/jruby-embed/pages/Home, for details.

At kenai.com, you might find the project whose name is “Red Bridge.” When I moved my project to kenai.com right after I got the invitation from Charles Oliver Nutter, the name was Red Bridge, the same one at Google Code. A couple of weeks later, I talked with Charles and Thomas Enebo about Red Bridge. They liked Core part of two layers of Red Bridge and wanted to have just core layer bundled in JRuby. Following their choice, I started “JRuby Embed” project just for Embed Core. After that, JSR223 and BSF were added to the list to be bundled in JRuby, and Red Bridge was merged into JRuby Embed project. Merged into Red Bridge was definitely another choice. However, I chose JRuby Embed because people were interested in Embed Core part more than JSR 223 implementation and more members have been subscribed in JRuby Embed. Besides, the package name is org.jruby.embed, no redbridge in it. Since the name, “Red Bridge,” is easy to memorize and nice compared to banal name, “JRuby Embed,” I’ll keep using Red Bridge. While BSF implementation never had its own project ever. The implementation was added after JSR223 was merged in and took for a week or so.

Having JRuby 1.4.0RC1, users might be confusing JRuby’s JavaEmbedUtils and Red Bridge, and which one they should use. Definitely, new users should use Red Bridge since it is easy to use and powerful. (I’m working hard to update documents, so some of them are old. Sorry!) Right now, JavaEmbedUtils as well as other embed related interfaces are on a discussion to seek how they can be obsolete. API of JavaEmbedUtils and others have been used in many packages including JRuby Rack, so making them obsolete would be influential. Red Bridge will probably need to have bug fixes and improve its performance. Also API of Red Bridge probably needs to be reviewed and modified. I think it takes a time to eliminate JavaEmbedUtils.

Then, what will be next? I want to add a feature to run compiled Ruby scripts on Red Bridge. Currently, JIT and Force compiled modes are supported, but those are different from executing *.class files generated from *.rb. “Rails on Red Bridge” will be my exciting challenge. If people can write Struts’ action by Ruby using Red Bridge, it might be interesting. I don’t have a clear load map right now, but I want to keep going.

Have fun with Red Bridge!

Sunday, September 06, 2009

Splitting jruby-complete.jar up for Google App Engine

When we write a web application using JRuby, we need jruby-complete.jar included in a war to use builtin libraries. The builtin libraries are supposed to be located under jruby.home, so the jruby.home system property is expected to be set correctly. However, we need an alternative to set the property since setting jruby.home on the web application doesn't make sense. The answer is jruby-complete.jar, which has builtin libraries under META-INF/jruby.home directory in it.

When we use Google App Engine, another problem pops up. GAE has 10MB limit per each file to upload (http://googleappengine.blogspot.com/2009/02/skys-almost-limit-high-cpu-is-no-more.html). The size of jruby-complete.jar is unfortunately over 10MB. The shell script to split jruby-complete.jar up into two jar archives has been introduced at http://olabini.com/blog/2009/04/jruby-on-rails-on-google-app-engine/. However, I learned the way in the blog was already obsolete when I filed JRUBY-3949. The smart way of doing that was already out there. The latest JRuby, I mean, JRuby 1.4.0dev in git HEAD has had a Rakefile to create jars.

This is what I acutually did on a terminal:

$ git clone git://kenai.com/jruby~main
$ cd jruby~main
$ export JRUBY_HOME=`pwd`
$ PATH=$JRUBY_HOME/bin:$PATH
$ ant
$ gem install rake
$ gem install hoe
$ cd gem
$ rake update

Then, jruby-core-1.4.0dev.jar and jruby-stdlib-1.4.0dev.jar was built in jruby~main/gem/lib directory.

Wednesday, September 02, 2009

Finally yaml worked on Google App Engine

In my previous post, I wrote about my struggle over an application on Google App Engine that uses JRuby's builtin library, yaml. I found the reason of the error at JRUBY-3892. Builtin library needs jruby.home environment variable to be set correctly, and it should be done in jruby-complete.jar. But, jruby-complete.jar built from old JRuby 1.4.0dev had a somewhat broken path. Since the issue has been resolved in the end of Auguest, I tried again using the latest JRuby 1.4.0dev cloned out from git repo. It worked. So, the final release of JRuby 1.4.0 won't have this problem.

Now, my sample Servlet, ParsenRunServlet is working at http://servletgarden-in-red.appspot.com/. If you are interested in the code, those are in JRuby Embed API Wiki.

Saturday, August 29, 2009

Yaml doesn't work on Google App Engine

While I was testing JRuby Embed API on Google App Engine, I encountered this awkward problem. Yaml never worked on GAE. Exactly the same Servlet successfully worked on GlassFish. Servlet and Ruby codes were:

# yaml_snippet.rb

require 'yaml'

content = YAML::load @text

def format element
case element
when String: print "<p>#{element}</p>"
when Array:
print "<ul>"
element.each do |child|
print "<li>"
format child
print "</li>"
end
puts "</ul>"
when Hash:
element.each do |key, value|
print "<ul><li>#{key}"
format value
print "</li></ul>"
end
end
end

content.each do |heading, paragraph|
puts "<h4>#{heading}</h4>"
paragraph.each do |element|
format element
end
end

package olive.jruby.example;

import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.PrintWriter;
import java.util.Arrays;
import java.util.List;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.jruby.embed.ScriptingContainer;
import org.jruby.javasupport.JavaEmbedUtils.EvalUnit;

public class YamlSampleServlet extends HttpServlet {
private ScriptingContainer container;
private EvalUnit yaml_unit;
private String text =
"Trees:\n" +
"- This is a small example to general HTML.\n" +
"- - Quince\n" +
" - flower: Red\n" +
"- - Apple\n" +
" - fruit: Red\n" +
"- - Maple\n" +
" - leaf: Red";

@Override
public void init() {
String classpath = getServletContext().getRealPath("/WEB-INF/classes");
List<String> loadPaths = Arrays.asList(classpath.split(File.pathSeparator));
container = new ScriptingContainer();
container.getProvider().setLoadPaths(loadPaths);
String filename = "ruby/yaml_snippet.rb";
InputStream istream = container.getRuntime().getJRubyClassLoader().getResourceAsStream(filename);
yaml_unit = container.parse(istream, filename);
}

protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
response.setContentType("text/html;charset=UTF-8");
PrintWriter out = response.getWriter();
container.setWriter(out);
try {
out.println("<html>");
out.println("<head>");
out.println("<title>Servlet YamlSampleServlet</title>");
out.println("</head>");
out.println("<body>");
out.println("<h3>Servlet YamlSampleServlet at " + request.getContextPath() + "</h3>");
container.put("@text", text);
yaml_unit.run();
out.println("</pre></body>");
out.println("</html>");
} finally {
out.close();
}
}

@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}

@Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}

@Override
public String getServletInfo() {
return "Yaml Sample";
}
}

And the programs produced:

<html>
<head>
<title>Servlet YamlSampleServlet</title>
</head>
<body>
<h3>Servlet YamlSampleServlet at /Olive</h3>
<h4>Trees</h4>
<p>This is a small example to general HTML.</p><ul><li><p>Quince</p></li><li><ul><li>flower<p>Red</p></li></ul></li></ul>
<ul><li><p>Apple</p></li><li><ul><li>fruit<p>Red</p></li></ul></li></ul>
<ul><li><p>Maple</p></li><li><ul><li>leaf<p>Red</p></li></ul></li></ul>
</pre></body>
</html>

However, this test servlet never worked on GAE because of the exception:

[java] yaml/constants:15:in `const_missing': uninitialized constant YAML::Yecht::Resolver (NameError)
[java] from yaml:88
[java] from yaml:2:in `require'
[java] from ruby/yaml_snippet.rb:2
[java] ...internal jruby stack elided...
[java] from Module.const_missing(yaml:88)
[java] from (unknown).(unknown)(yaml:2)
[java] from (unknown).(unknown)(yaml:2)
[java] from Kernel.require(ruby/yaml_snippet.rb:2)
[java] from (unknown).(unknown)(:1)

JRuby has yaml library under the "builtin" directory in its source tree, so all of necessary scripts here are in jruby.jar as well as jruby-complete.jar. But, yaml library needs jruby.home system property to be set correctly, so jruby-complete.jar is the only choice on a web application. In case of an application, I could set jruby.home explicitly as in below and got the result as I expected:

package brick;

import java.io.InputStream;
import org.jruby.embed.ScriptingContainer;
import org.jruby.javasupport.JavaEmbedUtils.EvalUnit;

public class YamlSample {

private String filename = "ruby/yaml_snippet.rb";
private String text =
"Trees:\n" +
"- This is a small example to general HTML.\n" +
"- - Quince\n" +
" - flower: Red\n" +
"- - Apple\n" +
" - fruit: Red\n" +
"- - Maple\n" +
" - leaf: Red";

private YamlSample() {
System.setProperty("jruby.home", "/Users/yoko/Works/080909-jruby/jruby~main");
ScriptingContainer container = new ScriptingContainer();
InputStream istream = container.getRuntime().getJRubyClassLoader().getResourceAsStream(filename);
EvalUnit yaml_unit = container.parse(istream, filename);
container.put("@text", text);
yaml_unit.run();
}

public static void main(String[] args) {
new YamlSample();
}
}

I tried to copy all yaml library under the source tree of my web application, and clean & build, redeploy, and request again... with no luck. Sigh... I tried to set the path to the yaml library in jar archive,

String classpath = getServletContext().getRealPath("/WEB-INF/lib/jruby-complete.jar!/builtin");
List loadPaths = Arrays.asList(classpath.split(File.pathSeparator));
container = new ScriptingContainer();
container.getProvider().setLoadPaths(loadPaths);

... no luck. The same exception, as ever.

Since the program worked on GlassFish, I guess the difference in class loading mechanism between GlassFish and GAE might have caused the exception on GAE. But, I haven't figured the culprit out so far. Any idea?