Sunday, February 27, 2011

Rails on RedBridge, Scaffolded App to Work

In my blog post, "The second step to Rails on RedBridge", I used Rails simple controller and showed how it could get run on Servlet. That was easy since there was no interaction with a Web browser. That controller just returned a result to the browser. That's it. So, Rails didn't need much information to work. However, a controller-only application isn't common. People start using Rails from scaffolding. Every Rails book talks about scaffolding first. Thus, I tried to make scaffolded Rails app to work on the Servlet. So far so good. Though it was a newbie app, scaffolding has been covered by Rails on RedBridge. The scaffolded app worked via JRuby's embedding API on the Servlet. The code is on github, https://github.com/yokolet/RailsCrossing, and the sample app by Scala is also on github, https://github.com/yokolet/RailsCrossingSamples.

The way to make it work was a bit hard. Rails API doc was not so helpful in terms of hooking it up on the Servlet. To figure out how it should have worked, I used rails console a lot, read sources, watched what were supposed to be in HTTP requests/responses over FireBug, wrote snippets to see what was missing. So, I'm going to write down how RailsCrossing (Rails on RedBridge) made scaffolded app to work.

The idea of RailsCrossing is to hit "SomeController.action('some_action').call(env)" method. Thus,

  1. Find the controller class
  2. Find the action
  3. Stuff all necessary information into "env" hash

are all to make it work.


Finding controller class is not so complicated. For example, we can confirm routes mapping on rails console:

irb(main):001:0> ActionDispatch::Routing::Routes.routes.to_s
=> "GET /Acacia/home/index(.:format) {:controller=>\"Acacia/home\", :action=>\"index\"}GET /Acacia/users(.:format) {:action=>\"index\", :controller=>\"users\"}POST /Acacia/users(.:format) {:action=>\"create\", :controller=>\"users\"}GET /Acacia/users/new(.:format) {:action=>\"new\", :controller=>\"users\"}GET /Acacia/users/:id/edit(.:format) {:action=>\"edit\", :controller=>\"users\"}GET /Acacia/users/:id(.:format) {:action=>\"show\", :controller=>\"users\"}PUT /Acacia/users/:id(.:format) {:action=>\"update\", :controller=>\"users\"}DELETE /Acacia/users/:id(.:format) {:action=>\"destroy\", :controller=>\"users\"}ANY /rails/info/properties(.:format) {:controller=>\"rails/info\", :action=>\"properties\"}"

Odd thing was the scope name "Acacia" appeared in a controller name when I created controller by "generate controller" but not by "generate scaffold." Although I surrounded whole stuff by a scope "/Acaia," routes.rb, it happened. I'm not sure this is intended or a bug, but, anyways, I took the scope name out from to find controller class if it is included.


Finding an action needed closer look at a Rails behavior. As you may know, Rails RESTful app uses GET, POST, PUT, and DELETE HTTP requests for index/show/new/edit, create, update, and destroy actions respectively. Even if the given path is the same, say /Acacia/users/1, it is for show when HTTP method is GET while update when PUT. Rails uses "_method" hidden HTML form field to know the difference. So, I added lines to get that parameter from HTTP request, then put in env hash with a "rack.methodoverride.original_method" key. Also the _method value was used to find out matched action and controller combination.
(See getEnv() and findMatchedRoute() methods in CrossingHelpers.java)


Stuffing all necessary information in the "env" hash was rather baffling. JRuby's Java proxy didn't work well since Rails expected hash keys could be referenced as a symbol. For example, a hash is created by h = {"name" => "foo"}, it should be referenced by h[:name]. JRuby's Java String proxy seemed not to be effective for this usage. To avoid this problem, I directly used JRuby's RubyHash class.

Next problem was keys to assign input parameters. "rack.input" is mandatory, but, as far as sending HTML form parameters, "rack.input" isn't used for that purpose. It is used to compare the value of "rack.request.form_input," and not used afterward as well as "rack.request.form_input." (If input is multipart/form-data (file upload), "rack.input" is used to read binary data, though.) Instead, "rack.request.form_hash" key is used to send input values. There's one more key-value pair involved in sending form parameters. It is "action_dispatch.request.path_parameters." The id, like user id, seems to be expected to be there. I put the hash got by Rails.application.routes.recognize_path("some path") as a value of ""action_dispatch.request.path_parameters." The example of a hash value related to sending form parameters became as in below:

"action_dispatch.request.path_parameters" => {:action=>"show", :controller=>"users", :id=>"1"}
"rack.input" => ""
"rack.request.form_input" => ""
"rack.request.form_hash" => {"utf8"=>"✓", authenticity_token"=>"fi/oOym8i+mZTwM7+h+rZBQL/s9hv62+
mnNRv6mNQnw=", "user"=>{"name"=>"foo", "email"=>"foo@bar"}, "commit"=>"Update User"}


Moreover, there are "flash" messages, such as green "User was successfully updated." Those also should be in "env" hash. This sort of messages are in a HTTP response returned as a result of form input processing. When the response is redirected and back to Rails app, the messages show up to browser. To keep the messages over a sequence of HTTP request/response has been completed, we need cookie like mechanism. Rails uses cookie to save such messages. On Servlet, we can use Servlet API's session, which is available to save an object. So, I took the hash for flash massages out from the response, then put it in Servlet's session without any modification.

String script =
"response = " + route.getName() + ".action('" + route.getAction() + "').call(env)\n" +
"return response[0], response[1], response[2].body, response[2].request.flash";
RubyArray responseArray = (RubyArray)container.runScriptlet(script);
CrossingResponse response = new CrossingResponse();
response.context_path = context_path;
response.status = ((Long)responseArray.get(0)).intValue(); //status code; Fixnum
response.responseHeader = (Map)responseArray.get(1); // response header; Hash
response.body = (String)responseArray.get(2); // response body; HTML
response.flash = (Map) responseArray.get(3); // flash messages; Hash

....

request.getSession().setAttribute("action_dispatch.request.flash_hash", crossingResponse.getFlash()); //save falsh messages to Servlet's session

....

// before sending a request to Rails app
Map map = (Map) request.getSession().getAttribute("action_dispatch.request.flash_hash");
if (map != null) env.put("action_dispatch.request.flash_hash", map);



Serving static assets, such as stylesheets and javascripts, needed to have another way. Even though I added the scope in routes.rb, Rails didn't add the scope name to static assets not like paths to displayed in HTML body part. There should have the Rails way to add some path to static assets always, but I didn't search that. Because I wanted to keep Rails app as much as it is. Instead, RailsCrossing rewrites the URI so that the Servlet context path (the same as the Rails scope name) will be in the URI just before the response is sent back to the browser. RailsCrossing has its own default Servlet (CrossingDefaultServlet.java) to serve such static files. This is necessary so that Servlet Container dispatches the HTTP request to Acacia web app. Unless, the request is dispatched to a Servlet Container's default Servlet that doesn't know where the files are.


Above are what I tried to get scaffolded Rails app to work. Probably, there are the right ways to get this done, but I couldn't find by googling. (I want to know where I should go.)


RailsCrossing is still on the long way to serve Rails app satisfactory, but getting closer and closer. As I wrote, simple scaffolded Rails app has been covered. Next would be Ajax request handling, file upload, complicated database schema, etc. ... maybe, I need to work more.

Tuesday, February 15, 2011

Using Rails from Scala

Using Rails from Scala? Yes, it is possible if you use RailsCrossing (https://github.com/yokolet/RailsCrossing). This is a following chapter of my previous blog post, The second step to Rails on RadBridge. As I wrote "I'll try Rails from Scala next." at the end of that blog post, I tried it today. How did it go? It worked just fine. Here's how I used Rails from Scala.


1. Preparation of Java web application project and Rails app

These are exactly the same as the previous post. See steps 1 through 3. This time, I named Java web application "Sycamore," but the rest of all are the same.


2. Scala Servlet for Rails

Writing Servlet by Scala is really easy. I read the article, The busy Java developer's guide to Scala: Scala and servlets for that and could easily follow. I don't explain about what is Scala Servlet because that's not my purpose. So, it's good for you to google or goto some sites to learn what it is.

Here's the Scala Servlet that initializes Rails app and dispatches HTTP request to Rails app:

package com.servletgarden.sycamore

import javax.servlet.ServletConfig
import javax.servlet.http.{HttpServlet, HttpServletRequest => HSReq, HttpServletResponse => HSResp}
import com.servletgarden.railsxing.CrossingServlet

class RailsScalaServlet extends CrossingServlet {
override def init(config: ServletConfig) = super.init(config)

override def doGet(request: HSReq, response: HSResp) = dispatch(request, response)

override def doPost(request: HSReq, response: HSResp) = dispatch(request, response)
}

Compare RailsScalaServlet above and SimpleSample Servlet in the previous post. Basically, both Servlet do the same thing. See the previous post for details about what those two servlets are doing.

That's it.


3. Scala Servlet Compilation

Before executing Scala Servlet, the Servlet needs to be compiled. The article mentioned above, "The busy...," explains how to do that. Also, Scala Ant Tasks section of Developer's Guides will help you to understand how to write build.xml. I'm using NetBeans and its Java web application projects, so I directly edited NetBeans' build.xml. My build.xml is https://gist.github.com/828420. It assumes the directory tree below:

Sycamore -+- src -+- java -+- com -+- servletgarden -+- sycamore -+- RailsScalaServlet.scala
+- web -+- META-INF -+- context.xml
+- WEB-INF -+- lib -+- Gemfile
| +- Gemfile.lock
| +- RailsCrossing.jar
| +- blog -+- Gemfile
| | +- README
| | +- Rakefile
| | +- ...
| +- jruby-complete.jar
| +- jruby -+- 1.8 -+- bin -+- bundle
| | | +- erubis
| | | +- ....
| | +- cache -+- ...
| | +- doc
| | +- gem -+- abstract-1.0.0 -+- ..
| | +- gem -+- .... -+- ..
| | +- gem -+- bundler-1.0.10 -+- ..
| | +- gem -+- .... -+- ..
| | +- specifications -+- ...
| +- scala-compiler.jar
| +- scala-library.jar
|
+- index.jsp

In my case, NetBeans' build menu compiles Scala files and puts compiled classes under Sycamore/build/web/WEB-INF/classes directory.


4. Configurations

SimpleSample Servlet in the previous post uses annotation to configure the Servlet, thus, no web.xml is there. On the other hand, RailsScalaServlet uses web.xml to set rail_path and gem_path, and path mapping. The web.xml file is https://gist.github.com/828572.

Double check the context name of this Java web application has been changed to "blog." See "5. One More Setup" of the previous post.


5. Run Scala Servlet to use Rails

After starting tomcat, request http://localhost:8080/blog/home/index from a web client. Like in the previous post, familiar Rails default page will show up.



As I wrote how to here, using Rails from Scala is relatively easy. If you want to mix Scala code and Rails results, you can use methods of CrossingHelpers bypassing CrossingServlet. Also, you can use JRuby's ScriptingContainer to take Rails classes out to call methods of them. Try this new Rails way.

Monday, February 14, 2011

The second step to Rails on RedBridge

This is the second attempt to make Rails work on RedBrdige (JRuby embedding API). I believe Rails on RedBridge got closer to a real application. The first attempt is also in this blog, A small step to Rails on RedBridge. The blog post had some impact on a few people who want to control Rails (or Sinatra) from Java. The example there successfully showed how we could wake Rails up from Java Servlet, but was almost a hello world example. Meanwhile, I demonstrated simple Rails app from Groovy (Groovy Servlet, precisely) in my session at RubyConf 2010, "RubyGems to All JVM Languages." Rails on RedBridge way is an approach from Java API on Servlet, so all JVM languages are possibly available to use Rails *gem* from them over JRuby's Java API. That demonstration at RubyConf 2010 impressed the audiences there enough to have a clap, but I knew it was still a hello world example. Recently, I tried to develop Rails on RedBridge more so that Rails app got work like other rackup style frameworks. RailsCrossing is the outcome. It is the Java API to use Rails.


OK. I'm going to write how to setup and write code using RailsCrossing.


1. Creating Java Web Application Project

The first job is to create a Java web application project. I used NetBeans for that. You can use whatever IDE you like that has the feature to create a Java Servlet based web application project. My web app project has a name "Spruce" and the server to run it is Tomcat 7.0.4.

Initially, a directory tree of the Spruce project looks like below (I didn't write NetBeans specific files/directories):

Spruce -+- src
+- web -+- META-INF -+- context.xml
+- WEB-INF
+- index.jsp


The directory tree is slightly different when other IDEs are used. That's not the matter. Just look at WEB-INF directory. The WEB-INF directory is special for Java Servlet web app. It is the place to put libraries and is protected from accesses originated from web clients.


2. Prepare for Rails APP

RailsCrossing assumes Rails app is located under WEB-INF directory. It could not be necessarily there. However, I made that constraint since Java Servlet web application should be portable. Everything needed to the web app work should be in the web app directory tree. It's the policy.

Let's prepare the app.

First, get jruby-complete.jar and put it in WEB-INF/lib directory. (Get the latest (master) branch of JRuby. Or, you can use jruby-complete.jar of RailsCrossing project)

Spruce -+- src
+- web -+- META-INF -+- context.xml
+- WEB-INF -+- lib -+- jruby-complete.jar
+- index.jsp


Next, install bundler gem. Make sure bundler will be installed under WEB-INF directory. So, let's go to lib directory and type as in below (you can use jruby command instead):

$ java -jar jruby-complete.jar -S gem install bundler --no-ri --no-rdoc -i jruby/1.8
Fetching: bundler-1.0.10.gem (100%)
Successfully installed bundler-1.0.10
1 gem installed

Don't forget -i option to install bundler gem under lib directory. The command above installs bundler gem in WEB-INF/lib/jruby/1.8. So, you can find bundle command in the path, WEB-INF/lib/jruby/1.8/bin/bundle.

Spruce -+- src
+- web -+- META-INF -+- context.xml
+- WEB-INF -+- lib -+- jruby-complete.jar
| +- jruby -+- 1.8 -+- bin -+- bundle
| +- cache -+- ...
| +- doc
| +- gem -+- bundler-1.0.10 -+- ..
| +- specifications -+- ...
|
+- index.jsp


Let's set GEM_PATH to use bundler to install other gems.
$ export GEM_PATH=[path to lib directory]/jruby/1.8


Write or create Gemfile. If you want to use bundle command rather than open a new file by editor, type as in below:
java -jar jruby-complete.jar -S jruby/1.8/bin/bundle init

Gemfile should be something like this:

# A sample Gemfile
source "http://rubygems.org"

gem "rails"

I just deleted the comment sign (#) on the rails line after bundle init command. The last preparation is to install rails gem.
java -Xmx500m -jar jruby-complete.jar -S jruby/1.8/bin/bundle install --path .

Don't forget "--path ." to install gems under the same path as bundler has been installed in. When bundle install command successfully finished, bunch of gems are in jruby/1.8.

Spruce -+- src
+- web -+- META-INF -+- context.xml
+- WEB-INF -+- lib -+- Gemfile
| +- Gemfile.lock
| +- jruby-complete.jar
| +- jruby -+- 1.8 -+- bin -+- bundle
| | +- erubis
| | +- ....
| +- cache -+- ...
| +- doc
| +- gem -+- abstract-1.0.0 -+- ..
| +- gem -+- .... -+- ..
| +- gem -+- bundler-1.0.10 -+- ..
| +- gem -+- .... -+- ..
| +- specifications -+- ...
|
+- index.jsp



3. Create Rails App

Probably, I don't need to say anything. I typed below:
java -jar jruby-complete.jar -S jruby/1.8/bin/rails new blog --template http://jruby.org

Now, blog Rails app is created under WEB-INF/lib/blog.

Spruce -+- src
+- web -+- META-INF -+- context.xml
+- WEB-INF -+- lib -+- Gemfile
| +- Gemfile.lock
| +- blog -+- Gemfile
| | +- README
| | +- Rakefile
| | +- ...
| +- jruby-complete.jar
| +- jruby -+- 1.8 -+- bin -+- bundle
| | +- erubis
| | +- ....
| +- cache -+- ...
| +- doc
| +- gem -+- abstract-1.0.0 -+- ..
| +- gem -+- .... -+- ..
| +- gem -+- bundler-1.0.10 -+- ..
| +- gem -+- .... -+- ..
| +- specifications -+- ...
|
+- index.jsp


In general, database setup follows. But, to make it understandable to know how it works, let's create a controller.

$ cd blog (Now, I'm in Spruce/web/WEB-INF/lib/blog directory)
$ java -Xmx500m -jar ../jruby-complete.jar ../jruby/1.8/bin/bundle install --path ../. (Don't forget --path ../. option!)
$ java -jar ../jruby-complete.jar script/rails g controller home index

So far so good. These steps to creating Rails app should be familiar with.


4. Simple Sample Servlet to run Rails' controller

Here comes RailsCrossing API. You can write Servlet using methods in CrossingHelpers or subclassing CrossingServlet. The CrossingServlet does everything to run Rails' controller. For example, SimpleSample below would be the simplest example.

package com.servletgarden.spruce;

import com.servletgarden.railsxing.CrossingServlet;
import java.io.IOException;
import javax.servlet.Servlet;
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.annotation.WebInitParam;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
*
* @author Yoko Harada
*/
@WebServlet(name="SimpleSample",
urlPatterns={"/*"},
initParams={@WebInitParam(name="rails_path", value="/lib/blog"), @WebInitParam(name="gem_path", value="/lib/jruby/1.8")})
public class SimpleSample extends CrossingServlet {

/**
* @see Servlet#init(ServletConfig)
*/
@Override
public void init(ServletConfig config) throws ServletException {
super.init(config);
}

/**
* Processes requests for both HTTP GET and POST methods.
* @param request servlet request
* @param response servlet response
* @throws ServletException if a servlet-specific error occurs
* @throws IOException if an I/O error occurs
*/
protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {

dispatch(request, response);
}

/**
* Handles the HTTP GET method.
* @param request servlet request
* @param response servlet response
* @throws ServletException if a servlet-specific error occurs
* @throws IOException if an I/O error occurs
*/
@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}

/**
* Handles the HTTP POST method.
* @param request servlet request
* @param response servlet response
* @throws ServletException if a servlet-specific error occurs
* @throws IOException if an I/O error occurs
*/
@Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}

/**
* Returns a short description of the servlet.
* @return a String containing servlet description
*/
@Override
public String getServletInfo() {
return "the simplest example used RailsCrossing";
}

}


Let's look at what's going on in detail.

In the SimpleSample init method, it calls init method of a super class, CrossingServlet. In CrossingServlet init method, initialization of ScriptingContainer (JRuby embedding API a.k.a. RedBridge) and Rails are done. Then, Rails routes mappings are parsed.

public void init(ServletConfig config) throws ServletException {
container = CrossingHelpers.initialize(config);
routes = CrossingHelpers.parseRoutes(container);
}

While initialization is going on CrossingServlet uses "rails_path" and "gem_path" initParams relative to WEB-INF directory. These are necessary to fire up Rails.


Well, how Rails routes mapping can be parsed? Probably, many of you have used rails console. RailsCrossing does the same thing on a Servlet as people do on rails console. For example,

$ java -jar ../jruby-complete.jar script/rails c
irb(main):007:0> route_ary = ActionDispatch::Routing::Routes.routes
=> [#<ActionDispatch::Routing::Route:0x555669ae @name="home_index"...(snip)
irb(main):004:0> route_ary[0].to_a[1]
=> {:path_info=>/\A\/home\/index(?:\.([^\/.?]+))?\Z/, :request_method=>/^GET$/}
irb(main):005:0> route_ary[0].to_a[2]
=> {:controller=>"home", :action=>"index"}
irb(main):006:0> route_ary[0].to_a[3]
=> "home_index"

CrossingHelpers.parseRoutes method exactly evaluates these commands to draw mapping info out from Rails.


Let get back to SimpleSample Servlet. Look at the processRequest method. In this method, CrossingServlet's dispatch method is called. Just that. What's this dispatch?
Here it is:

protected void dispatch(HttpServletRequest request, HttpServletResponse response) throws IOException {
CrossingRoute route = findMatchedRoute(request.getContextPath(), request.getPathInfo(), request.getMethod());
if (route == null) return;
Map env = getEnvMap(request);
CrossingResponse crossingResponse = CrossingHelpers.dispatch(container, route, env);
response.setStatus(crossingResponse.getStatus());
Set keys = crossingResponse.getResponseHeader().keySet();
for (String key : keys) {
String value = crossingResponse.getResponseHeader().get(key);
response.setHeader(key, value);
}
PrintWriter writer = response.getWriter();
writer.write(crossingResponse.getBody());
}

Based on the parsed routes, CrossingServlet finds what controller should be used from path_info in HttpServletRequest. If matched controller is found, it creates a map of each HTTP request params including HTTP request header and query string. Then, dispatching. Here, RailsCrossing does rails console commands on Servlet again.

irb(main):006:0> env = {"rack.input" => "", "REQUEST_METHOD" => "GET"}
=> {"rack.input"=>"", "REQUEST_METHOD"=>"GET"}
irb(main):008:0> HomeController.action('index').call(env)[0]
=> 200
irb(main):009:0> HomeController.action('index').call(env)[1]
=> {"Content-Type"=>"text/html; charset=utf-8", "ETag"=>"\"a5e60d2fa2208dc316b0f09cef107bba\"", "Cache-Control"=>"max-age=0, private, must-revalidate"}
irb(main):010:0> HomeController.action('index').call(env)[2].body
=> "<!DOCTYPE html>\n<html>\n<head>\n <title>Blog</title>\n...

Once CrossingServlet gets the response, it sets HTTP response status code and header params, then writes html part out to Servlet's writer.

These are what RaisCrossing does.


5. One More Setup

The difference of a context path lies between Java web app and Rails app. In general, Java web application is referenced by http://servername:port/context_name/servlet_name/path_info, while rails is done by http://servername:port/path_info. We need to fix these to make them coincide.

Let's change Rails first editing config/routes.rb:

Blog::Application.routes.draw do
scope "/blog" do
get "home/index"
...
end
end

I added scope in config/routes.rb. Now, blog app is referenced by http://servername:port/blog/path_info. Next is a tomcat setting. I edited META_INF/context.xml:

<?xml version="1.0" encoding="UTF-8"?>
<Context antiJARLocking="true" path="/blog"/>

Originally, it was path="/Spruce" since I named the web app Spruce. After Spruce has been changed to "blog," every HTTP request to http://servername:port/blog/* has been directed to SimpleSample Servlet.


6. Run Servlet

Since I created web app on NetBeans, I can get SimpleSample Servlet started from NetBeans menu with the path, "/home/index" If not, start tomcat up and request "http://localhost:8080/blog/home/index" You'll see familiar controller default output on a browser.


7. Thoughts

RailsCrossing uses Rails metal introduced in version 3. This rack style invocation was easy to get it run. However, it is mostly undocumented area. I attempted many commands on rails console and tried to find the best way; however, there might be still better way.

A good side of RailsCrossing is ... it is a Java API. This means, JVM languages such as Scala, Groovy, or other can use RailsCrossing over java integration feature. I'll try Rails from Scala next.

Wednesday, January 12, 2011

Embedding API refinements for JRuby 1.6.0RC1

TThe biggest release ever --- JRuby 1.6.0RC1 has been released on Jan. 10, 2011. Headius wrote a blog about this biggest-ever release, JRuby 1.6.0 RC1 Released. Ruby 1.9 compatibility, performance improvement, built-in graph profiler, and more. How about embedding API (RedBridge)? In the release note (http://jruby.org/2011/01/10/jruby-1-6-0-rc1.html), you'll find a line mentioned about RedBridge, "Embedding API refinements." I'm going to write what refinements RedBridge had in JRuby 1.6.0RC1 in this blog post.


1. New Context Instance Type, concurrent

So far, RedBridge has had three types of context models, singleton, singlethread, and threadsafe. JRuby 1.6.0 added one more type, concurrent. The concurrent type is somehow mixture of singleton and threadsafe. Ruby runtime is a singleton, variables (except global variables) are thread local. This model would be good to use RedBridge on servlet based web applications. For example, we can instantiate Ruby runtime and evaluate all Ruby codes during servlet’s initialization, then use them in HTTP request processing methods, such as doGet() or doPost(). Threadsafe context model is similar, but Ruby runtimes are on each thread (each HTTP request on a web app). On the other hand, the runtime in the same classloader will be used on the concurrent context model on all threads, if Ruby runtime has been already initialized by another app.

The concurrent model is a bit complicated and assuming users have enough knowledge about Java web application and Ruby's variable/constant scope. Global variables are totally unprotected from concurrent processing since Ruby runtime is a singleton. As you may know the scope of a global variable is global on Ruby runtime, so is the concurrent context. Global variables of concurrent model are looked up from all threads (all HTTP requests on a web app) simultaneously. The scope of other variable types and a constant depends on how those are used. If those are top-level variables/constants, the scope will be global since Runtime is a singleton. If not, those are thread local and protected from concurrent processing.

When you use the concurrent context model, specify as in below:

Embed core
ScriptingContainer container = new ScriptingContainer(LocalContextScope.CONCURRENT);

JSR223
System.property(“org.jruby.embed.localcontext.scope”, “concurrent”);



2. Receiver aware sharing variables

I’ve written about this new feature in the blog post, http://yokolet.blogspot.com/2010/09/new-featues-of-embedding-api-for-jruby.html ScriptingContainer.get()/put()/remove() methods had a receiver object in those method arguments. If the receiver object is not specified, runtime top-self is used as a receiver. This means those operations are not thread safe on concurrent context type. Be careful.
Also, see the lazy variable retrieval feature of the same blog post. This is for performance improvement.


3. Classloader option for JSR223

The new option to set classloader into ScriptingContainer via ScriptEngineFactory has been added from JRuby 1.6.0RC1. This is for users who want to use JSR223 ScriptEngine for JRuby on some custom classloader. Setting a classloader of ScriptingContainer itself to Ruby runtime is for now well-known technique on custom classloader. The problem was JSR223 doesn’t have a method for that. So, JRuby 1.6.0 sets the classloader that is used to instantiate ScriptingContainer to Ruby runtime by default. This is done in ScriptEngineFactory#getScriptEngine() method internally. This behavior can be disabled by a system property, org.jruby.embed.classloader.

System.setProperty(“org.jruby.embed.classloader”, “current”); // default, sets classloader
System.setProperty(“org.jruby.embed.classloader”, “none”); // doesn't set classloader


The option is JSR223 only. The org.jruby.embed.classloader system property has no effect when you directly instantiate ScriptingContainer. Set whatever appropriate classloader using ScriptingContainer.setClassLoader() method.


4. New Map proxy

This is not a embedding API but JRuby core. Also the feature is not only for RedBridge users but also for all JRuby users. However, the new proxy would make RedBridge users happy especially, so I’ll add this feature here.

Since JRuby 1.6.0RC1, a java.util.Map type object has all Ruby Hash methods by default. Let’s see example code.

ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
ConcurrentHashMap map1 = new ConcurrentHashMap();
map1.put("a", 100);
map1.put("b", 200);
Map map2 = new HashMap();
map2.put("b", 254);
map2.put("c", 300);
container.put("map1", map1);
container.put("map2", map2);
String script =
"map1.merge!(map2) {|k,o,n| o+n }\n" +
"puts \"#{map1.inspect}, length = #{map1.length}\"\n" +
"puts \"class: #{map1.class}\"";
container.runScriptlet(script);

Set entries = map1.entrySet();
for (Map.Entry entry : entries) {
System.out.print(entry.getKey() + ": " + entry.getValue() + ", ");
}

Above code prints:

{"a"=>100, "b"=>454, "c"=>300}, length = 3
class: Java::JavaUtilConcurrent::ConcurrentHashMap
a: 100, b: 454, c: 300,

Two Map type objects were merged as instructed, map1.merge!(map2) {|k,o,n| o+n }. Also in Java world, map1 was changed exactly the same way as was in Ruby world.


These are the brief summary of embedding API refinement done in JRuby 1.6.0RC1. Have fun and please report if you find a bug or encounter a weird behavior.

Friday, October 22, 2010

Gems in a Jar with RedBridge

Using JRuby embed API (RedBridge), it is easy to use Ruby gems from Java. However, packaging might not be simple, so people occasionally struggle to create a "portable" package. Being portable is important for a Java app. All stuffs of the Java app should be packaged in a jar archive, which should work on another PC, or even different OS. This is the good side of Java. This blog illustrates one solution for packaging.

I intentionally didn't use jruby command since the command hides what are going on behind. You might be exhausted by lines of "java -jar ....," sorry. But, it helps to understand a packaging process.

Directories


Since I used bundler to install gems, I created directories below to fit them to bundler.

Linden -+- lib -+- jruby -+- 1.8
+- src -+- linden
+- build

"Linden" is a top directory and the name doesn't have any special meaning. You can name it whatever you like. "lib/jruby/1.8" is the directory gems will be installed. "src" is for Java code, and "build" is for compiled *.class files.

Bundler Installation


I intentionally used jruby-complete.jar to assure gems won't be installed in the default location. The path to jruby-complete.jar can be anything since typing a full path to jruby-complete.jar works. But, I like a short path to type, so I copied jruby-complete.jar under "Linden/lib." Now, the directories/files were as in below:

Linden -+- lib -+- jruby -+- 1.8
+- jruby-complete.jar
+- src -+- linden
+- build


Then, the bundler installation went:

cd lib
java -jar jruby-complete.jar -S gem install bundler jruby-openssl --no-ri --no-rdoc -i jruby/1.8

Bundler and jruby-openssl gems were installed and the directories became as in below:

Linden -+- lib -+- jruby -+- 1.8 -+- bin -+- bundle
| +- cache -+- ...
| +- doc
| +- gems -+- bouncy-castle-java-1.5.0145.2 -+- ...
| | +- bundler-1.0.3 -+- ...
| | +- jruby-openssl-0.7.1 -+- ...

| +- specifications -+- ...
+- jruby-complete.jar
+- src -+- linden
+- build

Let's see whether gems are really installed.

export GEM_PATH=`pwd`/jruby/1.8
java -jar jruby-complete.jar -S gem list

This command should print out the installed three gems and pre-installed gems.

*** LOCAL GEMS ***

bouncy-castle-java (1.5.0145.2)
bundler (1.0.3)
columnize (0.3.1)
jruby-openssl (0.7.1)
rake (0.8.7)
rspec (1.3.0)
ruby-debug (0.10.3)
ruby-debug-base (0.10.3.2)
sources (0.0.1)

OK. Bundler was installed successfully.

Installation of other gems


Next step is to install other gems using bundler, so I used bundle command. Theoretically, PATH environment variable should work to find bundle command by ruby. Unfortunately, this didn't work in my "java -jar ..." usage. Instead, I typed a path to bundle command after -S option.

java -jar jruby-complete.jar -S jruby/1.8/bin/bundle init

Then, I added "twitter" gem to Gemfile.

# A sample Gemfile
source "http://rubygems.org"

# gem "rails"

gem "twitter"

The installation went on:

java -Xmx500m -jar jruby-complete.jar -S jruby/1.8/bin/bundle install --path=.

The java command option "-Xmx500m" is for performance and to avoid memory outage. Now, 6 gems were installed and the directories/files became:

Linden -+- lib -+- jruby -+- 1.8 -+- bin -+- bundle
| | +- httparty
| | +- oauth

| +- cache -+- ...
| +- doc
| +- gems -+- bouncy-castle-java-1.5.0145.2 -+- ...
| | +- bundler-1.0.3 -+- ...
| | +- crack-0.1.8 -+- ...
| | +- hashie-0.4.0 -+- ...
| | +- httparty-0.6.1 -+- ...

| | +- jruby-openssl-0.7.1 -+- ...
| | +- multi_json-0.0.4 -+- ...
| | +- oauth-0.4.3 -+- ...
| | +- twitter-0.9.12 -+- ...

| +- specifications -+- ...
+- jruby-complete.jar
+- Gemfile
+- Gemfile.lock

+- src -+- linden
+- build


Java code to use twitter gem


Since all gems were ready, I wrote Java code to see gems worked. This time, I relied on GEM_PATH environment variable to find gems. The fist code had Java package, linden, and class name, SearchSample. So, I created SearchSample.java file under "src/linden" directory.

package linden;

import org.jruby.embed.LocalContextScope;
import org.jruby.embed.ScriptingContainer;

public class SearchSample {
private String jarname = "sample.jar";

private SearchSample() {
String basepath = System.getProperty("user.dir");
System.out.println("basepath: " + basepath);
ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
System.out.println("jrubyhome: " + container.getHomeDirectory());
container.runScriptlet("ENV['GEM_PATH']='" + basepath + "/lib/jruby/1.8'");

String script =
"require 'rubygems'\n" +
"require 'twitter'\n" +
"require 'pp'\n" +
"pp Twitter::Search.new('#jruby').fetch.results.first";
container.runScriptlet(script);
}

public static void main(String[] args) {
new SearchSample();
}
}

Linden -+- lib -+- jruby -+- 1.8 -+- bin -+- bundle
| | +- httparty
| | +- oauth
| +- cache -+- ...
| +- doc
| +- gems -+- bouncy-castle-java-1.5.0145.2 -+- ...
| | +- bundler-1.0.3 -+- ...
| | +- crack-0.1.8 -+- ...
| | +- hashie-0.4.0 -+- ...
| | +- httparty-0.6.1 -+- ...
| | +- jruby-openssl-0.7.1 -+- ...
| | +- multi_json-0.0.4 -+- ...
| | +- oauth-0.4.3 -+- ...
| | +- twitter-0.9.12 -+- ...
| +- specifications -+- ...
+- jruby-complete.jar
+- Gemfile
+- Gemfile.lock
+- src -+- linden -+- SearchSample.java
+- build



Compile and run using rake-ant integration



Everything was ready. OK, how do I compile and run it? Of course, classic "javac" and "java" command were the options. But, these commands are not very convenient to repeat compile/run. So, I used JRuby's rake-ant integration. Yes, I wrote "Rakefile" to compile and run Java code. Isn't it nice? Here's Rakefile:

require 'ant'

namespace :ant do
task :compile => :clean do
ant.javac :srcdir => "src", :destdir => "build"
end
end

namespace :ant do
task :java => :compile do
ant.java :classname => "linden.SearchSample" do
classpath do
pathelement :location => "lib/jruby-complete.jar"
pathelement :path => "build"
end
end
end
end

require 'rake/clean'

CLEAN.include '*.class', '*.jar'

I created Rakefile under the Linden directory, so now the directories/files were:

Linden -+- lib -+- jruby -+- 1.8 -+- bin -+- bundle
| | +- httparty
| | +- oauth
| +- cache -+- ...
| +- doc
| +- gems -+- bouncy-castle-java-1.5.0145.2 -+- ...
| | +- bundler-1.0.3 -+- ...
| | +- crack-0.1.8 -+- ...
| | +- hashie-0.4.0 -+- ...
| | +- httparty-0.6.1 -+- ...
| | +- jruby-openssl-0.7.1 -+- ...
| | +- multi_json-0.0.4 -+- ...
| | +- oauth-0.4.3 -+- ...
| | +- twitter-0.9.12 -+- ...
| +- specifications -+- ...
+- jruby-complete.jar
+- Gemfile
+- Gemfile.lock
+- src -+- linden -+- SearchSample.java
+- build
+- Rakefile


When I typed "rake ant:java", the Java code above worked and printed out one tweet.

java -jar lib/jruby-complete.jar -S rake ant:java
(in /Users/yoko/Works/tmp/Linden)
basepath: /Users/yoko/Works/tmp/Linden
jrubyhome: file:/Users/yoko/Works/tmp/Linden/lib/jruby-complete.jar!/META-INF/jruby.home
{"profile_image_url"=>
"http://a3.twimg.com/profile_images/76084835/web-profile_normal.jpg",
"created_at"=>"Fri, 22 Oct 2010 15:38:09 +0000",
"from_user"=>"mccrory",
"metadata"=>{"result_type"=>"recent"},
"to_user_id"=>nil,
"text"=>
"RT @carlosqt: #IronRuby 1.1.1 a released! download from: http://ironruby.codeplex.com/ #programming #JRuby #Ruby #Rails #dotnet #code",
"id"=>28415816774,
"from_user_id"=>1757577,
"geo"=>nil,
"iso_language_code"=>"en",
"source"=>"<a href="http://twitter.com/">web</a>"}


Packaging


This time I can't rely on GEM_PATH environment variable. If GEM_PATH had worked also for the path in a jar, unfortunately, it didn't. Instead of GEM_PATH, I set all path to gems to ScriptingContainer. This was not complicated since all gems were installed in the same directory. I fixed the jar name, "sample.jar," so this name appeared in the Java code. This name could have been given via a command line argument. Edited SearchSample.java is in below:

package linden;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import java.util.zip.ZipEntry;

import org.jruby.embed.LocalContextScope;
import org.jruby.embed.ScriptingContainer;

public class SearchSample {
private String jarname = "sample.jar";

private SearchSample() throws IOException {
String basepath = System.getProperty("user.dir");
System.out.println("basepath: " + basepath);
ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
System.out.println("jrubyhome: " + container.getHomeDirectory());
container.setLoadPaths(getGemPaths(jarname, basepath));

String script =
"require 'rubygems'\n" +
"require 'twitter'\n" +
"require 'pp'\n" +
"pp Twitter::Search.new('#jruby').fetch.results.first";
container.runScriptlet(script);
}

private List<String> getGemPaths(String jarname, String basepath) throws IOException {
JarFile jarFile = new JarFile(basepath + "/" + jarname);
Enumeration<JarEntry> entries = jarFile.entries();
String gempath = "lib/jruby/1.8/gems/";
Set<String> gemnames = new HashSet<String>();
while (entries.hasMoreElements()) {
ZipEntry entry = (ZipEntry) entries.nextElement();
String entryName = entry.getName();
if (entryName.startsWith(gempath) && entryName.length() > gempath.length()) {
String n = entryName.substring(gempath.length());
String m = n.substring(0, n.indexOf("/"));
gemnames.add(m);
}
}
List<String> gemPaths = new ArrayList<String>();
for (String gem : gemnames) {
gemPaths.add("file:" + basepath + "/" + jarname + "!/lib/jruby/1.8/gems/" + gem + "/lib");
}
return gemPaths;
}


public static void main(String[] args) throws IOException {
new SearchSample();
}
}

To make jar archive, I added jar task to my Rakefile.

require 'ant'

namespace :ant do
task :compile => :clean do
ant.javac :srcdir => "src", :destdir => "build"
end
end

namespace :ant do
task :java => :compile do
ant.java :classname => "linden.SearchSample" do
classpath do
pathelement :location => "lib/jruby-complete.jar"
pathelement :path => "build"
end
end
end
end

namespace :ant do
task :jar => :compile do
ant.jar :basedir => ".", :destfile => "sample.jar" do
fileset :dir => "build" do
include :name => "**/*.class"
end
include :name => "lib/jruby/1.8/gems/**/*"
manifest do
attribute :name => "Main-Class", :value => "linden.SearchSample"
end
end
end
end


require 'rake/clean'

CLEAN.include '*.class', '*.jar'

All right, let's create jar archive.

java -jar lib/jruby-complete.jar -S rake ant:jar

The ant:jar task created the sample.jar archive in Linden directory. To test this archive was really gems in a jar, I unset GEM_PATH environment variable first. Then, I typed java command and got one tweet, Yay!

unset GEM_PATH; echo $GEM_PATH
java -cp lib/jruby-complete.jar:sample.jar linden.SearchSample

To ensure that the jar archive had gems in the jar, I moved sample.jar to a different directory and typed java command with the full path to jruby-complete.jar. It worked.

cp sample.jar ../.
cd ..
java -cp Linden/lib/jruby-complete.jar:sample.jar linden.SearchSample



This might be one answer of packaging gems in a jar and using gems from RedBridge.

Monday, October 11, 2010

pure Java Nokogiri - XSLT extension function -

Here's a memo of pure Java Nokogiri about an implementation of XSLT extension function. In short, I concluded pure Java version is unable to support Nokogiri style XSLT extension function. I've tried possible ways to make it happen, but for an inevitable reason, I settled to this conclusion. However, in future, this part might be reconsidered when XML libraries and APIs are replaced to others. For the future version of pure Java Nokogiri, I'm going to write down what I did and what was the problem. Hopefully, this memo will help to retry the implementation later.


1. What is XSLT extension function?

XSLT extension is defined in "14 Extensions" of XSL Transformations (XSLT)
Version 1.0 (http://www.w3.org/TR/xslt), which allows users to delegate an XSLT processing to a specified function/method written in a programming languages such as Ruby, Java, JavaScript. As in Nokogiri's test case,

<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:f="http://e.org/functions"
extension-element-prefixes="f">

<xsl:template match="text()">
<xsl:copy-of select="f:capitalize(.)"/>
</xsl:template>
....

the element, "extension-element-prefixes," indicates the function is tied to this namespace. This XSL file should be common to all languages used to write the function.


2. Nokogiri style function mapping

Nokogiri maps the namesapce to the function as in below:

foo = Class.new do
def capitalize nodes
nodes.first.content.upcase
end
end

XSLT.register "http://e.org/functions", foo

Thus, a receiver object is registered to XSLT processor with the URL tag. It is nice Ruby friendly design. <xsl:copy-of select="f:capitalize(.)"/> above executes a "capitalize" method of the "foo" object.


3. How Java handles this?

As far as I googled about a Java way, not many documents, blogs, articles were out there. Probably, Extending XSLT with Java - Chapter 17. XSLT would be the best described one. This explains how to delegate the process to a Java method tied to the namespace. Xalan has the document, Xalan-Java Extensions; however, this uses BSF (Bean Scripting Framework: http://jakarta.apache.org/bsf/) to execute a function/method written in an XSL file. So, the first one has a possibility to realize Nokogiri style.


While I tried a couple of patterns, the "extension-element-prefixes" element seemed not to have a much meaning. Instead, "xmlns:java="http://xml.apache.org/xslt/java" and "xmlns:foo="xalan://[fully qualified class name]" worked. OK, so pure Java version of Nokogiri needs a specific rule to use XSLT extension function. This might be better than unsupported. Then, I wrote a Java class below to see whether it worked or not:

package Canna;

public class ExtensionFoo {
public static Object exec(String method, Object value) {
.....
}
}

The method should be static to be called from XSLT processor. The first argument is a method name to make the style resemble to the Nokogiri way. Since I thought

foo = Class.new do
def capitalize nodes
nodes.first.content.upcase
end
end

XSLT.register "http://e.org/functions", foo

xsl = Nokogiri.XSLT(<<-EOXSL)
<?xml version="1.0"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:foo="xalan://nokogiri.internals.XsltExtensionFunction"
extension-element-prefixes="foo">

<xsl:template match="text()">
<xsl:copy-of select="foo:exec('capitalize' .)"/>
</xsl:template>
...

would not be a bad substitution. Users need to have just a small rule only in the xsl file.


Here are entire files that I tried XSLT extension function in action by Java.

[extension.xsl]
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:foo="xalan://Canna.ExtensionFoo"
extension-element-prefixes="foo"
version="1.0">
<xsl:template match="text()">
<xsl:copy-of select="foo:exec('capitalize', .)"/>
</xsl:template>
</xsl:stylesheet>

[extension.xml]
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1 plus MathML 2.0//EN"
"http://www.w3.org/TR/MathML2/dtd/xhtml-math11-f.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta http-equiv="Content-type" content="application/xhtml+xml"/>
<title>Foo</title>
</head>
<body>
<h1>Foo</h1>
<p>Lorem ipsum.</p>
</body>
</html>

[ExtensionFoo.java]
package Canna;

public class ExtensionFoo {
public static Object exec(String method, Object value) {
if (value != null && (value instanceof String)) {
return ((String)value).toUpperCase();
} else {
return "hello?";
}
}
}

[TransformSample.java]
package Canna;

import java.io.File;
import javax.xml.transform.Result;
import javax.xml.transform.Source;
import javax.xml.transform.Templates;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerConfigurationException;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;

public class TransformSample {
private static String userdir = System.getProperty("user.dir");
private static String templateName = "extension.xsl";
private static String documentName = "extension.xml";

private TransformSample() throws TransformerConfigurationException, TransformerException {
Source templateSource = new StreamSource(new File(userdir + "/ext/java/Canna/" + templateName));
Source documentSource = new StreamSource(new File(userdir + "/ext/java/Canna/" + documentName));
Result result = new StreamResult(System.out);
TransformerFactory factory = TransformerFactory.newInstance();
Templates templates = factory.newTemplates(templateSource);
Transformer transformer = templates.newTransformer();
transformer.transform(documentSource, result);
}

public static void main(String[] args) throws TransformerConfigurationException, TransformerException {
new TransformSample();
}
}

What was the result? The program output a bunch of "hello?"s. Why? The given object of a method argument wasn't the String type but DTMNodeProxy. What's DTMNodeProxy? This is com.sun.org.apache.xml.internal.dtm.ref.DTMNodeProxy, org.apache.xml.internal.dtm.ref.DTMNodeProxy, or another XSLT processor's internal type. The most affordable choice would be org.apache.xml.internal.dtm.ref.DTMNodeProxy, but Nokogiri needs to add xalan.jar to its jar list. Definitely, Nokogiri will be fat. Otherwise, pure Java Nokogiri will lose portability. Thankfully, there is an option. Users can convert values from XSL to Java and hand a desired type in to the method. So, I changed one line in XSL file:

<xsl:copy-of select="foo:exec('capitalize', string(.))"/>

OK, this worked. Every text became an upper case.


4. Inevitable API conflict

Although there were pure Java specific rules, Nokogiri style XSLT extension function seemed to work. However, the result was NOT. Puzzled. I moved the sample code above under Nokogir source tree then figured out the culprit.

When xercesImpl.jar or jing.jar is on a classpath, the sample code failed to parse the XSL file.

Sigh... I haven't found what's wrong with that yet, but the conflict lies there. Pure Java Nokogiri uses an internal API of Xerces for SAX and Jing for Relaxng processing. Both xercesImpl.jar and jing.jar are necessary APIs.

Probably, the best choice would be not to support XSLT extension of Nokogiri style right now. In future, pure Java Nokogiri might choose other XML APIs. Or, someone might give me a good advice to avoid the conflict. So, still, there is a possibility to make it happen later. At the time, this memo hopefully helps to restart implementing the XSLT extension feature.

Monday, September 20, 2010

New featues of embedding API for JRuby 1.6

This month, a lot of work for JRuby's embedding API (RedBridge) has been done. Mainly, bug fixing. While I was fixing bugs, I eventually landed to add a new feature and change the area of sharing variables. These will be in JRuby 1.6. Currently, snapshot is available at http://ci.jruby.org/snapshots/ if you want to test it. I believe the changes are good to use Ruby more naturally, but those might affect the code already written a little bit. That why I'm writing this. If you are the user of JRuby's embedding API or JSR223, be aware of upcoming changes.


1. Sharing variables becomes a receiver sensitive

Before the changes, embedding API didn't mind the difference of receivers. A receiver means Ruby's receiver, which is returned as a result of evaluation. Variables and constants to be shared are injected to a top level, in other words, runtime's top self. Also, those should be retrieved from the top level variables and constants. However, this logic didn't work perfectly.

Firstly, a trouble happened in sharing instance variables. The reason is that embedding API didn't use consistent receivers to inject and retrieve instance variables. When multiple receivers were involved, multiple values were assigned to the same key. This ended up unwanted results occasionally.

In light of this, I added three methods to ScriptingContainer.

get(receiver, key)
put(receiver, key, value)
remove(receiver, key, value)

These methods explicitly interact with a given receiver. Existing get/put/remove methods will have top self receivers for the argument.

Let me show you example. I'm going to use the Ruby code, tree_sample.rb below:

class Tree
attr_accessor :name, :shape, :type

def initialize name, shape, type
@name = name
@shape = shape
@type = type
end

def name= name
@name = name
end

def to_s
"#{name.capitalize} is a(n) #{shape} shaped, #{type} tree."
end
end

When the code is evaluated by:

ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
container.runScriptlet(PathType.CLASSPATH, "ruby/tree_sample.rb");

The runtime caches the Tree class and returns nil, which is converted to null for Java code. Then, suppose two objects are instantiated:

Object tree1 = container.runScriptlet("Tree.new('any', 'pyramidal', 'evergreen')");
Object tree2 = container.runScriptlet("Tree.new('any', 'oval', 'deciduous')");

Tree1 and tree2 above are receivers. Before the changes, container retrieved instance variables from the receivers and saved in an internal variable table tied to the instance name at the end of runScriptlet method. As you know, there're two objects. Two values of each instance variable were assigned to the single key without any receiver info. The retrieval should be receiver sensitive as well as injecting.

After the change, the instance variable values are tied to both the key and receiver. In JRuby 1.6, you'll get expected results even though multiple receivers of the same class are there. For example, suppose callMethod methods are run for each receiver object:

container.callMethod(tree1, "name=", "pine");
container.callMethod(tree2, "name=", "poplar");
System.out.println(container.callMethod(tree1, "to_s", String.class));
System.out.println(container.callMethod(tree2, "to_s", String.class));

The result is the one expected:

Pine is a(n) pyramidal shaped, evergreen tree.
Poplar is a(n) oval shaped, deciduous tree.

For clarity, let me add more lines of sharing variable related method usages:

container.put(tree1, "@name", "camellia");
container.put(tree2, "@name", "cherry");
container.put(tree1, "@shape", "oval");
container.put(tree2, "@shape", "round");

System.out.println(container.callMethod(tree1, "to_s", String.class));
System.out.println(container.callMethod(tree2, "to_s", String.class));

System.out.println("@type of tree1: " + container.get(tree1, "@type"));
System.out.println("@type of tree2: " + container.get(tree2, "@type"));

Above prints:

Camellia is a(n) oval shaped, evergreen tree.
Cherry is a(n) round shaped, deciduous tree.
@type of tree1: evergreen
@type of tree2: deciduous


So far, I talked about instance variables. Constants are also receiver sensitive in JRuby 1.6. Look at the code below:

ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
String script =
"COLOR = 'pink'\n" +
"class ColorSample\n" +
" COLOR = 'orange'\n" +
"end\n" +
"ColorSample.new";
Object receiver = container.runScriptlet(script);
System.out.println("top level: " + container.get("COLOR"));
System.out.println("class: " + container.get(receiver, "COLOR"));

There're two constants which have the same name, COLOR. While container.get("COLOR")) gets a top level contant, container.get(receiver, "COLOR")) gets a constant from a given receiver. So, the output is:

top level: pink
class: orange


How about global and local variables? Global variables are receiver insensitive because those should be referenced globally. Local variables are always injected to/retrieved from the top level. It might have been possible to tie to the receiver, but that is not a good idea because Ruby code might use gems or third party libraries. The injected local variables from Java might be extraterrestrials for those libraries and cause unexpected results.

For JSR223 user, I added an "org.jruby.embed.receiver" attribute, but this needs more work to make it happen.


2. Variable/constant value retrieval becomes lazy by default


Before the change ScriptingContainer tried to retrieve variable/constant values as much as possible at the end of runScriptlet/callMethod and EvalUnit.run methods. This was convenient to get variables/constants defined in Ruby ready for Java. However, too many values were saved in the internal variable table. Those variables were injected to Ruby code in succeeding evaluations implicitly. Obviously, this behavior made performance and memory usage worse. When multiple gems were used, it would be serious.

In JRuby 1.6, all variables/constants are retrieved lazily except persistent local variables. This means the internal variable table will have minimum key-value pairs. When a get method of ScriptingContainer is called, the requested key-value pair is saved in the variable table. When key-value pairs have been put to the internal variable table before the evaluation, those will be updated right after the evaluation. Others are not. However, persistent local variables are exception. The values are retrieved eagerly as it was done before. This is because of the internal of JRuby and is helped by the policy that local variables to be shared are only top level ones. Probably, I can consider top level local variables are few.

The change will bring a good result; however, JSR223 users might be affected. The existence of JSR223's SimpleBindings and SimpleScriptContext are always headache to me. It's very hard to add a trick to those two. Workaround is to set false in lazy option and to use the method, ScriptEngine#getContext().getAttrubute() method. Otherwise, set a key and dummy value pair to the bindings. Then, JRubyEngine updates the value right after the evaluation. By the release of 1.6, I'll improve JSR223 support.


By this change, users program will be more natural in terms of Ruby coding even on JRuby's embedding API. My work on these has not yet finished. I'll add more tests and some measures to make these on JSR223.

Saturday, September 04, 2010

My Presentation at JRubyKaigi 2010

I spoke at JRubyKaigi 2010 about JRuby embed API (RedBridge). Since attendees were almost all Japanese, I spoke in Japanese. However, my slide was written using simple English words, so that even non-Japanese speanker could understand what's going on. Don't worry about the Language of my slide.

Now, you can see my slide at http://servletgarden-point.appspot.com/slideshow. As the url shows, my slide itself was a demo built on Google App Engine using JRuby on Rails. This app uses jQuery UI and ajax. Click on buttons on the left side, then right side contents will change, and you'll see accordion. Also, you can try examples I showed in my presentation. Cloning the git repo,

git://github.com/yokolet/jrubykaigi_examples.git

and look at README to run examples.

Thursday, July 15, 2010

Cucumber on RedBridge

With JRuby's RedBridge, Ruby applications get started from Java. For example Cucumber, does. Let's see how Cucumber runs in Java code.

Firstly, people use Cucumber like this:

jruby -S cucumber addition.feature

"cucumber" is installed in $JRUBY_HOME/bin directory and looks like a command. But "cucumber" is a Ruby script, so "cucumber" can be evaluated using JRuby Embed, RedBridge, API. How about "addition.feature"? Cucumber receives the feature name via ARGV constant. This means adding a file name to ARGV will work. From these analysis, I wrote CucumberRunner.

package evergreen;

import org.jruby.embed.LocalContextScope;
import org.jruby.embed.PathType;
import org.jruby.embed.ScriptingContainer;

public class CucumberRunner {
private String jrubyhome = "/Users/yoko/Projects/jruby";
private String cucumber = jrubyhome + "/bin/cucumber";
private String feature = jrubyhome + "/lib/ruby/gems/1.8/gems/cucumber-0.8.5/examples/i18n/en/features/addition.feature";

private CucumberRunner() {
ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
container.setHomeDirectory(jrubyhome);
container.runScriptlet("ARGV << '" + feature + "'");
container.runScriptlet(PathType.ABSOLUTE, cucumber);
}

public static void main(String[] args) {
new CucumberRunner();
}
}

This printed out:

# language: en
Feature: Addition
In order to avoid silly mistakes
As a math idiot
I want to be told the sum of two numbers

Scenario Outline: Add two numbers # /Users/yoko/Projects/jruby/lib/ruby/gems/1.8/gems/cucumber-0.8.5/examples/i18n/en/features/addition.feature:7
Given I have entered <input_1> into the calculator # cucumber-0.8.5/examples/i18n/en/features/step_definitons/calculator_steps.rb:14
And I have entered <input_2> into the calculator # cucumber-0.8.5/examples/i18n/en/features/step_definitons/calculator_steps.rb:14
When I press <button> # cucumber-0.8.5/examples/i18n/en/features/step_definitons/calculator_steps.rb:18
Then the result should be <output> on the screen # cucumber-0.8.5/examples/i18n/en/features/step_definitons/calculator_steps.rb:22

Examples:
| input_1 | input_2 | button | output |
| 20 | 30 | add | 50 |
| 2 | 5 | add | 7 |
| 0 | 40 | add | 40 |

3 scenarios (3 passed)
12 steps (12 passed)
0m0.179s
/Users/yoko/Projects/jruby/lib/ruby/gems/1.8/gems/cucumber-0.8.5/bin/cucumber:19:in `load': exit (SystemExit)
from /Users/yoko/Projects/jruby/bin/cucumber:19
Exception in thread "main" org.jruby.embed.EvalFailedException: exit
at org.jruby.embed.internal.EmbedEvalUnitImpl.run(EmbedEvalUnitImpl.java:127)
at org.jruby.embed.ScriptingContainer.runUnit(ScriptingContainer.java:1149)
at org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1194)
at evergreen.CucumberRunner.(CucumberRunner.java:16)
at evergreen.CucumberRunner.main(CucumberRunner.java:20)
Caused by: org.jruby.exceptions.RaiseException: exit
at (unknown).(unknown)(/Users/yoko/Projects/jruby/lib/ruby/gems/1.8/gems/cucumber-0.8.5/bin/cucumber:19)
at Kernel.load(/Users/yoko/Projects/jruby/bin/cucumber:19)
at (unknown).(unknown)(:1)

OK. Cucumber seems to execute SystemExit in the end. Java code isn't happy with this sort of exception since evaluation itself has been succeeded. So, I'm going to catch org.jruby.embed.EmbedEvalFailedException and error message raised in Ruby.

package evergreen;

import java.io.StringWriter;

import org.jruby.embed.EvalFailedException;
import org.jruby.embed.LocalContextScope;
import org.jruby.embed.PathType;
import org.jruby.embed.ScriptingContainer;

public class CucumberRunner1 {
private String jrubyhome = "/Users/yoko/Projects/jruby";
private String cucumber = jrubyhome + "/bin/cucumber";
private String feature = jrubyhome + "/lib/ruby/gems/1.8/gems/cucumber-0.8.5/examples/i18n/en/features/addition.feature";

private CucumberRunner1() {
StringWriter errorWriter = new StringWriter();
try {
ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
container.setError(errorWriter);
container.setHomeDirectory(jrubyhome);
container.runScriptlet("ARGV << '" + feature + "'");
container.runScriptlet(PathType.ABSOLUTE, cucumber);
} catch (EvalFailedException e) {
System.out.println("Cuke says: " + e.getMessage());
} finally {
System.out.println("Cuke also says: " + errorWriter.toString());
}
}

public static void main(String[] args) {
new CucumberRunner1();
}
}

All right, CuumberRunner1 printed out:

# language: en
Feature: Addition
In order to avoid silly mistakes
As a math idiot
I want to be told the sum of two numbers

Scenario Outline: Add two numbers # /Users/yoko/Projects/jruby/lib/ruby/gems/1.8/gems/cucumber-0.8.5/examples/i18n/en/features/addition.feature:7
Given I have entered <input_1> into the calculator # cucumber-0.8.5/examples/i18n/en/features/step_definitons/calculator_steps.rb:14
And I have entered <input_2> into the calculator # cucumber-0.8.5/examples/i18n/en/features/step_definitons/calculator_steps.rb:14
When I press <button> # cucumber-0.8.5/examples/i18n/en/features/step_definitons/calculator_steps.rb:18
Then the result should be <output> on the screen # cucumber-0.8.5/examples/i18n/en/features/step_definitons/calculator_steps.rb:22

Examples:
| input_1 | input_2 | button | output |
| 20 | 30 | add | 50 |
| 2 | 5 | add | 7 |
| 0 | 40 | add | 40 |

3 scenarios (3 passed)
12 steps (12 passed)
0m0.238s
Cuke says: exit
Cuke also says: /Users/yoko/Projects/jruby/lib/ruby/gems/1.8/gems/cucumber-0.8.5/bin/cucumber:19:in `load': exit (SystemExit)
from /Users/yoko/Projects/jruby/bin/cucumber:19

Now, everything is controllable from Java.

Then, what if Cucumber result is showed up in Swing window? Isn't it interesting? So, I set StringWriter to ScriptingContainer so that all standard output from Ruby will be caught in StringWriter. Then, I added small Swing code.

package evergreen;

import java.io.StringWriter;

import javax.swing.JFrame;
import javax.swing.JTextArea;

import org.jruby.embed.EvalFailedException;
import org.jruby.embed.LocalContextScope;
import org.jruby.embed.PathType;
import org.jruby.embed.ScriptingContainer;

public class CucumberRunner2 {
private String jrubyhome = "/Users/yoko/Projects/jruby";
private String cucumber = jrubyhome + "/bin/cucumber";
private String feature = jrubyhome + "/lib/ruby/gems/1.8/gems/cucumber-0.8.5/examples/i18n/en/features/addition.feature";

private CucumberRunner2() {
StringWriter writer = new StringWriter();
StringWriter errorWriter = new StringWriter();
try {
ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
container.setError(errorWriter);
container.setHomeDirectory(jrubyhome);
container.setWriter(writer);
container.runScriptlet("ARGV << '" + feature + "'");
container.runScriptlet(PathType.ABSOLUTE, cucumber);
} catch (EvalFailedException e) {
System.out.println("Cuke says: " + e.getMessage());
} finally {
System.out.println("Cuke also says: " + errorWriter.toString());
writeOnWindow(writer.toString());
}
}

private void writeOnWindow(String message) {
JFrame frame = new JFrame();
JTextArea text = new JTextArea(message);
frame.add(text);
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.pack();
frame.setVisible(true);
}

public static void main(String[] args) {
new CucumberRunner2();
}
}

When I ran above code, a simple Swing window showed up with the result of Cucumber test.

Cucumber is Swing friendly with RedBridge. NetBeans plugin for Cucumber might be possible. Or, other Java based plugins for Cucumber might be easily implemented with RedBrdige.

Monday, July 12, 2010

Clojure uses DataMapper

Clojure is one of the JVM languages. People know this fact well. Also, people know well JRuby is among JVM languages. On the JVM languages, many people have used Java APIs from Clojure or Ruby code. But, we can do more since the JVM languages are able to communicate each other. The communication is done over the API exposed to Java such as JSR223. This means libraries and tools for a particular language are available to use in other JVM languages. For example, Clojure can choose DataMapper gem to interact databases. Usage is not so complicated. How could I make it happen? Here's a small example.


1. Installation

Clojure and JRuby themselves don't need to be installed. Grab the archives and unzip them. JRuby has installers, so you can use those if you like. After setting the path to jruby command, install 3 DataMapper gems. In this example, I used Sqlite3 for my DBMS. This is why I installed the DataMapper adapter for Sqlite3.

jruby -S gem install --no-ri --no-rdoc dm-core dm-sqlite-adapter dm-migrations



2. Run Clojure with the classpath to jruby.jar

My JRuby resides in /Users/yoko/Tools/jruby-1.5.1, so the path to jruby.jar is /Users/yoko/Tools/jruby-1.5.1/lib/jruby.jar. I added one more path. It is a directory for Ruby code for DataMapper, and the path name is /Users/yoko/Works/Samples/datamapper.
Move to the directory where Clojure was expanded, and run Clojure as in below:

java -cp clojure.jar:/Users/yoko/Tools/jruby-1.5.1/lib/jruby.jar:/Users/yoko/Works/Samples/datamapper clojure.main



3. Setup ScriptingContainer

This example uses JRuby's embed core, RedBridge, since it is easier to use compared to JSR223. The first step is to instantiate ScriptingContainer and set it up so that gems can be loaded.

user=> (import '(org.jruby.embed ScriptingContainer PathType))
org.jruby.embed.PathType
user=> (def c (ScriptingContainer.))
#'user/c
user=> (. c setHomeDirectory "/Users/yoko/Tools/jruby-1.5.1")
nil

To verify the setting was correct, I printed out Ruby's $LOAD_PATH.

user=> (. c runScriptlet "p $LOAD_PATH")
["clojure.jar", "/Users/yoko/Tools/jruby-1.5.1/lib/jruby.jar", "/Users/yoko/Works/Samples/datamapper",
"/Users/yoko/Tools/jruby-1.5.1/lib/ruby/site_ruby/1.8",
"/Users/yoko/Tools/jruby-1.5.1/lib/ruby/site_ruby/shared", "/Users/yoko/Tools/jruby-1.5.1/lib/ruby/1.8", "."]
nil

The path seems OK. Let's go forward.


4. Load gems and connect to the database

The second step is to load gems.

user=> (. c runScriptlet "require 'rubygems'; require 'dm-core'; require 'dm-migrations'")
true


Then, setup DataMapper.

user=> (. c runScriptlet "DataMapper.setup(:default, 'sqlite::memory:')")
#<RubyObject #<DataMapper::Adapters::SqliteAdapter:0xa75865>>


Writing Ruby code in method argument is not very nice. Instead, I wrote Ruby code in the *.rb files and evaluated each file loaded from classpath, /Users/yoko/Works/Samples/datamapper.

user=> (. c runScriptlet PathType/CLASSPATH "category_def.rb")
#<RubyObject #<DataMapper::Model::DescendantSet:0x3b9617>>

See category_def.rb below:

# Definition of the Category model
class Category
include DataMapper::Resource

property :id, Serial
property :category, String
property :created_at, DateTime
end

# Migration
DataMapper.auto_migrate!

Since the Category table has been created, I added three new records:

user=> (. c runScriptlet PathType/CLASSPATH "categories.rb")
true

See categories.rb below:

# Create new records
c1 = Category.new
c1.category = 'Kitchen & Food'
c1.save

c2 = Category.new
c2.category = 'Bed & Bath'
c2.save

c3 = Category.new
c3.category = 'Dining'
c3.save

Let's see what were input to Sqlite3.

user=> (. c runScriptlet "p Category.all")
[#<Category @id=1 @category="Kitchen & Food" @created_at=nil>, #<Category
@id=2 @category="Bed & Bath" @created_at=nil>, #<Category @id=3 @category="Dining" @created_at=nil>]
nil



As in this example, DataMapper worked with Clojure! MySQL and PostgreSQL are available to use from Clojure via DataMapper. Not just DataMapper. Other Ruby gems are also friends of Clojure.

Tuesday, May 04, 2010

A Small Step to Rails on RedBridge

When RedBridge (JRuby Embed) was released included in JRuby 1.4.0RC1 for the first time ever, I wrote“'Rails on Red Bridge' will be my exciting challenge" in my blog, What's the embedding API of JRuby 1.4.0RC1?. Since then, RedBridge had many improvements and bug fixes, so I tackled the issue this week. As a result, it went good. I could successfully made a small step to Rials on RedBridge. However, the process was a bit tricky, so I'm going to write down how I could make it, step by step.

The most difficult part was gem bundling. Since Rails on RedBridge is built on Java web application, all gems should be in a Java based web application structure. Besides, the gems should be looked up by a Rails app on a servlet. After I tried some, I concluded Rails 3 was the best choice because of its built-in feature of gem bundler. So, I created a Java web application project with Apache Tomcat on Eclipse and Rails 3 app within.

First, I built jruby-complete.jar from JRuby 1.5.0 source archive. At this moment, JRuby 1.5.0.RC3 is the latest.

$ tar zxfv /Users/yoko/Downloads/jruby-src-1.5.0.RC3.tar.gz
$ cd jruby-1.5.0.RC3
$ ant jar-complete
$ export JRUBY_HOME=`pwd`
$ PATH=$JRUBY_HOME/bin:$PATH

Then, I installed rails 3 and jdbc adapter gems. In my case, I used sqlite3, so I typed a gem name for jdbc sqlite3 adapter.

$ jruby -S gem install rails --pre --no-rdoc --no-ri
$ jruby -S gem install activerecord-jdbcsqlite3-adapter --no-rdoc --no-ri


Next, I created a Java web application project, Hemlock, on Eclipse. Once the project was built, I had the tree structure below:

Hemlock -+- WebContent -+- META-INF --- MANIFEST.MF
| +- WEB-INF -+- lib
| +- web.xml
+- build -+- classes
+- src

Then, I created a Rails app under WebContent/WEB-INF/lib. This is because a web application adds the path, WEB-INF/lib, into the classpath automatically. WebContent/WEB-INF/classes would have been among the choices for the same reason.

My Rails app is based on a classic tutorial, "Four Days on Rails." This original document got outdated, but Japanese version was updated up to Rails 2.3.2. You can download the PDF file from Ita-san's blog: Four Days on Rails 2.0. Online translation service might help you to read the PDF.

$ cd [path-to-workspace]/Hemlock/WebContent/WEB-INF/lib
$ jruby -S rails todo
$ cd todo

Then, I edited Gemfile and config/database.yml to replace adapter gem and setting from sqlite3 to jdbcsqlite3.

Gemfile

#gem 'sqlite3-ruby', :require => 'sqlite3'
gem 'activerecord-jdbcsqlite3-adapter'

config/database.yml

#adapter: sqlite3
adapter: jdbcsqlite3

Before proceeding further, I did scaffolding to see database connection was set up right.

$ jruby -S rails g scaffold Category category:string
$ jruby -S rake db:migrate
$ jruby -S rails server

Requesting http://localhost:3000/categories on my browser, I added three categories.


I did one more setup on Rails --- Rails metal. Rails metal was originally invented to improve perfomance. But, metal exposes Rack's bare interface, so metal makes it easy to handle Rails app in the servlet.

$ jruby -S rails g metal poller


OK, let's setup gems for a servlet.

$ jruby -S bundle install vendor --disable-shared-gems

This command installed all necessary gems in WebContent/WEB-INF/lib/todo/vendor/gems directory.

Then, I moved on to the web application setting and the servlet. I copied jruby-compelete.jar to WEB-INF/lib, first. Next, after refreshing the Hemlock Eclipse project, I added jruby-complete.jar by Properties -> Java Build Path -> Add External JARS -> [on a JAR Selection window, I chose Hemlock/WebContent/WEB-INF/lib/jruby-complete.jar] -> OK. My servlet became as in below:

package com.servletgarden.hemlock;

import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.jruby.embed.LocalContextScope;
import org.jruby.embed.ScriptingContainer;

public class SimpleMetalServlet extends HttpServlet {
private static final long serialVersionUID = 1L;
private String basepath;
private ScriptingContainer container;
private List loadPaths;

public SimpleMetalServlet() {
super();
}

@Override
public void init() {
basepath = getServletContext().getRealPath("/WEB-INF");
String[] paths = {
basepath + "/lib/todo/vendor/gems/bundler-0.9.25/lib/",
basepath + "/lib/todo"
};
loadPaths = Arrays.asList(paths);
container = new ScriptingContainer(LocalContextScope.THREADSAFE);
}

@Override
protected void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
container.setLoadPaths(loadPaths);
container.runScriptlet("ENV['BUNDLE_GEMFILE'] = \"" + basepath + "/lib/todo/Gemfile\"");
container.runScriptlet("require 'config/environment'");
Map env = getEnvMap(request);
container.put("env", env);
List result = (List) container.runScriptlet("Poller.call env");
returnResult(response, result);
}

private Map getEnvMap(HttpServletRequest request) {
Map map = new HashMap();
map.put("PATH_INFO", "/poller");
return map;
}

private void returnResult(HttpServletResponse response, List result) throws IOException {
response.setStatus(((Long)result.get(0)).intValue());
Map headers = (Map)result.get(1);
Set keys = headers.keySet();
for (String key : keys) {
if ("Content-Type".equals(key)) {
response.setContentType((String)headers.get(key));
}
}
List contents = (List)result.get(2);
for (String content : contents) {
response.getWriter().write(content);
}
}
}

When I right-clicked on the servlet source and chose Run As -> Run on Server, my Eclipse showed that simple "Hello, World!"


OK, let's look at what I did in SimpleMetalServlet. The first one is a load path setting.

Hemlock tree

Hemlock -+- WebContent -+- META-INF --- MANIFEST.MF
| +- WEB-INF -+- lib -+- todo -+- Gemfile
| | | +- app -+- controllers --- ...
| | | | +- metal --- poller.rb
| | | | +- ...
| | | +- config -+- environment.rb
| | | +- vendor -+- gems -+- bundler-0.9.25 -+- lib
| | | -+- ...
| | | +- ...
| | +- jruby-complete.jar
| +- web.xml
+- build -+- classes
+- src -+- com -+- servletgarden -+- hemlock -+- SimpleMetalServlet.java

in init() method:

basepath = getServletContext().getRealPath("/WEB-INF");
String[] paths = {
basepath + "/lib/todo/vendor/gems/bundler-0.9.25/lib/",
basepath + "/lib/todo"
};
loadPaths = Arrays.asList(paths);

in service() method:

container.setLoadPaths(loadPaths);

The path to bundler is to use bundler gem to load bundled gems. Bundler gem is installed in the todo/vendor/gems directory, but before loading bundled gem, SimpleMetalServlet can't load bundler itself. So, the servlet is telling JRuby runtime where the bundler gem is. The path to todo, Rails app top directory, is to tell JRuby where Rails app top directory is located.

The second one is the path to Gemfile.

container.runScriptlet("ENV['BUNDLE_GEMFILE'] = \"" + basepath + "/lib/todo/Gemfile\"");

Gemfile matters to Rails. To make Rails up and running, SimpleMetalServlet needs to give info about Gemfile. In this case, SimpleMetalServlet set the real path to Gemfile tied to the key "BUNDLE_GEMFILE" in ENV hash.

Then,

container.runScriptlet("require 'config/environment'");

Todo Rails app should get started by this.

Everything should be setup now, so SimpleMetalServlet is able to kick the metal method. Before talking about how the servlet kicked it, let's see what code Rails tailored.

poller.rb

# Allow the metal piece to run in isolation
require File.expand_path('../../../config/environment', __FILE__) unless defined?(Rails)

class Poller
def self.call(env)
if env["PATH_INFO"] =~ /^\/poller/
[200, {"Content-Type" => "text/html"}, ["Hello, World!"]]
else
[404, {"Content-Type" => "text/html", "X-Cascade" => "pass"}, ["Not Found"]]
end
end
end

Poller class has a class method, call, which needs an argument, env. The value, env, is a hash, in which PATH_INFO key and the value tied to it is expected. So, SimpleMetalServlet creates java.util.Map type object, puts to ScriptingContainer and evaluates Poller.call method with the argument, env.

Map env = getEnvMap(request);
container.put("env", env);
List result = (List) container.runScriptlet("Poller.call env");

private Map getEnvMap(HttpServletRequest request) {
Map map = new HashMap();
map.put("PATH_INFO", "/poller");
return map;
}


At last, response handling comes in.

returnResult(response, result);

private void returnResult(HttpServletResponse response, List result) throws IOException {
response.setStatus(((Long)result.get(0)).intValue());
Map headers = (Map)result.get(1);
Set keys = headers.keySet();
for (String key : keys) {
if ("Content-Type".equals(key)) {
response.setContentType((String)headers.get(key));
}
}
List contents = (List)result.get(2);
for (String content : contents) {
response.getWriter().write(content);
}
}

Rack based application returns the response as an array of a status code, hash of http response headers, and array of body contents. SimpleMetalServlet parses the response and sends back to a browser. Since Poller class returns so simple reponse, SimpleMetalServlet doesn't do much in this case.

I tweaked poller.rb a bit so that registered categories are showed up.

poller.rb

#[200, {"Content-Type" => "text/html"}, ["Hello, World!"]]
[200, {"Content-Type" => "application/xhtml+xml"}, [Category.all.to_xml]]

Refreshing Hemlock project on Eclipse, restarting Tomcat also on Eclipse, then requesting http://localhost:8080/Hemlock/SimpleMetalServlet gave me the XML below.


It seems Rails on RedBridge worked.