Sunday, February 27, 2011

Rails on RedBridge, Scaffolded App to Work

In my blog post, "The second step to Rails on RedBridge", I used Rails simple controller and showed how it could get run on Servlet. That was easy since there was no interaction with a Web browser. That controller just returned a result to the browser. That's it. So, Rails didn't need much information to work. However, a controller-only application isn't common. People start using Rails from scaffolding. Every Rails book talks about scaffolding first. Thus, I tried to make scaffolded Rails app to work on the Servlet. So far so good. Though it was a newbie app, scaffolding has been covered by Rails on RedBridge. The scaffolded app worked via JRuby's embedding API on the Servlet. The code is on github, https://github.com/yokolet/RailsCrossing, and the sample app by Scala is also on github, https://github.com/yokolet/RailsCrossingSamples.

The way to make it work was a bit hard. Rails API doc was not so helpful in terms of hooking it up on the Servlet. To figure out how it should have worked, I used rails console a lot, read sources, watched what were supposed to be in HTTP requests/responses over FireBug, wrote snippets to see what was missing. So, I'm going to write down how RailsCrossing (Rails on RedBridge) made scaffolded app to work.

The idea of RailsCrossing is to hit "SomeController.action('some_action').call(env)" method. Thus,

  1. Find the controller class
  2. Find the action
  3. Stuff all necessary information into "env" hash

are all to make it work.


Finding controller class is not so complicated. For example, we can confirm routes mapping on rails console:

irb(main):001:0> ActionDispatch::Routing::Routes.routes.to_s
=> "GET /Acacia/home/index(.:format) {:controller=>\"Acacia/home\", :action=>\"index\"}GET /Acacia/users(.:format) {:action=>\"index\", :controller=>\"users\"}POST /Acacia/users(.:format) {:action=>\"create\", :controller=>\"users\"}GET /Acacia/users/new(.:format) {:action=>\"new\", :controller=>\"users\"}GET /Acacia/users/:id/edit(.:format) {:action=>\"edit\", :controller=>\"users\"}GET /Acacia/users/:id(.:format) {:action=>\"show\", :controller=>\"users\"}PUT /Acacia/users/:id(.:format) {:action=>\"update\", :controller=>\"users\"}DELETE /Acacia/users/:id(.:format) {:action=>\"destroy\", :controller=>\"users\"}ANY /rails/info/properties(.:format) {:controller=>\"rails/info\", :action=>\"properties\"}"

Odd thing was the scope name "Acacia" appeared in a controller name when I created controller by "generate controller" but not by "generate scaffold." Although I surrounded whole stuff by a scope "/Acaia," routes.rb, it happened. I'm not sure this is intended or a bug, but, anyways, I took the scope name out from to find controller class if it is included.


Finding an action needed closer look at a Rails behavior. As you may know, Rails RESTful app uses GET, POST, PUT, and DELETE HTTP requests for index/show/new/edit, create, update, and destroy actions respectively. Even if the given path is the same, say /Acacia/users/1, it is for show when HTTP method is GET while update when PUT. Rails uses "_method" hidden HTML form field to know the difference. So, I added lines to get that parameter from HTTP request, then put in env hash with a "rack.methodoverride.original_method" key. Also the _method value was used to find out matched action and controller combination.
(See getEnv() and findMatchedRoute() methods in CrossingHelpers.java)


Stuffing all necessary information in the "env" hash was rather baffling. JRuby's Java proxy didn't work well since Rails expected hash keys could be referenced as a symbol. For example, a hash is created by h = {"name" => "foo"}, it should be referenced by h[:name]. JRuby's Java String proxy seemed not to be effective for this usage. To avoid this problem, I directly used JRuby's RubyHash class.

Next problem was keys to assign input parameters. "rack.input" is mandatory, but, as far as sending HTML form parameters, "rack.input" isn't used for that purpose. It is used to compare the value of "rack.request.form_input," and not used afterward as well as "rack.request.form_input." (If input is multipart/form-data (file upload), "rack.input" is used to read binary data, though.) Instead, "rack.request.form_hash" key is used to send input values. There's one more key-value pair involved in sending form parameters. It is "action_dispatch.request.path_parameters." The id, like user id, seems to be expected to be there. I put the hash got by Rails.application.routes.recognize_path("some path") as a value of ""action_dispatch.request.path_parameters." The example of a hash value related to sending form parameters became as in below:

"action_dispatch.request.path_parameters" => {:action=>"show", :controller=>"users", :id=>"1"}
"rack.input" => ""
"rack.request.form_input" => ""
"rack.request.form_hash" => {"utf8"=>"✓", authenticity_token"=>"fi/oOym8i+mZTwM7+h+rZBQL/s9hv62+
mnNRv6mNQnw=", "user"=>{"name"=>"foo", "email"=>"foo@bar"}, "commit"=>"Update User"}


Moreover, there are "flash" messages, such as green "User was successfully updated." Those also should be in "env" hash. This sort of messages are in a HTTP response returned as a result of form input processing. When the response is redirected and back to Rails app, the messages show up to browser. To keep the messages over a sequence of HTTP request/response has been completed, we need cookie like mechanism. Rails uses cookie to save such messages. On Servlet, we can use Servlet API's session, which is available to save an object. So, I took the hash for flash massages out from the response, then put it in Servlet's session without any modification.

String script =
"response = " + route.getName() + ".action('" + route.getAction() + "').call(env)\n" +
"return response[0], response[1], response[2].body, response[2].request.flash";
RubyArray responseArray = (RubyArray)container.runScriptlet(script);
CrossingResponse response = new CrossingResponse();
response.context_path = context_path;
response.status = ((Long)responseArray.get(0)).intValue(); //status code; Fixnum
response.responseHeader = (Map)responseArray.get(1); // response header; Hash
response.body = (String)responseArray.get(2); // response body; HTML
response.flash = (Map) responseArray.get(3); // flash messages; Hash

....

request.getSession().setAttribute("action_dispatch.request.flash_hash", crossingResponse.getFlash()); //save falsh messages to Servlet's session

....

// before sending a request to Rails app
Map map = (Map) request.getSession().getAttribute("action_dispatch.request.flash_hash");
if (map != null) env.put("action_dispatch.request.flash_hash", map);



Serving static assets, such as stylesheets and javascripts, needed to have another way. Even though I added the scope in routes.rb, Rails didn't add the scope name to static assets not like paths to displayed in HTML body part. There should have the Rails way to add some path to static assets always, but I didn't search that. Because I wanted to keep Rails app as much as it is. Instead, RailsCrossing rewrites the URI so that the Servlet context path (the same as the Rails scope name) will be in the URI just before the response is sent back to the browser. RailsCrossing has its own default Servlet (CrossingDefaultServlet.java) to serve such static files. This is necessary so that Servlet Container dispatches the HTTP request to Acacia web app. Unless, the request is dispatched to a Servlet Container's default Servlet that doesn't know where the files are.


Above are what I tried to get scaffolded Rails app to work. Probably, there are the right ways to get this done, but I couldn't find by googling. (I want to know where I should go.)


RailsCrossing is still on the long way to serve Rails app satisfactory, but getting closer and closer. As I wrote, simple scaffolded Rails app has been covered. Next would be Ajax request handling, file upload, complicated database schema, etc. ... maybe, I need to work more.

Tuesday, February 15, 2011

Using Rails from Scala

Using Rails from Scala? Yes, it is possible if you use RailsCrossing (https://github.com/yokolet/RailsCrossing). This is a following chapter of my previous blog post, The second step to Rails on RadBridge. As I wrote "I'll try Rails from Scala next." at the end of that blog post, I tried it today. How did it go? It worked just fine. Here's how I used Rails from Scala.


1. Preparation of Java web application project and Rails app

These are exactly the same as the previous post. See steps 1 through 3. This time, I named Java web application "Sycamore," but the rest of all are the same.


2. Scala Servlet for Rails

Writing Servlet by Scala is really easy. I read the article, The busy Java developer's guide to Scala: Scala and servlets for that and could easily follow. I don't explain about what is Scala Servlet because that's not my purpose. So, it's good for you to google or goto some sites to learn what it is.

Here's the Scala Servlet that initializes Rails app and dispatches HTTP request to Rails app:

package com.servletgarden.sycamore

import javax.servlet.ServletConfig
import javax.servlet.http.{HttpServlet, HttpServletRequest => HSReq, HttpServletResponse => HSResp}
import com.servletgarden.railsxing.CrossingServlet

class RailsScalaServlet extends CrossingServlet {
override def init(config: ServletConfig) = super.init(config)

override def doGet(request: HSReq, response: HSResp) = dispatch(request, response)

override def doPost(request: HSReq, response: HSResp) = dispatch(request, response)
}

Compare RailsScalaServlet above and SimpleSample Servlet in the previous post. Basically, both Servlet do the same thing. See the previous post for details about what those two servlets are doing.

That's it.


3. Scala Servlet Compilation

Before executing Scala Servlet, the Servlet needs to be compiled. The article mentioned above, "The busy...," explains how to do that. Also, Scala Ant Tasks section of Developer's Guides will help you to understand how to write build.xml. I'm using NetBeans and its Java web application projects, so I directly edited NetBeans' build.xml. My build.xml is https://gist.github.com/828420. It assumes the directory tree below:

Sycamore -+- src -+- java -+- com -+- servletgarden -+- sycamore -+- RailsScalaServlet.scala
+- web -+- META-INF -+- context.xml
+- WEB-INF -+- lib -+- Gemfile
| +- Gemfile.lock
| +- RailsCrossing.jar
| +- blog -+- Gemfile
| | +- README
| | +- Rakefile
| | +- ...
| +- jruby-complete.jar
| +- jruby -+- 1.8 -+- bin -+- bundle
| | | +- erubis
| | | +- ....
| | +- cache -+- ...
| | +- doc
| | +- gem -+- abstract-1.0.0 -+- ..
| | +- gem -+- .... -+- ..
| | +- gem -+- bundler-1.0.10 -+- ..
| | +- gem -+- .... -+- ..
| | +- specifications -+- ...
| +- scala-compiler.jar
| +- scala-library.jar
|
+- index.jsp

In my case, NetBeans' build menu compiles Scala files and puts compiled classes under Sycamore/build/web/WEB-INF/classes directory.


4. Configurations

SimpleSample Servlet in the previous post uses annotation to configure the Servlet, thus, no web.xml is there. On the other hand, RailsScalaServlet uses web.xml to set rail_path and gem_path, and path mapping. The web.xml file is https://gist.github.com/828572.

Double check the context name of this Java web application has been changed to "blog." See "5. One More Setup" of the previous post.


5. Run Scala Servlet to use Rails

After starting tomcat, request http://localhost:8080/blog/home/index from a web client. Like in the previous post, familiar Rails default page will show up.



As I wrote how to here, using Rails from Scala is relatively easy. If you want to mix Scala code and Rails results, you can use methods of CrossingHelpers bypassing CrossingServlet. Also, you can use JRuby's ScriptingContainer to take Rails classes out to call methods of them. Try this new Rails way.

Monday, February 14, 2011

The second step to Rails on RedBridge

This is the second attempt to make Rails work on RedBrdige (JRuby embedding API). I believe Rails on RedBridge got closer to a real application. The first attempt is also in this blog, A small step to Rails on RedBridge. The blog post had some impact on a few people who want to control Rails (or Sinatra) from Java. The example there successfully showed how we could wake Rails up from Java Servlet, but was almost a hello world example. Meanwhile, I demonstrated simple Rails app from Groovy (Groovy Servlet, precisely) in my session at RubyConf 2010, "RubyGems to All JVM Languages." Rails on RedBridge way is an approach from Java API on Servlet, so all JVM languages are possibly available to use Rails *gem* from them over JRuby's Java API. That demonstration at RubyConf 2010 impressed the audiences there enough to have a clap, but I knew it was still a hello world example. Recently, I tried to develop Rails on RedBridge more so that Rails app got work like other rackup style frameworks. RailsCrossing is the outcome. It is the Java API to use Rails.


OK. I'm going to write how to setup and write code using RailsCrossing.


1. Creating Java Web Application Project

The first job is to create a Java web application project. I used NetBeans for that. You can use whatever IDE you like that has the feature to create a Java Servlet based web application project. My web app project has a name "Spruce" and the server to run it is Tomcat 7.0.4.

Initially, a directory tree of the Spruce project looks like below (I didn't write NetBeans specific files/directories):

Spruce -+- src
+- web -+- META-INF -+- context.xml
+- WEB-INF
+- index.jsp


The directory tree is slightly different when other IDEs are used. That's not the matter. Just look at WEB-INF directory. The WEB-INF directory is special for Java Servlet web app. It is the place to put libraries and is protected from accesses originated from web clients.


2. Prepare for Rails APP

RailsCrossing assumes Rails app is located under WEB-INF directory. It could not be necessarily there. However, I made that constraint since Java Servlet web application should be portable. Everything needed to the web app work should be in the web app directory tree. It's the policy.

Let's prepare the app.

First, get jruby-complete.jar and put it in WEB-INF/lib directory. (Get the latest (master) branch of JRuby. Or, you can use jruby-complete.jar of RailsCrossing project)

Spruce -+- src
+- web -+- META-INF -+- context.xml
+- WEB-INF -+- lib -+- jruby-complete.jar
+- index.jsp


Next, install bundler gem. Make sure bundler will be installed under WEB-INF directory. So, let's go to lib directory and type as in below (you can use jruby command instead):

$ java -jar jruby-complete.jar -S gem install bundler --no-ri --no-rdoc -i jruby/1.8
Fetching: bundler-1.0.10.gem (100%)
Successfully installed bundler-1.0.10
1 gem installed

Don't forget -i option to install bundler gem under lib directory. The command above installs bundler gem in WEB-INF/lib/jruby/1.8. So, you can find bundle command in the path, WEB-INF/lib/jruby/1.8/bin/bundle.

Spruce -+- src
+- web -+- META-INF -+- context.xml
+- WEB-INF -+- lib -+- jruby-complete.jar
| +- jruby -+- 1.8 -+- bin -+- bundle
| +- cache -+- ...
| +- doc
| +- gem -+- bundler-1.0.10 -+- ..
| +- specifications -+- ...
|
+- index.jsp


Let's set GEM_PATH to use bundler to install other gems.
$ export GEM_PATH=[path to lib directory]/jruby/1.8


Write or create Gemfile. If you want to use bundle command rather than open a new file by editor, type as in below:
java -jar jruby-complete.jar -S jruby/1.8/bin/bundle init

Gemfile should be something like this:

# A sample Gemfile
source "http://rubygems.org"

gem "rails"

I just deleted the comment sign (#) on the rails line after bundle init command. The last preparation is to install rails gem.
java -Xmx500m -jar jruby-complete.jar -S jruby/1.8/bin/bundle install --path .

Don't forget "--path ." to install gems under the same path as bundler has been installed in. When bundle install command successfully finished, bunch of gems are in jruby/1.8.

Spruce -+- src
+- web -+- META-INF -+- context.xml
+- WEB-INF -+- lib -+- Gemfile
| +- Gemfile.lock
| +- jruby-complete.jar
| +- jruby -+- 1.8 -+- bin -+- bundle
| | +- erubis
| | +- ....
| +- cache -+- ...
| +- doc
| +- gem -+- abstract-1.0.0 -+- ..
| +- gem -+- .... -+- ..
| +- gem -+- bundler-1.0.10 -+- ..
| +- gem -+- .... -+- ..
| +- specifications -+- ...
|
+- index.jsp



3. Create Rails App

Probably, I don't need to say anything. I typed below:
java -jar jruby-complete.jar -S jruby/1.8/bin/rails new blog --template http://jruby.org

Now, blog Rails app is created under WEB-INF/lib/blog.

Spruce -+- src
+- web -+- META-INF -+- context.xml
+- WEB-INF -+- lib -+- Gemfile
| +- Gemfile.lock
| +- blog -+- Gemfile
| | +- README
| | +- Rakefile
| | +- ...
| +- jruby-complete.jar
| +- jruby -+- 1.8 -+- bin -+- bundle
| | +- erubis
| | +- ....
| +- cache -+- ...
| +- doc
| +- gem -+- abstract-1.0.0 -+- ..
| +- gem -+- .... -+- ..
| +- gem -+- bundler-1.0.10 -+- ..
| +- gem -+- .... -+- ..
| +- specifications -+- ...
|
+- index.jsp


In general, database setup follows. But, to make it understandable to know how it works, let's create a controller.

$ cd blog (Now, I'm in Spruce/web/WEB-INF/lib/blog directory)
$ java -Xmx500m -jar ../jruby-complete.jar ../jruby/1.8/bin/bundle install --path ../. (Don't forget --path ../. option!)
$ java -jar ../jruby-complete.jar script/rails g controller home index

So far so good. These steps to creating Rails app should be familiar with.


4. Simple Sample Servlet to run Rails' controller

Here comes RailsCrossing API. You can write Servlet using methods in CrossingHelpers or subclassing CrossingServlet. The CrossingServlet does everything to run Rails' controller. For example, SimpleSample below would be the simplest example.

package com.servletgarden.spruce;

import com.servletgarden.railsxing.CrossingServlet;
import java.io.IOException;
import javax.servlet.Servlet;
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.annotation.WebInitParam;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
*
* @author Yoko Harada
*/
@WebServlet(name="SimpleSample",
urlPatterns={"/*"},
initParams={@WebInitParam(name="rails_path", value="/lib/blog"), @WebInitParam(name="gem_path", value="/lib/jruby/1.8")})
public class SimpleSample extends CrossingServlet {

/**
* @see Servlet#init(ServletConfig)
*/
@Override
public void init(ServletConfig config) throws ServletException {
super.init(config);
}

/**
* Processes requests for both HTTP GET and POST methods.
* @param request servlet request
* @param response servlet response
* @throws ServletException if a servlet-specific error occurs
* @throws IOException if an I/O error occurs
*/
protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {

dispatch(request, response);
}

/**
* Handles the HTTP GET method.
* @param request servlet request
* @param response servlet response
* @throws ServletException if a servlet-specific error occurs
* @throws IOException if an I/O error occurs
*/
@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}

/**
* Handles the HTTP POST method.
* @param request servlet request
* @param response servlet response
* @throws ServletException if a servlet-specific error occurs
* @throws IOException if an I/O error occurs
*/
@Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}

/**
* Returns a short description of the servlet.
* @return a String containing servlet description
*/
@Override
public String getServletInfo() {
return "the simplest example used RailsCrossing";
}

}


Let's look at what's going on in detail.

In the SimpleSample init method, it calls init method of a super class, CrossingServlet. In CrossingServlet init method, initialization of ScriptingContainer (JRuby embedding API a.k.a. RedBridge) and Rails are done. Then, Rails routes mappings are parsed.

public void init(ServletConfig config) throws ServletException {
container = CrossingHelpers.initialize(config);
routes = CrossingHelpers.parseRoutes(container);
}

While initialization is going on CrossingServlet uses "rails_path" and "gem_path" initParams relative to WEB-INF directory. These are necessary to fire up Rails.


Well, how Rails routes mapping can be parsed? Probably, many of you have used rails console. RailsCrossing does the same thing on a Servlet as people do on rails console. For example,

$ java -jar ../jruby-complete.jar script/rails c
irb(main):007:0> route_ary = ActionDispatch::Routing::Routes.routes
=> [#<ActionDispatch::Routing::Route:0x555669ae @name="home_index"...(snip)
irb(main):004:0> route_ary[0].to_a[1]
=> {:path_info=>/\A\/home\/index(?:\.([^\/.?]+))?\Z/, :request_method=>/^GET$/}
irb(main):005:0> route_ary[0].to_a[2]
=> {:controller=>"home", :action=>"index"}
irb(main):006:0> route_ary[0].to_a[3]
=> "home_index"

CrossingHelpers.parseRoutes method exactly evaluates these commands to draw mapping info out from Rails.


Let get back to SimpleSample Servlet. Look at the processRequest method. In this method, CrossingServlet's dispatch method is called. Just that. What's this dispatch?
Here it is:

protected void dispatch(HttpServletRequest request, HttpServletResponse response) throws IOException {
CrossingRoute route = findMatchedRoute(request.getContextPath(), request.getPathInfo(), request.getMethod());
if (route == null) return;
Map env = getEnvMap(request);
CrossingResponse crossingResponse = CrossingHelpers.dispatch(container, route, env);
response.setStatus(crossingResponse.getStatus());
Set keys = crossingResponse.getResponseHeader().keySet();
for (String key : keys) {
String value = crossingResponse.getResponseHeader().get(key);
response.setHeader(key, value);
}
PrintWriter writer = response.getWriter();
writer.write(crossingResponse.getBody());
}

Based on the parsed routes, CrossingServlet finds what controller should be used from path_info in HttpServletRequest. If matched controller is found, it creates a map of each HTTP request params including HTTP request header and query string. Then, dispatching. Here, RailsCrossing does rails console commands on Servlet again.

irb(main):006:0> env = {"rack.input" => "", "REQUEST_METHOD" => "GET"}
=> {"rack.input"=>"", "REQUEST_METHOD"=>"GET"}
irb(main):008:0> HomeController.action('index').call(env)[0]
=> 200
irb(main):009:0> HomeController.action('index').call(env)[1]
=> {"Content-Type"=>"text/html; charset=utf-8", "ETag"=>"\"a5e60d2fa2208dc316b0f09cef107bba\"", "Cache-Control"=>"max-age=0, private, must-revalidate"}
irb(main):010:0> HomeController.action('index').call(env)[2].body
=> "<!DOCTYPE html>\n<html>\n<head>\n <title>Blog</title>\n...

Once CrossingServlet gets the response, it sets HTTP response status code and header params, then writes html part out to Servlet's writer.

These are what RaisCrossing does.


5. One More Setup

The difference of a context path lies between Java web app and Rails app. In general, Java web application is referenced by http://servername:port/context_name/servlet_name/path_info, while rails is done by http://servername:port/path_info. We need to fix these to make them coincide.

Let's change Rails first editing config/routes.rb:

Blog::Application.routes.draw do
scope "/blog" do
get "home/index"
...
end
end

I added scope in config/routes.rb. Now, blog app is referenced by http://servername:port/blog/path_info. Next is a tomcat setting. I edited META_INF/context.xml:

<?xml version="1.0" encoding="UTF-8"?>
<Context antiJARLocking="true" path="/blog"/>

Originally, it was path="/Spruce" since I named the web app Spruce. After Spruce has been changed to "blog," every HTTP request to http://servername:port/blog/* has been directed to SimpleSample Servlet.


6. Run Servlet

Since I created web app on NetBeans, I can get SimpleSample Servlet started from NetBeans menu with the path, "/home/index" If not, start tomcat up and request "http://localhost:8080/blog/home/index" You'll see familiar controller default output on a browser.


7. Thoughts

RailsCrossing uses Rails metal introduced in version 3. This rack style invocation was easy to get it run. However, it is mostly undocumented area. I attempted many commands on rails console and tried to find the best way; however, there might be still better way.

A good side of RailsCrossing is ... it is a Java API. This means, JVM languages such as Scala, Groovy, or other can use RailsCrossing over java integration feature. I'll try Rails from Scala next.