Friday, October 22, 2010

Gems in a Jar with RedBridge

Using JRuby embed API (RedBridge), it is easy to use Ruby gems from Java. However, packaging might not be simple, so people occasionally struggle to create a "portable" package. Being portable is important for a Java app. All stuffs of the Java app should be packaged in a jar archive, which should work on another PC, or even different OS. This is the good side of Java. This blog illustrates one solution for packaging.

I intentionally didn't use jruby command since the command hides what are going on behind. You might be exhausted by lines of "java -jar ....," sorry. But, it helps to understand a packaging process.

Directories


Since I used bundler to install gems, I created directories below to fit them to bundler.

Linden -+- lib -+- jruby -+- 1.8
+- src -+- linden
+- build

"Linden" is a top directory and the name doesn't have any special meaning. You can name it whatever you like. "lib/jruby/1.8" is the directory gems will be installed. "src" is for Java code, and "build" is for compiled *.class files.

Bundler Installation


I intentionally used jruby-complete.jar to assure gems won't be installed in the default location. The path to jruby-complete.jar can be anything since typing a full path to jruby-complete.jar works. But, I like a short path to type, so I copied jruby-complete.jar under "Linden/lib." Now, the directories/files were as in below:

Linden -+- lib -+- jruby -+- 1.8
+- jruby-complete.jar
+- src -+- linden
+- build


Then, the bundler installation went:

cd lib
java -jar jruby-complete.jar -S gem install bundler jruby-openssl --no-ri --no-rdoc -i jruby/1.8

Bundler and jruby-openssl gems were installed and the directories became as in below:

Linden -+- lib -+- jruby -+- 1.8 -+- bin -+- bundle
| +- cache -+- ...
| +- doc
| +- gems -+- bouncy-castle-java-1.5.0145.2 -+- ...
| | +- bundler-1.0.3 -+- ...
| | +- jruby-openssl-0.7.1 -+- ...

| +- specifications -+- ...
+- jruby-complete.jar
+- src -+- linden
+- build

Let's see whether gems are really installed.

export GEM_PATH=`pwd`/jruby/1.8
java -jar jruby-complete.jar -S gem list

This command should print out the installed three gems and pre-installed gems.

*** LOCAL GEMS ***

bouncy-castle-java (1.5.0145.2)
bundler (1.0.3)
columnize (0.3.1)
jruby-openssl (0.7.1)
rake (0.8.7)
rspec (1.3.0)
ruby-debug (0.10.3)
ruby-debug-base (0.10.3.2)
sources (0.0.1)

OK. Bundler was installed successfully.

Installation of other gems


Next step is to install other gems using bundler, so I used bundle command. Theoretically, PATH environment variable should work to find bundle command by ruby. Unfortunately, this didn't work in my "java -jar ..." usage. Instead, I typed a path to bundle command after -S option.

java -jar jruby-complete.jar -S jruby/1.8/bin/bundle init

Then, I added "twitter" gem to Gemfile.

# A sample Gemfile
source "http://rubygems.org"

# gem "rails"

gem "twitter"

The installation went on:

java -Xmx500m -jar jruby-complete.jar -S jruby/1.8/bin/bundle install --path=.

The java command option "-Xmx500m" is for performance and to avoid memory outage. Now, 6 gems were installed and the directories/files became:

Linden -+- lib -+- jruby -+- 1.8 -+- bin -+- bundle
| | +- httparty
| | +- oauth

| +- cache -+- ...
| +- doc
| +- gems -+- bouncy-castle-java-1.5.0145.2 -+- ...
| | +- bundler-1.0.3 -+- ...
| | +- crack-0.1.8 -+- ...
| | +- hashie-0.4.0 -+- ...
| | +- httparty-0.6.1 -+- ...

| | +- jruby-openssl-0.7.1 -+- ...
| | +- multi_json-0.0.4 -+- ...
| | +- oauth-0.4.3 -+- ...
| | +- twitter-0.9.12 -+- ...

| +- specifications -+- ...
+- jruby-complete.jar
+- Gemfile
+- Gemfile.lock

+- src -+- linden
+- build


Java code to use twitter gem


Since all gems were ready, I wrote Java code to see gems worked. This time, I relied on GEM_PATH environment variable to find gems. The fist code had Java package, linden, and class name, SearchSample. So, I created SearchSample.java file under "src/linden" directory.

package linden;

import org.jruby.embed.LocalContextScope;
import org.jruby.embed.ScriptingContainer;

public class SearchSample {
private String jarname = "sample.jar";

private SearchSample() {
String basepath = System.getProperty("user.dir");
System.out.println("basepath: " + basepath);
ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
System.out.println("jrubyhome: " + container.getHomeDirectory());
container.runScriptlet("ENV['GEM_PATH']='" + basepath + "/lib/jruby/1.8'");

String script =
"require 'rubygems'\n" +
"require 'twitter'\n" +
"require 'pp'\n" +
"pp Twitter::Search.new('#jruby').fetch.results.first";
container.runScriptlet(script);
}

public static void main(String[] args) {
new SearchSample();
}
}

Linden -+- lib -+- jruby -+- 1.8 -+- bin -+- bundle
| | +- httparty
| | +- oauth
| +- cache -+- ...
| +- doc
| +- gems -+- bouncy-castle-java-1.5.0145.2 -+- ...
| | +- bundler-1.0.3 -+- ...
| | +- crack-0.1.8 -+- ...
| | +- hashie-0.4.0 -+- ...
| | +- httparty-0.6.1 -+- ...
| | +- jruby-openssl-0.7.1 -+- ...
| | +- multi_json-0.0.4 -+- ...
| | +- oauth-0.4.3 -+- ...
| | +- twitter-0.9.12 -+- ...
| +- specifications -+- ...
+- jruby-complete.jar
+- Gemfile
+- Gemfile.lock
+- src -+- linden -+- SearchSample.java
+- build



Compile and run using rake-ant integration



Everything was ready. OK, how do I compile and run it? Of course, classic "javac" and "java" command were the options. But, these commands are not very convenient to repeat compile/run. So, I used JRuby's rake-ant integration. Yes, I wrote "Rakefile" to compile and run Java code. Isn't it nice? Here's Rakefile:

require 'ant'

namespace :ant do
task :compile => :clean do
ant.javac :srcdir => "src", :destdir => "build"
end
end

namespace :ant do
task :java => :compile do
ant.java :classname => "linden.SearchSample" do
classpath do
pathelement :location => "lib/jruby-complete.jar"
pathelement :path => "build"
end
end
end
end

require 'rake/clean'

CLEAN.include '*.class', '*.jar'

I created Rakefile under the Linden directory, so now the directories/files were:

Linden -+- lib -+- jruby -+- 1.8 -+- bin -+- bundle
| | +- httparty
| | +- oauth
| +- cache -+- ...
| +- doc
| +- gems -+- bouncy-castle-java-1.5.0145.2 -+- ...
| | +- bundler-1.0.3 -+- ...
| | +- crack-0.1.8 -+- ...
| | +- hashie-0.4.0 -+- ...
| | +- httparty-0.6.1 -+- ...
| | +- jruby-openssl-0.7.1 -+- ...
| | +- multi_json-0.0.4 -+- ...
| | +- oauth-0.4.3 -+- ...
| | +- twitter-0.9.12 -+- ...
| +- specifications -+- ...
+- jruby-complete.jar
+- Gemfile
+- Gemfile.lock
+- src -+- linden -+- SearchSample.java
+- build
+- Rakefile


When I typed "rake ant:java", the Java code above worked and printed out one tweet.

java -jar lib/jruby-complete.jar -S rake ant:java
(in /Users/yoko/Works/tmp/Linden)
basepath: /Users/yoko/Works/tmp/Linden
jrubyhome: file:/Users/yoko/Works/tmp/Linden/lib/jruby-complete.jar!/META-INF/jruby.home
{"profile_image_url"=>
"http://a3.twimg.com/profile_images/76084835/web-profile_normal.jpg",
"created_at"=>"Fri, 22 Oct 2010 15:38:09 +0000",
"from_user"=>"mccrory",
"metadata"=>{"result_type"=>"recent"},
"to_user_id"=>nil,
"text"=>
"RT @carlosqt: #IronRuby 1.1.1 a released! download from: http://ironruby.codeplex.com/ #programming #JRuby #Ruby #Rails #dotnet #code",
"id"=>28415816774,
"from_user_id"=>1757577,
"geo"=>nil,
"iso_language_code"=>"en",
"source"=>"<a href="http://twitter.com/">web</a>"}


Packaging


This time I can't rely on GEM_PATH environment variable. If GEM_PATH had worked also for the path in a jar, unfortunately, it didn't. Instead of GEM_PATH, I set all path to gems to ScriptingContainer. This was not complicated since all gems were installed in the same directory. I fixed the jar name, "sample.jar," so this name appeared in the Java code. This name could have been given via a command line argument. Edited SearchSample.java is in below:

package linden;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import java.util.zip.ZipEntry;

import org.jruby.embed.LocalContextScope;
import org.jruby.embed.ScriptingContainer;

public class SearchSample {
private String jarname = "sample.jar";

private SearchSample() throws IOException {
String basepath = System.getProperty("user.dir");
System.out.println("basepath: " + basepath);
ScriptingContainer container = new ScriptingContainer(LocalContextScope.SINGLETHREAD);
System.out.println("jrubyhome: " + container.getHomeDirectory());
container.setLoadPaths(getGemPaths(jarname, basepath));

String script =
"require 'rubygems'\n" +
"require 'twitter'\n" +
"require 'pp'\n" +
"pp Twitter::Search.new('#jruby').fetch.results.first";
container.runScriptlet(script);
}

private List<String> getGemPaths(String jarname, String basepath) throws IOException {
JarFile jarFile = new JarFile(basepath + "/" + jarname);
Enumeration<JarEntry> entries = jarFile.entries();
String gempath = "lib/jruby/1.8/gems/";
Set<String> gemnames = new HashSet<String>();
while (entries.hasMoreElements()) {
ZipEntry entry = (ZipEntry) entries.nextElement();
String entryName = entry.getName();
if (entryName.startsWith(gempath) && entryName.length() > gempath.length()) {
String n = entryName.substring(gempath.length());
String m = n.substring(0, n.indexOf("/"));
gemnames.add(m);
}
}
List<String> gemPaths = new ArrayList<String>();
for (String gem : gemnames) {
gemPaths.add("file:" + basepath + "/" + jarname + "!/lib/jruby/1.8/gems/" + gem + "/lib");
}
return gemPaths;
}


public static void main(String[] args) throws IOException {
new SearchSample();
}
}

To make jar archive, I added jar task to my Rakefile.

require 'ant'

namespace :ant do
task :compile => :clean do
ant.javac :srcdir => "src", :destdir => "build"
end
end

namespace :ant do
task :java => :compile do
ant.java :classname => "linden.SearchSample" do
classpath do
pathelement :location => "lib/jruby-complete.jar"
pathelement :path => "build"
end
end
end
end

namespace :ant do
task :jar => :compile do
ant.jar :basedir => ".", :destfile => "sample.jar" do
fileset :dir => "build" do
include :name => "**/*.class"
end
include :name => "lib/jruby/1.8/gems/**/*"
manifest do
attribute :name => "Main-Class", :value => "linden.SearchSample"
end
end
end
end


require 'rake/clean'

CLEAN.include '*.class', '*.jar'

All right, let's create jar archive.

java -jar lib/jruby-complete.jar -S rake ant:jar

The ant:jar task created the sample.jar archive in Linden directory. To test this archive was really gems in a jar, I unset GEM_PATH environment variable first. Then, I typed java command and got one tweet, Yay!

unset GEM_PATH; echo $GEM_PATH
java -cp lib/jruby-complete.jar:sample.jar linden.SearchSample

To ensure that the jar archive had gems in the jar, I moved sample.jar to a different directory and typed java command with the full path to jruby-complete.jar. It worked.

cp sample.jar ../.
cd ..
java -cp Linden/lib/jruby-complete.jar:sample.jar linden.SearchSample



This might be one answer of packaging gems in a jar and using gems from RedBridge.

4 comments:

Sunil Pradhan said...

Thanks a lot Yoko ...this article helped me a lot.But i have some doubts creating the jar.as m not familiar with ant m not able to understand how to make jar using that.

Can we make the jar manually from the command prompt.will it work?

Could you pls help me to make a jar manually without using ant?

yokolet said...

Sunil,
Of course, you can make a jar file using "jar" command. If you have JDK (not JRE), you already have jar command. Go to http://download.oracle.com/javase/tutorial/deployment/jar/build.html how to use it. It's quite similar to tar command of linux/bsd/solaris...

walt said...

if you are using Windows you cannot export with cygwin, ruby gets confused by the cygpath, but other than that this guide was helpful to me. thanks

sarabjeet said...

I am basically not a programmer and I am comparatively new to Java technology , so I was wondering what all topics should be covered up if i have to start java from the start and has any one
studied or got any info regarding this 6 week java training online course http://www.wiziq.com/course/12145-the-6-week-complete-java-primer-with-training-certificate and should we also have knowledge of C language before we further move on to Advance Java topics??