Tender Lovemaking » johnson

Testing JavaScript Outside the Browser

Aaron Patterson — Mon, 06 Apr 2009 03:15:24 +0000

The other day at LA RubyConf during the Johnson presentation, I showed a few slides which I don’t think were given the time that they deserve. Not that we didn’t have enough time, I just don’t think I made as big a deal about them as I should have. Those particular slides demonstrated HTML Document Object manipulation executed in JavaScript outside any web browser. Those particular slides, and that particular code, is the culmination of over a year worth of work (and Yak Shaving) and I would like to talk about it a little more in detail here.

Since I started doing any sort of non-trivial browser dependent JavaScript, I’ve wanted to be able to test the code which I wrote. Hitting refresh on a webpage seems like a hack. Setting up a special browser to refresh the page for me also seems like a hack. I want to run “rake test” and have my JavaScript DOM manipulations tested right along with everything else, no browser dependence required. As far as I could tell, we need three things to make that happen:

A JavaScript runtime that can be used in Ruby
A parser with browser-like HTML correction schemes
A DOM interface that mirrors a browsers DOM interface

Over the weekend, I think we’ve come a lot closer. John has finally released Johnson. Johnson solves problem number 1. Johnson provides a JavaScript runtime that is fully accessible in Ruby. Watch our RubyConf 2008 presentation about Johnson for more details about that project.

Number 2, I believe, has been solved by nokogiri. As far as I can tell, the tree generated inside libxml2 is very similar to one found in the browser. Nokogiri was partly a Yak Shave for number 3. Since I had writing a DOM interface in mind, nokogiri’s api lends well to writing a DOM api.

Number 3 was partly solved this weekend. I’ve been working on a DOM api called taka. Taka sits a DOM api on top of nokogiri. The goal of the project is to mirror a browser’s DOM api in Ruby.

With these three tools in place, I believe that we have a good start on a browserless JavaScript testing environment.

Codes

Enough talk. Let’s look at some codes. Take this HTML page for example:


  
    
  
  
    Behold the Johnson

The JavaScript in this HTML will add a few option tags as children of the select tag. Effectively populating the drop down for our user. It would be nice if we could write a test to assert that when this JavaScript executes, the option tags are actually added as children of the select tag.

With Johnson and Taka, it is possible to write such a test:

require 'rubygems'
require 'taka'
require 'johnson'
require 'test/unit'

class OptionTagsAppendedTest < Test::Unit::TestCase
  def setup
    # Create our DOM object
    @document = Taka::DOM::HTML(DATA.read)

    # Create a new JavaScript runtime
    @rt = Johnson::Runtime.new

    # Set the document in the runtime
    @rt['document'] = @document

    # Execute any script tags
    @document.getElementsByTagName('script').each do |script|
      @rt.evaluate(script.textContent);
    end
  end

  def test_options_populated_by_onload
    # 0 option tags before onload is executed
    assert_equal 0, @document.getElementsByTagName('option').length

    # Execute the onload body attribute
    @rt.evaluate(@document.getElementsByTagName('body')[0].onload)

    # 4 option tags after onload is executed
    assert_equal 4, @document.getElementsByTagName('option').length
  end
end

There. It’s done. This test executes the JavaScript and manipulates your HTML the same way the browser would. You can run this code today, just make sure to install the johnson and taka gems first.

Problems

There are at least a few problems. This HTML code is, admittedly, carefully crafted. So far, taka only implements the DOM 1 interface. That means taka is missing many methods that are available in browsers. The good news though is that Taka is pure ruby and open source. As soon as you find methods that are missing, fork the repo, add a test, and send a pull request. I will be sure to merge it.

Conclusion

We are making progress towards testing JavaScript without a browser. We have to do it a step at a time. The solution I have presented to you, while not complete, has promise. I think that the only thing standing in our way right now is time and man power. The methods that need to be implemented on Taka to make it mirror a browser are not hard (take a look at the taka source). These methods just need to be written.

Write your Rails view in……. JavaScript?

Aaron Patterson — Tue, 06 May 2008 18:33:30 +0000

In my last post about Johnson, I said that next time I would talk about the JavaScript parse tree that Johnson provides. Well, I changed my mind. Sorry.

I want to write about a rails plugin that I added to Johnson. Brohuda Katz wrote an ERb type parser in JavaScript, and added it to the (yet to be released) Johnson distribution. With that in mind, and looking at the new template handlers in edge rails, I was able to throw together a rails plugin that allows me to use JavaScript in my rails view code.

Lets get to the code. Here is my controller:

class JohnsonController < ApplicationController
  def index
    @users = User.find(:all)
  end
end

And my EJS view (the file is named index.html.ejs):

<% for(var user in at.users) { %>
  <%= user.first_name() %>

<% } %>

The johnson rails plugin puts controller instance variables in to a special javascript variable called “at”. The “at” variable is actually a proxy to the controller, lazily fetching instance variables from the controller and importing those objects in to javascript land.

Lets take a look at the plugin, its only a few lines:

class EJSHandler < ActionView::TemplateHandler
  class EJSProxy # :nodoc:
    def initialize(controller)
      @controller = controller
    end

    def key?(pooperty)
      @controller.instance_variables.include?("@#{pooperty}")
    end

    def [](pooperty)
      @controller.instance_variable_get("@#{pooperty}")
    end

    def []=(pooperty, value)
      @controller.instance_variable_set("@#{pooperty}", value)
    end
  end

  def initialize(view)
    @view = view
  end

  def render(template)
    ctx = Johnson::Context.new
    ctx.evaluate('Johnson.require("johnson/template");')
    ctx['template'] = template.source
    ctx['controller'] = @view.controller
    ctx['at'] = EJSProxy.new(@view.controller)

    ctx.evaluate('Johnson.templatize(template).call(at)')
  end
end

ActionView::Template.register_template_handler("ejs", EJSHandler)

When the template gets rendered (the render method), I wrap the controller with an EJS proxy, then compile the template into a javascript function, and call that function. The “at” variable is set to the EJSProxy before executing the template, and all property accessing on the “at” variable is passed along to fetching instance variables from the controller.

Server side javascript coding in rails. Weird, eh?

Take it to the limit one more time

Aaron Patterson — Wed, 23 Apr 2008 18:22:29 +0000

Sup bros. I need to post in this thing more often. Yesterday, someone tipped over my scooter again. I’m getting kind of tired of that.

Anyway, its time for me to write about this. RKelly is pretty much dead. For the past few months, John and I have been working on RKelly’s replacement called Johnson. Basically we’re now putting a ruby wrapper around Mozilla’s Spidermonkey. The project is coming along quite nicely. Ruby objects can be passed in to javascript land, and javascript objects can be passed back in to ruby land.

For example, we can define an alert function in our javascript context:

require 'johnson'

ctx = Johnson::Context.new
ctx['alert'] = lambda { |x| puts x }
ctx.evaluate('alert("Hello world!");')

Johnson::Context#evaluate will also return the last statement evaluated. We can evaluate an expression, and manipulate that expression in ruby land. For example, I’ll create an object in javascript, return it to ruby land, then access a property of the javascript object:

require 'johnson'

ctx = Johnson::Context.new
obj = ctx.evaluate('var foo = { x: "hello world" }; foo')
puts obj.x  # => 'hello world'

We can even do the reverse by stuffing ruby objects in to the context:

A = Struct.new(:foo)

ctx = Johnson::Context.new
ctx['alert'] = lambda { |x| puts x }
ctx['a'] = A.new("bar")
ctx.evaluate('alert(a.foo);') # => 'bar'

But it gets better. We added a top level variable called “Ruby” that lets you access constants and globals from Ruby land. We can rewrite the previous example completely in javascript:

ctx = Johnson::Context.new
ctx.evaluate("var x = new (new Ruby.Struct(Johnson.symbolize('foo')));")
ctx.evaluate("x.foo = 'bar'")
puts ctx.evaluate('x').foo # => 'bar'
puts ctx.evaluate('x').class # => #

Since the ‘Ruby’ constant delegates to Object, you can access any constant. Including ones you’ve defined yourself. We could, for example, look up a bunch of User records through rails:

ctx = Johnson::Context.new
ctx['alert'] = lambda { |x| puts x }
ctx.evaluate(<<-END
             for(var user in Ruby.User.find(Johnson.symbolize('all'))) {
               alert(user.first_name());
             }
             END
            )

You might be wondering what this Johnson.symbolize business is about. Since Javascript doesn’t have a concept of a symbol, we’ve created a helper to “mark” a string as a symbol and pass it back in to ruby land.

To conclude this update about my Johnson, I’d like to show off an interactive shell for Johnson (thanks to Brohuda Katz). Johnson has an interactive shell that lets you try things out in javascript land or ruby land, and let you quickly switch between the two. Typing ‘js’ will put in you the javascript shell, ‘rb’ will switch you to the ruby shell. In the ruby shell, you can use the ‘cx’ variable to get ahold of you javascript context:
$ ruby -I lib bin/johnson js> var x = { foo: 'bar', hello: function() { return 'world' } }; => nil js> rb rb> cx['x'].foo => "bar" rb> cx['x'].hello() => "world" rb> [/sourcecode] We aren't quite ready for a release yet, but if you'd like to play around with Johnson, you can pull it down from github here. Just run 'rake', and you should have it compiled and running!


My next Johnson related post will be about Javascript parse trees and Javascript code generation.