Tenderlove Making

Predicting Test Failures

Running tests is the worst. Seriously. It takes forever, and by the time they’re all done running, I forgot what I was doing. Some apps take 20 to 30 min to run all the tests, and I just can’t wait that long. What bothers me even more than waiting so long, is that after I make a change to the code, 99% of the tests aren’t even running the code that I changed! Why am I spending time running code that is unrelated to the change I made?

On the Rails team, we keep talking about ways to generally make Rails app test suites faster by introducing things like parallel testing. But what if we could predict which tests are likely to fail after we’ve changed the code? If we run just those tests, then we could send a pull request and let Travis run the thousands of unrelated tests in the whole suite.

An excellent way to cut your test time in half is to only run half the tests!

Regression Test Selection

Picking the tests that may fail depending on the code you’ve changed is called Regression Test Selection. Today we’re going to build a regression test selector for Minitest and RSpec. The goal of this program is to answer the question:

If I modify line A of file B, what tests should I run?

In order to build this, we need to:

  • Collect code coverage info on a per test basis
  • Figure out what code has changed
  • Map changed code back to existing tests

Lets start with collecting per test coverage.

Collecting per-test coverage

At a high level, in order to collect per-test coverage, what we’ll do is:

  1. Record a snapshot of the current coverage info
  2. Run the test
  3. Record a snapshot of the current coverage info

In order to do this, you’ll need to be running trunk Ruby which has this patch. Our code will look somewhat like this:

require 'coverage'

Coverage.start # start the coverage tool

before = Coverage.peek_result # grab the current coverage
after = Coverage.peek_result # grab the current coverage

save_the_test_info_with_coverage(before, after)

For each test framework we need to figure out how to wrap each test case with the code to collect coverage information, and record that coverage information along with enough information to rerun just that one test.

First we’ll implement this using Minitest, then implement with RSpec.

Here is our code for Minitest:

require 'coverage'
require 'json'


require 'minitest'

class Minitest::Runnable
  LOGS = []

  Minitest.after_run {
    File.open('run_log.json', 'w') { |f| f.write JSON.dump LOGS }

  class << self
    alias :old_run_one_method :run_one_method

    def run_one_method klass, method_name, reporter
      before = Coverage.peek_result
      old_run_one_method klass, method_name, reporter
      after = Coverage.peek_result
      LOGS << [ klass.name, method_name.to_s, before, after ]

To integrate in Minitest, we need to monkey patch it. I couldn’t figure out a better way to do this than by adding a monkey patch. Anyway, the run_one_method method is the method that will run one test case. We alias off Minitest’s implementation, then add our own. Our implementation grabs the current coverage info, then calls the old implementation which runs the test, then grabs coverage info again. Once we have coverage info and test info, we add that in to the LOGS array. When Minitest is done running, it will execute the block we provided to after_run, where we write out the test and coverage information.

Now the RSpec version:

require 'coverage'
require 'json'
require 'rspec'

LOGS = []

RSpec.configuration.after(:suite) {
  File.open('run_log.json', 'w') { |f| f.write JSON.dump LOGS }

RSpec.configuration.around(:example) do |example|
  before = Coverage.peek_result
  after = Coverage.peek_result
  LOGS << [ example.full_description, before, after ]

There’s really not much difference between the two. The main changes in the RSpec version are that I don’t have to monkey patch anything, and we record example.full_description rather than the class and method name.

Now that we’ve got this code, we can run the whole suite and collect coverage information that is split by test. We can figure out what code each test executes. Next we need to figure out what code changed.

What code changed?

This example is only going to work with git repositories. To figure out what code changed, we’ll be using the rugged gem. The rugged gem wraps up libgit2 and gives you access to information about git repositories. With it, we can figure out what files and lines were modified.

To keep these examples short, we’ll take a very naive approach and just say that we’re only interested in lines that have been added or deleted. If the lines were added or deleted, we want to run that same line.

require 'rugged'
require 'set'

repo = Rugged::Repository.new '.'
lines_to_run = Set.new

repo.index.diff.each_patch { |patch|
  file = patch.delta.old_file[:path]

  patch.each_hunk { |hunk|
    hunk.each_line { |line|
      case line.line_origin
      when :addition
        lines_to_run << [file, line.new_lineno]
      when :deletion
        lines_to_run << [file, line.old_lineno]
      when :context
        # do nothing

This code opens the git repository, gets a diff from the index, and iterates over each patch. For each patch, it looks at each hunk, and each line of the hunk. If the line was an addition or deletion, we store the file name and line number of the change.

So if the output of git diff looks like this:

diff --git a/lib/my_thing.rb b/lib/my_thing.rb
index 806deff..eb057b9 100644
--- a/lib/my_thing.rb
+++ b/lib/my_thing.rb
@@ -4,7 +4,7 @@ class Whatever
   def bar
-    "bar #{@foo}"
+    raise
   def baz

The lines_to_run set will contain one array like this:

#<Set: {["lib/my_thing.rb", 7]}>

Now that we have the lines to execute, lets map those back to tests.

Mapping back to tests

For each test, we recorded two pieces of coverage information. We recorded the coverage before the test ran, then we recorded the coverage after the test ran. We need to compute the difference between the two in order to figure out what lines the test ran.

The following function computes the difference:

def diff before, after
  after.each_with_object({}) do |(file_name,line_cov), res|
    before_line_cov = before[file_name]

    # skip arrays that are exactly the same
    next if before_line_cov == line_cov

    # subtract the old coverage from the new coverage
    cov = line_cov.zip(before_line_cov).map do |line_after, line_before|
      if line_after
        line_after - line_before

    # add the "diffed" coverage to the hash
    res[file_name] = cov

Coverage information is returned from the coverage tool as a hash, where the keys are file names, and the value is an array where each index of the array represents one line in the source. The value at that index represents how many times that line has been run.

The above function iterates through the before and after hashes, subtracting the “before” coverage from the “after” coverage and produces a hash where the keys are file names and the value is the coverage information just for that test.

Now that we can compute per-test coverage information, we need to map the code changes back to test methods. The modified file and line numbers are the key. We need to be able to look up tests by file name and line number.

cov_map = Hash.new { |h, file|
  h[file] = Hash.new { |i, line|
    i[line] = []

File.open('run_log.json') do |f|
  # Read in the coverage info
  JSON.parse(f.read).each do |desc, before, after|

    # calculate the per test coverage
    delta = diff before, after

    delta.each_pair do |file, lines|
      file_map = cov_map[file]

      lines.each_with_index do |val, i|
        # skip lines that weren't executed
        next unless val && val > 0

        # add the test name to the map. Multiple tests can execute the same
        # line, so we need to use an array.  Arrays are 0 indexed, but `rugged`
        # gives line numbers starting at 1, so we need to add one to `i`.
        file_map[i + 1] << desc

The above snippet reads in the coverage JSON, calculates the coverage for each test, then inserts the test in to cov_map, where the file name and line number are the key, and the value is a list of tests. More than one test can run any particular line of source, so we need to keep a list of tests for each file name and line number.

Now we need to combine the information from Rugged, and the information from our coverage map to produce a list of tests to run:

lines_to_run.each do |file, line|
  cov_map[File.expand_path(file)][line].each do |desc|
    puts desc

lines_to_run came from Rugged, and of course cov_map came from our coverage information. All this snippet does is iterate over the lines of code that have changed, and looks up tests that will execute that particular line, then prints it out.

I guess this is pretty anti-climactic, but now you are able to predict which tests will fail given a change to your codebase.

All of this code is available here. If you won’t actually want to type stuff, you can see a video of me predicting which tests will fail here.


I think that failure prediction and regression test selection can be a great tool for people like me that work on legacy code bases. Obviously, the value of a tool like this diminishes as your tests get faster, but if you have to work with slow tests, then I think this is a good way to save time.

Also, please take this idea. Please please please take this idea. I want this tool to exist as a gem, but I don’t have time to maintain it. So if you want to take this idea and package it up, then please do it!

Other ideas

Now that we can collect coverage information incrementally, I was thinking it would be nice if we made a Rack handler that recorded the coverage information on a per-request basis. Then we could do interesting things like cross reference code execution in the real world with what our tests actually execute.

I hope you enjoyed this article. Please have a good day!!

read more »

Weird stuff with hashes

Ok, so this isn’t really weird stuff, I don’t think. But maybe people don’t know about it so I thought I would post.

Hashes dup string keys

When you assign a string key to a hash, the hash copies the string and freezes it. Here is an example:

>> x = 'string'
=> "string"
>> y = {}
=> {}
>> y[x] = :value
=> :value
>> { x => y.keys.first }
=> {"string"=>"string"}
>> { x.object_id => y.keys.first.object_id }
=> {70157126548460=>70157126526760} # object ids are different
>> { x.frozen? => y.keys.first.frozen? }
=> {false=>true} # frozen value is different

You can prevent the string from being duped by freezing the string before putting it in the hash:

>> x = 'string'
=> "string"
>> x.freeze
=> "string"
>> y = {}
=> {}
>> y[x] = :value
=> :value
>> { x.object_id => y.keys.first.object_id }
=> {70157126414020=>70157126414020} # note that both object ids are the same

Why is this important?

I’ve been working with @eileencodes to improve the performance of integration tests in Rails. It turns out that the tests are spending a significant amount of time in the garbage collector. We started tracking down allocations, and found that setting values in the header hash was creating many strings.

To partially fix this, we added a freeze to the string before inserting it in to the header hash. This cut the allocations down. The code used to allocate one string for the “downcased” version, then another when the downcased version was inserted in to the hash.

Setting strings as hash keys may be a source of unnecessary object allocation in your application. Now, don’t go freezing all the strings. Make sure to measure where object allocations are happening in your app first (a subject that I’ll cover in a later blog post), then freeze strings where appropriate.

Why does the Hash dup strings?

I think there are a few reasons.

One, if you mutate the string after adding it to the hash, it would be surprising (to me anyway) if the key for the hash was mutated too. For example:

>> x = 'string'
=> "string"
>> y = {}
=> {}
>> y[x] = :value
=> :value
>> x.sub!(/r/, '')
=> "sting"
>> y.key? x
=> false
>> { y.keys => x }
=> {["string"]=>"sting"}

I think it would be pretty surprising if, after mutating the object in place, it would still be part of the hash.

I think the other more technical reason is that mutating the string changes the hash value of the string, and means your hash would need to be rehashed in order to find the same key:

>> x.hash
=> 4078764570319932244
>> x.sub!(/r/, '')
=> "sting"
>> x.hash
=> -1518905602476999392

Note that the hash value changed.

Hash value, what does that have to do with anything?

If you change the hash value of a hash key, then the key will be in the wrong place in the hash. I think this might be more clear with a code example.

You may know about the two methods you need to implement for creating a custom hash key: hash indicates where in the hash the object will be stored, and eql? is used in the case there is a hash collision.

Lets make an object that allows us to mutate the hash value, and see how it behaves after inserting it in a hash:

class Foo
  attr_accessor :hash

x = Foo.new
x.hash = 10

hash = {}
hash[x] = :hello

p hash.key?(x)          # => true
p hash.keys.include?(x) # => true

x.hash = 11 # change the hash value

p hash.key?(x)          # => false
p hash.keys.include?(x) # => true


p hash.key?(x)          # => true
p hash.keys.include?(x) # => true

First we see that x is a key to the hash, and is in the list of keys. After we mutate the hash value, Hash#key? returns false, yet Array#include? returns true when we check for the object in the list of keys.

The reason Hash#key? returns false is because it’s using the value of hash to find the object in the hash buckets. When we first inserted the object in the hash, it went in to the “10” bucket. But now that we’ve changed the value to “11”, the hash will look for the object in the “11” bucket, which is the wrong place!

To fix this, we call the rehash method on the hash. The rehash method will redistribute the keys. Since the object in the “10” bucket is now supposed to be in the “11” bucket, it will move the object to the right place, and Hash#key? will work correctly.

Wouldn’t it be annoying if you had to do the same song and dance as this if you mutated your strings? I think this is the main reason why hashes dup and freeze string keys.

What is the lesson?

Don’t mutate the hash value on hash keys! You’ll have a bad time.

Anyway, I don’t think this is weird or anything, it’s just something that we don’t have to deal with on a day to day basis. Maybe I should have titled the post “Uncommon things to do with hashes”.

Thanks for reading!!! <3<3<3<3<3

Performance P.S. (is that a P.P.S.?)

An interesting thing is that Hash#dup will rehash the newly created hash, where Hash[] will not:

class Foo
  attr_accessor :hash

x = Foo.new
x.hash = 10

hash = {}
hash[x] = :hello

p hash.key?(x)          # => true

x.hash = 11 # change the hash value

p hash.key?(x)          # => false
p hash.dup.key?(x)      # => true
p Hash[hash].key?(x)    # => false

This means that duping a hash with Hash[] can end up being faster than using Hash#dup. Here is a benchmark to demonstrate:

require 'benchmark/ips'

hash = Hash[*('a'..'z').to_a]
Benchmark.ips do |x|
  x.report("Hash#dup") do

  x.report("Hash[]") do

Results on my machine:

Calculating -------------------------------------
            Hash#dup     7.705k i/100ms
              Hash[]    15.978k i/100ms
            Hash#dup     93.648k (± 4.9%) i/s -    470.005k
              Hash[]    230.497k (±11.2%) i/s -      1.150M

Does this mean that you should switch to Hash[]? Only if your benchmarks can prove that it’s a bottleneck. Please please please don’t change all of your code because this shows it’s faster. Make sure to measure your app performance first. But you already knew that, which is why you’re reading the PS section. ;-)

read more »

My experience with Minitest and RSpec

I’ve been using RSpec in earnest for the past 6 months now, so I thought it’s time to write a blurrrgh poast comparing RSpec with Minitest. I’ve used Minitest for years, and RSpec only for 6 months, so please keep that in mind when reading!

PLEASE REMEMBER, I don’t care what test framework you use, as long as you are testing your code. This is a post just about my experience with these two frameworks. In other words, this is an opinion piece.


I really believe that all test frameworks are essentially the same. No test framework can do something that’s impossible for another to do. It’s just code. So what differentiates test frameworks? I would say the main difference would be user interface. So I’m really going to compare user interfaces between these frameworks.

Things I like about RSpec.

By far and away, my favorite thing about RSpec is that if there is a failure, it prints how to run just that one test at the bottom of the output. I can easily copy and paste that line and run just the one test. For people who don’t know, here is an example:

describe "something" do
  it "works" do
    expect(10).to equal(11)

  it "really works" do
    expect(11).to equal(11)

When you run this, the output looks like this:

[aaron@TC tlm.com (master)]$ rspec code/fail_spec.rb 


  1) something works
     Failure/Error: expect(10).to eq(11)
       expected: 11
            got: 10
       (compared using ==)
     # ./code/fail_spec.rb:3:in `block (2 levels) in <top (required)>'

Finished in 0.00504 seconds (files took 0.20501 seconds to load)
2 examples, 1 failure

Failed examples:

rspec ./code/fail_spec.rb:2 # something works
[aaron@TC tlm.com (master)]$ 

All you have to do to rerun just the one failing spec is to copy and paste that one line like this:

rerun demo

I don’t have to think or type, I just “copy, paste, run”. I like to think of this as a “learn as you play” feature.

The other thing I like about RSpec is that it ships with a colored output formatter, so if you do rspec --color, you’ll get colored output. I’ve used Minitest for years without colored output, but I’ve come to really like the colored output from RSpec. It helps me quickly see the most important parts of the test output, the assertion failure, and stack traces (though I highly suspect that the RSpec core team all use black backgrounds in their terminals because sometimes the colors aren’t so nice on my white terminal).

Things I like about Minitest

The main thing that I like about Minitest is that a Minitest test is just a Ruby class. Here is an example test similar to the RSpec sample I showed earlier:

require 'minitest/autorun'

class Something < Minitest::Test
  def test_works
    assert_equal 11, 10

  def test_really_works
    assert_equal 11, 11

I like this because I know exactly where the test_works method is defined. It’s no different than any other class in Ruby, so I don’t have to learn anything new. I like it because I am a heavy user of CTags, so I can easily jump to test methods or classes inside my editor. Another thing that’s nice about using regular Ruby classes is that when I need to refactor my tests, I just use my normal refactoring skills: I extract methods, classes, modules, change to inheritance, etc. I use the same refactoring techniques on my test files as I do on my library files. I think that is where Minitest really shines.

Things I dislike about RSpec

RSpec is a DSL for writing tests. I think this is probably RSpec’s largest weakness. I am a programmer, I can read code, so I don’t particularly care whether or not my test code “reads like English”. I don’t understand the value of the DSL, especially when I have a nearly 4000 line specfile that I’d like to refactor. How do I go about refactoring? I can extract methods:

describe "something" do
  def helper

  it "works" do
    expect(10).to eq(helper)

  it "really works" do
    expect(11).to eq(helper)

But where is the helper method defined? What’s its visibility? Can I put it in a module? Can I use inheritance? Who can call it? Can I call super from the extracted method? If so, where does super go? These are not the types of questions I want to be pondering when I have 3000 test failures and really long spec files to read through. My mind spins off in to thoughts like “I wonder how RSpec works?”, and “I wonder what my cat is doing?”

From what I can gather, calling describe in RSpec essentially defines a class. But if that’s the case, why not just use Ruby classes? Then I don’t need to guess about method visibility, modules, inheritance, etc. Not to mention, the library code that I’m working with is just using Ruby classes. Why do I need to switch to some other language for testing?

Nesting a describe seems to perform inheritance, but only for certain things like before blocks. If you run this code:

describe "something" do
  before do
    puts "hi!"

  it "works" do
    expect(10).to eq(helper)

  it "really works" do
    expect(11).to eq(helper)

  context "another thing" do
    before do
      puts "hello!"

    it "really really works" do
      expect(11).to eq(helper)

  def helper

You’ll see “hi!” printed 3 times, once for each it, and “hello!” printed once. For the nested context, I would like to print “hello!” before it prints “hi!”. In a normal inheritance situation, I would change the call to super. I am not sure how to do what I just described without learning a new thing (I think I would need let but I don’t know). For now, I’m coping with this by refactoring to classes outside of the specs themselves, then calling the objects from the specs. Essentially “composition” in my tests.

Another thing that bugs me is that if I do try to refactor my RSpec tests, and I change any lines, then the line that I copy and pasted to run just the one test won’t work anymore. I have to figure out the new line. I can use the name of the test by specifying the -e flag on RSpec, but then I lose the “copy, paste, run” feature that I love so much.

Anyway, I will stop picking on RSpec now. To me, RSpec seems to have some sort of identity crisis and is caught somewhere between “a language for writing tests” and “just Ruby”. That is RSpec’s biggest weakness.

Things I dislike about Minitest

If RSpec does one thing better than Minitest, I would say that the command line UI easily outshines Minitest. Here is one example.

I can run one test at a time like so:

$ ruby code/fail_test.rb -n Something#test_works

This will run the test_works method on the Something class. The problem for me is that it’s not clear to do that when you see the failing test output. If we run the whole suite, the output looks like this:

[aaron@TC tlm.com (master)]$ ruby code/fail_test.rb 
Run options: --seed 5990

# Running:


Finished in 0.002047s, 977.0396 runs/s, 977.0396 assertions/s.

  1) Failure:
Something#test_works [code/fail_test.rb:5]:
Expected: 11
  Actual: 10

2 runs, 2 assertions, 1 failures, 0 errors, 0 skips
[aaron@TC tlm.com (master)]$

Looking at this, how would you know how to run just that one test? With RSpec, it’s no problem. I was able to learn how to run just one test the first time I ever used RSpec. I didn’t read any docs, just ran the tests, and the info was right there. Coming from RSpec, you might need to think you could do this:

$ ruby code/fail_test.rb:5

But of course that doesn’t work because you’re just using Ruby on the command line.

Here is a short video of me running the single test:


The disadvantages of this over RSpec’s output is I have to type a few characters and know a commandline interface in advance. You might think “it’s just a few characters”, but every time I have to type something is a potential for a mistake that I’ll have to correct, which takes even more time. I really think the advantages of “copy, paste, run” can’t be understated. (Yes, I know there are tools like autotest for both Minitest and RSpec that will run tests for you, but sometimes I can’t / don’t use them)

The other thing I wish Minitest would do is to ship with a color formatter like RSpec does. I am used to the color output from RSpec, and I don’t really want to go back. It really helps me focus on what’s important about my test failures.


Both of these frameworks have things I like and dislike. Being able to reuse my debugging and refactoring skills in my tests is pretty important to me, especially since I deal with lots of legacy code. This is why I’ll stick with Minitest in my personal projects. However, I am sure I will get better at refactoring and debugging in RSpec tests, and when I get better at it, I will share what I learn.

Now, do I care which one you use? No. As long as you test your code, I am happy. A professional developer should be able to work in either one of these because they essentially do the same thing: test your code.

read more »

YAGNI methods slow us down

TL;DR: OutputBuffer subclasses SafeBuffer which forces us to do runtime checks that are probably unnecessary

I made a post about YAGNI methods hurting you where I said I would provide two examples, but then I got tired of writing the article so I just did one example. Here is the other example! The previous example demonstrated a memory leak that was introduced because the “footprint” (the number of methods implemented on the object) was too large.

This example will show how these YAGNI methods are impacting performance of your Rails application, and we’ll talk about how to fix it.

This problem is in ActionView::OutputBuffer which inherits from ActiveSupport::SafeBuffer, which inherits from Ruby’s String. Let’s talk about the behavior ActiveSupport::SafeBuffer first, then we’ll see how it impacts to the performance of ActionView::OutputBuffer.


This class is the class that’s used to mark a string as being “html safe”. It’s how Rails detects whether a string has been html escaped or not. In Rails, a normal Ruby string is considered to be not “html safe”. For example:

>> x = "foo"
=> "foo"
>> x.class
=> String
>> x.html_safe?
=> false

If we call html_safe on the string, it returns an instance of ActiveSupport::SafeBuffer that is “tagged” as html safe:

>> x = "foo"
=> "foo"
>> y = x.html_safe
=> "foo"
>> y.class
=> ActiveSupport::SafeBuffer
>> y.html_safe?
=> true

Whenever we html escape a string in Rails, it returns a safe buffer:

>> x = "<html>"
=> "<html>"
>> x.html_safe?
=> false
>> y = ERB::Util.html_escape x
=> "&lt;html&gt;"
>> y.class
=> ActiveSupport::SafeBuffer
>> y.html_safe?
=> true

Now, using the html_safe? predicate, we can easily tell the difference between strings that have been tagged as “html safe” and strings that haven’t been tagged as “html safe” (side note: just like encodings, tagging something does not mean that it actually is correct. We can tag things as “html safe” without them actually being html safe).

We can also concatenate unsafe strings to a safe string, and it will be automatically escaped:

>> x = ERB::Util.html_escape "<html>"
=> "&lt;html&gt;"
>> x.concat "<blink>"
=> "&lt;html&gt;&lt;blink&gt;"
>> x.html_safe?
=> true

Finally, we can concatenate safe strings and they will not be double escaped:

>> x = ERB::Util.html_escape "<html>"
=> "&lt;html&gt;"
>> x.concat ERB::Util.html_escape "<blink>"
=> "&lt;html&gt;&lt;blink&gt;"
>> x.html_safe?
=> true

Infecting a SafeBuffer

So far, the html_safe? predicate is a 1:1 relationship with the class. Meaning that if the class of the string is ActiveSupport::SafeBuffer, then html_safe? would return true, and if the class was String, html_safe? would return false.

Unfortunately it is not a 1:1 relationship. We can mutate a SafeBuffer, making it unsafe again. For example:

>> x = ERB::Util.html_escape "<html>"
=> "&lt;html&gt;"
>> dangerous = "&gt;<script>"
=> "&gt;<script>"
>> x.gsub!(/&gt;/, dangerous)
=> "&lt;html&gt;<script>"
>> x
=> "&lt;html&gt;<script>"
>> x.class
=> ActiveSupport::SafeBuffer
>> x.html_safe?
=> false

You can see that the string in dangerous has been embedded in to the SafeBuffer without being escaped. The class of x is still SafeBuffer, but calling html_safe? will return false. As you can see, the return value of html_safe? is not a 1:1 relationship with the class.

Concatenating to an infected SafeBuffer

Our infected SafeBuffer still supports concatenation:

>> x
=> "&lt;html&gt;<script>"
>> x.class
=> ActiveSupport::SafeBuffer
>> x.html_safe?
=> false
>> x.concat "<blink>"
=> "&lt;html&gt;<script><blink>"

But you can see that the “<blink>” string was not escaped. This means that the concat behavior on a SafeBuffer changes depending on the value of html_safe?. If you look at the implementation of concat, along with it’s helper method, you can indeed see that this is true.

What is OutputBuffer?

OutputBuffer is a buffer that is fed to Rack and then output to the client over the wire. Templates instantiate an OutputBuffer and concatenate computed output to the buffer.

Impact of SafeBuffer on OutputBuffer

How does SafeBuffer impact OutputBuffer? Let’s take a look at a compiled ERB template to find out. Here is an ERB template after it’s been compiled to Ruby:

@output_buffer = output_buffer || ActionView::OutputBuffer.new
@output_buffer.safe_append='      <tr>
@output_buffer.append=( book.name )

The output isn’t pretty, but you can see that we essentially call two methods on the output buffer: append=, and safe_append= (why we have an = is a mystery for the ages). append= is used when there are dynamic values like <%= link_to ... %>, and safe_append= is used for string literals in your template.

Let’s look at the implementation of safe_append=:

def safe_concat(value)
  return self if value.nil?
alias :safe_append= :safe_concat

First, notice that value is never nil. The ERB compiler ensures that fact, so the value.nil? check is useless. However, this blurrrggghhh post is about how YAGNI methods are hurting us. We do need to concatenate, so you are gonna need this.

OutputBuffer is a subclass of SafeBuffer, and safe_concat supers in to SafeBuffer. Let’s look at the implementation of safe_concat on SafeBuffer:

def safe_concat(value)
  raise SafeConcatError unless html_safe?

safe_concat raises an exception if the current object is not html_safe?. Every time our ERB template concatenates a string, it checks whether or not the current object is “html safe”.

As we saw earlier, the only way to make an instance of a SafeBuffer not html safe is by calling an unsafe method on that instance.

YAGNI method pain

Every time we concatenate to an OutputBuffer object, we’re forced to check a flag. This flag is directly related to people calling gsub! on the OutputBuffer. This bothers me because, as you can imagine, we concatenate on to the OutputBuffer extremely frequently.

I’m betting that it’s extremely rare for someone to call gsub! on an OutputBuffer. I’m also willing to bet that people calling gsub! on an OutputBuffer would be willing to call to_s on it before doing their mutation, so why do we “support” this?

Our OutputBuffer is punishing most people for a feature that is extremely rare. This is thanks to the features we got “for free” from our superclass.

Secondly, if you look in the same file that defines OutputBuffer, you’ll see a class called StreamingBuffer. This class is meant to be a replacement for OutputBuffer, but it streams to the client. Notice that this class does not inherit from SafeBuffer. Further evidence that gsub! on OutputBuffer is a YAGNI method.

How do we fix this?

I think we can fix this by disconnecting OutputBuffer from it’s superclass. I suspect that the methods people actually use on OutputBuffer are extremely few. I’d also say that this class should be completely private. In other words, we (the Rails team) should design our API such that people never actually touch instances of the OutputBuffer.

If we disconnect OutputBuffer from it’s superclass, I think we can even do some tricks with the ERB compiler to attain faster output.

read more »

YAGNI methods are killing me

TL;DR: Inheriting from Hash will bite you. ¯\_(ツ)_/¯

This is Yet Another Post about preferring composition over inheritance, but I will try to drive it home with Real World Examples™. I’m not saying “don’t use inheritance”, I am saying “use inheritance conservatively, and when it is appropriate” (I know it’s not very controversial). I think I can confidently say “don’t inherit from String, Hash, or Array in Ruby”, and in this post we’ll look at two concrete examples about why you shouldn’t inherit from those classes.

YAGNI methods

YAGNI methods are baggage that your subclasses are carrying around. They’re methods that you may never use, but you have to deal with them because your parent class implemented them. The thing about these methods is that when you sit at your keyboard, innocently pecking out the characters class Parameters < Hash, you don’t realize that these YAGNI methods are there waiting in the shadows getting ready to ruin your day.

Let’s look at a couple examples of how these YAGNI methods make our lives harder.

Memory Leaks

I showed this off during my Keynote at RailsConf, but we’ll dive in a little more here. As of e2a97adb, this code leaks memory:

require 'action_controller'

params = ActionController::Parameters.new({

loop do
  params.delete :foo
  params[:foo] = [Object.new]

If you run this code, you’ll see that the process’s memory grows unbounded.

ActionController::Parameters is used for the parameters hash in your controller. When you do params[:id], params returns an instance of ActionController::Parameters. This class inherits from ActiveSupport::HashWithIndifferentAccess, which in turn inherits from Ruby’s Hash.

Why does this leak?

There are two methods involved in our leak. One method is the delete method, and ActionController::Parameters does not implement that method. The other method is the [] method, so let’s look there.

If you read the implementation of the square brace method, you’ll see it calls convert_hashes_to_parameters, which calls convert_value_to_parameters.

Here is the convert_value_to_parameters method:

def convert_value_to_parameters(value)
  if value.is_a?(Array) && !converted_arrays.member?(value)
    converted = value.map { |_| convert_value_to_parameters(_) }
    converted_arrays << converted
  elsif value.is_a?(Parameters) || !value.is_a?(Hash)

This method seems to do some sort of conversions on Array, and appends the conversion to the converted_arrays object. Each time we iterate through the loop, we delete a key, but that value is never deleted from the converted_arrays object. Each time we access the Array value, it gets “converted”, and that converted array is added to the converted_arrays object. So, the converted_arrays object grows unbounded.

Why do we need to convert stuff? Why do we need to mutate a thing? This method leaves me with more questions than I have time to deal with here, but presumably the function is necessary.

How do we fix it?

Well, we need to implement the delete method. Whenever the delete method is called, we need to remove any “converted arrays” from the converted_arrays list.

This solution may work for the delete method, but what about the delete_if method? What about the merge! method? How about keep_if? How about reject!? How about select!? The list goes on, and soon we feel like we’re playing whack-a-mole with method implementations.

Rather than “How do we fix it?” I think a better question is “Do we need it?”. For these mutation methods, I think the answer is “no”. The author of convert_value_to_parameters probably didn’t think about these mutation methods, and I’m willing to bet that very few people actually mutate their own params object. I’ll also bet that of the people who do mutate their params object, 100% of those people would be OK with calling to_hash on params before making mutations.

Ok! Lets remove the method!

Whoa! Not so fast there, friend. We can’t just remove the method. We need to add deprecation warnings so that we don’t break applications when people upgrade. That means we need to add a warning on merge!, and keep_if, and delete_if, and select!, and reject!, etc.

Feels like we’re playing method whack-a-mole again, doesn’t it?

These methods are what I think of as YAGNI methods. Methods that we probably don’t need, but we got them for “free” through inheritance. Remember that even in life, inheriting something isn’t always a good thing.

So how do we really fix it?


We should probably switch this to use composition, but it will be a breaking change so it must wait until Rails 5.

The End

I said I was going to write about two issues with YAGNI methods, but I don’t really want to right now. I will write more later. However, the next installment will be about ActionView::OutputBuffer which is (eventually) a subclass of String. We will talk about how the YAGNI methods on OutputBuffer are hurting performance of your Rails application.

Remember: stay conservative with your API. It’s easier to add a new method than it is to take one away.

read more »

Webcam photos with Ruby

Let’s do something fun! In this post we’ll take a photo using your webcam in Ruby.

NOTE: This only works on OS X

I haven’t tried making it work on other operating systems. Not that I don’t like other operating systems, I just haven’t made it work. :-)

Installing the Gem

We’ll use the av_capture gem. It wraps the AVCapture framework on OS X. To install it, just do:

$ gem install av_capture

Using the Gem

I’ll paste the code here first, then explain it:

require 'av_capture'

# Create a recording session
session = AVCapture::Session.new

# Find the first video capable device
dev = AVCapture.devices.find(&:video?)

# Output the camera's name
$stderr.puts dev.name

# Connect the camera to the recording session
session.run_with(dev) do |connection|

  # Capture an image and write it to $stdout
  $stdout.write connection.capture

First the program creates a new capture session. OS X can capture from many multimedia devices, and we hook them together through a session. Next, it grabs the first device attached to the machine that has video capability. If your machine has multiple cameras, you may want to adjust this code. After it outputs the name of the camera, we tell the session to start and use that device.

The camera can only be used while the session is open, inside the block provided to run_with. Inside the block, we ask the connection to capture an image, then write the image to $stdout.

Running the code

I’ve saved the program in a file called thing.rb. If you run the program like this:

$ ruby thing.rb | open -f -a /Applications/Preview.app

it should open Preview.app with an image captured from the camera.

Taking Photos Interactively

Let’s make this program a little more interactive:

require 'av_capture'
require 'io/console'

session = AVCapture::Session.new
dev = AVCapture.devices.find(&:video?)

session.run_with(dev) do |connection|
  loop do
    case $stdin.getch
    when 'q' then break # quit when you hit 'q'
      IO.popen("open -g -f -a /Applications/Preview.app", 'w') do |f|
        f.write connection.capture

This program will just sit there until you press a button. Press ‘q’ to quit, any other button to take a photo and display it in Preview.app. Requiring io/console lets us read one character from $stdin as soon as possible, and the call to IO.popen lets us write the data to Preview.app.

A Photo Server using DRb

It takes a little time for the camera to turn on before the program can take a photo. This causes a little lag time when we want to take a photo. In the spirit of over-engineering things, lets create a photo server using DRb. The server will keep the camera on and ready to take photos. The client will ask the server for photos.

Server code

Here is our server code:

require 'av_capture'
require 'drb'

class PhotoServer
  attr_reader :photo_request, :photo_response

  def initialize
    @photo_request  = Queue.new
    @photo_response = Queue.new
    @mutex          = Mutex.new

  def take_photo
    @mutex.synchronize do
      photo_request << "x"

server = PhotoServer.new

Thread.new do
  session = AVCapture::Session.new
  dev = AVCapture.devices.find(&:video?)

  session.run_with(dev) do |connection|
    while server.photo_request.pop
      server.photo_response.push connection.capture

URI = "druby://localhost:8787"
DRb.start_service URI, server

The PhotoServer object has a request queue and a response queue. When a client asks to take a photo by calling the take_photo method, it writes a request to the queue, then waits for a photo to be pushed on to the response queue.

The AVCapture session’s run block waits for a request to appear on the photo_request queue. When it gets a request on the queue, it takes a photo and writes the photo to the response queue.

At the bottom of the file, we connect the PhotoServer object to DRb on port 8787, and join the DRb server thread.

Client Code

Here is our client code:

require 'drb'

SERVER_URI = "druby://localhost:8787"

photoserver = DRbObject.new_with_uri SERVER_URI
print photoserver.take_photo

The client code connects to the DRb server on port 8787, requests a photo, then writes the photo to $stdout.

Running the code

In one terminal, run the server code like this:

$ ruby server.rb

Then in another terminal, run the client code like this:

$ ruby client.rb | open -f -a /Applications/Preview.app

You should have a photo show up in Preview.app. You can kill the server program by doing Ctrl-C.

Speed comparison

Just for fun, let’s compare the speed of the first program to the speed of the client program just using time. Here is the first program:

$ time ruby thing.rb > /dev/null
FaceTime HD Camera (Built-in)

real	0m3.217s
user	0m0.151s
sys	0m0.069s

Here is the client program:

$ time ruby client.rb > /dev/null

real	0m0.183s
user	0m0.070s
sys	0m0.038s

The first program takes about 3 seconds to take a photo where the second “client” program only takes 200ms or so. The reason the second program is much faster is because the server keeps the camera “hot”. Most of our time is spent getting the camera ready rather than taking photos.

Weird photos

Here are some weird photos that I made while I was writing this:

one two three

Happy Wednesday! <3<3<3<3

read more »

AdequateRecord Pro™: Like ActiveRecord, but more adequate

TL;DR: AdequateRecord is a set of patches that adds cache stuff to make ActiveRecord 2x faster

I’ve been working on speeding up Active Record, and I’d like to share what I’ve been working on! First, here is a graph:


This graph shows the number of times you can call Model.find(id) and Model.find_by_name(name) per second on each stable branch of Rails. Since it is “iterations per second”, a higher value is better. I tried running this benchmark with Rails 1.15.6, but it doesn’t work on Ruby 2.1.

Here is the benchmark code I used:

require 'active_support'
require 'active_record'

p ActiveRecord::VERSION::STRING

ActiveRecord::Base.establish_connection adapter: 'sqlite3', database: ':memory:'
ActiveRecord::Base.connection.instance_eval do
  create_table(:people) { |t| t.string :name }

class Person < ActiveRecord::Base; end

person = Person.create! name: 'Aaron'

id   = person.id
name = person.name

Benchmark.ips do |x|
  x.report('find')         { Person.find id }
  x.report('find_by_name') { Person.find_by_name name }

Now let’s talk about how I made these performance improvements.

What is AdequateRecord Pro™?

AdequateRecord Pro™ is a fork of ActiveRecord with some performance enhancements. In this post, I want to talk about how we achieved high performance in this branch. I hope you find these speed improvements to be “adequate”.

Group discounts for AdequateRecord Pro™ are available depending on the number of seats you wish to purchase.

How Does ActiveRecord Work?

ActiveRecord constructs SQL queries after doing a few transformations. Here’s an overview of the transformations:


The first transformation comes from your application code. When you do something like this in your application:


Active Record creates an instance of an ActiveRecord::Relation that contains the information that you passed to where, or order, or whatever you called. As soon as you call a method that turns this Relation instance in to an array, Active Record does a transformation on the relation objects. It turns the relation objects in to ARel objects which represent the SQL query AST. Finally, it converts the AST to an actually SQL string and passes that string to the database.

These same transformations happen when you run something like Post.find(id), or Post.find_by_name(name).

Separating Static Data

Let’s consider this statement:


In previous versions of Rails, when this code was executed, if you watched your log files, you would see something like this go by:

SELECT * FROM posts WHERE id = 10
SELECT * FROM posts WHERE id = 12
SELECT * FROM posts WHERE id = 22
SELECT * FROM posts WHERE id = 33

In later versions of Rails, you would see log messages that looked something like this:

SELECT * FROM posts WHERE id = ? [id, 10]
SELECT * FROM posts WHERE id = ? [id, 12]
SELECT * FROM posts WHERE id = ? [id, 22]
SELECT * FROM posts WHERE id = ? [id, 33]

This is because we started separating the dynamic parts of the SQL statement from the static parts of the SQL statement. In the first log file, the SQL statement changed on every call. In the second log file, you see the SQL statement never changes.

Now, the problem is that even though the SQL statement never changes, Active Record still performs all the translations we discussed above. In order to gain speed, what do we do when a known input always produces the same output? Cache the computation.

Keeping the static data separated from the dynamic data allows AdequateRecord to cache the static data computations. What’s even more cool is that even databases that don’t support prepared statements will see an improvement.

Supported Forms

Not every call can benefit from this caching. Right now the only forms that are supported look like this:

Post.find_by(name: name)

This is because calculating a cache key for these calls is extremely easy. We know these statements don’t do any joins, have any “OR” clauses, etc. Both of these statements indicate the table to query, the columns to select, and the where clauses right in the Ruby code.

This isn’t to say that queries like this:


can’t benefit from the same techniques. In those cases we just need to be smarter about calculating our cache keys. Also, this type of query will never be able to match speeds with the find_by_XXX form because the find_by_XXX form can completely skip creating the ActiveRecord::Relation objects. The “finder” form is able to skip the translation process completely.

Using the “chained where” form will always create the relation objects, and we would have to calculate our cache key from those. In the “chained where” form, we could possibly skip the “relation -> AST” and “AST -> SQL statement” translations, but you still have to pay the price of allocating ActiveRecord::Relation objects.

When can I use this?

You can try the code now by using the adequaterecord branch on GitHub. I think we will merge this code to the master branch after Rails 4.1 has been released.

What’s next?

Before merging this to master, I’d like to do this:

  1. The current incarnation of AdequateRecord needs to be refactored a bit. I have finished the “red” and “green” phases, and now it’s time for the “refactor” step.
  2. The cache should probably be an LRU. Right now, it just caches all of the things, when we should probably be smarter about cache expiry. The cache should be bounded by number of tables and combination of columns, but that may get too large.

After merging to master I’d like to start exploring how we can integrate this cache to the “chained where” form.

On A Personal Note

Feel free to quit reading now. :-)

The truth is, I’ve been yak shaving on this performance improvement for years. I knew it was possible in theory, but the code was too complex. Finally I’ve payed off enough technical debt to the point that I was able to make this improvement a reality. Working on this code was at times extremely depressing. Paying technical debt is really not fun, but at least it is very challenging. Some time I will blurrrgh about it, but not today!

Thanks to work (AT&T) for giving me the time to do this. I think we can make the next release of Rails (the release after 4.1) the fastest version ever.

EDIT: I forgot to add that newer Post.find_by(name: name) syntax is supported, so I put it in the examples.

read more »

Me And Facebook Open Academy

TL;DR: I am working with many students to improve Rails and it is fun!

WARNING: This is a non-technical blog post. I usually like to write about tech stuff, but I think this is pertinent so I’m going to write about it!

What is the Open Academy?

Facebook Open Academy is basically a program that gives Computer Science students at different universities the opportunity to work with various projects in the Open Source community. I think there are about 20 Open Source projects involved, and Rails is one of them. This year there are about 250 students participating, and the Rails team has 22 students. This is the second year that the Open Academy has been running, and I participated last year too.

How I got involved

I was contacted by Jay Borenstein who is a professor at Stanford and also works for Facebook. He told me that the idea of the program is that there are many CS students learning theory, but maybe not learning the skills they need to work on older code bases, with remote teams, or with Open Source code bases, and that he wanted to provide students with the opportunity to work closely with people in the Open Source community so that they could get these skills.

This struck a chord with me because I had to learn this stuff on my own, and I would have loved to have this kind of opportunity. For payment, I asked Jay to give me an Honorary Degree from Stanford. He has not gotten back to me on that.

Our Team

I’m not sure how many schools are involved total, but our team of 22 students is from 8 different schools: Sichuan University, University of Helsinki, Cornell University, Harvard, Princeton, University of Waterloo, UC Berkeley, and University of Washington.

I am extremely honored to work with these students, but I have to admit that as a college dropout it is extremely intimidating. The students we’re working with are very bright and have lots of potential. It’s inspiring to work with them, and I am slightly jealous because they are much better than I was at their age. I think this is a good thing because it means that the next generation of programmers will be even better than my generation.

Last year I was the only mentor. This year I was able to trick convince @_matthewd, @pixeltrix, and @bitsweat to help out as mentors.

Our Timeline

We started a couple weeks ago. But this weekend Facebook sponsored travel and lodging for all students, faculty, and mentors, to go to Palo Alto for a hackathon. So this weekend I was able to work with our team in person.

The timeline for this program is rather challenging because each school has a different schedule, so we will be working with all students for a different length of time.

Our Work

For the first few weeks we will have the students work on bug fixes in Rails. This will help us to assess their skill levels so that we can give them longer term projects that are a good fit. Fixing bugs has turned out to be a challenge because most “easy” bugs in Rails have been fixed, so most of the bugs left in the tracker are particularly challenging. I haven’t figured out a solution to this problem yet, but we are managing (I think). So if you find easy bugs in Rails, don’t fix them, but tell my students to fix them. ;-)

Anyway, after a couple weeks we will give them larger projects to work on (and I’ll describe those in a later post).

Improvements for the Rails team

Getting the students up and running with a dev environment was a huge challenge. Some students tried the Rails dev box, but running a VM on their hardware was just too slow, so we ended up with all of the students developing on their native hardware. Of course this was also a pain because some students had issues like having Fink, MacPorts, and homebrew all installed at the same time. For the Agile Web Development with Rails book, Sam Ruby has tests that start with a completely blank Ubuntu VM, and go all the way through getting a Rails app up and running. I think we need to do something like that, but for doing dev work against Rails.

One extremely frustrating example of why we need tests for setting up the dev environment is that in the guides it says that you should execute bundle install --without db. This is fine, except that when you do that, the --without db option is written to .bundle/config and persists between runs of bundler. We (the mentors) didn’t know the students had done that, and were ripping our hair out for an hour or so trying to figure out why ActiveRecord couldn’t find the adapter gems, or why running bundle install wouldn’t install the database gems. I find this to be an extremely frustrating behavior of bundler.

These are some things that I think we can improve.

Improvements for Universities

All of our students can program. Maybe not in Ruby, but they know some other language. They have to pick up Ruby, but I don’t think that’s a big deal. A larger problem is that not all of them knew about git, or about VCS in general. Jay tried to help with this problem by inviting @chacon to give a talk on git. I think this helped, but I really think that git should be part of normal CS curriculum. It’s very difficult for me to imagine writing any amount of code without some kind of VCS. What if you mess up? What if your computer dies? How do you remember what the code used to be? Not using a VCS seems like a huge waste of time. Some of the students were already familiar with git, but some weren’t. I am unclear about the role VCS plays among Universities, but I strongly believe it should be taught in all CS programs.

Maybe I can get people at GitHub in contact with people at different Universities?

Departing thoughts

It blows my mind that we have students from such prestigious Universities, and that these Universities are forward thinking enough to participate in this program. I am extremely honored to work with them. I also have to say thank you to Facebook for sponsoring this work even though Rails has basically nothing to do with Facebook. I also have to say thanks to my work (AT&T) for allowing me to have the job that I do, and also giving me some time to work with these students.

All of our students are part of the Friends of Rails organization on GitHub so you can watch some of our progress there. I will try to make more blurrrrgggghhh posts as we progress.

Post Script

Since I am a College dropout, I have an extreme case of Impostor Syndrome. Working with such prestigious schools made me extremely nervous. Last year I went to dinner with some of the faculty members of University of Helsinki. After a few beers, I was able to muster my courage and said “Before we start this program, I have to admit something. I never graduated college.” Their response: “So?”


read more »

Enabling ttyO1 on BeagleBone

I’m really just making this post because I will eventually forget how to do this, and maybe it will come in handy to others as well.


I’ve got an MSP430 reading temperature and humidity information from an RHT03 sensor. It sends the data over a serial port, and I’d like to read that data on my BeagleBone.

I have P1.1 and P1.2 on the launchpad connected to pin 24 and 26 on the BeagleBone. Pin 24 and 26 map to UART1, so I need to enable UART1 on the BeagleBone so it will show up in /dev.


The BeagleBone is running Angstrom v2012.12, and I believe this is the same version that is on the BeagleBone Black, so it should work there as well:

root@beaglebone:~# uname -a
Linux beaglebone 3.8.13 #1 SMP Tue Jun 18 02:11:09 EDT 2013 armv7l GNU/Linux
root@beaglebone:~# cat /etc/version 
Angstrom v2012.12


To enable the UART, you just need to do this:

root@beaglebone:~# echo BB-UART1 > /sys/devices/bone_capemgr.*/slots

After running that, there should be a message in dmesg about enabling the UART, and /dev/ttyO1 should be available.

Where does BB-UART1 come from?

“BB-UART1” is the name of a device cape, and it seems Angstrom already ships with a bunch of these. If you look under /lib/firmware, there will be a bunch of them. “BB-UART1” came from the file /lib/firmware/BB-UART1-00A0.dts. If you don’t have that file, then echoing to the slots file will not work.

Enabling at boot

I want the tty to be available every time I boot the machine. Angstrom doesn’t use normal System V init scripts, so you have to do something different. You need two files, and a symbolic link.

First I created /usr/local/bin/enable_uart1, and it looks like this:

root@beaglebone:/# cat /usr/local/bin/enable_uart1 

echo BB-UART1 > /sys/devices/bone_capemgr.7/slots


(make sure enable_uart1 is executable).

Then I created /lib/systemd/enable_uart1.service, and it looks like this:

root@beaglebone:/# cat /lib/systemd/enable_uart1.service
Description=Enable UART1
After=syslog.target network.target



Then I created a symbolic link:

root@beaglebone:/# cd /etc/systemd/system/
root@beaglebone:/# ln /lib/systemd/enable_uart1.service enable_uart1.service

Then I loaded and enabled the service:

root@beaglebone:/# systemctl daemon-reload
root@beaglebone:/# systemctl start enable_uart1.service
root@beaglebone:/# systemctl enable enable_uart1.service

After running these commands, /dev/ttyO1 should be available even after rebooting the machine.

(Most of this systemd information came from this blog post)


Next I need to get wifi working, and my BeagleBone will be perfect for real-time monitoring of Sausage Box One.

read more »

One Danger of Freedom Patches

I posted a benchmark on twitter about comparing a DateTime with a string. This is a short blurrrggghhh post about the benchmark and why there is such a performance discrepancy.

Here is the benchmark:

require 'benchmark/ips'
require 'active_support/all' if ENV['AS']
require 'date'

now = DateTime.now

Benchmark.ips do |x|
  x.report("lhs") { now == "foo" }
  x.report("rhs") { "foo" == now }

First we’ll run the benchmark without Active Support, then we’ll run the benchmark with Active Support.

The Benchmarks

Without Active Support

[aaron@higgins rails (master)]$ bundle exec ruby argh.rb 
Calculating -------------------------------------
                 lhs     57389 i/100ms
                 rhs     76222 i/100ms
                 lhs  2020064.6 (±14.7%) i/s -    9870908 in   5.015172s
                 rhs  3066573.4 (±13.2%) i/s -   15091956 in   5.012879s
[aaron@higgins rails (master)]$

With Active Support

[aaron@higgins rails (master)]$ AS=1 bundle exec ruby argh.rb 
Calculating -------------------------------------
                 lhs      4786 i/100ms
                 rhs     26327 i/100ms
                 lhs    62858.4 (±23.6%) i/s -     296732 in   5.019005s
                 rhs  2866546.6 (±26.6%) i/s -   13031865 in   4.996482s
[aaron@higgins rails (master)]$


In the benchmarks without Active Support, the performance is fairly close. The standard deviation is pretty big, but the numbers are within the ballpark of each other.

In the benchmarks with Active Support, the difference is enormous. It’s not even close. Why is this?

What is the deal?

This speed difference is due to a Freedom Patch that Active Support applies to the DateTime class:

class DateTime
  # Layers additional behavior on DateTime#<=> so that Time and
  # ActiveSupport::TimeWithZone instances can be compared with a DateTime.
  def <=>(other)
    super other.to_datetime

DateTime includes the Comparable module which will call the <=> method whenever you call the == method. This Freedom Patch calls to_datetime on whatever is on the right hand side of the comparison. Rails Monkey Patches the String class to add a to_datetime method, but “foo” is not a valid Date, so it raises an exception.

The Comparable module rescues any exception that happens inside <=> and returns false. This means that any time you call DateTime#== with something that doesn’t respond to to_datetime, an exception is raised and immediately thrown away.

The original implementation just does object equality comparisons, returns false, and it’s done. This is why the original implementation is so much faster than the implementation with the Freedom Patch.

My 2 Cents

These are the dangers of Freedom Patching. As a Rails Core member, I know this is a controversial opinion, but I am not a fan of Freedom Patching. It seems convienient until one day you wonder why is this code:

date == "foo"

so much slower than this code?

"foo" == date

Freedom Patching hides complexity behind a familiar syntax. It flips your world upside down; making code that seems reasonable do something unexpected. When it comes to code, I do not like the unexpected.

EDIT: I clarified the section about strings raising an exception. The actual exception occurrs in another monkeypatch in Rails.

read more »