Tender Lovemaking

Identifying unknown music with Ruby

Aaron Patterson — Mon, 04 Aug 2008 18:44:49 +0000

Ben Bleything inspired me (or rather distracted me from my yak shaving) to get my music library cleaned up and remove duplicates. Unfortunately my duplicates don't necessarily have ID3 tags, and they may be in different formats, so I wrote a gem called "earworm" which will identify unknown music. First I'll give you a code sample, then explain how to get earworm working, and finally explain how earworm works.

Here is the code sample:

ew = Earworm::Client.new('MY Music DNS Key')

info = ew.identify(:file => '/home/aaron/unknown.wav')

puts "#{info.artist_name} - #{info.title}"

Earworm just needs a MusicDNS key and a file name to return information about your unknown audio.

Okay, now for getting the gem to work. Unfortunately this takes a little work right now. You need to install a few libraries before earworm will work. These libraries let you decode and identify mp3, ogg, and wav files.

For OS X (assuming you have MacPorts already installed) do this:

$ sudo port install libogg libvorbis lame libofa

For Linux:

$ sudo yum install libogg-devel libvorbis-devel lame-devel libofa

Then just install the gem like so:

$ sudo gem install earworm

The final thing you need is a key from MusicDNS which you can obtain here. The key is free for non-commercial applications. Now you should be set! Earworm will automatically decode mp3 files and ogg files for you, so you don't have to do that yourself.

How does earworm work? First, it relies on icanhasaudio for audio decoding. Second, it uses DL to wrap libofa. libofa takes between 10 and 135 seconds of audio and generates a hash. After getting the hash for the audio snippet, earworm posts the hash (along with your key and some other info) to MusicDNS's web service. The web services returns an XML document with information about the audio snippet. Earworm parses the XML document and returns a nice object containing the information you requested. Thats it! No magic, and very simple (IMO) source code.

Enjoy!

Back Home!

Aaron Patterson — Tue, 08 Jul 2008 18:34:06 +0000

I'm finally back home. I went to Japan a few weeks ago for vacation, and I also spoke at Ruby Kaigi 2008. Ruby Kaigi was so much fun! I've been studying Japanese for a little over a year, but I've never been to Japan. It was exciting and fun to talk to people, and I made a bunch of new Japanese friends. I'd really like to thank Leonard Chin for helping out at the Kaigi. My language skills aren't good enough, and he was kind enough to fill in the gaps. Thank you!

While I was in Japan, I noticed QR Codes everywhere. QR Codes are basically really awesome bar codes. They can hold much more information in a smaller amount of space. They can be easily decoded from images taken with digital cameras. They have these codes everywhere in Japan, and the idea is that people can take a photo with the camera on their cell phone, then the phone decodes the QR Code. I believe most of the QR codes contain information about the company, or possibly a URL to the company's website.

The company that created the format says that the format is open, but unfortunately I have to pay for the spec. I can download the spec in Japanese for free, but my Japanese isn't that good! So unfortunately I'm stuck with either the ISO spec (which is over $200) or the AIM spec ($85). I don't understand why they are so expensive..... I think I'll buy the AIM one, and hope that it is the same as the ISO one.

Meow meow meow meow meow

Aaron Patterson — Fri, 06 Jun 2008 19:56:52 +0000

The other day I wrote an app called dejour to give me growl notifications from all the *jour gems out there. I used Eric Hodel's awesome ruby-growl library. Unfortunately it does all communications over the interweb, so you have to tweak some knobs in Growl to get it to work. I stumbled across a ruby/cocoa example using Growl, fixed it up, and released a gem called "Meow".

Meow lets you post notifications to your local machine without adjusting Growl. If you're on OS X 10.5, just do:

$ gem install meow

Then you can do this:

$ ruby -r rubygems -e'require "meow"; Meow.notify("meow", "meow", "meow")'

No growl tweaks required! Here is a code sample that is a little more explanatory:

require 'rubygems'
require 'meow'
meep = Meow.new('My Application Name')

meep.notify('Message Title', 'Message Description')

Be sure to check out the documentation.

Write your Rails view in……. JavaScript?

Aaron Patterson — Tue, 06 May 2008 18:33:30 +0000

In my last post about Johnson, I said that next time I would talk about the JavaScript parse tree that Johnson provides. Well, I changed my mind. Sorry.

I want to write about a rails plugin that I added to Johnson. Brohuda Katz wrote an ERb type parser in JavaScript, and added it to the (yet to be released) Johnson distribution. With that in mind, and looking at the new template handlers in edge rails, I was able to throw together a rails plugin that allows me to use JavaScript in my rails view code.

Lets get to the code. Here is my controller:

class JohnsonController < ApplicationController

  def index

    @users = User.find(:all)

  end

end

And my EJS view (the file is named index.html.ejs):

<% for(var user in at.users) { %>

  <%= user.first_name() %>

<% } %>

The johnson rails plugin puts controller instance variables in to a special javascript variable called "at". The "at" variable is actually a proxy to the controller, lazily fetching instance variables from the controller and importing those objects in to javascript land.

Lets take a look at the plugin, its only a few lines:

class EJSHandler < ActionView::TemplateHandler
class EJSProxy # :nodoc:
def initialize(controller)

      @controller = controller
end
    def key?(pooperty)

      @controller.instance_variables.include?("@#{pooperty}")

    end
    def [](pooperty)

      @controller.instance_variable_get("@#{pooperty}")

    end
    def []=(pooperty, value)

      @controller.instance_variable_set("@#{pooperty}", value)

    end

  end
  def initialize(view)

    @view = view

  end
  def render(template)

    ctx = Johnson::Context.new

    ctx.evaluate('Johnson.require("johnson/template");')

    ctx['template'] = template.source

    ctx['controller'] = @view.controller

    ctx['at'] = EJSProxy.new(@view.controller)
    ctx.evaluate('Johnson.templatize(template).call(at)')

  end

end
ActionView::Template.register_template_handler("ejs", EJSHandler)

When the template gets rendered (the render method), I wrap the controller with an EJS proxy, then compile the template into a javascript function, and call that function. The "at" variable is set to the EJSProxy before executing the template, and all property accessing on the "at" variable is passed along to fetching instance variables from the controller.

Server side javascript coding in rails. Weird, eh?

Take it to the limit one more time

Aaron Patterson — Wed, 23 Apr 2008 18:22:29 +0000

Sup bros. I need to post in this thing more often. Yesterday, someone tipped over my scooter again. I'm getting kind of tired of that.

Anyway, its time for me to write about this. RKelly is pretty much dead. For the past few months, John and I have been working on RKelly's replacement called Johnson. Basically we're now putting a ruby wrapper around Mozilla's Spidermonkey. The project is coming along quite nicely. Ruby objects can be passed in to javascript land, and javascript objects can be passed back in to ruby land.

For example, we can define an alert function in our javascript context:

require 'johnson'
ctx = Johnson::Context.new

ctx['alert'] = lambda { |x| puts x }

ctx.evaluate('alert("Hello world!");')

Johnson::Context#evaluate will also return the last statement evaluated. We can evaluate an expression, and manipulate that expression in ruby land. For example, I'll create an object in javascript, return it to ruby land, then access a property of the javascript object:

require 'johnson'
ctx = Johnson::Context.new

obj = ctx.evaluate('var foo = { x: "hello world" }; foo')

puts obj.x  # => 'hello world' 

We can even do the reverse by stuffing ruby objects in to the context:

A = Struct.new(:foo)
ctx = Johnson::Context.new

ctx['alert'] = lambda { |x| puts x }

ctx['a'] = A.new("bar")

ctx.evaluate('alert(a.foo);') # => 'bar' 

But it gets better. We added a top level variable called "Ruby" that lets you access constants and globals from Ruby land. We can rewrite the previous example completely in javascript:

ctx = Johnson::Context.new

ctx.evaluate("var x = new (new Ruby.Struct(Johnson.symbolize('foo')));")

ctx.evaluate("x.foo = 'bar'")

puts ctx.evaluate('x').foo # => 'bar'

puts ctx.evaluate('x').class # => #

Since the 'Ruby' constant delegates to Object, you can access any constant. Including ones you've defined yourself. We could, for example, look up a bunch of User records through rails:

ctx = Johnson::Context.new

ctx['alert'] = lambda { |x| puts x }

ctx.evaluate(<<-END

             for(var user in Ruby.User.find(Johnson.symbolize('all'))) {

               alert(user.first_name());

             }

             END

            )

You might be wondering what this Johnson.symbolize business is about. Since Javascript doesn't have a concept of a symbol, we've created a helper to "mark" a string as a symbol and pass it back in to ruby land.

To conclude this update about my Johnson, I'd like to show off an interactive shell for Johnson (thanks to Brohuda Katz). Johnson has an interactive shell that lets you try things out in javascript land or ruby land, and let you quickly switch between the two. Typing 'js' will put in you the javascript shell, 'rb' will switch you to the ruby shell. In the ruby shell, you can use the 'cx' variable to get ahold of you javascript context:

$ ruby -I lib bin/johnson
js> var x = { foo: 'bar', hello: function() { return 'world' } };
=> nil
js> rb
rb> cx['x'].foo
=> "bar"
rb> cx['x'].hello()
=> "world"
rb>

We aren't quite ready for a release yet, but if you'd like to play around with Johnson, you can pull it down from github here. Just run 'rake', and you should have it compiled and running!

My next Johnson related post will be about Javascript parse trees and Javascript code generation.

New Ruby Implementation - Brobinius

Aaron Patterson — Tue, 01 Apr 2008 22:00:51 +0000

Brobinius version 1.0.0 has been released!

I am happy to annouce the first release of my new fork of Ruby called
Brobinius. The goal of Brobinius is to implement new language features
that I have noticed to be completely missing.

For example, Object#tase! The tase method is featured in many other languages
but is sadly missing from Ruby. Brobinius has a fully implemented tase! method.

>> x = Class.new
=> #
>> x.tase!
RuntimeError: Don't tase me bro
from (irb):2:in `tase!'
from (irb):6
>>

Brobinius also has fully serializable kittenuations. You can create and
serialize your kittenuations, then pick up your snuggling where you left off.
For example:

>> lol = Kittenuation.new {
?> look_cute
>> throw :yarn
>> look_cute
>> }
=> #
>> lol.snuggle
>> Marshal.load(Marshal.dump(lol)).snuggle

Since Brobinius's kittenuations are serializable, you can share them over the
network with friends!

Brobinius also features screencasts with automatic YouTube uploads. All you
have to do write your program, then pass the --screencast option to Brobinius.
Brobinius will automatically create a screencast of your program and upload it
to YouTube:

$ brobinius --screencast my_code.rb

You can even add the --geoff flag to create screencasts with Geoffrey
Grosenbach doing the voice over.

Laser Etch My Macbook Air

Aaron Patterson — Thu, 20 Mar 2008 22:03:03 +0000

Dear Lazyweb,

I would really like to Laser Etch my Macbook Air with Martha Stewart's face. Where can I get that done? How much would it cost?

mechanize version 0.7.5 has been released!

Aaron Patterson — Sat, 15 Mar 2008 17:19:47 +0000

The Mechanize library is used for automating interaction with websites.
Mechanize automatically stores and sends cookies, follows redirects,
can follow links, and submit forms. Form fields can be populated and
submitted. Mechanize also keeps track of the sites that you have visited as
a history.

Changes:

# Mechanize CHANGELOG

## 0.7.5

Fixed a bug when fetching files and not pages. Thanks Mat Schaffer!
http://mechanize.rubyforge.org/

Profiling Database Queries in Rails

Aaron Patterson — Thu, 13 Mar 2008 21:16:37 +0000

Despite the recent Ruby webserver speed contests, most of the slowness at my job results from slow (or too many) database queries.

To help keep database queries down, I added a stats to every page that shows the number of queries vs. cache hits, the number of rows returned, and the amount of data transferred from the database. In this screenshot I'm using the "live" environment, 3 cache hits, 169 misses, 577 rows returned, and 458.9k data transferred. Clicking the box hides it, and clicking "Super Hide!" hides the box and sets a cookie so that the box doesn't show up again for a while.

To get this working, first I monkey patch the MysqlAdapter to collect database stats:

ActiveRecord::ConnectionAdapters::MysqlAdapter.module_evaldo

    @@stats_queries = @@stats_bytes = @@stats_rows = 0
    def self.get_stats

      { :queries => @@stats_queries,

        :rows => @@stats_rows,

        :bytes => @@stats_bytes }

    end

    def self.reset_stats

      @@stats_queries = @@stats_bytes = @@stats_rows = 0

    end
    def select_with_stats(sql, name)

      bytes = 0

      rows = select_without_stats(sql, name)

      rows.each do |row|

        row.each do |key, value|

          bytes += key.length

          bytes += value.length if value

        end

      end

      @@stats_queries += 1

      @@stats_rows += rows.length

      @@stats_bytes += bytes

      rows

    end

    alias_method_chain :select, :stats

  end

Next I patched the QueryCache to keep track of hits and misses:

ActiveRecord::ConnectionAdapters::QueryCache.module_evaldo

    @@hits = @@misses = 0
    def self.get_stats

      { :hits => @@hits,

        :misses => @@misses }

    end

    def self.reset_stats

      @@hits = @@misses = 0

    end
    def cache_sql_with_stats(sql, &block)

      if @query_cache.has_key?(sql)

        @@hits += 1

      else

        @@misses += 1

      end

      cache_sql_without_stats(sql, &block)

    end

    alias_method_chain :cache_sql, :stats

  end

Then modify ActionController to reset stats for each request:

ActionController::Base.module_evaldo
def perform_action_with_reset

      ActiveRecord::ConnectionAdapters::MysqlAdapter::reset_stats

      ActiveRecord::ConnectionAdapters::QueryCache::reset_stats

      perform_action_without_reset
end
    alias_method_chain :perform_action, :reset
    def active_record_runtime(runtime)

      stats = ActiveRecord::ConnectionAdapters::MysqlAdapter::get_stats

      "#{super} #{sprintf("%.1fk", stats[:bytes].to_f / 1024)} queries: #{stats[:queries]}"

    end

  end

Just drop all that inside the after_initialize in your development.rb and you'll get the nice stats. After that, just create a partial that displays the stats and include the partial at the bottom of your layout. Our partial looks like this:

<% unless %w(production test).include?(RAILS_ENV) -%>

   id="debug" onclick="$(this).remove()" style="background:pink;text-align:center;position:absolute;top:16px;left:35%;padding:0.5em;border: 2px solid red;">

  <%= RAILS_ENV %>

   />

  <% if ActiveRecord::ConnectionAdapters::QueryCache.respond_to?(:get_stats) %>

    <% stats = ActiveRecord::ConnectionAdapters::QueryCache.get_stats %>

    Queries: <%= stats[:hits] %> / <%= stats[:misses] %> /

    <%= number_to_percentage((stats[:hits].to_f / (stats[:hits] + stats[:misses])) * 100, :precision => 0) %>

    |

  <% end %>

  <% if ActiveRecord::ConnectionAdapters::MysqlAdapter.respond_to?(:get_stats) %>

    <% stats = ActiveRecord::ConnectionAdapters::MysqlAdapter.get_stats %>

    Rows: <%= stats[:rows] %> |

    Transfer: <%= sprintf("%.1fk", stats[:bytes].to_f / 1024) %>

  <% end %>

   style="margin:0">

     style="color:magenta" href="#" onclick="superHide()">super hide!

   type="text/javascript">

    function superHide() {

      document.cookie = 'debug=hidden; path=/; domain=<%= request.host %>; max-age=14400';

    }

    if(document.cookie.indexOf('debug=hidden') != -1) {

      $('debug').hide();

    }

<% end -%>

It's a little work, but it helps keep my mind on reducing the queries. With enough work, one of these days the speed of the webserver will matter to me. Thanks to Adam Doppelt for the basis of this monkey patch. Any bugs are mine, not his!

mechanize version 0.7.1 has been released!

Aaron Patterson — Wed, 05 Mar 2008 03:42:26 +0000

Changes:

# Mechanize CHANGELOG

## 0.7.1

Added iPhone to the user agent aliases. [#17572]
Fixed a bug with EOF errors in net/http. [#17570]
Handling 0 length gzipped responses. [#17471]
http://mechanize.rubyforge.org/