2010-12-09 @ 10:07

Ruby Advent Calendar

Every year Japanese Rubyists participate in what is called the “Ruby Advent Calendar”.

What is the Ruby Advent Calendar?

Each day leading up to December 25th, one person posts an article to their blog and adds a link to their blog on the Advent Calendar. So, 25 blog posts total. The posts can be about anything related to Ruby.

In my opinion, the Ruby Advent Calendar is about encouraging people to blog about Ruby and help others participate in the Ruby community.

How I can participate?

In past years, the articles have mainly been in Japanese. This year, we’re trying to do an English language Ruby Advent Calendar.

@yhara_en was kind enough to add English instructions. I will add screen shots with notes here so that you can more easily participate.

Step 1: Get some Coffee

As I’ve said in previous blog posts, I love it when instructions tell you to get coffee because I always do. So here is your opportunity. Go get some coffee!

Step 2: Click a Button

First, go to this link. Then click the button that says “このエベントに参加登録する”:

Ruby Advent Calendar jp-en: 2010 : ATND

Once you’ve clicked that button, you need to register.

Step 3: Register

The easiest way to register is with your Twitter account. Click the link that says “Twitterでログイン”:

Ruby Advent Calendar jp-en: 2010 : ATND

You’ll be taken through the normal Twitter authorization path. That part is in English, so I’m not going to cover it.

Step 4: Edit account details

Enter your name and website and click save (the button that says “保存する”):

users/profile : ATND

Step 5: Register for the Ruby Advent Calendar

Go back to the Ruby Advent Calendar and click the giant red button again:

Ruby Advent Calendar jp-en: 2010 : ATND

This time it will ask you to enter a comment:

Ruby Advent Calendar jp-en: 2010 : ATND

So that we can coordinate better, I just said which day I would like to participate.

Step 6: Publish a Blurgh Post

On your day, publish your blog post, then head back to the advent calendar, scroll to the bottom of the page, and link to your blog in the comments section:

Ruby Advent Calendar jp-en: 2010 : ATND

That’s it!

Happy Holidays, and Happy Blurghing!!!

<3<3<3<3<3<3 –tenderlove

read more »

2010-12-11 @ 16:25

Writing Ruby C Extensions: Part 2

OMG! It's been a year since I posted Writing Ruby C Extensions: Part 1. The first post I did was for the Ruby Advent Calendar in 2009. I guess it's fitting that I write a blog post for the Ruby Advent Calendar 2010. Anyway, if you haven't read part 1, please go read it now.

In Part 2, we'll modify our extconf.rb file to find important files in libstree, then we'll create a Ruby class that is backed by a C structure.

The final code associated with this part of of my Writing Ruby C Extensions series can be found here.

Using mkmf to find libraries

As I mentioned in the previous post, extconf.rb is used when installing a native gem to locate libraries, header files, and test various things about the target system before installing. We're going to teach our extconf.rb file to locate the libstree dynamic library along with the header files. We're also going to allow people to tell the gem where to find libstree, and set up our extconf.rb with some sensible defaults.

mkmf configuration with dir_config

The first thing we'll do is tell mkmf where to look for libstree files by default. We do this using the dir_config method. dir_config takes three arguments:

  • An arbitrary string, but usually the library name (like "stree")
  • A list of paths to search for header files
  • A list of paths to search for library files

The dir_config method also allows users installing our gem to configure where mkmf should look for various files. Let's take a look at our call to dir_config and talk about what it does:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
LIBDIR      = Config::CONFIG['libdir']
INCLUDEDIR  = Config::CONFIG['includedir']

HEADER_DIRS = [
  # First search /opt/local for macports
  '/opt/local/include',

  # Then search /usr/local for people that installed from source
  '/usr/local/include',

  # Check the ruby install locations
  INCLUDEDIR,

  # Finally fall back to /usr
  '/usr/include',
]

LIB_DIRS = [
  # First search /opt/local for macports
  '/opt/local/lib',

  # Then search /usr/local for people that installed from source
  '/usr/local/lib',

  # Check the ruby install locations
  LIBDIR,

  # Finally fall back to /usr
  '/usr/lib',
]

dir_config('stree', HEADER_DIRS, LIB_DIRS)

First, this code builds a two lists of sensible defaults for finding header files and library files. The HEADER_FILES and LIB_DIRS constants contain lists of common places to find libraries. These settings will be nice for our users because if they have libstree installed in /opt/local/ or /usr/local/ it will find the library without any user intervention.

Finally, we call dir_config with the string "stree" and two lists. This call to dir_config only configures mkmf with directories to search. We actually haven't done any searching at this point. The dir_config call also allows users to configure the gem on installation. The call sets up the following flags for our user to configure:

  • --with-stree-dir
  • --with-stree-include
  • --with-stree-lib

Finding headers and libraries

Now that we've configured mkmf with where we can find libraries and headers, we need to search for required header files and libraries. We'll do that with two functions: find_header and find_library.

We need to find the stree/lst_string.h header file, so we'll just supply that to the find_header method like so:

1
2
3
unless find_header('stree/lst_string.h')
  abort "libstree is missing.  please install libstree"
end

This code will tell mkmf to find the header file we need. If the header file can't be found, find_header will return false, and we can abort installation and provide some instructions. If the find_header method is a success, the directory where the header file was found will be added to the -I flags that get passed to your compiler.

Next, we need to find the libstree dynamic library. For this task, we'll use the find_library function call:

1
2
3
unless find_library('stree', 'lst_stree_free')
  abort "libstree is missing.  please install libstree"
end

The find_library function takes two arguments. The first argument is the library that we need to link against. This string will be passed to the -l flags. The second argument is a symbol we need to find in the library.

In this code example, mkmf will create a test C program that tries to link against stree and find the function lst_stree_free. If linking is successful, the path will be added to the -L flags provided to your compiler. If it fails, we abort installation and provide an error message.

Creating the Makefile

Just like the last article we still need the call to create_makefile in our extconf.rb:

1
create_makefile('stree/stree')

You can find the complete extconf.rb here.

Wrapping LST_String from libstree

libstree defines a String type structure. We're going to define a class in Ruby to wrap up this string type structure. Eventually, we'll have some Ruby code that looks like this:

1
2
string = STree::String.new 'foo'
assert_equal 3, string.length

In fact, since we're doing TDD let's start with a test for the length method. We'll also add a test to ensure that objects other than String objects will raise a TypeError:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
require 'stree'
require 'test/unit'

module STree
  class TestString < Test::Unit::TestCase
    def test_length
      string = STree::String.new 'foo'
      assert_equal 3, string.length
    end

    def test_type_error
      assert_raises(TypeError) do
        STree::String.new Object.new
      end
    end
  end
end

File structure

In my C projects, I like to make one C file per class. We have to make an entry point though, so we'll keep stree.h and stree.c from our previous project. Then we'll write stree_string.h and stree_string.c to keep our String class.

Library entry point

The entry point to our C code will be in stree.c. The stree.c file will initialize the String class. Here is the new stree.h file that includes libstree:

1
2
3
4
5
6
7
8
9
10
11
#ifndef RUBY_STREE
#define RUBY_STREE

#include <ruby.h>;
#include <stree/lst_string.h>;

#include <stree_string.h>;

extern VALUE mSTree;

#endif

We include header files from libstree, we include the header file for the string class, then we declare a global variable which will hold a reference to our Ruby "STree" module.

The new stree.c file looks like this:

1
2
3
4
5
6
7
8
9
10
#include <stree.h>

VALUE mSTree;

void Init_stree()
{
  mSTree = rb_define_module("STree");

  Init_stree_string();
}

When our library is required, Init_stree is called, then we'll define the STree module (assigning it to the global module variable) and initialize our String class. Now we need to define Init_stree_string in stree_string.h and stree_string.c.

Defining the String class

First we'll create the header file for our string class. We'll only have one public function called Init_stree_string, so our header file will look like this:

1
2
3
4
5
6
7
8
#ifndef RUBY_STREE_STRING
#define RUBY_STREE_STRING

#include <stree.h>

void Init_stree_string();

#endif

We include the main stree.h header file, then define our public initialize function. Now we need to define the body of the Init_stree_string function in stree_string.c:

1
2
3
4
5
6
#include <stree_string.h>

void Init_stree_string()
{
  VALUE cSTreeString = rb_define_class_under(mSTree, "String", rb_cObject);
}

The rb_define_class_under function will define a class "String" in the module pointed to by mSTree with a parent class of Object. This C code is equivalent to the following Ruby code:

1
2
3
4
module STree
  class String
  end
end

At this point, you should be able to compile the project and run the tests. We haven't defined any methods on the STree::String class in Ruby yet, but our project should compile, and the tests should execute. If you're following along, you should see test output like this:

  1) Error:
test_length(STree::TestString):
ArgumentError: wrong number of arguments (1 for 0)
    ./test/test_stree_string.rb:7:in `initialize'
    ./test/test_stree_string.rb:7:in `new'
    ./test/test_stree_string.rb:7:in `test_length'

Allocating the String class

The first thing we're going to do is teach Ruby how to allocate our String class. Ruby gives us a hook when the allocate method is called where we can allocate internal structures (we're actually defining the allocate method on the STree::String class).

First, let's modify the init function to tell ruby about our allocate function:

1
2
3
4
5
6
void Init_stree_string()
{
  VALUE cSTreeString = rb_define_class_under(mSTree, "String", rb_cObject);

  rb_define_alloc_func(cSTreeString, allocate);
}

rb_define_alloc_func tells Ruby to call a function pointer allocate when this class gets allocated. New we need to define our allocate function:

1
2
3
4
5
6
static VALUE allocate(VALUE klass)
{
  LST_String * string = malloc(sizeof(LST_String));

  return Data_Wrap_Struct(klass, NULL, deallocate, string);
}

In our allocate function, we allocate enough memory to hold an LST_String struct. Then we call Data_Wrap_Struct to return our actual Ruby object. Data_Wrap_Struct takes four arguments:

  • The Ruby class we're dealing with (in this case it's cSTreeString
  • A function pointer that is called when the object is marked
  • A function pointer that is called with the object is freed
  • A void pointer of the data we want to wrap

You'll notice we're referencing a function deallocate that isn't defined yet. Let's define that function now:

1
2
3
4
static void deallocate(void * string)
{
  lst_string_free((LST_String *)string);
}

The deallocate function is called with the pointer we passed to Data_Wrap_Struct, in this case an LST_String pointer. We'll use the lst_string_free function from libstree to free our pointer.

Defining STree::String#initialize

Now we need to define the initialize method. This method will take one argument (a string), and we'll populate the underlying LST_String struct with information from the Ruby string.

To define the initialize method, first we call rb_define_method:

1
2
3
4
5
6
7
void Init_stree_string()
{
  VALUE cSTreeString = rb_define_class_under(mSTree, "String", rb_cObject);

  rb_define_alloc_func(cSTreeString, allocate);
  rb_define_method(cSTreeString, "initialize", initialize, 1);
}

rb_define_method takes 4 arguments:

  • The class on which we want to define a method
  • The name of the method we're defining
  • A function pointer that will be called when our method is called
  • The number of parameters passed to that function

Next we need to define our initialize C function:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
static VALUE initialize(VALUE self, VALUE rb_string)
{
  LST_String * string;
  void * data;

  Check_Type(rb_string, T_STRING);

  Data_Get_Struct(self, LST_String, string);

  data = calloc(RSTRING_LEN(rb_string), sizeof(char));
  memcpy(data, StringValuePtr(rb_string), RSTRING_LEN(rb_string));

  lst_string_init(
      string,
      data,
      sizeof(char),
      RSTRING_LEN(rb_string));

  return self;
}

The initialize function has two parameters, the first is the instance of our STree::String object, the second is the single parameter for our method.

After declaring our variables we check the type of the required argument. Check_Type is a macro provided by Ruby to let us perform type checking on objects. We use this macro to ensure that the user passed us a Ruby string. If not, the Check_Type macro will automatically raise a type error.

Next we make a call to a macro provided by Ruby: Data_Get_Struct. Our LST_String pointer is stored inside the Ruby VALUE object, and Data_Get_Struct will extract our pointer. We give this macro the ruby object self, followed by the struct type we want to extract (LST_String), followed by the pointer where it will be assigned (string).

We need to copy the contents of the Ruby string to a buffer that our LST_String can keep. To do that, we use:

  • calloc to allocate the memory
  • RSTRING_LEN to get the number of bytes in our string
  • memcpy to copy the memory contents
  • StringValuePtr to get the underlying character pointer from Ruby

We give the data to libstree by calling lst_string_init, then finally return self.

At this point, we should have one passing test and one failing test:

  1) Error:
test_length(STree::TestString):
NoMethodError: undefined method `length' for #<STree::String:0x101f0e6f8>
    ./test/test_stree_string.rb:8:in `test_length'

Next we need to define the length method.

Defining STree::String#length

The hard part is over. Defining the length method should be much easier than the initialize method. Just like the initialize method, we need to call rb_define_method:

1
2
3
4
5
6
7
8
void Init_stree_string()
{
  VALUE cSTreeString = rb_define_class_under(mSTree, "String", rb_cObject);

  rb_define_alloc_func(cSTreeString, allocate);
  rb_define_method(cSTreeString, "initialize", initialize, 1);
  rb_define_method(cSTreeString, "length", length, 0);
}

This time, we're defining a function length that takes 0 arguments. Now lets define the length C function:

1
2
3
4
5
6
7
8
static VALUE length(VALUE self)
{
  LST_String * string;

  Data_Get_Struct(self, LST_String, string);

  return INT2NUM(lst_string_get_length(string));
}

Just like the initialize function, we declare our variables, then unwrap our struct. We use the lst_string_get_length function from libstree to get the string length as an integer. Then we use a macro provided by Ruby, INT2NUM, that converts the integer to a Ruby Numeric object and return that object.

After we've defined this method, all of our tests should pass:

Loaded suite -e
Started
..
Finished in 0.000873 seconds.

2 tests, 2 assertions, 0 failures, 0 errors

Yay!

Conclusion

OMG! C CODE WRAPPED WITH RUBY!

We've scratched the surface for writing C extensions in Ruby. In this part, we:

  • taught our system how to find the library we want to use
  • (briefly) dealt with memory management of our objects
  • defined modules and classes
  • defined methods on our classes

You can grab the code for part 2 here.

Happy holidays to EVERYONE! I hope you liked Part 2 of Writing Ruby C Extensions!

<3<3<3<3<3<3<3<3 –tenderlove

read more »