Barking Iguana

Symbol#to_proc is slow... is it slow enough to matter?

2008-11-18T23:45:20Z

It's common knowledge that using the to_proc hack is slower than not. Just how much slower is it? I decided to put together a few benchmarks to find out.

Environment

These tests were run on Ruby 1.8.6-pl111 and Rails 2.1.

Benchmarking

Say there's a database of 1,000 items that for some reason you want to iterate over. Let's forget that if you're showing 1,000 items you probably have usability issues and just roll with it.

1_000.times { |n| Bar.create :name => "bar-#{n}" }
bars = Bar.find(:all)

Here's how much slower it is over a dataset of 1,000 ActiveRecord instances.

Benchmark.measure { bars.map(&:name) }.real
#=> 0.00645709037780762

Benchmark.measure { bars.map { |b| b.name } }.real
#=> 0.00141692161560059

Now that's a horrific increase: it takes more than 350% longer to run the to_proc hack than the plain block... but let's be realistic here, over 1,000 records it's taken 0.0065 seconds. Big woop. Who cares?

How about over 1,000,000 rows? We already have 1,000 rows, let's top that up.

(1_000_000 - 1_000).times { Bar.create :name => Time.now.to_f.to_s }
bars = Bar.find(:all)

That makes it 1,000,000 rows in the table. By this stage your database is probably thinking you hate it. I'm pretty confident that presenting 1,000,000 rows to the person using your application is a bit of an edge case, but hey, here's how long it takes.

Benchmark.measure { bars.map(&:name) }.real
#=> 6.25304508209229

Benchmark.measure { bars.map { |b| b.name } }.real
#=> 1.38965106010437

Almost 5 seconds extra over a million rows. Okay so 5 seconds is a pretty big hit, but how long will your application be running before you hit a million rows in one of your tables and you need to iterate over all million rows?

Don't optimise your code prematurely. By the time to_proc becomes an issue you'll already have hit many, many other problems.

Benchmark.measure { Bar.find(:all) }.real
#=> 406.738657951355

Worry about those first.

Handling error feedback from Ajax requests to Rails applications

2008-11-17T22:30:17Z

Ajax is frequently used to provide a richer user experience. Why then are important error messages rarely handled properly in Ajax enabled applications? Handling errors gracefully and in a way that helps the visitor to solve them adds a really high quality feel to your application. We've got all the necessary machinery, all it takes is a little care and attention.

class FoosController < ApplicationController
  def update
    @foo = Foo.find(params[:id])
    respond_to do |format|
      if @foo.save
        format.html do
          flash[:info] = "Your foo has been created."
          redirect_to @foo
        end
        format.js { head :ok }
      else
        format.html do
          flash.now[:warning] = "I could not update the foo."
          render :action => :edit
        end
        format.json do
          head :unprocessable_entity, :json => @foo.errors.to_json
        end
      end
    end
  end
end

Using the above code we get a lovely fallback HTML based behaviour and when we work with Ajax we get to use JSON behaviours. When JSON is requested and something goes wrong we get back an array of arrays that takes the following form.

[
  [ "attribute1", "error1", "error2" ],
  [ "attribute2", "error3"]
]

Just think of the awesome things you could do with such rich feedback...

new Ajax.Request('/foo.json', {
  method: 'PUT',
  parameters: {
    authenticity_token: window._token,
    "foo[subject]": $F('foo_subject'),
    "foo[body]"   : $F('foo_body')
  },
  onSuccess: function(transport) {
    // This is Web 2.0: celebrate with a yellow highlight.
  },
  onFailure: function(transport) {
    var errors = transport.responseJSON;
    errors.each(function(error) {
      var attribute = error.shift();
      var messages = error.join(", ");
      var errorMessage = attribute + " " + messages;
      var inputNode = $("foo_" + attribute);
      if(inputNode) {
        // Show that something is wrong with this field.
        inputNode.addClassName("error");
        // Do something better than an alert box. Alert boxes suck.
        alert(errorMessage);
      }
    });
  }
});

Ajax and the Rails request authenticity token

2008-11-17T20:21:36Z

Rails 1.2.6 introduced CSRF protection in the form of an authenticity token which is a reasonably long string used to make sure that any PUT / POST / DELETE request you've made to an application was really generated by you (or at least your browser) doing something in the application and that you weren't tricked into submitting it by some nefarious third party.

Rails automatically adds this token to any forms generated by it's helpers, but when building rich Ajax applications it can be useful to be able to generate the Javascript by hand.

Fire this snippet into your layout just above including all the other Javascript files to get access to the authenticity token in Javascript and let you submit requests using Ajax.

<%= javascript_tag "window._token = '#{form_authenticity_token}';" %>

Now you can build Ajax requests that are allowed to do stuff to the application.

new Ajax.Request('/foo.json', {
  method: 'PUT',
  parameters: {
    authenticity_token: window._token,
    text: $F('foo_text')
  }
  /* callbacks omitted for brevity */
})

Writing a Story: why, when, where, who, what, how and a bunch of other questions and answers

2008-11-16T14:35:47Z

It's quite a head-shift to start using story-driven development. Often it's hard to work out what a story should contain and who should be involved in writing it. Wouldn't it be nice to have some guidelines to help you get started? Here are mine.

Get the current Git branch in your command prompt

2008-11-15T20:27:47Z

It seems that everyone and their dog has their own way to represent the current Git branch in the command prompt. Here's mine. Stick it in your ~/.profile.

export PS1='\[\033[01;32m\]\u@\h\[\033[00m\] \[\033[01;34m\]\w\[\033[00m\]$(git branch &>/dev/null; if [ $? -eq 0 ]; then echo "\[\033[01;33m\]($(git branch | grep ^*|sed s/\*\ //))\[\033[00m\]"; fi)$ '

craig@shiny ~/sandbox/addressbook(master)$

Now with 50% cleaner code

Not long after I posted this snippet I found a post about enabling git auto-completion. If you use the auto-complete file that's bundled with Git you can use the following code to get the same prompt (and get some nifty tab-based goodies too).

export PS1='\[\033[01;32m\]\u@\h\[\033[00m\] \[\033[01;34m\]\w\[\033[00m\]$(__git_ps1 "\[\033[01;33m\](%s)\[\033[00m\]")$ '

Setting up a public Git repository

2008-11-15T19:34:21Z

More and more these days I'm using Git as my version control system. I want to make code available to the general public. The easy choice would be to use a service like GitHub or repo.or.cz but I'm vain and want to serve my code from barkingiguana.com. I don't need to support multiple committers and I'd like to learn more about how Git works so I don't want to use Gitosis. It turns out that it's pretty easy to setup your own public repository...

Getting started with Story Driven Development for Rails with Cucumber

2008-11-11T12:07:31Z

I've been hearing about Story Driven Development (SDD) for a while but I haven't tried it out because I was under the impression that there was a huge amount to learn and setup before I could get going. I'm not sure if that used to be true, but I started using Cucumber yesterday and it was really easy.

Install and configure

You'll need to install a bunch of RubyGems.

sudo gem install nokogiri term-ansicolor treetop diff-lcs hpricot cucumber

Install Cucumber into your Rails app.

ruby script/generate cucumber

Install WebRAT. Unfortuantely this doesn't seem to be available as a RubyGem. If you're using Git then install this as a submodule instead. We're not so I clone the repository then svn add it.

git clone git://github.com/brynary/webrat.git vendor/plugin/webrat

Writing your first story

Stories have three components: a business value, the role of a person that uses the feature and some description of the feature.

In order to [do something with business value]
As [role]
Should [describe the feature]

An example might be the ability to order a pizza from the online ordering system of a pizza delivery company.

Feature: Order Pizza
  In order to get some hot, tasty pizza
  A hungry pizza lover
  Should be able to order pizza

Next we need to define some scenarios. Scenarios are things that can happen during the story. Most pizza places aren't open 24 hours a day so two simple scenarios are (1) the pizza shop is closed, and (2) the pizza shop is open.

  Scenario: The pizza shop is closed
    Given the pizza shop is closed
    And I am on the home page
    And I click "Feed Me!"
    Then I should see "Sorry, the shop is closed"

  Scenario: The pizza shop is open
    Given the pizza shop is open
    And I am on the home page
    And I click "Feed Me!"
    Then I should see "Your pizza will be with you soon"

The above description should go in a file called something like features/order_pizza.feature where it is lovely and version controlled and safe.

So, we now have a story that describes how a feature should behave. How does that get turned into acceptance tests? Well, you could pass these descriptions off to your testing team, or you could turn them into part of your test suite.

Automated tests: better than cake

You might notice that when installing Cucumber you got the directory features/steps. That's where you tell your test suite how to understand your stories. There are already two files here: common_webrat.rb which gives your test suite a few funky things like the ability to click links and env.rb which does pretty much the same stuff as spec/spec_helper.rb except for Cucumber. You can ignore env.rb, but common_webrat.rb will provide a few examples of how to start writing story steps.

Create a new file, order_pizza_steps.rb. This is where you define the steps involved in ordering pizza. It's pretty much just regular expressions which match each line of a scenario to some Ruby code.


Given /the pizza shop is open/ do
  PizzaShop.open = true
end

Given /the pizza shop is closed/ do
  PizzaShop.open = false
end

And /I am on the home page/ do
  visits "/"
end

That's all we need to do. The common WebRAT steps provide the necessary mapping for clicking buttons and checking for feedback.

Running your stories

This is pretty simple: run rake features. You should get some rather pretty coloured output, and if anything has gone wrong Cucumber is pretty good at suggesting ways to fix it.

Found this article useful?

If you enjoyed this article I'd appreciate recommendations at Working with Rails.

Content_for is the new GOTO

2008-11-06T17:06:29Z

I don't like content_for. Your view code jumps around up and down files and makes it hard to work out what's going on. It smells a lot like GOTO. When was the last time you saw someone recommend you use a GOTO?

content_for :javascript and content_for :css

Use of content_for can be easily avoided, at least for including CSS and Javascript files. Include the controller name and action name in the body tag in your layout and properly qualify your CSS declarations.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
                         "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
  <title><%= page_title %></title>
  <meta http-equiv="Content-Language" content="English" />
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
  <link rel="stylesheet" type="text/css" href="/stylesheets/simple.css" media="screen" />
</head>
<body id="<%= "#{controller.controller_name.tableize.singularize}_#{controller.action_name}" %> class="<%= "#{controller.controller_name.tableize.singularize} #{controller.action_name}" %>">
  <%= yield %>
</body>
</html>

Say you were looking at the Posts views in your app, you can now style these using something like this.

.post.index .article .title { 
  font-size: 1.25em;
}

.post.show .article .title { 
  font-size: 0.9em;
}

Or, in case you need to support browsers that don't let you specify two classes as a selector for a single element, you can write it like this.

#post_index .article .title { 
  font-size: 1.25em;
}

#post_show .article .title { 
  font-size: 0.9em;
}

Since all your Javascript is unobtrusive anyway (right?), it should be pretty easy to qualify the selectors used there with the same CSS selectors shown above.

As an added bonus, by specifying your Javascript / CSS like the above you can package it all in one Javascript or CSS file on deployment to your production environment and save yourself a bunch of HTTP requests.

Make sure you're @importing files that exist

2008-11-03T22:56:15Z

A few people near me have heard the start of my rumblings about optimising the number of HTTP requests required for each page. There are a variety of reasons that you might want to do this, but that discussion is for another post. For now simply assume that I don't like unnecessary HTTP requests. Extrapolating from that you would reasonably assume that I really don't like wasted HTTP requests such as might happen when the url in a CSS @import directive 404's. If you assumed that, you'd be right.

I got fed up of tracking down the source of these errors in a few applications that I maintain so I fired up a TextMate session and scraped together this nasty little snippet of Ruby to do my dirty work for me.

#! /usr/bin/env ruby

css_root = File.expand_path(`pwd`.strip)
css_files = Dir[File.join(css_root, "**", "*.css")]

missing_imports = Hash.new([])
css_files.each_with_index do |css_file, index|
  imports = File.read(css_file).split(/\n|\r/).grep(/\@import url\((.*)\)/)
  imports.each do |import|
    desired_path = import.scan(/url\((["'\ ])?(.*)\1\)/).to_a.first.to_a.last
    desired_root = desired_path[0,1] == "/" ? css_root : File.dirname(css_file)
    filesystem_path = File.expand_path(File.join(desired_root, desired_path))
    if !File.exists?(filesystem_path)
      missing_imports[css_file] += [{ :path => filesystem_path, :directive => import }]
    end
  end
end

if missing_imports.any?
  puts "Missing files declared as imports in CSS:\n\n"

  missing_imports.keys.each do |origin|
    puts "Origin:               #{origin}"
    missing_imports[origin].each do |import|
      puts "Missing @import file: #{import[:path]}"
      puts "Directive:            #{import[:directive]}"
    end
    puts ""
  end
else
  puts "No imported files are missing. Well done."
end

Run it in the directory which serves your document root - for Rails applications this would be RAILS_ROOT/public/ - and it'll either spit out a list of missing files you've tried to @import or tell you that you've done well.

Missing files declared as imports in CSS:

Origin:               /Users/craig/projects/1.8/public/stylesheets/.../find_by_service.css
Missing @import file: /Users/craig/projects/1.8/public/stylesheets/.../a_to_z.css
Directive:            @import url('.../a_to_z.css');

Now, just to be clear, I don't like @import directives. I'd prefer they were completely removed from the CSS. They are popular with a lot of people though so I'll compromise for the moment: if you must use them, please make sure they're going to work.

Scaling: Using MogileFS for storing uploaded images

2008-10-31T23:39:57Z

As you might have guessed from several of my previous posts, The team I've been working in has recently been scaling an application. I've learned a bunch of things along the way, several of which I've got half-written articles about and which I'll totally finish one day, honest.

One of the most awesome technologies I've started using is MogileFS, a distributed BLOB store. In out application we use this to store user-generated assets such as uploaded images and syndication feeds. I'll not go into the pros and cons of the technology here (I might do that another time), rather I'd like to share some code that we've found rather useful when handling image uploads and adding them to MogileFS: the MogileFilesystemBackend for AttachmentFu.

It's necessary to use a shared filestore for uploaded images when the application cluster you're using for uploads needs to scale beyond one physical box as otherwise the uploaded images land on several disks and there's no telling if they'll be available to a particular request to your application (that depends on which application server serves the request).

Getting stuck in

I've done some rather ugly preparation for this work and monkey-patched Kernel to provide an attr_accessor called filestore which is just an instance of MogileFS::MogileFS from the rather excellent MogileFS client by the clever people over at Seattle RB. The patch, which I'm sure will make experienced Rubyists cry, looks like this.

module Kernel
  # Oh noes, I'm screwing with Kernel.
  # 
  mattr_accessor :filestore
end

During the Rails initializer execution the filestore is setup using configuration values pulled from a YAML file in RAILS_ROOT/config/.

Kernel.filestore = MogileFS::MogileFS.new(
  :domain => "APPNAME-#{RAILS_ENV}",
  :hosts => array_of_hosts_from_yaml_file
)

(What I actually do is quite a bit different from this but that's because I've done evil things to the MogileFS client library which I'll probably share in the future. For now, believe the magic).

Now that the setup is complete, how do we get AttachmentFu to work with the filestore? We use the MogileFilesystemBackend of course!

class Image << ActiveRecord::Base
  has_attachment :content_type => :image,
    :storage => :mogile_filesystem,
    :max_size => 5.megabytes,
    :thumbnails => {
      :canonical => '1024x'
    },
    :processor => "MiniMagick"

  validates_as_attachment
end

The power behind the man

Of course, without the actual backend code not much is going to happen. The implementation was pretty heavily influenced by the existing Amazon S3 backend, mostly because the idea behind S3 and MogileFS is very similar.

module MogileFilesystemBackend
  def full_filename(thumbnail = nil)
    "#{class_prefix}:#{filestore_tag(thumbnail)}"
  end

  def filestore_tag(thumbnail = nil)
    "#{parent_id || id}:#{thumbnail || :original}"
  end

  def current_content
    temp_path ? File.read(temp_path) : temp_data
  end
  
  def public_filename(thumbnail = nil)
    [
      editorial_object_type.demodularize.tableize,
      editorial_object_id,
      "#{class_prefix}.#{file_extension}#{thumbnail && "?size=#{thumbnail}"}"
    ].join("/")
  end

  def file_extension
    Mime::Type.lookup(content_type).to_sym
  end

  def filestore_paths(thumbnail = nil)
    filestore.get_paths(full_filename(thumbnail))
  end

  def file_data(thumbnail = nil)
    filestore.get_file_data(full_filename(thumbnail))
  end

  protected
  def current_content_location
    temp_path ? :temp_path : :temp_data
  end

  def destroy_file
    filestore.delete full_filename
  end

  def rename_file
    filestore.rename @old_filename, full_filename
  end

  def save_to_storage
    logger.info "Storing #{self.class.name}\##{id} as #{full_filename(thumbnail)} (class: #{replication_policy}) from #{current_content_location == :temp_path ? temp_path : :memory}"
    filestore.store_content full_filename(thumbnail), replication_policy, current_content
  end

  def class_prefix
    self.class.name.demodularize.underscore.downcase
  end
  alias :replication_policy, :class_prefix
end

Technoweenie::AttachmentFu::Backends::MogileFilesystemBackend = ::MogileFilesystemBackend

Serving the public

So now you can get images into MogileFS, but in order to be useful we also need to serve them to the visitors of our application. That'll require a little work in the controller to make it read from the ever-present filestore instead of the database (if you're storing files in the database I will HURT you) or the local filesystem.

class ImageController < ApplicationController
  before_filter :load_image

  def show
  respond_to do |format|
    format.html
    format.any(:png, :jpg, :gif) do
      send_data @image.file_data(params[:size]),
        :type => @image.content_type,
        :disposition => 'inline'
    end
  end
  
  protected
  def load_image
    @image = Image.find(params[:id])
  end
end

There we have it. Images can now be requested through the ImageController and served to your adoring fans.

Found this article useful?

If you enjoyed this article I'd appreciate recommendations at Working with Rails.