A Starling Adapter for SMQueue

Starling is a persistent, lightweight work queue implemented in Ruby which talks the memcache protocol. I've recently started playing with it because I don't have the resource to look after, or the requirement for, a full blown service bus. Starling is easier to install and configure than ActiveMQ, but it's nowhere near as fully featured. Both have their place but a discussion of when and where to use them is outside the scope of this article.

I knew that I wanted to use a message bus to turn synchronous requests into asynchronous requests, pushing work off to some background process somewhere. What I didn't know was the form that the message bus would take. If you're familiar with the Gang of Four patterns book you've probably picked out the pattern that I should use here, but to be honest I'm buggered if I know what it's called. SMQueue which I'm familiar with provides a nice abstraction that makes it easy to swap out the message bus implementation while the code remains identical. Lovely, but SMQueue doesn't come with an adaptor for Starling.

"How hard," thought I, "would it be to implement a Starling adapter for SMQueue?"

I blinked and suddenly it existed. Awesome.

require 'rubygems'
require 'smqueue'
require 'starling'
require 'yaml'

module BarkingIguana
  module Messaging
    module SMQueue
      class StarlingAdapter < ::SMQueue::Adapter
        class Configuration < ::SMQueue::AdapterConfiguration
          DEFAULT_SERVER = '127.0.0.1:22122'

          has :queue
          has :server, :default => DEFAULT_SERVER
        end

        def initialize(*args)
          super
          options = args.first
          @configuration = options[:configuration]
          @configuration[:server] ||= Configuration::DEFAULT_SERVER

          @client = ::Starling.new(@configuration[:server])
        end

        def put(*args, &block)
          @client.set @configuration[:queue], args[0].to_yaml
        end

        def get(*args, &block)
          if block_given?
            loop do
              yield next_message
            end
          else
            next_message
          end
        end

        private
        def next_message
          ::SMQueue::Message(:headers => {},
            :body => YAML.load(@client.get(@configuration[:queue])))
        end
      end
    end
  end
end

Want to use it? You'll need Starling running somewhere. After that you can implement a producer in two lines of code:

producer = SMQueue(:adapter => BarkingIguana::Messaging::SMQueue::StarlingAdapter, :queue => "some.queue.name")
producer.put "Quack quack"

On the other side of the connection, here's a sample consumer:

consumer = SMQueue(:adapter => BarkingIguana::Messaging::SMQueue::StarlingAdapter, :queue => "some.queue.name")
consumer.get do |message|
  puts message.body.inspect
  # => "Quack quack"
end

One thing that's different about this adapter compared to the current SMQueue adapters is that it assumes you want to use YAML as a transport format. I'd prefer to use XML or JSON but it is at the moment just a preference, YAML is the easiest to implement, and I'm lazy.

There's also a bunch of work to do around failover - this adapter only supports one server. I still don't know enough about how Starling would handle failover so I don't want to rush into implementing that and discover I've done it wrong.

If you can help by providing patches for either other transport formats or failover please do.

written by

Craig

published

2009-05-08