Starling: A Ruby Persistent Queue Server That Speaks Memcached - The Diigo Meta page

• 20 Aug 09
  

  Harjeet Singh

  Starling: A Ruby Persistent Queue Server
  
  RabbitMQ
  RudeQ
  Scaling
  Beanstalkd

  async ruby rails activemq starling

  • It's been around for a while now, but Starling is a "light-weight persistent queue server that speaks the MemCache protocol." Starling makes it ridiculously easy to set up a network-accessible queue (or many queues) for, say, asynchronous job processing between multiple processes and machines. It was developed by Twitter to handle the heavy amount of queueing necessary to keep their service ticking over.
  • FiveRuns have even created their own fork that, they say, is significantly faster.
  • 5 more annotations...
  • • An interesting alternative to Starling is also presented within the comments on Glenn's post - RudeQ. RudeQ uses the same API as Starling but is ActiveRecord / database based, meaning there's no extra process to monitor. I suspect it's nowhere near as fast, but if you'd rather avoid the headache of monitoring another persistent process or don't have the option of having a persistent process at all (shared hosting, perhaps) it's worth checking out.

    • I highly recommend Beanstalkd as an alternative lightweight (albeit non-persistent queue) - its been rock solid for us at Reevoo; its lack of persistance has not proved to be an issue with its current uptime of 185 days, i.e. since we first started it up in production, and its very fast too.

      
      

      http://labs.reevoo.com/plugins/beanstalk-messaging

    • Another of our engineers has an implementation of Starling in Scala, called Scarling, that's more suited to certain operational scenarios: http://github.com/robey/scarling/. We may deploy Scarling, but we expect to move to a robust queuing system within the next several months.

    • RabbitMQ in this context.

      
      

      There is a shiny new ruby client that you might like to play with. No need to know any erlang code, but you still get all the scalability and stability of the platform. Plus you can turn on persistence, do pub-sub, etc - these are all config options.

      
      

      The ruby client is described here: http://hopper.squarespace.com/blog/2008/7/22/simple-amqp-library-for-ruby.html

    • Apache ActiveMQ
      
      http://activemq.apache.org/
      
      (needs Java J2EE, and speaks many protocols, including the simple STOMP protocol which has a ruby client / command-line)

      
      

      Sparrow
      
      http://code.google.com/p/sparrow/
      
      (persistent in files/SQLite, speaks memcached)

      
      

      RQ (by Ara T. Howard)
      
      http://www.codeforpeople.com/lib/ruby/rq/
      
      (based on NFS?)

      
      

      StompServer
      
      http://stompserver.rubyforge.org/
      
      (speaks STOMP, of course)

• 18 Nov 08
  

  Damon Snyder

  Erlang

  queue starling workqueue

• 24 Jul 08
  

  Brent Sordyl

  Starling is a "light-weight persistent queue server that speaks the MemCache protocol." Starling makes it ridiculously easy to set up a network-accessible queue

  ruby memcached queue messaging rubyonrails

• 22 Jul 08
  

  Ariejan de Vroom

  starling ruby queues memcached rails share

Would you like to comment?

Join Diigo for a free account, or sign in if you are already a member.