• 28 Jan 10
  

  Jyotsna oswal

  "Starling: A Ruby Persistent Queue Server That Speaks Memcached
  
  By Peter Cooper — July 22nd, 2008 — 13 Comments
  
  In Elsewhere, Tutorials
  
  starling.png
  
  It's been around for a while now, but Starling is a "light-weight persistent queue server that speaks the MemCache protocol." Starling makes it ridiculously easy to set up a network-accessible queue (or many queues) for, say, asynchronous job processing between multiple processes and machines. It was developed by Twitter to handle the heavy amount of queueing necessary to keep their service ticking over. Starling is proven in production, with not only Twitter using it in anger, but FiveRuns too. FiveRuns have even created their own fork that, they say, is significantly faster."

  starling rails

  •   

    

      

     

     

    Starling: A Ruby Persistent Queue Server That Speaks Memcached

     

    By Peter Cooper — July 22nd, 2008 — 13 Comments

     

    In Elsewhere, Tutorials

     

     

     

    

     

    It's been around for a while now, but Starling is a "light-weight persistent queue server that speaks the MemCache protocol." Starling makes it ridiculously easy to set up a network-accessible queue (or many queues) for, say, asynchronous job processing between multiple processes and machines. It was developed by Twitter to handle the heavy amount of queueing necessary to keep their service ticking over. Starling is proven in production, with not only Twitter using it in anger, but FiveRuns too. FiveRuns have even created their own fork that, they say, is significantly faster.
• 14 Dec 09
  

  Paul Brown

  ruby starling memcached rails messaging

• 08 Nov 09
  

  Francisco Cabrita

  ruby queue distributed scalability

• 10 Sep 09
  

  Don Hill

  ruby queue server mq

• 20 Aug 09
  

  Harjeet Singh

  Starling: A Ruby Persistent Queue Server
  
  RabbitMQ
  RudeQ
  Scaling
  Beanstalkd

  async ruby rails activemq starling

  • It's been around for a while now, but Starling is a "light-weight persistent queue server that speaks the MemCache protocol." Starling makes it ridiculously easy to set up a network-accessible queue (or many queues) for, say, asynchronous job processing between multiple processes and machines. It was developed by Twitter to handle the heavy amount of queueing necessary to keep their service ticking over.
  • FiveRuns have even created their own fork that, they say, is significantly faster.
  • An interesting alternative to Starling is also presented within the comments on Glenn's post - RudeQ. RudeQ uses the same API as Starling but is ActiveRecord / database based, meaning there's no extra process to monitor. I suspect it's nowhere near as fast, but if you'd rather avoid the headache of monitoring another persistent process or don't have the option of having a persistent process at all (shared hosting, perhaps) it's worth checking out.

  • I highly recommend Beanstalkd as an alternative lightweight (albeit non-persistent queue) - its been rock solid for us at Reevoo; its lack of persistance has not proved to be an issue with its current uptime of 185 days, i.e. since we first started it up in production, and its very fast too.

     

    http://labs.reevoo.com/plugins/beanstalk-messaging
  • Another of our engineers has an implementation of Starling in Scala, called Scarling, that's more suited to certain operational scenarios: http://github.com/robey/scarling/. We may deploy Scarling, but we expect to move to a robust queuing system within the next several months.

  • RabbitMQ in this context.

     

    There is a shiny new ruby client that you might like to play with. No need to know any erlang code, but you still get all the scalability and stability of the platform. Plus you can turn on persistence, do pub-sub, etc - these are all config options.

     

    The ruby client is described here: http://hopper.squarespace.com/blog/2008/7/22/simple-amqp-library-for-ruby.html

  • Apache ActiveMQ
     http://activemq.apache.org/
     (needs Java J2EE, and speaks many protocols, including the simple STOMP protocol which has a ruby client / command-line)

     

    Sparrow
     http://code.google.com/p/sparrow/
     (persistent in files/SQLite, speaks memcached)

     

    RQ (by Ara T. Howard)
     http://www.codeforpeople.com/lib/ruby/rq/
     (based on NFS?)

     

    StompServer
     http://stompserver.rubyforge.org/
     (speaks STOMP, of course)

  5 more annotations...

• 10 Feb 09
  

  Andrew Gilmartin

  ruby queue harvest

• 18 Nov 08
  

  Damon Snyder

  Erlang

  queue starling workqueue

• 22 Aug 08
  

  Rodrigo Chappa

  rails starling

• 28 Jul 08
  

  Marcin Bajer

  memcache Ruby twitter reference @implement from-delicious

• 24 Jul 08
  

  Brent Sordyl

  Starling is a "light-weight persistent queue server that speaks the MemCache protocol." Starling makes it ridiculously easy to set up a network-accessible queue

  ruby memcached queue messaging rubyonrails

• 23 Jul 08
  

  zeroeth

  ruby queue

• 22 Jul 08
  

  miles woodroffe

  ruby starling mq

Would you like to comment?

Join Diigo for a free account, or sign in if you are already a member.