Skip to main content

Joel Liu's Library tagged architecture →  View Popular

17 Jul 09

Stack Overflow Architecture | High Scalability

    • 16 million page views a month

    • 3 million unique visitors a month (Facebook reaches 77 million unique visitors a month)

    • 6 million visits a month

    • 86% of traffic comes from Google

    • 9 million active programmers in the world and 30% have used Stack Overflow.
    • If you’re comfortable managing servers then buy them. The two biggest problems with renting costs were: 1) the insane cost of memory and disk upgrades 2) the fact that they [hosting providers] really couldn’t manage anything.

    • Make larger one time up front investments to avoid recurring monthly costs which are more expensive in the long term.
  • 3 more annotations...
03 Jul 09

Gojko Adzic » QCon London 2009: Upgrading Twitter without service disruptions

  • A very interesting observation during the talk was that Twitter started up with a CMS model and that they gradually moved towards a messaging model. I’ve seen this in a few applications so far, including a casino system, where the messaging model seems to fit best an application intended to power massive community of online users, it seems regardless of what the application actually does business wise. Applications start out completely different, but then more and more functionality gets bolted on top of user messaging capabilities that the whole architecture on the end gets refactored to utilise the messaging channels as the core information transport. With Twitter, I’d expect this to be more obvious from the start as it was intended to help people notify each other.
  • The interesting thing, however, was that all the upgrades were done live, without shutting down the system. The changes were always introduced to one node, then regression issues were sorted out, and then the software would be rolled out to the whole cluster. They went as far as building a whole messaging system based on memcached APIs in order to be able to slot in such changes.
  • 3 more annotations...

InfoQ: Twitter, an Evolving Architecture

  • Most of the tools used by Twitter are open source. The stack is made up of Rails for the front side, C, Scala and Java for the middle business layer, and MySQL for storing data. Everything is kept in RAM and the database is just a backup.
  • image
  • 1 more annotations...

GoogleTalk Architecture | High Scalability

  • Measure the right thing.

    - People ask about how many IMs do you deliver or how many active users. Turns out not to be the right engineering question.

    - Hard part of IM is how to show correct present to all connected users because growth is non-linear: ConnectedUsers * BuddyListSize * OnlineStateChanges

    - A linear user grown can mean a very non-linear server growth which requires serving many billions of presence packets per day.

    - Have a large number friends and presence explodes. The number IMs not that

    big of deal.
25 Apr 09

Elven Architecture at amdev

    • Elf Island requires the following servers to run:


      • app server (n)
      • smartfox socket server (n)
      • db server (1+n)
      • web server for assets like swfs, images etc. (n)
      • load balancer
27 Dec 07

ArchitectNotes - Varnish - Trac

  • What happens is this: Squid creates a HTTP object in "RAM" and it gets used some
    times rapidly after creation. Then after some time it get no more hits and the
    kernel notices this. Then somebody tries to get memory from the kernel for
    something and the kernel decides to push those unused pages of memory out to
    swap space and use the (cache-RAM) more sensibly for some data which is actually
    used by a program. This however, is done without squid knowing about it. Squid
    still thinks that these http objects are in RAM, and they will be, the very
    second it tries to access them, but until then, the RAM is used for something
    productive.

SEDA - Architecture for Highly-Concurrent Server Applications


  • SEDA is an acronym for staged event-driven architecture, and
    decomposes a complex, event-driven application into a set of
    stages connected by queues. This design
    avoids the high overhead associated with thread-based concurrency
    models, and decouples event and thread scheduling from application
    logic. By performing admission control on each
    event queue, the service can be well-conditioned to load, preventing
    resources from being overcommitted when demand exceeds service
    capacity.
    SEDA employs dynamic control to automatically tune runtime parameters
    (such as the scheduling parameters of each stage), as well as to
    manage load, for example, by performing adaptive load shedding.
    Decomposing services into a set of stages also enables modularity and
    code reuse, as well as the development of debugging tools for complex
    event-driven applications.
1 - 19 of 19
Showing 20▼ items per page

Selected Tags

Related Tags

Diigo is about better ways to research, share and collaborate on information. Learn more »

Join Diigo