Colin Henderson's personal annotations on this page
Bankwatch bookmarked
on 2008-05-22
-
The retrieval system is where things are not as simple. Unlike webmail services where refreshing a user’s inbox only queries a very simple data set (is there anything new in MY inbox?), refreshing a user’s home page on Twitter queries a much more complex data set (are there any new updates in ALL my friends’ pages?) and the nature of the service means that the ratio of reads to writes is significantly different from most other web services.
This link has been bookmarked by 21 people . It was first bookmarked on 14 May 2008, by Scott Moody.
-
-
Twitter probably does use an inbox sort of model;
-
-
Ricky RobinsonThe problems involved in architecting a microblogging service like Twitter. Shows that Ruby ain't the problem, but if you're developing a messaging service, it probably shouldn't look like a CMS under the covers.
-
Scaling a Microblogging Service - Part I
-
-
-
Scaling a Microblogging Service - Part I
-
-
Martin Lindnera short & very clear introduction to scaling 2.0 services.
-
-
The retrieval system is where things are not as simple. Unlike webmail services where refreshing a user’s inbox only queries a very simple data set (is there anything new in MY inbox?), refreshing a user’s home page on Twitter queries a much more complex data set (are there any new updates in ALL my friends’ pages?) and the nature of the service means that the ratio of reads to writes is significantly different from most other web services.
-
-
-
The retrieval system is where things are not as simple. Unlike webmail services where refreshing a user’s inbox only queries a very simple data set (is there anything new in MY inbox?), refreshing a user’s home page on Twitter queries a much more complex data set (are there any new updates in ALL my friends’ pages?) and the nature of the service means that the ratio of reads to writes is significantly different from most other web services.
It is these constant queries that bring Twitter down during popular events. The fact that a single request for a user’s home page or an API request for a user’s friends timeline must perform a complex iterative query is what causing some requests to take too long, at best timeout and at worst cause the entire system to lag behind. These custom views are expensive and mean that it is much more difficult to cache the final result of a request.
Going through a timeline request, the server first looks up the list of people the user is following. Then for each person, checks if their timeline is public or private, and if private, if the requesting user has the rights to view it. If the user has rights, the last few messages are retrieved, and the same is repeated for each person being followed. When done, all the messages are collected, sorted, and the latest messages are converted from their internal representation to the requested format (JASON, XML, RSS, or ATOM).
-
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.