This link has been bookmarked by 226 people . It was first bookmarked on 19 Jan 2009, by Tim Goh.
-
20 Mar 12
-
04 Mar 12
-
11 Jan 12
carlos puentesPerhaps you’re considering using a dedicated key-value or document store instead of a traditional relational database.
database distributed scalability storage db cloud programming RDBMS
-
28 Oct 11
-
28 Sep 11
-
23 Sep 11
-
21 May 11
-
Scalaris is a key-value store - it uses a modified version of the Chord algorithm to form a DHT, and stores the keys in lexicographical order, so range queries are possible.
-
On top of the DHT they use an improved version of Paxos to guarantee ACID properties when dealing with multiple concurrent transactions. So it’s a key-value store, but it can guarantee the ACID properties and do proper distributed transactions over multiple keys.
- 2 more annotation(s)...
-
-
Oh, and to demonstrate how you can scale a webservice based on such a system, the Scalaris folk implemented their own version of Wikipedia on Scalaris, loaded in the Wikipedia data, and benchmarked their setup to prove it can do more transactions/sec on equal hardware than the classic PHP/MySQL combo that Wikipedia use. Yikes.
-
From what I can tell, Scalaris is only memory-resident at the moment and doesn’t persist data to disk.
-
-
-
12 May 11
-
27 Apr 11
-
19 Apr 11
-
21 Mar 11
-
04 Mar 11
-
28 Feb 11
-
27 Feb 11
-
17 Dec 10
-
19 Oct 10
Grant SladeThis has some great links to other related articles / information regarding the direction which dbms's are moving
-
14 Oct 10
-
28 Jul 10
-
07 Jul 10
-
06 Jul 10
-
09 Jun 10
-
13 May 10
-
22 Apr 10
-
15 Apr 10
-
28 Mar 10
-
13 Mar 10
-
11 Mar 10
-
09 Mar 10
-
21 Feb 10
-
18 Feb 10
-
11 Feb 10
-
09 Feb 10
-
14 Jan 10
-
23 Dec 09
-
14 Dec 09
-
02 Dec 09
-
01 Dec 09
-
30 Nov 09
-
18 Nov 09
-
13 Nov 09
-
11 Nov 09
-
04 Nov 09
-
01 Nov 09
-
28 Oct 09
-
25 Oct 09
-
24 Oct 09
-
22 Oct 09
-
17 Oct 09
-
14 Oct 09
-
13 Oct 09
Tim Lossen"This article represents my notes and research to date on distributed key-value stores (and some other stuff) that might be suitable as RDBMS replacements under the right conditions."
-
11 Oct 09
-
10 Oct 09
danintersting run around various non-sql data stores and which work for last.fm
-
06 Oct 09
-
20 Sep 09
-
19 Sep 09
-
15 Sep 09
-
13 Sep 09
-
07 Sep 09
-
06 Sep 09
-
21 Aug 09
-
04 Aug 09
-
03 Aug 09
-
01 Aug 09
-
Dan DascalescuComparison overview of key-value storage backends
RDBMS database key value distributed hash table DHT scalability performance
-
29 Jul 09
-
27 Jul 09
-
22 Jul 09
-
09 Jul 09
-
08 Jul 09
-
07 Jul 09
-
06 Jul 09
-
04 Jul 09
-
03 Jul 09
rawwell"Here is a list of projects that could potentially replace a group of relational database shards. Some of these are much more than key-value stores, and aren’t suitable for low-latency data serving, but are interesting none-the-less."
-
28 Jun 09
-
25 Jun 09
-
23 Jun 09
-
21 Jun 09
-
16 Jun 09
-
14 Jun 09
-
27 May 09
-
17 May 09
-
13 May 09
-
12 May 09
-
08 May 09
-
28 Apr 09
-
24 Apr 09
-
23 Apr 09
-
07 Apr 09
-
06 Apr 09
-
05 Apr 09
-
31 Mar 09
-
27 Mar 09
Ian ForresterPerhaps you’re considering using a dedicated key-value or document store instead of a traditional relational database.
programming development data storage database cloud store performance db list distributed comparison architecture couchdb scalability rdbms erlang simpledb memcached dht
-
-
- You’re suffering from Cloud-computing Mania.
- You need an excuse to ‘get your Erlang on’
- You heard CouchDB was cool.
- You hate MySQL, and although PostgreSQL is much better, it still doesn’t have decent replication. There’s no chance you’re buying Oracle licenses.
- Your data is stored and retrieved mainly by primary key, without complex joins.
- You have a non-trivial amount of data, and the thought of managing lots of RDBMS shards and replication failure scenarios gives you the fear.
Perhaps you’re considering using a dedicated key-value or document store instead of a traditional relational database. Reasons for this might include:
-
At Last.fm we do a lot of batch computation in Hadoop, then dump it out to other machines where it’s indexed and served up over HTTP and Thrift as an internal service (stuff like ‘most popular songs in London, UK this week’ etc). Presently we’re using a home-grown index format which points into large files containing lots of data spanning many keys, similar to the Haystack approach mentioned in this article about Facebook photo storage. It works, but rather than build our own replication and partitioning system on top of this, we are looking to potentially replace it with a distributed, resilient key-value store for reasons 4, 5 and 6 above.
- 1 more annotation(s)...
-
-
- Distributed Hash Table (DHT) and algorithms such as Chord or Kadmelia
- Amazon’s Dynamo Paper, and this ReadWriteWeb article about Dynamo which explains why such a system is invaluable
- Amazon’s SimpleDB Service, and some commentary
- Google’s BigTable paper
- The Paxos Algorithm - read this page in order to appreciate that knocking up a Paxos implementation isn’t something you’d want to do whilst hungover on a Saturday morning.
Glossary and Background Reading
-
-
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.