This link has been bookmarked by 38 people . It was first bookmarked on 27 Sep 2007, by someone privately.
-
07 Apr 14
Navneet KumarIf you want to scale to the petabyte level, or the billion requests a day, you need to be:
- Distributed. The data has to be distributed across multiple machines.
- Joinless. No joins, and no referential integrity, at least at the data store level.
- De-Normalized. No one said this explicily, but I presume there is a lot of de-normalization going on if you are avoiding joins.
- Transcationless. No transactions.
Those constraints represent something fundamentally different from a relational database.Distributed-Systems Distributed-Database Distributed-Storage Scalability
-
- The requirements were:
- Fast realtime key/value lookup.
- Complicated non realtime queries mainly for operation and maintenance.
- Distributed data due to distributed applications.
- High fault tolerance.
- Dynamic re configuration.
- Complex objects.
-
The query syntax is part of Erlang itself, while the data model is akin to an object-relational DBMS:
-
Mnesia is also interesting due to its tight coupling to the programming language Erlang, thus almost turning Erlang into a database programming language.
-
This has many benefits, the foremost being that the impedance mismatch between data format used by the DBMS and data format used by the programming language which is being used to manipulate the data, completely disappears.
-
"Organizing the telecommunications data in third (or even first) normal form is usually not possible." Storing complex values in columns means that no joins need to be made.
-
The DBMS runs in the same process space as the application. This allows a lookup to return a pointer to the object without the need to marshall the object to or from different data formats, or access it over the wire.
-
"Erlang processes have the efficiency advantage of running in the same address space but they do not have the possibility to explicitly read or write each others memory."
-
Provide the ability for the application developer to access tables transparently - i.e. regardless of if it is remote or local, or a replica.
-
Queries are expressed using a so-called "list comprehension syntax".
-
The big volumes mean you need to be able to write data and not care where it went. And you need keyed lookup for reads built on top of the FS, not in the RDBMS (on the basis that an RDBMS with no joins, constraints or triggers is an indexed filesystem). That will end looking looking something like hadoop, mogilefs or S3 - a data parallel architecture.
-
-
16 Aug 10
-
28 Nov 09
-
09 Jul 09
menbomSection 2 of the paper provides an overview of Mnesia components. Mnesia is comprised of several Erlang applications that perform the essential DBMS services such as locking, transaction management and replication. The authors note that Erlang is well sui
-
14 Jan 09
-
17 Nov 08
-
28 Sep 08
-
11 Sep 08
-
29 Aug 08
-
21 Aug 08
-
16 Aug 08
-
26 May 08
-
14 Oct 07
-
05 Oct 07
-
20 Sep 07
Hendy IrawanThese attributes can be used to describe Mnesia , the Erlang distributed DBMS, that supports high scalability and fault tolerance through replication, with the ability to lookup records without the need for a traditional relational database join. Mnesia i
performance distributed programming dbms scalability mnesia database erlang amqp architecture article code concurrency db development
-
04 Sep 07
-
25 Aug 07
-
22 Aug 07
-
21 Aug 07
-
20 Aug 07
-
Bruno Martins2007 could well be dubbed "the year the normalized database myth was debunked". We have seen interesting discussions on how large websites have moved away from the traditional relational database approach in order to service the millions of hits per hour
-
-
19 Aug 07
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.