Joel Liu's Library tagged filesystem → View Popular

10 May 09

Another reason to not use a DBMS for a search engine is that typical implementations of transaction-oriented SQL databases are a terrible fit for the performance requirements of a search engine. For example, search engines don't need concurrent writes or ACID transactions, or SQL-like query language; search engines want to optimize for large-scale updates, not small, random writes; typical DBMS index structures (btree) don't work well for search engine indices.
Eric Brewer has an interesting paper that lays out an architecture for a search engine that is consistent with DBMS design principles, but differs significantly in the implementation details:
http://www.cs.berkeley.edu/~brewer/papers/SearchDB.pdf

20 Apr 07

23 Jul 06

24 Mar 06

02 Mar 06

MogileFS is our open source distributed filesystem. Its properties and features include:
In this way, MogileFS (without RAID) can save money on disks that would otherwise be storing multiple copies of data unnecessarily.
1 more annotations...
- No single point of failure -- all three components of a MogileFS setup (storage nodes, trackers, and the tracker's database(s)) can be run on multiple machines, so there's no single point of failure. (you can run trackers on the same machines as storage nodes, too, so you don't need 4 machines...) A minimum of 2 machines is recommended.