Skip to main content

Diigo Home

Hacker News | Follow on: Use DBMS or fs? - The Diigo Meta page

news.ycombinator.com/item - Cached - Annotated View

Joel Liu's personal annotations on this page

joel
Joel bookmarked on 2009-05-10 database filesystem search engine
  • Another reason to not use a DBMS for a search engine is that typical implementations of transaction-oriented SQL databases are a terrible fit for the performance requirements of a search engine. For example, search engines don't need concurrent writes or ACID transactions, or SQL-like query language; search engines want to optimize for large-scale updates, not small, random writes; typical DBMS index structures (btree) don't work well for search engine indices.

    Eric Brewer has an interesting paper that lays out an architecture for a search engine that is consistent with DBMS design principles, but differs significantly in the implementation details:

    http://www.cs.berkeley.edu/~brewer/papers/SearchDB.pdf

This link has been bookmarked by 2 people . It was first bookmarked on 10 May 2009, by Joel Liu.

  • 01 Jul 09
  • 10 May 09
    • Another reason to not use a DBMS for a search engine is that typical implementations of transaction-oriented SQL databases are a terrible fit for the performance requirements of a search engine. For example, search engines don't need concurrent writes or ACID transactions, or SQL-like query language; search engines want to optimize for large-scale updates, not small, random writes; typical DBMS index structures (btree) don't work well for search engine indices.

      Eric Brewer has an interesting paper that lays out an architecture for a search engine that is consistent with DBMS design principles, but differs significantly in the implementation details:

      http://www.cs.berkeley.edu/~brewer/papers/SearchDB.pdf