The way I imagined building an incremental mapreduce mechanism, without storing
the intermediate data and just recomputing chunks that are out-of-date (which
would be lame), is to add one extra concept into the system: call it "demap". It
will basically create "negative entries" for the old data. This is basically
what Damien did by providing both the old and new data map calls, all the time,
just said differently, and I think my way might make the average call a lot
simpler. And I don't see any reason why my version wouldn't be parallelizable,
chainable, and generally yummy.
Ads by Google
Brown University - Data Management Group » A Comparison of Approaches to Large-Scale Data Analysis: MapReduce vs. DBMS Benchmarks
more fromdatabase.cs.brown.edu
Map-reduce-merge: simplified relational data processing on large clusters | Lambda the Ultimate
more fromlambda-the-ultimate.org
Running Hadoop On Ubuntu Linux (Single-Node Cluster) - Michael G. Noll
more fromwww.michael-noll.com
Typical Programmer - Relational Database Experts Jump The MapReduce Shark
more fromtypicalprogrammer.com
Welcome to Hadoop!
more fromlucene.apache.org
Notation: * = Private bookmark and comment|… = Clipping [?] | … = Public highlight [?]
Joel Liu's Related Tags
Selected tags
Related tags
See More Top Contributors
Related Groups on Diigo
Related Lists on Diigo
-
Parallel Databases
Links related to mapreduce ...
Items: 11 | Visits: 3
Created by: Dmitry Serebrennikov