Skip to main content

Joel Liu's Library tagged MapReduce →  View Popular

09 Apr 08

Tv's cobweb: Incremental mapreduce

  • The way I imagined building an incremental mapreduce mechanism, without storing
    the intermediate data and just recomputing chunks that are out-of-date (which
    would be lame), is to add one extra concept into the system: call it "demap". It
    will basically create "negative entries" for the old data. This is basically
    what Damien did by providing both the old and new data map calls, all the time,
    just said differently, and I think my way might make the average call a lot
    simpler. And I don't see any reason why my version wouldn't be parallelizable,
    chainable, and generally yummy.
1 - 16 of 16
Showing 20▼ items per page

Selected Tags

Related Tags

Diigo is about better ways to research, share and collaborate on information. Learn more »

Join Diigo