kinaze 's List: bigdata

• he end goal.  This is of course the ability to handle an unlimited volume of data, processing all requests in real time.
• Keeping in mind all the different platforms that Big Data is having an impact on (web, cloud, enterprise, mobile) combined with all the Big Data domain challenges (transaction processing, analytics, data mining, visualization) as well as many of the Big Data characteristic requirements (volume, timeliness, availability, consistency), it is easy to see how no single technology will provide a cover-all solution for the eclectic mix of needs.
• However, we are now facing the problem that there is fast becoming too much data for us to digest in its raw form.  To move forward in our understanding we will need to rely on technology to provide timely, summarized & relevant data across all aspects of our lives.  This is what those working in Big Data are setting out to achieve.

1 more annotation...

• Nous étions dans le royaume de la science informatique et du génie; maintenant avec une telle somme de données, il est possible de générer de la valeur dans le département marketing: faire émerger des patterns, remonter les signaux faibles, croiser les bases pour faire apparaître des tendances

• Google's founding philosophy is that we don't know why this page is better than that one: If the statistics of incoming links say it is, that's good enough. No semantic or causal analysis is required. That's why Google can translate languages without actually "knowing" them (given equal corpus data, Google can translate Klingon into Farsi as easily as it can translate French into German). And why it can match ads to content without any knowledge or assumptions about the ads or the content.
• The scientific method is built around testable hypotheses. These models, for the most part, are systems visualized in the minds of scientists. The models are then tested, and experiments confirm or falsify theoretical models of how the world works. This is the way science has worked for hundreds of years.
• Once you have a model, you can connect the data sets with confidence. Data without a model is just noise.
• Petabytes allow us to say: "Correlation is enough."
• We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.

3 more annotations...

• Any theoretic model is a reduction and simplification of reality to its relevant essence; and so for two purposes. The first is that models help understand reality. Second is they help to predict what will happen once we figure out the underlying mechanisms (simplified even as they are). Throwing lots of data and an algorithm can indeed replace models for understanding reality. (Like the example of the wine quality prediction in the Authors@Google talk). But it will not help in understanding anything, you still need models for that.

• Megan Smith, Google.org: Uberuser, ubercreator, ubercataloguer of data. Google.org leverages Google employees to do good things with data. Three examples of google.org’s use of big data to do good.

   
   
  • Google.org is trying to use Landsat images over time to start tracking the deforrstation footprint anywhere across the world for past 30 years – Google Earth Engine.
   
  • Flu trends – tracking 36 search terms for flu related symptoms across globe. Found it to be over 95% effective in anticipating flu outbreaks across the world.
   
  • Google Crisis Response – integrating the response to crisis with easy use of big data.
   
  • Google Books - Not a google.org initiative but shows what can be done by putting lots of information from disparate sources online.