This link has been bookmarked by 465 people . It was first bookmarked on 24 Jun 2008, by Steven Rafferty.
-
30 Mar 14
-
13 Mar 14
-
oogle's founding philosophy is that we don't know why this page is better than that one: If the statistics of incoming links say it is, that's good enough. No semantic or causal analysis is required. That's why Google can translate languages without actually "knowing" them
-
George Box's maxim: "All models are wrong, and increasingly you can succeed without them."
-
Scientists are trained to recognize that correlation is not causation, that no conclusions should be drawn simply on the basis of correlation between X and Y (it could just be a coincidence). Instead, you must understand the underlying mechanisms that connect the two. Once you have a model, you can connect the data sets with confidence. Data without a model is just noise.
-
t faced with massive data, this approach to science — hypothesize, model, test — is becoming obsole
-
here is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science canno
-
The best practical example of this is the shotgun gene sequencing by J. Craig Venter.
-
that case, Venter can make some guesses about the animals — that they convert sunlight into energy in a particular way, or that they descended from a common ancestor. But besides that, he has no better model of this species than Google has of your MySpace page. It's just data. By analyzing it with Google-quality computing resources, though, Venter has advanced biology more than anyone else of his generation.
-
he new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Correlation supersedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic explanation at all.
There's no reason to cling to our old ways. It's time to ask: What can science learn from Google
-
-
28 Jan 14
-
13 Jan 14
-
17 Dec 13
-
05 Dec 13
Adam MarcusChris Anderson's 2008 article on the rise of big data and its implication on methods of scientific research.
-
02 Nov 13
-
21 Oct 13
-
16 Oct 13
-
16 Sep 13
-
20 Aug 13
-
14 Aug 13
-
21 Jul 13
-
"All models are wrong, and increasingly you can succeed without them."
-
Scientists are trained to recognize that correlation is not causation, that no conclusions should be drawn simply on the basis of correlation between X and Y (it could just be a coincidence). Instead, you must understand the underlying mechanisms that connect the two. Once you have a model, you can connect the data sets with confidence. Data without a model is just noise.
But faced with massive data, this approach to science — hypothesize, model, test — is becoming obsolete. Consider physics: Newtonian models were crude approximations of the truth (wrong at the atomic level, but still useful). A hundred years ago, statistically based quantum mechanics offered a better picture — but quantum mechanics is yet another model, and as such it, too, is flawed, no doubt a caricature of a more complex underlying reality. The reason physics has drifted into theoretical speculation about n-dimensional grand unified models over the past few decades (the "beautiful story" phase of a discipline starved of data) is that we don't know how to run the experiments that would falsify the hypotheses — the energies are too high, the accelerators too expensive, and so on.
Now biology is heading in the same direction. The models we were taught in school about "dominant" and "recessive" genes steering a strictly Mendelian process have turned out to be an even greater simplification of reality than Newton's laws. The discovery of gene-protein interactions and other aspects of epigenetics has challenged the view of DNA as destiny and even introduced evidence that environment can influence inheritable traits, something once considered a genetic impossibility.
In short, the more we learn about biology, the further we find ourselves from a model that can explain it.
There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.
-
J. Craig Venter
-
Venter can tell you almost nothing about the species he found. He doesn't know what they look like, how they live, or much of anything else about their morphology. He doesn't even have their entire genome. All he has is a statistical blip — a unique sequence that, being unlike any other sequence in the database, must represent a new species.
-
In February, the National Science Foundation announced the Cluster Exploratory, a program that funds research designed to run on a large-scale distributed computing platform developed by Google and IBM in conjunction with six pilot universities. The cluster will consist of 1,600 processors, several terabytes of memory, and hundreds of terabytes of storage, along with the software, including IBM's Tivoli and open source versions of Google File System and MapReduce.
-
The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Correlation supersedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic explanation at all.
-
It's time to ask: What can science learn from Google?
-
-
02 Jul 13
-
The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Correlation supersedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic explanation at all.
-
-
01 Jul 13
-
25 May 13
-
24 May 13
-
20 May 13
-
19 May 13
-
12 May 13
-
07 May 13
-
02 May 13
-
"All models are wrong, but some are useful."
-
-
30 Apr 13
-
29 Apr 13
-
16 Apr 13
-
Google's founding philosophy is that we don't know why this page is better than that one: If the statistics of incoming links say it is, that's good enough. No semantic or causal analysis is required. That's why Google can translate languages without actually "knowing" them (given equal corpus data, Google can translate Klingon into Farsi as easily as it can translate French into German). And why it can match ads to content without any knowledge or assumptions about the ads or the content.
-
-
01 Apr 13
-
11 Mar 13
-
15 Feb 13
-
11 Feb 13
-
15 Jan 13
-
21 Dec 12
-
This sequence may correlate with other sequences that resemble those of species we do know more about. In that case, Venter can make some guesses about the animals — that they convert sunlight into energy in a particular way, or that they descended from a common ancestor. But besides that, he has no better model of this species than Google has of your MySpace page. It's just data. By analyzing it with Google-quality computing resources, though, Venter has advanced biology more than anyone else of his generation.
-
-
20 Dec 12
-
11 Dec 12
-
08 Dec 12
-
All models are wrong, and increasingly you can succeed without them."
-
All models are wrong, and increasingly you can succeed without them
-
All models are wrong, and increasingly you can succeed without them."
-
All models are wrong, and increasingly you can succeed without them
-
All models are wrong, and increasingly you can succeed without them
-
All models are wrong, and increasingly you can succeed without them
-
-
27 Nov 12
-
09 Oct 12
-
05 Oct 12
-
10 Sep 12
-
04 Sep 12
-
31 Jul 12
-
13 Jul 12
stepstoderivative universe - google doesn't need a theory, just a process to apply over and over on the data
-
14 Jun 12
-
08 May 12
-
t the petabyte scale, information is not a matter of simple three- and four-dimensional taxonomy and order but of dimensionally agnostic statistics.
-
, one that requires us to lose the tether of data as something that can be visualized in its totality.
-
view data mathematically first and establish a context for it later.
-
Google's founding philosophy is that we don't know why this page is better than that one: If the statistics of incoming links say it is, that's good enough. No semantic or causal analysis is required.
-
Peter Norvig, Google's research director, offered an update to George Box's maxim: "All models are wrong, and increasingly you can succeed without them."
-
Forget taxonomy, ontology, and psychology. Who knows why people do what they do
-
Scientists are trained to recognize that correlation is not causation, that no conclusions should be drawn simply on the basis of correlation between X and Y (it could just be a coincidence). Instead, you must understand the underlying mechanisms that connect the two
-
A hundred years ago, statistically based quantum mechanics offered a better picture — but quantum mechanics is yet another model, and as such it, too, is flawed, no doubt a caricature of a more complex underlying reality.
-
The discovery of gene-protein interactions and other aspects of epigenetics has challenged the view of DNA as destiny and even introduced evidence that environment can influence inheritable traits, something once considered a genetic impossibility.
-
In February, the National Science Foundation announced the Cluster Exploratory, a program that funds research designed to run on a large-scale distributed computing platform developed by Google and IBM in conjunction with six pilot universities
-
Correlation supersedes causation
-
-
18 Apr 12
-
17 Apr 12
-
16 Apr 12
-
09 Apr 12
-
12 Mar 12
Joaquin Borrego DíazGet the latest in science news, including space, physics, planet earth, discoveries, NASA, satellites, and space travel from Wired.com
-
10 Mar 12
-
07 Feb 12
Sam DonovanThis essay by Chris Anderson at Wired is a couple of years old - but it just shows he was ahead of the game.
-
06 Feb 12
Charles van der HaegenThe Petabyte Age:
Sensors everywhere. Infinite storage. Clouds of processors. Our ability to capture, warehouse, and understand massive amounts of data is changing science, medicine, business, and technology. As our collection of facts and figures grows, so will the opportunity to find answers to fundamental questions. Because in the era of big data, more isn't just more. More is different.
The End of Theory:
Essay: The Data Deluge Makes the Scientific Method Obsolete
Feeding the Masses
Chasing the Quark
Winning the Lawsuit
Tracking the News
Spotting the Hot Zones
Sorting the World
Watching the Skies
Scanning Our Skeletons
Tracking Air Fares
Predicting the Vote
Pricing Terrorism
Visualizing Big Data-
order but of dimensionally agnostic statistics
-
All models are wrong, and increasingly you can succeed without them."
-
The discovery of gene-protein interactions and other aspects of epigenetics has challenged the view of DNA as destiny and even introduced evidence that environment can influence inheritable traits, something once considered a genetic impossibility.
-
Correlation supersedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic explanation at all.
-
-
13 Dec 11
-
05 Dec 11
-
01 Dec 11
-
Google conquered the advertising world with nothing more than applied mathematics
-
Petabytes allow us to say: "Correlation is enough."
-
-
18 Nov 11
-
10 Oct 11
-
05 Oct 11
max_ritscher"All models are wrong, but some are useful."
So proclaimed statistician George Box 30 years ago, and he was right. But what choice did we have? Only models, from cosmological equations to theories of human behavior, seemed to be able to consistently, if -
02 Oct 11
-
Doug Peterson"All models are wrong, but some are useful."So proclaimed statistician George Box 30 years ago, and he was right. But what choice did we have? Only models, from cosmological equations to theories of human behavior, seemed to be able to consistently, if imperfectly, explain the world around us. Until now. Today companies like Google, which have grown up in an era of massively abundant data, don't have to settle for wrong models.
Science Data statistics theory google technology philosophy information
-
29 Sep 11
-
Billy Halla possible way to open and think through how poetry is both a model and exists outside of models and rather points toward techne, knowing how to use data rather than contain the whole of data in representaion.
-
28 Sep 11
-
27 Sep 11
bradkincaidThe End of Theory: The Data Deluge Makes the Scientific Method Obsolete
-
Scott Garrigansept 2011 - Chris Anderson article on role of models (samples) when you have data on the whole population! When is theory useful? For prediction?
Science Data statistics theory philosophy information prediction
-
-
But faced with massive data, this approach to science — hypothesize, model, test — is becoming obsolete.
-
There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.
-
In February, the National Science Foundation announced the Cluster Exploratory
-
-
-
Learning to use a "computer" of this scale may be challenging. But the opportunity is great: The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world
-
-
Glenda Bakerarticle discussing whether math models can replace other tools for understanding the world.
-
sociology. Forget taxonomy, ontology, and psychology. Who knows why people do what they do? The point is they do it, and we can track and measure it with unprecedented fidelity. With enough data, the numbers speak for themselves.
-
The big target here isn't advertising, though. It's science. The scientific method is built around testable hypotheses. These models, for the most part, are systems visualized in the minds of scientists. The models are then tested, and experiments confirm or falsify theoretical models of how the world works.
-
But faced with massive data, this approach to science — hypothesize, model, test — is becoming obsolete.
-
Petabytes allow us to say: "Correlation is enough."
-
There's no reason to cling to our old ways. It's time to ask: What can science learn from Google?
-
It's time to ask: What can science learn from Google?
-
-
17 Sep 11
-
Sensors everywhere. Infinite storage. Clouds of processors. Our ability to capture, warehouse, and understand massive amounts of data is changing science, medicine, business, and technology. As our collection of facts and figures grows, so will the opportunity to find answers to fundamental questions. Because in the era of big data, more isn't just more. More is different.
-
-
22 Jul 11
-
07 Jul 11
-
06 Jul 11
-
20 Jun 11
-
04 Jun 11
aaron tayThe End of Theory: The Data Deluge Makes the Scientific Method Obsolete http://j.mp/loXRsD "All models are wrong, but some are useful."
-
17 May 11
-
07 May 11
-
07 Mar 11
-
Google's founding philosophy is that we don't know why this page is better than that one: If the statistics of incoming links say it is, that's good enough. No semantic or causal analysis is required. That's why Google can translate languages without actually "knowing" them (given equal corpus data, Google can translate Klingon into Farsi as easily as it can translate French into German). And why it can match ads to content without any knowledge or assumptions about the ads or the content.
-
The scientific method is built around testable hypotheses. These models, for the most part, are systems visualized in the minds of scientists. The models are then tested, and experiments confirm or falsify theoretical models of how the world works. This is the way science has worked for hundreds of years.
-
Once you have a model, you can connect the data sets with confidence. Data without a model is just noise.
-
Petabytes allow us to say: "Correlation is enough."
-
We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.
-
-
27 Feb 11
Patrick SavalleBut faced with massive data, this approach to science — hypothesize, model, test — is becoming obsolete. Consider physics: Newtonian models were crude approximations of the truth (wrong at the atomic level, but still useful). A hundred years ago, statisti
science scientific method models paradigm-shift scientific-method
-
24 Feb 11
Pablo StafforiniWith enough data, the numbers speak for themselves.
-
12 Feb 11
-
18 Jan 11
-
Son los hijos de la Edad Petabyte.
-
La Edad Petabyte es diferente porque es más diferente. Kilobytes fueron almacenados en disquetes. Megabytes fueron almacenados en los discos duros. Terabytes fueron almacenados en arrays de disco. Petabytes se almacenan en la nube. Mientras avanzábamos que la progresión, pasamos de la analogía carpeta a la analogía del gabinete a la analogía de la colección - y, en petabytes nos quedamos sin analogías de organización.
-
Este es un mundo donde las cantidades masivas de datos y matemáticas aplicadas reemplazar cualquier otra herramienta que pueda ser ejercida. A cabo con todas las teorías del comportamiento humano, de la lingüística a la sociología. Olvídese de la taxonomía, la ontología y la psicología. Quién sabe por qué las personas hacen lo que hacen? El punto es que lo hacen, y que puede realizar un seguimiento y medir con una fidelidad sin precedentes. Con suficientes datos, los números hablan por sí mismos.
-
-
17 Jan 11
-
10 Jan 11
-
-
It calls for an entirely different approach, one that requires us to lose the tether of data as something that can be visualized in its totality. It forces us to view data mathematically first and establish a context for it later.
-
-
-
"All models are wrong, but some are useful."
-
The End of Theory:
Essay: The Data Deluge Makes the Scientific Method Obsolete
<!-- pageType= magazinewide slug= pb_theory section= science subsection= discoveries headline= Petabyte Age: The Next Big Thing authorName= Chris Anderson creditType= illustration credit= marian bantjes -->"All models are wrong, but some are useful."
-
So proclaimed statistician George Box 30 years ago, and he was right
-
The Petabyte Age is different because more is different
-
At the petabyte scale, information is not a matter of simple three- and four-dimensional taxonomy and order but of dimensionally agnostic statistics. It calls for an entirely different approach, one that requires us to lose the tether of data as something that can be visualized in its totality
-
Peter Norvig, Google's research director, offered an update to George Box's maxim: "All models are wrong, and increasingly you can succeed without them."
-
The big target here isn't advertising, though. It's science.
-
Consider physics: Newtonian models were crude approximations of the truth (wrong at the atomic level, but still useful). A hundred years ago, statistically based quantum mechanics offered a better picture — but quantum mechanics is yet another model, and as such it, too, is flawed, no doubt a caricature of a more complex underlying reality.
-
The discovery of gene-protein interactions and other aspects of epigenetics has challenged the view of DNA as destiny and even introduced evidence that environment can influence inheritable traits, something once considered a genetic impossibility.
-
Petabytes allow us to say: "Correlation is enough."
-
This kind of thinking is poised to go mainstream. In February, the National Science Foundation announced the Cluster Exploratory, a program that funds research designed to run on a large-scale distributed computing platform developed by Google and IBM in conjunction with six pilot universities.
-
There's no reason to cling to our old ways. It's time to ask: What can science learn from Google?
-
-
13 Dec 10
-
18 Nov 10
-
08 Nov 10
-
There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.
-
There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing cluste
-
"All models are wrong, but some are useful."
So proclaimed statistician George Box 30 years ago, and he was right.
-
The End of Theory:
Essay: The Data Deluge Makes the Scientific Method Obsolete
<!-- pageType= magazinewide slug= pb_theory section= science subsection= discoveries headline= Petabyte Age: The Next Big Thing authorName= Chris Anderson creditType= illustration credit= marian bantjes -->"All models are wrong, but some are useful."
So procla
-
The Petabyte Age
-
Petabytes are stored in the cloud
-
Google's founding philosophy is that we don't know why this page is better than that one: If the statistics of incoming links say it is, that's good enough. No semantic or causal analysis is required.
-
There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.
-
The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Correlation supersedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic explanation at all.
There's no reason to cling to our old ways. It's time to ask: What can science learn from Google?
-
Public Stiky Notes
Page Comments
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.