an example of how consultation works.
This link has been bookmarked by 162 people and liked by 2 people. It was first bookmarked on 01 Jul 2010, by someone privately.
-
11 May 12
-
03 May 12
-
06 Oct 11
-
Dunbar knew that scientists often don’t think the way the textbooks say they are supposed to.
-
Philosophy of science is about as useful to scientists as ornithology is to birds
-
The scientists were looking for X, but they found Y.
-
-
05 Sep 11
-
29 Jun 11
-
01 Dec 10
Maarten HoekstraKevin Dunbar is a researcher who studies how scientists study things — how they fail and succeed. In the early 1990s, he began an unprecedented research project: observing four biochemistry labs at Stanford University. Philosophers have long theorized about how science happens, but Dunbar wanted to get beyond theory. He wasn’t satisfied with abstract models of the scientific method — that seven-step process we teach schoolkids before the science fair — or the dogmatic faith scientists place in logic and objectivity. Dunbar knew that scientists often don’t think the way the textbooks say they are supposed to. He suspected that all those philosophers of science — from Aristotle to Karl Popper — had missed something important about what goes on in the lab. (As Richard Feynman famously quipped, “Philosophy of science is about as useful to scientists as ornithology is to birds.”) So Dunbar decided to launch an “in vivo” investigation, attempting to learn from the messiness of real experiments.
scientists science confirmation bias failure scoop neuroscience research psychology Dunbar study diversity
-
20 Oct 10
-
31 Aug 10
-
02 Aug 10
-
27 Jun 10
-
14 Jun 10
-
07 Jun 10
-
31 May 10
-
24 May 10
-
23 May 10
-
09 May 10
-
18 Mar 10
Robert Stephenshttp://diigo.com/09ylt
-
But they made the scope too sensitive.
-
Their telescope was a failure.
-
In the early 1990s, he began an unprecedented research project: observing four biochemistry labs at Stanford University.
-
So Dunbar decided to launch an “in vivo” investigation, attempting to learn from the messiness of real experiments.
-
Dunbar came away from his in vivo studies with an unsettling insight: Science is a deeply frustrating pursuit.
-
Although the researchers were mostly using established techniques, more than 50 percent of their data was unexpected. (In some labs, the figure exceeded 75 percent.
-
The details always changed, but the story remained the same: The scientists were looking for X, but they found Y.
-
However, when experiments were observed up close — and Dunbar interviewed the scientists about even the most trifling details — this idealized version of the lab fell apart, replaced by an endless supply of disappointing surprises.
-
“These weren’t sloppy people,” Dunbar says. “They were working in some of the finest labs in the world. But experiments rarely tell us what we think they’re going to tell us. That’s the dirty secret of science.”
-
How did the researchers cope with all this unexpected data? How did they deal with so much failure? Dunbar realized that the vast majority of people in the lab followed the same basic strategy. First, they would blame the method. The surprising finding was classified as a mere mistake; perhaps a machine malfunctioned or an enzyme had gone stale. “The scientists were trying to explain away what they didn’t understand,” Dunbar says. “It’s as if they didn’t want to believe it.”
-
People have to pick and choose what’s interesting and what’s not, but they often choose badly.
-
The scientists had discovered a new fact, but they called it a failure.
-
the real reason researchers automatically assume that every unexpected result is a stupid mistake — is rooted in the way the human brain works.
-
The fact is, we carefully edit our reality, searching for evidence that confirms what we already believe. Although we pretend we’re empiricists — our views dictated by nothing but the facts — we’re actually blinkered, especially when it comes to information that contradicts our theories. The problem with science, then, isn’t that most experiments fail — it’s that most failures are ignored.
-
But there’s another region of the brain that can be activated as we go about editing reality. It’s called the dorsolateral prefrontal cortex, or DLPFC. It’s located just behind the forehead and is one of the last brain areas to develop in young adults. It plays a crucial role in suppressing so-called unwanted representations, getting rid of those thoughts that don’t square with our preconceptions. For scientists, it’s a problem.
-
If the ACC is the “Oh shit!” circuit, the DLPFC is the Delete key. When the ACC and DLPFC “turn on together, people aren’t just noticing that something doesn’t look right,” Dunbar says. “They’re also inhibiting that information.”
-
Belief, in other words, is a kind of blindness.
-
Check Your Assumptions
Ask yourself why this result feels like a failure. What theory does it contradict? Maybe the hypothesis failed, not the experiment.
2 Seek Out the IgnorantTalk to people who are unfamiliar with your experiment. Explaining your work in simple terms may help you see it in a new light.
3 Encourage DiversityIf everyone working on a problem speaks the same language, then everyone has the same set of assumptions.
4 Beware of Failure-BlindnessIt’s normal to filter out information that contradicts our preconceptions. The only way to avoid that bias is to be aware of it.
-
if Einstein had gotten tenure at an elite German university, he would have become just another physics professor with a vested interest in the space-time status quo. He would never have noticed the anomalies that led him to develop the theory of relativity.
-
There are advantages to thinking on the margin.
-
the romantic narrative of the brilliant and perceptive newcomer left something out
-
The diverse lab, in contrast, mulled the problem at a group meeting. None of the scientists were protein experts, so they began a wide-ranging discussion of possible solutions. At first, the conversation seemed rather useless. But then, as the chemists traded ideas with the biologists and the biologists bounced ideas off the med students, potential answers began to emerge. “After another 10 minutes of talking, the protein problem was solved,” Dunbar says. “They made it look easy.”
-
The diverse lab, in contrast, mulled the problem at a group meeting. None of the scientists were protein experts, so they began a wide-ranging discussion of possible solutions. At first, the conversation seemed rather useless. But then, as the chemists traded ideas with the biologists and the biologists bounced ideas off the med students, potential answers began to emerge. “After another 10 minutes of talking, the protein problem was solved,” Dunbar says. “They made it look easy.”
-
generated a distinct type of interaction in which the scientists were forced to rely on metaphors and analogies to express themselves.
-
These abstractions proved essential for problem-solving
-
This is why other people are so helpful: They shock us out of our cognitive box.
-
What turned out to be so important, of course, was the unexpected result, the experimental error that felt like a failure.
-
the breakthrough was the result of a casual conversation with an outsider.
-
-
09 Mar 10
-
05 Mar 10
-
04 Mar 10
-
03 Mar 10
Vahid MasrourHow does this interect with the "individual search for truth" bahá'í principle?
-
Add Sticky NoteDunbar found that most new scientific ideas emerged from lab meetings, those weekly sessions in which people publicly present their data. Interestingly, the most important element of the lab meeting wasn’t the presentation — it was the debate that followed. Dunbar observed that the skeptical (and sometimes heated) questions asked during a group session frequently triggered breakthroughs, as the scientists were forced to reconsider data they’d previously ignored. The new theory was a product of spontaneous conversation,
-
-
the second lab lacked a specialized language that everyone could understand
-
It’s not until we talk to a colleague or translate our idea into an analogy that we glimpse the meaning in our mistake.
-
-
02 Mar 10
-
26 Feb 10
-
25 Feb 10
-
24 Feb 10
ujyvbkhHow To Fail Screw ups, disasters, misfires, flops. Why losing big can be a winning strategy. • Accept Defeat: The Neuroscience of Screwing Up Stay
-
17 Feb 10
-
16 Feb 10
Simon CarrGreat article on "The Neuroscience of Screwing Up" http://bit.ly/a1RoAv (via Wired Magazine)
-
15 Feb 10
-
14 Feb 10
Fire crackerWhile the scientific process is typically seen as a lonely pursuit — researchers solve problems by themselves — Dunbar found that most new scientific ideas emerged from lab meetings, those weekly sessions in which people publicly present their data. Inter
-
August JacksonWhile the students were watching the footage, Dunbar asked them to select the more accurate representation of gravity. Not surprisingly, undergraduates without a physics background disagreed with Galileo. (Intuitively, we’re all Aristotelians.) They found
-
Jacque Priorvery relevant, especially considering how last year turned out and in relation to honours/qute mag/un mag/hockey/life/everything
-
Suvi KorhonenDunbarin tutkimukset tieteensosiologiassa/filosofiassa.
science psychology neuroscience failure research learning innovation creativity dunbar wired tutkimus tieto methodology
-
09 Feb 10
-
01 Feb 10
-
Although the researchers were mostly using established techniques, more than 50 percent of their data was unexpected.
-
But experiments rarely tell us what we think they’re going to tell us. That’s the dirty secret of science
-
The scientists had discovered a new fact, but they called it a failure.
The reason we’re so resistant to anomalous information — the real reason researchers automatically assume that every unexpected result is a stupid mistake — is rooted in the way the human brain works. Over the past few decades, psychologists have dismantled the myth of objectivity.
-
The problem with science, then, isn’t that most experiments fail — it’s that most failures are ignored.
-
But there’s another region of the brain that can be activated as we go about editing reality. It’s called the dorsolateral prefrontal cortex, or DLPFC. It’s located just behind the forehead and is one of the last brain areas to develop in young adults. It plays a crucial role in suppressing so-called unwanted representations, getting rid of those thoughts that don’t square with our preconceptions. For scientists, it’s a problem.
-
The lesson is that not all data is created equal in our mind’s eye: When it comes to interpreting our experiments, we see what we want to see and disregard the rest.
-
Belief, in other words, is a kind of blindness.
-
How to Learn
From FailureToo often, we assume that a failed experiment is a wasted effort. But not all anomalies are useless. Here’s how to make the most of them. —J.L.
1 Check Your AssumptionsAsk yourself why this result feels like a failure. What theory does it contradict? Maybe the hypothesis failed, not the experiment.
2 Seek Out the IgnorantTalk to people who are unfamiliar with your experiment. Explaining your work in simple terms may help you see it in a new light.
3 Encourage DiversityIf everyone working on a problem speaks the same language, then everyone has the same set of assumptions.
4 Beware of Failure-BlindnessIt’s normal to filter out information that contradicts our preconceptions. The only way to avoid that bias is to be aware of it.
-
But his larger point is crucial: There are advantages to thinking on the margin. When we look at a problem from the outside, we’re more likely to notice what doesn’t work.
-
Modern science is populated by expert insiders, schooled in narrow disciplines. Researchers have all studied the same thick textbooks, which make the world of fact seem settled.
-
to argue that the only scientists capable of acknowledging the anomalies — and thus shifting paradigms and starting revolutions — are “either very young or very new to the field.” In other words, they are classic outsiders, naive and untenured. They aren’t inhibited from noticing the failures that point toward new possibilities.
-
Dunbar found that most new scientific ideas emerged from lab meetings, those weekly sessions in which people publicly present their data. Interestingly, the most important element of the lab meeting wasn’t the presentation — it was the debate that followed.
-
questions asked during a group session frequently triggered breakthroughs
-
The new theory was a product of spontaneous conversation, not solitude; a single bracing query was enough to turn scientists into temporary outsiders, able to look anew at their own work.
-
he found that the intellectual mix generated a distinct type of interaction in which the scientists were forced to rely on metaphors and analogies to express themselves.
-
These abstractions proved essential for problem-solving, as they encouraged the scientists to reconsider their assumptions.
-
This is why other people are so helpful: They shock us out of our cognitive box.
-
It’s not until we talk to a colleague or translate our idea into an analogy that we glimpse the meaning in our mistake.
-
There’s no success quite like failure.
-
-
26 Jan 10
-
Kevin Dunbar is a researcher who studies how scientists study things — how they fail and succeed.
-
Dunbar came away from his in vivo studies with an unsettling insight: Science is a deeply frustrating pursuit. Although the researchers were mostly using established techniques, more than 50 percent of their data was unexpected. (In some labs, the figure exceeded 75 percent.) “The scientists had these elaborate theories about what was supposed to happen,” Dunbar says. “But the results kept contradicting their theories. It wasn’t uncommon for someone to spend a month on a project and then just discard all their data because the data didn’t make sense.”
-
Dunbar was fascinated by these statistics. The scientific process, after all, is supposed to be an orderly pursuit of the truth, full of elegant hypotheses and control variables. (Twentieth-century science philosopher Thomas Kuhn, for instance, defined normal science as the kind of research in which “everything but the most esoteric detail of the result is known in advance.”) However, when experiments were observed up close — and Dunbar interviewed the scientists about even the most trifling details — this idealized version of the lab fell apart, replaced by an endless supply of disappointing surprises. There were models that didn’t work and data that couldn’t be replicated and simple studies riddled with anomalies. “These weren’t sloppy people,” Dunbar says. “They were working in some of the finest labs in the world. But experiments rarely tell us what we think they’re going to tell us. That’s the dirty secret of science.”
-
First, they would blame the method. The surprising finding was classified as a mere mistake; perhaps a machine malfunctioned or an enzyme had gone stale. “The scientists were trying to explain away what they didn’t understand,” Dunbar says. “It’s as if they didn’t want to believe it.”
-
The experiment would then be carefully repeated. Sometimes, the weird blip would disappear, in which case the problem was solved. But the weirdness usually remained, an anomaly that wouldn’t go away.
-
According to Dunbar, even after scientists had generated their “error” multiple times — it was a consistent inconsistency — they might fail to follow it up. “Given the amount of unexpected data in science, it’s just not feasible to pursue everything,” Dunbar says. “People have to pick and choose what’s interesting and what’s not, but they often choose badly.” And so the result was tossed aside, filed in a quickly forgotten notebook. The scientists had discovered a new fact, but they called it a failure.
-
The reason we’re so resistant to anomalous information — the real reason researchers automatically assume that every unexpected result is a stupid mistake — is rooted in the way the human brain works. Over the past few decades, psychologists have dismantled the myth of objectivity. The fact is, we carefully edit our reality, searching for evidence that confirms what we already believe. Although we pretend we’re empiricists — our views dictated by nothing but the facts — we’re actually blinkered, especially when it comes to information that contradicts our theories. The problem with science, then, isn’t that most experiments fail — it’s that most failures are ignored.
-
In one 2003 study, he had undergraduates at Dartmouth College watch a couple of short videos of two different-size balls falling. The first clip showed the two balls falling at the same rate. The second clip showed the larger ball falling at a faster rate. The footage was a reconstruction of the famous (and probably apocryphal) experiment performed by Galileo, in which he dropped cannonballs of different sizes from the Tower of Pisa. Galileo’s metal balls all landed at the exact same time — a refutation of Aristotle, who claimed that heavier objects fell faster.
-
While the students were watching the footage, Dunbar asked them to select the more accurate representation of gravity. Not surprisingly, undergraduates without a physics background disagreed with Galileo. (Intuitively, we’re all Aristotelians.) They found the two balls falling at the same rate to be deeply unrealistic, despite the fact that it’s how objects actually behave. Furthermore, when Dunbar monitored the subjects in an fMRI machine, he found that showing non-physics majors the correct video triggered a particular pattern of brain activation: There was a squirt of blood to the anterior cingulate cortex, a collar of tissue located in the center of the brain. The ACC is typically associated with the perception of errors and contradictions — neuroscientists often refer to it as part of the “Oh shit!” circuit — so it makes sense that it would be turned on when we watch a video of something that seems wrong.
-
But Dunbar also conducted the experiment with physics majors. As expected, their education enabled them to see the error, and for them it was the inaccurate video that triggered the ACC.
-
But there’s another region of the brain that can be activated as we go about editing reality. It’s called the dorsolateral prefrontal cortex, or DLPFC. It’s located just behind the forehead and is one of the last brain areas to develop in young adults. It plays a crucial role in suppressing so-called unwanted representations, getting rid of those thoughts that don’t square with our preconceptions. For scientists, it’s a problem.
-
When physics students saw the Aristotelian video with the aberrant balls, their DLPFCs kicked into gear and they quickly deleted the image from their consciousness. In most contexts, this act of editing is an essential cognitive skill. (When the DLPFC is damaged, people often struggle to pay attention, since they can’t filter out irrelevant stimuli.) However, when it comes to noticing anomalies, an efficient prefrontal cortex can actually be a serious liability. The DLPFC is constantly censoring the world, erasing facts from our experience. If the ACC is the “Oh shit!” circuit, the DLPFC is the Delete key. When the ACC and DLPFC “turn on together, people aren’t just noticing that something doesn’t look right,” Dunbar says. “They’re also inhibiting that information.”
-
not all data is created equal in our mind’s eye: When it comes to interpreting our experiments, we see what we want to see and disregard the rest. The physics students, for instance, didn’t watch the video and wonder whether Galileo might be wrong. Instead, they put their trust in theory, tuning out whatever it couldn’t explain. Belief, in other words, is a kind of blindness.
-
Too often, we assume that a failed experiment is a wasted effort. But not all anomalies are useless. Here’s how to make the most of them.
-
How to Learn
From Failure -
4 Beware of Failure-Blindness
It’s normal to filter out information that contradicts our preconceptions. The only way to avoid that bias is to be aware of it.
-
1 Check Your Assumptions
Ask yourself why this result feels like a failure. What theory does it contradict? Maybe the hypothesis failed, not the experiment.
-
2 Seek Out the Ignorant
Talk to people who are unfamiliar with your experiment. Explaining your work in simple terms may help you see it in a new light.
-
3 Encourage Diversity
If everyone working on a problem speaks the same language, then everyone has the same set of assumptions.
-
If humans — scientists included — are apt to cling to their beliefs, why is science so successful? How do our theories ever change? How do we learn to reinterpret a failure so we can see the answer?
-
In 1918, sociologist Thorstein Veblen was commissioned by a popular magazine devoted to American Jewry to write an essay on how Jewish “intellectual productivity” would be changed if Jews were given a homeland.
-
He argued instead that the scientific achievements of Jews — at the time, Albert Einstein was about to win the Nobel Prize and Sigmund Freud was a best-selling author — were due largely to their marginal status. In other words, persecution wasn’t holding the Jewish community back — it was pushing it forward.
-
The reason, according to Veblen, was that Jews were perpetual outsiders, which filled them with a “skeptical animus.” Because they had no vested interest in “the alien lines of gentile inquiry,” they were able to question everything, even the most cherished of assumptions. Just look at Einstein, who did much of his most radical work as a lowly patent clerk in Bern, Switzerland. According to Veblen’s logic, if Einstein had gotten tenure at an elite German university, he would have become just another physics professor with a vested interest in the space-time status quo. He would never have noticed the anomalies that led him to develop the theory of relativity.
-
There are advantages to thinking on the margin. When we look at a problem from the outside, we’re more likely to notice what doesn’t work. Instead of suppressing the unexpected, shunting it aside with our “Oh shit!” circuit and Delete key, we can take the mistake seriously. A new theory emerges from the ashes of our surprise.
-
Modern science is populated by expert insiders, schooled in narrow disciplines. Researchers have all studied the same thick textbooks, which make the world of fact seem settled. This led Kuhn, the philosopher of science, to argue that the only scientists capable of acknowledging the anomalies — and thus shifting paradigms and starting revolutions — are “either very young or very new to the field.” In other words, they are classic outsiders, naive and untenured. They aren’t inhibited from noticing the failures that point toward new possibilities.
-
But Dunbar, who had spent all those years watching Stanford scientists struggle and fail, realized that the romantic narrative of the brilliant and perceptive newcomer left something out.
-
While the scientific process is typically seen as a lonely pursuit — researchers solve problems by themselves — Dunbar found that most new scientific ideas emerged from lab meetings, those weekly sessions in which people publicly present their data. Interestingly, the most important element of the lab meeting wasn’t the presentation — it was the debate that followed. Dunbar observed that the skeptical (and sometimes heated) questions asked during a group session frequently triggered breakthroughs, as the scientists were forced to reconsider data they’d previously ignored. The new theory was a product of spontaneous conversation, not solitude; a single bracing query was enough to turn scientists into temporary outsiders, able to look anew at their own work.
-
intellectual mix generated a distinct type of interaction in which the scientists were forced to rely on metaphors and analogies to express themselves.
-
These abstractions proved essential for problem-solving, as they encouraged the scientists to reconsider their assumptions. Having to explain the problem to someone else forced them to think, if only for a moment, like an intellectual on the margins, filled with self-skepticism.
-
This is why other people are so helpful: They shock us out of our cognitive box. “I saw this happen all the time,” Dunbar says. “A scientist would be trying to describe their approach, and they’d be getting a little defensive, and then they’d get this quizzical look on their face. It was like they’d finally understood what was important.”
-
What turned out to be so important, of course, was the unexpected result, the experimental error that felt like a failure. The answer had been there all along — it was just obscured by the imperfect theory, rendered invisible by our small-minded brain. It’s not until we talk to a colleague or translate our idea into an analogy that we glimpse the meaning in our mistake. Bob Dylan, in other words, was right: There’s no success quite like failure.
-
Their failure was the answer to a different question.
-
-
-
25 Jan 10
arobertThis article should be required reading for lawyers regarding confirmation bias. It generated several thoughts: (1) the need to create a google search engine for the blawgs I like to see if they have discussed this article; (2) the necessity of having br
science neuroscience psychology research failure learning innovation creativity wired lawpracticecle blog delicious
-
22 Jan 10
Christian DScrew ups, disasters, misfires, flops. Why losing big can be a winning strategy.
-
19 Jan 10
-
When Dunbar reviewed the transcripts of the meeting, he found that the intellectual mix generated a distinct type of interaction in which the scientists were forced to rely on metaphors and analogies to express themselves. (That’s because, unlike the E. coli group, the second lab lacked a specialized language that everyone could understand.) These abstractions proved essential for problem-solving, as they encouraged the scientists to reconsider their assumptions. Having to explain the problem to someone else forced them to think, if only for a moment, like an intellectual on the margins, filled with self-skepticism.
-
-
17 Jan 10
David WilcoxEdge people, different ways of thinking, challenging hypotheses are all important to innovation
-
16 Jan 10
-
13 Jan 10
-
12 Jan 10
-
Paul OppenheimerHow not to learn from failure
science research education information academia brain psychology neuroscience article learning mind articles creativity productivity belief innovation behavior theory bias diversity read wired failure toread mistakes mistake decisionmaking discovery fail
-
11 Jan 10
Adam Crowe'Dunbar found that most new scientific ideas emerged from lab meetings, those weekly sessions in which people publicly present their data. When Dunbar reviewed the transcripts of the meeting, he found that the intellectual mix generated a distinct type of
science fallibilism skepticism criticaldistance abstraction metaphor context ideas #diversity
-
08 Jan 10
-
06 Jan 10
-
05 Jan 10
-
04 Jan 10
-
03 Jan 10
-
02 Jan 10
-
01 Jan 10
-
ken .Kevin Dunbar - "experiments rarely tell us what we think they’re going to tell us. That’s the dirty secret of science" - seven step scientific method - theories and observations (but +50% of observations 'fail' expectations, generate anomalies, discarded)
belief discovery edge failure knowledge intuition learning po progress quotes science theory wtf
-
Diane Courtxperiments rarely tell us what we expect
science neuroscience research failure learning creativity innovation education
-
Katie DayThe messy way science works... through lots of failure...
<< When Dunbar reviewed the transcripts of the meeting, he found that the intellectual mix generated a distinct type of interaction in which the scientists were forced to rely on metaphors and analscience inquiry_based_learning failure metaphors scientists cognitive_science imported_from_delicious
-
31 Dec 09
-
Chris WillmottHow did the researchers cope with all this unexpected data? How did they deal with so much failure? Dunbar realized that the vast majority of people in the lab followed the same basic strategy. First, they would blame the method. The surprising finding wa
science research neuroscience brain Wired PUS research_methods
-
30 Dec 09
-
"This is why other people are so helpful: They shock us out of our cognitive box."
johah_lehrer wired kevin_dunbar neuroscience failure dissonant_data cognition DLPFC editing_reality metaphors
-
28 Dec 09
-
hldnmyown unfortunatelyHow To Fail Screw ups, disasters, misfires, flops. Why losing big can be a winning strategy. • Accept Defeat: The Neuroscience of Screwing Up Stay
-
27 Dec 09
-
-
There’s no success quite like failure.
-
-
26 Dec 09
-
25 Dec 09
-
24 Dec 09
-
-
Kevin Dunbar is a researcher who studies how scientists study things — how they fail and succeed. In the early 1990s, he began an unprecedented research project: observing four biochemistry labs at Stanford University. Philosophers have long theorized about how science happens, but Dunbar wanted to get beyond theory. He wasn’t satisfied with abstract models of the scientific method — that seven-step process we teach schoolkids before the science fair — or the dogmatic faith scientists place in logic and objectivity. Dunbar knew that scientists often don’t think the way the textbooks say they are supposed to. He suspected that all those philosophers of science — from Aristotle to Karl Popper — had missed something important about what goes on in the lab. (As Richard Feynman famously quipped, “Philosophy of science is about as useful to scientists as ornithology is to birds.”) So Dunbar decided to launch an “in vivo” investigation, attempting to learn from the messiness of real experiments.
-
Dunbar was fascinated by these statistics. The scientific process, after all, is supposed to be an orderly pursuit of the truth, full of elegant hypotheses and control variables. (Twentieth-century science philosopher Thomas Kuhn, for instance, defined normal science as the kind of research in which “everything but the most esoteric detail of the result is known in advance.”) However, when experiments were observed up close — and Dunbar interviewed the scientists about even the most trifling details — this idealized version of the lab fell apart, replaced by an endless supply of disappointing surprises. There were models that didn’t work and data that couldn’t be replicated and simple studies riddled with anomalies. “These weren’t sloppy people,” Dunbar says. “They were working in some of the finest labs in the world. But experiments rarely tell us what we think they’re going to tell us. That’s the dirty secret of science.”
-
How did the researchers cope with all this unexpected data? How did they deal with so much failure? Dunbar realized that the vast majority of people in the lab followed the same basic strategy. First, they would blame the method. The surprising finding was classified as a mere mistake; perhaps a machine malfunctioned or an enzyme had gone stale. “The scientists were trying to explain away what they didn’t understand,” Dunbar says. “It’s as if they didn’t want to believe it.”
-
This is when things get interesting. According to Dunbar, even after scientists had generated their “error” multiple times — it was a consistent inconsistency — they might fail to follow it up. “Given the amount of unexpected data in science, it’s just not feasible to pursue everything,” Dunbar says. “People have to pick and choose what’s interesting and what’s not, but they often choose badly.” And so the result was tossed aside, filed in a quickly forgotten notebook. The scientists had discovered a new fact, but they called it a failure.
-
The reason we’re so resistant to anomalous information — the real reason researchers automatically assume that every unexpected result is a stupid mistake — is rooted in the way the human brain works. Over the past few decades, psychologists have dismantled the myth of objectivity. The fact is, we carefully edit our reality, searching for evidence that confirms what we already believe. Although we pretend we’re empiricists — our views dictated by nothing but the facts — we’re actually blinkered, especially when it comes to information that contradicts our theories. The problem with science, then, isn’t that most experiments fail — it’s that most failures are ignored.
-
As he tried to further understand how people deal with dissonant data, Dunbar conducted some experiments of his own. In one 2003 study, he had undergraduates at Dartmouth College watch a couple of short videos of two different-size balls falling. The first clip showed the two balls falling at the same rate. The second clip showed the larger ball falling at a faster rate. The footage was a reconstruction of the famous (and probably apocryphal) experiment performed by Galileo, in which he dropped cannonballs of different sizes from the Tower of Pisa. Galileo’s metal balls all landed at the exact same time — a refutation of Aristotle, who claimed that heavier objects fell faster.
-
While the students were watching the footage, Dunbar asked them to select the more accurate representation of gravity. Not surprisingly, undergraduates without a physics background disagreed with Galileo. (Intuitively, we’re all Aristotelians.) They found the two balls falling at the same rate to be deeply unrealistic, despite the fact that it’s how objects actually behave. Furthermore, when Dunbar monitored the subjects in an fMRI machine, he found that showing non-physics majors the correct video triggered a particular pattern of brain activation: There was a squirt of blood to the anterior cingulate cortex, a collar of tissue located in the center of the brain. The ACC is typically associated with the perception of errors and contradictions — neuroscientists often refer to it as part of the “Oh shit!” circuit — so it makes sense that it would be turned on when we watch a video of something that seems wrong.
-
So far, so obvious: Most undergrads are scientifically illiterate. But Dunbar also conducted the experiment with physics majors. As expected, their education enabled them to see the error, and for them it was the inaccurate video that triggered the ACC.
-
But there’s another region of the brain that can be activated as we go about editing reality. It’s called the dorsolateral prefrontal cortex, or DLPFC. It’s located just behind the forehead and is one of the last brain areas to develop in young adults. It plays a crucial role in suppressing so-called unwanted representations, getting rid of those thoughts that don’t square with our preconceptions. For scientists, it’s a problem.
-
When physics students saw the Aristotelian video with the aberrant balls, their DLPFCs kicked into gear and they quickly deleted the image from their consciousness. In most contexts, this act of editing is an essential cognitive skill. (When the DLPFC is damaged, people often struggle to pay attention, since they can’t filter out irrelevant stimuli.) However, when it comes to noticing anomalies, an efficient prefrontal cortex can actually be a serious liability. The DLPFC is constantly censoring the world, erasing facts from our experience. If the ACC is the “Oh shit!” circuit, the DLPFC is the Delete key. When the ACC and DLPFC “turn on together, people aren’t just noticing that something doesn’t look right,” Dunbar says. “They’re also inhibiting that information.”
-
The lesson is that not all data is created equal in our mind’s eye: When it comes to interpreting our experiments, we see what we want to see and disregard the rest. The physics students, for instance, didn’t watch the video and wonder whether Galileo might be wrong. Instead, they put their trust in theory, tuning out whatever it couldn’t explain. Belief, in other words, is a kind of blindness.
-
There are advantages to thinking on the margin. When we look at a problem from the outside, we’re more likely to notice what doesn’t work. Instead of suppressing the unexpected, shunting it aside with our “Oh shit!” circuit and Delete key, we can take the mistake seriously. A new theory emerges from the ashes of our surprise.
-
Modern science is populated by expert insiders, schooled in narrow disciplines. Researchers have all studied the same thick textbooks, which make the world of fact seem settled. This led Kuhn, the philosopher of science, to argue that the only scientists capable of acknowledging the anomalies — and thus shifting paradigms and starting revolutions — are “either very young or very new to the field.” In other words, they are classic outsiders, naive and untenured. They aren’t inhibited from noticing the failures that point toward new possibilities.
-
While the scientific process is typically seen as a lonely pursuit — researchers solve problems by themselves — Dunbar found that most new scientific ideas emerged from lab meetings, those weekly sessions in which people publicly present their data. Interestingly, the most important element of the lab meeting wasn’t the presentation — it was the debate that followed. Dunbar observed that the skeptical (and sometimes heated) questions asked during a group session frequently triggered breakthroughs, as the scientists were forced to reconsider data they’d previously ignored. The new theory was a product of spontaneous conversation, not solitude; a single bracing query was enough to turn scientists into temporary outsiders, able to look anew at their own work.
-
But not every lab meeting was equally effective. Dunbar tells the story of two labs that both ran into the same experimental problem: The proteins they were trying to measure were sticking to a filter, making it impossible to analyze the data. “One of the labs was full of people from different backgrounds,” Dunbar says. “They had biochemists and molecular biologists and geneticists and students in medical school.” The other lab, in contrast, was made up of E. coli experts. “They knew more about E. coli than anyone else, but that was what they knew,” he says. Dunbar watched how each of these labs dealt with their protein problem. The E. coli group took a brute-force approach, spending several weeks methodically testing various fixes. “It was extremely inefficient,” Dunbar says. “They eventually solved it, but they wasted a lot of valuable time.”
-
The diverse lab, in contrast, mulled the problem at a group meeting. None of the scientists were protein experts, so they began a wide-ranging discussion of possible solutions. At first, the conversation seemed rather useless. But then, as the chemists traded ideas with the biologists and the biologists bounced ideas off the med students, potential answers began to emerge. “After another 10 minutes of talking, the protein problem was solved,” Dunbar says. “They made it look easy.”
-
When Dunbar reviewed the transcripts of the meeting, he found that the intellectual mix generated a distinct type of interaction in which the scientists were forced to rely on metaphors and analogies to express themselves. (That’s because, unlike the E. coli group, the second lab lacked a specialized language that everyone could understand.) These abstractions proved essential for problem-solving, as they encouraged the scientists to reconsider their assumptions. Having to explain the problem to someone else forced them to think, if only for a moment, like an intellectual on the margins, filled with self-skepticism.
-
This is why other people are so helpful: They shock us out of our cognitive box. “I saw this happen all the time,” Dunbar says. “A scientist would be trying to describe their approach, and they’d be getting a little defensive, and then they’d get this quizzical look on their face. It was like they’d finally understood what was important.”
-
What turned out to be so important, of course, was the unexpected result, the experimental error that felt like a failure. The answer had been there all along — it was just obscured by the imperfect theory, rendered invisible by our small-minded brain. It’s not until we talk to a colleague or translate our idea into an analogy that we glimpse the meaning in our mistake. Bob Dylan, in other words, was right: There’s no success quite like failure.
-
-
23 Dec 09
Public Stiky Notes
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.