This link has been bookmarked by 200 people . It was first bookmarked on 18 Sep 2008, by Gerhard Stoltz.
-
11 Jan 13
babsyco babsycoThe 4th Quadrant - and Edge.org article by Taleb.
-
06 Oct 12
-
Statistical and applied probabilistic knowledge is the core of knowledge; statistics is what tells you if something is true, false, or merely anecdotal; it is the "logic of science"; it is the instrument of risk-taking; it is the applied tools of epistemology
-
Black Swans, the highly improbable and unpredictable events that have massive impact
-
"no true statisticians", merely people using statistics either without understanding them, or in a self-serving manner. "The current subprime crisis did wonders to help me drill my point about the limits of statistically driven claims,"
-
"the banking system, betting against Black Swans, has lost over 1 Trillion dollars (so far), more than was ever made in the history of banking"
-
A map is a useful thing because you know where you are safe and where your knowledge is questionable.
-
So I drew for the Edge readers a tableau showing the boundaries where statistics works well and where it is questionable or unreliable.
-
Clearly, with current International Monetary Fund estimates of the costs of the 2007-2008 subprime crisis, the banking system seems to have lost more on risk taking (from the failures of quantitative risk management) than every penny banks ever earned taking risks.
-
But it was easy to see from the past that the pilot did not have the qualifications to fly the plane and was using the wrong navigation tools: The same happened in 1983 with money center banks losing cumulatively every penny ever made, and in 1991-1992 when the Savings and Loans industry became history.
-
It appears that financial institutions earn money on transactions (say fees on your mother-in-law's checking account) and lose everything taking risks they don't understand.
-
And we are beyond suckers: not only, for socio-economic and other nonlinear, complicated variables, we are riding in a bus driven a blindfolded driver, but we refuse to acknowledge it in spite of the evidence, which to me is a pathological problem with academia.
-
After 1998, when a "Nobel-crowned" collection of people (and the crème de la crème of the financial economics establishment) blew up Long Term Capital Management, a hedge fund, because the "scientific" methods they used misestimated the role of the rare event, such methodologies and such claims on understanding risks of rare events should have been discredited. Yet the Fed helped their bailout and exposure to rare events (and model error) patently increased exponentially (as we can see from banks' swelling portfolios of derivatives that we do not understand).
-
Are we using models of uncertainty to produce certainties?
-
Recently, the American Statistical Association had a special panel session on the "black swan" concept at the annual Joint Statistical Meeting in Denver last August. They insistently made a distinction between the "statisticians" (those who deal with the subject itself and design the tools and methods) and those in other fields who pick up statistical tools from textbooks without really understanding them. For them it is a problem with statistical education and half-baked expertise. Alas, this category of blind users includes regulators and risk managers, whom I accuse of creating more risk than they reduce.
-
Indeed, I am moving on: my new project is about methods on how to domesticate the unknown, exploit randomness, figure out how to live in a world we don't understand very well. While most human thought (particularly since the enlightenment) has focused us on how to turn knowledge into decisions, my new mission is to build methods to turn lack of information, lack of understanding, and lack of "knowledge" into decisions—how, as we will see, not to be a "turkey".
-
This piece has a technical appendix that presents mathematical points and empirical evidence. (See link below.) It includes a battery of tests showing that no known conventional tool can allow us to make precise statistical claims in the Fourth Quadrant. While in the past I limited myself to citing research papers, and evidence compiled by others (a less risky trade), here I got hold of more than 20 million pieces of data (includes 98% of the corresponding macroeconomics values of transacted daily, weekly, and monthly variables for the last 40 years) and redid a systematic analysis that includes recent years.
-
My anger with "empirical" claims in risk management does not come from research. It comes from spending twenty tense (but entertaining) years taking risky decisions in the real world managing portfolios of complex derivatives, with payoffs that depend on higher order statistical properties —and you quickly realize that a certain class of relationships that "look good" in research papers almost never replicate in real life (in spite of the papers making some claims with a "p" close to infallible). But that is not the main problem with research.
-
So the central lesson from decision-making (as opposed to working with data on a computer or bickering about logical constructions) is the following: it is the exposure (or payoff) that creates the complexity —and the opportunities and dangers— not so much the knowledge ( i.e., statistical distribution, model representation, etc.). In some situations, you can be extremely wrong and be fine, in others you can be slightly wrong and explode.
-
So knowledge (i.e., if some statement is "true" or "false") matters little, very little in many situations. In the real world, there are very few situations where what you do and your belief if some statement is true or false naively map into each other. Some decisions require vastly more caution than others—or highly more drastic confidence intervals.
-
You need evidence of safety—not evidence of lack of safety— a central asymmetry that affects us with rare events. This asymmetry in skepticism makes it easy to draw a map of danger spots.
-
Because, not only have economists been unable to prove that their models work, but no one managed to prove that the use of a model that does not work is neutral, that it does not increase blind risk taking, hence the accumulation of hidden risks.
-
No known econometric statistical method can capture the probability of the event with any remotely acceptable accuracy (except, of course, in hindsight, and "on paper").
-
trade-weighted data
-
A then-Princeton economist called Ben Bernanke made a pronouncement in late 2004 about the "new moderation" in economic life: the world getting more and more stable—before becoming the Chairman of the Federal Reserve. Yet the system was getting riskier and riskier as we were turkey-style sitting on more and more barrels of dynamite—and Prof. Bernanke's predecessor the former Federal Reserve Chairman Alan Greenspan was systematically increasing the hidden risks in the system, making us all more vulnerable to blowups.
-
with graphs, jargon, curves, the kind of facade-of-knowledge that you find in economics textbooks
-
narrative fallacy
-
This is the find of glib, snake-oil facade of knowledge—even more dangerous because of the mathematics
-
I have nothing against economists: you should let them entertain each others with their theories and elegant mathematics, and help keep college students inside buildings. But beware: they can be plain wrong, yet frame things in a way to make you feel stupid arguing with them. So make sure you do not give any of them risk-management responsibilities.
-
Things are made simple by the following. There are two distinct types of decisions, and two distinct classes of randomness.
-
The first type of decisions is simple, "binary", i.e. you just care if something is true or false.
-
The second type of decisions is more complex. You do not just care of the frequency—but of the impact as well, or, even more complex, some function of the impact.
-
When you invest you do not care how many times you make or lose, you care about the expectation: how many times you make or lose times the amount made or lost.
-
There are two classes of probability domains—very distinct qualitatively and quantitatively. The first, thin-tailed: Mediocristan", the second, thick tailed Extremistan.
-
In Mediocristan, exceptions occur but don't carry large consequences. Add the heaviest person on the planet to a sample of 1000. The total weight would barely change. In Extremistan, exceptions can be everything (they will eventually, in time, represent everything). Add Bill Gates to your sample: the wealth will jump by a factor of >100,000. So, in Mediocristan, large deviations occur but they are not consequential—unlike Extremistan.
-
Mediocristan corresponds to "random walk" style randomness that you tend to find in regular textbooks (and in popular books on randomness). Extremistan corresponds to a "random jump" one. The first kind I can call "Gaussian-Poisson", the second "fractal" or Mandelbrotian (after the works of the great Benoit Mandelbrot linking it to the geometry of nature).
-
But note here an epistemological question: there is a category of "I don't know" that I also bundle in Extremistan for the sake of decision making—simply because I don't know much about the probabilistic structure or the role of large events.
-
Now it lets see where the traps are:
-
Simple binary decisions, in Mediocristan: Statistics does wonders. These situations are, unfortunately, more common in academia, laboratories, and games than real life—what I call the
-
"ludic fallacy"
-
In other words, these are the situations in casinos, games, dice, and we tend to study them because we are successful in modeling them.
-
Complex decisions in Extremistan: Welcome to the Black Swan domain. Here is where your limits are. Do not base your decisions on statistically based claims. Or, alternatively, try to move your exposure type to make it third-quadrant style ("clipping tails").
-
-
13 Aug 12
-
26 Jul 12
B.L. Ochman@Hamlet Thanks for the Retweet. Probably the best blog post I've read in ages. The word NEEDS to spread on this: http://tinyurl.com/6dngnx
-
20 Jun 12
-
19 May 12
-
06 May 12
-
THE FOURTH QUADRANT: A MAP OF THE LIMITS OF STATISTICS [9.15.08]
By Nassim Nicholas Taleb
-
-
25 Mar 12
-
17 Jan 12
-
29 Mar 11
-
09 Feb 11
-
30 Jan 11
-
16 Oct 10
-
04 Oct 10
-
08 Sep 10
-
19 May 10
-
14 May 10
-
11 Feb 10
-
28 Jan 10
Bruce LewinStatistical and applied probabilistic knowledge is the core of knowledge; statistics is what tells you if something is true, false, or merely anecdotal; it is the "logic of science"; it is the instrument of risk-taking; it is the applied tools of epistemo
taleb uncertainty economics statistics research quantification
-
02 Jan 10
-
06 Dec 09
-
06 Nov 09
-
24 Sep 09
-
08 Sep 09
-
04 Sep 09
-
26 Aug 09
-
18 Aug 09
-
Inverse Problems. It is the greatest epistemological difficulty I know. In real life we do not observe probability distributions (not even in Soviet Russia, not even the French government). We just observe events. So we do not know the statistical properties—until, of course, after the fact. Given a set of observations, plenty of statistical distributions can correspond to the exact same realizations—each would extrapolate differently outside the set of events on which it was derived. The inverse problem is more acute when more theories, more distributions can fit a set a data.
-
Alas, the rarer the event, the more theory you need (since we don't observe it). So the rarer the event, the worse its inverse problem. And theories are fragile (just think of Doctor Bernanke).
-
I used to give the same mathematical finance lectures for both graduate students and practitioners before giving up on academic students and grade-seekers. Students cannot understand the value of "this is what we don't know"—they think it is not information, that they are learning nothing. Practitioners on the other hand value it immensely. Likewise with statisticians: I never had a disagreement with statisticians (who build the field)—only with users of statistical methods.
-
1) Avoid Optimization, Learn to Love Redundancy. Psychologists tell us that getting rich does not bring happiness—if you spend it. But if you hide it under the mattress, you are less vulnerable to a black swan. Only fools (such as Banks) optimize, not realizing that a simple model error can blow through their capital (as it just did). In one day in August 2007, Goldman Sachs experienced 24 x the average daily transaction volume—would 29 times have blown up the system? The only weak point I know of financial markets is their ability to drive people & companies to "efficiency" (to please a stock analyst’s earnings target) against risks of extreme events.
-
2) Avoid prediction of remote payoffs—though not necessarily ordinary ones. Payoffs from remote parts of the distribution are more difficult to predict than closer parts.
-
3) Beware the "atypicality" of remote events. There is a sucker's method called "scenario analysis" and "stress testing"—usually based on the past (or some "make sense" theory). Yet I show in the appendix how past shortfalls that do not predict subsequent shortfalls. Likewise, "prediction markets" are for fools. They might work for a binary election, but not in the Fourth Quadrant. Recall the very definition of events is complicated: success might mean one million in the bank ...or five billions!
-
-
10 Aug 09
Fabio de MirandaStatistical and applied probabilistic knowledge is the core of knowledge; statistics is what tells you if something is true, false, or merely anecdotal; it is the "logic of science"; it is the instrument of risk-taking; it is the applied tools of epistemology; you can't be a modern intellectual and not think probabilistically—but... let's not be suckers. The problem is much more complicated than it seems to the casual, mechanistic user who picked it up in graduate school. Statistics can fool you. In fact it is fooling your government right now. It can even bankrupt the system (let's face it: use of probabilistic methods for the estimation of risks did just blow up the banking system).
-
04 Aug 09
Ryan Mullerexplains how statistical analysis breaks down for complex decisions with heavy tails (black swans)
-
Fourth Quadrant: Complex decisions in Extremistan: Welcome to the Black Swan domain. Here is where your limits are. Do not base your decisions on statistically based claims. Or, alternatively, try to move your exposure type to make it third-quadrant style ("clipping tails").
-
he rarer the event, the worse its inverse problem
-
our empirical knowledge about the potential contribution—or role—of rare events (probability × consequence) is inversely proportional to their impact.
-
if you don't know what a "typical" event is, fractal power laws are the most effective way to discuss the extremes mathematically
-
Avoid Optimization, Learn to Love Redundancy
-
Conventional metrics based on type 1 randomness don't work.
-
Do not confuse absence of volatility with absence of risks.
-
-
14 Jul 09
Sam VAn opinionated essay about 'domains' or applications of probability and knowledge (or lack thereof) under uncertainty.
statistics economics finance risk science probability philosophy knowledge crisis
-
29 Jun 09
-
18 Jun 09
-
17 Jun 09
-
09 Jun 09
agujas1207Excellent article on the limits of statistical analysis "Pre-asymptotics. Theories are, of course, bad, but they can be worse in some situations when they were derived in idealized situations, the asymptote, but are used outside the asymptote (its limit,
statistics economics research history policy probability finance taleb crisis financialcrisis
-
14 May 09
-
02 May 09
-
28 Apr 09
-
27 Apr 09
-
26 Apr 09
Michael TillerIn August 2007, The Wall Street Journal published a statement by one financial economist, expressing his surprise that financial markets experienced a string of events that "would happen once in 10,000 years". A portrait of the gentleman accompanying the
-
24 Apr 09
-
28 Mar 09
-
20 Mar 09
Adam Crowe'Statistics can fool you. In fact it is fooling your government right now. It can even bankrupt the system (let's face it: use of probabilistic methods for the estimation of risks did just blow up the banking system). I want this to stop, and stop now— th
epistemology economics finance mathematics statistics literacy probability risk predictions criticism rearviewmirror fallacy myopia blackswans NassimNicholasTaleb
-
19 Mar 09
-
18 Mar 09
-
14 Mar 09
-
11 Mar 09
-
10 Mar 09
-
03 Mar 09
-
27 Feb 09
-
25 Feb 09
-
16 Feb 09
Oliver MayorSlightly more cohesive and specific overview of Taleb's ideas in The Black Swan. Presents a clearer picture of how to characterize risk.
economics science math probability taleb statistics finance research blackswan risk nnt
-
15 Feb 09
-
08 Feb 09
-
03 Feb 09
-
01 Feb 09
-
28 Jan 09
Nick HortovanyiAnd we are beyond suckers: not only, for socio-economic and other nonlinear, complicated variables, we are riding in a bus driven a blindfolded driver, but we refuse to acknowledge it in spite of the evidence, which to me is a pathological problem with ac
statistics science risk economics probability finance taleb for:aqualung for:mikeziv for:shaynephillips for:dahowlett
-
19 Jan 09
Ari REssentially NNT's statement of purpose: "Statistical and applied probabilistic knowledge is the core of knowledge; statistics is what tells you if something is true, false, or merely anecdotal; it is the "logic of science"; it is the instrument of risk-ta
statistics math logic risk subprime error nassimtaleb probability blackswan epistemology finance induction benoitmandelbrot
-
09 Jan 09
mrGTaleb, looking at the cataclysmic situation facing financial institutions today, points out that "the banking system, betting against Black Swans, has lost over 1 Trillion dollars (so far)
game-theory probability mathematicians economics statistics science
-
06 Jan 09
-
25 Dec 08
-
24 Dec 08
-
23 Dec 08
-
19 Dec 08
-
-
Figure 1 My classical metaphor: A Turkey is fed for a 1000 days—every days confirms to its statistical department that the human race cares about its welfare "with increased statistical significance". On the 1001st day, the turkey has a surprise.
-
Let us start with the inverse problem of rare events and proceed with a simple, nonmathematical argument. In August 2007, The Wall Street Journal published a statement by one financial economist, expressing his surprise that financial markets experienced a string of events that "would happen once in 10,000 years". A portrait of the gentleman accompanying the article revealed that he was considerably younger than 10,000 years; it is therefore fair to assume that he was not drawing his inference from his own empirical experience (and not from history at large), but from some theoretical model that produces the risk of rare events, or what he perceived to be rare events.
Alas, the rarer the event, the more theory you need (since we don't observe it). So the rarer the event, the worse its inverse problem. And theories are fragile (just think of Doctor Bernanke).
The tragedy is as follows. Suppose that you are deriving probabilities of future occurrences from the data, assuming (generously) that the past is representative of the future. Now, say that you estimate that an event happens every 1,000 days. You will need a lot more data than 1,000 days to ascertain its frequency, say 3,000 days. Now, what if the event happens once every 5,000 days? The estimation of this probability requires some larger number, 15,000 or more. The smaller the probability, the more observations you need, and the greater the estimation error for a set number of observations. Therefore, to estimate a rare event you need a sample that is larger and larger in inverse proportion to the occurrence of the event.
-
If small probability events carry large impacts, and (at the same time) these small probability events are more difficult to compute from past data itself, then: our empirical knowledge about the potential contribution—or role—of rare events (probability × consequence) is inversely proportional to their impact. This is why we should worry in the fourth quadrant!
For rare events, the confirmation bias (the tendency, Bernanke-style, of finding samples that confirm your opinion, not those that disconfirm it) is very costly and very distorting. Why? Most of histories of Black Swan prone events is going to be Black Swan free! Most samples will not reveal the black swans—except after if you are hit with them, in which case you will not be in a position to discuss them. Indeed I show with 40 years of data that past Black Swans do not predict future Black Swans in socio-economic life.
-
-
10 Dec 08
-
05 Dec 08
-
02 Dec 08
Sam RoseSome thoughts on the limits of statistics
foresight futuresstudies math economics statistics knowledge maps blackswan mathematics decisionmaking
-
29 Nov 08
Mariana Bondila
Statistics and economics...you like maths as far as I remember...on the home page there are some lectures on behavioral economics - they must be interesting ... -
25 Nov 08
-
18 Nov 08
-
11 Nov 08
-
10 Nov 08
-
07 Nov 08
-
05 Nov 08
-
04 Nov 08
-
03 Nov 08
-
30 Oct 08
-
25 Oct 08
-
23 Oct 08
-
22 Oct 08
Daniel SeitzStatistical and applied probabilistic knowledge is the core of knowledge; statistics is what tells you if something is true, false, or merely anecdotal; it is the "logic of science"; it is the instrument of risk-taking; it is the applied tools of epistemo
research science statistics economics finance economy probability risk bankenkrise finanzkrise
-
19 Oct 08
-
17 Oct 08
-
16 Oct 08
-
-
classical metaphor: A Turkey is fed for a 1000 days—every days confirms to its statistical department that the human race cares about its welfare "with increased statistical significance". On the 1001st day, the turkey has a surprise.
-
-
15 Oct 08
-
13 Oct 08
-
12 Oct 08
Christopher RiceNassim Taleb lays out the case against statistics and certainty. Cold comfort in the wake of global financial meltdown.
economics research science statistics decisionmaking politicaltheory
-
11 Oct 08
-
03 Oct 08
Kore7Taleb, looking at the cataclysmic situation facing financial institutions today, points out that "the banking system, betting against Black Swans, has lost over 1 Trillion dollars (so far), more than was ever made in the history of banking".
statistics science mathematics markets investment black_swan logic irrationality risk economics banking crisis taleb wall_street probability edge imported
-
30 Sep 08
Turker KThe problem is much more complicated than it seems to the casual, mechanistic user who picked it up in graduate school. Statistics can fool you. In fact it is fooling your government right now. It can even bankrupt the system (let's face it: use of probabilistic methods for the estimation of risks did just blow up the banking system).
nassim nicholas taleb economy 2008 economic crisis black swan statistics
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.