Black Swan PDF: Mastering Unpredictability

ON THE PLUMAGE OF BIRDS Before the discovery of Australia, people in the Old World were convinced that all swans were white, an unassailable belief as it seemed completely con rmed by empirical evidence. The sighting of the rst black swan might have been an interesting surprise for a few orni...

ON THE PLUMAGE OF BIRDS Before the discovery of Australia, people in the Old World were convinced that all swans were white, an unassailable belief as it seemed completely con rmed by empirical evidence. The sighting of the rst black swan might have been an interesting surprise for a few ornithologists (and others extremely concerned with the coloring of birds), but that is not where the signi cance of the story lies. It illustrates a severe limitation to our learning from observations or experience and the fragility of our knowledge. One single observation can invalidate a general statement derived from millennia of con rmatory sightings of millions of white swans. All you need is one single (and, I am told, quite ugly) black bird.* I push one step beyond this philosophical-logical question into an empirical reality, and one that has obsessed me since childhood.† What we call here a Black Swan (and capitalize it) is an event with the following three attributes. First, it is an outlier, as it lies outside the realm of regular expectations, because nothing in the past can convincingly point to its possibility. Second, it carries an extreme impact (unlike the bird). Third, in spite of its outlier status, human nature makes us concoct explanations for its occurrence after the fact, making it explainable and predictable. I stop and summarize the triplet: rarity, extreme impact, and retrospective (though not prospective) predictability.* A small number of Black Swans explain almost everything in our world, from the success of ideas and religions, to the dynamics of historical events, to elements of our own personal lives. Ever since we left the Pleistocene, some ten millennia ago, the e ect of these Black Swans has been increasing. It started accelerating during the industrial revolution, as the world started getting more complicated, while ordinary events, the ones we study and discuss and try to predict from reading the newspapers, have become increasingly inconsequential. Just imagine how little your understanding of the world on the eve of the events of 1914 would have helped you guess what was to happen next. (Don’t cheat by using the explanations drilled into your cranium by your dull high school teacher.) How about the rise of Hitler and the subsequent war? How about the precipitous demise of the Soviet bloc? How about the consequences of the rise of Islamic fundamentalism? How about the e ect of the spread of the Internet? How about the market crash of 1987 (and the more unexpected recovery)? Fads, epidemics, fashion, ideas, the emergence of art genres and schools. All follow these Black Swan dynamics. Literally, just about everything of signi cance around you might qualify. This combination of low predictability and large impact makes the Black Swan a great puzzle; but that is not yet the core concern of this book. Add to this phenomenon the fact that we tend to act as if it does not exist! I don’t mean just you, your cousin Joey, and me, but almost all “social scientists” who, for over a century, have operated under the false belief that their tools could measure uncertainty. For the applications of the sciences of uncertainty to real-world problems has had ridiculous e ects; I have been privileged to see it in nance and economics. Go ask your portfolio manager for his de nition of “risk,” and odds are that he will supply you with a measure that excludes the possibility of the Black Swan—hence one that has no better predictive value for assessing the total risks than astrology (we will see how they dress up the intellectual fraud with mathematics). This problem is endemic in social matters. The central idea of this book concerns our blindness with respect to randomness, particularly the large deviations: Why do we, scientists or nonscientists, hotshots or regular Joes, tend to see the pennies instead of the dollars? Why do we keep focusing on the minutiae, not the possible signi cant large events, in spite of the obvious evidence of their huge in uence? And, if you follow my argument, why does reading the newspaper actually decrease your knowledge of the world? It is easy to see that life is the cumulative e ect of a handful of signi cant shocks. It is not so hard to identify the role of Black Swans, from your armchair (or bar stool). Go through the following exercise. Look into your own existence. Count the signi cant events, the technological changes, and the inventions that have taken place in our environment since you were born and compare them to what was expected before their advent. How many of them came on a schedule? Look into your own personal life, to your choice of profession, say, or meeting your mate, your exile from your country of origin, the betrayals you faced, your sudden enrichment or impoverishment. How often did these things occur according to plan? What You Do Not Know Black Swan logic makes what you don’t know far more relevant than what you do know.* Consider that many Black Swans can be caused and exacerbated by their being unexpected. Think of the terrorist attack of September 11, 2001: had the risk been reasonably conceivable on September 10, it would not have happened. If such a possibility were deemed worthy of attention, ghter planes would have circled the sky above the twin towers, airplanes would have had locked bulletproof doors, and the attack would not have taken place, period. Something else might have taken place. What? I don’t know. Isn’t it strange to see an event happening precisely because it was not supposed to happen? What kind of defense do we have against that? Whatever you come to know (that New York is an easy terrorist target, for instance) may become inconsequential if your enemy knows that you know it. It may be odd that, in such a strategic game, what you know can be truly inconsequential.* This extends to all businesses. Think about the “secret recipe” to making a killing in the restaurant business. If it were known and obvious, then someone next door would have already come up with the idea and it would have become generic. The next killing in the restaurant industry needs to be an idea that is not easily conceived of by the current population of restaurateurs. It has to be at some distance from expectations. The more unexpected the success of such a venture, the smaller the number of competitors, and the more successful the entrepreneur who implements the idea. The same applies to the shoe and the book businesses—or any kind of entrepreneurship. The same applies to scienti c theories—nobody has interest in listening to trivialities. The payo of a human venture is, in general, inversely proportional to what it is expected to be. Consider the Indian Ocean tsunami of December 2004. Had it been expected, it would not have caused the damage it did—the areas a ected would have been less populated, an early warning system would have been put in place. What you know cannot really hurt you. Experts and “Empty Suits” The inability to predict outliers implies the inability to predict the course of history, given the share of these events in the dynamics of events. But we act as though we are able to predict historical events, or, even worse, as if we are able to change the course of history. We produce thirty-year projections of social security de cits and oil prices without realizing that we cannot even predict these for next summer—our cumulative prediction errors for political and economic events are so monstrous that every time I look at the empirical record I have to pinch myself to verify that I am not dreaming. What is surprising is not the magnitude of our forecast errors, but our absence of awareness of it. This is all the more worrisome when we engage in deadly con icts: wars are fundamentally unpredictable (and we do not know it). Owing to this misunderstanding of the causal chains between policy and actions, we can easily trigger Black Swans thanks to aggressive ignorance—like a child playing with a chemistry kit. Our inability to predict in environments subjected to the Black Swan, coupled with a general lack of the awareness of this state of a airs, means that certain professionals, while believing they are experts, are in fact not. Based on their empirical record, they do not know more about their subject matter than the general population, but they are much better at narrating—or, worse, at smoking you with complicated mathematical models. They are also more likely to wear a tie. Black Swans being unpredictable, we need to adjust to their existence (rather than naïvely try to predict them). There are so many things we can do if we focus on antiknowledge, or what we do not know. Among many other bene ts, you can set yourself up to collect serendipitous Black Swans (of the positive kind) by maximizing your exposure to them. Indeed, in some domains—such as scienti c discovery and venture capital investments—there is a disproportionate payo from the unknown, since you typically have little to lose and plenty to gain from a rare event. We will see that, contrary to social-science wisdom, almost no discovery, no technologies of note, came from design and planning— they were just Black Swans. The strategy for the discoverers and entrepreneurs is to rely less on top-down planning and focus on maximum tinkering and recognizing opportunities when they present themselves. So I disagree with the followers of Marx and those of Adam Smith: the reason free markets work is because they allow people to be lucky, thanks to aggressive trial and error, not by giving rewards or “incentives” for skill. The strategy is, then, to tinker as much as possible and try to collect as many Black Swan opportunities as you can. Learning to Learn Another related human impediment comes from excessive focus on what we do know: we tend to learn the precise, not the general. What did people learn from the 9/11 episode? Did they learn that some events, owing to their dynamics, stand largely outside the realm of the predictable? No. Did they learn the built-in defect of conventional wisdom? No. What did they gure out? They learned precise rules for avoiding Islamic prototerrorists and tall buildings. Many keep reminding me that it is important for us to be practical and take tangible steps rather than to “theorize” about knowledge. The story of the Maginot Line shows how we are conditioned to be speci c. The French, after the Great War, built a wall along the previous German invasion route to prevent reinvasion—Hitler just (almost) e ortlessly went around it. The French had been excellent students of history; they just learned with too much precision. They were too practical and exceedingly focused for their own safety. We do not spontaneously learn that we don’t learn that we don’t learn. The problem lies in the structure of our minds: we don’t learn rules, just facts, and only facts. Metarules (such as the rule that we have a tendency to not learn rules) we don’t seem to be good at getting. We scorn the abstract; we scorn it with passion. Why? It is necessary here, as it is my agenda in the rest of this book, both to stand conventional wisdom on its head and to show how inapplicable it is to our modern, complex, and increasingly recursive environment.* But there is a deeper question: What are our minds made for? It looks as if we have the wrong user’s manual. Our minds do not seem made to think and introspect; if they were, things would be easier for us today, but then we would not be here today and I would not have been here to talk about it—my counterfactual, introspective, and hard-thinking ancestor would have been eaten by a lion while his nonthinking but faster-reacting cousin would have run for cover. Consider that thinking is time-consuming and generally a great waste of energy, that our predecessors spent more than a hundred million years as nonthinking mammals and that in the blip in our history during which we have used our brain we have used it on subjects too peripheral to matter. Evidence shows that we do much less thinking than we believe we do—except, of course, when we think about it. A NEW KIND OF INGRATITUDE It is quite saddening to think of those people who have been mistreated by history. There were the poètes maudits, like Edgar Allan Poe or Arthur Rimbaud, scorned by society and later worshipped and force-fed to schoolchildren. (There are even schools named after high school dropouts.) Alas, this recognition came a little too late for the poet to get a serotonin kick out of it, or to prop up his romantic life on earth. But there are even more mistreated heroes—the very sad category of those who we do not know were heroes, who saved our lives, who helped us avoid disasters. They left no traces and did not even know that they were making a contribution. We remember the martyrs who died for a cause that we knew about, never those no less e ective in their contribution but whose cause we were never aware of—precisely because they were successful. Our ingratitude toward the poètes maudits fades completely in front of this other type of thanklessness. This is a far more vicious kind of ingratitude: the feeling of uselessness on the part of the silent hero. I will illustrate with the following thought experiment. Assume that a legislator with courage, in uence, intellect, vision, and perseverance manages to enact a law that goes into universal e ect and employment on September 10, 2001; it imposes the continuously locked bulletproof doors in every cockpit (at high costs to the struggling airlines)—just in case terrorists decide to use planes to attack the World Trade Center in New York City. I know this is lunacy, but it is just a thought experiment (I am aware that there may be no such thing as a legislator with intellect, courage, vision, and perseverance; this is the point of the thought experiment). The legislation is not a popular measure among the airline personnel, as it complicates their lives. But it would certainly have prevented 9/11. The person who imposed locks on cockpit doors gets no statues in public squares, not so much as a quick mention of his contribution in his obituary. “Joe Smith, who helped avoid the disaster of 9/11, died of complications of liver disease.” Seeing how super uous his measure was, and how it squandered resources, the public, with great help from airline pilots, might well boot him out of o ce. Vox clamantis in deserto. He will retire depressed, with a great sense of failure. He will die with the impression of having done nothing useful. I wish I could go attend his funeral, but, reader, I can’t nd him. And yet, recognition can be quite a pump. Believe me, even those who genuinely claim that they do not believe in recognition, and that they separate labor from the fruits of labor, actually get a serotonin kick from it. See how the silent hero is rewarded: even his own hormonal system will conspire to o er no reward. Now consider again the events of 9/11. In their aftermath, who got the recognition? Those you saw in the media, on television performing heroic acts, and those whom you saw trying to give you the impression that they were performing heroic acts. The latter category includes someone like the New York Stock Exchange chairman Richard Grasso, who “saved the stock exchange” and received a huge bonus for his contribution (the equivalent of several thousand average salaries). All he had to do was be there to ring the opening bell on television—the television that, we will see, is the carrier of unfairness and a major cause of Black Swan blindness. Who gets rewarded, the central banker who avoids a recession or the one who comes to “correct” his predecessors’ faults and happens to be there during some economic recovery? Who is more valuable, the politician who avoids a war or the one who starts a new one (and is lucky enough to win)? It is the same logic reversal we saw earlier with the value of what we don’t know; everybody knows that you need more prevention than treatment, but few reward acts of prevention. We glorify those who left their names in history books at the expense of those contributors about whom our books are silent. We humans are not just a super cial race (this may be curable to some extent); we are a very unfair one. LIFE IS VERY UNUSUAL This is a book about uncertainty; to this author, the rare event equals uncertainty. This may seem like a strong statement—that we need to principally study the rare and extreme events in order to gure out common ones—but I will make myself clear as follows. There are two possible ways to approach phenomena. The rst is to rule out the extraordinary and focus on the “normal.” The examiner leaves aside “outliers” and studies ordinary cases. The second approach is to consider that in order to understand a phenomenon, one needs rst to consider the extremes—particularly if, like the Black Swan, they carry an extraordinary cumulative e ect. I don’t particularly care about the usual. If you want to get an idea of a friend’s temperament, ethics, and personal elegance, you need to look at him under the tests of severe circumstances, not under the regular rosy glow of daily life. Can you assess the danger a criminal poses by examining only what he does on an ordinary day? Can we understand health without considering wild diseases and epidemics? Indeed the normal is often irrelevant. Almost everything in social life is produced by rare but consequential shocks and jumps; all the while almost everything studied about social life focuses on the “normal,” particularly with “bell curve” methods of inference that tell you close to nothing. Why? Because the bell curve ignores large deviations, cannot handle them, yet makes us con dent that we have tamed uncertainty. Its nickname in this book is GIF, Great Intellectual Fraud. PLATO AND THE NERD At the start of the Jewish revolt in the rst century of our era, much of the Jews’ anger was caused by the Romans’ insistence on putting a statue of Caligula in their temple in Jerusalem in exchange for placing a statue of the Jewish god Yahweh in Roman temples. The Romans did not realize that what the Jews (and the subsequent Levantine monotheists) meant by god was abstract, all embracing, and had nothing to do with the anthropomorphic, too human representation that Romans had in mind when they said deus. Critically, the Jewish god did not lend himself to symbolic representation. Likewise, what many people commoditize and label as “unknown,” “improbable,” or “uncertain” is not the same thing to me; it is not a concrete and precise category of knowledge, a nerdified eld, but its opposite; it is the lack (and limitations) of knowledge. It is the exact contrary of knowledge; one should learn to avoid using terms made for knowledge to describe its opposite. What I call Platonicity, after the ideas (and personality) of the philosopher Plato, is our tendency to mistake the map for the territory, to focus on pure and well-de ned “forms,” whether objects, like triangles, or social notions, like utopias (societies built according to some blueprint of what “makes sense”), even nationalities. When these ideas and crisp constructs inhabit our minds, we privilege them over other less elegant objects, those with messier and less tractable structures (an idea that I will elaborate progressively throughout this book). Platonicity is what makes us think that we understand more than we actually do. But this does not happen everywhere. I am not saying that Platonic forms don’t exist. Models and constructions, these intellectual maps of reality, are not always wrong; they are wrong only in some speci c applications. The di culty is that a) you do not know beforehand (only after the fact) where the map will be wrong, and b) the mistakes can lead to severe consequences. These models are like potentially helpful medicines that carry random but very severe side e ects. The Platonic fold is the explosive boundary where the Platonic mind- set enters in contact with messy reality, where the gap between what you know and what you think you know becomes dangerously wide. It is here that the Black Swan is produced. TOO DULL TO WRITE ABOUT It was said that the artistic lmmaker Luchino Visconti made sure that when actors pointed at a closed box meant to contain jewels, there were real jewels inside. It could be an e ective way to make actors live their part. I think that Visconti’s gesture may also come out of a plain sense of aesthetics and a desire for authenticity—somehow it may not feel right to fool the viewer. This is an essay expressing a primary idea; it is neither the recycling nor repackaging of other people’s thoughts. An essay is an impulsive meditation, not science reporting. I apologize if I skip a few obvious topics in this book out of the conviction that what is too dull for me to write about might be too dull for the reader to read. (Also, to avoid dullness may help to lter out the nonessential.) Talk is cheap. Someone who took too many philosophy classes in college (or perhaps not enough) might object that the sighting of a Black Swan does not invalidate the theory that all swans are white since such a black bird is not technically a swan since whiteness to him may be the essential property of a swan. Indeed those who read too much Wittgenstein (and writings about comments about Wittgenstein) may be under the impression that language problems are important. They may certainly be important to attain prominence in philosophy departments, but they are something we, practitioners and decision makers in the real world, leave for the weekend. As I explain in the chapter called “The Uncertainty of the Phony,” for all of their intellectual appeal, these niceties have no serious implications Monday to Friday as opposed to more substantial (but neglected) matters. People in the classroom, not having faced many true situations of decision making under uncertainty, do not realize what is important and what is not—even those who are scholars of uncertainty (or particularly those who are scholars of uncertainty). What I call the practice of uncertainty can be piracy, commodity speculation, professional gambling, working in some branches of the Ma a, or just plain serial entrepreneurship. Thus I rail against “sterile skepticism,” the kind we can do nothing about, and against the exceedingly theoretical language problems that have made much of modern philosophy largely irrelevant to what is derisively called the “general public.” (In the past, for better or worse, those rare philosophers and thinkers who were not self-standing depended on a patron’s support. Today academics in abstract disciplines depend on one another’s opinion, without external checks, with the severe occasional pathological result of turning their pursuits into insular prowess-showing contests. Whatever the shortcomings of the old system, at least it enforced some standard of relevance.) The philosopher Edna Ullmann-Margalit detected an inconsistency in this book and asked me to justify the use of the precise metaphor of a Black Swan to describe the unknown, the abstract, and imprecise uncertain—white ravens, pink elephants, or evaporating denizens of a remote planet orbiting Tau Ceti. Indeed, she caught me red handed. There is a contradiction; this book is a story, and I prefer to use stories and vignettes to illustrate our gullibility about stories and our preference for the dangerous compression of narratives.* You need a story to displace a story. Metaphors and stories are far more potent (alas) than ideas; they are also easier to remember and more fun to read. If I have to go after what I call the narrative disciplines, my best tool is a narrative. Ideas come and go, stories stay. THE BOTTOM LINE The beast in this book is not just the bell curve and the self-deceiving statistician, nor the Platoni ed scholar who needs theories to fool himself with. It is the drive to “focus” on what makes sense to us. Living on our planet, today, requires a lot more imagination than we are made to have. We lack imagination and repress it in others. Note that I am not relying in this book on the beastly method of collecting selective “corroborating evidence.” For reasons I explain in Chapter 5, I call this overload of examples naïve empiricism— successions of anecdotes selected to t a story do not constitute evidence. Anyone looking for con rmation will nd enough of it to deceive himself—and no doubt his peers.* The Black Swan idea is based on the structure of randomness in empirical reality. To summarize: in this (personal) essay, I stick my neck out and make a claim, against many of our habits of thought, that our world is dominated by the extreme, the unknown, and the very improbable (improbable according our current knowledge)—and all the while we spend our time engaged in small talk, focusing on the known, and the repeated. This implies the need to use the extreme event as a starting point and not treat it as an exception to be pushed under the rug. I also make the bolder (and more annoying) claim that in spite of our progress and the growth in knowledge, or perhaps because of such progress and growth, the future will be increasingly less predictable, while both human nature and social “science” seem to conspire to hide the idea from us. Chapters Map  The sequence of this book follows a simple logic; it ows from what can be labeled purely literary (in subject and treatment) to what can be deemed entirely scienti c (in subject, though not in treatment). Psychology will be mostly present in Part One and in the early part of Part Two; business and natural science will be dealt with mostly in the second half of Part Two and in Part Three. Part One, “Umberto Eco’s Antilibrary,” is mostly about how we perceive historical and current events and what distortions are present in such perception. Part Two, “We Just Can’t Predict,” is about our errors in dealing with the future and the unadvertised limitations of some “sciences”—and what to do about these limitations. Part Three, “Those Gray Swans of Extremistan,” goes deeper into the topic of extreme events, explains how the bell curve (that great intellectual fraud) is generated, and reviews the ideas in the natural and social sciences loosely lumped under the label “complexity.” Part Four, “The End,” will be very short. I derived an unexpected amount of enjoyment writing this book—in fact, it just wrote itself—and I hope that the reader will experience the same. I confess that I got hooked on this withdrawal into pure ideas after the constraints of an active and transactional life. After this book is published, my aim is to spend time away from the clutter of public activities in order to think about my philosophical-scienti c idea in total tranquillity. * The spread of camera cell phones has a orded me a large collection of pictures of black swans sent by traveling readers. Last Christmas I also got a case of Black Swan Wine (not my favorite), a videotape (I don’t watch videos), and two books. I prefer the pictures. † I used the logical metaphor of the black swan (not capitalized) for  Black Swan Events (capitalized), but this problem should not be confused with the logical problem raised by many philosophers. This is not so much about exceptions as it is about the oversize role of extreme events in many domains in life. Furthermore, the logical problem is about the possibility of the exception (black swan); mine is about the role of the exceptional event (Black Swan) leading to the degradation of predictability and the need to be robust to negative Black Swans and exposed to positive ones. * The highly expected not happening is also a Black Swan. Note that, by symmetry, the occurrence of a highly improbable event is the equivalent of the nonoccurrence of a highly probable one. * The Black Swan is the result of collective and individual epistemic limitations (or distortions), mostly con dence in knowledge; it is not an objective phenomenon. The most severe mistake made in the interpretation of my Black Swan is to try to de ne an “objective Black Swan” that would be invariant in the eyes of all observers. The events of September 11, 2001, were a Black Swan for the victims, but certainly not to the perpetrators. The Postscript provides an additional discussion of the point. * The Idea of Robustness: Why do we formulate theories leading to projections and forecasts without focusing on the robustness of these theories and the consequences of the errors? It is much easier to deal with the Black Swan problem if we focus on robustness to errors rather than improving predictions. * Recursive here means that the world in which we live has an increasing number of feedback loops, causing events to be the cause of more events (say, people buy a book because other people bought it), thus generating snowballs and arbitrary and unpredictable planet-wide winner-take-all e ects. We live in an environment where information  ows too rapidly, accelerating such epidemics. Likewise, events can happen because they are not supposed to happen. (Our intuitions are made for an environment with simpler causes and e ects and slowly moving information.) This type of randomness did not prevail during the Pleistocene, as socioeconomic life was far simpler then. * The metaphor of the black swan is not at all a modern one—contrary to its usual attribution to Popper, Mill, Hume, and others. I selected it because it corresponds to the ancient idea of a “rare bird.” The Latin poet Juvenal refers to a “bird as rare as the black swan”—rara avis in terris nigroque simillima cygno. * It is also naïve empiricism to provide, in support of some argument, series of eloquent con rmatory quotes by dead authorities. By searching, you can always nd someone who made a well-sounding statement that con rms your point of view—and, on every topic, it is possible to nd another dead thinker who said the exact opposite. Almost all of my non–Yogi Berra quotes are from people I disagree with.  The writer Umberto Eco belongs to that small class of scholars who are encyclopedic, insightful, and nondull. He is the owner of a large personal library (containing thirty thousand books), and separates visitors into two categories: those who react with “Wow! Signore professore dottore Eco, what a library you have! How many of these books have you read?” and the others— a very small minority—who get the point that a private library is not an ego- boosting appendage but a research tool. Read books are far less valuable than unread ones. The library should contain as much of what you do not know as your nancial means, mortgage rates, and the currently tight real-estate market allow you to put there. You will accumulate more knowledge and more books as you grow older, and the growing number of unread books on the shelves will look at you menacingly. Indeed, the more you know, the larger the rows of unread books. Let us call this collection of unread books an antilibrary.  We tend to treat our knowledge as personal property to be protected and defended. It is an ornament that allows us to rise in the pecking order. So this tendency to o end Eco’s library sensibility by focusing on the known is a human bias that extends to our mental operations. People don’t walk around with anti-résumés telling you what they have not studied or experienced (it’s the job of their competitors to do that), but it would be nice if they did. Just as we need to stand library logic on its head, we will work on standing knowledge itself on its head. Note that the Black Swan comes from our misunderstanding of the likelihood of surprises, those unread books, because we take what we know a little too seriously. Let us call an antischolar—someone who focuses on the unread books, and makes an attempt not to treat his knowledge as a treasure, or even a possession, or even a self-esteem enhancement device—a skeptical empiricist. The chapters in this section address the question of how we humans deal with knowledge—and our preference for the anecdotal over the empirical. Chapter 1 presents the Black Swan as grounded in the story of my own obsession. I will make a central distinction between the two varieties of randomness in Chapter 3. After that, Chapter 4 brie y returns to the Black Swan problem in its original form: how we tend to generalize from what we see. Then I present the three facets of the same Black Swan problem: a) The error of confirmation, or how we are likely to undeservedly scorn the virgin part of the library (the tendency to look at what con rms our knowledge, not our ignorance), in Chapter 5; b) the narrative fallacy, or how we fool ourselves with stories and anecdotes (Chapter 6); c) how emotions get in the way of our inference (Chapter 7); and d) the problem of silent evidence, or the tricks history uses to hide Black Swans from us (Chapter 8). Chapter 9 discusses the lethal fallacy of building knowledge from the world of games. Chapter One THE APPRENTICESHIP OF AN EMPIRICAL SKEPTIC Anatomy of a Black Swan—The triplet of opacity—Reading books backward—The rearview mirror—Everything becomes explainable— Always talk to the driver (with caution)—History doesn’t crawl; it jumps —“It was so unexpected”—Sleeping for twelve hours This is not an autobiography, so I will skip the scenes of war. Actually, even if it were an autobiography, I would still skip the scenes of war. I cannot compete with action movies or memoirs of adventurers more accomplished than myself, so I will stick to my specialties of chance and uncertainty. ANATOMY OF A BLACK SWAN For more than a millennium the eastern Mediterranean seaboard called Syria Libanensis, or Mount Lebanon, had been able to accommodate at least a dozen di erent sects, ethnicities, and beliefs—it worked like magic. The place resembled major cities of the eastern Mediterranean (called the Levant) more than it did the other parts in the interior of the Near East (it was easier to move by ship than by land through the mountainous terrain). The Levantine cities were mercantile in nature; people dealt with one another according to a clear protocol, preserving a peace conducive to commerce, and they socialized quite a bit across communities. This millennium of peace was interrupted only by small occasional friction within Moslem and Christian communities, rarely between Christians and Moslems. While the cities were mercantile and mostly Hellenistic, the mountains had been settled by all manner of religious minorities who claimed to have ed both the Byzantine and Moslem orthodoxies. A mountainous terrain is an ideal refuge from the mainstream, except that your enemy is the other refugee competing for the same type of rugged real estate. The mosaic of cultures and religions there was deemed an example of coexistence: Christians of all varieties (Maronites, Armenians, Greco-Syrian Byzantine Orthodox, even Byzantine Catholic, in addition to the few Roman Catholics left over from the Crusades); Moslems (Shiite and Sunni); Druzes; and a few Jews. It was taken for granted that people learned to be tolerant there; I recall how we were taught in school how far more civilized and wiser we were than those in the Balkan communities, where not only did the locals refrain from bathing but also fell prey to fractious ghting. Things appeared to be in a state of stable equilibrium, evolving out of a historical tendency for betterment and tolerance. The terms balance and equilibrium were often used. Both sides of my family came from the Greco-Syrian community, the last Byzantine outpost in northern Syria, which included what is now called Lebanon. Note that the Byzantines called themselves “Romans”—Roumi (plural Roum) in the local languages. We originate from the olive-growing area at the base of Mount Lebanon—we chased the Maronite Christians into the mountains in the famous battle of Amioun, my ancestral village. Since the Arab invasion in the seventh century, we had been living in mercantile peace with the Moslems, with only some occasional harassment by the Lebanese Maronite Christians from the mountains. By some (literally) Byzantine arrangement between the Arab rulers and the Byzantine emperors, we managed to pay taxes to both sides and get protection from both. We thus managed to live in peace for more than a millennium almost devoid of bloodshed: our last true problem was the later troublemaking crusaders, not the Moslem Arabs. The Arabs, who seemed interested only in warfare (and poetry) and, later, the Ottoman Turks, who seemed only concerned with warfare (and pleasure), left to us the uninteresting pursuit of commerce and the less dangerous one of scholarship (like the translation of Aramaic and Greek texts). By any standard the country called Lebanon, to which we found ourselves suddenly incorporated after the fall of the Ottoman Empire, in the early twentieth century, appeared to be a stable paradise; it was also cut in a way to be predominantly Christian. People were suddenly brainwashed to believe in the nation-state as an entity.* The Christians convinced themselves that they were at the origin and center of what is loosely called Western culture yet with a window on the East. In a classical case of static thinking, nobody took into account the di erentials in birthrate between communities and it was assumed that a slight Christian majority would remain permanent. Levantines had been granted Roman citizenship, which allowed Saint Paul, a Syrian, to travel freely through the ancient world. People felt connected to everything they felt was worth connecting to; the place was exceedingly open to the world, with a vastly sophisticated lifestyle, a prosperous economy, and temperate weather just like California, with snow-covered mountains jutting above the Mediterranean. It attracted a collection of spies (both Soviet and Western), prostitutes (blondes), writers, poets, drug dealers, adventurers, compulsive gamblers, tennis players, après-skiers, and merchants—all professions that complement one another. Many people acted as if they were in an old James Bond movie, or the days when playboys smoked, drank, and, instead of going to the gym, cultivated relationships with good tailors. The main attribute of paradise was there: cabdrivers were said to be polite (though, from what I remember, they were not polite to me). True, with hindsight, the place may appear more Elysian in the memory of people than it actually was. I was too young to taste the pleasures of the place, as I became a rebellious idealist and, very early on, developed an ascetic taste, averse to the ostentatious signaling of wealth, allergic to Levantine culture’s overt pursuit of luxury and its obsession with things monetary. As a teenager, I could not wait to go settle in a metropolis with fewer James Bond types around. Yet I recall something that felt special in the intellectual air. I attended the French lycée that had one of the highest success rates for the French baccalauréat (the high school degree), even in the subject of the French language. French was spoken there with some purity: as in prerevolutionary Russia, the Levantine Christian and Jewish patrician class (from Istanbul to Alexandria) spoke and wrote formal French as a language of distinction. The most privileged were sent to school in France, as both my grandfathers were—my paternal namesake in 1912 and my mother’s father in 1929. Two thousand years earlier, by the same instinct of linguistic distinction, the snobbish Levantine patricians wrote in Greek, not the vernacular Aramaic. (The New Testament was written in the bad local patrician Greek of our capital, Antioch, prompting Nietzsche to shout that “God spoke bad Greek.”) And, after Hellenism declined, they took up Arabic. So in addition to being called a “paradise,” the place was also said to be a miraculous crossroads of what are super cially tagged “Eastern” and “Western” cultures. On Walking Walks  My ethos was shaped when, at fteen, I was put in jail for (allegedly) attacking a policeman with a slab of concrete during a student riot—an incident with strange rami cations since my grandfather was then the minister of the interior, and the person who signed the order to crush our revolt. One of the rioters was shot dead when a policeman who had been hit on the head with a stone panicked and randomly opened re on us. I recall being at the center of the riot, and feeling a huge satisfaction upon my capture while my friends were scared of both prison and their parents. We frightened the government so much that we were granted amnesty. There were some obvious bene ts in showing one’s ability to act on one’s opinions, and not compromising an inch to avoid “o ending” or bothering others. I was in a state of rage and didn’t care what my parents (and grandfather) thought of me. This made them quite scared of me, so I could not a ord to back down, or even blink. Had I concealed my participation in the riot (as many friends did) and been discovered, instead of being openly de ant, I am certain that I would have been treated as a black sheep. It is one thing to be cosmetically de ant of authority by wearing unconventional clothes—what social scientists and economists call “cheap signaling”—and another to prove willingness to translate belief into action. My paternal uncle was not too bothered by my political ideas (these come and go); he was outraged that I used them as an excuse to dress sloppily. To him, inelegance on the part of a close family member was the mortal o ense. Public knowledge of my capture had another major bene t: it allowed me to avoid the usual outward signs of teenage rebellion. I discovered that it is much more e ective to act like a nice guy and be “reasonable” if you prove willing to go beyond just verbiage. You can a ord to be compassionate, lax, and courteous if, once in a while, when it is least expected of you, but completely justi ed, you sue someone, or savage an enemy, just to show that you can walk the walk. “Paradise” Evaporated The Lebanese “paradise” suddenly evaporated, after a few bullets and mortar shells. A few months after my jail episode, after close to thirteen centuries of remarkable ethnic coexistence, a Black Swan, coming out of nowhere, transformed the place from heaven to hell. A erce civil war began between Christians and Moslems, including the Palestinian refugees who took the Moslem side. It was brutal, since the combat zones were in the center of the town and most of the ghting took place in residential areas (my high school was only a few hundred feet from the war zone). The con ict lasted more than a decade and a half. I will not get too descriptive. It may be that the invention of gun re and powerful weapons turned what, in the age of the sword, would have been just tense conditions into a spiral of uncontrollable tit-for-tat warfare. Aside from the physical destruction (which turned out to be easy to reverse with a few motivated contractors, bribed politicians, and naïve bondholders), the war removed much of the crust of sophistication that had made the Levantine cities a continuous center of great intellectual re nement for three thousand years. Christians had been leaving the area since Ottoman times— those who moved to the West took Western rst names and melded in. Their exodus accelerated. The number of cultured people dropped below some critical level. Suddenly the place became a vacuum. Brain drain is hard to reverse, and some of the old re nement may be lost forever. The Starred Night The next time you experience a blackout, take some solace by looking at the sky. You will not recognize it. Beirut had frequent power shutdowns during the war. Before people bought their own generators, one side of the sky was clear at night, owing to the absence of light pollution. That was the side of town farthest from the combat zone. People deprived of television drove to watch the erupting lights of nighttime battles. They appeared to prefer the risk of being blown up by mortar shells to the boredom of a dull evening. So you could see the stars with great clarity. I had been told in high school that the planets are in something called equilibrium, so we did not have to worry about the stars hitting us unexpectedly. To me, that eerily resembled the stories we were also told about the “unique historical stability” of Lebanon. The very idea of assumed equilibrium bothered me. I looked at the constellations in the sky and did not know what to believe. HISTORY AND THE TRIPLET OF OPACITY History is opaque. You see what comes out, not the script that produces events, the generator of history. There is a fundamental incompleteness in your grasp of such events, since you do not see what’s inside the box, how the mechanisms work. What I call the generator of historical events is di erent from the events themselves, much as the minds of the gods cannot be read just by witnessing their deeds. You are very likely to be fooled about their intentions. This disconnect is similar to the di erence between the food you see on the table at the restaurant and the process you can observe in the kitchen. (The last time I brunched at a certain Chinese restaurant on Canal Street in downtown Manhattan, I saw a rat coming out of the kitchen.) The human mind su ers from three ailments as it comes into contact with history, what I call the triplet of opacity. They are: 1. the illusion of understanding, or how everyone thinks he knows what is going on in a world that is more complicated (or random) than they realize; 2. the retrospective distortion, or how we can assess matters only after the fact, as if they were in a rearview mirror (history seems clearer and more organized in history books than in empirical reality); and 3. the overvaluation of factual information and the handicap of authoritative and learned people, particularly when they create categories —when they “Platonify.” Nobody Knows What’s Going On  The rst leg of the triplet is the pathology of thinking that the world in which we live is more understandable, more explainable, and therefore more predictable than it actually is. I was constantly told by adults that the war, which ended up lasting close to seventeen years, was going to end in “only a matter of days.” They seemed quite con dent in their forecasts of duration, as can be evidenced by the number of people who sat waiting in hotel rooms and other temporary quarters in Cyprus, Greece, France, and elsewhere for the war to nish. One uncle kept telling me how, some thirty years earlier, when the rich Palestinians ed to Lebanon, they considered it a very temporary solution (most of those still alive are still there, six decades later). Yet when I asked him if it was going to be the same with our con ict, he replied, “No, of course not. This place is di erent; it has always been di erent.” Somehow what he detected in others did not seem to apply to him. This duration blindness in the middle-aged exile is quite a widespread disease. Later, when I decided to avoid the exile’s obsession with his roots (exiles’ roots penetrate their personalities a bit too deeply), I studied exile literature precisely to avoid the traps of a consuming and obsessive nostalgia. These exiles seemed to have become prisoners of their memory of idyllic origin—they sat together with other prisoners of the past and spoke about the old country, and ate their traditional food while some of their folk music played in the background. They continuously ran counterfactuals in their minds, generating alternative scenarios that could have happened and prevented these historical ruptures, such as “if the Shah had not named this incompetent man as prime minister, we would still be there.” It was as if the historical rupture had a speci c cause, and that the catastrophe could have been averted by removing that speci c cause. So I pumped every displaced person I could nd for information on their behavior during exile. Almost all act in the same way. One hears endless stories of Cuban refugees with suitcases still half packed who came to Miami in the 1960s for “a matter of a few days” after the installation of the Castro regime. And of Iranian refugees in Paris and London who ed the Islamic Republic in 1978 thinking that their absence would be a brief vacation. A few are still waiting, more than a quarter century later, for the return. Many Russians who left in 1917, such as the writer Vladimir Nabokov, settled in Berlin, perhaps to be close enough for a quick return. Nabokov himself lived all his life in temporary housing, in both indigence and luxury, ending his days at the Montreux Palace hotel on Lake Geneva. There was, of course, some wishful thinking in all of these forecasting errors, the blindness of hope, but there was a knowledge problem as well. The dynamics of the Lebanese con ict had been patently unpredictable, yet people’s reasoning as they examined the events showed a constant: almost all those who cared seemed convinced that they understood what was going on. Every single day brought occurrences that lay completely outside their forecast, but they could not gure out that they had not forecast them. Much of what took place would have been deemed completely crazy with respect to the past. Yet it did not seem that crazy after the events. This retrospective plausibility causes a discounting of the rarity and conceivability of the event. I later saw the exact same illusion of understanding in business success and the nancial markets. History Does Not Crawl, It Jumps Later, upon replaying the wartime events in my memory as I formulated my ideas on the perception of random events, I developed the governing impression that our minds are wonderful explanation machines, capable of making sense out of almost anything, capable of mounting explanations for all manner of phenomena, and generally incapable of accepting the idea of unpredictability. These events were unexplainable, but intelligent people thought they were capable of providing convincing explanations for them— after the fact. Furthermore, the more intelligent the person, the better sounding the explanation. What’s more worrisome is that all these beliefs and accounts appeared to be logically coherent and devoid of inconsistencies. So I left the place called Lebanon as a teenager, but, since a large number of my relatives and friends remained there, I kept coming back to visit, especially during the hostilities. The war was not continuous: there were periods of ghting interrupted by “permanent” solutions. I felt closer to my roots during times of trouble and experienced the urge to come back and show support to those left behind who were often demoralized by the departures—and envious of the fair-weather friends who could seek economic and personal safety only to return for vacations during these occasional lulls in the con ict. I was unable to work or read when I was outside Lebanon while people were dying, but, paradoxically, I was less concerned by the events and able to pursue my intellectual interests guilt-free when I was inside Lebanon. Interestingly, people partied quite heavily during the war and developed an even bigger taste for luxuries, making the visits quite attractive in spite of the ghting. There were a few di cult questions. How could one have predicted that people who seemed a model of tolerance could become the purest of barbarians overnight? Why was the change so abrupt? I initially thought that perhaps the Lebanese war was truly not possible to predict, unlike other con icts, and that the Levantines were too complicated a race to gure out. Later I slowly realized, as I started to consider all the big events in history, that their irregularity was not a local property. The Levant has been something of a mass producer of consequential events nobody saw coming. Who predicted the rise of Christianity as a dominant religion in the Mediterranean basin, and later in the Western world? The Roman chroniclers of that period did not even take note of the new religion —historians of Christianity are ba ed by the absence of contemporary mentions. Apparently, few of the big guns took the ideas of a seemingly heretical Jew seriously enough to think that he would leave traces for posterity. We only have a single contemporary reference to Jesus of Nazareth —in The Jewish Wars of Josephus—which itself may have been added later by a devout copyist. How about the competing religion that emerged seven centuries later; who forecast that a collection of horsemen would spread their empire and Islamic law from the Indian subcontinent to Spain in just a few years? Even more than the rise of Christianity, it was the spread of Islam (the third edition, so to speak) that carried full unpredictability; many historians looking at the record have been taken aback by the swiftness of the change. Georges Duby, for one, expressed his amazement about how quickly close to ten centuries of Levantine Hellenism were blotted out “with a strike of a sword.” A later holder of the same history chair at the Collège de France, Paul Veyne, aptly talked about religions spreading “like bestsellers”—a comparison that indicates unpredictability. These kinds of discontinuities in the chronology of events did not make the historian’s profession too easy: the studious examination of the past in the greatest of detail does not teach you much about the mind of History; it only gives you the illusion of understanding it. History and societies do not crawl. They make jumps. They go from fracture to fracture, with a few vibrations in between. Yet we (and historians) like to believe in the predictable, small incremental progression. It struck me, a belief that has never left me since, that we are just a great machine for looking backward, and that humans are great at self-delusion. Every year that goes by increases my belief in this distortion. Dear Diary: On History Running Backward Events present themselves to us in a distorted way. Consider the nature of information: of the millions, maybe even trillions, of small facts that prevail before an event occurs, only a few will turn out to be relevant later to your understanding of what happened. Because your memory is limited and ltered, you will be inclined to remember those data that subsequently match the facts, unless you are like the eponymous Funes in the short story by Jorge Luis Borges, “Funes, the Memorious,” who forgets nothing and seems condemned to live with the burden of the accumulation of unprocessed information. (He does not manage to live too long.) I had my rst exposure to the retrospective distortion as follows. During my childhood I had been a voracious, if unsteady, reader, but I spent the rst phase of the war in a basement, diving body and soul into all manner of books. School was closed and it was raining mortar shells. It is dreadfully boring to be in basements. My initial worries were mostly about how to ght boredom and what to read next*—though being forced to read for lack of other activities is not as enjoyable as reading out of one’s own volition. I wanted to be a philosopher (I still do), so I felt that I needed to make an investment by forcibly studying others’ ideas. Circumstances motivated me to study theoretical and general accounts of wars and con icts, trying to get into the guts of History, to get into the workings of that big machine that generates events. Surprisingly, the book that in uenced me was not written by someone in the thinking business but by a journalist: William Shirer’s Berlin Diary: The Journal of a Foreign Correspondent, 1934–1941. Shirer was a radio correspondent, famous for his book The Rise and Fall of the Third Reich. It occurred to me that the Journal o ered an unusual perspective. I had already read (or read about) the works of Hegel, Marx, Toynbee, Aron, and Fichte on the philosophy of history and its properties and thought that I had a vague idea of the notions of dialectics, to the extent that there was something to understand in these theories. I did not grasp much, except that history had some logic and that things developed through contradiction (or opposites) in a way that elevated mankind into higher forms of society—that kind of thing. This sounded awfully similar to the theorizing around me about the war in Lebanon. To this day I surprise people who put the ludicrous question to me about what books “shaped my thinking” by telling them that this book taught me (albeit inadvertently) the most about philosophy and theoretical history— and, we will see, about science as well, since I learned the di erence between forward and backward processes. How? Simply, the diary purported to describe the events as they were taking place, not after. I was in a basement with history audibly unfolding above me (the sound of mortar shells kept me up all night). I was a teenager attending the funerals of classmates. I was experiencing a nontheoretical unfolding of History and I was reading about someone apparently experiencing history as it went along. I made e orts to mentally produce a movielike representation of the future and realized it was not so obvious. I realized that if I were to start writing about the events later they would seem more … historical. There was a di erence between the before and the after. The journal was purportedly written without Shirer knowing what was going to happen next, when the information available to him was not corrupted by the subsequent outcomes. Some comments here and there were quite illuminating, particularly those concerning the French belief that Hitler was a transitory phenomenon, which explained their lack of preparation and subsequent rapid capitulation. At no time was the extent of the ultimate devastation deemed possible. While we have a highly unstable memory, a diary provides indelible facts recorded more or less immediately; it thus allows the xation of an unrevised perception and enables us to later study events in their own context. Again, it is the purported method of description of the event, not its execution, that was important. In fact, it is likely that Shirer and his editors did some cheating, since the book was published in 1941 and publishers, I am told, are in the business of delivering texts to the general public instead of providing faithful depictions of the authors’ mind-sets stripped of retrospective distortions. (By “cheating,” I mean removing at the time of publication elements that did not turn out to be relevant to what happened, thus enhancing those that may interest the public. Indeed the editing process can be severely distorting, particularly when the author is assigned what is called a “good editor.”) Still, encountering Shirer’s book provided me with an intuition about the workings of history. One would suppose that people living through the beginning of WWII had an inkling that something momentous was taking place. Not at all.* Shirer’s diary turned out to be a training program in the dynamics of uncertainty. I wanted to be a philosopher, not knowing at the time what most professional philosophers did for a living. The idea led me to adventure (rather to the adventurous practice of uncertainty) and also to mathematical and scienti c pursuits instead. Education in a Taxicab I will introduce the third element of the triplet, the curse of learning, as follows. I closely watched my grandfather, who was minister of defense, and later minister of the interior and deputy prime minister in the early days of the war, before the fading of his political role. In spite of his position he did not seem to know what was going to happen any more than did his driver, Mikhail. But unlike my grandfather, Mikhail used to repeat “God knows” as his main commentary on events, transferring the task of understanding higher up. I noticed that very intelligent and informed persons were at no advantage over cabdrivers in their predictions, but there was a crucial di erence. Cabdrivers did not believe that they understood as much as learned people— really, they were not the experts and they knew it. Nobody knew anything, but elite thinkers thought that they knew more than the rest because they were elite thinkers, and if you’re a member of the elite, you automatically know more than the nonelite. It is not just knowledge but information that can be of dubious value. It came to my notice that almost everybody was acquainted with current events in their smallest details. The overlap between newspapers was so large that you would get less and less information the more you read. Yet everyone was so eager to become familiar with every fact that they read every freshly printed document and listened to every radio station as if the great answer was going to be revealed to them in the next bulletin. People became encyclopedias of who had met with whom and which politician said what to which other politician (and with what tone of voice: “Was he more friendly than usual?”). Yet to no avail. CLUSTERS I also noticed during the Lebanese war that journalists tended to cluster not necessarily around the same opinions but frequently around the same framework of analyses. They assign the same importance to the same sets of circumstances and cut reality into the same categories—once again the manifestation of Platonicity, the desire to cut reality into crisp shapes. What Robert Fisk calls “hotel journalism” further increased the mental contagion. While Lebanon in earlier journalism was part of the Levant, i.e., the eastern Mediterranean, it now suddenly became part of the Middle East, as if someone had managed to transport it closer to the sands of Saudi Arabia. The island of Cyprus, around sixty miles from my village in northern Lebanon, and with almost identical food, churches, and habits, suddenly became part of Europe (of course the natives on both sides became subsequently conditioned). While in the past a distinction had been drawn between Mediterranean and non-Mediterranean (i.e., between the olive oil and the butter), in the 1970s the distinction suddenly became that between Europe and non-Europe. Islam being the wedge between the two, one does not know where to place the indigenous Arabic-speaking Christians (or Jews) in that story. Categorizing is necessary for humans, but it becomes pathological when the category is seen as de nitive, preventing people from considering the fuzziness of boundaries, let alone revising their categories. Contagion was the culprit. If you selected one hundred independent-minded journalists capable of seeing factors in isolation from one another, you would get one hundred di erent opinions. But the process of having these people report in lockstep caused the dimensionality of the opinion set to shrink considerably—they converged on opinions and used the same items as causes. For instance, to depart from Lebanon for a moment, all reporters now refer to the “roaring eighties,” assuming that there was something particularly distinct about that exact decade. And during the Internet bubble of the late 1990s, journalists agreed on crazy indicators as explanatory of the quality of worthless companies that everyone wanted very badly.* If you want to see what I mean by the arbitrariness of categories, check the situation of polarized politics. The next time a Martian visits earth, try to explain to him why those who favor allowing the elimination of a fetus in the mother’s womb also oppose capital punishment. Or try to explain to him why those who accept abortion are supposed to be favorable to high taxation but against a strong military. Why do those who prefer sexual freedom need to be against individual economic liberty? I noticed the absurdity of clustering when I was quite young. By some farcical turn of events, in that civil war of Lebanon, Christians became pro- free market and the capitalistic system—i.e., what a journalist would call “the Right”—and the Islamists became socialists, getting support from Communist regimes (Pravda, the organ of the Communist regime, called them “oppression ghters,” though subsequently when the Russians invaded Afghanistan, it was the Americans who sought association with bin Laden and his Moslem peers). The best way to prove the arbitrary character of these categories, and the contagion e ect they produce, is to remember how frequently these clusters reverse in history. Today’s alliance between Christian fundamentalists and the Israeli lobby would certainly seem puzzling to a nineteenth-century intellectual—Christians used to be anti-Semites and Moslems were the protectors of the Jews, whom they preferred to Christians. Libertarians used to be left-wing. What is interesting to me as a probabilist is that some random event makes one group that initially supports an issue ally itself with another group that supports another issue, thus causing the two items to fuse and unify … until the surprise of the separation. Categorizing always produces reduction in true complexity. It is a manifestation of the Black Swan generator, that unshakable Platonicity that I de ned in the Prologue. Any reduction of the world around us can have explosive consequences since it rules out some sources of uncertainty; it drives us to a misunderstanding of the fabric of the world. For instance, you may think that radical Islam (and its values) are your allies against the threat of Communism, and so you may help them develop, until they send two planes into downtown Manhattan. It was a few years after the beginning of the Lebanese war, as I was attending the Wharton School, at the age of twenty-two, that I was hit with the idea of e cient markets—an idea that holds that there is no way to derive pro ts from traded securities since these instruments have automatically incorporated all the available information. Public information can therefore be useless, particularly to a businessman, since prices can already “include” all such information, and news shared with millions gives you no real advantage. Odds are that one or more of the hundreds of millions of other readers of such information will already have bought the security, thus pushing up the price. I then completely gave up reading newspapers and watching television, which freed up a considerable amount of time (say one hour or more a day, enough time to read more than a hundred additional books per year, which, after a couple of decades, starts mounting). But this argument was not quite the entire reason for my dictum in this book to avoid the newspapers, as we will see further bene ts in avoiding the toxicity of information. It was initially a great excuse to avoid keeping up with the minutiae of business, a perfect alibi since I found nothing interesting about the details of the business world —inelegant, dull, pompous, greedy, unintellectual, sel sh, and boring. Where Is the Show? Why someone with plans to become a “philosopher” or a “scienti c philosopher of history” would wind up in business school, and the Wharton School no less, still escapes me. There I saw that it was not merely some inconsequential politician in a small and antique country (and his philosophical driver Mikhail) who did not know what was going on. After all, people in small countries are supposed to not know what is going on. What I saw was that in one of the most prestigious business schools in the world, in the most potent country in the history of the world, the executives of the most powerful corporations were coming to describe what they did for a living, and it was possible that they too did not know what was going on. As a matter of fact, in my mind it was far more than a possibility. I felt in my spine the weight of the epistemic arrogance of the human race.* I became obsessive. At the time, I started becoming conscious of my subject —the highly improbable consequential event. And it was not only well-dressed, testosterone-charged corporate executives who were usually fooled by this concentrated luck, but persons of great learning. This awareness turned my Black Swan from a problem of lucky or unlucky people in business into a problem of knowledge and science. My idea is that not only are some scienti c results useless in real life, because they underestimate the impact of the highly improbable (or lead us to ignore it), but that many of them may be actually creating Black Swans. These are not just taxonomic errors that can make you unk a class in ornithology. I started to see the consequences of the idea. 8¾ LBS LATER Four and a half years after my graduation from Wharton (and 8¾ pounds heavier), on October 19, 1987, I walked home from the o ces of the investment bank Credit Suisse First Boston in Midtown Manhattan to the Upper East Side. I walked slowly, as I was in a bewildered state. That day saw a traumatic nancial event: the largest market drop in (modern) history. It was all the more traumatic in that it took place at a time when we thought we had become su ciently sophisticated with all these intelligent-talking Platoni ed economists (with their phony bell curve-based equations) to prevent, or at least forecast and control, big shocks. The drop was not even the response to any discernible news. The occurrence of the event lay outside anything one could have imagined on the previous day—had I pointed out its possibility, I would have been called a lunatic. It quali ed as a Black Swan, but I did not know the expression then. I ran into a colleague of mine, Demetrius, on Park Avenue, and, as I started talking to him, an anxiety-ridden woman, losing all inhibitions, jumped into the conversation: “Hey, do the two of you know what’s going on?” People on the sidewalk looked dazed. Earlier I had seen a few adults silently sobbing in the trading room of First Boston. I had spent the day at the epicenter of the events, with shell-shocked people running around like rabbits in front of headlights. When I got home, my cousin Alexis called to tell me that his neighbor committed suicide, jumping from his upper- oor apartment. It did not even feel eerie. It felt like Lebanon, with a twist: having seen both, I was struck that nancial distress could be more demoralizing than war (just consider that nancial problems and the accompanying humiliations can lead to suicide, but war doesn’t appear to do so directly). I feared a Pyrrhic victory: I had been vindicated intellectually, but I was afraid of being too right and seeing the system crumble under my feet. I did not really want to be that right. I will always remember the late Jimmy P. who, seeing his net worth in the process of melting down, kept half-jokingly begging the price on the screen to stop moving. But I realized then and there that I did not give a hoot about the money. I experienced the strangest feeling I have ever had in my life, this deafening trumpet signaling to me that I was right, so loudly that it made my bones vibrate. I have never had it since and will never be able to explain it to those who have never experienced it. It was a physical sensation, perhaps a mixture of joy, pride, and terror. And I felt vindicated? How? During the one or two years after my arrival at Wharton, I had developed a precise but strange specialty: betting on rare and unexpected events, those that were on the Platonic fold, and considered “inconceivable” by the Platonic “experts.” Recall that the Platonic fold is where our representation of reality ceases to apply—but we do not know it. For I was early to embrace, as a day job, the profession of “quantitative nance.” I became a “quant” and trader at the same time—a quant is a brand of industrial scientist who applies mathematical models of uncertainty to nancial (or socioeconomic) data and complex nancial instruments. Except that I was a quant exactly in reverse: I studied the aws and the limits of these models, looking for the Platonic fold where they break down. Also I engaged in speculative trading, not “just tawk,” which was rare for quants since they were prevented from “taking risks,” their role being con ned to analysis, not decision making. I was convinced that I was totally incompetent in predicting market prices—but that others were generally incompetent also but did not know it, or did not know that they were taking massive risks. Most traders were just “picking pennies in front of a streamroller,” exposing themselves to the high-impact rare event yet sleeping like babies, unaware of it. Mine was the only job you could do if you thought of yourself as risk-hating, risk-aware, and highly ignorant. Also, the technical baggage that comes with being a quant (a mixture of applied mathematics, engineering, and statistics), in addition to the immersion in practice, turned out to be very useful for someone wanting to be a philosopher.* First, when you spend a couple of decades doing mass-scale empirical work with data and taking risks based on such studies, you can easily spot elements in the texture of the world that the Platoni ed “thinker” is too brainwashed, or threatened, to see. Second, it allowed me to become formal and systematic in my thinking instead of wallowing in the anecdotal. Finally, both the philosophy of history and epistemology (the philosophy of knowledge) seemed inseparable from the empirical study of times series data, which is a succession of numbers in time, a sort of historical document containing numbers instead of words. And numbers are easy to process on computers. Studying historical data makes you conscious that history runs forward, not backward, and that it is messier than narrated accounts. Epistemology, the philosophy of history, and statistics aim at understanding truths, investigating the mechanisms that generate them, and separating regularity from the coincidental in historical matters. They all address the question of what one knows, except that they are all to be found in di erent buildings, so to speak. The Four-Letter Word of Independence That night, on October 19, 1987, I slept for twelve hours straight. It was hard to tell my friends, all hurt in some manner by the crash, about this feeling of vindication. Bonuses at the time were a fraction of what they are today, but if my employer, First Boston, and the nancial system survived until year-end, I would get the equivalent of a fellowship. This is sometimes called “f*** you money,” which, in spite of its coarseness, means that it allows you to act like a Victorian gentleman, free from slavery. It is a psychological bu er: the capital is not so large as to make you spoiled-rich, but large enough to give you the freedom to choose a new occupation without excessive consideration of the nancial rewards. It shields you from prostituting your mind and frees you from outside authority—any outside authority. (Independence is person-speci c: I have always been taken aback at the high number of people in whom an astonishingly high income led to additional sycophancy as they became more dependent on their clients and employers and more addicted to making even more money.) While not substantial by some standards, it literally cured me of all nancial ambition—it made me feel ashamed whenever I diverted time away from study for the pursuit of material wealth. Note that the designation f*** you corresponds to the exhilarating ability to pronounce that compact phrase before hanging up the phone. These were the days when it was extremely common for traders to break phones when they lost money. Some resorted to destroying chairs, tables, or whatever would make noise. Once, in the Chicago pits, another trader tried to strangle me and it took four security guards to drag him away. He was irate because I was standing in what he deemed his “territory.” Who would want to leave such an environment? Compare it to lunches in a drab university cafeteria with gentle-mannered professors discussing the latest departmental intrigue. So I stayed in the quant and trading businesses (I’m still there), but organized myself to do minimal but intense (and entertaining) work, focus only on the most technical aspects, never attend business “meetings,” avoid the company of “achievers” and people in suits who don’t read books, and take a sabbatical year for every three on average to ll up gaps in my scienti c and philosophical culture. To slowly distill my single idea, I wanted to become a âneur, a professional meditator, sit in cafés, lounge, unglued to desks and organization structures, sleep as long as I needed, read voraciously, and not owe any explanation to anybody. I wanted to be left alone in order to build, small steps at a time, an entire system of thought based on my Black Swan idea. Limousine Philosopher  The war in Lebanon and the crash of 1987 seemed identical phenomena. It became obvious to me that nearly everyone had a mental blindspot in acknowledging the role of such events: it was as if they were not able to see these mammoths, or that they rapidly forgot about them. The answer was looking straight at me: it was a psychological, perhaps even biological, blindness; the problem lay not in the nature of events, but in the way we perceived them. I end this autobiographical prelude with the following story. I had no de ned specialty (outside of my day job), and wanted none. When people at cocktail parties asked me what I did for a living, I was tempted to answer, “I am a skeptical empiricist and a âneur-reader, someone committed to getting very deep into an idea,” but I made things simple by saying that I was a limousine driver. Once, on a transatlantic ight, I found myself upgraded to rst class next to an expensively dressed, high-powered lady dripping with gold and jewelry who continuously ate nuts (low-carb diet, perhaps), insisted on drinking only Evian, all the while reading the European edition of The Wall Street Journal. She kept trying to start a conversation in broken French, since she saw me reading a book (in French) by the sociologist-philosopher Pierre Bourdieu— which, ironically, dealt with the marks of social distinction. I informed her (in English) that I was a limousine driver, proudly insisting that I only drove “very upper-end” cars. An icy silence lasted the whole ight, and, although I could feel the tension, it allowed me to read in peace. * It is remarkable how fast and how e ectively you can construct a nationality with a ag, a few speeches, and a national anthem; to this day I avoid the label “Lebanese,” preferring the less restrictive “Levantine” designation. * Benoît Mandelbrot, who had a similar experience at about the same age, though close to four decades earlier, remembers his own war  episode as long stretches of painful boredom punctuated by brief moments of extreme fear. * The historian Niall Ferguson showed that, despite all the standard accounts of the buildup to the Great War, which describe “mounting tensions” and “escalating crises,” the con ict came as a surprise. Only retrospectively was it seen as unavoidable by backward-looking historians. Ferguson used a clever empirical argument to make his point: he looked at the prices of imperial bonds, which normally include investors’ anticipation of government’s nancing needs and decline in expectation of con icts since wars cause severe de cits. But bond prices did not re ect the anticipation of war. Note that this study illustrates, in addition, how working with prices can provide a good understanding of history. * We will see in Chapter 10 some clever quantitative tests done to prove such herding; they show that, in many subject matters, the distance between opinions is remarkably narrower than the distance between the average of opinions and truth. * I then realized that the great strength of the free-market system is the fact that company executives don’t need to know what’s going on. * I specialized in complicated nancial instruments called “derivatives,” those that required advanced mathematics—but for which the errors for using the wrong mathematics were the greatest. The subject was new and attractive enough for me to get a doctorate in it. Note that I was not able to build a career just by betting on Black Swans—there were not enough tradable opportunities. I could, on the other hand, avoid being exposed to them by protecting my portfolio against large losses. So, in order to eliminate the dependence on randomness, I focused on technical ine ciencies between complicated instruments, and on exploiting these opportunities without exposure to the rare event, before they disappeared as my competitors became technologically advanced. Later on in my career I discovered the easier (and less randomness laden) business of protecting, insurance-style, large portfolios against the Black Swan. Chapter Two YEVGENIA’S BLACK SWAN Pink glasses and success—How Yevgenia stops marrying philosophers— I told you so Five years ago, Yevgenia Nikolayevna Krasnova was an obscure and unpublished novelist, with an unusual background. She was a neuroscientist with an interest in philosophy (her rst three husbands had been philosophers), and she got it into her stubborn Franco-Russian head to express her research and ideas in literary form. She dressed up her theories as stories, and mixed them with all manner of autobiographical commentary. She avoided the journalistic prevarications of contemporary narrative non ction (“On a clear April morning, John Smith left his house. …”). Foreign dialogue was always written in the original language, with translations appended like movie subtitles. She refused to dub into bad English conversations that took place in bad Italian.* No publisher would have given her the time of day, except that there was, at the time, some interest in those rare scientists who could manage to express themselves in semi-understandable sentences. A few publishers agreed to speak with her; they hoped that she would grow up and write a “popular science book on consciousness.” She received enough attention to get the courtesy of rejection letters and occasional insulting comments instead of the far more insulting and demeaning silence. Publishers were confused by her manuscript. She could not even answer their rst question: “Is this ction or non ction?” Nor could she respond to the “Who is this book written for?” on the publishers’ book proposal forms. She was told, “You need to understand who your audience is” and “amateurs write for themselves, professionals write for others.” She was also told to conform to a precise genre because “bookstores do not like to be confused and need to know where to place a book on the shelves.” One editor protectively added, “This, my dear friend, will only sell ten copies, including those bought by your ex-husbands and family members.” She had attended a famous writing workshop ve years earlier and came out nauseated. “Writing well” seemed to mean obeying arbitrary rules that had grown into gospel, with the con rmatory reinforcement of what we call “experience.” The writers she met were learning to retro t what was deemed successful: they all tried to imitate stories that had appeared in past issues of The New Yorker—not realizing that most of what is new, by de nition, cannot be modeled on past issues of The New Yorker. Even the idea of a “short story” was a me-too concept to Yevgenia. The workshop instructor, gentle but rm in his delivery, told her that her case was utterly hopeless. Yegvenia ended up posting the entire manuscript of her main book, A Story of Recursion, on the Web. There it found a small audience, which included the shrewd owner of a small unknown publishing house, who wore pink-rimmed glasses and spoke primitive Russian (convinced that he was uent). He o ered to publish her, and agreed to her condition to keep her text completely unedited. He o ered her a fraction of the standard royalty rate in return for her editorial stricture—he had so little to lose. She accepted since she had no choice. It took ve years for Yevgenia to graduate from the “egomaniac without anything to justify it, stubborn and di cult to deal with” category to “persevering, resolute, painstaking, and ercely independent.” For her book slowly caught re, becoming one of the great and strange successes in literary history, selling millions of copies and drawing so-called critical acclaim. The start-up house has since become a big corporation, with a (polite) receptionist to greet visitors as they enter the main o ce. Her book has been translated into forty languages (even French). You see her picture everywhere. She is said to be a pioneer of something called the Consilient School. Publishers now have a theory that “truck drivers who read books do not read books written for truck drivers” and hold that “readers despise writers who pander to them.” A scienti c paper, it is now understood, can hide trivialities or irrelevance with equations and jargon; consilient prose, by exposing an idea in raw form, allows it to be judged by the public. Today, Yevgenia has stopped marrying philosophers (they argue too much), and she hides from the press. In classrooms, literary scholars discuss the many clues indicating the inevitability of the new style. The distinction between ction and non ction is considered too archaic to withstand the challenges of modern society. It was so evident that we needed to remedy the fragmentation between art and science. After the fact, her talent was so obvious. Many of the editors she later met blamed her for not coming to them, convinced that they would have immediately seen the merit in her work. In a few years, a literary scholar will write the essay “From Kundera to Krasnova,” showing how the seeds of her work can be found in Kundera—a precursor who mixed essay and metacommentary (Yevgenia never read Kundera, but did see the movie version of one of his books—there was no commentary in the movie). A prominent scholar will show how the in uence of Gregory Bateson, who injected autobiographical scenes into his scholarly research papers, is visible on every page (Yevgenia has never heard of Bateson). Yevgenia’s book is a Black Swan. * Her third husband was an Italian philosopher. Chapter Three THE SPECULATOR AND THE PROSTITUTE On the critical difference between speculators and prostitutes— Fairness, unfairness, and Black Swans—Theory of knowledge and professional incomes—How Extremistan is not the best place to visit, except, perhaps, if you are a winner Yevgenia’s rise from the second basement to superstar is possible in only one environment, which I call Extremistan.* I will soon introduce the central distinction between the Black Swan–generating province of Extremistan and the tame, quiet, and uneventful province of Mediocristan. THE BEST (WORST) ADVICE When I play back in my mind all the “advice” people have given me, I see that only a couple of ideas have stuck with me for life. The rest has been mere words, and I am glad that I did not heed most of it. Most consisted of recommendations such as “be measured and reasonable in your statements,” contradicting the Black Swan idea, since empirical reality is not “measured,” and its own version of “reasonableness” does not correspond to the conventional middlebrow de nition. To be genuinely empirical is to re ect reality as faithfully as possible; to be honorable implies not fearing the appearance and consequences of being outlandish. The next time someone pesters you with unneeded advice, gently remind him of the fate of the monk whom Ivan the Terrible put to death for delivering uninvited (and moralizing) advice. It works as a short-term cure. The most important piece of advice was, in retrospect, bad, but it was also, paradoxically, the most consequential, as it pushed me deeper into the dynamics of the Black Swan. It came when I was twenty-two, one February afternoon, in the corridor of a building at 3400 Walnut Street in Philadelphia, where I lived. A second-year Wharton student told me to get a profession that is “scalable,” that is, one in which you are not paid by the hour and thus subject to the limitations of the amount of your labor. It was a very simple way to discriminate among professions and, from that, to generalize a separation between types of uncertainty—and it led me to the major philosophical problem, the problem of induction, which is the technical name for the Black Swan. It allowed me to turn the Black Swan from a logical impasse into an easy-to-implement solution, and, as we will see in the next chapters, to ground it in the texture of empirical reality. How did career advice lead to such ideas about the nature of uncertainty? Some professions, such as dentists, consultants, or massage professionals, cannot be scaled: there is a cap on the number of patients or clients you can see in a given period of time. If you are a prostitute, you work by the hour and are (generally) paid by the hour. Furthermore, your presence is (I assume) necessary for the service you provide. If you open a fancy restaurant, you will at best steadily ll up the room (unless you franchise it). In these professions, no matter how highly paid, your income is subject to gravity. Your revenue depends on your continuous e orts more than on the quality of your decisions. Moreover, this kind of work is largely predictable: it will vary, but not to the point of making the income of a single day more signi cant than that of the rest of your life. In other words, it will not be Black Swan driven. Yevgenia Nikolayevna would not have been able to cross the chasm between underdog and supreme hero overnight had she been a tax accountant or a hernia specialist (but she would not have been an underdog either). Other professions allow you to add zeroes to your output (and your income), if you do well, at little or no extra e ort. Now being lazy, considering laziness as an asset, and eager to free up the maximum amount of time in my day to meditate and read, I immediately (but mistakenly) drew a conclusion. I separated the “idea” person, who sells an intellectual product in the form of a transaction or a piece of work, from the “labor” person, who sells you his work. If you are an idea person, you do not have to work hard, only think intensely. You do the same work whether you produce a hundred units or a thousand. In quant trading, the same amount of work is involved in buying a hundred shares as in buying a hundred thousand, or even a million. It is the same phone call, the same computation, the same legal document, the same expenditure of brain cells, the same e ort in verifying that the transaction is right. Furthermore, you can work from your bathtub or from a bar in Rome. You can use leverage as a replacement for work! Well, okay, I was a little wrong about trading: one cannot work from a bathtub, but, when done right, the job allows considerable free time. The same property applies to recording artists or movie actors: you let the sound engineers and projectionists do the work; there is no need to show up at every performance in order to perform. Similarly, a writer expends the same e ort to attract one single reader as she would to capture several hundred million. J. K. Rowling, the author of the Harry Potter books, does not have to write each book again every time someone wants to read it. But this is not so for a baker: he needs to bake every single piece of bread in order to satisfy each additional customer. So the distinction between writer and baker, speculator and doctor, fraudster and prostitute, is a helpful way to look at the world of activities. It separates those professions in which one can add zeroes of income with no greater labor from those in which one needs to add labor and time (both of which are in limited supply)—in other words, those subjected to gravity. BEWARE THE SCALABLE But why was the advice from my fellow student bad? If the advice was helpful, and it was, in creating a classi cation for ranking uncertainty and knowledge, it was a mistake as far as choices of profession went. It might have paid o for me, but only because I was lucky and happened to be “in the right place at the right time,” as the saying goes. If I myself had to give advice, I would recommend someone pick a profession that is not scalable! A scalable profession is good only if you are successful; they are more competitive, produce monstrous inequalities, and are far more random, with huge disparities between e orts and rewards—a few can take a large share of the pie, leaving others out entirely at no fault of their own. One category of profession is driven by the mediocre, the average, and the middle-of-the-road. In it, the mediocre is collectively consequential. The other has either giants or dwarves—more precisely, a very small number of giants and a huge number of dwarves. Let us see what is behind the formation of unexpected giants—the Black Swan formation. The Advent of Scalability Consider the fate of Giaccomo, an opera singer at the end of the nineteenth century, before sound recording was invented. Say he performs in a small and remote town in central Italy. He is shielded from those big egos at La Scala in Milan and other major opera houses. He feels safe as his vocal cords will always be in demand somewhere in the district. There is no way for him to export his singing, and there is no way for the big guns to export theirs and threaten his local franchise. It is not yet possible for him to store his work, so his presence is needed at every performance, just as a barber is (still) needed today for every haircut. So the total pie is unevenly split, but only mildly so, much like your calorie consumption. It is cut in a few pieces and everyone has a share; the big guns have larger audiences and get more invitations than the small guy, but this is not too worrisome. Inequalities exist, but let us call them mild. There is no scalability yet, no way to double the largest in-person audience without having to sing twice. Now consider the e ect of the rst music recording, an invention that introduced a great deal of injustice. Our ability to reproduce and repeat performances allows me to listen on my laptop to hours of background music of the pianist Vladimir Horowitz (now extremely dead) performing Rachmanino ’s Preludes, instead of to the local Russian émigré musician (still living), who is now reduced to giving piano lessons to generally untalented children for close to minimum wage. Horowitz, though dead, is putting the poor man out of business. I would rather listen to Vladimir Horowitz or Arthur Rubinstein for $10.99 a CD than pay $9.99 for one by some unknown (but very talented) graduate of the Juilliard School or the Prague Conservatory. If you ask me why I select Horowitz, I will answer that it is because of the order, rhythm, or passion, when in fact there are probably a legion of people I have never heard about, and will never hear about—those who did not make it to the stage, but who might play just as well. Some people naïvely believe that the process of unfairness started with the gramophone, according to the logic that I just presented. I disagree. I am convinced that the process started much, much earlier, with our DNA, which stores information about our selves and allows us to repeat our performance without our being there by spreading our genes down the generations. Evolution is scalable: the DNA that wins (whether by luck or survival advantage) will reproduce itself, like a bestselling book or a successful record, and become pervasive. Other DNA will vanish. Just consider the di erence between us humans (excluding nancial economists and businessmen) and other living beings on our planet.  Furthermore, I believe that the big transition in social life came not with the gramophone, but when someone had the great but unjust idea to invent the alphabet, thus allowing us to store information and reproduce it. It accelerated further when another inventor had the even more dangerous and iniquitous notion of starting a printing press, thus promoting texts across boundaries and triggering what ultimately grew into a winner-take-all ecology. Now, what was so unjust about the spread of books? The alphabet allowed stories and ideas to be replicated with high delity and without limit, without any additional expenditure of energy on the author’s part for the subsequent performances. He didn’t even have to be alive for them—death is often a good career move for an author. This implies that those who, for some reason, start getting some attention can quickly reach more minds than others and displace the competitors from the bookshelves. In the days of bards and troubadours, everyone had an audience. A storyteller, like a baker or a coppersmith, had a market, and the assurance that none from far away could dislodge him from his territory. Today, a few take almost everything; the rest, next to nothing. By the same mechanism, the advent of the cinema displaced neighborhood actors, putting the small guys out of business. But there is a di erence. In pursuits that have a technical component, like being a pianist or a brain surgeon, talent is easy to ascertain, with subjective opinion playing a relatively small part. The inequity comes when someone perceived as being marginally better gets the whole pie. In the arts—say the cinema—things are far more vicious. What we call “talent” generally comes from success, rather than its opposite. A great deal of empiricism has been done on the subject, most notably by Art De Vany, an insightful and original thinker who singlemindedly studied wild uncertainty in the movies. He showed that, sadly, much of what we ascribe to skills is an after-the-fact attribution. The movie makes the actor, he claim