Psychology of Intelligence Analysis PDF

 of by Richards J. Heuer, Jr. CENTER for the STUDY of INTELLIGENCE Central Intelligence Agency 1999 Tis book was prepared primarily for the use of US Government ofcials, and the format, coverage, and content were designed to meet their s...

 of by Richards J. Heuer, Jr. CENTER for the STUDY of INTELLIGENCE Central Intelligence Agency 1999 Tis book was prepared primarily for the use of US Government ofcials, and the format, coverage, and content were designed to meet their spe- cifc requirements. Because this book is now out of print, this Portable Document File (PDF) is formatted for two-sided printing to facilitate desktop publishing. It may be used by US Government agencies to make copies for govern- ment purposes and by non-governmental organizations to make copies for educational purposes. Because this book may be subject to copyright restriction, copies may not be made for any commercial purpose. Tis book will be available at www.odci.gov/csi. All statements of fact, opinion, or analysis expressed in the main text of this book are those of the author. Similarly, all such statements in the Forward and the Introduction are those of the respective authors of those sections. Such statements of fact, opinion, or analysis do not necessarily refect the ofcial positions or views of the Central Intelligence Agency or any other component of the US Intelligence Community. Nothing in the contents of this book should be con- strued as asserting or implying US Government endorsement of fac- tual statements or interpretations. ISBN 1 929667-00-0 Originally published in 1999. iii  Psychology of Intelligence Analysis by Richards J. Heuer, Jr. Author’s Preface....................................................vi Foreword...............................................................ix Introduction.......................................................xiii PART I—OUR MENTAL MACHINERY...............1 Chapter 1: Tinking About Tinking...........................1 Chapter 2: Perception: Why Can’t We See What Is Tere To Be Seen?............................................7 Chapter 3: Memory: How Do We Remember What We Know?.........................................................17 PART II—TOOLS FOR THINKING..................31 Chapter 4: Strategies for Analytical Judgment: Transcending the Limits of Incomplete Information...31 Chapter 5: Do You Really Need More Information?...51 Chapter 6: Keeping an Open Mind............................65 Chapter 7: Structuring Analytical Problems................85 Chapter 8: Analysis of Competing Hypotheses...........95 PART III—COGNITIVE BIASES......................111 Chapter 9: What Are Cognitive Biases?.....................111 Chapter 10: Biases in Evaluation of Evidence............115 v  Chapter 11: Biases in Perception of Cause and Efect127 Chapter 12: Biases in Estimating Probabilities..........147 Chapter 13: Hindsight Biases in Evaluation of Intelligence Reporting...............................................161 PART IV—CONCLUSIONS.............................173 Chapter 14: Improving Intelligence Analysis.............173 vi  Author’s Preface Tis volume pulls together and republishes, with some editing, updating, and additions, articles written during 1978–86 for internal use within the CIA Directorate of Intelligence. Four of the articles also appeared in the Intelligence Community journal Studies in Intelligence during that time frame. Te information is relatively timeless and still relevant to the never-ending quest for better analysis. Te articles are based on reviewing cognitive psychology literature concerning how people process information to make judgments on in- complete and ambiguous information. I selected the experiments and fndings that seem most relevant to intelligence analysis and most in need of communication to intelligence analysts. I then translated the techni- cal reports into language that intelligence analysts can understand and interpreted the relevance of these fndings to the problems intelligence analysts face. Te result is a compromise that may not be wholly satisfactory to either research psychologists or intelligence analysts. Cognitive psychol- ogists and decision analysts may complain of oversimplifcation, while the non-psychologist reader may have to absorb some new terminology. Unfortunately, mental processes are so complex that discussion of them does require some specialized vocabulary. Intelligence analysts who have read and thought seriously about the nature of their craft should have no difculty with this book. Tose who are plowing virgin ground may require serious efort. I wish to thank all those who contributed comments and suggestions on the draft of this book: Jack Davis (who also wrote the Introduction); four former Directorate of Intelligence (DI) analysts whose names cannot be cited here; my current colleague, Prof. Teodore Sarbin; and my edi- tor at the CIA’s Center for the Study of Intelligence, Hank Appelbaum. All made many substantive and editorial suggestions that helped greatly to make this a better book. —Richards J. Heuer, Jr. vii  Foreword By Douglas MacEachin1 My frst exposure to Dick Heuer’s work was about 18 years ago, and I have never forgotten the strong impression it made on me then. Tat was at about the midpoint in my own career as an intelligence analyst. After another decade and a half of experience, and the opportunity dur- ing the last few years to study many historical cases with the beneft of archival materials from the former USSR and Warsaw Pact regimes, read- ing Heuer’s latest presentation has had even more resonance. I know from frst-hand encounters that many CIA ofcers tend to react skeptically to treatises on analytic epistemology. Tis is understand- able. Too often, such treatises end up prescribing models as answers to the problem. Tese models seem to have little practical value to intelligence analysis, which takes place not in a seminar but rather in a fast-breaking world of policy. But that is not the main problem Heuer is addressing. What Heuer examines so clearly and efectively is how the human thought process builds its own models through which we process infor- mation. Tis is not a phenomenon unique to intelligence; as Heuer’s research demonstrates, it is part of the natural functioning of the human cognitive process, and it has been demonstrated across a broad range of felds ranging from medicine to stock market analysis. Te process of analysis itself reinforces this natural function of the human brain. Analysis usually involves creating models, even though they may not be labeled as such. We set forth certain understandings and expectations about cause-and-efect relationships and then process and interpret information through these models or flters. Te discussion in Chapter 5 on the limits to the value of additional information deserves special attention, in my view—particularly for an 1. Douglas MacEachin is a former CIA Deputy Director of Intelligence. After 32 years with the Agency, he retired in 1997 and became a Senior Fellow at Harvard University’s John F. Kennedy School of Government. ix intelligence organization. What it illustrates is that too often, newly ac- quired information is evaluated and processed through the existing ana- lytic model, rather than being used to reassess the premises of the model itself. Te detrimental efects of this natural human tendency stem from the raison d’etre of an organization created to acquire special, critical in- formation available only through covert means, and to produce analysis integrating this special information with the total knowledge base. I doubt that any veteran intelligence ofcer will be able to read this book without recalling cases in which the mental processes described by Heuer have had an adverse impact on the quality of analysis. How many times have we encountered situations in which completely plausible premises, based on solid expertise, have been used to construct a logically valid forecast—with virtually unanimous agreement—that turned out to be dead wrong? In how many of these instances have we determined, with hindsight, that the problem was not in the logic but in the fact that one of the premises—however plausible it seemed at the time—was incorrect? In how many of these instances have we been forced to admit that the erroneous premise was not empirically based but rather a conclu- sion developed from its own model (sometimes called an assumption)? And in how many cases was it determined after the fact that information had been available which should have provided a basis for questioning one or more premises, and that a change of the relevant premise(s) would have changed the analytic model and pointed to a diferent outcome? Te commonly prescribed remedy for shortcomings in intelligence analysis and estimates—most vociferously after intelligence “failures”—is a major increase in expertise. Heuer’s research and the studies he cites pose a serious challenge to that conventional wisdom. Te data show that expertise itself is no protection from the common analytic pitfalls that are endemic to the human thought process. Tis point has been demon- strated in many felds beside intelligence analysis. A review of notorious intelligence failures demonstrates that the an- alytic traps caught the experts as much as anybody. Indeed, the data show that when experts fall victim to these traps, the efects can be aggravated by the confdence that attaches to expertise—both in their own view and in the perception of others. Tese observations should in no way be construed as a denigration of the value of expertise. On the contrary, my own 30-plus years in the business of intelligence analysis biased me in favor of the view that, end- x less warnings of information overload notwithstanding, there is no such thing as too much information or expertise. And my own observations of CIA analysts sitting at the same table with publicly renowned experts have given me great confdence that attacks on the expertise issue are grossly misplaced. Te main diference is that one group gets to promote its reputations in journals, while the other works in a closed environment in which the main readers are members of the intelligence world’s most challenging audience—the policymaking community. Te message that comes through in Heuer’s presentation is that in- formation and expertise are a necessary but not sufcient means of mak- ing intelligence analysis the special product that it needs to be. A compa- rable efort has to be devoted to the science of analysis. Tis efort has to start with a clear understanding of the inherent strengths and weaknesses of the primary analytic mechanism—the human mind—and the way it processes information. I believe there is a signifcant cultural element in how intelligence analysts defne themselves: Are we substantive experts employed by CIA, or are we professional analysts and intelligence ofcers whose expertise lies in our ability to adapt quickly to diverse issues and problems and analyze them efectively? In the world at large, substantive expertise is far more abundant than expertise on analytic science and the human mental processing of information. Dick Heuer makes clear that the pitfalls the hu- man mental process sets for analysts cannot be eliminated; they are part of us. What can be done is to train people how to look for and recognize these mental obstacles, and how to develop procedures designed to ofset them. Given the centrality of analytic science for the intelligence mission, a key question that Heuer’s book poses is: Compared with other areas of our business, have we committed a commensurate efort to the study of analytic science as a professional requirement? How do the efort and re- source commitments in this area compare to, for example, the efort and commitment to the development of analysts’ writing skills? Heuer’s book does not pretend to be the last word on this issue. Hopefully, it will be a stimulant for much more work. xi  Introduction Improving Intelligence Analysis at CIA: Dick Heuer’s Contribution to Intelligence Analysis by Jack Davis I applaud CIA’s Center for the Study of Intelligence for making the work of Richards J. Heuer, Jr. on the psychology of intelligence analysis available to a new generation of intelligence practitioners and scholars. Dick Heuer’s ideas on how to improve analysis focus on helping analysts compensate for the human mind’s limitations in dealing with complex problems that typically involve ambiguous information, multi- ple players, and fuid circumstances. Such multi-faceted estimative chal- lenges have proliferated in the turbulent post-Cold War world. Heuer’s message to analysts can be encapsulated by quoting two sentences from Chapter 4 of this book: Intelligence analysts should be self-conscious about their rea- soning processes. Tey should think about how they make judgments and reach conclusions, not just about the judgments and conclusions themselves. Heuer’s ideas are applicable to any analytical endeavor. In this Introduction, I have concentrated on his impact—and that of other pio- neer thinkers in the intelligence analysis feld—at CIA, because that is the institution that Heuer and his predecessors, and I myself, know best, having spent the bulk of our intelligence careers there. 2. Jack Davis served with the Directorate of Intelligence (DI), the National Intelligence Council, and the Ofce of Training during his CIA career. He is now an independent contrac- tor who specializes in developing and teaching analytic tradecraft. Among his publications is Uncertainty, Surprise, and Warning (1996). xiii Leading Contributors to Quality of Analysis Intelligence analysts, in seeking to make sound judgments, are al- ways under challenge from the complexities of the issues they address and from the demands made on them for timeliness and volume of pro- duction. Four Agency individuals over the decades stand out for having made major contributions on how to deal with these challenges to the quality of analysis. My short list of the people who have had the greatest positive im- pact on CIA analysis consists of Sherman Kent, Robert Gates, Douglas MacEachin, and Richards Heuer. My selection methodology was simple. I asked myself: Whose insights have infuenced me the most during my four decades of practicing, teaching, and writing about analysis? Sherman Kent Sherman Kent’s pathbreaking contributions to analysis cannot be done justice in a couple of paragraphs, and I refer readers to fuller treat- ments elsewhere.3 Here I address his general legacy to the analytical pro- fession. Kent, a professor of European history at Yale, worked in the Research and Analysis branch of the Ofce of Strategic Services during World War II. He wrote an infuential book, Strategic Intelligence for American World Power, while at the National War College in the late 1940s. He served as Vice Chairman and then as Chairman of the DCI’s Board of National Estimates from 1950 to 1967. Kent’s greatest contribution to the quality of analysis was to defne an honorable place for the analyst—the thoughtful individual “applying the instruments of reason and the scientifc method”—in an intelligence world then as now dominated by collectors and operators. In a second (1965) edition of Strategic Intelligence, Kent took account of the coming computer age as well as human and technical collectors in proclaiming the centrality of the analyst: Whatever the complexities of the puzzles we strive to solve and whatever the sophisticated techniques we may use to collect 3. See, in particular, the editor’s unclassifed introductory essay and “Tribute” by Harold P. Ford in Donald P. Steury, Sherman Kent and the Board of National Estimates: Collected Essays (CIA, Center for the Study of Intelligence, 1994). Hereinafter cited as Steury, Kent. xiv  the pieces and store them, there can never be a time when the thoughtful man can be supplanted as the intelligence device supreme. More specifcally, Kent advocated application of the techniques of “scientifc” study of the past to analysis of complex ongoing situations and estimates of likely future events. Just as rigorous “impartial” analysis could cut through the gaps and ambiguities of information on events long past and point to the most probable explanation, he contended, the powers of the critical mind could turn to events that had not yet trans- pired to determine the most probable developments.4 To this end, Kent developed the concept of the analytic pyramid, featuring a wide base of factual information and sides comprised of sound assumptions, which pointed to the most likely future scenario at the apex.5 In his proselytizing and in practice, Kent battled against bureaucrat- ic and ideological biases, which he recognized as impediments to sound analysis, and against imprecise estimative terms that he saw as obstacles to conveying clear messages to readers. Although he was aware of what is now called cognitive bias, his writings urge analysts to “make the call” without much discussion of how limitations of the human mind were to be overcome. Not many Agency analysts read Kent nowadays. But he had a pro- found impact on earlier generations of analysts and managers, and his work continues to exert an indirect infuence among practitioners of the analytic profession. Robert Gates Bob Gates served as Deputy Director of Central Intelligence (1986– 1989) and as DCI (1991–1993). But his greatest impact on the quality of CIA analysis came during his 1982–1986 stint as Deputy Director for Intelligence (DDI). 4. Sherman Kent, Writing History, second edition (1967). Te frst edition was published in 1941, when Kent was an assistant professor of history at Yale. In the frst chapter, “Why History,” he presented ideas and recommendations that he later adapted for intelligence analy- sis. 5. Kent, “Estimates and Infuence” (1968), in Steury, Kent. xv  Initially schooled as a political scientist, Gates earned a Ph.D. in Soviet studies at Georgetown while working as an analyst at CIA. As a member of the National Security Council staf during the 1970s, he gained invaluable insight into how policymakers use intelligence anal- ysis. Highly intelligent, exceptionally hard-working, and skilled in the bureaucratic arts, Gates was appointed DDI by DCI William Casey in good part because he was one of the few insiders Casey found who shared the DCI’s views on what Casey saw as glaring defciencies of Agency ana- lysts.6 Few analysts and managers who heard it have forgotten Gates’ blis- tering criticism of analytic performance in his 1982 “inaugural” speech as DDI. Most of the public commentary on Gates and Agency analysis concerned charges of politicization levied against him, and his defense against such charges, during Senate hearings for his 1991 confrmation as DCI. Te heat of this debate was slow to dissipate among CIA analysts, as refected in the pages of Studies in Intelligence, the Agency journal founded by Sherman Kent in the 1950s.7 I know of no written retrospective on Gates’ contribution to Agency analysis. My insights into his ideas about analysis came mostly through an arms-length collaboration in setting up and running an Agency training course entitled “Seminar on Intelligence Successes and Failures.”8 During his tenure as DDI, only rarely could you hold a conversation with ana- lysts or managers without picking up additional viewpoints, thoughtful and otherwise, on what Gates was doing to change CIA analysis. Gates’s ideas for overcoming what he saw as insular, fabby, and in- coherent argumentation featured the importance of distinguishing be- tween what analysts know and what they believe—that is, to make clear what is “fact” (or reliably reported information) and what is the analyst’s opinion (which had to be persuasively supported with evidence). Among his other tenets were the need to seek the views of non-CIA experts, in- 6. Casey, very early in his tenure as DCI (1981-1987), opined to me that the trouble with Agency analysts is that they went from sitting on their rear ends at universities to sitting on their rear ends at CIA, without seeing the real world. 7. “Te Gates Hearings: Politicization and Soviet Analysis at CIA”, Studies in Intelligence (Spring 1994). “Communication to the Editor: Te Gates Hearings: A Biased Account,” Studies in Intelligence (Fall 1994). 8. DCI Casey requested that the Agency’s training ofce provide this seminar so that, at the least, analysts could learn from their own mistakes. DDI Gates carefully reviewed the statement of goals for the seminar, the outline of course units, and the required reading list. xvi cluding academic specialists and policy ofcials, and to present alternate future scenarios. Gates’s main impact, though, came from practice—from his direct involvement in implementing his ideas. Using his authority as DDI, he reviewed critically almost all in-depth assessments and current intelli- gence articles prior to publication. With help from his deputy and two rotating assistants from the ranks of rising junior managers, Gates raised the standards for DDI review dramatically—in essence, from “looks good to me” to “show me your evidence.” As the many drafts Gates rejected were sent back to managers who had approved them—accompanied by the DDI’s comments about in- consistency, lack of clarity, substantive bias, and poorly supported judg- ments—the whole chain of review became much more rigorous. Analysts and their managers raised their standards to avoid the pain of DDI rejec- tion. Both career advancement and ego were at stake. Te rapid and sharp increase in attention paid by analysts and man- agers to the underpinnings for their substantive judgments probably was without precedent in the Agency’s history. Te longer term benefts of the intensifed review process were more limited, however, because insuf- fcient attention was given to clarifying tradecraft practices that would promote analytic soundness. More than one participant in the process observed that a lack of guidelines for meeting Gates’s standards led to a large amount of “wheel-spinning.” Gates’s impact, like Kent’s, has to be seen on two planes. On the one hand, little that Gates wrote on the craft of analysis is read these days. But even though his pre-publication review process was discontinued under his successors, an enduring awareness of his standards still gives pause at jumping to conclusions to many managers and analysts who experienced his criticism frst-hand. Douglas MacEachin Doug MacEachin, DDI from 1993 to 1996, sought to provide an essential ingredient for ensuring implementation of sound analytic stan- dards: corporate tradecraft standards for analysts. Tis new tradecraft was aimed in particular at ensuring that sufcient attention would be paid to cognitive challenges in assessing complex issues. xvii  MacEachin set out his views on Agency analytical faults and correc- tives in Te Tradecraft of Analysis: Challenge and Change in the CIA.9 My commentary on his contributions to sound analysis is also informed by a series of exchanges with him in 1994 and 1995. MacEachin’s university major was economics, but he also showed great interest in philosophy. His Agency career—like Gates’—included an extended assignment to a policymaking ofce. He came away from this experience with new insights on what constitutes “value-added” in- telligence usable by policymakers. Subsequently, as CIA’s senior manager on arms control issues, he dealt regularly with a cadre of tough-minded policy ofcials who let him know in blunt terms what worked as efective policy support and what did not. By the time MacEachin became DDI in 1993, Gates’s policy of DDI front-ofce pre-publication review of nearly all DI analytical stud- ies had been discontinued. MacEachin took a diferent approach; he read—mostly on weekends—and refected on numerous already-pub- lished DI analytical papers. He did not like what he found. In his words, roughly a third of the papers meant to assist the policymaking process had no discernible argumentation to bolster the credibility of intelligence judgments, and another third sufered from fawed argumentation. Tis experience, along with pressures on CIA for better analytic performance in the wake of alleged “intelligence failures” concerning Iraq’s invasion of Kuwait, prompted his decision to launch a major new efort to raise analytical standards.10 MacEachin advocated an approach to structured argumentation called “linchpin analysis,” to which he contributed muscular terms de- signed to overcome many CIA professionals’ distaste for academic no- menclature. Te standard academic term “key variables” became driv- ers. “Hypotheses” concerning drivers became linchpins—assumptions underlying the argument—and these had to be explicitly spelled out. MacEachin also urged that greater attention be paid to analytical pro- cesses for alerting policymakers to changes in circumstances that would increase the likelihood of alternative scenarios. 9. Unclassifed paper published in 1994 by the Working Group on Intelligence Reform, which had been created in 1992 by the Consortium for the Study of Intelligence, Washington, DC. 10. Discussion between MacEachin and the author of this Introduction, 1994. xviii  MacEachin thus worked to put in place systematic and transparent standards for determining whether analysts had met their responsibili- ties for critical thinking. To spread understanding and application of the standards, he mandated creation of workshops on linchpin analysis for managers and production of a series of notes on analytical tradecraft. He also directed that the DI’s performance on tradecraft standards be tracked and that recognition be given to exemplary assessments. Perhaps most ambitious, he saw to it that instruction on standards for analysis was incorporated into a new training course, “Tradecraft 2000.” Nearly all DI managers and analysts attended this course during 1996–97. As of this writing (early 1999), the long-term staying power of MacEachin’s tradecraft initiatives is not yet clear. But much of what he advocated has endured so far. Many DI analysts use variations on his linchpin concept to produce soundly argued forecasts. In the training realm, “Tradecraft 2000” has been supplanted by a new course that teach- es the same concepts to newer analysts. But examples of what MacEachin would label as poorly substantiated analysis are still seen. Clearly, ongo- ing vigilance is needed to keep such analysis from fnding its way into DI products. Richards Heuer Dick Heuer was—and is—much less well known within the CIA than Kent, Gates, and MacEachin. He has not received the wide acclaim that Kent enjoyed as the father of professional analysis, and he has lacked the bureaucratic powers that Gates and MacEachin could wield as DDIs. But his impact on the quality of Agency analysis arguably has been at least as important as theirs. Heuer received a degree in philosophy in 1950 from Williams College, where, he notes, he became fascinated with the fundamental epistemological question, “What is truth and how can we know it?” In 1951, while a graduate student at the University of California’s Berkeley campus, he was recruited as part of the CIA’s buildup during the Korean War. Te recruiter was Richard Helms, OSS veteran and rising player in the Agency’s clandestine service. Future DCI Helms, according to Heuer, was looking for candidates for CIA employment among recent graduates of Williams College, his own alma mater. Heuer had an added advantage xix as a former editor of the college’s newspaper, a position Helms had held some 15 years earlier.11 In 1975, after 24 years in the Directorate of Operations, Heuer moved to the DI. His earlier academic interest in how we know the truth was rekindled by two experiences. One was his involvement in the con- troversial case of Soviet KGB defector Yuriy Nosenko. Te other was learning new approaches to social science methodology while earning a Master’s degree in international relations at the University of Southern California’s European campus. At the time he retired in 1979, Heuer headed the methodology unit in the DI’s political analysis ofce. He originally prepared most of the chapters in this book as individual articles between 1978 and 1986; many of them were written for the DI after his retirement. He has updated the articles and prepared some new material for inclusion in this book. Heuer’s Central Ideas Dick Heuer’s writings make three fundamental points about the cognitive challenges intelligence analysts face: Te mind is poorly "wired" to deal efectively with both inherent uncertainty (the natural fog surrounding complex, indeterminate intelligence issues) and induced uncertainty (the man-made fog fabricated by denial and deception operations). Even increased awareness of cognitive and other "unmotivated" biases, such as the tendency to see information confrming an al- ready-held judgment more vividly than one sees "disconfrming" information, does little by itself to help analysts deal efectively with uncertainty. Tools and techniques that gear the analyst's mind to apply higher levels of critical thinking can substantially improve analysis on complex issues on which information is incomplete, ambiguous, and often deliberately distorted. Key examples of such intellectu- 11. Letter to the author of this Introduction, 1998. xx  al devices include techniques for structuring information, chal- lenging assumptions, and exploring alternative interpretations. Te following passage from Heuer’s 1980 article entitled “Perception: Why Can’t We See What Is Tere to be Seen?” shows that his ideas were similar to or compatible with MacEachin’s concepts of linchpin analy- sis. Given the difculties inherent in the human processing of com- plex information, a prudent management system should: Encourage products that (a) clearly delineate their as- sumptions and chains of inference and (b) specify the degree and source of the uncertainty involved in the conclusions. Emphasize procedures that expose and elaborate al- ternative points of view—analytic debates, devil’s ad- vocates, interdisciplinary brainstorming, competitive analysis, intra-ofce peer review of production, and elicitation of outside expertise. Heuer emphasizes both the value and the dangers of mental models, or mind-sets. In the book’s opening chapter, entitled “Tinking About Tinking,” he notes that: [Analysts] construct their own version of “reality” on the ba- sis of information provided by the senses, but this sensory in- put is mediated by complex mental processes that determine which information is attended to, how it is organized, and the meaning attributed to it. What people perceive, how readily they perceive it, and how they process this information after receiving it are all strongly infuenced by past experience, edu- cation, cultural values, role requirements, and organizational norms, as well as by the specifcs of the information received. Tis process may be visualized as perceiving the world through a lens or screen that channels and focuses and thereby may dis- tort the images that are seen. To achieve the clearest possible image... analysts need more than information... Tey also xxi  need to understand the lenses through which this information passes. Tese lenses are known by many terms—mental mod- els, mind-sets, biases, or analytic assumptions. In essence, Heuer sees reliance on mental models to simplify and interpret reality as an unavoidable conceptual mechanism for intelligence analysts—often useful, but at times hazardous. What is required of ana- lysts, in his view, is a commitment to challenge, refne, and challenge again their own working mental models, precisely because these steps are cen- tral to sound interpretation of complex and ambiguous issues. Troughout the book, Heuer is critical of the orthodox prescription of “more and better information” to remedy unsatisfactory analytic per- formance. He urges that greater attention be paid instead to more inten- sive exploitation of information already on hand, and that in so doing, analysts continuously challenge and revise their mental models. Heuer sees mirror-imaging as an example of an unavoidable cogni- tive trap. No matter how much expertise an analyst applies to interpret- ing the value systems of foreign entities, when the hard evidence runs out the tendency to project the analyst’s own mind-set takes over. In Chapter 4, Heuer observes: To see the options faced by foreign leaders as these leaders see them, one must understand their values and assumptions and even their misperceptions and misunderstandings. Without such insight, interpreting foreign leaders’ decisions or forecast- ing future decisions is often nothing more than partially in- formed speculation. Too frequently, foreign behavior appears “irrational” or “not in their own best interest.” Such conclu- sions often indicate analysts have projected American values and conceptual frameworks onto the foreign leaders and soci- eties, rather than understanding the logic of the situation as it appears to them. Competing Hypotheses To ofset the risks accompanying analysts’ inevitable recourse to mir- ror-imaging, Heuer suggests looking upon analysts’ calculations about xxii foreign beliefs and behavior as hypotheses to be challenged. Alternative hypotheses need to be carefully considered—especially those that cannot be disproved on the basis of available information. Heuer’s concept of “Analysis of Competing Hypotheses” (ACH) is among his most important contributions to the development of an in- telligence analysis methodology. At the core of ACH is the notion of competition among a series of plausible hypotheses to see which ones survive a gauntlet of testing for compatibility with available information. Te surviving hypotheses—those that have not been disproved—are sub- jected to further testing. ACH, Heuer concedes, will not always yield the right answer. But it can help analysts overcome the cognitive limitations discussed in his book. Some analysts who use ACH follow Heuer’s full eight-step method- ology. More often, they employ some elements of ACH—especially the use of available information to challenge the hypotheses that the analyst favors the most. Denial and Deception Heuer’s path-breaking work on countering denial and deception (D&D) was not included as a separate chapter in this volume. But his brief references here are persuasive. He notes, for example, that analysts often reject the possibility of de- ception because they see no evidence of it. He then argues that rejection is not justifed under these circumstances. If deception is well planned and properly executed, one should not expect to see evidence of it readily at hand. Rejecting a plausible but unproven hypothesis too early tends to bias the subsequent analysis, because one does not then look for the evidence that might support it. Te possibility of deception should not be rejected until it is disproved or, at least, until a systematic search for evidence has been made and none has been found. Heuer’s Impact Heuer’s infuence on analytic tradecraft began with his frst articles. CIA ofcials who set up training courses in the 1980s as part of then- DDI Gates’s quest for improved analysis shaped their lesson plans partly on the basis of Heuer’s fndings. Among these courses were a seminar on intelligence successes and failures and another on intelligence analysis. xxiii Te courses infuenced scores of DI analysts, many of whom are now in the managerial ranks. Te designers and teachers of Tradecraft 2000 clearly were also infuenced by Heuer, as refected in reading selections, case studies, and class exercises. Heuer’s work has remained on reading lists and in lesson plans for DI training courses ofered to all new analysts, as well as courses on warn- ing analysis and on countering denial and deception. Senior analysts and managers who have been directly exposed to Heuer’s thinking through his articles, or through training courses, continue to pass his insights on to newer analysts. Recommendations Heuer’s advice to Agency leaders, managers, and analysts is pointed: To ensure sustained improvement in assessing complex issues, analysis must be treated as more than a substantive and organizational process. Attention also must be paid to techniques and tools for coping with the inherent limitations on analysts’ mental machinery. He urges that Agency leaders take steps to: Establish an organizational environment that promotes and re- wards the kind of critical thinking he advocates—or example, analysis on difcult issues that considers in depth a series of plau- sible hypotheses rather than allowing the frst credible hypothesis to sufce. Expand funding for research on the role such mental processes play in shaping analytical judgments. An Agency that relies on sharp cognitive performance by its analysts must stay abreast of studies on how the mind works—i.e., on how analysts reach judgments. Foster development of tools to assist analysts in assessing informa- tion. On tough issues, they need help in improving their mental models and in deriving incisive fndings from information they already have; they need such help at least as much as they need more information. xxiv  I ofer some concluding observations and recommendations, rooted in Heuer’s fndings and taking into account the tough tradeofs facing intelligence professionals: Commit to a uniform set of tradecraft standards based on the insights in this book. Leaders need to know if analysts have done their cognitive homework before taking corporate responsibility for their judgments. Although every analytical issue can be seen as one of a kind, I suspect that nearly all such topics ft into about a dozen recurring patterns of challenge based largely on varia- tions in substantive uncertainty and policy sensitivity. Corporate standards need to be established for each such category. And the burden should be put on managers to explain why a given ana- lytical assignment requires deviation from the standards. I am convinced that if tradecraft standards are made uniform and transparent, the time saved by curtailing personalistic review of quick-turnaround analysis (e.g., “It reads better to me this way”) could be “re-invested” in doing battle more efectively against cognitive pitfalls. (“Regarding point 3, let’s talk about your as- sumptions.”) Pay more honor to "doubt." Intelligence leaders and policymakers should, in recognition of the cognitive impediments to sound analysis, establish ground rules that enable analysts, after doing their best to clarify an issue, to express doubts more openly. Tey should be encouraged to list gaps in information and other ob- stacles to confdent judgment. Such conclusions as “We do not know” or “Tere are several potentially valid ways to assess this issue” should be regarded as badges of sound analysis, not as der- eliction of analytic duty. Find a couple of successors to Dick Heuer. Fund their research. Heed their fndings. xxv  PART I—OUR MENTAL MACHINERY Chapter 1 Tinking About Tinking Of the diverse problems that impede accurate intelligence analysis, those inherent in human mental processes are surely among the most important and most difcult to deal with. Intelligence analysis is fundamentally a men- tal process, but understanding this process is hindered by the lack of conscious awareness of the workings of our own minds. A basic fnding of cognitive psychology is that people have no conscious experience of most of what happens in the human mind. Many functions as- sociated with perception, memory, and information processing are conducted prior to and independently of any conscious direction. What appears sponta- neously in consciousness is the result of thinking, not the process of thinking. Weaknesses and biases inherent in human thinking processes can be demonstrated through carefully designed experiments. Tey can be alleviated by conscious application of tools and techniques that should be in the analyti- cal tradecraft toolkit of all intelligence analysts. ******************* “When we speak of improving the mind we are usually referring to the acquisition of information or knowledge, or to the type of thoughts one should have, and not to the actual functioning of the mind. We spend little time monitoring our own thinking and comparing it with a more sophisticated ideal.”12 When we speak of improving intelligence analysis, we are usually referring to the quality of writing, types of analytical products, relations between intelligence analysts and intelligence consumers, or organization 12. James L. Adams, Conceptual Blockbusting: A Guide to Better Ideas (New York: W.W. Norton, second edition, 1980), p. 3. 1 of the analytical process. Little attention is devoted to improving how analysts think. Tinking analytically is a skill like carpentry or driving a car. It can be taught, it can be learned, and it can improve with practice. But like many other skills, such as riding a bike, it is not learned by sitting in a classroom and being told how to do it. Analysts learn by doing. Most people achieve at least a minimally acceptable level of analytical perfor- mance with little conscious efort beyond completing their education. With much efort and hard work, however, analysts can achieve a level of excellence beyond what comes naturally. Regular running enhances endurance but does not improve tech- nique without expert guidance. Similarly, expert guidance may be re- quired to modify long-established analytical habits to achieve an optimal level of analytical excellence. An analytical coaching staf to help young analysts hone their analytical tradecraft would be a valuable supplement to classroom instruction. One key to successful learning is motivation. Some of CIA’s best analysts developed their skills as a consequence of experiencing analytical failure early in their careers. Failure motivated them to be more self-con- scious about how they do analysis and to sharpen their thinking pro- cess. Tis book aims to help intelligence analysts achieve a higher level of performance. It shows how people make judgments based on incomplete and ambiguous information, and it ofers simple tools and concepts for improving analytical skills. Part I identifes some limitations inherent in human mental process- es. Part II discusses analytical tradecraft—simple tools and approaches for overcoming these limitations and thinking more systematically. Chapter 8, “Analysis of Competing Hypotheses,” is arguably the most important single chapter. Part III presents information about cognitive biases—the technical term for predictable mental errors caused by simplifed infor- mation processing strategies. A fnal chapter presents a checklist for ana- lysts and recommendations for how managers of intelligence analysis can help create an environment in which analytical excellence fourishes. Herbert Simon frst advanced the concept of “bounded” or limited rationality.13 Because of limits in human mental capacity, he argued, the 13. Herbert Simon, Models of Man, 1957. 2 mind cannot cope directly with the complexity of the world. Rather, we construct a simplifed mental model of reality and then work with this model. We behave rationally within the confnes of our mental model, but this model is not always well adapted to the requirements of the real world. Te concept of bounded rationality has come to be recognized widely, though not universally, both as an accurate portrayal of human judgment and choice and as a sensible adjustment to the limitations in- herent in how the human mind functions.14 Much psychological research on perception, memory, attention span, and reasoning capacity documents the limitations in our “mental machinery” identifed by Simon. Many scholars have applied these psy- chological insights to the study of international political behavior.15 A similar psychological perspective underlies some writings on intelligence failure and strategic surprise.16 Tis book difers from those works in two respects. It analyzes prob- lems from the perspective of intelligence analysts rather than policymak- ers. And it documents the impact of mental processes largely through 14. James G. March., “Bounded Rationality, Ambiguity, and the Engineering of Choice,” in David E. Bell, Howard Raifa, and Amos Tversky, eds., Decision Making: Descriptive, Normative, and Prescriptive Interactions (Cambridge University Press, 1988). 15. Among the early scholars who wrote on this subject were Joseph De Rivera, Te Psychological Dimension of Foreign Policy (Columbus, OH: Merrill, 1968), Alexander George and Richard Smoke, Deterrence in American Foreign Policy (New York: Columbia University Press, 1974), and Robert Jervis, Perception and Misperception in International Politics (Princeton, NJ: Princeton University Press, 1976). 16. Christopher Brady, “Intelligence Failures: Plus Ca Change...” Intelligence and National Security, Vol. 8, No. 4 (October 1993). N. Cigar, “Iraq’s Strategic Mindset and the Gulf War: Blueprint for Defeat,” Te Journal of Strategic Studies, Vol. 15, No. 1 (March 1992). J. J. Wirtz, Te Tet Ofensive: Intelligence Failure in War (New York, 1991). Ephraim Kam, Surprise Attack (Harvard University Press, 1988). Richard Betts, Surprise Attack: Lessons for Defense Planning (Brookings, 1982). Abraham Ben-Zvi, “Te Study of Surprise Attacks,” British Journal of International Studies, Vol. 5 (1979). Iran: Evaluation of Intelligence Performance Prior to November 1978 (Staf Report, Subcommittee on Evaluation, Permanent Select Committee on Intelligence, US House of Representatives, January 1979). Richard Betts, “Analysis, War and Decision: Why Intelligence Failures Are Inevitable,” World Politics, Vol. 31, No. 1 (October 1978). Richard W. Shryock, “Te Intelligence Community Post-Mortem Program, 1973- 1975,” Studies in Intelligence, Vol. 21, No. 1 (Fall 1977). Avi Schlaim, “Failures in National Intelligence Estimates: Te Case of the Yom Kippur War,” World Politics, Vol. 28 (April 1976). Michael Handel, Perception, Deception, and Surprise: Te Case of the Yom Kippur War (Jerusalem: Leonard Davis Institute of International Relations, Jerusalem Paper No. 19, 1976). Klaus Knorr, “Failures in National Intelligence Estimates: Te Case of the Cuban Missiles,” World Politics, Vol. 16 (1964). 3 experiments in cognitive psychology rather than through examples from diplomatic and military history. A central focus of this book is to illuminate the role of the observer in determining what is observed and how it is interpreted. People construct their own version of “reality” on the basis of information provided by the senses, but this sensory input is mediated by complex mental processes that determine which information is attended to, how it is organized, and the meaning attributed to it. What people perceive, how readily they perceive it, and how they process this information after receiving it are all strongly infuenced by past experience, education, cultural values, role requirements, and organizational norms, as well as by the specifcs of the information received. Tis process may be visualized as perceiving the world through a lens or screen that channels and focuses and thereby may distort the im- ages that are seen. To achieve the clearest possible image of China, for example, analysts need more than information on China. Tey also need to understand their own lenses through which this information passes. Tese lenses are known by many terms—mental models, mind-sets, bi- ases, or analytical assumptions. In this book, the terms mental model and mind-set are used more or less interchangeably, although a mental model is likely to be better developed and articulated than a mind-set. An analytical assumption is one part of a mental model or mind-set. Te biases discussed in this book result from how the mind works and are independent of any substantive mental model or mind-set. Before obtaining a license to practice, psychoanalysts are required to undergo psychoanalysis themselves in order to become more aware of how their own personality interacts with and conditions their observa- tions of others. Te practice of psychoanalysis has not been so success- ful that its procedures should be emulated by the intelligence and for- eign policy community. But the analogy highlights an interesting point: Intelligence analysts must understand themselves before they can under- stand others. Training is needed to (a) increase self-awareness concerning generic problems in how people perceive and make analytical judgments concerning foreign events, and (b) provide guidance and practice in over- coming these problems. Not enough training is focused in this direction—that is, inward toward the analyst’s own thought processes. Training of intelligence ana- 4 lysts generally means instruction in organizational procedures, method- ological techniques, or substantive topics. More training time should be devoted to the mental act of thinking or analyzing. It is simply assumed, incorrectly, that analysts know how to analyze. Tis book is intended to support training that examines the thinking and reasoning processes involved in intelligence analysis. As discussed in the next chapter, mind-sets and mental models are inescapable. Tey are, in essence, a distillation of all that we think we know about a subject. Te problem is how to ensure that the mind re- mains open to alternative interpretations in a rapidly changing world. Te disadvantage of a mind-set is that it can color and control our perception to the extent that an experienced specialist may be among the last to see what is really happening when events take a new and un- expected turn. When faced with a major paradigm shift, analysts who know the most about a subject have the most to unlearn. Tis seems to have happened before the reunifcation of Germany, for example. Some German specialists had to be prodded by their more generalist supervi- sors to accept the signifcance of the dramatic changes in progress toward reunifcation of East and West Germany. Te advantage of mind-sets is that they help analysts get the produc- tion out on time and keep things going efectively between those water- shed events that become chapter headings in the history books.17 A generation ago, few intelligence analysts were self-conscious and introspective about the process by which they did analysis. Te accepted wisdom was the “common sense” theory of knowledge—that to perceive events accurately it was necessary only to open one’s eyes, look at the facts, and purge oneself of all preconceptions and prejudices in order to make an objective judgment. Today, there is greatly increased understanding that intelligence analysts do not approach their tasks with empty minds. Tey start with a set of assumptions about how events normally transpire in the area for which they are responsible. Although this changed view is becoming conventional wisdom, the Intelligence Community has only begun to scratch the surface of its implications. If analysts’ understanding of events is greatly infuenced by the mind-set or mental model through which they perceive those events, 17. Tis wording is from a discussion with veteran CIA analyst, author, and teacher Jack Davis. 5 should there not be more research to explore and document the impact of diferent mental models?18 Te reaction of the Intelligence Community to many problems is to collect more information, even though analysts in many cases already have more information than they can digest. What analysts need is more truly useful information—mostly reliable HUMINT from knowledge- able insiders—to help them make good decisions. Or they need a more accurate mental model and better analytical tools to help them sort through, make sense of, and get the most out of the available ambiguous and conficting information. Psychological research also ofers to intelligence analysts additional insights that are beyond the scope of this book. Problems are not limited to how analysts perceive and process information. Intelligence analysts often work in small groups and always within the context of a large, bu- reaucratic organization. Problems are inherent in the processes that occur at all three levels—individual, small group, and organization. Tis book focuses on problems inherent in analysts’ mental processes, inasmuch as these are probably the most insidious. Analysts can observe and get a feel for these problems in small-group and organizational processes, but it is very difcult, at best, to be self-conscious about the workings of one’s own mind. 18. Graham Allison’s work on the Cuban missile crisis (Essence of Decision, Little, Brown & Co., 1971) is an example of what I have in mind. Allison identifed three alternative assump- tions about how governments work--a rational actor model, an organizational process model, and a bureaucratic politics model. He then showed how an analyst’s implicit assumptions about the most appropriate model for analyzing a foreign government’s behavior can cause him or her to focus on diferent evidence and arrive at diferent conclusions. Another example is my own analysis of fve alternative paths for making counterintelligence judgments in the contro- versial case of KGB defector Yuriy Nosenko: Richards J. Heuer, Jr., “Nosenko: Five Paths to Judgment,” Studies in Intelligence, Vol. 31, No. 3 (Fall 1987), originally classifed Secret but de- classifed and published in H. Bradford Westerfeld, ed., Inside CIA’s Private World: Declassifed Articles from the Agency’s Internal Journal 1955-1992 (New Haven: Yale University Press, 1995). 6  Chapter  Perception: Why Can’t We See What Is Tere To Be Seen? Te process of perception links people to their environment and is criti- cal to accurate understanding of the world about us. Accurate intelligence analysis obviously requires accurate perception. Yet research into human per- ception demonstrates that the process is beset by many pitfalls. Moreover, the circumstances under which intelligence analysis is conducted are precisely the circumstances in which accurate perception tends to be most difcult. Tis chapter discusses perception in general, then applies this information to il- luminate some of the difculties of intelligence analysis.19 ******************* People tend to think of perception as a passive process. We see, hear, smell, taste or feel stimuli that impinge upon our senses. We think that if we are at all objective, we record what is actually there. Yet percep- tion is demonstrably an active rather than a passive process; it constructs rather than records “reality.” Perception implies understanding as well as awareness. It is a process of inference in which people construct their own version of reality on the basis of information provided through the fve senses. As already noted, what people in general and analysts in particular perceive, and how readily they perceive it, are strongly infuenced by their past experience, education, cultural values, and role requirements, as well as by the stimuli recorded by their receptor organs. Many experiments have been conducted to show the extraordinary extent to which the information obtained by an observer depends upon the observer’s own assumptions and preconceptions. For example, when 19. An earlier version of this article was published as part of “Cognitive Factors in Deception and Counterdeception,” in Donald C. Daniel and Katherine L. Herbig, eds., Strategic Military Deception (Pergamon Press, 1982). 7 you looked at Figure 1 above, what did you see? Now refer to the foot- note for a description of what is actually there.0 Did you perceive Figure 1 correctly? If so, you have exceptional powers of observation, were lucky, or have seen the fgure before. Tis simple experiment demonstrates one of the most fundamental principles concerning perception: We tend to perceive what we expect to perceive. A corollary of this principle is that it takes more information, and more unambiguous information, to recognize an unexpected phenom- enon than an expected one. One classic experiment to demonstrate the infuence of expecta- tions on perception used playing cards, some of which were gimmicked so the spades were red and the hearts black. Pictures of the cards were fashed briefy on a screen and, needless to say, the test subjects identifed the normal cards more quickly and accurately than the anomalous ones. After test subjects became aware of the existence of red spades and black hearts, their performance with the gimmicked cards improved but still did not approach the speed or accuracy with which normal cards could be identifed.1 20. Te article is written twice in each of the three phrases. Tis is commonly overlooked because perception is infuenced by our expectations about how these familiar phrases are normally written. 21. Jerome S. Bruner and Leo Postman, “On the Perception of Incongruity: A Paradigm,” in Jerome S. Bruner and David Kraut, eds., Perception and Personality: A Symposium (New York: Greenwood Press, 1968). 8  Tis experiment shows that patterns of expectation become so deeply embedded that they continue to infuence perceptions even when people are alerted to and try to take account of the existence of data that do not ft their preconceptions. Trying to be objective does not ensure accurate perception. Te position of the test subject identifying playing cards is analo- gous to that of the intelligence analyst or government leader trying to make sense of the paper fow that crosses his or her desk. What is actually perceived in that paper fow, as well as how it is interpreted, depends in part, at least, on the analyst’s patterns of expectation. Analysts do not just have expectations about the color of hearts and spades. Tey have a set of assumptions and expectations about the motivations of people and the processes of government in foreign countries. Events consistent with these expectations are perceived and processed easily, while events that contradict prevailing expectations tend to be ignored or distorted in perception. Of course, this distortion is a subconscious or pre-conscious process, as illustrated by how you presumably ignored the extra words in the triangles in Figure 1. Tis tendency of people to perceive what they expect to perceive is more important than any tendency to perceive what they want to per- ceive. In fact, there may be no real tendency toward wishful thinking. Te commonly cited evidence supporting the claim that people tend to perceive what they want to perceive can generally be explained equally well by the expectancy thesis. Expectations have many diverse sources, including past experience, professional training, and cultural and organizational norms. All these infuences predispose analysts to pay particular attention to certain kinds of information and to organize and interpret this information in certain ways. Perception is also infuenced by the context in which it occurs. Diferent circumstances evoke diferent sets of expectations. People are more attuned to hearing footsteps behind them when walking in an alley at night than along a city street in daytime, and the meaning attributed to the sound of footsteps will vary under these difering circumstances. A military intelligence analyst may be similarly tuned to perceive indicators of potential confict. 22. For discussion of the ambiguous evidence concerning the impact of desires and fears on judgment, see Robert Jervis, Perception and Misperception in International Politics (Princeton, NJ: Princeton University Press, 1976), Chapter 10. 9  Patterns of expectations tell analysts, subconsciously, what to look for, what is important, and how to interpret what is seen. Tese pat- terns form a mind-set that predisposes analysts to think in certain ways. A mind-set is akin to a screen or lens through which one perceives the world. Tere is a tendency to think of a mind-set as something bad, to be avoided. According to this line of argument, one should have an open mind and be infuenced only by the facts rather than by preconceived no- tions! Tat is an unreachable ideal. Tere is no such thing as “the facts of the case.” Tere is only a very selective subset of the overall mass of data to which one has been subjected that one takes as facts and judges to be relevant to the question at issue. Actually, mind-sets are neither good nor bad; they are unavoidable. People have no conceivable way of coping with the volume of stimuli that impinge upon their senses, or with the volume and complexity of the data they have to analyze, without some kind of simplifying precon- ceptions about what to expect, what is important, and what is related to what. “Tere is a grain of truth in the otherwise pernicious maxim that an open mind is an empty mind.”23 Analysts do not achieve objective analysis by avoiding preconceptions; that would be ignorance or self-de- lusion. Objectivity is achieved by making basic assumptions and reason- ing as explicit as possible so that they can be challenged by others and analysts can, themselves, examine their validity. One of the most important characteristics of mind-sets is: Mind-sets tend to be quick to form but resistant to change. Figure 2 illustrates this principle by showing part of a longer series of progressively modifed drawings that change almost imperceptibly from a man into a woman.4 Te right-hand drawing in the top row, when viewed alone, has equal chances of being perceived as a man or a woman. When test subjects are shown the entire series of drawings one by one, their perception of this intermediate drawing is biased according to which end of the series they started from. Test subjects who start by viewing a picture that is clearly a man are biased in favor of continuing 23. Richard Betts, “Analysis, War and Decision: Why Intelligence Failures are Inevitable”, World Politics, Vol. XXXI (October 1978), p. 84. 24. Drawings devised by Gerald Fisher in 1967. 10 to see a man long after an “objective observer” (for example, an observer who has seen only a single picture) recognizes that the man is now a woman. Similarly, test subjects who start at the woman end of the series are biased in favor of continuing to see a woman. Once an observer has formed an image—that is, once he or she has developed a mind-set or expectation concerning the phenomenon being observed—this condi- tions future perceptions of that phenomenon. Tis is the basis for another general principle of perception: New information is assimilated to existing images. Tis principle explains why gradual, evolutionary change often goes unnoticed. It also explains the phenomenon that an intelligence analyst assigned to work on a topic or country for the frst time may generate accurate insights that have been overlooked by experienced analysts who have worked on the same problem for 10 years. A fresh perspective is sometimes useful; past experience can handicap as well as aid analysis. Tis tendency to assimilate new data into pre-existing images is greater “the more ambiguous the information, the more confdent the actor is of 11 the validity of his image, and the greater his commitment to the estab- lished view.”5 Te drawing in Figure 3 provides the reader an opportunity to test for him or herself the persistence of established images.26 Look at Figure 3. What do you see—an old woman or a young woman? Now look again to see if you can visually and mentally reorganize the data to form a dif- ferent image—that of a young woman if your original perception was of an old woman, or of the old woman if you frst perceived the young one. If necessary, look at the footnote for clues to help you identify the other 25. Jervis, p. 195. 26. Tis picture was originally published in Puck magazine in 1915 as a cartoon entitled “My Wife and My Mother-in-Law.” 12 image.7 Again, this exercise illustrates the principle that mind-sets are quick to form but resistant to change. When you have seen Figure 3 from both perspectives, try shifting back and forth from one perspective to the other. Do you notice some initial difculty in making this switch? One of the more difcult men- tal feats is to take a familiar body of data and reorganize it visually or mentally to perceive it from a diferent perspective. Yet this is what in- telligence analysts are constantly required to do. In order to understand international interactions, analysts must understand the situation as it appears to each of the opposing forces, and constantly shift back and forth from one perspective to the other as they try to fathom how each side interprets an ongoing series of interactions. Trying to perceive an adversary’s interpretations of international events, as well as US interpre- tations of those same events, is comparable to seeing both the old and young woman in Figure 3. Once events have been perceived one way, there is a natural resistance to other perspectives. A related point concerns the impact of substandard conditions of perception. Te basic principle is: Initial exposure to blurred or ambiguous stimuli interferes with accurate perception even after more and better information be- comes available. Tis efect has been demonstrated experimentally by projecting onto a screen pictures of common, everyday subjects such as a dog standing on grass, a fre hydrant, and an aerial view of a highway cloverleaf inter- section.28 Te initial projection was blurred in varying degrees, and the pictures were then brought into focus slowly to determine at what point test subjects could identify them correctly. Tis experiment showed two things. First, those who started view- ing the pictures when they were most out of focus had more difculty identifying them when they became clearer than those who started view- 27. Te old woman’s nose, mouth, and eye are, respectively, the young woman’s chin, necklace, and ear. Te old woman is seen in profle looking left. Te young woman is also looking left, but we see her mainly from behind so most facial features are not visible. Her eyelash, nose, and the curve of her cheek may be seen just above the old woman’s nose. 28. Jerome S. Bruner and Mary C. Potter, “Interference in Visual Recognition,” Science, Vol. 144 (1964), pp. 424-25. 13 ing at a less blurred stage. In other words, the greater the initial blur, the clearer the picture had to be before people could recognize it. Second, the longer people were exposed to a blurred picture, the clearer the picture had to be before they could recognize it. What happened in this experiment is what presumably happens in real life; despite ambiguous stimuli, people form some sort of tenta- tive hypothesis about what they see. Te longer they are exposed to this blurred image, the greater confdence they develop in this initial and per- haps erroneous impression, so the greater the impact this initial impres- sion has on subsequent perceptions. For a time, as the picture becomes clearer, there is no obvious contradiction; the new data are assimilated into the previous image, and the initial interpretation is maintained until the contradiction becomes so obvious that it forces itself upon our con- sciousness. Te early but incorrect impression tends to persist because the amount of information necessary to invalidate a hypothesis is consider- ably greater than the amount of information required to make an initial interpretation. Te problem is not that there is any inherent difculty in grasping new perceptions or new ideas, but that established perceptions are so difcult to change. People form impressions on the basis of very little information, but once formed, they do not reject or change them unless they obtain rather solid evidence. Analysts might seek to limit the adverse impact of this tendency by suspending judgment for as long as possible as new information is being received. Implications for Intelligence Analysis Comprehending the nature of perception has signifcant implica- tions for understanding the nature and limitations of intelligence analy- sis. Te circumstances under which accurate perception is most difcult are exactly the circumstances under which intelligence analysis is gener- ally conducted—dealing with highly ambiguous situations on the basis of information that is processed incrementally under pressure for early judgment. Tis is a recipe for inaccurate perception. Intelligence seeks to illuminate the unknown. Almost by defnition, intelligence analysis deals with highly ambiguous situations. As previ- ously noted, the greater the ambiguity of the stimuli, the greater the impact of expectations and pre-existing images on the perception of that 14 stimuli. Tus, despite maximum striving for objectivity, the intelligence analyst’s own preconceptions are likely to exert a greater impact on the analytical product than in other felds where an analyst is working with less ambiguous and less discordant information. Moreover, the intelligence analyst is among the frst to look at new problems at an early stage when the evidence is very fuzzy indeed. Te analyst then follows a problem as additional increments of evidence are received and the picture gradually clarifes—as happened with test sub- jects in the experiment demonstrating that initial exposure to blurred stimuli interferes with accurate perception even after more and better information becomes available. If the results of this experiment can be generalized to apply to intelligence analysts, the experiment suggests that an analyst who starts observing a potential problem situation at an early and unclear stage is at a disadvantage as compared with others, such as policymakers, whose frst exposure may come at a later stage when more and better information is available. Te receipt of information in small increments over time also fa- cilitates assimilation of this information into the analyst’s existing views. No one item of information may be sufcient to prompt the analyst to change a previous view. Te cumulative message inherent in many pieces of information may be signifcant but is attenuated when this informa- tion is not examined as a whole. Te Intelligence Community’s review of its performance before the 1973 Arab-Israeli War noted: Te problem of incremental analysis—especially as it applies to the current intelligence process—was also at work in the period preceding hostilities. Analysts, according to their own accounts, were often proceeding on the basis of the day’s take, hastily comparing it with material received the previous day. Tey then produced in ‘assembly line fashion’ items which may have refected perceptive intuition but which [did not] accrue from a systematic consideration of an accumulated body of in- tegrated evidence.9 And fnally, the intelligence analyst operates in an environment that exerts strong pressures for what psychologists call premature closure. 29. Te Performance of the Intelligence Community Before the Arab-Israeli War of October 1973: A Preliminary Post-Mortem Report, December 1973. Te one-paragraph excerpt from this post- mortem, as quoted in the text above, has been approved for public release, as was the title of the post-mortem, although that document as a whole remains classifed. 15 Customer demand for interpretive analysis is greatest within two or three days after an event occurs. Te system requires the intelligence analyst to come up with an almost instant diagnosis before sufcient hard infor- mation, and the broader background information that may be needed to gain perspective, become available to make possible a well-grounded judgment. Tis diagnosis can only be based upon the analyst’s precon- ceptions concerning how and why events normally transpire in a given society. As time passes and more information is received, a fresh look at all the evidence might suggest a diferent explanation. Yet, the perception experiments indicate that an early judgment adversely afects the forma- tion of future perceptions. Once an observer thinks he or she knows what is happening, this perception tends to resist change. New data received incrementally can be ft easily into an analyst’s previous image. Tis per- ceptual bias is reinforced by organizational pressures favoring consistent interpretation; once the analyst is committed in writing, both the analyst and the organization have a vested interest in maintaining the original assessment. Tat intelligence analysts perform as well as they do is testimony to their generally sound judgment, training, and dedication in performing a dauntingly difcult task. Te problems outlined here have implications for the management as well as the conduct of analysis. Given the difculties inherent in the human processing of complex information, a prudent management sys- tem should: Encourage products that clearly delineate their assumptions and chains of inference and that specify the degree and source of un- certainty involved in the conclusions. Support analyses that periodically re-examine key problems from the ground up in order to avoid the pitfalls of the incremental approach. Emphasize procedures that expose and elaborate alternative points of view. Educate consumers about the limitations as well as the capabili- ties of intelligence analysis; defne a set of realistic expectations as a standard against which to judge analytical performance. 16  Chapter 3 Memory: How Do We Remember What We Know? Diferences between stronger and weaker analytical performance are at- tributable in large measure to diferences in the organization of data and experience in analysts’ long-term memory. Te contents of memory form a continuous input into the analytical process, and anything that infuences what information is remembered or retrieved from memory also infuences the outcome of analysis. Tis chapter discusses the capabilities and limitations of several com- ponents of the memory system. Sensory information storage and short-term memory are beset by severe limitations of capacity, while long-term memory, for all practical purposes, has a virtually infnite capacity. With long-term memory, the problems concern getting information into it and retrieving in- formation once it is there, not physical limits on the amount of information that may be stored. Understanding how memory works provides insight into several analytical strengths and weaknesses. ******************* Components of the Memory System What is commonly called memory is not a single, simple function. It is an extraordinarily complex system of diverse components and pro- cesses. Tere are at least three, and very likely more, distinct memory processes. Te most important from the standpoint of this discussion and best documented by scientifc research are sensory information stor- 17 age (SIS), short-term memory (STM), and long-term memory (LTM).30 Each difers with respect to function, the form of information held, the length of time information is retained, and the amount of information- handling capacity. Memory researchers also posit the existence of an in- terpretive mechanism and an overall memory monitor or control mech- anism that guides interaction among various elements of the memory system. Sensory Information Storage Sensory information storage holds sensory images for several tenths of a second after they are received by the sensory organs. Te functioning of SIS may be observed if you close your eyes, then open and close them again as rapidly as possible. As your eyes close, notice how the visual image is maintained for a fraction of a second before fading. Sensory information storage explains why a movie flm shot at 16 separate frames per second appears as continuous movement rather than a series of still pictures. A visual trace is generally retained in SIS for about one-quarter of a second. It is not possible to consciously extend the time that sensory information is held in SIS. Te function of SIS is to make it possible for the brain to work on processing a sensory event for longer than the dura- tion of the event itself. Short-Term Memory Information passes from SIS into short-term memory, where again it is held for only a short period of time—a few seconds or minutes. Whereas SIS holds the complete image, STM stores only the interpreta- tion of the image. If a sentence is spoken, SIS retains the sounds, while STM holds the words formed by these sounds. Like SIS, short-term memory holds information temporarily, pend- ing further processing. Tis processing includes judgments concerning meaning, relevance, and signifcance, as well as the mental actions nec- essary to integrate selected portions of the information into long-term 30. Memory researchers do not employ uniform terminology. Sensory information storage is also known as sensory register, sensory store, and eidetic and echoic memory. Short- and long- term memory are also referred to as primary and secondary memory. A variety of other terms are in use as well. I have adopted the terminology used by Peter H. Lindsay and Donald A. Norman in their text on Human Information Processing (New York: Academic Press, 1977). Tis entire chapter draws heavily from Chapters 8 through 11 of the Lindsay and Norman book. 18 memory. When a person forgets immediately the name of someone to whom he or she has just been introduced, it is because the name was not transferred from short-term to long-term memory. A central characteristic of STM is the severe limitation on its ca- pacity. A person who is asked to listen to and repeat a series of 10 or 20 names or numbers normally retains only fve or six items. Commonly it is the last fve or six. If one focuses instead on the frst items, STM be- comes saturated by this efort, and the person cannot concentrate on and recall the last items. People make a choice where to focus their attention. Tey can concentrate on remembering or interpreting or taking notes on information received moments ago, or pay attention to information cur- rently being received. Limitations on the capacity of short-term memory often preclude doing both. Retrieval of information from STM is direct and immediate because the information has never left the conscious mind. Information can be maintained in STM indefnitely by a process of “rehearsal”—repeating it over and over again. But while rehearsing some items to retain them in STM, people cannot simultaneously add new items. Te severe limita- tion on the amount of information retainable in STM at any one time is physiological, and there is no way to overcome it. Tis is an important point that will be discussed below in connection with working memory and the utility of external memory aids. Long-Term Memory Some information retained in STM is processed into long-term memory. Tis information on past experiences is fled away in the re- cesses of the mind and must be retrieved before it can be used. In contrast to the immediate recall of current experience from STM, retrieval of information from LTM is indirect and sometimes laborious. Loss of detail as sensory stimuli are interpreted and passed from SIS into STM and then into LTM is the basis for the phenomenon of selective perception discussed in the previous chapter. It imposes limits on subsequent stages of analysis, inasmuch as the lost data can never be retrieved. People can never take their mind back to what was actually there in sensory information storage or short-term memory. Tey can only retrieve their interpretation of what they thought was there as stored in LTM. 19  Tere are no practical limits to the amount of information that may be stored in LTM. Te limitations of LTM are the difculty of processing information into it and retrieving information from it. Tese subjects are discussed below. Te three memory processes comprise the storehouse of informa- tion or database that we call memory, but the total memory system must include other features as well. Some mental process must determine what information is passed from SIS into STM and from STM into LTM; decide how to search the LTM data base and judge whether further memory search is likely to be productive; assess the relevance of retrieved information; and evaluate potentially contradictory data. To explain the operation of the total memory system, psycholo- gists posit the existence of an interpretive mechanism that operates on the data base and a monitor or central control mechanism that guides and oversees the operation of the whole system. Little is known of these mechanisms and how they relate to other mental processes. Despite much research on memory, little agreement exists on many critical points. What is presented here is probably the lowest common denominator on which most researchers would agree. Organization of Information in Long-Term Memory. Physically, the brain consists of roughly 10 billion neurons, each analogous to a computer chip capable of storing information. Each neuron has octopus- like arms called axons and dendrites. Electrical impulses fow through these arms and are ferried by neurotransmitting chemicals across what is called the synaptic gap between neurons. Memories are stored as patterns of connections between neurons. When two neurons are activated, the connections or “synapses” between them are strengthened. As you read this chapter, the experience actually causes physical changes in your brain. “In a matter of seconds, new circuits are formed that can change forever the way you think about the world.”31 Memory records a lifetime of experience and thoughts. Such a mas- sive data retrieval mechanism, like a library or computer system, must have an organizational structure; otherwise information that enters the system could never be retrieved. Imagine the Library of Congress if there were no indexing system. 31. George Johnson, In the Palaces of Memory: How We Build the Worlds Inside Our Heads. Vintage Books, 1992, p. xi. 20  Tere has been considerable research on how information is orga- nized and represented in memory, but the fndings remain speculative. Current research focuses on which sections of the brain process various types of information. Tis is determined by testing patients who have sufered brain damage from strokes and trauma or by using functional magnetic resonance imaging (fMRI) that “lights up” the active portion of the brain as a person speaks, reads, writes, or listens. None of the current theories seems to encompass the full range or complexity of memory processes, which include memory for sights and sounds, for feelings, and for belief systems that integrate information on a large number of concepts. However useful the research has been for other purposes, analysts’ needs are best served by a very simple image of the structure of memory. Imagine memory as a massive, multidimensional spider web. Tis image captures what is, for the purposes of this book, perhaps the most important property of information stored in memory—its interconnect- edness. One thought leads to another. It is possible to start at any one point in memory and follow a perhaps labyrinthine path to reach any other point. Information is retrieved by tracing through the network of interconnections to the place where it is stored. Retrievability is infuenced by the number of locations in which information is stored and the number and strength of pathways from this information to other concepts that might be activated by incom- ing information. Te more frequently a path is followed, the stronger that path becomes and the more readily available the information located along that path. If one has not thought of a subject for some time, it may be difcult to recall details. After thinking our way back into the appropriate context and fnding the general location in our memory, the interconnections become more readily available. We begin to remember names, places, and events that had seemed to be forgotten. Once people have started thinking about a problem one way, the same mental circuits or pathways get activated and strengthened each time they think about it. Tis facilitates the retrieval of information. Tese same pathways, however, also become the mental ruts that make it difcult to reorganize the information mentally so as to see it from a diferent perspective. Tat explains why, in the previous chapter, once you saw the picture of the old woman it was difcult to see the young 21 woman, or vice versa. A subsequent chapter will consider ways of break- ing out of mental ruts. One useful concept of memory organization is what some cogni- tive psychologists call a “schema.” A schema is any pattern of relationships among data stored in memory. It is any set of nodes and links between them in the spider web of memory that hang together so strongly that they can be retrieved and used more or less as a single unit. For example, a person may have a schema for a bar that when acti- vated immediately makes available in memory knowledge of the proper- ties of a bar and what distinguishes a bar, say, from a tavern. It brings back memories of specifc bars that may in turn stimulate memories of thirst, guilt, or other feelings or circumstances. People also have schemata (plural for schema) for abstract concepts such as a socialist economic system and what distinguishes it from a capitalist or communist system. Schemata for phenomena such as success or failure in making an accurate intelligence estimate will include links to those elements of memory that explain typical causes and implications of success or failure. Tere must also be schemata for processes that link memories of the various steps in- volved in long division, regression analysis, or simply making inferences from evidence and writing an intelligence report. Any given point in memory may be connected to many diferent overlapping schemata. Tis system is highly complex and not well un- derstood. Tis conception of a schema is so general that it begs many impor- tant questions of interest to memory researchers, but it is the best that can be done given the current state of knowledge. It serves the purpose of emphasizing that memory does have structure. It also shows that how knowledge is connected in memory is critically important in determin- ing what information is retrieved in response to any stimulus and how that information is used in reasoning. Concepts and schemata stored in memory exercise a powerful in- fuence on the formation of perceptions from sensory data. Recall the experiment discussed in the previous chapter in which test subjects were exposed very briefy to playing cards that had been doctored so that some hearts were black and spades red. When retained in SIS for a fraction of a second, the spades were indeed red. In the course of interpreting the sen- sory impression and transferring it to STM, however, the spades became black because the memory system has no readily available schema for a 22 red spade to be matched against the sensory impression. If information does not ft into what people know, or think they know, they have great difculty processing it. Te content of schemata in memory is a principal factor distin- guishing stronger from weaker analytical ability. Tis is aptly illustrated by an experiment with chess players. When chess grandmasters and mas- ters and ordinary chess players were given fve to 10 seconds to note the position of 20 to 25 chess pieces placed randomly on a chess board, the masters and ordinary players were alike in being able to remember the places of only about six pieces. If the positions of the pieces were taken from an actual game (unknown to the test subjects), however, the grandmasters and masters were usually able to reproduce almost all the positions without error, while the ordinary players were still able to place correctly only a half-dozen pieces.3 Tat the unique ability of the chess masters did not result from a pure feat of memory is indicated by the masters’ inability to perform better than ordinary players in remembering randomly placed positions. Teir exceptional performance in remembering positions from actual games stems from their ability to immediately perceive patterns that enable them to process many bits of information together as a single chunk or schema. Te chess master has available in long-term memory many schemata that connect individual positions together in coherent patterns. When the position of chess pieces on the board corresponds to a recognized schema, it is very easy for the master to remember not only the positions of the pieces, but the outcomes of previous games in which the pieces were in these positions. Similarly, the unique abilities of the master analyst are attributable to the schemata in long-term memory that enable the analyst to perceive patterns in data that pass undetected by the average observer. Getting Information Into and Out of Long-Term Memory. It used to be that how well a person learned something was thought to depend upon how long it was kept in short-term memory or the number of times they repeated it to themselves. Research evidence now suggests that nei- ther of these factors plays the critical role. Continuous repetition does not necessarily guarantee that something will be remembered. Te key 32. A. D. deGroot, Tought and Choice in Chess (Te Hague: Mouton, 1965) cited by Herbert A. Simon, “How Big Is a Chunk?” Science, Vol. 183 (1974), p. 487. 23 factor in transferring information from short-term to long-term memory is the development of associations between the new information and schemata already available in memory. Tis, in turn, depends upon two variables: the extent to which the information to be learned relates to an already existing schema, and the level of processing given to the new information. Take one minute to try to memorize the following items from a shopping list: bread, eggs, butter, salami, corn, lettuce, soap, jelly, chick- en, and cofee. Chances are, you will try to burn the words into your mind by repeating them over and over. Such repetition, or maintenance rehearsal, is efective for maintaining the information in STM, but is an inefcient and often inefective means of transferring it to LTM. Te list is difcult to memorize because it does not correspond with any schema already in memory. Te words are familiar, but you do not have available in memory a schema that connects the words in this particular group to each other. If the list were changed to juice, cereal, milk, sugar, bacon, eggs, toast, butter, jelly, and cofee, the task would be much easier because the data would then correspond with an existing schema—items commonly eat- en for breakfast. Such a list can be assimilated to your existing store of knowledge with little difculty, just as the chess master rapidly assimi- lates the positions of many chessmen. Depth of processing is the second important variable in determin- ing how well information is retained. Depth of processing refers to the amount of efort and cognitive capacity employed to process informa- tion, and the number and strength of associations that are thereby forged between the data to be learned and knowledge already in memory. In ex- periments to test how well people remember a list of words, test subjects might be asked to perform diferent tasks that refect diferent levels of processing. Te following illustrative tasks are listed in order of the depth of mental processing required: say how many letters there are in each word on the list, give a word that rhymes with each word, make a mental image of each word, make up a story that incorporates each word. It turns out that the greater the depth of processing, the greater the ability to recall words on a list. Tis result holds true regardless of whether the test subjects are informed in advance that the purpose of the experiment is to test them on their memory. Advising test subjects to expect a test makes almost no diference in their performance, presum- 24 ably because it only leads them to rehearse the information in short-term memory, which is inefective as compared with other forms of process- ing. Tere are three ways in which information may be learned or com- mitted to memory: by rote, assimilation, or use of a mnemonic device. Each of these procedures is discussed below.33 By Rote. Material to be learned is repeated verbally with sufcient frequency that it can later be repeated from memory without use of any memory aids. When information is learned by rote, it forms a separate schema not closely interwoven with previously held knowledge. Tat is, the mental processing adds little by way of elaboration to the new infor- mation, and the new information adds little to the elaboration of existing schemata. Learning by rote is a brute force technique. It seems to be the least efcient way of remembering. By Assimilation. Information is learned by assimilation when the structure or substance of the information fts into some memory schema already possessed by the learner. Te new information is assimilated to or linked to the existing schema and can be retrieved readily by frst accessing the existing schema and then reconstructing the new informa- tion. Assimilation involves learning by comprehension and is, therefore, a desirable method, but it can only be used to learn information that is somehow related to our previous experience. By Using A Mnemonic Device. A mnemonic device is any means of organizing or encoding information for the purpose of making it easier to remember. A high school student cramming for a geography test might use the acronym “HOMES” as a device for remembering the frst letter of each of the Great Lakes—Huron, Ontario, etc. To learn the frst grocery list of disconnected words, you would cre- ate some structure for linking the words to each other and/or to informa- tion already in LTM. You might imagine yourself shopping or putting the items away and mentally picture where they are located on the shelves at the market or in the kitchen. Or you might imagine a story concerning one or more meals that include all these items. Any form of processing information in this manner is a more efective aid to retention than rote repetition. Even more efective systems for quickly memorizing lists of 33. Tis discussion draws on Francis S. Bellezza, “Mnemonic Devices: Classifcation, Characteristics, and Criteria” (Athens, Ohio: Ohio University, pre-publication manuscript, January 1980). 25 names or words have been devised by various memory experts, but these require some study and practice in their use. Mnemonic devices are useful for remembering info