Psychological Testing and Assessment (9th Ed) PDF

Psychological Testing and Assessment An Introduction to Tests and Measurement Ninth Edition Ronald Jay Cohen Mark E. Swerdlik Psychological Testing and Assessment An Introduction to Tests and Measurement NINTH EDITION Ronald Jay Cohen RJ COHEN CONSULTING Mark E. Swerdlik ILLINOIS STATE UNIVERSITY PSYCHOLOGICAL TESTING AND ASSESSMENT: AN INTRODUCTION TO TESTS AND MEASUREMENT, NINTH EDITION Published by McGraw-Hill Education, 2 Penn Plaza, New York, NY 10121. Copyright © 2018 by McGraw-Hill Education. All rights reserved. Printed in the United States of America. Previous editions © 2013, 2010, and 2005. No part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written consent of McGraw-Hill Education, including, but not limited to, in any network or other electronic storage or transmission, or broadcast for distance learning. Some ancillaries, including electronic and print components, may not be available to customers outside the United States. This book is printed on acid-free paper. 1 2 3 4 5 6 7 8 9 LWI 21 20 19 18 17 16 ISBN 978-1-259-87050-7 MHID 1-259-87050-2 Chief Product Officer, SVP Products & Markets: G. Scott Virkler Vice President, General Manager, Products & Markets: Michael Ryan Vice President, Content Design & Delivery: Betsy Whalen Managing Director: David Patterson Brand Manager: Jamie Laferrera Product Developer: Alex Preiss Marketing Manager: Meredith Leo Editorial Coordinator: Jasmine Staton Director, Content Design & Delivery: Terri Schiesl Program Manager: Jennifer L. Shekleton Content Project Managers: Jennifer L. Shekleton; George Theofanopoulos Buyer: Susan K. Culbertson Design: Studio Montage Content Licensing Specialists: Melisa Seegmiller Cover Image: Leonardo Carneiro de Almeida/Getty Images Compositor: Aptara, Inc. Printer: LSC Communications All credits appearing on page or at the end of the book are considered to be an extension of the copyright page. Library of Congress Cataloging-in-Publication Data Names: Cohen, Ronald Jay, author. | Swerdlik, Mark E., author. Title: Psychological testing and assessment / Ronald Jay Cohen, RJ Cohen Consulting, Mark E. Swerdlik, Illinois State University. Description: Ninth Edition. | Dubuque : McGraw-Hill Education, 2018. | Revised edition of the authors’ Psychological testing and assessment, c2013. Identifiers: LCCN 2016059701| ISBN 9781259870507 | ISBN 1259870502 Subjects: LCSH: Psychological tests. | Psychometrics. | BISAC: PSYCHOLOGY / General. Classification: LCC BF176.C63 2017 | DDC 150.28/7—dc23 LC record available at https://lccn.loc.gov/2016059701 The Internet addresses listed in the text were accurate at the time of publication. The inclusion of a website does not indicate an endorsement by the authors or McGraw-Hill Education, and McGraw-Hill Education does not guarantee the accuracy of the information presented at these sites. mheducation.com/highered This book is dedicated with love to the memory of Edith and Harold Cohen. © 2017 Ronald Jay Cohen. All rights reserved. Contents Preface xiii P A R T I An Overview 1 Psychological Testing and Assessment 1 TESTING AND ASSESSMENT 1 Psychological Testing and Assessment Defined 1 THE TOOLS OF PSYCHOLOGICAL ASSESSMENT 7 The Test 7 The Interview 9 The Portfolio 11 Case History Data 12 Behavioral Observation 12 Role-Play Tests 13 Computers as Tools 14 Other Tools 17 WHO, WHAT, WHY, HOW, AND WHERE? 17 Who Are the Parties? 17 In What Types of Settings Are Assessments Conducted, and Why? 20 How Are Assessments Conducted? 26 Where to Go for Authoritative Information: Reference Sources 30 CLOSE-UP Behavioral Assessment Using Smartphones 5 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Alan Ogle 24 EVERYDAY PSYCHOMETRICS Everyday Accommodations 31 SELF-ASSESSMENT 35 2 Historical, Cultural, and Legal/Ethical Considerations 36 A HISTORICAL PERSPECTIVE 36 Antiquity to the Nineteenth Century 36 The Twentieth Century 39 CULTURE AND ASSESSMENT 42 Evolving Interest in Culture-Related Issues 42 Some Issues Regarding Culture and Assessment 47 Tests and Group Membership 53 LEGAL AND ETHICAL CONSIDERATIONS 55 The Concerns of the Public 55 The Concerns of the Profession 63 The Rights of Testtakers 69 iv CLOSE-UP The Controversial Career of Henry Herbert Goddard 44 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Neil Krishan Aggarwal 51 EVERYDAY PSYCHOMETRICS Life-or-Death Psychological Assessment 66 SELF-ASSESSMENT 74 P A R T II The Science of Psychological Measurement 3 A Statistics Refresher 75 SCALES OF MEASUREMENT 76 Nominal Scales 77 Ordinal Scales 78 Interval Scales 79 Ratio Scales 79 Measurement Scales in Psychology 79 DESCRIBING DATA 81 Frequency Distributions 81 Measures of Central Tendency 84 Measures of Variability 89 Skewness 93 Kurtosis 94 THE NORMAL CURVE 95 The Area Under the Normal Curve 95 STANDARD SCORES 98 z Scores 98 T Scores 99 Other Standard Scores 99 CORRELATION AND INFERENCE 102 The Concept of Correlation 102 The Pearson r 104 The Spearman Rho 106 Graphic Representations of Correlation 107 Meta-Analysis 110 EVERYDAY PSYCHOMETRICS Consumer (of Graphed Data), Beware! 85 CLOSE-UP The Normal Curve and Psychological Tests 96 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Joni L. Mihura 112 SELF-ASSESSMENT 114 4 Of Tests and Testing 115 SOME ASSUMPTIONS ABOUT PSYCHOLOGICAL TESTING AND ASSESSMENT 115 Assumption 1: Psychological Traits and States Exist 115 Assumption 2: Psychological Traits and States Can Be Quantified and Measured 117 Contents v Assumption 3: Test-Related Behavior Predicts Non-Test-Related Behavior 119 Assumption 4: Tests and Other Measurement Techniques Have Strengths and Weaknesses 119 Assumption 5: Various Sources of Error Are Part of the Assessment Process 119 Assumption 6: Testing and Assessment Can Be Conducted in a Fair and Unbiased Manner 121 Assumption 7: Testing and Assessment Benefit Society 121 WHAT’S A “GOOD TEST”? 122 Reliability 122 Validity 122 Other Considerations 123 NORMS 123 Sampling to Develop Norms 126 Types of Norms 132 Fixed Reference Group Scoring Systems 135 Norm-Referenced Versus Criterion-Referenced Evaluation 136 Culture and Inference 138 EVERYDAY PSYCHOMETRICS Putting Tests to the Test 124 CLOSE-UP How “Standard” Is Standard in Measurement? 127 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Steve Julius and Dr. Howard W. Atlas 138 SELF-ASSESSMENT 139 5 Reliability 141 THE CONCEPT OF RELIABILITY 141 Sources of Error Variance 143 RELIABILITY ESTIMATES 145 Test-Retest Reliability Estimates 145 Parallel-Forms and Alternate-Forms Reliability Estimates 149 Split-Half Reliability Estimates 150 Other Methods of Estimating Internal Consistency 152 Measures of Inter-Scorer Reliability 156 USING AND INTERPRETING A COEFFICIENT OF RELIABILITY 158 The Purpose of the Reliability Coefficient 159 The Nature of the Test 160 The True Score Model of Measurement and Alternatives to It 163 RELIABILITY AND INDIVIDUAL SCORES 169 The Standard Error of Measurement 169 The Standard Error of the Difference Between Two Scores 172 CLOSE-UP Psychology’s Replicability Crisis 147 EVERYDAY PSYCHOMETRICS The Importance of the Method Used for Estimating Reliability 157 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Bryce B. Reeve 168 SELF-ASSESSMENT 174 vi Contents 6 Validity 175 THE CONCEPT OF VALIDITY 175 Face Validity 177 Content Validity 178 CRITERION-RELATED VALIDITY 182 What Is a Criterion? 182 Concurrent Validity 184 Predictive Validity 184 CONSTRUCT VALIDITY 187 Evidence of Construct Validity 187 VALIDITY, BIAS, AND FAIRNESS 192 Test Bias 192 Test Fairness 196 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Adam Shoemaker 179 CLOSE-UP The Preliminary Validation of a Measure of Individual Differences in Constructive Versus Unconstructive Worry 193 EVERYDAY PSYCHOMETRICS Adjustment of Test Scores by Group Membership: Fairness in Testing or Foul Play? 197 SELF-ASSESSMENT 199 7 Utility 200 WHAT IS UTILITY? 201 Factors That Affect a Test’s Utility 201 UTILITY ANALYSIS 206 What Is a Utility Analysis? 206 How Is a Utility Analysis Conducted? 207 Some Practical Considerations 221 METHODS FOR SETTING CUT SCORES 225 The Angoff Method 225 The Known Groups Method 226 IRT-Based Methods 226 Other Methods 228 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Delphine Courvoisier 204 CLOSE-UP Utility Analysis: An Illustration 208 EVERYDAY PSYCHOMETRICS The Utility of Police Use of Body Cameras 218 SELF-ASSESSMENT 228 8 Test Development 229 TEST CONCEPTUALIZATION 230 Some Preliminary Questions 232 Pilot Work 234 TEST CONSTRUCTION 234 Scaling 234 Writing Items 239 Scoring Items 246 Contents vii TEST TRYOUT 246 What Is a Good Item? 247 ITEM ANALYSIS 248 The Item-Difficulty Index 248 The Item-Reliability Index 249 The Item-Validity Index 250 The Item-Discrimination Index 250 Item-Characteristic Curves 253 Other Considerations in Item Analysis 256 Qualitative Item Analysis 258 TEST REVISION 260 Test Revision as a Stage in New Test Development 260 Test Revision in the Life Cycle of an Existing Test 262 The Use of IRT in Building and Revising Tests 266 INSTRUCTOR-MADE TESTS FOR IN-CLASS USE 270 Addressing Concerns About Classroom Tests 270 CLOSE-UP Creating and Validating a Test of Asexuality 231 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Scott Birkeland 254 EVERYDAY PSYCHOMETRICS Adapting Tools of Assessment for Use with Specific Cultural Groups 261 SELF-ASSESSMENT 272 P A R T III The Assessment of Intelligence 9 Intelligence and Its Measurement 273 WHAT IS INTELLIGENCE? 273 Perspectives on Intelligence 275 MEASURING INTELLIGENCE 285 Some Tasks Used to Measure Intelligence 285 Some Tests Used to Measure Intelligence 287 ISSUES IN THE ASSESSMENT OF INTELLIGENCE 308 Culture and Measured Intelligence 308 The Flynn Effect 314 The Construct Validity of Tests of Intelligence 315 A PERSPECTIVE 315 CLOSE-UP Factor Analysis 277 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Rebecca Anderson 287 EVERYDAY PSYCHOMETRICS The Armed Services Vocational Aptitude Battery (ASVAB): A Test You Can Take 304 SELF-ASSESSMENT 316 viii Contents 10 Assessment for Education 317 THE ROLE OF TESTING AND ASSESSMENT IN EDUCATION 317 THE CASE FOR AND AGAINST EDUCATIONAL TESTING IN THE SCHOOLS 318 THE COMMON CORE STATE STANDARDS 319 Response to Intervention (RtI) 320 Dynamic Assessment 324 ACHIEVEMENT TESTS 328 Measures of General Achievement 328 Measures of Achievement in Specific Subject Areas 329 APTITUDE TESTS 331 The Preschool Level 333 The Elementary-School Level 338 The Secondary-School Level 340 The College Level and Beyond 341 DIAGNOSTIC TESTS 344 Reading Tests 345 Math Tests 346 PSYCHOEDUCATIONAL TEST BATTERIES 346 The Kaufman Assessment Battery for Children (K-ABC) and the Kaufman Assessment Battery for Children, Second Edition (KABC-II) 346 The Woodcock-Johnson IV (WJ IV) 348 OTHER TOOLS OF ASSESSMENT IN EDUCATIONAL SETTINGS 348 Performance, Portfolio, and Authentic Assessment 349 Peer Appraisal Techniques 351 Measuring Study Habits, Interests, and Attitudes 352 EVERYDAY PSYCHOMETRICS The Common Core Controversy 321 MEET AN ASSESSMENT PROFESSIONAL Meet Eliane Keyes, M.A. 325 CLOSE-UP Educational Assessment: An Eastern Perspective 339 SELF-ASSESSMENT 353 P A R T IV The Assessment of Personality 11 Personality Assessment: An Overview 354 PERSONALITY AND PERSONALITY ASSESSMENT 354 Personality 354 Personality Assessment 355 Traits, Types, and States 355 PERSONALITY ASSESSMENT: SOME BASIC QUESTIONS 359 Who? 360 What? 366 Where? 368 How? 368 Contents ix DEVELOPING INSTRUMENTS TO ASSESS PERSONALITY 377 Logic and Reason 377 Theory 380 Data Reduction Methods 380 Criterion Groups 383 PERSONALITY ASSESSMENT AND CULTURE 395 Acculturation and Related Considerations 396 CLOSE-UP The Personality of Gorillas 361 EVERYDAY PSYCHOMETRICS Some Common Item Formats 372 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Rick Malone 378 SELF-ASSESSMENT 400 12 Personality Assessment Methods 401 OBJECTIVE METHODS 401 How Objective Are Objective Methods of Personality Assessment? 402 PROJECTIVE METHODS 402 Inkblots as Projective Stimuli 404 Pictures as Projective Stimuli 410 Words as Projective Stimuli 418 Sounds as Projective Stimuli 421 The Production of Figure Drawings 422 Projective Methods in Perspective 425 BEHAVIORAL ASSESSMENT METHODS 429 The Who, What, When, Where, Why, and How of It 431 Varieties of Behavioral Assessment 435 Issues in Behavioral Assessment 442 A PERSPECTIVE 444 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Monica Webb Hooper 433 EVERYDAY PSYCHOMETRICS Confessions of a Behavior Rater 436 CLOSE-UP General (g) and Specific (s) Factors in the Diagnosis of Personality Disorders 445 SELF-ASSESSMENT 447 P A R T V Testing and Assessment in Action 13 Clinical and Counseling Assessment 448 AN OVERVIEW 448 The Diagnosis of Mental Disorders 450 The Interview in Clinical Assessment 453 Case History Data 460 Psychological Tests 460 x Contents CULTURALLY INFORMED PSYCHOLOGICAL ASSESSMENT 462 Cultural Aspects of the Interview 464 SPECIAL APPLICATIONS OF CLINICAL MEASURES 467 The Assessment of Addiction and Substance Abuse 467 Forensic Psychological Assessment 469 Diagnosis and evaluation of emotional injury 475 Profiling 475 Custody Evaluations 476 CHILD ABUSE AND NEGLECT 479 Elder Abuse and Neglect 481 Suicide Assessment 483 THE PSYCHOLOGICAL REPORT 484 The Barnum Effect 484 Clinical Versus Mechanical Prediction 486 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Stephen Finn 456 CLOSE-UP PTSD in Returning Veterans and Military Culture 465 EVERYDAY PSYCHOMETRICS Measuring Financial Competency 473 SELF-ASSESSMENT 494 14 Neuropsychological Assessment 489 THE NERVOUS SYSTEM AND BEHAVIOR 489 Neurological Damage and the Concept of Organicity 490 THE NEUROPSYCHOLOGICAL EVALUATION 493 When a Neuropsychological Evaluation Is Indicated 493 General Elements of a Neuropsychological Evaluation 495 The Physical Examination 498 NEUROPSYCHOLOGICAL TESTS 504 Tests of General Intellectual Ability 504 Tests to Measure the Ability to Abstract 506 Tests of Executive Function 507 Tests of Perceptual, Motor, and Perceptual-Motor Function 510 Tests of Verbal Functioning 512 Tests of Memory 513 Neuropsychological Test Batteries 515 OTHER TOOLS OF NEUROPSYCHOLOGICAL ASSESSMENT 519 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Jeanne P. Ryan 505 EVERYDAY PSYCHOMETRICS Medical Diagnostic Aids and Neuropsychological Assessment 520 CLOSE-UP A Typical In-Office Dementia Evaluation 522 SELF-ASSESSMENT 523 Contents xi 15 Assessment, Careers, and Business 524 CAREER CHOICE AND CAREER TRANSITION 524 Measures of Interest 524 Measures of Ability and Aptitude 527 Measures of Personality 533 Other Measures 536 SCREENING, SELECTION, CLASSIFICATION, AND PLACEMENT 538 The Résumé and the Letter of Application 539 The Application Form 539 Letters of Recommendation 540 Interviews 541 Portfolio Assessment 541 Performance Tests 541 Physical Tests 546 COGNITIVE ABILITY, PRODUCTIVITY, AND MOTIVATION MEASURES 548 Measures of Cognitive Ability 548 Productivity 549 Motivation 551 JOB SATISFACTION, ORGANIZATIONAL COMMITMENT, AND ORGANIZATIONAL CULTURE 554 Job Satisfaction 555 Organizational Commitment 555 Organizational Culture 556 OTHER TOOLS OF ASSESSMENT FOR BUSINESS APPLICATIONS 558 Consumer Psychology 558 The Measurement of Attitudes 558 Surveys 560 Motivation Research Methods 563 CLOSE-UP Validity Generalization and the GATB 530 EVERYDAY PSYCHOMETRICS The Selection of Personnel for the Office of Strategic Services (OSS): Assessment and Psychometrics in Action 543 MEET AN ASSESSMENT PROFESSIONAL Meet Dr. Jed Yalof 557 SELF-ASSESSMENT 566 References R-1 Name Index I-1 Glossary/Index I-12 Timeline T-1 xii Contents Preface W e are proud to welcome instructors of a measurement course in psychology to this ninth edition of Psychological Testing and Assessment. Thank you for the privilege of assisting in the exciting task of introducing the world of tests and measurement to your students. In this preface, we impart our vision for a measurement textbook, as well as the philosophy that has driven, and that continues to drive, the organization, content, writing style, and pedagogy of this book. We’ll briefly look back at this book’s heritage and discuss what is new and distinctive about this ninth edition. Of particular interest to instructors, this preface will overview the authors’ general approach to the course content and distinguish how that approach differs from other measurement textbooks. For students who happen to be curious enough to read this preface (or ambitious enough to read it despite the fact that it was not assigned), we would hope that your takeaway from it has to do with the authors’ genuine dedication to making this book the far-and-away best available textbook for your measurement course. Our Vision for a Textbook on Psychological Testing and Assessment First and foremost, let’s get out there that the subject matter of this course is psychological testing and assessment—a fact that is contrary to the message conveyed by an array of would-be competitor books, all distinguished by their anachronistic “psychological testing” title. Of course we cover tests and testing, and no available textbook does it better or more comprehensively. But it behooves us to observe that we are now well into the twenty-first century and it has long been recognized that tests are only one tool of assessment. Psychological testing is a process that can be—perhaps reminiscent of those books with the same title— impersonal, noncreative, uninspired, routine, and even robotic in nature. By contrast, psychological assessment is a very human, dynamic, custom, creative, and collaborative enterprise. These aspects of the distinction between psychological testing and psychological assessment are not trivial. Paralleling important differences between our book’s title and that of other books in this area are key differences in the way that the subject matter of the course is approached. In routine writing and through a variety of pedagogical tools, we attempt to draw students into the world of testing and assessment by humanizing the material. Our very human approach to the course material stands in stark contrast to the “by-the-numbers” approach of some of our competitors; the latter approach can easily alienate readers, prompting them to “tune out.” Let’s briefly elaborate on this critical point. Although most of our competitors begin by organizing their books with an outline that for the most part mimics our own—right down to the inclusion of the Statistics Refresher that we innovated some 30 years ago—the way that they cover that subject matter, and the pedagogical tools they rely on to assist student learning, bear only cosmetic resemblance to our approach. We take every opportunity to illustrate the course material by putting a human face to it, and by providing practical, “every day” examples of the principles and procedures at work. This approach differs in key ways from the approach of other books in the area, where a “practical approach” may instead be equated with the intermingling of statistical or other exercises within every chapter of the book. Presumably, according to the latter vision, a textbook is a simultaneous delivery system for both course-related information and course-related exercises. Students are expected to read their textbooks until such time that their reading is interrupted by an exercise. After the completion of the exercise, students are expected to go back to the reading, but only xiii until they happen upon another exercise. It is thus the norm to interrupt absorption in assigned reading on a relatively random (variable ratio) schedule in order to have students complete general, one-size-fits-all exercises. Students using such a book are not encouraged to concentrate on assigned reading; they may even be tacitly encouraged to do the opposite. The emphasis given to students having to complete exercises scattered within readings seems especially misplaced when, as is often the case with such one-size-fits-all tasks, some of the exercises will be way too easy for students in some classes and way too difficult for students in others. This brings to mind our own experience with testing-related exercises being assigned to varied groups of introductory students. For several years and through several editions, our textbook was published with a supplementary exercises workbook. After extensive feedback from many instructors, some of whom used our book in their classes and some of whom did not, we determined that matters related to the choice, content, and level of supplementary exercises were better left to individual instructors as opposed to textbook authors. In general, instructors preferred to assign their own supplementary exercises, which could be custom-designed for the needs of their particular students and the goals of their particular course. A workbook of exercises, complete with detailed, step-by-step, illustrated solutions of statistical and psychometric problems, was determined by us to add little value to our textbook and it is therefore no longer offered. What we learned, and what we now believe, is that there is great value to supplementary, ancillary exercises for students taking an introductory course in measurement. However, these exercises are of optimal use to the student when they are custom-designed (or selected) by the instructor based on factors such as the level and interest of the students in the class, and the students’ in-class and out-of-class study schedule. To be clear, supplemental exercises randomly embedded in a textbook work, in our view, not to facilitate students’ immersion and concentration in assigned reading, but to obliterate it.1 Given the fact that decisions regarding supplementary exercises are best left to individual instructors, the difference between our own approach to the subject matter of the course and that of other approaches are even more profound. In this ninth edition, we have concentrated our attention and effort to crafting a textbook that will immerse and involve students in assigned readings and motivate them to engage in critical and generative thinking about what they have read. Contrast that vision with one in which author effort is divided between writing text and writing nonsupplementary exercises. Could the net result of the latter approach be a textbook that divides student attention between assigned readings and assigned (or unassigned) exercises? Seasoned instructors may concur with our view that most students will skip the intrusive and distracting exercises when they are not specifically assigned for completion by the instructor. In the case where the exercises are assigned, students may well skim the reading to complete the exercises. No available textbook is more focused on being practical, timely, and “real-life” oriented than our’s. Further, no other textbook provides students in an introductory course with a more readable or more comprehensive account of how psychological tests and assessment-related procedures are used in practice. That has been the case for some 30 years and it most certainly is the case today. With that as background, let’s briefly sum up some of our concerns with regard to certain members of the current community of “psychological testing” books. In our view, red flags rise when books equate “practical” with the systematic burdening of readers with obstacles to immersion in assigned readings. Red flags rise when what appears to be 1. We urge any instructors curious about this assertion to informally evaluate it by asking a student or two how they feel about the prospect of scattering statistical exercises in their assigned reading. If the assigned reading is at all immersive, the modal response may be something like “maddening.” xiv Preface well-intentioned pedagogy, perhaps paradoxically, frustrates rather than advances student learning. Red flags rise when randomly placed, one-size-fits-all exercises are used to ostensibly compensate for an apparent inability to write in a way that involves, teaches, and inspires. Especially with regard to a textbook at the introductory level, what is critical is the breadth and depth of coverage of how tests and other tools of assessment are actually used in practice. Practice-level proficiency and hands-on experience are always nice, but may in some cases be too ambitious. For example, a practical approach to factor analysis in a textbook for an introductory measurement course need not equip the student to conduct a factor analysis. Rather, the coverage ideally provides the student with a sound grounding in what this widely used set of techniques are, as well as how and why they are used. Similarly a practical approach to test utility, as exemplified in Chapter 7, provides students with a sound grounding in what that construct is, as well as how and why it is applied in practice. Of course when it comes to breadth and depth of coverage of how tests and other tools of assessment are actually used in practice, we have long been the standard by which other books are measured. Consider in this context a small sampling of what is new, timely, and relevant in this ninth edition. The subject of our Chapter 1 Close-Up is behavioral assessment using smart phones. The subject of our Everyday Psychometrics in Chapter 7 on utility is the utility of police use of body cameras.2 Terrorism is a matter of worldwide concern and in Chapter 11, the professional profiled in our Meet an Assessment Professional feature is Colonel Rick Malone of the United States Army’s Criminal Investigation Command. Dr. Malone shares some intriguing insights regarding his area of expertise: threat assessment. New to this ninth edition is a section of online-only boxes, apps, and links or “OOBALs.” Among the OOBALs presented online for chapter-specific, supplementary work are a group of original essays, apps, and podcasts, as well as links to stories, videos, or articles from the popular press or the scholarly literature. After reading Chapter 10 (Assessment of Education), for example, students may wish to check out OOBAL-10-B1, which is an original essay that was written especially for this book by guest author Daniel Teichman. In his essay entitled “Educational Evaluation in the Age of the MOOC,” Teichman discusses the challenge of educational evaluation when conducting a massive open online class (MOOC) for some 200,000 students. Much more about our vision for this textbook and its supplements, as well as more previews of what is new and exciting in this ninth edition, is presented in what follows. Organization From the first edition of our book forward, we have organized the information to be presented into five major sections. Part I, An Overview, contains two chapters that do just that. Chapter 1 provides a comprehensive overview of the field, including some important definitional issues, a general description of tools of assessment, and related important information couched as answers to questions regarding the who, what, why, how, and where of the enterprise. The foundation for the material to come continues to be laid in the second chapter of the overview, which deals with historical, cultural, and legal/ethical issues. The material presented 2. This essay is an informative and timely discussion of the utility of police-worn body cameras in reducing use-of-force complaints. Parenthetically, let’s share our view that the concept of utility seems lost in, or at least given inadequate coverage in other measurement books. It seems that we may have caught many of those “psychological testing” books off-guard by devoting a chapter to this construct beginning with our seventh edition—this at a time when utility was not even an indexed term in most of them. Attempts to compensate have ranged from doing nothing at all to doing near nothing at all by equating “utility” with “validity.” For the record, although utility is related to validity, much as reliability is related to validity, we believe it is misleading to even intimate that “utility” and “validity” are synonymous. Preface xv in Chapter 2 clearly sets a context for everything that will follow. To relegate such material to the back of the book (as a kind of elective topic, much like the way that legal/ethical issues are treated in some books), or to ignore presentation of such material altogether (as most other books have done with regard to cultural issues in assessment), is, in our estimation, a grave error. “Back page infrequency” (to borrow an MMPI-2 term) is too often the norm, and relegation of this critically important information to the back pages of a textbook too often translates to a potential shortchanging of students with regard to key cultural, historical, and legal/ethical information. The importance of exposure early on to relevant historical, cultural, and legal/ethical issues cannot be overemphasized. This exposure sets a context for succeeding coverage of psychometrics and creates an essential lens through which to view and process such material. Part II, The Science of Psychological Measurement, contains Chapters 3 through 8. These six chapters were designed to build—logically and sequentially—on the student’s knowledge of psychometric principles. Part II begins with a chapter reviewing basic statistical principles and ends with a chapter on test construction. In between, there is extensive discussion of assumptions inherent in the enterprise, the elements of good test construction, as well as the concepts of norms, correlation, inference, reliability, and validity. All of the measurement textbooks that came before us were written based on the assumption that every student taking the course was up to speed on all of the statistical concepts that would be necessary to build on learning about psychometrics. In theory, at least, there was no reason not to assume this; statistics was a prerequisite to taking the course. In practice, a different picture emerged. It was simply not the case that all students were adequately and equally prepared to begin learning statistics-based measurement concepts. Our remedy for this problem, some 30 years ago, was to include a “Statistics Refresher” chapter early on, just prior to building on students’ statistics-based knowledge. The rest, as they say, is history... Our book forever changed for the better the way the measurement course was taught and the way all subsequent textbooks for the course would be written. Our unique coverage of the assessment of intelligence and personality, as well as our coverage of assessment for various applications (ranging from neuropsychological to business and organizational applications), made relics of the typical “psychological testing” course outline as it existed prior to the publication of our first edition in 1988. In our seventh edition, in response to increasing general interest in test utility, we added a chapter on this important construct right after our chapters on the constructs of reliability and validity. Let’s note here that topics such as utility and utility analysis can get extremely complicated. However, we have never shied away from the presentation of complicated subject matter. For example, we were the first introductory textbook to present detailed information related to factor analysis. As more commercial publishers and other test users have adopted the use of item response theory (IRT) in test construction, our coverage of IRT has kept pace. As more test reviews have begun to evaluate tests not only in terms of variables such as reliability and validity but in terms of utility, we saw a need for the inclusion of a chapter on that topic. Of course, no matter how “difficult” the concepts we present are, we never for a moment lose sight of the appropriate level of presentation. This book is designed for students taking a first course in psychological testing and assessment. Our objective in presenting material on methods such as IRT and utility analysis is simply to acquaint the introductory student with these techniques. The depth of the presentation in these and other areas has always been guided and informed by extensive reviews from a geographically diverse sampling of instructors who teach measurement courses. For users of this textbook, what currently tends to be required is a conceptual understanding of commonly used IRT methods. We believe our presentation of this material effectively conveys such an understanding. Moreover, it does so without unnecessarily burdening students with level-inappropriate formulas and calculations. xvi Preface Part III of this book, The Assessment of Abilities and Aptitudes, contains two chapters, one on intelligence and its assessment, and the other on assessment in schools and other educational settings. In past editions of this book, two chapters were devoted to the assessment of intelligence. To understand why, it is instructive to consider what the coverage of intelligence testing looked like in the then available introductory measurement textbooks three decades ago. While the books all covered tests of intelligence, they devoted little or no attention to defining and discussing the construct of intelligence. We called attention to this problem and attempted to remedy it by differentiating our book with a chapter devoted to imparting a conceptual understanding of intelligence. Although revolutionary at the time, the logic of our approach had widespread appeal. Before long, the typical “psychological testing” course of the 1980s was being restructured to include conceptual discussions of concepts such as “intelligence” and “personality” before proceeding to discuss their measurement. The “psychological testing” textbooks of the day also followed our lead. And so, to the present day, two-chapter-coverage of the assessment of intelligence (with the first chapter providing a discussion of the construct of intelligence) has become the norm. That is, of course, until the publication of this ninth edition; more on the news there in just a moment. In retrospect, it seems reasonable to conclude that our addition of a chapter on the nature of intelligence, much like our addition of a statistics refresher, did more than remedy a serious drawback in existing measurement textbooks; it forever revolutionized the way that the measurement course was taught in classrooms around the world. It did this first of all by making the teaching of the course more logical. This is so because the logic of our guiding principle—fully define and discuss the psychological construct being measured before discussing its measurement—had wide appeal. In our first edition, we also extended that logic to the discussion of the measurement of other psychological constructs such as personality. Another benefit we saw in adding the conceptual coverage was that such coverage would serve to “humanize” the content. After all, “Binet” was more than just the name of a psychological test; it was the name of a living, breathing person. Also, since our first edition, we have revolutionized textbook coverage of psychological tests—this by a philosophy of “less is more” when it comes to such coverage. Back in the 1980s, the “psychological testing” books of the day had elements reminiscent of Tests in Print. They provided reliability, validity, and related psychometric data on dozens of psychological tests. But we raised the question, “Why duplicate in a textbook information about dozens of tests that is readily available from reference sources?” We further resolved to limit detailed coverage of psychological tests to a handful of representative tests. Once again, the simple logic of our approach had widespread appeal, and other textbooks in the area—both then, and to the present day—all followed suit. There is another trend in textbook coverage of the measurement course that also figured prominently in our decision to cover the assessment of intelligence in a single chapter. This trend has to do with the widespread availability of online resources to supplement coverage of a specific topic. We have long taken advantage of this fact by making available various supplementary materials online to our readers, or by supplying links to such materials. Some three decades after we revolutionized the organization of textbook coverage of the measurement course in so many significant ways, it was time to re-evaluate whether two chapters to cover the subject of intelligence assessment was still necessary. We gave thoughtful consideration to this question and sought-out the opinion of trusted colleagues. In the end, we determined that coverage of the construct and assessment of intelligence could be accomplished in a single chapter. And so, in the interest of streamlining this book in length, Chapter 9 in this ninth edition incorporates text formerly in Chapters 9 and 10 of the eighth edition. By the way, some of the text from each of these eighth edition chapters has been preserved in the Chapter 9 supplementary package of online-only boxes, apps, and links (OOBALs). See, for example, OOBAL-9-B2 for an essay entitled “Nature versus Nurture in Measured Intelligence.” Preface xvii Part IV, The Assessment of Personality, contains two chapters, which respectively overview how personality assessments are conducted, and the various methods used. Part V, Testing and Assessment in Action, is designed to convey to students a sense of how a sampling of tests and other tools of assessment are actually used in clinical, counseling, business, and other settings. Content In addition to a logical organization that sequentially builds on student learning, we view content selection as another key element of our appeal. The multifaceted nature and complexity of the discipline affords textbook authors wide latitude in terms of what material to elaborate on, what material to ignore, and what material to highlight, exemplify, or illustrate. In selecting content to be covered for chapters, the primary question for us was most typically “What do students need to know?” So, for example, since the publication of previous editions of this book, the field of educational evaluation has been greatly influenced by the widespread implementation of the Common Core Standards. Accordingly, we take cognizance of these changes in the K-through-12 education landscape and their implications for evaluation in education. Students of educational assessment need to know about the Common Core Standards and relevant coverage of these standards can be found in this ninth edition in our chapter on educational assessment. While due consideration is given to creating content that students need to know, consideration is also given to relevant topics that will engage interest and serve as stimuli for critical or generative thinking. In the area of neuropsychological assessment, for example, the topic of Alzheimer’s disease is one that generates a great deal of interest. Most students have seen articles or feature stories in the popular media that review the signs and symptoms of this disease. However, while students are aware that such patients are typically referred to a neurologist for formal diagnosis, many questions remain about how a diagnosis of Alzheimer’s disease is clinically made. The Close-Up in our chapter on neuropsychological assessment addresses those frequently asked questions. It was guest-authored by an experienced neurologist and written especially for students of psychological assessment reading this textbook. Let’s note here that in this ninth edition, more than in any previous edition of this textbook, we have drawn on the firsthand knowledge of psychological assessment experts from around the world. Specifically, we have asked these experts to guest-author brief essays in the form of Close-Up, Everyday Psychometrics, or Meet an Assessment Professional features. For example, in one of our chapters that deal with personality assessment, two experts on primate behavior (including one who is currently working at Dian Fossey’s research center in Karisoke, in Rwanda) prepared an essay on evaluating the personality of gorillas. Written especially for us, this Close-Up makes an informative contribution to the literature on cross-species personality assessment. In our chapter on test construction, an Australian team of behavioral scientists guest-authored a Close-Up entitled “Adapting Tools of Assessment for Use with Specific Cultural Groups.” This essay recounts some of the intriguing culture-related challenges inherent in the psychological assessment of clients from the Aboriginal community. Sensitivity to cultural issues in psychological testing and assessment is essential, and this textbook has long set the standard for coverage of such issues. Coverage of cultural issues begins in earnest in Chapter 2, where we define culture and overview the importance of cultural considerations in everything from test development to standards of evaluation. Then, much like an identifiable musical theme that recurs throughout a symphony, echoes of the importance of culture repeat in various chapters throughout this book. For example, the echo is heard in Chapter 4 where, among other things, we continue a long tradition of acquainting students with the “do’s and don’ts” of culturally informed assessment. In Chapter 13, our chapter on xviii Preface assessment in clinical and counseling settings, there is a discussion of acculturation and culture as these issues pertain to clinical assessment. Also in that chapter, students will find a thought- provoking Close-Up entitled, “PTSD in Veterans and the Idealized Culture of Warrior Masculinity.” Guest-authored especially for us by Duncan M. Shields, this timely contribution to the clinical literature sheds light on the diagnosis and treatment of post-traumatic stress disorder (PTSD) from a new and novel, cultural perspective. It is important to note that our presentation of culture-related issues (as well as most other issues relevant to measurement in psychology) does not end with the final page of the final chapter of this book. The breadth and depth of our coverage has been greatly extended by this book’s supplementary package of “OOBALs” (our acronym for online-only boxes, apps and links) available at mhhe.cohentesting9/OOBALS. In the Chapter 9 OOBAL, for example, students will find an original essay that explores the nature of intelligence from an Eastern perspective. By way of background, this contribution stemmed from an invited presentation on psychological assessment made by the senior author of this textbook (RJC) to the students and faculty of the School of Psychology at Shaanxi Normal University (SNNU) in China, in December, 2015. After the presentation, Cohen was a guest at a dinner hosted by the SNNU faculty and administration. The dinner was memorable not only for an array of tantalizing delicacies, but for the intriguing cultural insights advanced by the SNNU scholars with respect to measurement-related issues. During the course of the evening, Cohen proposed that Dean Xuqun You and Vice Dean Ning He collaborate on brief essays contrasting Eastern versus Western perspectives on topics such as the measurement of intelligence and educational assessment. Professors You and He, along with some of their colleagues, accepted Cohen’s invitation. One of the resulting “Eastern Perspective” essays can be found herein as the Close-Up in our chapter on educational assessment. This essay, guest-authored by Chengting Ju, Ning He, and Xuqun You is entitled, “Educational Assessment: An Eastern Perspective.” The other original essay, guest-authored by Yuanbo Gu, Ning He, and Xuqun is “Intelligence: An Eastern Perspective.” This informative essay supplements our coverage of the assessment of intelligence as OOBAL-9-B12. Of course, whether published in the textbook as a Close-Up or published in the online supplementary package as an “OOBAL,” each of these “Eastern perspective” essays are well worth reading for their valuable contribution to the literature on cross-cultural considerations in psychological assessment. In addition to standard-setting content related to cultural issues, mention must also be made of our leadership role with respect to coverage of historical and legal/ethical aspects of measurement in psychology. Our own appreciation for the importance of history is emphasized by the listing of noteworthy historical events that is set within the front and back covers of this textbook. As such, readers may be greeted with some aspect of the history of the enterprise on every occasion that they open the book. Although historical vignettes are distributed throughout the book to help set a context or advance understanding, formal coverage begins in Chapter 2. Important historical aspects of testing and assessment may also be found in Close-Ups. See, for example, the fascinating account of the controversial career of Henry Goddard found in Chapter 2. In a Close-Up in Chapter 15, students will discover what contemporary assessment professionals can learn from World War II-vintage assessment data collected by the Office of Strategic Services (OSS). In this engrossing essay, iconic data meets contemporary data analytic methods with brilliant new insights as a result. This Close-Up was guest-authored by Mark F. Lenzenweger, who is a State University of New York (SUNY) Distinguished Professor in the Department of Psychology at The State University of New York at Binghamton. Much like content pertaining to relevant historical and culture-related material, our discussion of legal–ethical issues, from our first edition through to the present day, has been standard-setting. Discussion of legal and ethical issues as they apply to psychological testing and assessment provides students not only with context essential for understanding psychometric Preface xix principles and practice, but another lens through which to filter understanding of tests and measurement. In the first edition, while we got the addition of this pioneering content right, we could have done a better job in terms of placement. In retrospect, the first edition would have benefitted from the discussion of such issues much earlier than the last chapter. But in response to the many compelling arguments reviewers and users of that book, discussion of legal/ethical issues was prioritized in Chapter 2 by the time that our second edition was published. The move helped ensure that students were properly equipped to appreciate the role of legal and ethical issues in the many varied settings in which psychological testing and assessment takes place.3 Another element of our vision for the content of this book has to do with the art program; that is, the photos, drawings, and other types of illustrations used in a textbook. Before the publication of our ground-breaking first edition, what passed for an art program in the available “psychological testing” textbooks were some number-intensive graphs and tables, as well as photos of test kits or test materials. In general, photos and other illustrations seemed to be inserted more to break up text than to complement it. For us, the art program is an important element of a textbook, not a device for pacing. Illustrations can help draw students into the narrative, and then reinforce learning by solidifying meaningful visual associations to the written words. Photos can be powerful tools to stir the imagination. See, for example, the photo of Army recruits being tested in Chapter 1, or the photo of Ellis Island immigrants being tested in Chapter 2. Photos can bring to life and “humanize” the findings of measurement-related research. See, for example, the photo in Chapter 3 regarding the study that examined the relationship between grades and cell phone use in class. Photos of many past and present luminaries in the field (such as John Exner, Jr. and Ralph Reitan), and photos accompanying the persons featured in our Meet an Assessment Professional boxes all serve to breathe life into their respective accounts and descriptions. In the world of textbooks, photos such as the sampling of the ones described here may not seem very revolutionary. However, in the world of measurement textbooks, our innovative art program has been and remains quite revolutionary. One factor that has always distinguished us from other books in this area is the extent to which we have tried to “humanize” the course subject matter; the art program is just another element of this textbook pressed into the service of that objective. “Humanization” of Content This ninth edition was conceived with a commitment to continuing our three-decade tradition of exemplary organization, exceptional writing, timely content, and solid pedagogy. Equally important was our desire to spare no effort in making this book as readable and as involving for students as it could possibly be. Our “secret sauce” in accomplishing this is, at this point, not much of a secret. We have the highest respect for the students for whom this book is written. We try to show that respect by never underestimating their capacity to become immersed in course-relevant narratives that are presented clearly and 3. Apparently, some of those “psychological testing” books were so enamored with our first edition table of contents that they failed to revise with the times. For example, to date, they still present legal/ethical issues at the end of the book, instead of where it will do students the most good (i.e., near the beginning of the book). Also, in the faded footsteps of an Appendix to our first edition, one can still find in the current edition of a “psychological testing” book a hard copy Appendix that lists test publishers along with contact information. But in the age of the Internet, and in the rare circumstance that current contact information for a test publisher was required, the Appendix of a textbook would not be the first (or best) choice to find such information. Today, there really is no defensible reason to weight down a textbook with these extra, thoroughly unnecessary pages. In all probability, some of the information in the Appendix was out-of-date even before it went to press. So, while imitation is the sincerest form of flattery, and we are flattered by attempts to emulate our first edition (circa 1988), those “psychological testing” books that still have that Appendix, need to have it removed. xx Preface straightforwardly. With the goal of further drawing the student into the subject matter, we make every effort possible to “humanize” the presentation of topics covered. So, what does “human- ization” in this context actually mean? While other authors in this discipline impress us as blindly intent on viewing the field as Greek letters to be understood and formulas to be memorized, we view an introduction to the field to be about people as much as anything else. Students are more motivated to learn this material when they can place it in a human context. Many psychology students simply do not respond well to endless presentations of psychometric concepts and formulas. In our opinion, to not bring a human face to the field of psychological testing and assessment, is to risk perpetuating all of those unpleasant (and now unfair) rumors about the course that first began circulating long before the time that the senior author himself was an undergraduate. Our effort to humanize the material is evident in the various ways we have tried to bring a face (if not a helping voice) to the material. The inclusion of Meet an Assessment Professional is a means toward that end, as it quite literally “brings a face” to the enterprise. Our inclusion of interesting biographical facts on historical figures in assessment is also representative of efforts to humanize the material. Consider in this context the photo and brief biographical statement of MMPI-2 senior author James Butcher in Chapter 11 (p. 389). Whether through such images of historical personages or by other means, our objective has been made to truly involve students via intriguing, real-life illustrations of the material being discussed. See, for example, the discussion of life-or-death psychological assessment and the ethical issues involved in the Close-Up feature of Chapter 2. Or check out the candid “confessions” of a behavior rater in the Everyday Psychometrics feature in Chapter 12. So how has our “humanization” of the material in this discipline been received by some of its more “hard core” and “old school” practitioners? Very well, thank you—at least from all that we have heard, and the dozens of reviews that we have read over the years. What stands out prominently in the mind of the senior author (RJC) was the reaction of one particular psychometrician whom I happened to meet at an APA convention not long after the first edition of this text was published. Lee J. Cronbach was quite animated as he shared with me his delight with the book, and how refreshingly different he thought that it was from anything comparable that had been published. I was so grateful to Lee for his encouragement, and felt so uplifted by that meeting, that I subsequently requested a photo from Lee for use in the second edition. The photo he sent was indeed published in the second edition of this book—this despite the fact that at that time, Lee had a measurement book that could be viewed as a direct competitor to ours. Regardless, I felt it was important not only to acknowledge Lee’s esteemed place in measurement history, but to express my sincere gratitude in this way for his kind, inspiring, and motivating words, as well as for what I perceived as his most valued “seal of approval.” Pedagogical Tools The objective of incorporating timely, relevant, and intriguing illustrations of assessment-related material is furthered by several pedagogical tools built into the text. One pedagogical tool we created several editions ago is Everyday Psychometrics. In each chapter of the book, relevant, practical, and “everyday” examples of the material being discussed are highlighted in an Everyday Psychometrics box. For example, in the Everyday Psychometrics presented in Chapter 1 (“Everyday Accommodations”), students will be introduced to accommodations made in the testing of persons with handicapping conditions. In Chapter 4, the Everyday Psychometrics feature (“Putting Tests to the Test”) equips students with a working overview of the variables they need to be thinking about when reading about a test and evaluating how satisfactory the test really is for a particular purpose. In Chapter 5, the subject of the Everyday Psychometrics is how the method used to estimate diagnostic reliability may affect the obtained estimate of reliability. Preface xxi A pedagogical tool called Meet an Assessment Professional was first introduced in the seventh edition. This feature provides a forum through which everyday users of psychological tests from various fields can share insights, experiences, and advice with students. The result is that in each chapter of this book, students are introduced to a different test user and provided with an intriguing glimpse of their professional life—this in the form of a Meet an Assessment Professional (MAP) essay. For example, in Chapter 4, students will meet a team of test users, Drs. Steve Julius and Howard Atlas, who have pressed psychometric knowledge into the service of professional sports. They provide a unique and fascinating account of how application of their knowledge of was used to improve the on-court of achievement of the Chicago Bulls. A MAP essay from Stephen Finn, the well-known proponent of therapeutic assessment is presented in Chapter 13. Among the many MAP essays that are new to this edition are essays from two mental-health professionals serving in the military. Dr. Alan Ogle introduces readers to aspects of the work of an Air Force psychologist in Chapter 1. In Chapter 11, army psychiatrist Dr. Rick Malone shares his expertise in the area of threat assessment. In other MAPs new to this edition, the senior author of an oft-cited meta-analysis that was published in Psychological Bulletin shares her insights on meta-analytic methods in Chapter 3, while a psychiatrist who specializes in cultural issues introduces himself to students in Chapter 2. Our use of the pedagogical tool referred to as a “Close-Up,” is reserved for more in-depth and detailed consideration of specific topics related to those under discussion. The Close-Up in our chapter on test construction, for example, acquaints readers with the trials and tribulations of test developers working to create a test to measure asexuality. The Close-Up in one of our chapters on personality assessment raises the intriguing question of whether it is meaningful to speak of general (g) and specific (s) factors in the diagnosis of personality disorders. There are other pedagogical tools that readers (as well as other textbook authors) may take for granted—but we do not. Consider, in this context, the various tables and figures found in every chapter. In addition to their more traditional use, we view tables as space-saving devices in which a lot of information may be presented. For example, in the first chapter alone, tables are used to provide succinct but meaningful comparisons between the terms testing and assessment, the pros and cons of computer-assisted psychological assessment, and the pros and cons of using various sources of information about tests. Critical thinking may be defined as “the active employment of judgment capabilities and evaluative skills in the thought process” (Cohen, 1994, p. 12). Generative thinking may be defined as “the goal-oriented intellectual production of new or creative ideas” (Cohen, 1994, p. 13). The exercise of both of these processes, we believe, helps optimize one’s chances for success in the academic world as well as in more applied pursuits. In the early editions of this textbook, questions designed to stimulate critical and generative thinking were raised “the old-fashioned way.” That is, they were right in the text, and usually part of a paragraph. Acting on the advice of reviewers, we made this special feature of our writing even more special beginning with the sixth edition of this book; we raised these critical thinking questions in the margins with a Just Think heading. Perhaps with some encouragement from their instructors, motivated students will, in fact, give thoughtful consideration to these (critical and generative thought-provoking) Just Think questions. In addition to critical thinking and generative thinking questions called out in the text, other pedagogical aids in this book include original cartoons created by the authors, original illustrations created by the authors (including the model of memory in Chapter 14), and original acronyms created by the authors.4 Each chapter ends with a Self-Assessment feature that students may use to test themselves with respect to key terms and concepts presented in the 4. By the way, our use of the French word for black (noir) as an acronym for levels of measurement (nominal, ordinal, interval, and ratio) now appears in other textbooks. xxii Preface text. Also at the end of each chapter is a reminder to students to check out the supplementary package of online-only boxes, apps, and links (OOBALs). And speaking of online resources... Connect Logo The 9th edition of Psychological Testing and Assessment is now available online with Connect, McGraw-Hill Education’s integrated assignment and assessment platform. Connect also offers SmartBook for the new edition, which is the first adaptive reading experience proven to improve grades and help students study more effectively. All of the title’s website and ancillary content is also available through Connect, including: An Instructor’s Manual for each chapter. A full Test Bank of multiple choice questions that test students on central concepts and ideas in each chapter. Lecture Slides for instructor use in class. Writing Style What type of writing style or author voice works best with students being introduced to the field of psychological testing and assessment? Instructors familiar with the many measurement books that have come (and gone) may agree with us that the “voice” of too many authors in this area might best be characterized as humorless and academic to the point of arrogance or pomposity. Students do not tend to respond well to textbooks written in such styles, and their eagerness and willingness to spend study time with these authors (and even their satisfaction with the course as a whole) may easily suffer as a consequence. In a writing style that could be characterized as somewhat informal and—to the extent possible, given the medium and particular subject being covered—“conversational,” we have made every effort to convey the material to be presented as clearly as humanly possible. In practice, this means: keeping the vocabulary of the presentation appropriate (without ever “dumbing-down” or trivializing the material); presenting so-called difficult material in step-by-step fashion where appropriate, and always preparing students for its presentation by placing it in an understandable context; italicizing the first use of a key word or phrase and then bolding it when a formal definition is given; providing a relatively large glossary of terms to which students can refer; supplementing material where appropriate with visual aids, tables, or other illustrations. supplementing material where appropriate with intriguing historical facts (as in the Chapter 12 material on projectives and the projective test created by B. F. Skinner); incorporating timely, relevant, and intriguing illustrations of assessment-related material in the text as well as in the online materials. In addition, we have interspersed some elements of humor in various forms (original cartoons, illustrations, and vignettes) throughout the text. The judicious use of humor to engage and maintain student interest is something of a novelty among measurement textbooks. Where else would one turn for pedagogy that employs an example involving a bimodal distribution of test scores from a new trade school called The Home Study School of Elvis Presley Impersonators? As readers learn about face validity, they discover why it “gets no respect” and how it has been characterized as “the Rodney Dangerfield of psychometric variables.” Preface xxiii ® Required=Results ©Hero Images/Getty Images RF McGraw-Hill Connect® Learn Without Limits Connect is a teaching and learning platform that is proven to deliver better results for students and instructors. Connect empowers students by continually adapting to deliver precisely what they need, when they need it, and how they need it, so your class time is more engaging and eﬀective. 73% of instructors who use Connect require it; instructor satisfaction increases by 28% Using Connect improves retention when Connect is required. rates by 19.8%, passing rates by 12.7%, and exam scores by 9.1%. Analytics Connect Insight® Connect Insight is Connect’s new one-of-a-kind visual analytics dashboard—now available for both instructors and students—that provides at-a- glance information regarding student performance, which is immediately actionable. By presenting assignment, assessment, and topical performance results together with a time metric that is easily visible for aggregate or individual results, Connect Insight gives the user the ability to take a just-in-time approach to teaching and Students can view learning, which was never before available. Connect Insight presents their results for any data that empowers students and helps instructors improve class Connect course. performance in a way that is efficient and eﬀective. Mobile Connect’s new, intuitive mobile interface gives students and instructors ﬂexible and convenient, anytime–anywhere access to all components of the Connect platform. Adaptive THE ADAPTIVE READING EXPERIENCE DESIGNED TO TRANSFORM THE WAY STUDENTS READ More students earn A’s and B’s when they use McGraw-Hill Education Adaptive products. SmartBook® Proven to help students improve grades and study more efficiently, SmartBook contains the same content within the print book, but actively tailors that content to the needs of the individual. SmartBook’s adaptive technology provides precise, personalized instruction on what the student should do next, guiding the student to master and remember key concepts, targeting gaps in knowledge and oﬀering customized feedback, and driving the student toward comprehension and retention of the subject matter. Available on tablets, SmartBook puts learning at the student’s ﬁngertips—anywhere, anytime. Over 5.7 billion questions have been answered, making McGraw- Hill Education products more intelligent, reliable, and precise. www.mheducation.com Numerous other illustrations could be cited here. But let’s reserve those smiles as a pleasant surprise when readers happen to come upon them. Acknowledgments Thanks to the members of the academic community who have wholeheartedly placed their confidence in this book through all or part of its nine-edition life-cycle to date. Your trust in our ability to help your students navigate the complex world of measurement in psychology is a source of inspiration to us. We appreciate the privilege of assisting you in the education and professional growth of your students, and we will never take that privilege for granted. Every edition of this book has begun with blueprinting designed with the singular objective of making this book far-and-away best in the field of available textbooks in terms of organization, content, pedagogy, and writing. Helping the authors to meet that objective were developmental editor Erin Guendelsberger and project supervisor Jamie Laferrera along with a number of guest contributors who graciously gave of their time, talent, and expertise. To be the all-around best textbook in a particular subject area takes, as they say, “a village.” On behalf of the authors, a hearty “thank you” is due to many “villagers” in the academic and professional community who wrote or reviewed something for this book, or otherwise contributed to it. First and foremost, thank you to all of the following people who wrote essays designed to enhance and enrich the student experience of the course work. In order of appearance of the ninth edition chapter that their essay appeared in, we say thanks to the following contributors of guest-authored Meet an Assessment Professional, Everyday Psychometrics, Close-Up, or OOBAL essays: Alan, D. Ogle of the 559th Medical Group, Military Training Consult Service of the United States Air Force; Dror Ben-Zeev of the Department of Psychiatry of the Geisel School of Medicine at Dartmouth; Neil Krishan Aggarwal of the New York State Psychiatric Institute; Joni L. Mihura of the Department of Psychology of the University of Toledo; Michael Chmielewski of the Department of Psychology of Southern Methodist University; Jason M. Chin of the University of Toronto Faculty of Law; Ilona M. McNeill of the University of Melbourne (Australia); Patrick D. Dunlop of the University of Western Australia; Delphine Courvoisier of Beau-Séjour Hospital, Geneva, Switzerland; Alex Sutherland of RAND Europe, Cambridge, United Kingdom; Barak Ariel of the Institute of Criminology of the University of Cambridge (United Kingdom); Lori A. Brotto of the Department of Gynaecology of the University of British Columbia; Morag Yule of the Department of Gynaecology of the University of British Columbia; Sivasankaran Balaratnasingam of the School of Psychiatry and Clinical Neurosciences of the University of Western Australia; Zaza Lyons of the School of Psychiatry and Clinical Neurosciences of the University of Western Australia; Aleksander Janca of the School of Psychiatry and Clinical Neurosciences of the University of Western Australia; Yuanbo Gu of the School of Psychology of Shaanxi Normal University (China); Ning He of the School of Psychology of Shaanxi Normal University (China); Xuqun You of the School of Psychology of Shaanxi Normal University (China); xxvi Preface Chengting Ju of the School of Psychology of Shaanxi Normal University (China); Rick Malone of the U.S. Army Criminal Investigation Command, Quantico, VA; Winnie Eckardt of The Dian Fossey Gorilla Fund International, Atlanta, GA; Alexander Weiss of the Department of Psychology of the University of Edinburgh (UK); Monica Webb Hooper of the Case Comprehensive Cancer Center at Case Western Reserve University; Carla Sharp of the Department of Psychology at the University of Houston (TX); Liliana B. Sousa of the Faculty of Psychology and Educational Sciences of the University of Coimbra (Portugal); Duncan M. Shields of the Faculty of Medicine of the University of British Columbia; Eric Kramer of Medical Specialists of the Palm Beaches, (Neurology), Atlantis, Florida; Jed Yalof of the Department of Graduate Psychology of Immaculata University; Mark F. Lenzenweger of the Department of Psychology of the State University of New York at Binghamton; Jessica Klein of the Department of Psychology of the University of Florida (Gainesville); Anna Taylor of the Department of Psychology of Illinois State University; Suzanne Swagerman of the Department of Biological Psychology of Vrije Universiteit (VU), Amsterdam, The Netherlands; Eco J.C. de Geus of the Department of Biological Psychology of Vrije Universiteit (VU), Amsterdam, The Netherlands; Kees-Jan Kan of the Department of Biological Psychology of Vrije Universiteit (VU), Amsterdam, The Netherlands; Dorret I. Boomsma of the Department of Biological Psychology of Vrije Universiteit (VU), Amsterdam, The Netherlands; Faith Miller of the Department of Educational Psychology of the University of Minnesota; and, Daniel Teichman formerly of the Department of Computer and Information Science and Engineering of the University of Florida (Gainesville). For their enduring contribution to this and previous editions of this book, we thank Dr. Jennifer Kisamore for her work on the original version of our chapter on test utility, and Dr. Bryce Reeve who not only wrote a Meet an Assessment Professional essay, but another informative essay on item response theory (see OOBAL-5-B2). Thanks to the many assessment professionals who, whether in a past or the current edition, took the time to introduce students to what they do. For being a potential source of inspiration to the students who they “met” in these pages, we thank the following assessment professionals: Dr. Rebecca Anderson, Dr. Howard W. Atlas, Dr. Scott Birkeland, Dr. Anthony Bram, Dr. Stephen Finn, Dr. Chris Gee, Dr. Joel Goldberg, Ms. Eliane Hack, Dr. Steve Julius, Dr. Nathaniel V. Mohatt, Dr. Barbara C. Pavlo, Dr. Jeanne P. Ryan, Dr. Adam Shoemaker, Dr. Benoit Verdon, Dr. Erik Viirre, and Dr. Eric A. Zillmer. Thanks also to Dr. John Garruto for his informative contribution to Chapter 10. Special thanks to two student assistants whose help was invaluable to Ronald Jay Cohen in the preparation of this book and its supplementary package of OOBALs. Jessica Klein of the University of Florida (Gainesville) and Anna Taylor of Illinois State University helped gather publications that were needed for updating this ninth edition. They both also helped in identifying OOBALs that would be of particular interest and value to fellow students. As if that were not enough, Jessica and Anna each also authored OOBALs designed to welcome fellow students to a course in measurement in psychology (see OOBAL-1-B1 and OOBAL-1-B2). Preface xxvii While thanking all who contributed in many varied ways, including Susan Cohen who assisted with keyboarding, we remind readers that the present authorship team takes sole responsibility for any possible errors that may have somehow found their way into this ninth edition. Meet the Authors Ronald Jay Cohen, Ph.D., ABPP, ABAP, is a Diplomate of the American Board of Professional Psychology in Clinical Psychology, and a Diplomate of the American Board of Assessment Psychology. He is licensed to practice psychology in New York and Florida, and a “scientist-practitioner” and “scholar-professional” in the finest traditions of each of those terms. During a long and gratifying professional career in which he has published numerous journal articles and books, Dr. Cohen has had the privilege of personally working alongside some of the luminaries in the field of psychological assessment, including David Wechsler (while Cohen was a clinical psychology intern at Bellevue Psychiatric Hospital in New York City) and Doug Bray (while working as an assessor for AT&T in its Management Progress Study). After serving his clinical psychology internship at Bellevue, Dr. Cohen was appointed Senior Psychologist there, and his clinical duties entailed not only psychological assessment but the supervision and training of others in this enterprise. Subsequently, as an independent practitioner in the New York City area, Dr. Cohen taught various courses at local universities on an adjunct basis, including undergraduate and graduate courses in psychological assessment. Asked by a colleague to conduct a qualitative research study for an advertising agency, Dr. Cohen would quickly become a sought-after qualitative research consultant with a client list of major companies and organizations—among them Paramount Pictures, Columbia Pictures, NBC Television, the Campbell Soup Company, Educational Testing Service, and the College Board. Dr. Cohen’s approach to qualitative research, referred to by him as dimensional qualitative research, has been emulated and written about by qualitative researchers around the world. Dr. Cohen is a sought-after speaker and has delivered invited addresses at the Sorbonne in Paris, Peking University in Beijing, and numerous other universities throughout the world. It was Dr. Cohen’s work in the area of qualitative assessment that led him to found the scholarly journal Psychology & Marketing. Since the publication of the journal’s first issue in 1984, Dr. Cohen has served as its Editor-in-Chief. Mark E. Swerdlik, Ph.D., ABPP, is Professor of Psychology at Illinois State University, where he has taught the undergraduate psychological measurement course, conducted professional seminars addressing legal/ethical issues in assessment, and supervised practicum students in assessment. He has served as an editorial board member of several journals, written test reviews for several journals, reviewed test-scoring software for a major test publisher, and served as a reviewer for the Mental Measurements Yearbook. In various professional capacities, he has participated in the standardization of many psychological tests, including, for example, the WISC-R, the WISC-III, the Kaufman Assessment Battery for Children (K-ABC), the Stanford-Binet IV, the Peabody Picture Vocabulary Test (PPVT), the Kaufman Test of Educational Achievement, the Vineland Adaptive Behavior Scale, the Psychological Processing Checklist (PPC), and the Psychological Processing Checklist-Revised (PPC-R). As a licensed clinical psychologist, a nationally certified school psychologist, independent practitioner, and consultant, Dr. Swerdlik administers and interprets psychological tests, and conducts seminars to train fellow professionals in proper test administration, scoring, and interpretation procedures. He has also served as a program evaluator for many programs, a partial listing of which would include the Heart of Illinois Low Incidence Association (HILA), the Autism/Pervasive Developmental Delays Training and Technical Assistance Project, and the Illinois National xxviii Preface Guard Statewide Reintegration Program for Combat Veterans (for veterans who served in Iraq and Afghanistan, from 2006 to the present). And on a Personal Note... I think back to the time when we were just wrapping up work on the sixth edition of this book. At that time, I received the unexpected and most painful news that my mother had suffered a massive and fatal stroke. It is impossible to express the sense of sadness and loss experienced by myself, my brother, and my sister, as well as the countless other people who knew this gentle, loving, and much-loved person. To this day, we continue to miss her counsel, her sense of humor, and just knowing that she’s there for us. We continue to miss her genuine exhilaration, which in turn exhilarated us, and the image of her welcoming, outstretched arms whenever we came to visit. Her children were her life, and the memory of her smiling face, making each of us feel so special, survives as a private source of peace and comfort for us all. She always kept a copy of this book proudly displayed on her coffee table, and I am very sorry that a copy of more recent editions did not make it to that most special place. My dedication of this book is one small way I can meaningfully acknowledge her contribution, as well as that of my beloved, deceased father, to my personal growth. As in the sixth edition, I am using my parents’ wedding photo in the dedication. They were so good together in life. And so there Mom is, reunited with Dad. Now, that is something that would make her very happy. As the reader might imagine, given the depth and breadth of the material covered in this textbook, it requires great diligence and effort to create and periodically re-create an instructional tool such as this that is timely, informative, and readable. Thank you, again, to all of the people who have helped through the years. Of course, I could not do it myself were it not for the fact that even through nine editions, this truly Herculean undertaking remains a labor of love. Ronald Jay Cohen, Ph.D., ABPP, ABAP Diplomate, American Board of Professional Psychology (Clinical) Diplomate, American Board of Assessment Psychology Preface xxix 1 C H A P T E R Psychological Testing and Assessment A ll fields of human endeavor use measurement in some form, and each field has its own set of measuring tools and measuring units. For example, if you’re recently engaged or thinking about becoming engaged, you may have learned about a unit of measure called the carat. If you’ve been shopping for a computer, you may have learned something about a unit of measurement called a byte. As a student of psychological measurement, you need a working familiarity with some of the commonly used units of measure in psychology as well as knowledge of some of the many measuring tools employed. In the pages that follow, you will gain that knowledge as well as an acquaintance with the history of measurement in psychology and an understanding of its theoretical basis. Testing and Assessment The roots of contemporary psychological testing and assessment can be found in early twentieth- century France. In 1905, Alfred Binet and a colleague published a test designed to help place Paris schoolchildren in appropriate classes. Binet’s test would have consequences well beyond the Paris school district. Within a decade an English-language version of Binet’s test was prepared for use in schools in the United States. When the United States declared war on Germany and entered World War I in 1917, the military needed a way to screen large numbers of recruits quickly for intellectual and emotional problems. Psychological testing provided this methodology. During World War II, the military would depend even more on psychological tests to screen recruits for service. Following the war, more and more tests purporting to measure an ever-widening array of psychological variables were developed and used. There were tests to measure not only intelligence but also personality, brain functioning, performance at work, and many other aspects of psychological and social functioning. Psychological Testing and Assessment Defined The world’s receptivity to Binet’s test in the early twentieth century spawned not only more tests but more test developers, more test publishers, more test users, and the emergence of what, logically enough, has become known as a testing enterprise. “Testing” was the term used to refer to everything from the administration of a test (as in “Testing in progress”) to the interpretation of a test score (“The testing indicated that...”). During World War I, the term “testing” aptly described the group screening of thousands of military recruits. We suspect that it was then that the term gained a powerful foothold in the vocabulary of professionals and laypeople. The use of “testing” to denote everything from test administration to test interpretation can be found in postwar textbooks (such as Chapman, 1921; Hull, 1922; Spearman, 1927) as well as in various test-related writings for decades thereafter. However, by World War II a semantic distinction between testing and a more inclusive term, “assessment,” began to emerge. 1 During World War II the U.S. Office of Strategic Services (OSS), a predecessor to today’s Central Intelligence Agency (CIA), used a variety of procedures and measurement tools— psychological tests among them—in selecting military personnel for highly specialized positions involving espionage, intelligence gathering, and the like. For example, one of the tools employed was a very uncomfortable, group-on-one interview technique to evaluate how well candidates might respond to Gestapo-like interrogation. With a light harshly pointed at their face, interviewees would have to draw on their own creativity, persuasive abilities, and other resources to satisfactorily defend and explain a given scenario to a group of increasingly hostile interviewers. Candidates might have to explain why they were in a particular building that was off-limits to them, and doing something that they were not authorized to do, such as looking at or removing classified files. Candidates were evaluated on a number of variables, such as their ability to maintain noncontradictory responses. Today, such an assessment method, or any assessment method that has potential for harming the persons being assessed, would be likely to raise serious ethical concerns. As summarized in Assessment of Men (OSS Assessment Staff, 1948) and elsewhere (Murray & MacKinnon, 1946), the assessment data generated were subjected to thoughtful integration and evaluation by highly trained assessment center staff. The OSS model—using an innovative variety of evaluative tools along with data from the evaluations of highly trained assessors—would later inspire what is now referred to as the assessment center approach to personnel evaluation (Bray, 1982). Military, clinical, educational, and business settings are but a few of the many contexts that entail behavioral observation and active integration by assessors of test scores and other data. In such situations, the term assessment may be preferable to testing. In contrast to testing, assessment acknowledges that tests are only one type of tool used by professional assessors (along with other tools, such as the interview), and that the value of a test, or of any other tool of assessment, is intimately linked to the knowledge, skill, and experience of the assessor. The semantic distinction between psychological testing and psychological assessment is blurred in everyday conversation. Somewhat surprisingly, the distinction between the two terms still remains blurred in some published “psychological testing” textbooks. Yet the distinction is important. Society at large is best served by a clear definition of and J U ST THI N K... differentiation between these two terms as well as related terms such as psychological test user and psychological assessor. Clear Describe a situation in which testing is more distinctions between such terms may also help avoid the turf wars appropriate than assessment. By contrast, now brewing between psychology professionals and members of describe a situation in which assessment is other professions seeking to use various psychological tests. In many more appropriate than testing. psychological evaluation contexts, conducting an assessment requires greater education, training, and skill than simply administering a test. We define psychological assessment as the gathering and integration of psychology-related data for the purpose of making a psychological evaluation that is accomplished through the use of tools such as tests, interviews, case studies, behavioral observation, and specially designed apparatuses and measurement procedures. We define psychological testing as the process of measuring psychology-related variables by means of devices or procedures designed to obtain a sample of behavior. Some of the differences between these two processes are presented in Table 1–1.1 1. Especially when discussing general principles related to the creation of measurement procedures, as well as the creation, manipulation, or interpretation of data generated from such procedures, the word test (as well as related terms, such as test score) may be used in the broadest and most generic sense; that is, “test” may be used in shorthand fashion to apply to almost any procedure that entails measurement (including, for example, situational performance measures). Accordingly, when we speak of “test development” in Chapter 8, many of the principles set forth will apply to the development of other measurements that are not, strictly speaking, “tests” (such as situational performance measures, as well as other tools of assessment). Having said that, let’s reemphasize that a real and meaningful distinction exists between the terms psychological testing and psychological assessment, and that effort should continually be made not to confuse the meaning of these two terms. 2 Part 1: An Overview Table 1–1 Testing in Contrast to Assessment In contrast to the process of administering, scoring, and interpreting psychological tests (psychological testing), psychological assessment may be conceived as a problem-solving process that can take many different forms. How psychological assessment proceeds depends on many factors, not the least of which is the reason for assessing. Different tools of evaluation—psychological tests among them— might be marshaled in the process of assessment, depending on the particular objectives, people, and circumstances involved as well as on other variables unique to the particular situation. Admittedly, the line between what constitutes testing and what constitutes assessment is not always as clear as we might like it to be. However, by acknowledging that such ambiguity exists, we can work to sharpen our definition and use of these terms. It seems useful to distinguish the differences between testing and assessment in terms of the objective, process, and outcome of an evaluation and also in terms of the role and skill of the evaluator. Keep in mind that, although these are useful distinctions to consider, exceptions can always be found. Testing Assessment Objective Typically, to obtain some gauge, usually numerical in Typically, to answer a referral question, solve a problem, or nature, with regard to an ability or attribute. arrive at a decision through the use of tools of evaluation. Process Testing may be individual or group in nature. After test Assessment is typically individualized. In contrast to testing, administration, the tester will typically add up “the assessment more typically focuses on how an individual number of correct answers or the number of certain processes rather than simply the results of that processing. types of responses... with little if any regard for the how or mechanics of such content” (Maloney & Ward, 1976, p. 39). Role of Evaluator The tester is not key to the process; practically speaking, The assessor is key to the process of selecting tests and/or one tester may be substituted for another tester without other tools of evaluation as well as in drawing conclusions appreciably affecting the evaluation. from the entire evaluation. Skill of Evaluator Testing typically requires technician-like skills in terms Assessment typically requires an educated selection of tools of administering and scoring a test as well as in of evaluation, skill in evaluation, and thoughtful interpreting a test result. organization and integration of data. Outcome Typically, testing yields a test score or series of test Typically, assessment entails a logical problem-solving scores. approach that brings to bear many sources of data designed to shed light on a referral question. Varieties of assessment The term assessment may be modified in a seemingly endless number of ways, each such modification referring to a particular variety or area of assessment. Sometimes the meaning of the specialty area can be readily discerned just from the word or term that modifies “assessment.” For example, the term “therapeutic psychological assessment” refers to assessment that has a therapeutic component to it. Also intuitively obvious, the term educational assessment refers to, broadly speaking, the use of tests and other tools to evaluate abilities and skills relevant to success or failure in a school or pre-school context. Intelligence Chapter 1: Psychological Testing and Assessment 3 tests, achievement tests, and reading comprehension tests are some of the evaluative tools that may spring to mind with the mention of the term “educational assessment.” But what springs to mind with the mention of other, less common assessment terminology? Consider, for example, terms like retrospective assessment, remote assessment, and ecological momentary assessment. For the record, the term retrospective assessment may be defined as the use of evaluative tools to draw conclusions about psychological aspects of a person as they existed at some point in time prior to the assessment. There are unique challenges and hurdles to be overcome when conducting retrospective assessments regardless if the subject of the evaluation is deceased (Reyman et al., 2015) or alive (Teel et al., 2016). Remote assessment refers to the use of tools of psychological evaluation to gather data and draw conclusions about a subject who is not in physical proximity to the person or people conducting the evaluation. One example of how psychological assessments may be conducted remotely was provided in this chapter’s Close-Up feature. In each chapter of this book, we will spotlight one topic for “a closer look.” In this chapter, the Close-Up box explored how the smartphone revolution in communication may also signal a revolution in the way that psychological assessments are conducted. Psychological assessment by means of smartphones also serves as an example of an approach to assessment called ecological momentary assessment (EMA). EMA refers to the “in the moment” evaluation of specific problems and related cognitive and behavioral variables at the very time and place tha

Psychological Testing and Assessment (9th Ed) PDF

Document Details

Tags

Related

Summary

Full Transcript

Upgrade to continue