Psychometrics 1 Introduction + History.pdf

PSYCHOMETRICS 1 BES3149 1st Semester AY 2024-2025 PROCESS OF ASSESSMENT INTRO: THE ASSESSMENT PROCESS Begins with REFERRAL FOR...

PSYCHOMETRICS 1 BES3149 1st Semester AY 2024-2025 PROCESS OF ASSESSMENT INTRO: THE ASSESSMENT PROCESS Begins with REFERRAL FOR ASSESSMENT from teacher, school psychologist, counselor, TOPIC OVERVIEW judge, clinician, or corporate HR I. The Process of Assessment specialist A. Tools of Psychological Referral questions: Does a child Assessment have a reading disability? Does a B. Parties in the Assessment young adult have schizophrenia? Enterprise Can the child be eligible for C. Assessment Settings certain services or D. Obligations of the Assessment accommodations? Is the Professional defendant competent to stand E. Uses of Psychological Tests F. Assessment vs. Testing trial? How well can this employee G. Psychometrics Defined be expected to perform if II. Tests promoted to a managerial A. What a test measures position? B. Psychological Tests The assessor selects tools of C. Types of Tests assessment. Consider here the D. Essential Test Elements assessor’s competence in test use E. Control and Use of Psychological (i.e., levels). Research can also Tests F. Test Administration guide the test selection process. G. Examiner & Situational Variables LEVELS OF TEST A School-made Test I. PROCESS OF ASSESSMENT Group administered B (E.g. USTET; done by RPm) WHAT CAN BE ASSESSED? Individually administered C (One on one, Intelligence, Personality, Psychological Qualities Rorschach; Done by RPsy) ○ Intelligence Formal assessment begins. ○ Aptitude After assessment, the assessor ○ Achievement writes a report of the findings that ○ Personality is designed to answer the referral ○ Interests question. Physical Qualities ○ Height I. A. TOOLS OF PSYCH ASSESSMENT ○ Age ○ Temperature ○ Body Language The Test Test Content (i.e., subject matter ASSESSMENT of the test) The evaluation or estimation of Test Format (i.e., form, plan, the nature, quality, or ability of structure, arrangement, layout of someone or something test items, and manner of test A guide to decision making administration – computerized, e.g. the assessment of economic paper-and-pencil) needs Test Administration Procedures (individual versus group tests) 1 BES3149 Psychometrics 1 — 1st Semester AY 2024-2025  Psychometric soundness (or - Must be QUALIFIED test users, technical quality) of tests: how and also PERMITTED TO consistently and accurately a PURCHASE psychological test measures. 3. The Test Taker - Issues to Reliability and validity of a test. consider: Psychometrics: the science of a. Test Anxiety psychological measurement b. Understanding and Psychometric test utility: agreement with the rationale usefulness or practical value that of the assessment a test or other tool of assessment c. Capacity and willingness to has for a particular purpose cooperate with the examiner The Interview: gathering d. Physical pain and emotional information through direct distress experienced; communication involving physical discomfort; reciprocal exchange. alertness, wakefulness ○ Methods: face-to-face, during assessment telephone, online, e-mail, text, e. Test acquiescence; social sign language desirability issues; ○ Types: panel interview, f. Prior coaching received motivational interviewing g. “psychological autopsy” 4. Society at large - creates needs Notes for new variables to measure Subjective 5. Other parties BEI - Behavior Event Interviewing STAR - Situation, Tasks, I. C. ASSESSMENT SETTINGS Activities, Results The Portfolio: samples of one’s 1. Educational Settings ability and accomplishment Achievement tests – Case History Data: records, measure amount of transcripts, and other accounts in learning written, pictorial, or other form Diagnostic tests – narrow that preserve archival information, down areas of deficit official and informal accounts, and Most frequent user of tests other data and items relevant to 2. Clinical Settings an assessee Hospitals, in-patient and Behavioral Observation: out-patient clinics, private monitoring the actions of others practice consulting rooms, or oneself by visual or electronic schools, other institutions means while recording Intelligence tests, quantitative and/or qualitative personality tests, information regarding those neuropsychological tests, actions and other specialized instruments PARTIES IN THE ASSESSMENT Mostly individual I. B. ENTERPRISE assessment, with group testing usually for screening 1. The Test Developer and (i.e., determining individuals Publisher - the creators and who need further distributors of tests and other diagnostic evaluation methods of assessment 3. Counseling Settings 2. The Test User - clinicians, schools, prisons, counselors, school psychologists, government and private human resources personnel, institutions consumer psychologists, measures of social and experimental psychologists, social academic skills, psychologists, etc. 2 BES3149 Psychometrics 1 — 1st Semester AY 2024-2025  personality, interest, compromise any rules of the test attitudes, values administration process. 4. Geriatric Settings Safeguarding test protocols. housing designed for Conveying results in a clearly assisted living understandable fashion. long-term care facilities Report presence of third parties (hospitals, “home for the during testing, or anything out of aged”) the ordinary that happened assessment of quality of during testing. life (whether self-report or Scoring and interpretation must observed) conform to established assessment of cognitive procedures and ethical guidelines. decline (Dementia, Alzheimer’s Disease, I. E. USES OF PSYCHOLOGICAL TESTS Pseudodementia) 5. Business and Military Settings Earliest use of tests: To identify Decision making about mentally retarded, insane, and careers of personnel psychotic persons Achievement, aptitude, Assessment needs in education interest, motivational tests ○ Intelligence testing (affecting decision to hire, ○ Classification of students as to promote, or transfer) their ability to profit from 6. Governmental and instruction organizational credentialing ○ Identification of fast or slow Licensures, certification, learners membership in ○ Educational and occupational organizations counseling 7. Academic Research Settings Selection and classification of Measuring variables being industrial personnel explored by the researcher Individual counseling Research and data gathering OBLIGATIONS OF THE WHAT IS A TEST? I. D. ASSESSMENT PROFESSIONAL A test is a measurement device or technique used to quantify Select and use tests that are most behavior or aid in the appropriate for the individuals understanding and prediction of being tested. behavior. (Kaplan and Saccuzzo, The test should be stored in a way 2018) that reasonably ensures that its A psychological test is a set of contents will not be known to the items that are designed to test taker in advance. measure characteristics of human Ensure that a prepared and beings that pertain to behavior. suitably trained person (Kaplan and Saccuzzo, 2018) administers the test properly. A psychological test is an Examiners must be familiar with objective and standardized test materials and procedures; all measure of a sample of behavior materials (stop watch, pencils, (Anastasi and Urbina, 1997) protocols – form/sheet/booklet where test taker’s responses are I. F ASSESSMENT VS. TESTING entered) must be ready to properly administer the test. Psychological Assessment is the Ensure a conducive testing area. gathering and integration of Rapport is important, especially in psychology-related data for the one-on-one or small group purpose of making a testing. Rapport must not psychological evaluation that is accomplished through the use of 3 BES3149 Psychometrics 1 — 1st Semester AY 2024-2025  tools such as tests, interviews, 16PF - one of the samples of case studies, behavioral psychological tests and observation, and specially measures of personality designed apparatuses and measurement procedures. SIMILARITIES AND DIFFERENCES AMONG Notes PSYCHOLOGICAL TESTS Using more than one source of information SIMILARITIES DIFFERENCES Broader All psychological All psychological Psychological Testing is the tests require an tests require an process of measuring individual to perform individual to perform psychology-related variables by a behavior a behavior. means of devices or procedures The behavior The attribute they designed to obtain a sample of performed is used to measure behavior. (Cohen and Swerdlik, measure some personal attribute, 2018) trait, or characteristic. Notes Test the test taker This personal Their content Administer, score, interpret test attribute, trait, or Testing is part of assessment characteristic is thought to be important in I. G. describing or PSYCHOMETRICS DEFINED understanding behavior. Psychometrics is the science of psychological measurement The behavior How they are performed may also administered and Psychometric soundness (of a be formatted test) refers to how consistently used to predict and how accurately a test outcomes. measures what it purports to How they are scored measure. and interpreted ○ Reliability and Validity; ○ Most common psychometric Their psychometric quality quality II. TESTS Notes Behavior = encircling, drawing, shade, inkblot, etc. II. A. WHAT A TEST MEASURES Attribute = intelligence, attitude, personality, etc.... a sample of behavior... Just a PORTION, for instance, a child’s vocabulary, a clerk’s ability to TEST ITEM DEFINED perform arithmetic computations, A specific stimulus to which a or a pilot’s eye-hand coordination person responds overtly; this For example, give a test to response can be scored or measure the extent of a child’s evaluated vocabulary with only five (5) items, Specific questions that make up or 20 items consisting of purely the test biological terms Items may be an inkblot, drawing, Would it give an accurate question, etc. estimate of the child’s overall vocabulary? Most likely not. Notes More items = better 4 BES3149 Psychometrics 1 — 1st Semester AY 2024-2025 SAMPLE TEST ITEM An individual’s behavior, and therefore test scores, will remain unchanged over time. Individuals understand test items similarly. Individuals can report about themselves accurately. Individuals will report their thoughts and feelings honestly. The test score an individual receives is equal to his or her true ability plus some error. Notes Abstract Reasoning, measures Notes intelligence ASSUMPTIONS Nonverbal Test 1. Assume all tests are valid Culture fair, not bias 2. Assume all tests are reliable 3. Test maker only knows the true meaning of the item. MEASURED BEHAVIOR Conduct pilot testing Overt : observable activity 4. Assume test takers know Covert: takes place within the themselves individual and cannot be directly 5. Cases of faking good, faking observed (feelings, thoughts) bad self-report test, provide an 6. Test scores do not actually reflect the individuals’ true answer ability II. B. PSYCHOLOGICAL TESTS II. C. TYPES OF TESTS THREE DEFINING CHARACTERISTICS OF PSYCHOLOGICAL TESTS Individual Tests – given to one 1. A good test should person at a time (i.e. drawing) representatively sample the Group Tests – can be behaviors thought to measure an administered to more than one attribute or predict an outcome. person at a time by a single This ensures that the test examiner (used in school and measures what it says it industries) measures. TYPES OF TEST ACCORDING TO TYPE OF 2. The behavior samples should be BEHAVIOR MEASURED obtained under standardized ABILITY TEST - contains items conditions. That is, a test must be that can be scored in terms of administered exactly the same speed, accuracy, or both way to all individuals so that we Includes the following: can be confident that a person’s ○ Achievement: previous score accurately reflects the learning attribute being measured or the ○ Aptitude: potential for outcome being predicted. learning or acquiring a 3. There must be rules for scoring so specific skill that all examiners will score the ○ Intelligence: general potential test in the same way. to solve problems, adapt to changing circumstances, ASSUMPTIONS OF PSYCHOLOGICAL think abstractly, and profit TESTS from experience Psychological tests measure what PERSONALITY TEST - related to they say they measure, and any the overt and covert dispositions inferences that are drawn about of an individual test takers based on their test ○ May be self-report or scores are appropriate. projective 5 BES3149 Psychometrics 1 — 1st Semester AY 2024-2025  Notes CONTROL AND USE OF II. E. Ability test - objective is to get PSYCHOLOGICAL TESTS the highest possible score Projective - ambiguous Two reasons stimulus (inkblot, creating a ○ To ensure that the test is story) given by a qualified examiner Interest test - measures career and that the scores are orientations properly used; ○ To prevent general familiarity II. D. ESSENTIAL TEST ELEMENTS with the test content, which would invalidate the test Standardization implies Qualified Examiner is needed for uniformity of procedures in the three major aspects of the administering and scoring the testing situation: test. ○ Selection of the test; Consider ○ Directions technical aspects like validity, ○ Materials used reliability, difficulty level, and ○ Time limits norms ○ Oral instructions ○ Administration and scoring of ○ Preliminary demonstrations the test; and ○ Ways of handling questions ○ Interpretation of the test. from test takers SECURITY OF TEST CONTENT Establishment of NORMS imply Test content has to be restricted average or normal performance in order to prevent deliberate ○ Psychological tests have no efforts to fake scores predetermined standards of COMMUNICATION OF TEST passing or failing. An INFORMATION individual’s test score is Purposes of Test interpreted by comparing it Communication with the scores obtained by ○ Helps dispel any mystery others on the same test. associated with testing; Objective Measurement of ○ Concern with technical Difficulty procedures – reliability, ○ The administration, scoring, validity, other test properties; and interpretation of scores and are independent of the ○ Familiarize test takers with subjective judgment of the testing procedures, dispel individual examiner. anxiety, ensure good ○ Difficulty level of the test/test performance item is determined based on objective, empirical II. F. TEST ADMINISTRATION procedures ○ e.g., arrangement of order of A test score helps us to predict test items, as well as selection how the client will feel and act of items for inclusion in a test outside the test situation. Reliability is the consistency of ○ Achievement in college; scores obtained by the same ○ Job performance persons when retested with the ○ Etc. identical test or with any Advance Preparation of Examiners equivalent form of the test. ○ Memorize exact verbal Validity is the degree to which instructions the test measures what it ○ Prepare test materials purports to measure. ○ Familiarity with testing procedure, time limits Testing Conditions ○ Testing room (suitable) 6 BES3149 Psychometrics 1 — 1st Semester AY 2024-2025  Introducing the Test ○ Rapport refers to the examiner’s efforts to arouse the test-taker’s interest in the test, elicit their cooperation, and encourage them to respond in a manner that is appropriate to the test’s objectives. EXAMINER & SITUATIONAL II. G. VARIABLES Effects of certain extraneous factors ○ Age ○ Sex ○ Ethnicity ○ Professional/socio-econo mic status ○ Training and experience ○ Personality characteristics ○ Appearance Self-fulfilling prophecy (i.e., Rorschach case, open ended questions) Test-takers’ activity prior to the test (i.e., physical strenuous activity) The distinction between COACHING and EDUCATION A test score is invalidated only when a particular experience raises the score without appreciably affecting the behavior domain that the test is designed to measure. Test sophistication – effect of test taking practice Notes Coaching - review classes for board exams or CETs 7 BES3149 Psychometrics 1 — 1st Semester AY 2024-2025  treatment of mentally retarded HISTORY OF PSYCHOLOGICAL TESTING and insane persons It became necessary to TOPIC OVERVIEW distinguish between mentally retarded and insane persons I. Antiquity to the 19th century 1838: ESQUIROL II. Binet and the Rise of Intelligence French physician whose Scales III. Standardized Achievement Tests two-volume work made the first IV. Assessment of Personality explicit distinction between A. Projective Tests mentally retarded and insane B. New Approaches in Personality Testing individuals V. Post World War II More than 100 pages of his work VI. The Current Environment devoted to “mental retardation” A. The Philippine Scene B. Assessment During the Pointed out that there are many Pandemic degrees of mental retardation The individual’s use of language provides the most dependable ANTIQUITY TO THE 19TH criterion of his intellectual level I. CENTURY SEGUIN Another French physician Tests and testing programs first Pioneered in the training of came into being in China as early mentally retarded persons as 2200 B.C. (Dubois, 1966, 1970, in (particularly, children) Cohen and Swerdlik, 2018). 1837: established the first school Testing was instituted as a means devoted to the education of for selecting who would obtain mentally retarded children government jobs. 1848: migrated to the USA, made One’s position in society was suggestions regarding the largely determined by the family training of mentally retarded into which he was born, the fact persons that one could improve his lot by Some of the procedures scoring high in exams was a developed by Seguin were significant step forward. eventually incorporated into Ancient Greco-Roman writings performance or nonverbal tests of categorize people in terms of intelligence personality types (i.e., reference to ON THE ORIGIN OF SPECIES BY abundance or deficiency in some CHARLES DARWIN (1859) bodily fluid such as blood or Argued that chance variation in phlegm) species would be selected or rejected by nature according to Notes Blood - Sanguine - Happy adaptivity and survival value. Phlegm - Phlegmatic - Slow Darwin spurred interest in action individual differences. According Black bile - Melancholic - Sadness to him, individual differences are Yellow bile - Choleric - Easily of the highest importance, for angered they afford materials for natural 19th Century: strong awakening selection to act on. of interest in the humane FRANCIS GALTON English biologist; Darwin’s half cousin 8 BES3149 Psychometrics 1 — 1st Semester AY 2024-2025  Galton (1869) aspired to classify concerned largely with sensitivity people “according to their natural to visual, auditory, and other gifts” (p. 1) and to ascertain their sensory phenomena. This was “deviation from the average” (p. 11) reflected in the nature of the first His initial work on heredity was psychological tests. done with sweet peas, in part Emphasis on the need for because there tended to be fewer rigorous control of the conditions variations among the peas in a under which observations were single pod made Realized the need for measuring ○ Example: wording of the characteristics of related and directions, manuals for the unrelated persons administration of many tests Focused on INDIVIDUAL provide explicit instructions DIFFERENCES designed to hold constant or Galton was instrumental in “standardize” the conditions inducing a number of educational under which the test is institutions to keep systematic administered ANTHROPOMETRIC (physical JAMES MCKEEN CATTELL human abilities) RECORDS of American psychologist, student of their students Wilhelm Wundt 1884: Galton set up an In 1888, while lecturing at anthropometric laboratory at the Cambridge, regarded Francis International Exposition, where Galton, as “the greatest man I visitors could be measured on have known” (Roback, 1961; in certain variables such as height Cohen and Swerdlik, 2009); (standing), height (sitting), arm Galton stimulated his interest in span, weight, breathing capacity, the measurement of individual keenness of vision and hearing, differences strength of pull, strength of Boring (1950, p. 283): “Cattell, more squeeze, swiftness of blow, than any other person was, in this memory of form, discrimination of fashion, responsible for getting color, hand steadiness, reaction mental testing underway in time, and other simple America, and it is plain that his sensorimotor functions. motivation was similar to Galton’s, Primarily responsible for the and that he was influenced, or at launching of the testing least reinforced, by Galton.” movement He became active in the spread of Pioneered in the application of the testing movement; first to use rating scale and questionnaire the term MENTAL TEST (1890) methods (including self-report Instrumental in founding the inventories) PSYCHOLOGICAL CORPORATION, Responsible for the development which named 20 of the country’s of statistical methods for the leading psychologists as its analysis of data on individual directors differences (i.e., coefficient of The goal of the corporation was correlation) the “advancement of psychology WILHELM WUNDT and the promotion of the useful Wundt (1879): The problems applications of psychology” studied in their laboratories were 9 BES3149 Psychometrics 1 — 1st Semester AY 2024-2025 HERMAN EBBINGHAUS (1897) Made use of a standardization German psychologist sample of 50 children Administered tests of arithmetic Scale consisted of 30 problems or computation, memory span, and tests arranged in ascending order sentence completion to of difficulty, which measured schoolchildren JUDGMENT, COMPREHENSION, Only sentence completion, the and REASONING, seen by Binet as most complex of the three tests, essential components of showed a clear correspondence intelligence with the children’s scholastic The scale was designed to help achievement identify mentally retarded Paris ALFRED BINET schoolchildren French psychologist 1908 SCALE Urged that children who failed to Nearly twice as many items as the respond to normal schooling be 1905 Scale examined before dismissal, and if Some unsatisfactory tests in the considered educable, be assigned 1905 Scale were eliminated to special classes All tests were grouped into age His advocacy for the cause of levels mentally retarded children led to Score obtained corresponded to the establishment (in France) of a the child’s mental level (mental ministerial commission for the age) study of retarded children, to 1911 SCALE which he was appointed – Third revision, coincided with momentous event in the history Binet’s untimely death of psychological testing No fundamental changes, more 1895 tests added at several year levels, Alfred Binet and Victor Henri extended to the adult level criticized most of the available As a testing instrument, the tests as being too largely sensory BINET-SIMON Scale was and as concentrating unduly on outdistanced by the more simple, specialized abilities extensive and psychometrically An extensive and varied list of refined STANFORD-BINET Scale, tests was proposed, covering such developed by Lewis Terman and functions as memory, his associates at Stanford imagination, attention, University comprehension, and aesthetic Standardization sample was appreciation increased to 1000 Led to the development of the Terman was the first to use the famous BINET INTELLIGENCE term IQ SCALES Notes Lewis Terman - dept head of BINET AND THE RISE OF II. Psych in Stanford INTELLIGENCE SCALES - Handled Stanford-Binet Intelligence Scales 1905 SCALE (performance, test, In collaboration with Theodore individual) Simon KUHLMANN-BINET REVISION Also known as the Binet-Simon Extended the scale downward to Scale the age of three months (1912) 10 BES3149 Psychometrics 1 — 1st Semester AY 2024-2025  STANDARDIZED ACHIEVEMENT III. TESTS First standardized tests for measuring the outcomes of school instruction appeared in 1900; spearheaded by the work of E.L. Thorndike 1923 Stanford Achievement Test (Kelly, Rush, and Terman) 1930 phaseout of essay tests; introduction of test-scoring machines 1939 David Wechsler (clinical psychologist at Bellevue Hospital in New York City) Introduced a test designed to measure adult intelligence Defined intelligence as the aggregate or global capacity of the individual to think rationally, act purposefully, and deal effectively with his environment (1939, p. 3) Originally called the Wechsler- Bellevue Intelligence Scale, later renamed as the Wechsler Adult Intelligence Scale (WAIS) Standardization sample: 3,000 individuals RISE OF THE GROUP INTELLIGENCE TEST Response to an urgent need to screen military recruit 11 BES3149 Psychometrics 1 — 1st Semester AY 2024-2025