PSYC 513 Review Guide for Exam 1 Fall 2024 PDF

PSYC 513: Assessment in Rehabilitation Review Sheet for Exam 1 Assessment Overview  Assessment- “systematic method of obtaining information from tests and other sources, used to draw inferences about characteristics of people, objects or programs ...

PSYC 513: Assessment in Rehabilitation Review Sheet for Exam 1 Assessment Overview  Assessment- “systematic method of obtaining information from tests and other sources, used to draw inferences about characteristics of people, objects or programs  What is a test? How is assessment different from testing? Testing- systematic and often standardized process for sampling and describing a behavior of interest Assessment is more complex; may include tests plus other procedures (e.g.,observation, interviewing) Assessment is a process that attempts to meet a clearly defined objective Assessment usually results in a report with recommendations based on multiple sources Assessment requires specific knowledge(e.g., psychology, rehabilitation) as well as clinical judgment that goes beyondpsychometrics  How is assessment used in rehabilitation and counseling? Problem Orientation: Stimulate counselors and clients to consider how the problem is being viewed and recognized – A counselor uses an alcohol screening inventory to identify area of focus Problem Identification: Clarify nature of an issue (operationalize the problem) – Provide a diagnosis to classify a set of concerns Generate Alternatives: suggest alternative solutions – Use interest inventory to suggest other career choices; strengths Decision-Making/Planning: determine appropriate treatment for client – Use personality inventory to help select an intervention  Verification: evaluate effectiveness of solution – Client feedback to make changes to an intervention – Client self-monitoring exercise to assess maintenance of chang  What is vocational assessment? Career/Vocational Assessment: – career readiness, planning, maturity, career values, interests  Assessment vs. Testing Assessment: umbrella term (systematic procedures, multiple sources, inferences) – Evaluation, appraisal Tests: systematic and often standardize process for sampling and describing a behavior of interest Measurement: the assignment of numbers to attributes of persons according to rule  Purposes and Uses of Assessment Information To screen applicants for jobs, educational settings, and human service program eligibility To classify and place individuals in educational and business contexts For evaluation of work or academic performance (e.g., promotion, graduation) For personal counseling and guidance Diagnosis; determine baseline for treatment Evaluate effectiveness of interventions  Common Assessment Hazards (Anastasi 1992)  Hazard of the single score - use a score range with band of error  Hazard of the single time period - Be cautious with dated results or restricted time observations  Hazard of the single indicator - Use multiple methods to assess a construct (e.g., interest, anxiety)   Hazard of illusory precision -Be careful not to overweight numerical scores as more objective than observations because of quantification  Organizing Assessment Information The Assessment Report Identifying Information Reason for Referral Background Information Behavioral Observations Results and Interpretation Recommendations Summary Signature *Addendum (to clarify results shared back with client and their impressions  Informed Choice and Consumer Empowerment in Assessment Informed Choice- Individual sets goal, gathers information, considers range of options, takes responsibility for selecting the option that best meets his/her criteria Consumer Empowerment 1. Establish a collaborative relationship. 2. Understand and analyze the needs, expectations, values, goals of client. 3. Communicate info about the evaluation process. 4. Encourage active consumer involvement, consumer analyzes information about the process. 5. Consumer makes informed choices and expresses self-determination when selecting evaluation approaches. 6. The professional monitors consumer to ascertain that s/he is still actively participating in the identified, selected, and planned tasks of assessment  Formulating specific referral questions Possible questions Vocational/Educational Planning What type of employment is most suitable for this individual’s skill and ability levels? Does this job require additional training, and, if so, what type of training?  Work Behaviors How is this individual likely to respond to on-the-job stressors, such as time pressures? Is this individual able to apply consistent effort on work-related tasks? Work Readiness Is this individual capable of independently completing a job application and interviewing for employment? What type of employment supports, if any, are recommended? Self-Awareness/Self-Management Is this individual’s self-estimate of vocational skills, abilities, and limitations similar to the tested and demonstrated abilities? Functional Capacity/Stamina What is the individual’s stamina or endurance level as they relate to ability to engaging in fulltime employment? Part-time employment? What is the optimal work schedule for this individual? Work Environment How does this individual respond to supervision? What methods of supervision are optimal for this individual? How does this individual adapt and respond to changes in work routines and work environment? Ethical Considerations:  CRC Code of Ethics (Section H)  ACA Codes of Ethics (Section E) Basic Measurement Concepts & Principles  Raw scores vs. comparative information Raw scores are difficult to interpret in the absence of other information - Percent correct or number of items correct or endorsed provide no indication of the following: a. How other individuals scored b. What is an expected or cut-off score: or c. How this compares to previous performance of the same individual Comparative Information 1. Comparison to an internal frame of reference = self-referenced 2. Comparison with an absolute standard, external frame of reference = criterion-referenced interpretation (E.g. standards) 3. Comparison with scores obtained by other individuals, external frame of reference = norm-referenced interpretation (E. g. 89th percentile compared to other high school seniors)  Types of Score Interpretation: self-referenced, criterion-referenced, norm- referenced Self-Referenced (changes in time) Criterion-Referenced (focus on test taker, not comparison) o Focus on what a test taker can do in a specific domain, not comparison to others o Standards/values usually derived from an organization Norm-Referenced (comparison to others), o E.g., Developmental Norms  Types of scales: nominal, ordinal, interval, ratio Nominal - Classifies, assigns numerals but does not distinguish size, amount - E.g., any categorical variable, such as ethnicity, gender Ordinal -Indication of ordering; -no indication of distances between objects on the scale; - e.g., “Jamie ran the fastest race, Alison was second, and Pam was third Interval -equal intervals on scale - i.e., the difference between 80 and 90 is the same as the difference between 90 and 100 -No true 0 point -EX: temperature, IQ Ratio - Place objects in order, does so with equal intervals and has a true zero point -EX: Height  Normal curve/Normal frequency distribution A frequency distribution A bell-shaped curve Indicates relative standing within a comparison group of interest Some are based on the assumption that many characteristics are distributed according to the “normal curve” (e.g., height, ability, personality characteristics)  Indexes of Central Tendency: mean, mode, median, standard deviation Central tendency = how scores tend to cluster in a particular distribution Mode: most frequent score Median: midpoint Mean: sum of the scores in a distribution divided by number of scores Standard deviation -Considered best measure of dispersion or spread in a group of scores -Statistic that tell you how tightly all the various examples are clustered around the mean in a set of data. -When the examples are pretty tightly bunched together and the bell shaped curve is steep, the standard deviation is small. Mean, median and mode all have the same value since the curve is highest in the middle  Percentile scores: what are they, advantages, disadvantages. Percentiles -Rank order (Not the same as percentage correct) -Percentage of persons in the comparison group that attained a lower score -Advantages: ease of understanding; frequently used/reported  -Often used by organizations comparing candidates -Disadvantage: not equal units of measurement -For instance, a difference of 5 percentile points between two individual’s scores will have a different meaning depending on its position Problem: Tests with relatively few items, most scores clustered around the mean exaggerating differences  Standard Scores: z-scores, T-scores, IQ scores, SAT/GRE scores, stanines Transformed scores with a specified mean and standard deviation in relation to the comparison group z-scores (M = 0, SD = 1) T-scores (M = 50, SD=10) ▪ Used in many aptitude and personality measures, usually range 20-80 IQ scores (M=100, SD=15) SAT Total (M = 1060; SD = 217) max 1600 ACT Composite (M = 20.8; SD = 5.8) max 36 GRE General Tests (ranges 130-170; 0 to 6) o Verbal Reasoning (M =151, SD =8.3) o Quantitative Reasoning (M = 157, SD = 9.9) o Analytic Writing (M = 3.5; SD =0.88 Stanine/CAPS scores (M =5, SD=2)  Correlations, correlation coefficient (direction and strength of relationship), why used in testing? Correlation: degree of relationship between two variables Correlation coefficient = an index number expressing the degree of relationship: o 0.0= no relationship o From +1.0 = perfect positive correlation to -1.0 = perfect negative relationship close to 0 indicates weak relationship A number close to 1 (or -1) indicates strong relations ▪ Used to express validity - Where tests scores are correlated with criterion values ▪ To express reliability - where two sets of scores for the same test are correlated  Reliability: test-retest, alternate or parallel forms, internal consistency Is the test consistent Inter-Rater Reliability & Intra-Rater Reliability  o INTER: the extent to which scores are the same across raters (INTRA: by the same rater on different occasions) Test/retest: measures consistency over time o A test-retest correlation indicates relationship between scores obtained by individuals within the same group on two administrations of the test. Alternate or parallel forms o An alternate or parallel form correlation indicates consistency of scores of individuals within same group on two alternate but equivalent forms of the same test taken at the same time Internal Consistency o Split half correlation indicates consistency of scores obtained by individuals within the same group on two different parts of the test (e.g., odd vs. even items) on one form of the test taken on one occasion (no time fluctuation, one form)  What constitutes practical, acceptable reliability? If used to make diagnostic decisions, expected to have high reliability (.8 to.99), >.80 high  Validity: content validity, face validity, construct validity, criterion-related validity (concurrent and predictive), incremental validity Validity- Extent to which meaningful and appropriate inferences can be made from the instrument o Content validity: items measure what test is supposed to be measuring o Face validity: appraisal of test’s content, made on the "face" of the test o Construct validity: the extent to which the measure actually measures what the theory says they do o Criterion-Related Validity: Compares the test with another measure or outcome (the criterion) already held to be valid (or compared to “gold standard”) ▪ Concurrent: test scores and criterion performance scores collected at same time ▪ Predictive: performance or criterion measure is obtained some time after the test score o Incremental Validity: Predictor measures useful only if they demonstrate they can explain something about the criterion measure that was not already known ▪ Does psych GRE add to a prediction from general GRE Interview and Behavioral Observations  Interview structure: structured, semi-structured, unstructured  Interview- The most commonly used “assessment tool Structured Interviews o Composed of prescribed questions asked in predetermined sequence, with little if any variation Semi-structured Interviews o Topics and questions specified in advance, but more flexibility in the sequence, and extent to which particular responses are followed up on Unstructured Interviews o All decisions regarding the conduct of interviews to the judgment of the individual practitioner, offering maximum flexibility  Intake Interview, purpose of intake interview; informed consent Intake interview o A conversation between a counselor and a consumer with a definite mutually acceptable purpose 1. Provide information to the client regarding: ▪ The role of the agency or program and the services available, ▪ respective responsibilities of client and counselor, ▪ and informed consent issues 2. Initiating the assessment process (assessment process begins with the interview) 3. Establishing rapport as a basis for future service delivery  Common Screening Inventories at Intake (Inventory of Common Problems, Symptom Check List-90, Psychiatric Diagnostic Screen Questionnaire, Patient Health Questionnaire) Symptom Check List-90-Revised (SCL-90-R) o List of 90 symptoms (e.g., “headaches, “feeling tense or keyed up”) o Provides scores for 9 scales: Somatization, Obsessive-Compulsive, Interpersonal Sensitivity, Depression, Anxiety, Hostility, Phobic Anxiety, Paranoid Ideation, and Psychotocism Inventory of Common Problems (ICP) o Used in college populations, 24 items (depression, anxiety, academic problems, interpersonal problems, physical health problems, substance use problems) Psychiatric Diagnostic Screen Questionnaire (PDSQ) o 13 scales related to the DSM-IV-TR Patient Health Questionnaire (PHQ) o scales for common mental disorders  Using screening inventories in counseling 1. identify critical items o Did client endorse items that refer to thoughts of suicide or violent behavior? 2. examine general level of responses  o Ask clients to discuss responses. IF large number of extreme responses, consider need for immediate intervention 3. note responses for substance abuse and health items (these problems often overlooked) 4. readminister at later point to monitor change 5. use scores to consult with others regarding the treatment of a case 6. Add items to assess matters of importance to your agency o You can add items, e.g., “concern someone may harm me” 7. Administer screening inventories for specific topics when seem appropriate o E.g., MI Alcoholism Screening Test, Eating Attitudes Test 8. consider possibility client could be minimizing or exaggerating their problems 9. Use in conjunction with other assessment methods 10. use to monitor your caseload (what types of clients are you serving?)  Reliability and validity of interviews as assessment tools Reliability (consistency) o Different interviewers may address different topics or organize interviews differently, influencing the responses of clients o 2 interviewers follow same structure, the various verbal and nonverbal behaviors may influence results o Little research on inter-rater and test-retest reliability (i.e. different interviewers interviewing same clients at different points in time) o Wide variation in rating personality traits o Good to excellent reliability on most DSM-IV diagnoses Validity (Measure what its meant to) o Clients may intentionally distort info o History reports can be less than accurate o Possible clinical judgment errors when processing info o unstructured’ clinical interview is also among the least reliable and potentially least valid  Interview strategies and techniques 1. Comprehensive and accurate information that is relevant to the purpose of the assessment must be elicited from the client. o Ask client to “tell their stories” o Open-ended questions, encouragers, paraphrasing, summarizing 2. Practitioner must make valid clinical judgments in interpreting, integrating, and remembering information elicited and behaviors observed o Note recurring themes o Integrate sources  o Beware of cultural biases due to client race, ethnicity, social class, gender, disability  Observations Appearance o Is the person well groomed or unkempt? o What is person wearing? Are clothes appropriate for weather and situation? Attitude o Is the person calm, cooperative, uncooperative or belligerent? Behavior o Does person show any unusual movements or psychomotor changes? Speech (style and quality) o Is speech normal for rate/tone/volume? Affect (immediately expressed and observed emotion) o Reactive and mood congruen Mood (pervasive and sustained emotion, sometimes referred to as the feeling tone) o Euthymic (cheerfulness, tranquility) o Irritable (easily annoyed, angry, argumentative, or aggressive) o Elevated (impulsive, creative, thinking quickly, impatient, agitated, distracted and unfocused) o Anxious (worried, nervous, tense, overwhelmed) o Depressed (sad, down, Orientation o Oriented to time and place? Memory o Long-term (past events), recent (3/3 objects after 5 minutes), immediate (immediate recall, e.g., digit span) Fund of knowledge o Awareness of current events (e.g., who is president; name bordering states) Concentration o Serial 7s; spell WORLD backwards; follow 3-step command Thought Processes o Goal directed and logical o Disorganized or other? Thought Content o Suicidal or homicidal ideation? Plan, intent and/or means? o Delusions, phobias, obsessions/compulsions, other? Perception o Any signs of hallucinations or delusions during interview? Insight and Judgement (good, fair, poor) o Does the person have insight into their condition?   Mental Status examination an assessment of current mental capacity through evaluation of general appearance, behavior, any unusual or bizarre beliefs and perceptions (e.g., delusions, hallucinations), mood, and aspects of cognition (e.g., attention, orientation, memory) Cognitive and Intelligence Assessment  What is intelligence? single measurable characteristic affecting all mental ability Three Meanings of Intelligence(Gardner, 2003) 1. Intelligence as a species characteristic. ▪ Delineating the defining characteristics of human intelligence. 2. Intelligence as an individual difference. – ▪ “Susan displays more intelligence than John.” 3. Intelligence as fit execution of an assignment. ▪ What distinguishes beautiful piano playing is not the technique per se but the sheer intelligence of the player’s interpretations.  Uses of intelligence testing Helpful for determining specific skills, abilities, and knowledge of an individual Documented accuracy in predicting future behavior diagnostic purposes  Spearman’s g factor g factor o Postulated by Charles Spearman in 1904 o g stands for general intelligence, o is a statistic used in psychometrics in an attempt to quantify the mental ability underlying results of various tests of cognitive ability.  Flynn Effect Mean IQ scores have increased about 3 points per decade over past 50 years, but don’t see this rise in achievement and scholastic aptitude tests o Rising cognitive demands of work and leisure leads to environmental “social multiplier” effect o Rise in general intelligence or rise in special skills related to taking IQ tests o Better health,  o Test familiarity  Cattell-Horn-Carroll (CHC) Theory of Cognitive Abilities Cattell-Horn-Carroll (CHC) Theory of Cognitive Abilities – 10 broad stratum abilities (over 70 narrow abilities) o ◦Crystallized Intelligence (Gc): includes the breadth and depth of a person's acquired knowledge o Fluid Intelligence (Gf): includes the broad ability to reason, form concepts, and solve problems o Quantitative Reasoning (Gq): is the ability to comprehend quantitative concepts and to manipulate numerical symbols o Reading & Writing Ability (Grw): includes basic reading and writing skills. o Short-Term Memory (Gsm): is the ability to apprehend and hold information in immediate awareness and then use it within a few seconds. o Long-Term Storage and Retrieval (Glr): is the ability to store information and fluently retrieve it later in the process of thinking. o Visual Processing (Gv): is the ability to perceive, analyze, synthesize, and think with visual patterns, including the ability to store and recall visual representations. o Auditory Processing (Ga): is the ability to analyze, synthesize, and discriminate auditory stimuli o Processing Speed (Gs): is the ability to perform automatic cognitive tasks, particularly when measured under pressure to maintain focused attention. ◦ o Decision/Reaction Time/Speed (Gt): reflect the immediacy with which an individual can react to stimuli or a task  Validity evidence for IQ tests: test scores as predictors Research shows that general intelligence plays an important role in valued life outcomes. o academic success o job performance o socioeconomic advancement  Intelligence Tests: Stanford Binet-V o ages 2-89 years o comprised of 10 subtests o five verbal and five nonverbal subtests  o Nonverbal subtests can be used for people with hearing impairments, communication disorders, and limited English-language background. o Scales/Indexes ▪ Full scale IQ ▪ Verbal IQ and Nonverbal IQ – ▪ Five factor indexes: ✓ ◦ fluid reasoning, knowledge, quantitative reasoning, visual-spatial reasoning, and working memory Slosson Intelligence Test Revised o Oral intelligence test o individually administered o Ages 4- 65 o Populations: available for people with visual impairment, reading difficulty, and physical disabilities Peabody Picture Vocabulary Test o Age 2-90+ o untimed test o Orally administered 10-15 min o no reading level required/ Item responses are made by pointing o (M =100, SD = 15) o *Special Populations: applicable for people with intellectual disabilities Ravens Progressive Matrices o measure the ability to perceive and use relationships between nonverbal materials such as spatial aptitude, inductive reasoning, and perceptual accuracy o Spearman’s g o Age 5- older o Individual or group administration o No time limits o *Populations: Helpful for people with physical or emotional disabilities Comprehensive Test of Nonverbal Intelligence (CTONI) o Brief measure of general intelligence focused on abstract reasoning and problem solving o Matrix format o 15-20 minutes o Language-free test o M = 100, SD = 15 Kaufman Brief Intelligence Test (KBIT) o Approx 20 minutes  o Age 4-90+ o To identify individuals who may require further evaluation o To obtain a quick estimate of the intellectual ability of adults in institutional setting o Useful for reevaluating periodically intellectual ability of adults at work setting o To obtain information useful for vocational and rehabilitation setting o Verbal, nonverbal, and overall IQ score will be available o Scores provided on a familiar scale (M =100, SD = 15) Haptic Intelligence Scales o Performance-based intelligence measure o for blind and partially sighted adults o Age Range: 16.0 years and older o Time: 60- 90 minutes o designed to be used with, or independently of, the verbal scale of the WAIS o Scale is composed of 6 subtests: ✓ Digital Symbol — Consists of a plastic plate embossed with simple geometric forms. ✓ Object Assembly —Objects are to be assembled. ◦ ✓ Block Design — ✓ Object Completion ✓ Pattern Board —Subject examines a pattern with his fingers and then reproduces it after pegs are withdrawn. ✓ Bead Arithmetic — Consists of an abacus Woodcock Johnson Tests of Cognitive Abilities o 2 to 90+ o Standard (approx 60 minutes) and Extended Versions o WJ III Normative Update (NU) Cluster / Test Oral Expression Listening Comprehension Written Expression Basic Reading Skills Reading Comprehension Reading Fluency Math Calculation Skills Math Reasonin  Wechsler Adult Intelligence Scale-IV (index scores, scoring, etc) o 4 index scores representing major components of intelligence:  – Verbal Comprehension Index (VCI) ▪ application of verbal skills and information to the solution of new problems ▪ ability to process verbal information ▪ ability to think with words ▪ crystallized knowledge – –Perceptual Reasoning Index (PRI) ▪ ability to think in terms of visual images and manipulate them with fluency ▪ cognitive flexibility (including the ability to shift mental operations) – Working Memory Index (WMI) ▪ working memory ▪ short-term memory ▪ the ability to sustain attention, ▪ Numerical ability ▪ encoding ability – Processing Speed Index (PSI) ▪ rate of test taking ▪ perceptual discrimination ▪ speed of mental operation, ▪ psychomotor speed ▪ attentiion o 2 broad scores are also generated, which can be used to summarize general intellectual abilities: – Full Scale IQ (FSIQ), based on the total combined performance of the VCI, PRI, WMI, and PSI – General Ability Index (GAI), based only on the six subtests that comprise the VCI and PRI. – – It can be used as a measure of cognitive abilities that are less vulnerable to impairment. WAIS-IV Index Scores (IQ scores, M = 100, SD = 15) Score Description