Lecture 14 VP Tests & Measurements.pdf

Tests & Measurements POVD J u l y 16, 2024 1 Why is this important? ◦ O b t a i n objective m easurem en ts o f a child ’s p e r fo r m a n c e ◦ Which skills are strengths? Weaknesses? ◦ Is this c h ild gettin g b etter w ith treatm ent? ◦ How much of...

Tests & Measurements POVD J u l y 16, 2024 1 Why is this important? ◦ O b t a i n objective m easurem en ts o f a child ’s p e r fo r m a n c e ◦ Which skills are strengths? Weaknesses? ◦ Is this c h ild gettin g b etter w ith treatm ent? ◦ How much of an improvement do you need to see in a score to know if the child is getting better? ◦ Ta lk to p a rents ◦ Ta lk to o th er p rof ession als 2 Standardized Testing ◦ Test: objec tive and stand ard ized measure of a sample of behavior. ◦ Standardized: uniformity of p roc ed ures, in strumentation , and scorin g have been sp ec ified so that they c an be d uplic ated at d if ferent times and plac es. ◦ Norms: the p erformanc e d istribution o f a rep resentative sample o f the population on a p ar tic ular stand ard ized test. 3 Descriptive Norms ◦ M easurements ob tained w hen stand ard ized test is ad m in istered to a def ined p op ulation sample ◦ C h arac terize the population, not the in d ivid ual child 4  Normal Distribution ◦ Mean (x or μ): average of all values ◦ Range: d if ferenc e between the highest and lowest va lues ◦ Dev iation score: how far away is a p ar tic ular sc ore f rom the average ◦ Standard deviation (s or σ): the amount of variation or dispersion of a set of values ◦ L a rge sta n d a rd d ev iation à s c o r e s a r e r e a l ly s p r e a d o u t ◦ S m a ll sta n d a rd d ev ia tio n à r e a l ly s t e e p curve (leptokurtotic) -we prefer a larger SD 5 Percentile Distribution ◦ In all cases of normal distribution: ◦ 68% o f a ll obtained scores lie w ithin 1 stand ard deviation of the mean (34% on each side) ◦ 95% w ithin 2 stand ard deviation s (47.5% on each side) ◦ 99.7% w ithin 3 stand ard deviation s 6  NORMAL D ISTRIBUTIO N 7 Derived Scores ◦ A llo w c o m p a r is o n o f a n individual’s p e r fo r m a n c e to that o f the n orm ative p o p u la tio n ◦ In d ivid u a l’s p e r fo r m a n c e is b e in g c o m p a re d to a p o p u la tio n o f c h ild re n o f the sam e age ◦ Z scores, p e rce n tile s, s c a le d scores, s ta n d a rd sco re s c a n b e u se d , a n d a re a ll ex p re s s io n s o f the sam e c o n c e p t ◦ Pe rc e p tu a l eva lu a tio n – co nve rt to sca led score 8  Z score ◦ Expresses the distance of an individual’s raw score from the mean in terms of the standard deviation ◦ If μ = 12, σ = 2 ◦ x = 12, z = 0 ◦ x = 14, z = +1 ◦ x = 10, z = -1 ◦ x = 15, z = +1.5 -x is the raw score u is the mean over the SD -pos is faster and a better Z score 9 Z scores ◦ Positive z score – above average score ◦ Negative z score – below average score ◦ When the raw score is time, lower number (of seconds) is better 10 Percentiles ◦ The percentage of individuals in the distribution w hose sc ores fall below a given raw sc ore ◦ C o nversion table in lab manual (or google it…) ◦ Z=+1.25, w hat p erc entile ran k? 89.44% 11 12 The Myth of “ W ithin 1SD” ◦ Some believe that p erfo rmanc e is “normal” or ”within normal limits” if it is w ithin 1 stand ard deviation of the average ◦ z = -1.00 is the 16th perc entile! ◦ W hat if it’s your child? ◦ 0.5 SD or z = -0.50 is the M ozlin benchmark. W hat p erc entile? ◦ Tassinari ar ticle: A ssessin g the Assessment 13 Scaled Scores ◦ D erived f rom the z sc ore ◦ The mean is arbitrarily assigned a value of 10, and a standard dev iation of 3 ◦ S c aled sc ore = 10 + 3z ◦ I f z is negative – b e low average, 10 ◦ A lways whole numbers, never decimals ◦ D TLA uses sc aled sc ores 14  Standard Scores ◦ Similar to scaled scores, except mean is 100 and standard deviation is 15 ◦ Standard score = 100 + 15z ◦ Beery VMI, WI S C (IQ), DEM -IQ test is based off of this and 100 is the avg -intellectual disability is an IQ below 70 à bc its 2 SD below the mean 15  C O NVERTING BETWEEN SC O RES 99% 16 17  Lateral Transformations ◦If μ = 12,σ = 2,x = 9 ◦z = ◦Scaled sc ore = ◦Standard sc ore = ◦Perc entile = -z score à 9-12/2 = -1.5 -scaled score à 10 + 3(-1.5) = 5.5 à 6 -standard score à 100 + 15(-1.5) = 77.5 à 78 -percentile can be seen in previous slide 18 SANITY BREAK!! 19  Reliability – SEM ◦ M e a s u re m e n t o f the repeatability o f a test ◦ M e a s u re d a s the re lia b ility c o e ffic ie n t (r) ◦ To a p p ly the c o n c e p t o f re lia b ility c lin ic a lly, w e c a lc ulate th e S tan d ard E rro r o f M easurem en t ◦ T h e S E M p rov id e s a m e a su re o f the ra n ge o f flu c tu a tio n s w e m ight e x p e c t i n a n in d ivid u a l’s s c o re ◦ T h e greater the reliability, the lo w e r the SEM -Dr. Vaughn hinted if if you need to calculate this in a problem you will be given the value of the standard error of measurement 20 DTLA Design Sequenc e (S c aled Scores) ◦ r = 0.76 ◦ SEM = 1.47 (or 1.5) ◦ If ob tained sc aled sc ore = 9, then w hat is the ran ge of the true sc aled score? ◦ If we retested the child , then 68% of the time, their sc ore would be between 7.5 and 10.5 21  ScS = 9 ScS = 12 14.0 14.0 SEM 13.5 13.5 13.0 13.0 12.5 12.5 ◦ For a Sc S = 9 on D esign Seq uenc e (SEM 12.0 12.0 = 1.5), the true sc ore lies between 10.5 11.5 11.5 and 7.5 11.0 11.0 ◦ If Sc S = 12 on another test w ith the same 10.5 10.5 SEM , the true sc ore lies between 13.5 10.0 10.0 and 10.5 9.5 9.5 ◦ Relative stren gths and weaknesses 9.0 9.0 8.5 8.5 ◦ A kin to statistic sign if ic anc e in 8.0 8.0 scientific stud ies 7.5 7.5 7.0 7.0 6.5 6.5 22 Pre- and Post- Therapy ◦ How much improvement in a test score is necessary to feel confident that the change in the score is the result of therapy and not just chance? ◦ Look for a change that is at least twice the SEM 23  Has the patient improved? ScS = 9 ScS = 10 ScS = 9 ScS = 12 s/p 8 s/p 12 14.0 14.0 weeks 14.0 14.0 weeks 13.5 13.5 of VT 13.5 13.5 of VT 13.0 13.0 13.0 13.0 12.5 12.5 12.5 12.5 12.0 12.0 12.0 12.0 11.5 11.5 11.5 11.5 11.0 11.0 11.0 11.0 10.5 10.5 10.5 10.5 10.0 10.0 10.0 10.0 9.5 9.5 9.5 9.5 9.0 9.0 9.0 9.0 8.5 8.5 8.5 8.5 8.0 8.0 8.0 8.0 7.5 7.5 7.5 7.5 7.0 7.0 7.0 7.0 6.5 6.5 6.5 6.5 -want an improvement of twice the standard error of measurement -so if the kid started at 9 and 8 weeks later moved to 10, there’s still an overlap in numbers and bc 10 is in the range the first time we can say its true improvement however after 12 weeks the child got better and got to 12 24 Design Sequence Pre- and Post-VT ◦ S c a le d s c o re p re -Tx = 9 ◦ S c a led sc ore p ost-T x = 12 ◦ C h a n ge is +3 ◦ S E M = 1.5 ◦ T h e im p rove m e n t i n test s c o re m ust b e at le a st 3 in ord er to b e c on sid ered a s ig n ific a n t im p rove m e n t ◦ S i n c e the s c o re d i d in c re a s e b y 3, w e a re s e c u re i n o u r d e c is io n to either d ism iss the p atien t o r c o n c e n tra te o n o th e r s k ills 25 Summary ◦ I hate statistics ◦ Scaled score is the preferred method of scoring in visual perceptual testing ◦ You can convert between z score, scaled score, standard score, and percentile ◦ We can never know a patient’s true score on a given test ◦ Observed score represents a range of scores ◦ We want to see an improvement of twice the SEM to determine the patient has really improved 26