Chap 4.pdf.pdf

Chapter 4 VALIDITY Validity defined….  Validity The extent to which the test measures what it claims to measure Validity influences the accuracy of inference made on the basis of tests Always mentioned in relation to some purpose, or population No tool is universally va...

Chapter 4 VALIDITY Validity defined….  Validity The extent to which the test measures what it claims to measure Validity influences the accuracy of inference made on the basis of tests Always mentioned in relation to some purpose, or population No tool is universally valid Meaning of ‘validation’ Face validity Validity- Trinitarian View  CONTENT VALIDITY “ The judgment of how adequately a test samples behaviour representative of the universe of behaviour that the test was designed to sample” e.g. Making a content-valid educational achievement test/ employment test  Content validity- Quantification For each item, a panel of experts is asked if the item is- o Essential o Useful but not essential o Not necessary For each item, no of panelists stating that the item is essential is noted More than half panelists state that the item is essential- item has some content validity Greater the number of panelists stating that the item is essential- higher is the content validity Content Validity Ratio (CVR)- C.H. Lawshe CVR = ne – (N/2) (N/2)  Content Validity Ratio Negative CVR: When fewer than half the panellists indicate that the item is “essential”, CVR is negative Zero CVR: When exactly half the panellists indicate that the item is essential, CVR is zero Positive CVR: When more than half but not all the panellists indicate that  Criterion-related validity Judgment of how adequately a test score can be used to infer an individual’s most probable standing on some measure of interest (criterion) Criterion should be : Relevant Valid Uncontaminated  Types of criterion-related validity Concurrent validity Predictive validity Test scores and criterion scores are available at the same time Useful when a test is being developed to replace long process of selection/ diagnosis One has to wait till criterion scores mature/ are available Useful when a test is being developed to predict who is likely to suffer from disorder/ who is likely to succeed Calculating concurrent and predictive validity involves calculation of Pearson r correlation coefficient How to calculate Pearson r Product-Moment Correlation Coefficient? How to interpret Pearson r value? Which factors influence obtained r? How high should be validity coefficient? (Topics related to testing Assignment)  Concept of Incremental Validity Predictive validity- ‘Prediction’ Predictor variable and criterion variable Correlation of predictor and predicted variable Simple regression Multiple regression Additional (incremental validity) Regression line Multiple regression X1- Exam X2- Written X3- X4- Work Y- marks test Interview sample test Performanc e X1- Exam 1.00 0.56 marks X2- Written 0.98 1.00 0.92 test X3- Interview 0.79 0.64 1.00 0.50 0.87 X4- Work 0.55 0.45 0.50 1.00 0.89 sample test Y- 0.56 0.92 0.87 0.89 1.00 Performance  Multiple regression Y = a + b1X1 + b2X2 + b3X3 +……+ bnXn Expectancy Table Ratings at selection Satisfactory perf Unsatisfactory perf Excellent 94% 6% Very good 62% 38% Average 48% 54% Below average 25% 75% Poor 17% 83%  CONTRUCT validity Judgment about appropriateness of inferences drawn from test scores regarding individual’s standing on a variable called ‘construct’. Are the test scores as predicted by a theory? Construct validity- an umbrella concept Evidences for construct validity Convergent-discriminant validity Rotated factor matrix- result Sr no Test Factor 1 Factor 2 1 Vocabulary 0.91 -0/06 2 Analogies 0.75 0.02 3 Sentence completion 0.80 0.00 4 Disarranged sentences 0.39 -0.02 5 Reading comprehension 0.86 -0.04 6 Addition -0.09 0.55 7 Multiplication 0.07 0.64 8 Arithmetic problems 0.28 0.68 9 Equation relations 0.16 0.54 10 Number-series completion 0.13 0.38  Validity, Test bias and test fairness Test bias: Inherent characteristic of the test; test SYSTEMATICALLY overpredicts or underpredicts performance of a group, when logically no such differences exist Reasons: Content of items Bias (halo effect/ distributional errors) on the part of raters  Validity, Test bias and test fairness Test fairness: More about how the test is being used, for what purpose, for whom, how? Test may be unbiased but used in an unfair manner Important questions: Are the differences found result of unfair use/bias or they are real? What do you do (remedy) when you find group differences on a test?