Validity and Reliability of Classroom Tests PDF

VALIDITY AND RELIABILITY of CLASSROOM TESTS Sherry E. Belmoro, PhD  Learning Outcomes: 1. Identify the type of validity used in a given assessment scenario. 2. Propose ways to improve validity of classroom tests. 3. Identify type of reliability used in a given assessment sce...

VALIDITY AND RELIABILITY of CLASSROOM TESTS Sherry E. Belmoro, PhD  Learning Outcomes: 1. Identify the type of validity used in a given assessment scenario. 2. Propose ways to improve validity of classroom tests. 3. Identify type of reliability used in a given assessment scenario. 4. Propose ways to improve reliability of classroom tests.  Validity Reliability the extent to which a test measures or predicts what is the extent to which a test supposed to. yields consistent results If a test in English is created to Given a rubric to evaluate measure the students’ a product, judges will knowledge about figures of likely give similar ratings. speech then the test should Being reliable is being contain items about figures of consistent. speech. Being valid is being on target. 1 Types of Validity Face Validity The extent to which the assessment appears to test what it aims to test. Does the test look like what it intends to measure? Is often considered as the weakest form of validity. You might look at a test of English Proficiency, read through the items and decide that it seems to be a good measure of English Proficiency.  2 Content Validity The extent to which the content of the test matches the instructional objectives. Does the test contain items cover the “content domain”. A term exam that only includes content covered during the first two sessions is not a valid measure of the course's overall objectives -- it has very low content validity.  3 Criterion Validity The extent to which scores on the test are in agreement with (concurrent validity) or predict (predictive validity) an external criterion. Does the test relate to an existing similar (concurrent) or later (predictive) performance measure? Performance in the IQ Test is related to your performance in Metropolitan Achievement Test which was both given during the entrance exam. Entrance examination performance can predict success in the LET Board Examination. (predictive)  4 Construct Validity The extent to which the assessments is measuring the correct construct (trait/attribute/ability/skill) Does the test relate to underlying theoretical concepts adequately? A test that measures musical ability should have enough items to measure musical ability.  TAKE NOTE: Because validity is a matter of degree it is appropriate that you use relative terms such as high validity, moderate validity, and low validity.  Ways to Improve Validity of Tests 1. Make sure your goals and objectives are clearly defined and operationalized. 2. Match your assessment measure to your goals and objectives. 3. Have the test reviewed by other teachers in your field of specialization. 4. Get students involved; have the students look over the assessment for troublesome wording, or other difficulties (this process may be applied with the rubrics you made). 5. If possible, compare your measure with other measures, or data that may be available. II. RELIABILIT Y What does reliability have to do with the practical aspects of planning, administering and analyzing results of classroom tests?  One student should be able to take a test today and then again tomorrow and get very similar results. The results should be the same no matter who scores the test and when they score it. If a teacher needs to make different forms of a test, the test should have the same content and the same level of difficulty. A student should be able to perform comparably on both tests. TYPES OF RELIABILITY 1 TEST-RETEST RELIABILITY Measure of Stability Administer the same test to the same group of students at two different times. 2 EQUIVALENT/ALTERNATE FORMS RELIABILITY Measure of Equivalence Administer two different forms of the same test to the same group of students.  3 INTER-RATER RELIABILITY Measure of Agreement Have two or more raters rate the performance or product and then check the agreement between or among the raters  4 INTERNAL CONSISTENCY RELIABILITY Measure of how consistently each item measures the same concept Check the performance of students in each item with overall performance across participants. Reliability concerns and what you can do to address them: Nitko and Brookhart (2011). For all assessments: Encourage students to perform consistency within student their best Match the assessment difficulty to (not that they always do the the students’ ability levels same, but that they Have scoring criteria that are consistently try to show what available and well understood by they know). students before they start the assignment For objective assessments like Have enough items multiple choice tests: consistent Allow enough time for students to performance from item to item. complete the test Have clear enough directions for For papers, essays, and projects: students accuracy of rater judgment and Have a systematic scoring procedure consistency across forms Have multiple markers (scorers) (prompts or assignments). when possible Relationship between VALIDITY & RELIABILITY Relationship between VALIDITY & RELIABILITY Validity and reliability are closely related. Validity is more important feature. Reliability is a prerequisite to validity. Assessment results may be reliable but not valid. To be useful, a test should be both reasonably valid and reliable. Aim for validity first, and then try to make the test more reliable little by little. thanks! Do you have any questions? CREDITS: This presentation template was created by Slidesgo, including icons by Flaticon, and infographics & images by Freepik Please keep this slide for attribution