Classroom Assessment: Validity and Test Theory

• Assessment is a multifaceted process that includes collecting, interpreting, and using information to inform teachers' decisions.
• Teachers use assessments to document and improve student learning.
• Classical Test Theory is an approach to measurement.
• It considers the relationship between expected scores (true scores) and observed scores in an assessment.
• The formula for Classical Test Theory is X = T + e, the test score equals the true score plus error.
• Qualities of a good assessment are reliability, validity, appropriateness, practicality, positive motivation, and standardization.
• Validity refers to the appropriateness of the interpretation and use of assessment results for a specific group of students in a specific context.

Sources of Validity Evidence in Classroom Assessment

• Content-related evidence assesses how well the assessment represents the domain of interest, using methods like face validation and preparation of a Table of Specifications (TOS).
• Criterion-related evidence assesses the relationship between an assessment and another measure of the same trait.
• Construct evidence assesses the extent to which operationalizations of a construct measure what the theory says they do, using Factor Analysis.

Steps in Testing and Assessment

• Determine the purpose of measurement.
• Develop the Table of Specifications.
• Select appropriate assessment tasks.
• Prepare relevant assessment tasks.
• Assemble the assessment.
• Administer the test.
• Appraise the assessment.
• Use the results to improve instruction and learning.

Table of Specifications (TOS)

• A TOS is a two-way grid displaying the content, learning targets, and levels of thinking skills in an assessment, which helps with content validity.
• It allows teachers to design a test focusing on key areas with varied weights based on importance.

Elements of a TOS

• Content and/or Instructional Objectives
• Time spent teaching the content.
• Percent of class time on a topic.
• Number of test items.
• Levels of Thinking (Revised Bloom's Taxonomy).
• Item Placement.

Revised Bloom's Taxonomy of Educational Objectives

• A framework for classifying statements about what students should learn as a result of instruction.
• A scheme used to classify educational objectives, learning activities, and assessments to ensure standards are met.
• A classroom planning tool to express qualitatively the different levels of thinking.
• Lorin Anderson and David Krathwohl are the authors of the Revised Bloom's Taxonomy.

Knowledge Dimension

• Factual Knowledge: the basic elements students must know to understand a discipline or solve problems.
• Factual knowledge includes knowledge of terminology and specific details.
• Conceptual Knowledge: the interrelationships among basic elements within a larger structure.
• Conceptual knowledge includes classifications, categories, principles, models, and theories.
• Procedural Knowledge: How to do something, methods of inquiry, and criteria for using skills/algorithms.
• Procedural knowledge includes skills, algorithms, techniques, and methods.
• Metacognitive Knowledge: knowledge of cognition, including awareness of one's own cognition.
• Metacognitive knowledge includes strategic knowledge, knowledge of cognitive tasks, and self-knowledge.

Cognitive Process Dimension

• Remembering: Retrieving knowledge from long-term memory.
• Remembering includes recognizing and recalling.
• List of verbs includes cite, define, describe, identify, label, list, match, name, outline, quote, recall, report, reproduce, retrieve, show, state, tabulate, and tell.
• Understanding: Constructing meaning from instructional messages, including oral, written, and graphic communication.
• Understanding includes interpreting, exemplifying, classifying, summarizing, inferring, comparing, and explaining.
• List of verbs includes arrange, articulate, associate, categorize, clarify, classify, compare, compute, conclude, contrast, defend, diagram, differentiate, discuss, distinguish, estimate, exemplify, explain, extend, extrapolate, generalize, illustrate, infer, interpolate, interpret, match, outline, paraphrase, predict, rearrange, reorder, rephrase, represent, restate, summarize, transform, and translate.
• Applying: Carry out or use a procedure in a given situation.
• List of verbs includes apply, calculate, carry out, classify, complete, compute, demonstrate, dramatize, employ, examine, execute, experiment, generalize, illustrate, implement, infer, interpret, manipulate, modify, operate, organize, outline, predict, solve, transfer, translate, and use.
• Analyzing: Break materials into constituent parts and how the parts relate to one another and to the overall structure or purpose.
• Analyzing includes differentiating and organizing.
• List of verbs includes analyze, arrange, break down, categorize, classify, compare, connect, contrast, deconstruct, detect, diagram, differentiate, discriminate, distinguish, divide, explain, identify, integrate, inventory, order, organize, relate, separate, and structure.
• Evaluating: Make judgments based on criteria and standards.
• Evaluating includes checking and critiquing.
• List of verbs includes appraise, apprise, argue, assess, compare, conclude, consider, contrast, convince, criticize, critique, decide, determine, discriminate, evaluate, grade, judge, justify, measure, rank, rate, recommend, review, score, select, standardize, support, test, and validate.
• Creating: Put elements together to form a coherent or functional whole; reorganize elements into a new pattern or structure.
• Creating includes generating, planning, and producing.
• List of verbs includes arrange, assemble, build, collect, combine, compile, compose, constitute, construct, create, design, develop, devise, formulate, generate, hypothesize, integrate, invent, make, manage, modify, organize, perform, plan, prepare, produce, propose, rearrange, reconstruct, reorganize, revise, rewrite, specify, synthesize, and write

Why categorize objectives?

• Categorizing permits educators to examine objectives from the students' point of view.
• It helps educators consider the panorama of possibilities in education.
• It helps educators see the integral relationship between knowledge and cognitive processes inherent in objectives.
• It helps educators deal with assessment questions.
• Categorization makes more readily apparent the consistency, or lack of it, among stated objectives, teaching methods, and learning assessment.
• It helps educators make better sense of the wide variety of terms used in education.