Item Response Theory in Assessment

Study Notes

Item Response Theory (IRT)

A psychometric framework that examines the relationship between individuals' latent traits (unobservable characteristics or abilities) and their item responses (answers to questions or tasks).
Used in educational assessment, psychological testing, and other fields where measuring abilities or traits is essential.

Key Concepts of IRT

Latent Trait: The underlying characteristic or ability measured by IRT. This could include things like mathematical ability, reading comprehension, or anxiety levels.
Item Characteristic Curve (ICC): A graphical representation showing how item responses vary with the latent trait level. It describes the probability of a person answering an item correctly based on their latent trait level.
Parameters of IRT Models:
- Difficulty Parameter (b): Indicates how challenging an item is. Higher values mean more difficult items.
- Discrimination Parameter (a): Reflects how well an item differentiates between respondents with different latent trait levels. Higher values indicate better differentiation.
- Guessing Parameter (c): Accounts for the likelihood of a respondent guessing the correct answer, especially in multiple-choice formats.
Types of IRT Models:
- One-Parameter Logistic Model (1PL): Known as the Rasch model, assumes all items have the same discrimination parameter and only considers difficulty.
- Two-Parameter Logistic Model (2PL): Includes both difficulty and discrimination parameters, providing more flexibility in modeling item responses.
- Three-Parameter Logistic Model (3PL): Extends the 2PL model by adding a guessing parameter, suitable for items where guessing is a factor.
Test Information Function: Describes how much information a test provides about a respondent's latent trait across different trait levels. It helps evaluate the precision of the test in estimating latent traits.
Scoring and Scaling: IRT allows for the development of scales that provide more accurate estimates of an individual's latent trait. Scores from IRT are more informative than simple raw scores because they consider the properties of the items used in the assessment.

Advantages of Item Response Theory

Adaptive Testing: IRT underpins Computer Adaptive Testing (CAT), where item difficulty is adjusted based on a respondent's performance in real-time.
Detailed Item Analysis: Provides detailed information about each item, allowing for improved test design and refinement.
Comparison Across Different Tests: IRT allows for the comparison of scores from different tests measuring the same latent trait.

Applications of Item Response Theory

Educational Assessment: Used in standardized tests like the SAT and GRE to evaluate and compare student abilities.
Psychological Testing: Used in various psychological assessments, including personality inventories and mental health screenings.
Health Outcomes Measurement: Can be applied in health research to measure constructs like quality of life or health-related quality of life.

Conclusion

IRT offers a robust framework for understanding and measuring latent traits through item responses.
Its emphasis on the relationship between item characteristics and respondent abilities makes it a valuable tool in psychometrics and related fields.