Measurement Theories in Psychometrics

• The topic is about measurement theories in psychometrics.

Introduction

• In classical sciences like medicine, physics, and chemistry, instruments with specified error margins exist to measure characteristics like temperature, blood pressure, weight, and chemical concentrations.
• Psychology lacks such direct measurement instruments for traits like introversion, attitudes towards abortion, spatial aptitude, or reading ability.
• Mathematical or statistical theories are developed to measure psychological traits by inferring trait levels from observed performance.
• The question is whether a test score represents a good manifestation of the authentic trait of attention in that person.
• Psychologists need to be certain that the scores from tests produce good quantifications of the levels in their patients.

Classical Test Theory (CTT)

• Classical Test Theory (CTT) emerged at the beginning of the 20th century from the early work of psychologist Charles E. Spearman.
• CTT proposes a formal model to determine to what extent a test provides information about true trait levels.

CTT Scores and Types of Error

• A person's direct empirical score on a test (X) comprises two theoretical components which are the person's true trait level or score (V) and the error of measurement (E) that occurs when measuring the trait with the test: X = V + E
• X is the total score and empirically derived.
• V is the true score.
• E is the Measurement error.
• The total score (X) is the only element that can be observed.
• A true score (V) is constant for each person, or what they would score if no error existed.
• Measurement error (E) is a random variable that causes variability in scores.

Types of Error

• Errors occur due to transitory factors that are dependent on environmental factors, time, and motivation.
• Errors arise due to the specificity of the content of the items, where items should be representative and varied.
• Errors are from random factors that occur due to sporadic circumstances.

Parallel forms in the CTT

• Parallel forms refer to creating equivalent versions of a measurement test.
• These instruments measure the same construct with equivalent psychometric properties.
• Parallel forms have similar characteristics like the number of items, content, difficulty, instructions, and administration time.
• The versions are used interchangeably to measure the same construct.
• Repeated test administrations are permitted without risks of individuals recalling their responses from the previous administration.
• When administered to the same people simultaneously, the correlation between observed scores correlates in both forms, estimating reliability coefficient.

CTT Assumptions

• The expected value of errors is 0; the average is zero if applied infinitely: E(e) = 0.
• The correlation between true scores of people and their errors is zero: PVE = 0.
• The correlation between errors of both forms is zero: PE1E2 = 0.
• The correlation between the true scores of people and their errors in each form is zero: PVE1= PVE2 = 0.

CTT Limitations

• It is difficult to make comparisons because both subjects have distinct true scores because there is variability between subjects.
• It is difficult to generalize results because item parameters depend on the sample, such as difficulty and discrimination.
• There is inconsistency in the measurement error because the error is not constant between scores and can vary between skill levels.
• There is a lack of model fit indicators, which makes the CTT model unfalsifiable, so there are no mechanisms to verify if it functions.

Item Response Theory (IRT)

• Item Response Theory (IRT) emerged in the 1950s, in response to the limitations of CTT.
• IRT relies on the relationship between the latent trait (θ), representing an individual's ability taking the test, and the probability of correctly answering item P(θ).
• IRT offers a more flexible approach that doesn't rely on the sample and adapts to an evaluated person's different skill or trait levels.

Item Characteristic Curve

• Y-axis: probability of success, or P(0)
• X-axis: level of a trait (0)

Parameters of Items

• Discrimination Index (a) measures the item's capacity to distinguish between subjects with better or worse scores.
• Difficulty Index (b) signals the point at which one is more likely to get the item right around 0.5.
• Pseudo-Randomness (c) measures the probability of getting the item correctly by chance, even when they do not know the correct answer.

TRI: Main Models

• Models employ different complexity, parameters, and utility.
• The Rasch model uses parameter b for difficulty, is simple, and has easy interpretation.
• The 2-parameter (ML2P) model uses parameters b for difficulty and a for discrimination to better represent real data with discriminative capacity.
• The 3-parameter (ML3P) mode uses parameters b for difficulty, a for discrimination, and c for pseudo-guessing, to more closely model multiple-choice items by adjusting the probabilities for guessing.

IRT Assumptions

• Unidimensionality ensures that the items measure a single construct.
• Local independence occurs when the responses to an item do not depend on other items.

IRT Applications

• Developing tests allows for more precise measurement, which allows for improvements in the study of psychometric properties.
• The different models allow for different evaluations to assess more complex, multidimensional traits.
• It allows for comparison between populations that does not depend on estimating the sample parameters.
• It develops adaptive computer-based tests (CAT) where each question is different per evaluated subject and adapts to their trace level.
• Examines the differential functioning of Items (DIF), which determines whether an item functions better or worse based on a person's group.