PSYC332 - Item Response Theory Flashcards

Questions and Answers

What is the difference between Classical Test Theory and Item Response Theory?

CTT focuses more on total score of a scale or subscale; IRT focuses on relationship between items and total score or latent dimension underlying the test.

What is the key equation of CTT?

Test score = true score + error

Where does error come from?

Situational variables like tiredness, confusion, distraction; and in latent variables, there is always some error.

What is the main point of an item characteristic curve?

It explains the relationship between probability of a 'yes' response on a true/false item and the underlying dimension. Signup and view all the answers

Do estimates from IRT depend on the sample they are drawn from?

False Signup and view all the answers

What are the assumptions of IRT?

Unidimensionality and local dependence. Signup and view all the answers

What kind of test is IRT often used for?

Ability tests like mathematical tests. Signup and view all the answers

What does a steeper slope in ICC tell us?

It indicates more discrimination, making it easier to differentiate between high and low ability test-takers. Signup and view all the answers

What is a 3 Parameter ICC?

It includes a parameter for 'pseudo guessing' that estimates what individuals with low scores will achieve. Signup and view all the answers

What is ICC for multiple choice?

It plots the probability for each item response. Signup and view all the answers

Provide a summary of the different IRT models.

Parametric vs. Non-parametric, 1 Parameter (Rasch), 2 Parameter, and 3 Parameter models. Signup and view all the answers

What is Item Banking and Adaptive Testing?

Using established IRT parameters to change items shown to participants based on their ability. Signup and view all the answers

What is this pyramidal testing model?

A model where each participant starts at the same question and branches based on their responses. Signup and view all the answers

What is CAT logic?

Signup and view all the answers

Study Notes

Classical Test Theory vs Item Response Theory

CTT emphasizes total scores on scales, focusing on overall performance.
IRT examines item relationships and latent dimensions, providing deeper insights into test constructs.

Key Equation of Classical Test Theory

Test score is calculated as: true score + error.

Sources of Error in Testing

Error can arise from situational variables like fatigue, confusion, and distraction.
Latent variable assessments will inherently include error due to their indirect nature.

Item Characteristic Curve (ICC)

The ICC illustrates the connection between the likelihood of a positive response and the underlying trait being measured.
Usually shaped like a cumulative normal distribution, representing various ability levels.

Sample Invariance in IRT

IRT estimates are independent of the sample population from which they are drawn.

Assumptions Underlying IRT

Unidimensionality: each item measures only one construct.
Local dependence: items should not be excessively similar to maintain measurement integrity.

Common Application of IRT

Frequently used in ability assessments, particularly in mathematics.

Interpretation of ICC Slopes

A steeper slope indicates higher discrimination among test-takers, facilitating differentiation between high and low performers.

Three Parameter ICC

The 3-parameter ICC includes a 'pseudo-guessing' parameter, reflecting the performance of low ability individuals, typically represented by the Y-axis intercept.

ICC in Multiple Choice Tests

For multiple choice items, ICC plots the probability of each possible response, providing a visual representation of answer likelihood.

Summary of IRT Models

IRT models can be categorized as parametric (more widely used) vs. non-parametric, with Ramsay's TestGraf serving as a non-parametric method.
1-parameter or Rasch models assume uniform item discrimination with variations in difficulty.
2-parameter models allow for differences in both slope (discrimination) and difficulty.
3-parameter models expand on the 2-parameter framework by factoring in pseudo-guessing.

Item Banking and Adaptive Testing

Once IRT parameters are established, adaptive testing can adjust item presentation based on participant performance, starting with items that have high discrimination.
Subsequent items are chosen to maximize information gain, within a set time limit and desired standard error.

Pyramidal Testing Model

Participants begin with identical questions and navigate through branching questions based on their responses, allowing tailored assessment paths.

Computerized Adaptive Testing (CAT) Logic

CAT dynamically adjusts test difficulty and item selection based on individual performance, enhancing the measurement's precision and efficiency.

Description

This quiz focuses on key concepts from Item Response Theory and Classical Test Theory, essential for understanding psychological measurement. It includes important definitions and equations that distinguish these two theories. Ideal for students in psychology courses or those preparing for exams on measurement theory.