Psychometrics and Classical Test Theory

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is the field of study concerned with the measurement of psychological attributes and characteristics?

Psychometrics (correct)

Cognitive Psychology

Behavioral Science

Sociology

What is the purpose of psychometrics?

To design and evaluate reliable and valid psychological measures

What does Classical Test Theory rest on?

True scores

What is a true score (T)?

An individual's test score measured without error Signup and view all the answers

Classical Test Theory assumes T is ____ distributed on a normal distribution.

randomly Signup and view all the answers

An alternative way to think about the true score is to imagine an individual taking the same test ____ number of times.

an infinite Signup and view all the answers

How do we infer the values of true scores?

By administering appropriate test stimuli to individuals and calculating an observed score, denoted X. Signup and view all the answers

What will the observed score be?

An imperfect measure of the true score Signup and view all the answers

The discrepancy between true and observed scores is known as ____ error, denoted e.

measurement Signup and view all the answers

What is the observed score equation?

X=T+e Signup and view all the answers

What is measurement error?

A product of any factor(s) that causes discrepancy between true and observed scores. Signup and view all the answers

Match the following measurement error assumptions:

Error is randomly distributed = True Expected value of error is 0 = True Random errors are uncorrelated with each other = True No systematic relation between the true score and error = True Signup and view all the answers

What does the mean of the error term being 0 imply about the mean of the true scores?

Equals the mean of observed scores Signup and view all the answers

What is the expected variance of the observed score?

The variance of the true score plus the error variance Signup and view all the answers

What is the Standard Error of Measurement (SEM)?

The standard deviation of the distribution of random errors around the true score for an individual Signup and view all the answers

If an individual took the same test an infinite number of times, SEM would be the ____ of their collected scores.

standard deviation Signup and view all the answers

What is reliability in Classical Test Theory (CTT)?

Precision or consistency of measurement Signup and view all the answers

How can we define reliability?

As the proportion of variance in the observed score that is due to the construct being measured Signup and view all the answers

What can reliability be thought of as?

The squared correlation between the observed and true score Signup and view all the answers

What is test-retest reliability?

An administration of the same test to the same participant on 2 or more separate occasions Signup and view all the answers

What is an estimate of reliability?

A correlation coefficient between test scores at the 2 time points Signup and view all the answers

What does Classical Test Theory definition of reliability assume?

Error variance does not differ across the 2 sessions Signup and view all the answers

What is split-half reliability?

Used in data from a single session to calculate a correlation between scores on 2 randomly selected halves of a test Signup and view all the answers

What is Cronbach's alpha (alternative formula)?

alpha= (N * r)/(1+(N-1) * r) Signup and view all the answers

What does the alternative formula for Cronbach's alpha make clear?

Reliability is a function of test length and item covariance Signup and view all the answers

What tends to happen to reliability with more items?

Increase Signup and view all the answers

What are Cronbach's alpha scores?

alpha > 0.7 considered good; alpha > 0.9 may be an indicator of redundancy Signup and view all the answers

What do the reliabilities of 2 measures place constraints on?

The maximum correlation you can observe between the 2 measures Signup and view all the answers

What happens if you have 1 or 2 poor measures?

You will be underestimating the true relationship between these variables Signup and view all the answers

What is validity?

The degree to which a test measures what it claims/intends to measure Signup and view all the answers

What is face/content validity?

Whether the measure appears to measure what it should as judged by experts or experienced clinicians Signup and view all the answers

What is criterion validity?

Extent to which a variable is related to a relevant outcome Signup and view all the answers

What is concurrent validity?

The measure is compared against another measure that is a standard criterion for that construct, where both measures are assessed at the same time Signup and view all the answers

What is predictive validity?

Score on a measure is compared with a score obtained on a future criterion Signup and view all the answers

Correlations are used to assess the degree of ____ validity of a measure.

criterion Signup and view all the answers

What is construct validity?

Examines the degree to which a measure of a given construct correlates with measures of a similar construct and measures of dissimilar constructs Signup and view all the answers

What is convergent validity?

A new scale of depression should correlate with a standard, widely-used measure of depression Signup and view all the answers

What is discriminant validity?

This measure of depression should not correlate with a separate measure known to be unrelated Signup and view all the answers

What does a multitrait-multimethod matrix measure?

Multiple traits with multiple methods Signup and view all the answers

What does monotrait-heteromethod index?

Convergent validity Signup and view all the answers

What does heterotrait-monomethod reflect?

High correlation reflects method, indexes divergent validity if traits are supposed to be different Signup and view all the answers

What are two basic ways items can differ?

Item difficulty; Item discrimination Signup and view all the answers

How is item difficulty defined for dichotomous items?

By the number of people who get a particular item correct or choose a particular option Signup and view all the answers

What does item discrimination compare?

The relationship between performance on an item and performance on the whole test Signup and view all the answers

What are the 3 basic ways to measure item discrimination?

Extreme Group Method; Item-Total Correlation; Inter-Item Correlation Signup and view all the answers

What does the Extreme Group Method involve?

Take the top and bottom 1/3 of test scores and calculate the discrimination index Signup and view all the answers

What is Item Total Correlation?

The correlation between the score on an item and the total test score Signup and view all the answers

What can Inter-Item Correlation show?

What items positively or negatively relate to other items Signup and view all the answers

What is the Item-Characteristic Curve?

Graphical presentation of the proportion of people who got the item right versus the total test score Signup and view all the answers

What are the steps of drawing an Item-Characteristic Curve (ICC)?

Calculate total scale score for everyone, group individuals based on total score, and compute proportions for item correct responses. Signup and view all the answers

_ slopes mean an item is more discriminating than _ slopes.

Steeper; flatter Signup and view all the answers

What is Item Response Theory?

A modern measurement theory designed to model individual or group responses to items based on levels of a latent trait. Signup and view all the answers

What are the advantages of Item Response Theory?

Allows sample and item invariant measurement and can estimate SEM/reliability across different levels. Signup and view all the answers

In ICC, what does p stand for?

Probability of responding to an item Signup and view all the answers

What does high discrimination mean?

Probability of response increases more rapidly as ability increases Signup and view all the answers

What is the Test Information Function?

Plots the amount of information relative to ability Signup and view all the answers

What does Item Response Theory show?

Differential item functioning across different groups who take the same test Signup and view all the answers

What is an example of Differential Item Functioning (DIF) Analysis?

Females and males might perform differently on an item even if they have the same level of ability Signup and view all the answers

Study Notes

Psychometrics

Field dedicated to measuring psychological attributes and characteristics.
Aims to develop reliable and valid psychological assessments.

Classical Test Theory (CTT)

Fundamental premise is the concept of true scores, representing an individual's score without measurement error.
True scores are assumed to be normally distributed within the population.
Infers true scores by calculating observed scores through testing.

Measurement Concepts

Observed scores (X) are derived from the true score (T) and measurement error (e), represented by the equation X = T + e.
Measurement error reflects discrepancies between true and observed scores, influenced by various factors like fatigue or distractions.
Assumptions include random distribution of error, zero expected value of error, and no correlation between true scores and errors.

Reliability

Defined as the consistency and precision of measurement in assessments.
High reliability indicates that test items reliably represent the overall construct being measured.
Calculated as the proportion of observed score variance attributed to the true construct.

Types of Reliability

Test-retest reliability measures consistency of scores over time.
Split-half reliability assesses internal consistency by correlating scores of two halves of the same test.
Cronbach's alpha is a reliability coefficient, with values above 0.7 deemed acceptable, and above 0.9 indicating potential redundancy.

Validity

Validity gauges whether a test effectively measures what it is intended to.
Includes face/content validity, criterion validity (subdivided into concurrent and predictive validity), and construct validity.
Construct validity examines correlations between similar and dissimilar constructs, emphasizing convergent and discriminant validity.

Item Analysis

Items may vary based on difficulty (how many can answer correctly) and discrimination (how well they correlate with total test performance).
Item-Characteristic Curve (ICC) visually represents the success rates on test items against total scores.
Item Response Theory (IRT) enhances measurement by accounting for how individual traits influence responses.

Differential Item Functioning (DIF)

Analyzes how different groups respond to test items, important for ensuring fairness.
Example: May identify performance differences between genders on specific test items even when abilities are equal.

Summary of Key Methods

Extreme Group Method evaluates item discrimination by analyzing performance in the highest and lowest scoring groups.
Item-Total Correlation assesses an item’s effectiveness by correlating it with total test scores.
Inter-Item Correlation shows how items relate to one another, facilitating identification of strong or weak items in an assessment.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Description

Explore the fundamental concepts of psychometrics, including the principles of classical test theory (CTT) and measurement concepts. This quiz will assess your understanding of true scores, observed scores, measurement error, and the importance of reliability in psychological assessments.