Psychometrics and Validity Concepts

Psychometrics

• Psychometrics is the science of measuring psychological constructs
• Examples of psychological constructs include personality traits, cognitive skills, ability, and psychopathological disorders
• Examples of measuring depression are questionnaires such as the Beck Depression Inventory (BDI)

Measurement of Depression

• A depression inventory (e.g., BDI) uses items to measure constructs, such as hopelessness
• Example items include questions about feelings of hopelessness and concentration difficulties
• BDI questions include statements like "I am not particularly discouraged about the future" or "I can't do any work at all."

Construct Validity

• Construct validity is the extent to which a measure accurately measures what it claims to measure
• Two subjective methods include face validity (does the measure look like what it measures) and content validity (does the measure include all aspects of the construct)
• Empirical methods include criterion validity, convergent validity, and discriminant validity

Criterion Validity

• Concurrent criterion validity checks if a measure correlates with a relevant behavior at the same time
• Predictive criterion validity checks if a measure correlates with a relevant behavior at a different time

Convergent Validity

• Convergent validity checks if a measure correlates with other measures of the same construct

Discriminant Validity

• Discriminant validity checks if a measure does not correlate with measures of other constructs

Classical Test Theory (CTT)

• CTT frames test scores as the sum of a true score and an error score
• X=T+E, where X is the test score, T is the true score, and E is the error score
• A true score is the average score a person would get if they took the same test repeatedly in the same conditions.

Reliability

• Reliability is how consistent a test is in its measurement
• Reliability is the proportion of variance in the test score due to the true score
• Reliability is the squared correlation between a test score and a true score
• Acceptable levels for reliability include >0.7 for early research, >0.8 for applied research, and >0.9 for making important decisions.
• Reliability estimation methods include parallel tests, test-retest, and split-half reliability.

Internal Consistency Reliability (e.g., Cronbach's Alpha)

• Internal consistency assesses the extent to which items on a test correlate with each other. Cronbach's alpha is a commonly used measure of internal consistency.
• Values close to 1.0 indicate items are very highly correlated (redundant). A value of 0.8 is frequently considered desirable.

Modern Psychometrics

• Modern psychometrics uses latent variable models, where a latent variable (e.g., depression) is not directly observed but inferred from observed variables (items on a questionnaire).
• These models can account for relationships between observed variables, making inferences about the latent variable more accurate.

Latent Variable Modeling

• Key assumptions for latent variable models in psychometrics are local independence and unidimensionality.
• Local Independence - the unobserved variable explains all correlations between observed variables
• Unidimensionality - only one latent variable underlies all indicators/items.

Item Response Theory (IRT)

• IRT models the probability of a correct response on an item, given a person's ability level.
• It’s used for exams, educational tests, etc. and allows modeling item difficulty and discrimination between different ability levels. This contrasts with CTT which typically examines the total score.

Factor Analysis

• Factor analysis is a technique to analyze interrelationships among a set of items to identify latent variables (factors) that underlie observed variable scores. It assumes the observed variables are correlated due to their connection to shared latent factors (e.g., depression)

Structural Equation Modelling (SEM)

• SEM allows for modeling causal relationships between latent variables
• Factor analysis is a specific example of structural equation modeling
• Formative latent variables—the indicators themselves define the latent variable (e.g., socioeconomic status (SES), determined by income, occupation, and education)
• Reflective latent variables—the latent variable causes the indicators.

Network Psychometrics

• Network psychometrics sees psychological constructs as networks of interacting components
• This approach examines the unique interactions between observed variables, focusing on the network structure rather than the latent variables.

Test your knowledge on key concepts in psychometrics, focusing on various types of validity, including construct validity and concurrent criterion validity. This quiz also explores Classical Test Theory and its distinctions between test scores and true scores. It's an essential quiz for anyone studying measurement in psychology and education.