Psychometrics and Classical Test Theory
58 Questions
106 Views

Choose a study mode

Play Quiz
Study Flashcards
Spaced Repetition
Chat to Lesson

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is the field of study concerned with the measurement of psychological attributes and characteristics?

  • Psychometrics (correct)
  • Cognitive Psychology
  • Behavioral Science
  • Sociology

What is the purpose of psychometrics?

To design and evaluate reliable and valid psychological measures

What does Classical Test Theory rest on?

True scores

What is a true score (T)?

<p>An individual's test score measured without error</p> Signup and view all the answers

Classical Test Theory assumes T is ____ distributed on a normal distribution.

<p>randomly</p> Signup and view all the answers

An alternative way to think about the true score is to imagine an individual taking the same test ____ number of times.

<p>an infinite</p> Signup and view all the answers

How do we infer the values of true scores?

<p>By administering appropriate test stimuli to individuals and calculating an observed score, denoted X.</p> Signup and view all the answers

What will the observed score be?

<p>An imperfect measure of the true score</p> Signup and view all the answers

The discrepancy between true and observed scores is known as ____ error, denoted e.

<p>measurement</p> Signup and view all the answers

What is the observed score equation?

<p>X=T+e</p> Signup and view all the answers

What is measurement error?

<p>A product of any factor(s) that causes discrepancy between true and observed scores.</p> Signup and view all the answers

Match the following measurement error assumptions:

<p>Error is randomly distributed = True Expected value of error is 0 = True Random errors are uncorrelated with each other = True No systematic relation between the true score and error = True</p> Signup and view all the answers

What does the mean of the error term being 0 imply about the mean of the true scores?

<p>Equals the mean of observed scores</p> Signup and view all the answers

What is the expected variance of the observed score?

<p>The variance of the true score plus the error variance</p> Signup and view all the answers

What is the Standard Error of Measurement (SEM)?

<p>The standard deviation of the distribution of random errors around the true score for an individual</p> Signup and view all the answers

If an individual took the same test an infinite number of times, SEM would be the ____ of their collected scores.

<p>standard deviation</p> Signup and view all the answers

What is reliability in Classical Test Theory (CTT)?

<p>Precision or consistency of measurement</p> Signup and view all the answers

How can we define reliability?

<p>As the proportion of variance in the observed score that is due to the construct being measured</p> Signup and view all the answers

What can reliability be thought of as?

<p>The squared correlation between the observed and true score</p> Signup and view all the answers

What is test-retest reliability?

<p>An administration of the same test to the same participant on 2 or more separate occasions</p> Signup and view all the answers

What is an estimate of reliability?

<p>A correlation coefficient between test scores at the 2 time points</p> Signup and view all the answers

What does Classical Test Theory definition of reliability assume?

<p>Error variance does not differ across the 2 sessions</p> Signup and view all the answers

What is split-half reliability?

<p>Used in data from a single session to calculate a correlation between scores on 2 randomly selected halves of a test</p> Signup and view all the answers

What is Cronbach's alpha (alternative formula)?

<p>alpha= (N * r)/(1+(N-1) * r)</p> Signup and view all the answers

What does the alternative formula for Cronbach's alpha make clear?

<p>Reliability is a function of test length and item covariance</p> Signup and view all the answers

What tends to happen to reliability with more items?

<p>Increase</p> Signup and view all the answers

What are Cronbach's alpha scores?

<p>alpha &gt; 0.7 considered good; alpha &gt; 0.9 may be an indicator of redundancy</p> Signup and view all the answers

What do the reliabilities of 2 measures place constraints on?

<p>The maximum correlation you can observe between the 2 measures</p> Signup and view all the answers

What happens if you have 1 or 2 poor measures?

<p>You will be underestimating the true relationship between these variables</p> Signup and view all the answers

What is validity?

<p>The degree to which a test measures what it claims/intends to measure</p> Signup and view all the answers

What is face/content validity?

<p>Whether the measure appears to measure what it should as judged by experts or experienced clinicians</p> Signup and view all the answers

What is criterion validity?

<p>Extent to which a variable is related to a relevant outcome</p> Signup and view all the answers

What is concurrent validity?

<p>The measure is compared against another measure that is a standard criterion for that construct, where both measures are assessed at the same time</p> Signup and view all the answers

What is predictive validity?

<p>Score on a measure is compared with a score obtained on a future criterion</p> Signup and view all the answers

Correlations are used to assess the degree of ____ validity of a measure.

<p>criterion</p> Signup and view all the answers

What is construct validity?

<p>Examines the degree to which a measure of a given construct correlates with measures of a similar construct and measures of dissimilar constructs</p> Signup and view all the answers

What is convergent validity?

<p>A new scale of depression should correlate with a standard, widely-used measure of depression</p> Signup and view all the answers

What is discriminant validity?

<p>This measure of depression should not correlate with a separate measure known to be unrelated</p> Signup and view all the answers

What does a multitrait-multimethod matrix measure?

<p>Multiple traits with multiple methods</p> Signup and view all the answers

What does monotrait-heteromethod index?

<p>Convergent validity</p> Signup and view all the answers

What does heterotrait-monomethod reflect?

<p>High correlation reflects method, indexes divergent validity if traits are supposed to be different</p> Signup and view all the answers

What are two basic ways items can differ?

<p>Item difficulty; Item discrimination</p> Signup and view all the answers

How is item difficulty defined for dichotomous items?

<p>By the number of people who get a particular item correct or choose a particular option</p> Signup and view all the answers

What does item discrimination compare?

<p>The relationship between performance on an item and performance on the whole test</p> Signup and view all the answers

What are the 3 basic ways to measure item discrimination?

<p>Extreme Group Method; Item-Total Correlation; Inter-Item Correlation</p> Signup and view all the answers

What does the Extreme Group Method involve?

<p>Take the top and bottom 1/3 of test scores and calculate the discrimination index</p> Signup and view all the answers

What is Item Total Correlation?

<p>The correlation between the score on an item and the total test score</p> Signup and view all the answers

What can Inter-Item Correlation show?

<p>What items positively or negatively relate to other items</p> Signup and view all the answers

What is the Item-Characteristic Curve?

<p>Graphical presentation of the proportion of people who got the item right versus the total test score</p> Signup and view all the answers

What are the steps of drawing an Item-Characteristic Curve (ICC)?

<p>Calculate total scale score for everyone, group individuals based on total score, and compute proportions for item correct responses.</p> Signup and view all the answers

_____ slopes mean an item is more discriminating than _____ slopes.

<p>Steeper; flatter</p> Signup and view all the answers

What is Item Response Theory?

<p>A modern measurement theory designed to model individual or group responses to items based on levels of a latent trait.</p> Signup and view all the answers

What are the advantages of Item Response Theory?

<p>Allows sample and item invariant measurement and can estimate SEM/reliability across different levels.</p> Signup and view all the answers

In ICC, what does p stand for?

<p>Probability of responding to an item</p> Signup and view all the answers

What does high discrimination mean?

<p>Probability of response increases more rapidly as ability increases</p> Signup and view all the answers

What is the Test Information Function?

<p>Plots the amount of information relative to ability</p> Signup and view all the answers

What does Item Response Theory show?

<p>Differential item functioning across different groups who take the same test</p> Signup and view all the answers

What is an example of Differential Item Functioning (DIF) Analysis?

<p>Females and males might perform differently on an item even if they have the same level of ability</p> Signup and view all the answers

Flashcards

Psychometrics

The science behind measuring psychological qualities, like personality traits or intelligence.

Classical Test Theory (CTT)

A theory that assumes we have a true, hidden score for a trait, and we only see this score imperfectly through tests.

Observed Score

The score you get on a test, influenced by both your actual ability and random error.

True Score

The perfect score that reflects your true ability, free from any mistakes.

Signup and view all the flashcards

Measurement Error

Mistakes that happen when measuring psychological traits, like getting tired during a test or misunderstanding instructions.

Signup and view all the flashcards

Reliability

How consistent and reliable a test is, ensuring it measures what it should consistently.

Signup and view all the flashcards

Test-Retest Reliability

Measuring the same thing multiple times and comparing the results.

Signup and view all the flashcards

Split-Half Reliability

Dividing a test in half and seeing if the scores on both halves are similar.

Signup and view all the flashcards

Cronbach's Alpha

A statistic used to measure the internal consistency of a test. It reflects how closely related the items in a test are.

Signup and view all the flashcards

Validity

Whether a test actually measures what it claims to measure.

Signup and view all the flashcards

Face/Content Validity

Does the test appear to measure what it's supposed to measure? (i.e., content experts agree).

Signup and view all the flashcards

Criterion Validity

Does the test score correlate with other established measures of the same construct?

Signup and view all the flashcards

Predictive Validity

Does the test score predict future performance on a related criterion? (i.e., predicting job success).

Signup and view all the flashcards

Construct Validity

Whether a test measures the underlying concept it claims to measure. Convergent validity refers to the degree to which the test correlates with measures of similar constructs, while discriminant validity refers to the degree to which the test does not correlate with measures of different constructs.

Signup and view all the flashcards

Item Analysis

Analyzing individual test items to understand their difficulty and ability to discriminate between high and low scorers.

Signup and view all the flashcards

Item Difficulty

How difficult an item is, shown as the percentage of people who answer correctly.

Signup and view all the flashcards

Item Discrimination

How well an item separates those who score high on the test from those who score low. Higher discrimination means the item is more effective at distinguishing between high and low scorers.

Signup and view all the flashcards

Item-Characteristic Curve (ICC)

A visual graph showing how likely someone is to get an item right based on their overall score on the test.

Signup and view all the flashcards

Item Response Theory (IRT)

A more advanced theory for measuring traits, taking into account how individual differences affect item responses.

Signup and view all the flashcards

Differential Item Functioning (DIF)

Examining if different groups (e.g., gender, ethnicity) have different chances of answering an item correctly, even with equal ability.

Signup and view all the flashcards

Extreme Group Method

Compares the performance of those who score very high and very low on a test to check how good an item is at discriminating.

Signup and view all the flashcards

Item-Total Correlation

Calculates how well a single item correlates with the overall score on a test. Higher correlation suggests a better item.

Signup and view all the flashcards

Inter-Item Correlation

Measures the relationship between different items in a test. Strong correlations mean the items are similar, while weak correlations suggest a diverse set of items.

Signup and view all the flashcards

Study Notes

Psychometrics

  • Field dedicated to measuring psychological attributes and characteristics.
  • Aims to develop reliable and valid psychological assessments.

Classical Test Theory (CTT)

  • Fundamental premise is the concept of true scores, representing an individual's score without measurement error.
  • True scores are assumed to be normally distributed within the population.
  • Infers true scores by calculating observed scores through testing.

Measurement Concepts

  • Observed scores (X) are derived from the true score (T) and measurement error (e), represented by the equation X = T + e.
  • Measurement error reflects discrepancies between true and observed scores, influenced by various factors like fatigue or distractions.
  • Assumptions include random distribution of error, zero expected value of error, and no correlation between true scores and errors.

Reliability

  • Defined as the consistency and precision of measurement in assessments.
  • High reliability indicates that test items reliably represent the overall construct being measured.
  • Calculated as the proportion of observed score variance attributed to the true construct.

Types of Reliability

  • Test-retest reliability measures consistency of scores over time.
  • Split-half reliability assesses internal consistency by correlating scores of two halves of the same test.
  • Cronbach's alpha is a reliability coefficient, with values above 0.7 deemed acceptable, and above 0.9 indicating potential redundancy.

Validity

  • Validity gauges whether a test effectively measures what it is intended to.
  • Includes face/content validity, criterion validity (subdivided into concurrent and predictive validity), and construct validity.
  • Construct validity examines correlations between similar and dissimilar constructs, emphasizing convergent and discriminant validity.

Item Analysis

  • Items may vary based on difficulty (how many can answer correctly) and discrimination (how well they correlate with total test performance).
  • Item-Characteristic Curve (ICC) visually represents the success rates on test items against total scores.
  • Item Response Theory (IRT) enhances measurement by accounting for how individual traits influence responses.

Differential Item Functioning (DIF)

  • Analyzes how different groups respond to test items, important for ensuring fairness.
  • Example: May identify performance differences between genders on specific test items even when abilities are equal.

Summary of Key Methods

  • Extreme Group Method evaluates item discrimination by analyzing performance in the highest and lowest scoring groups.
  • Item-Total Correlation assesses an item’s effectiveness by correlating it with total test scores.
  • Inter-Item Correlation shows how items relate to one another, facilitating identification of strong or weak items in an assessment.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Quiz Team

Description

Explore the fundamental concepts of psychometrics, including the principles of classical test theory (CTT) and measurement concepts. This quiz will assess your understanding of true scores, observed scores, measurement error, and the importance of reliability in psychological assessments.

More Like This

Classical Test Theory and Domain Sampling Theory Quiz
35 questions

Teoría de Muestreo de Dominio: Prueba y Flashcards

ResourcefulStatueOfLiberty avatar
ResourcefulStatueOfLiberty
Klassieke Testtheorie en Meetfouten
18 questions

Klassieke Testtheorie en Meetfouten

SleekElectricOrgan avatar
SleekElectricOrgan
Classical Test Theory in Psychometrics
5 questions

Classical Test Theory in Psychometrics

RighteousCyclops avatar
RighteousCyclops
Classical Test Theory Overview
17 questions

Classical Test Theory Overview

InstrumentalSnail avatar
InstrumentalSnail
Use Quizgecko on...
Browser
Open
Browser
Browser