Psychometrics and Measurement Reliability

Systematic Error

• Refers to a constant or proportionate error in measurement.

Theory of Reliability

• Combines sampling error and correlation in measurement.
• Examines the impact of using a limited number of items to represent a larger concept.
• Conceptualizes reliability as the ratio of observed score variance to long-run true score variance.

Sources of Error Variance

• Test Construction: Variation among items within a test and between tests influences reliability.
• Test Administration: Factors like attention and motivation during test administration can affect reliability.

Domain Sampling Model

• This model suggests that more items lead to higher reliability.

Test Score Theory: Classical Test Theory (CTT)

• Known as the true score model.
• Formula: X = T + E (Observed score = True score + Error)
• Standard Error of Measurement: Represents the standard deviation of errors.
• Assumption: Error distribution is constant across individuals.
• Limitations: Requires identical test administration for each person.

Sources of Information About Tests

• Test Catalogues: Briefly describe tests but lack detailed technical information.
• Test Manuals: Provide comprehensive information on test development and technical details.
• Reference Volumes: Regularly updated volumes offering detailed test information (e.g., Mental Measurements Yearbook).
• Journal Articles: Include test reviews, psychometric soundness analyses, and application examples.

Early Experimental Psychologists

• Early focus on identifying commonalities, not individual differences.
• Individual differences were seen as sources of error.
• Key figures:
  • Johan Friedrich Herbart: Proposed mathematical models of the mind.
  • Ernst Heinrich Weber: Introduced concepts of sensory thresholds and Just Noticeable Differences (JND).
  • Gustav Theodor Fechner: Studied mathematical sensory thresholds and founded psychophysics.

Intelligence Testing

• Alfred Binet: Commissioned to develop a test for placing Paris schoolchildren into special classes.
• Binet-Simon Scale: The first intelligence test, standardized and using a sample for calibration.
• Lewis Madison Terman: Translated the Binet-Simon scale into English, published as the Stanford-Binet Intelligence Scale, and introduced the concept of IQ.
• Robert Yerkes: Commissioned by the US Army to develop structured tests of human abilities.
  • Army Alpha: Verbal test for literate soldiers.
  • Army Beta: Non-verbal test for illiterate soldiers.

Interest in Mental Deficiency

• Jean Etienne Esquirol: A French physician who distinguished insanity from mental retardation.
• Edouard Seguin: Pioneered training for individuals with mental retardation and rejected the notion of incurable retardation.
• Emil Kraepelin: Developed examinations for evaluating individuals with emotional impairments.

Charles Spearman

• Investigated the relationship between intelligence and visual acuity.
• Developed the concept of a general intelligence factor (g), present in all intellectual activities.

Karl Pearson

• Student of Galton.
• Invented the correlation coefficient formula (Pearson's r).

James McKeen Cattell

• First to use the term “mental test”.
• Linked measures of simple discrimination, perception, and association to independent estimates of intellectual level.
• Influenced by Galton's work.

Guy Montrose Whipple

• Influenced by Fechner and Titchner.
• Expanded the use of correlational methods.
• His seminar prompted the APA to issue its first standards for professional psychological testing.

Louis Leon Thurstone

• Contributed significantly to factor analysis.
• Developed the Law of Comparative Judgment as a measurement approach.

Personality Testing

• These tests aim to measure personality traits, with validity coefficients.
• First structured test:
• Rorschach Inkblot Test (RIT): Developed by Hermann Rorschach and used to assess personality.
• Thematic Apperception Test (TAT): Developed in 1935 and involves telling stories based on ambiguous pictures.

Evaluating Validity Coefficient

• Consider:
  • Changes in the cause of relationships.
  • The validity and reliability of the criterion.
  • The representativeness of the subject population in the validity study.
  • An adequate sample size for cross-validation.

Construct Validity

• Construct: An abstract scientific idea or hypothesis explaining behavior.
• Construct validation: Gathering evidence about the meaning and relationship of a test to other measures.
• Convergent Validity: Positive correlation between a test and established measures of the same construct.
• Evidence:
  • Consistency between the test and other measures.
  • Showing the test measures the same thing as established tests.
  • Moderate to high correlation coefficients.

Criterion vs. Predictor

• The criterion is the standard/desired outcome, while the predictor is a variable that affects the criterion.

This quiz covers key concepts in psychometrics, focusing on systematic errors, the theory of reliability, and sources of error variance in test scores. It also introduces the classical test theory and the domain sampling model. Test your understanding of these essential topics in measurement and reliability!