Psychological Assessment: Lesson 08

Understanding Reliability

• Reliability indicates consistency in measurement, reflecting the degree to which a test yields stable and consistent results.
• The reliability coefficient measures the ratio of true score variance to total variance, providing an index of reliability.

Sources of Error Variance

• Scorers and scoring systems can introduce error variance.
• Error sources include forgetting, misunderstanding instructions, and reporting biases (underreporting or overreporting).

Reliability Estimates

• Reliability can be estimated through three approaches:
  • Test-retest: Compares scores of the same individuals on two separate test administrations.
  • Alternate or parallel forms: Assesses consistency between different test versions measuring the same ability.
  • Internal or inter-item consistency: Evaluates the consistency of responses across items within the same test.

Test-Retest Reliability

• Best suited for stable abilities (e.g., personality traits) over time.
• Increased time between tests may lower reliability coefficients; typically, intervals over six months denote the coefficient of stability.

Impact of Test Administration and Environment

• Test construction variations impact reliability; effectively sampling items is crucial.
• Test environment factors (temperature, lighting, noise) can influence results, particularly in tests assessing perceptual judgments.

Testtaker Variables

• Factors such as emotional state, physical health, sleep quality, and medications can affect test performance and reliability.

Examiner-Related Variables

• The examiner's demeanor and appearance, as well as the interaction with the examinee, may also introduce variability in test scores.

Parallel-Forms and Alternate-Forms Reliability

• Evaluated through the degree of similarity between different forms of a test, calculated via an alternate-forms reliability coefficient.
• Steps to assess include dividing the test into equivalent halves and calculating the Pearson correlation coefficient between the scores.