Psychology of Testing and Measurement

Test Theory and Reliability

• Generalizability Theory: based on the idea that a person's test scores vary from testing to testing because of variables in the testing situation.
• Universe: in the context of reliability, refers to the particular test situation leading to a specific test score.
• Coefficient of Generalizability: represents the influence of particular facts on the test score.

Decision Study and Domain Sampling

• Decision Study: designed to inform how test scores should be used and how dependable those scores are as a basis for decisions.
• Domain Sampling Model: considers the problems created by using a limited number of items to represent a larger and more complicated construct.
• Domain Sampling: refers to defining a domain that represents a single trait or characteristic, and each item is an individual sample of this general characteristic.

Item Response Theory and Classical Test Theory

• Item Response Theory: uses a computer to focus on the range of item difficulty that helps assess an individual’s ability level.
• Classical Test Theory: each person has a true score that would be obtained if there were no errors in measurement.
• Error: in terms of reliability, implies the internal consistency of tests in which the items are not scored as 0 or 1.

Test Characteristics and Types

• Dynamic characteristics: traits, states, abilities presumed to be ever-changing as a function of situational and cognitive experiences.
• Speed Test: implies that the test is homogenous, meaning it is easy but short time.
• Criterion-Referenced Tests: designed to provide an indication of where a test-taker stands with respect to some variable or criterion.
• Homogeneity: in the context of test items, means that there will always be some inaccuracy in our measurements.

Classical Test Theory Formula and Reliability Coefficient

• X = T + E: represents the observed score is the sum of the true score and the error.
• Reliability Coefficient: indicates the ratio between the true score variance on a test and the total variance.

Reliability and Test Length

• Reliability: increases when the test length increases.
• Kuder-Richardson technique: considers all possible ways of splitting the items.
• Spearman-Brown formula: allows you to estimate what the correlation between the two halves would have been if each half had been the length of the whole test.
• Cronbach Alpha: estimates the reliability of a test.