True Scores Generalizability Theory Quiz

Reliability and Generalizability

• Test reliability is an objective measure of how precisely the test score reflects the domain from which the test draws a sample
• True score is a person's genuine ability level as measured by a particular test, which would be obtained if there were no errors
• Errors of measurement are assumed to be random, and the distribution of random errors is bell-shaped

Generalizability Theory

• Universe score is a person's test scores that vary from testing to testing due to variables in the testing situation
• Facets of the universe include the number of items in the test, the amount of training the test scorers have had, and the purpose of the test administration
• Coefficients of generalizability represent the influence of particular facets on the test score, similar to reliability coefficients in the true score model

Decision Study

• A decision study involves the application of information from a generalizability study to examine the usefulness of test scores in helping the test user make decisions
• It tells the test user how test scores should be used and how dependable those scores are as a basis for decisions, depending on the context of their use

Classical Test Theory (CTT)

• Also referred to as the true score model of measurement, it is the most widely used and accepted model in the psychometric literature today due to its simplicity
• Each person has a true score that would be obtained if there were no errors

Test Scores and Errors

• Observed scores are determined by true scores and measurement error
• A reliable test gives scores that closely approximate true scores, while a valid test gives scores that closely approximate construct scores
• The lower the test's reliability, the lower the test's validity
• A test can be reliable but not valid, such as a deeply flawed test that gives consistent measurements