Podcast
Questions and Answers
Which type of test primarily relies on content validity?
Which type of test primarily relies on content validity?
What is the primary focus of an aptitude test?
What is the primary focus of an aptitude test?
Which type of personality test involves ambiguous stimuli or responses?
Which type of personality test involves ambiguous stimuli or responses?
What is an example of a typical performance test?
What is an example of a typical performance test?
Signup and view all the answers
Which of the following is NOT a type of structured personality test?
Which of the following is NOT a type of structured personality test?
Signup and view all the answers
Which property is essential in a psychological test and pertains to its technical quality?
Which property is essential in a psychological test and pertains to its technical quality?
Signup and view all the answers
What type of interview method allows subjects to express their feelings freely without fear of disapproval?
What type of interview method allows subjects to express their feelings freely without fear of disapproval?
Signup and view all the answers
Which type of neuropsychological test is designed to measure intervals between events and behaviors?
Which type of neuropsychological test is designed to measure intervals between events and behaviors?
Signup and view all the answers
What type of psychological assessment involves the assessor and assessee working as 'partners'?
What type of psychological assessment involves the assessor and assessee working as 'partners'?
Signup and view all the answers
What differentiates psychological testing from psychological assessment?
What differentiates psychological testing from psychological assessment?
Signup and view all the answers
Which component of non-directive interviews involves biographical sketch of the client?
Which component of non-directive interviews involves biographical sketch of the client?
Signup and view all the answers
What is the primary purpose of a psychological test item?
What is the primary purpose of a psychological test item?
Signup and view all the answers
Which type of test measures one's likes, dislikes, and personality orientation towards the world of work?
Which type of test measures one's likes, dislikes, and personality orientation towards the world of work?
Signup and view all the answers
What is the main objective of a motivational interview?
What is the main objective of a motivational interview?
Signup and view all the answers
Which type of assessment is conducted without physical proximity between the subject and evaluator?
Which type of assessment is conducted without physical proximity between the subject and evaluator?
Signup and view all the answers
What is a 'cut-score' in the context of psychological testing?
What is a 'cut-score' in the context of psychological testing?
Signup and view all the answers
Which assessment approach encourages therapeutic self-discovery?
Which assessment approach encourages therapeutic self-discovery?
Signup and view all the answers
Which component of a psychological test refers to its form, plan, and structure?
Which component of a psychological test refers to its form, plan, and structure?
Signup and view all the answers
In psychological assessment, which approach involves evaluating specific problems at the moment they occur?
In psychological assessment, which approach involves evaluating specific problems at the moment they occur?
Signup and view all the answers
What primarily distinguishes a psychological trait from a psychological state?
What primarily distinguishes a psychological trait from a psychological state?
Signup and view all the answers
Which level of interpretation involves minimal concern with intervening processes and no concern with underlying constructs?
Which level of interpretation involves minimal concern with intervening processes and no concern with underlying constructs?
Signup and view all the answers
In psychological testing, what does 'scoring' entail?
In psychological testing, what does 'scoring' entail?
Signup and view all the answers
In the context of psychological assessment, what is Extra-Test Behavior?
In the context of psychological assessment, what is Extra-Test Behavior?
Signup and view all the answers
Which of the following best describes Mechanical Prediction?
Which of the following best describes Mechanical Prediction?
Signup and view all the answers
Who is primarily responsible for creating the tests or other methods of assessment?
Who is primarily responsible for creating the tests or other methods of assessment?
Signup and view all the answers
In actuarial assessment, what predominately influences the assessor's judgment?
In actuarial assessment, what predominately influences the assessor's judgment?
Signup and view all the answers
Which level of interpretation involves the development of a coherent and inclusive theory of the individual's life?
Which level of interpretation involves the development of a coherent and inclusive theory of the individual's life?
Signup and view all the answers
What is one of the main purposes of psychological traits, according to the assumptions?
What is one of the main purposes of psychological traits, according to the assumptions?
Signup and view all the answers
Which of the following is true about hypothetical constructs in psychological testing?
Which of the following is true about hypothetical constructs in psychological testing?
Signup and view all the answers
In psychological assessment, who controls the distribution of tests?
In psychological assessment, who controls the distribution of tests?
Signup and view all the answers
What type of variance is attributable to irrelevant random sources?
What type of variance is attributable to irrelevant random sources?
Signup and view all the answers
Which type of error can consistently affect the true score without changing the standard deviation?
Which type of error can consistently affect the true score without changing the standard deviation?
Signup and view all the answers
What term refers to variation among items within a test as well as to variation among items between tests?
What term refers to variation among items within a test as well as to variation among items between tests?
Signup and view all the answers
What is one effect of having a short test-retest interval?
What is one effect of having a short test-retest interval?
Signup and view all the answers
What is meant by 'practice effect' in the context of test-retest reliability?
What is meant by 'practice effect' in the context of test-retest reliability?
Signup and view all the answers
What is a common result when the time span between test-retest occasions is lengthy?
What is a common result when the time span between test-retest occasions is lengthy?
Signup and view all the answers
When does test sophistication primarily occur?
When does test sophistication primarily occur?
Signup and view all the answers
Which of the following contributes to inconsistency in test scores?
Which of the following contributes to inconsistency in test scores?
Signup and view all the answers
What factor is NOT typically associated with measurement error?
What factor is NOT typically associated with measurement error?
Signup and view all the answers
Which formula is used to calculate the true score?
Which formula is used to calculate the true score?
Signup and view all the answers
Which of the following accurately describes the relationship between reliability and the Standard Error of Measurement (SEM)?
Which of the following accurately describes the relationship between reliability and the Standard Error of Measurement (SEM)?
Signup and view all the answers
Which of the following is NOT used to increase the internal consistency of a test?
Which of the following is NOT used to increase the internal consistency of a test?
Signup and view all the answers
What does the Standard Error of the Difference help determine?
What does the Standard Error of the Difference help determine?
Signup and view all the answers
Which term refers to the generalizability of research results?
Which term refers to the generalizability of research results?
Signup and view all the answers
What does a higher confidence level in a confidence interval imply?
What does a higher confidence level in a confidence interval imply?
Signup and view all the answers
What aspect of validity is concerned with the degree to which test items represent a target domain?
What aspect of validity is concerned with the degree to which test items represent a target domain?
Signup and view all the answers
What does test sensitivity specifically detect?
What does test sensitivity specifically detect?
Signup and view all the answers
What outcome is characterized by predicting success that does not occur?
What outcome is characterized by predicting success that does not occur?
Signup and view all the answers
What does the term face validity refer to?
What does the term face validity refer to?
Signup and view all the answers
Which statement best describes internal validity?
Which statement best describes internal validity?
Signup and view all the answers
Which method assesses agreement between two or more raters with a categorical assessment scale?
Which method assesses agreement between two or more raters with a categorical assessment scale?
Signup and view all the answers
What does the Domain Sampling Theory aim to estimate?
What does the Domain Sampling Theory aim to estimate?
Signup and view all the answers
In Item Response Theory, what does the term 'Discrimination' refer to?
In Item Response Theory, what does the term 'Discrimination' refer to?
Signup and view all the answers
Which theory is concerned with the variables in testing situations causing score variations?
Which theory is concerned with the variables in testing situations causing score variations?
Signup and view all the answers
Which type of test has a time limit long enough to allow test takers to attempt all questions?
Which type of test has a time limit long enough to allow test takers to attempt all questions?
Signup and view all the answers
Standard Error of Measurement is designed to provide what?
Standard Error of Measurement is designed to provide what?
Signup and view all the answers
What does the 'Decision Study' in Generalizability Theory examine?
What does the 'Decision Study' in Generalizability Theory examine?
Signup and view all the answers
What does a high degree of internal consistency in a homogenous test indicate?
What does a high degree of internal consistency in a homogenous test indicate?
Signup and view all the answers
Which type of error provides an index of the expected error in an individual's test score?
Which type of error provides an index of the expected error in an individual's test score?
Signup and view all the answers
According to Generalizability Theory, what would result in the exact same test score being obtained?
According to Generalizability Theory, what would result in the exact same test score being obtained?
Signup and view all the answers
Who developed the methodology of factor analysis?
Who developed the methodology of factor analysis?
Signup and view all the answers
Which type of factor analysis involves testing the degree to which a hypothetical model fits the actual data?
Which type of factor analysis involves testing the degree to which a hypothetical model fits the actual data?
Signup and view all the answers
Which type of rating error involves the tendency to cluster ratings in the middle of the scale?
Which type of rating error involves the tendency to cluster ratings in the middle of the scale?
Signup and view all the answers
What term is used to describe the revalidation of a test to a criterion based on a different group?
What term is used to describe the revalidation of a test to a criterion based on a different group?
Signup and view all the answers
Which method is used to overcome rating errors by using rankings instead of ratings?
Which method is used to overcome rating errors by using rankings instead of ratings?
Signup and view all the answers
What is the name of the table that provides an estimate of the usefulness of a test in a selection system?
What is the name of the table that provides an estimate of the usefulness of a test in a selection system?
Signup and view all the answers
Which type of validity is assessed by using factor analysis to study the interrelationships among a set of variables?
Which type of validity is assessed by using factor analysis to study the interrelationships among a set of variables?
Signup and view all the answers
What is the term for preferential treatment that prevents impartial measurement in a test?
What is the term for preferential treatment that prevents impartial measurement in a test?
Signup and view all the answers
Which error occurs when a rater is excessively strict in scoring?
Which error occurs when a rater is excessively strict in scoring?
Signup and view all the answers
What basic element in utility analysis is described as both economic and noneconomic terms?
What basic element in utility analysis is described as both economic and noneconomic terms?
Signup and view all the answers
What is the primary concern of a test blueprint in psychological assessment?
What is the primary concern of a test blueprint in psychological assessment?
Signup and view all the answers
Which type of validity is concerned with the use of a test score to infer an individual's most probable standing on some measure of interest?
Which type of validity is concerned with the use of a test score to infer an individual's most probable standing on some measure of interest?
Signup and view all the answers
What does a zero Content Validity Ratio (CVR) indicate according to Lawshe's formula?
What does a zero Content Validity Ratio (CVR) indicate according to Lawshe's formula?
Signup and view all the answers
What is the main objective of examining incremental validity in the context of psychological testing?
What is the main objective of examining incremental validity in the context of psychological testing?
Signup and view all the answers
Which of the following is a method for improving the homogeneity of a test containing dichotomous items?
Which of the following is a method for improving the homogeneity of a test containing dichotomous items?
Signup and view all the answers
What is a primary characteristic of a criterion in validity testing?
What is a primary characteristic of a criterion in validity testing?
Signup and view all the answers
What does construct-irrelevant variance refer to?
What does construct-irrelevant variance refer to?
Signup and view all the answers
Which type of evidence demonstrates that test scores vary predictably based on group membership?
Which type of evidence demonstrates that test scores vary predictably based on group membership?
Signup and view all the answers
Which type of validity evidence would show a low correlation between the test being validated and other unrelated variables?
Which type of validity evidence would show a low correlation between the test being validated and other unrelated variables?
Signup and view all the answers
What term describes a failure to capture important components of a construct in a test?
What term describes a failure to capture important components of a construct in a test?
Signup and view all the answers
What is the main purpose of cumulative scoring in psychological assessment?
What is the main purpose of cumulative scoring in psychological assessment?
Signup and view all the answers
According to Assumption 3, what does test-related behavior aim to predict?
According to Assumption 3, what does test-related behavior aim to predict?
Signup and view all the answers
What does the reliability coefficient measure?
What does the reliability coefficient measure?
Signup and view all the answers
Which of the following is NOT considered a potential source of error variance?
Which of the following is NOT considered a potential source of error variance?
Signup and view all the answers
In Classical Test Theory, what does 'E' represent in the formula $X = T + E$?
In Classical Test Theory, what does 'E' represent in the formula $X = T + E$?
Signup and view all the answers
Competent test users understand the limitations of tests and:
Competent test users understand the limitations of tests and:
Signup and view all the answers
How is psychological reliability defined?
How is psychological reliability defined?
Signup and view all the answers
Which of the following best describes error variance?
Which of the following best describes error variance?
Signup and view all the answers
What does it mean when a test is said to be free from errors?
What does it mean when a test is said to be free from errors?
Signup and view all the answers
Error in psychological assessment refers to:
Error in psychological assessment refers to:
Signup and view all the answers
What is the main challenge associated with Parallel Forms/Alternate Forms Reliability?
What is the main challenge associated with Parallel Forms/Alternate Forms Reliability?
Signup and view all the answers
Which error is specifically associated with Split-Half Reliability?
Which error is specifically associated with Split-Half Reliability?
Signup and view all the answers
Which of the following tools are used for evaluating the coefficient of stability?
Which of the following tools are used for evaluating the coefficient of stability?
Signup and view all the answers
Homogeneity in tests is crucial for which type of reliability measurement?
Homogeneity in tests is crucial for which type of reliability measurement?
Signup and view all the answers
Under what condition should the KR-21 formula be used?
Under what condition should the KR-21 formula be used?
Signup and view all the answers
Why is counterbalancing used in psychological assessments?
Why is counterbalancing used in psychological assessments?
Signup and view all the answers
Which technique is suitable for testing internal consistencies of a test that contains non-dichotomous items?
Which technique is suitable for testing internal consistencies of a test that contains non-dichotomous items?
Signup and view all the answers
When is it impractical or undesirable to assess reliability with two tests or to administer a test twice, which reliability is useful?
When is it impractical or undesirable to assess reliability with two tests or to administer a test twice, which reliability is useful?
Signup and view all the answers
What statistical tool is used along with Pearson R for evaluating Parallel Forms/Alternate Forms Reliability?
What statistical tool is used along with Pearson R for evaluating Parallel Forms/Alternate Forms Reliability?
Signup and view all the answers
Which measure focuses on the degree of differences that exist between item scores to evaluate internal consistencies?
Which measure focuses on the degree of differences that exist between item scores to evaluate internal consistencies?
Signup and view all the answers
Study Notes
Psychological Assessment
Definition of Psychological Testing
- process of measuring psychology-related variables by means of devices or procedures designed to obtain a sample of behavior
- numerical in nature, individual or by group, administrators can be interchangeable without affecting the evaluation, requires technician-like skills in terms of administration and scoring, yield a test score or series of test scores, minutes to few hours
Psychological Assessment
- gathering and integration of psychology-related data for the purpose of making psychological evaluation
- answers referral question thru the use of different tools of evaluation
- individual, assessor is the key to the process of selecting tests and/or other tools of evaluation, requires an educated selection of tools of evaluation, skill in evaluation, and thoughtful organization and integration of data, entails logical problem-solving that brings to bear many sources of data assigned to answer the referral question
Types of Psychological Assessments
Ecological Momentary Assessment
- "in the moment" evaluation of specific problems and related cognitive and behavioral variables at the very time and place that they occur
Collaborative Assessment
- the assessor and assessee may work as "partners" from initial contact through final feedback
Therapeutic Assessment
- therapeutic self-discovery and new understanding are encouraged
Dynamic Assessment
- describe interactive approach to psychological assessment that usually follows the model: evaluation > intervention of some sort > evaluation
Psychological Test
- device or procedure designed to measure variables related to psychology
- components: content, format, item, administration procedures, score, scoring, cut-score, psychometric soundness, psychometrics, psychometrist or psychometrician
Ability or Maximal Performance Test
- Assess what a person can do
- types: achievement, aptitude, intelligence
Typical Performance Test
- Measure usual or habitual thoughts, feelings, and behavior
- indicate how test takers think and act on a daily basis
- use interval scales, no right and wrong answers
Personality Test
- measures individual dispositions and preferences
- designed to identify characteristic
- types: structured, projective, attitude, interest inventories
Other Tests
- speed, power, values inventory, trade, neuropsychological, norm-referenced, criterion-referenced
Interview
- method of gathering information through direct communication involving reciprocal exchange
- types: standardized/structured, non-standardized/unstructured, semi-standardized/focused, non-directive
Non-Directive Interview Components
- mental status examination, intake interview, social case, employment interview, panel interview, motivational interview, portfolio, case history data, case study, groupthink, behavioral observation
Assumptions of Psychological Assessment
-
- Psychological traits and states exist
-
- Psychological traits and states can be quantified and measured
-
- Test-related behavior predicts non-test-related behavior
-
- Test and other measurement techniques have strengths and weaknesses
-
- Various sources of error are part of the assessment process
-
- Testing and assessment can be conducted in a fair and unbiased manner
-
- Testing and assessment benefit society
Reliability
- dependability or consistency of the instrument or scores obtained by the same person when re-examined with the same test on different occasions, or with different sets of equivalent items
- estimate the range of possible random fluctuations that can be expected in an individual's score
- free from errors, minimizing error, using only representative sample to obtain an observed score
- true score cannot be found, reliability coefficient, index of reliability
Classical Test Theory
- score on an ability test is presumed to reflect not only the testtaker's true score on the ability being measured but also the error
- error: refers to the component of the observed test score that does not have to do with the testtaker's ability
- errors of measurement are random
Data Collection and Interpretation
- data collection, data interpretation, hit rate, profile, actuarial assessment, mechanical prediction
Levels of Interpretation
- level I: minimal amount of any sort of interpretation
- level II: descriptive generalizations, hypothetical construct
- level III: the effort to develop a coherent and inclusive theory of the individual life or a "working image" of the patient
Parties in Psychological Assessment
- test author/developer, test publishers, test reviewers, test users, test takers, test sponsors, society### Reliability Estimates
- Provide an estimate of the amount of error inherent in an observed score or measurement
- Used to estimate or infer the extent to which an observed score deviates from a true score
- Higher reliability, lower Standard Error of Measurement (SEM)
Types of Reliability
Parallel Forms Reliability
- Established when at least two different versions of the test yield almost the same scores
- Has the most universal applicability
- Alternate Forms: simply different versions of a test constructed to be parallel
Internal Consistency (Inter-Item Reliability)
- Used when tests are administered once
- Measures the internal consistency of the test, which is the degree to which each item measures the same construct
- Error: Item Sampling Homogeneity
- Statistical Tools:
- KR-20 for inter-item consistency of dichotomous items
- KR-21 for tests with equal variances and dichotomous scores
- Cronbach's Coefficient Alpha for non-dichotomous items with unequal variances
- Average Proportional Distance for internal consistency of a test
Split-Half Reliability
- Obtained by correlating two pairs of scores obtained from equivalent halves of a single test administered once
- Useful when it is impractical or undesirable to assess reliability with two tests or to administer a test twice
Validity
- A judgment or estimate of how well a test measures what it is supposed to measure
- Inferences: logical result or deduction
- Degree to which the measurement procedure measures the variables to be measured
- Validation: the process of gathering and evaluating evidence about validity
- Types of Validity:
- Face Validity: a test appears to measure what it is supposed to measure
- Content Validity: a test represents a adequate sample of a defined universe of situations or behaviors
- Criterion Validity: a test score is used to infer an individual's most probable standing on a measure of interest
- Construct Validity (Umbrella Validity): a judgment about the appropriateness of inferences drawn from test scores regarding individual standing on a variable called construct
Error
Scorer Differences
- Degree of agreement or consistency between two or more scorers regarding a particular measure
- Evaluated by calculating the percentage of times that two individuals assign the same scores to the performance of the examinees
Systematic Error
- Error that results from a flaw or bias in the measurement instrument or testing procedure
Domain Sampling Theory
- A theory that estimates the extent to which specific sources of variation under defined conditions are contributing to the test scores
- Generalizability Theory: a theory that considers the problem created by using a limited number of items to represent a larger and more complicated construct
Item Response Theory
- A theory that focuses on the probability that a person with a certain ability will be able to perform at a certain level on a test
- Difficulty: an attribute of not being easily accomplished, solved, or comprehended
- Discrimination: the degree to which an item differentiates among people with higher or lower levels of the trait, ability, etc.
Standard Error of Measurement
- Provides a measure of the precision of an observed test score
- Index of the amount of inconsistent or the amount of the expected error in an individual's score
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.
Description
A review quiz for psychological assessment, covering topics such as psychometric properties and more. Based on sources from Cohen & Swerdlik, Kaplan & Saccuzzo, and Groth & Wright.
