Podcast
Questions and Answers
What are the five characteristics of effective selection techniques?
What are the five characteristics of effective selection techniques?
Test-Retest reliability is the extent to which:
Test-Retest reliability is the extent to which:
What does 'temporal stability' refer to in the context of Test-Retest reliability?
What does 'temporal stability' refer to in the context of Test-Retest reliability?
Temporal stability refers to the consistency of test scores over time. This means that the test scores should be stable even when administered on different days or weeks, indicating that the test is not significantly influenced by random factors like illness or fatigue.
What is the purpose of 'counterbalancing' in the context of Alternate Forms reliability?
What is the purpose of 'counterbalancing' in the context of Alternate Forms reliability?
Signup and view all the answers
What is the extent to which the scores on two forms of a test are similar called?
What is the extent to which the scores on two forms of a test are similar called?
Signup and view all the answers
Internal consistency measures how consistently an applicant responds to similar items.
Internal consistency measures how consistently an applicant responds to similar items.
Signup and view all the answers
What are the four methods used to assess internal consistency?
What are the four methods used to assess internal consistency?
Signup and view all the answers
What is the extent to which two people scoring a test agree on the test score called?
What is the extent to which two people scoring a test agree on the test score called?
Signup and view all the answers
What is the degree to which inferences from test scores are justified by the evidence called?
What is the degree to which inferences from test scores are justified by the evidence called?
Signup and view all the answers
What are the five common strategies to investigate the validity of scores on a test?
What are the five common strategies to investigate the validity of scores on a test?
Signup and view all the answers
What does 'content validity' measure?
What does 'content validity' measure?
Signup and view all the answers
What does 'criterion validity' measure?
What does 'criterion validity' measure?
Signup and view all the answers
A test's reliability implies validity.
A test's reliability implies validity.
Signup and view all the answers
What does 'face validity' measure?
What does 'face validity' measure?
Signup and view all the answers
What strategy assumes that tests predicting specific job components can apply to similar jobs with shared components?
What strategy assumes that tests predicting specific job components can apply to similar jobs with shared components?
Signup and view all the answers
What does 'validity generalization' refer to?
What does 'validity generalization' refer to?
Signup and view all the answers
What are the three pieces of information needed to use the Taylor-Russell tables?
What are the three pieces of information needed to use the Taylor-Russell tables?
Signup and view all the answers
What does 'selection ratio' refer to?
What does 'selection ratio' refer to?
Signup and view all the answers
What is the 'base rate' of performance and how can it be obtained?
What is the 'base rate' of performance and how can it be obtained?
Signup and view all the answers
What is the name of the utility method that compares the percentage of times a selection decision was accurate with the percentage of successful employees?
What is the name of the utility method that compares the percentage of times a selection decision was accurate with the percentage of successful employees?
Signup and view all the answers
What does the Brogden-Cronbach-Gleser Utility Formula estimate?
What does the Brogden-Cronbach-Gleser Utility Formula estimate?
Signup and view all the answers
What is the term used to describe the technical aspects of a test that show group differences in test scores unrelated to the construct being measured?
What is the term used to describe the technical aspects of a test that show group differences in test scores unrelated to the construct being measured?
Signup and view all the answers
What term describes the situation in which a test predicts job success falsely favoring one group over another?
What term describes the situation in which a test predicts job success falsely favoring one group over another?
Signup and view all the answers
What term describes the situation where a test is significantly valid for two groups but more valid for one than the other?
What term describes the situation where a test is significantly valid for two groups but more valid for one than the other?
Signup and view all the answers
What are the two choices an organization has if differential group validity occurs?
What are the two choices an organization has if differential group validity occurs?
Signup and view all the answers
What method of selection is used when applicants are ranked by test scores and the top scorers are hired until all open positions are filled?
What method of selection is used when applicants are ranked by test scores and the top scorers are hired until all open positions are filled?
Signup and view all the answers
What is 'rule of three' used for?
What is 'rule of three' used for?
Signup and view all the answers
What is the purpose of passing scores in the context of selection?
What is the purpose of passing scores in the context of selection?
Signup and view all the answers
A multiple-cutoff approach is used when one score cannot compensate for another.
A multiple-cutoff approach is used when one score cannot compensate for another.
Signup and view all the answers
What method of selection focuses on reducing the cost of testing?
What method of selection focuses on reducing the cost of testing?
Signup and view all the answers
What is the name of the statistic used to account for error in test scores?
What is the name of the statistic used to account for error in test scores?
Signup and view all the answers
What is banding used for in the selection process?
What is banding used for in the selection process?
Signup and view all the answers
Flashcards
Reliability
Reliability
Consistency and freedom from error in a test or evaluation score.
Test-Retest Reliability
Test-Retest Reliability
Consistency of test scores when the same test is given twice.
Temporal Stability
Temporal Stability
Consistency of test scores over time.
Alternate-Forms Reliability
Alternate-Forms Reliability
Signup and view all the flashcards
Form Stability
Form Stability
Signup and view all the flashcards
Counterbalancing
Counterbalancing
Signup and view all the flashcards
Internal Reliability
Internal Reliability
Signup and view all the flashcards
Scorer Reliability
Scorer Reliability
Signup and view all the flashcards
Trait Anxiety
Trait Anxiety
Signup and view all the flashcards
State Anxiety
State Anxiety
Signup and view all the flashcards
Test-Retest Interval
Test-Retest Interval
Signup and view all the flashcards
Reliability Coefficient
Reliability Coefficient
Signup and view all the flashcards
Study Notes
Evaluating Selection Techniques and Decisions
- Effective selection techniques are reliable, valid, cost-efficient, fair, and legally defensible.
Characteristics of Effective Selection Techniques
- Effective selection techniques possess five key characteristics: reliability, validity, cost-efficiency, fairness, and legal defensibility.
Reliability
- Reliability measures the consistency of a test or evaluation. It considers how consistent a score from a test is, eliminating errors.
- Test reliability is determined in four ways:
- Test-retest reliability
- Alternate-forms reliability
- Internal reliability
- Scorer reliability
Test-Retest Reliability
- This method evaluates the consistency of test results over time.
- Scores from an initial test administration are compared to scores from a subsequent administration of the same test.
- A high correlation signifies good temporal stability. An example of a test that would use this would be a 'trait anxiety' test.
- A lower correlation coefficient indicates unstable scores over time.
- There is no fixed time interval between test administrations. The interval must balance the need to prevent test-taking practice effects and avoid time-related changes in the person being tested.
- The typical test-retest reliability coefficient for organizational tests is .86.
Alternate-Forms Reliability
- This method measures the consistency of test scores across different versions of the same test.
- Two parallel forms of the same test are constructed
- Scores from both forms are correlated to establish form stability.
- Counterbalancing is often used to control for order effects
Internal Reliability
- This measures how consistently an applicant answers similar test items.
- This approach focuses on items measuring the same concepts and reduces careless mistakes, as well as other issues.
- Factors affecting internal consistency include item homogeneity and test length. Longer tests often yield higher consistency scores because they include more items.
- The median internal reliability coefficient is .81, and coefficient alpha is the most frequently reported measure of this form of reliability in many studies.
Scorer Reliability
- This method checks how consistent different test scores are when evaluated by different scorers.
- Scorer reliability is important for tests with subjective elements, like projective tests, as well as even objective tests.
- Factors influencing this form of reliability include interrater reliability and consistency of test scoring.
Evaluating Test Validity
- Validity is the degree to which inferences from test scores are justified by the evidence.
- Reliability does not automatically equal validity.
- Key factors include reliability coefficients and test populations.
- Valid test populations have similar demographics as those being tested.
- Example, NEO personality scales have a lower reliability for males.
Evaluating Test Validity: 5 Strategies
- Content Validity: How well a test sample covers the intended content. (e.g. a final exam should cover all chapters).
- Criterion Validity: To what degree test scores relate to job performance.
- Example types: Concurrent validity (scores of current employees), Predictive validity (scores of job applicants to future job performance).
- Construct Validity: Whether the test measures intended constructs.
- Convergent validity: Measures similar, related constructs.
- Discriminant validity: Shows low correlations with dissimilar constructs.
- Known-group validity: Scores of known groups of different traits are compared.
- Validity Generalization: The extent to which a test is valid across different jobs.
- Synthetic Validity: Tests predicting specific job performance elements can be applied to similar jobs having shared components.
Challenges and Considerations
- Test validity is specific to an occupation. Correlation coefficients might not appear convincing in real-world applications.
- Conducting criterion validity studies can present challenges if results are not favorable.
Finding Reliability and Validity Information
- Resources like the Mental Measurement Yearbook and Tests in Print provide validity data.
- Cost-effectiveness is essential in using tests. Sometimes a cheaper test might be preferable.
Advancements in Testing
- Computer-assisted testing is gaining use to increase efficiency and cost-savings.
Establishing the Usefulness
- "Even when a test is reliable and valid, it is not necessarily useful."
Formulas to Determine How Useful A Test Is
- Taylor-Russell Tables: used to estimate success in the future after the hiring company uses them.
- Selection Ratio: the percentage of people a company will hire
- Base Rate: the percentage of employees currently on the job who succeed.
- Lawshe Table: determines the probability of an applicant future success based on the test results, validity coefficient, and base rate.
- Proportion of Correct Decisions (PCD) Formula: compares the percentage of accurate selection decisions with the percentage of successful employees to ascertain the accuracy of the test.
Brogden-Cronbach-Gleser Utility Formula
- Determines the degree to which an organization benefits from a selection system.
- Includes the number of employees hired, employee tenure, test validity, performance SD, and the average standardized predictor score.
Determining the Fairness of a Test
- Measurement bias
- Predictive bias
Making the Hiring Decision
- Unadjusted Top-Down Selection: used to rank applicants from the highest to the lowest scores. If there are multiple tests, score from one test can compensate a poor score from another.
- Rule of Three: the top three applicants, based on their test scores, are presented to the hiring manager.
- Passing Scores: helps in reducing adverse impact and increasing selection flexibility, this method suggests determining the lowest score in a test that predicts good performance on the job.
- Banding: considers the error associated with test scores. If there is only a few point difference between tested candidates, the difference may be attributed to chance rather than to true differences in ability.
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.