40 Questions
What is the purpose of conducting a study again with different samples or in different settings?
To replicate the study and ensure generalizability of the findings
Which of the following is a threat to external validity?
History
What is the term for the tendency of participants to change their behavior because they know they are being studied?
Hawthorne effect
What is the purpose of using inclusion and exclusion criteria in a study?
To reduce selection bias
Which of the following is a threat to internal validity?
History
What is the term for the consistency of a measure?
Reliability
What is the term for the accuracy of a measure?
Validity
Which of the following is NOT a threat to external validity?
Instrumentation
What is the primary purpose of checking the consistency of results across time?
To assess the reliability of a measurement
What is the main difference between a reliable and a valid measurement?
A reliable measurement is not always valid, but a valid measurement is always reliable
If a method is not reliable, what can be inferred about its validity?
It is probably not valid
What type of reliability assesses the consistency of a measure across different researchers?
Inter-rater reliability
What is an example of low reliability?
Several different doctors use the same questionnaire with the same patient but give different diagnoses
What is the purpose of assessing the reliability of a measurement?
To determine whether the measurement is reliable
What type of reliability assesses the consistency of a measure across different items?
Internal consistency
What is the relationship between reliability and validity?
Reliability is a prerequisite for validity
What should you do when devising questions or measures to improve internal consistency?
Formulate questions based on the same theory
Which type of reliability is relevant when measuring a property that is expected to stay the same over time?
Test-retest
What is validity?
How accurately a method measures what it is intended to measure
What is internal validity?
The extent to which the observed results represent the truth in the population we are studying
What is external validity?
The extent to which the observed results can be generalized to other situations
Which type of validity refers to whether the test appears to be suitable to its aims?
Face validity
What is content validity?
Whether the test is fully representative of what it aims to measure
What is criterion validity?
Whether the results correspond to a different test of the same thing
What is the primary goal when formulating questions to improve test-retest reliability?
To minimize the influence of external factors on participants' responses
How is inter-rater reliability typically measured?
By comparing the results of different researchers conducting the same measurement on the same sample
What is an example of a scenario where high inter-rater reliability is demonstrated?
A team of researchers observe the progress of wound healing in patients and there is a strong correlation between their ratings
What is a key step in improving inter-rater reliability?
Clearly defining variables and methods for measurement
What is the primary goal of internal consistency?
To measure the correlation between different responses to a set of statements
What is an example of a scenario where internal consistency is low?
A group of respondents rate high on depression indicators but the correlation between responses is very weak
What is a common method used to measure internal consistency?
Average inter-item correlation
Why is it important to ensure that all researchers have the same information and training when measuring inter-rater reliability?
To ensure that all researchers are using the same criteria and methods for measurement
What is the main threat to internal validity that occurs when participants are repeatedly tested using the same measures?
Testing
What type of validity is concerned with the extent to which study findings can be generalized to other situations, people, and settings?
External Validity
What is the term used to describe the statistical tendency for people who score extremely low or high on a test to score closer to the mean the next time?
Regression to the mean
What is the primary purpose of using statistical methods to adjust for problems related to external validity?
To do reprocessing or calibration
What is the main threat to internal validity that occurs when groups are not comparable at the beginning of the study?
Selection Bias
What is the term used to describe the process of making sure that participants experience the events of a study as a real event?
Realism
What is the term used to describe the extent to which findings of a qualitative study can be generalized to other situations, people, and settings?
Transferability
What is the main threat to internal validity that occurs as a natural result of time, such as participants growing older or becoming tired?
Maturation
Study Notes
Reliability and Validity
- Reliability refers to the consistency of a measure, whether the results can be reproduced under the same conditions
- Validity refers to the accuracy of a measure, whether the results really do represent what they are supposed to measure
Types of Reliability
- Test-retest reliability: consistency of a measure across time
- Inter-rater reliability: consistency of a measure across raters or observers
- Internal consistency: consistency of a measure across items or questions
Improving Reliability
- Formulate questions and statements to minimize external influences
- Ensure consistent testing conditions and minimize changes in participants over time
- Clearly define variables and methods for inter-rater reliability
- Use detailed, objective criteria for rating variables
- Ensure multiple researchers have the same information and training
Threats to External Validity
- Selection bias: sample not representative of the population
- History: unrelated events influence outcomes
- Experimenter effect: researchers unintentionally influence outcomes
- Hawthorne effect: participants change behavior due to being studied
- Testing effect: pre- or post-test affects outcomes
- Situation effect: setting, time of day, location, etc. limit generalizability
- Sample features: results influenced by specific sample characteristics
Validity
- Face validity: content of the test appears suitable for its aims
- Construct validity: test measures the concept it's intended to measure
- Content validity: test is fully representative of what it aims to measure
- Criterion validity: results correspond to a different test of the same thing
Ensuring Validity in Research
- Threats to internal validity: maturation, regression to the mean, testing, selection bias
- Threats to external validity: selection bias, history, experimenter effect, Hawthorne effect, testing effect, situation effect, sample features
- Use statistical methods to adjust for external validity problems
Learn about the importance of reliability and validity in measurements, and how to ensure accurate results in research and testing.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
