Podcast
Questions and Answers
What is a major criticism of quantitative research?
What is a major criticism of quantitative research?
What is Goodhart's Law related to?
What is Goodhart's Law related to?
What type of error occurs due to differences between sampling estimates and population parameters?
What type of error occurs due to differences between sampling estimates and population parameters?
What is a consequence of using quantitative social indicators for decision-making?
What is a consequence of using quantitative social indicators for decision-making?
Signup and view all the answers
What is a limitation of quantitative research?
What is a limitation of quantitative research?
Signup and view all the answers
What is a potential source of error in research?
What is a potential source of error in research?
Signup and view all the answers
What can lead to a static view of social life?
What can lead to a static view of social life?
Signup and view all the answers
What can be affected due to the reliance on instruments and procedures?
What can be affected due to the reliance on instruments and procedures?
Signup and view all the answers
What type of error is represented by the I-term in the equation X=T + I + e?
What type of error is represented by the I-term in the equation X=T + I + e?
Signup and view all the answers
What is a source of systematic measurement error in cross-national research?
What is a source of systematic measurement error in cross-national research?
Signup and view all the answers
Which type of reliability assesses whether measures are consistent over time?
Which type of reliability assesses whether measures are consistent over time?
Signup and view all the answers
What is the main concern of internal validity?
What is the main concern of internal validity?
Signup and view all the answers
What type of validity is concerned with whether a measure performs according to theoretical expectations?
What type of validity is concerned with whether a measure performs according to theoretical expectations?
Signup and view all the answers
What is the main concern of measurement invalidity?
What is the main concern of measurement invalidity?
Signup and view all the answers
What is the main difference between measurement error and true variation?
What is the main difference between measurement error and true variation?
Signup and view all the answers
Which type of reliability assesses whether measures are consistent between observers?
Which type of reliability assesses whether measures are consistent between observers?
Signup and view all the answers
What is the main concern of external validity?
What is the main concern of external validity?
Signup and view all the answers
What is an example of a systematic error or bias in measurement?
What is an example of a systematic error or bias in measurement?
Signup and view all the answers
What is the term for the inconsistency of a measurement instrument under repeated uses?
What is the term for the inconsistency of a measurement instrument under repeated uses?
Signup and view all the answers
What is the term for the difference between the observed variable and the true variable of interest?
What is the term for the difference between the observed variable and the true variable of interest?
Signup and view all the answers
What is an example of measurement invalidity?
What is an example of measurement invalidity?
Signup and view all the answers
What is the total value of the construct of interest, including both systematic and random error?
What is the total value of the construct of interest, including both systematic and random error?
Signup and view all the answers
The ______ approach involves using data to develop a theory, whereas the ______ approach involves using a theory to make predictions that are tested with data.
The ______ approach involves using data to develop a theory, whereas the ______ approach involves using a theory to make predictions that are tested with data.
Signup and view all the answers
In the ______ approach, data is collected in the form of numbers through instruments such as surveys.
In the ______ approach, data is collected in the form of numbers through instruments such as surveys.
Signup and view all the answers
Data quality in ______ research refers mainly to research design and the measurable quality of data.
Data quality in ______ research refers mainly to research design and the measurable quality of data.
Signup and view all the answers
The ______ cycle involves a process of deductive approach, with theory leading to observations and findings.
The ______ cycle involves a process of deductive approach, with theory leading to observations and findings.
Signup and view all the answers
In ______ research, data is collected in the form of words, and the researcher is the instrument.
In ______ research, data is collected in the form of words, and the researcher is the instrument.
Signup and view all the answers
Epistemology deals with the question of ______ can we know, whereas ontology deals with the question of what can we know.
Epistemology deals with the question of ______ can we know, whereas ontology deals with the question of what can we know.
Signup and view all the answers
[Blank] approach focuses on the use of statistics and methods to analyze data, whereas ______ approach focuses on the use of interviews, documents, and observations.
[Blank] approach focuses on the use of statistics and methods to analyze data, whereas ______ approach focuses on the use of interviews, documents, and observations.
Signup and view all the answers
The ______ approach involves testing a theory by making predictions that are then tested with data, whereas the ______ approach involves developing a theory based on observations and findings.
The ______ approach involves testing a theory by making predictions that are then tested with data, whereas the ______ approach involves developing a theory based on observations and findings.
Signup and view all the answers
The measure of ______ applies to both multiple indicators for latent variables and to single indicators.
The measure of ______ applies to both multiple indicators for latent variables and to single indicators.
Signup and view all the answers
X = ______ + I + e
X = ______ + I + e
Signup and view all the answers
______ reliability assesses whether measures are consistent over time.
______ reliability assesses whether measures are consistent over time.
Signup and view all the answers
______ validity refers to the extent to which a measure performs according to theoretical expectations.
______ validity refers to the extent to which a measure performs according to theoretical expectations.
Signup and view all the answers
______ validity refers to generalization.
______ validity refers to generalization.
Signup and view all the answers
______ error occurs due to differences in the translations in cross-national research.
______ error occurs due to differences in the translations in cross-national research.
Signup and view all the answers
______ validity is concerned with whether a measure reflects the concepts.
______ validity is concerned with whether a measure reflects the concepts.
Signup and view all the answers
______ reliability assesses whether measures are consistent between observers.
______ reliability assesses whether measures are consistent between observers.
Signup and view all the answers
The ______ stage involves collecting data to test a hypothesis.
The ______ stage involves collecting data to test a hypothesis.
Signup and view all the answers
Quantitative researchers rely heavily on ______ to collect and analyze data.
Quantitative researchers rely heavily on ______ to collect and analyze data.
Signup and view all the answers
One criticism of quantitative research is that it treats people and social institutions as part of ______.
One criticism of quantitative research is that it treats people and social institutions as part of ______.
Signup and view all the answers
According to ______'s Law, a measure becomes less reliable when it becomes a target.
According to ______'s Law, a measure becomes less reliable when it becomes a target.
Signup and view all the answers
A ______ design involves studying a single case or a small number of cases in depth.
A ______ design involves studying a single case or a small number of cases in depth.
Signup and view all the answers
Sampling error occurs due to differences between ______ estimates and population parameters.
Sampling error occurs due to differences between ______ estimates and population parameters.
Signup and view all the answers
______ error occurs when there are problems with the data collection process.
______ error occurs when there are problems with the data collection process.
Signup and view all the answers
A ______ design involves studying a phenomenon over a long period of time.
A ______ design involves studying a phenomenon over a long period of time.
Signup and view all the answers
Study Notes
Variation and Error
- Variation consists of true variation and variation due to error.
- A variable with little error has a smaller difference between true variation and variation due to error.
- A variable with considerable error has a larger difference between true variation and variation due to error.
Measurement Error
- Measurement error (ME) is how far the measured variable is free from measurement error.
- ME consists of systematic error/bias (validity issue) and random measurement error (reliability issue).
- Systematic error/bias is a validity issue, where the measure does not capture the concept of interest.
- Random measurement error is a reliability issue, where the measure is inconsistent under repeated uses.
Measurement Error and Validity
- At the empirical level, there is a distinction between the variable we want to measure (T) and the observed variable (X).
- The observed value (X) is a combination of the true value (T), invalidity (I), and random error (e).
Sources of Measurement Error
- Sources of systematic measurement error include:
- Response scales (different scale points)
- Wording of items
- Order of response categories
- Context of question
- Response styles
- Natural language differences in cross-national research
- Unjustified differences in translations
- Differences in interpretation of questions in different countries
Quality Indicators in Quantitative Business Research
- Quality indicators include:
- Reliability (are measures consistent?)
- Replication/replicability (is the study repeatable?)
- Measurement (or construct) validity (do measures reflect concepts?)
- Validity (are conclusions well-founded?)
- Internal validity (are causal relations between variables real?)
- External validity (can results be generalized beyond the research setting?)
- Ecological validity (are findings applicable?)
Types of Reliability
- Stability: is the measure stable over time? (e.g., test-retest method)
- Internal reliability: are the indicators consistent? (e.g., split-half method)
- Inter-observer consistency: is the measure consistent between observers?
- External reliability: replicability
Types of Validity
- Face validity: intuitive process
- Construct validity (or measurement validity): extent to which a measure performs according to theoretical expectations
- Internal validity: refers to causality; is X really causing Y? (linked to confounding variable bias)
- External validity: generalization
- Ecological validity: are research findings applicable?
Main Preoccupations of Quantitative Research
- Measurement
- Causality
- Generalization
- Replication
- Reliability and validity
Criticisms of Quantitative Research
- Quantitative researchers fail to distinguish people and social institutions from the natural world.
- The measurement process possesses an artificial and spurious sense of precision and accuracy.
- The reliance on instruments and procedures hinders the connection between research and everyday life.
- The analysis of relationships between variables creates a static view of social life that is independent of people's lives.
Quantitative Measures: Pitfalls
- Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure."
- Campbell's Law: "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor." (e.g., high-stakes testing; teaching to the test)
Sources of Error
- Sampling (-related) error:
- Sampling error: difference between sampling estimate and population parameter
- Sampling related error: size, selection, etc.
- Data collection error
Research Designs
- Experimental design
- Cross-sectional design
- Longitudinal design
- Case study design
- Comparative design
- Design science
Main Preoccupations of Quantitative Research
- Measurement
- Causality
- Generalization
- Replication
- Reliability and validity
Criticisms of Quantitative Research
- Failure to distinguish people and social institutions from the natural world
- Artificial and spurious sense of precision and accuracy in measurement
- Hindrance to connection between research and everyday life due to reliance on instruments and procedures
- Static view of social life that is independent of people's lives
Quantitative Measures: Pitfalls
- Goodhart's Law: a measure becomes a target, it ceases to be a good measure
- Campbell's Law: quantitative social indicator is used for social decision-making, it becomes subject to corruption pressures and distorts social processes
Sources of Error
- Sampling error: difference between sampling estimate and population parameter
- Sampling related error: size, selection, etc.
- Data collection error
Data Quality in Quantitative Research
- Data quality is a broad concept that refers to research design and measurable quality of data
- Refers to: % no opinion, differentiation in response patterns, item nonresponse, etc.
Epistemology and Ontology
- Epistemology: how can we know?
- Ontology: what can we know?
- Positivism
- Interpretivism
- Objectivism
- Constructionism
Deductive and Inductive Approach
- Deductive approach: theory → observations → findings
- Inductive approach: observations → theory → findings
- Quantitative methods: data as numbers, from instruments
- Qualitative methods: data in the form of words, from the field
Research Cycle
- Deductive approach process:
-
- Theory
-
- Data
-
- Measure of T (true value)
-
- Formula: X = T + I + e (observed value = true value + invalidity + random error)
Sources of Measurement Error (ME)
- I-term: invalidity or systematic error
- Sources of ME: response scales, wording of items, order of response categories, context of question, response styles, natural language differences, unjustified differences in translations, differences in interpretation of questions
Quality Indicators in Quantitative Business Research
- Reliability: are measures consistent?
- Replication/replicability: is study repeatable?
- Measurement (or construct) validity: do measures reflect concepts?
- Validity: are conclusions well-founded?
- Internal validity: are causal relations between variables real?
- External validity: can results be generalized beyond the research setting?
- Ecological validity: are findings applicable?
Types of Reliability
- Stability: is the measure stable over time?
- Internal reliability: are the indicators consistent?
- Inter-observer consistency: is the measure consistent between observers?
- External reliability: replicability
Types of Validity
- Face validity: intuitive process
- Construct validity (or measurement validity): extent to which a measure performs according to theoretical expectations
- Internal validity: refers to causality, linked to confounding variable bias
- External validity: generalization
- Ecological validity: are research findings applicable?
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.
Related Documents
Description
Understanding the concepts of variation and error in statistical measurements, including true variation and variation due to error, and measurement error comprising systematic and random errors.
