Research Methodology Quiz

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is a latent variable?

A variable whose existence is obtained through direct measurement.

A constant throughout the research process.

A variable inferred from indicators that are not directly measurable. (correct)

An observable measurement that can be quantified.

What is an example of self-report bias?

A participant forgetting their age.

Participants selecting random answers.

Participants responding in ways they believe are socially acceptable. (correct)

A researcher influencing participant responses.

What is a confound in research?

An unrelated variable that has no impact on the results.

A variable that is unintentionally manipulated during an experiment.

An expected variable that confirms the hypothesis.

A variable that may influence the outcome and complicate the results. (correct)

Which of the following concerns relates to sample size in research studies?

Underpowered studies may lead to false positives. Signup and view all the answers

What does 'correlation does not equal causation' imply?

Correlation can suggest a causal relationship, but other factors may influence both. Signup and view all the answers

What is essential for ensuring true informed consent by participants?

Participants understand the implications of algorithms used in research. Signup and view all the answers

Which of the following reflects an ethical concern regarding data accessibility?

Using data without consent may violate ethical standards. Signup and view all the answers

What is the main goal of risk minimization strategies in research?

To protect participant confidentiality and reduce potential harm. Signup and view all the answers

What is the focus of qualitative research?

Meaning Signup and view all the answers

Which of the following is a goal shared by both quantitative and qualitative researchers?

To deepen understanding of human behavior Signup and view all the answers

What does the term 'conceptual density' refer to in qualitative research?

The development and tight linkage of concepts Signup and view all the answers

Which method is driven by research questions according to the principles of qualitative research?

Inductive approach Signup and view all the answers

What is the purpose of triangulation in research?

To corroborate findings from different types of research Signup and view all the answers

What role do methods play in qualitative research?

They are tools for interpretation Signup and view all the answers

What type of data do qualitative researchers typically analyze?

Written or spoken words Signup and view all the answers

Which of the following represents a characteristic of a natural setting in qualitative research?

Diverse and real-world contexts Signup and view all the answers

What is a primary goal of ethnographic research?

To describe and interpret observable relationships in social practices. Signup and view all the answers

Which type of participant operates under pretense and is fully recognized as a member of the scene?

Complete Participant Signup and view all the answers

What limitations might researchers face when spending time in the field?

Limitations imposed by funding agencies Signup and view all the answers

What is meant by 'gatekeepers' in ethnographic research?

Individuals who can deny or grant access to a social setting. Signup and view all the answers

What is the significance of 'positionality' for a researcher in ethnography?

It refers to the researcher's understanding of their own social position and biases. Signup and view all the answers

What ethical dilemma can arise from the role of a complete participant in ethnographic research?

They may mislead participants about their intentions. Signup and view all the answers

Which researcher role is primarily invested in observation but may interact casually with participants?

Observer-As-Participant Signup and view all the answers

What is 'member validation' in the context of ethnographic research?

A process where participants contribute to the research design and findings. Signup and view all the answers

What does divergent validity assess in research measures?

Correlation with different variables. Signup and view all the answers

Which of the following is an internal validity threat related to participant behavior changes?

Hawthorne Effect Signup and view all the answers

How does ecological validity relate to research findings?

It examines if results are applicable outside the research environment. Signup and view all the answers

What is a common indicator of reliability in quantitative research?

High consistency among measures Signup and view all the answers

Which external validity threat relates to the artificial setting of a study?

Testing Interaction Signup and view all the answers

Which effect refers to the researcher’s behavior influencing the participants?

Experimenter Effect Signup and view all the answers

What concept assesses how well a variable is defined and operationalized in research?

Precision Signup and view all the answers

Which factor could lead to participant dropouts in a study?

Mortality Signup and view all the answers

What is the main purpose of a sampling interval in research?

To determine the distance between selected sample elements Signup and view all the answers

Which of the following is NOT a characteristic of stratified sampling?

Random selection from the entire population Signup and view all the answers

What is a key disadvantage of convenience sampling?

It relies solely on willing participants Signup and view all the answers

What is the main goal of purposive sampling?

To choose specific individuals based on research criteria Signup and view all the answers

Which approach is NOT recommended for increasing external validity?

Avoid measuring demographics Signup and view all the answers

What is one of the criteria necessary to establish causality?

X and Y must not be related in a spurious relationship Signup and view all the answers

Which of the following is a guideline for writing effective survey measures?

Avoid jargon and use understandable language Signup and view all the answers

What should NOT be included in survey questions according to best practices?

Terms like 'always' or 'never' Signup and view all the answers

What is the main objective of reflexivity in the interview process?

To make the researcher’s influence explicit. Signup and view all the answers

Which type of qualitative interview focuses on how participants express their own views?

Respondent Interviews Signup and view all the answers

What is a key characteristic of ethnographic interviews?

Informal and often occur in the midst of social actions. Signup and view all the answers

Which type of interview question is designed to help open up the conversation?

Introducing Questions Signup and view all the answers

Which method is noted as less intrusive than observational techniques?

Qualitative interviews Signup and view all the answers

The inverted funnel approach in qualitative interviews involves which of the following?

Beginning with specific questions leading to a general understanding. Signup and view all the answers

What is the primary goal of interviews in qualitative research?

To understand participant perspectives and social meanings. Signup and view all the answers

Which type of questions ask about things the interviewee may not have direct knowledge of?

Indirect Questions Signup and view all the answers

Study Notes

Research Orientations

Ways of knowing include experience, authority, tradition, and intuition.
Many of these rely on cognitive logic processes, which can be flawed.
Epistemology asks how we know what we know.
Necessary and sufficient conditions for knowledge must be determined.

Knowledge in social science relies on reliably observed phenomena.
Epistemology focuses on what counts as knowledge in the social sciences.

Different research questions require different methods.
Scientific/Quantitative methods include surveys and experiments.
Interpretive/Qualitative methods include interviews, focus groups, and ethnography/participant observation.

Quantitative/Deductive theory aims to create testable and falsifiable hypotheses.
Theoretical constructs in quantitative theory must be observable.
Interpretive/Inductive theory seeks plausible interpretations of social data.

Hypotheses Testing

Hypotheses propose relationships between variables.
Hypotheses commonly take the form of H1 (X is related to Y) or H2 (Group A will differ from Group B on X).
Examples include communication apprehension affecting public speaking grades or social media use affecting mediated activity time.

Quantitative Methodology

A quantitative methodology aims to produce unbiased and replicable knowledge.
Methodologies produce quantifiable results, ideally generalizable.

Elements of Quantitative Research

Understanding what is being studied (conceptualization).
How the study will be conducted (operationalization).
Who is participating (sampling).
Clarity of data collection methods (replicability).
Confidence in findings.

Variables

Variables are entities that take on different values.
Variables are the building blocks of research design.

Conceptualization

Refines and specifies abstract concepts.
Conceptualizations are working agreements, not dictionary definitions.

Language - The Problem

Key concepts in communication studies (e.g., love, leadership, satisfaction, control, culture) can be problematic due to language creating confusions and limitations in communication.
Conceptualization, indicators, and dimensions of communication apprehension include nervousness, butterflies in stomach, worry, speechlessness, fear, etc.

Quantitative Approaches: Confidence in Findings

All studies are likely flawed.
The "coffee problem" highlights how preconceived notions bias the interpretation of research findings.
Overall effects of an action (e.g., coffee consumption) may be complex, preventing clear conclusions from individual studies.
The Central Limit Theorem describes how sample statistics approximate population parameters.

Normal Distribution

Data clusters around the mean.
To gain confidence in quantitative findings:
- Carefully conceptualize the variables being studied.
- Precisely operationalize the variables.
- Employ appropriate sampling techniques.
- Ensure data collection methods allow for replication.

Variables (continued)

Variables are entities that take on diverse values.
Variables underpin research design.

Categorical Variables

Mutually exclusive categories; e.g., hair color, gender

Ordinal Variables

Ordered categories (e.g., paper grades); meaningful rank but not precise differences

Interval Variables

Ranked, meaningful differences, but no true zero point (e.g., temperature in Celsius)

Ratio Variables

Ranked, meaningful differences, and a meaningful zero point (e.g., height, weight).

Measure variables...

at the highest possible level whenever possible.

Conceptualization of Variables

Conceptualization involves defining abstract concepts in a study-specific manner.
Conceptualizations are working agreements, not dictionary definitions.
Careful analysis of study variables is essential, especially for abstract concepts.
Indicators are elements that demonstrate the presence or absence of a concept.
Variables can be multi-dimensional (e.g., communication apprehension).

Connecting Variables to Questions

Research questions propose relationships between variables, general in nature.
Hypotheses propose specific relationships between variables, usually in a directional form.

Hypotheses

Hypotheses can be correlational (e.g., a relationship between variables) or comparative (e.g., differences between groups).

Week 3: Measurements

Operationalization: The development of specific research procedures resulting in empirical observations representing those concepts in the real world.
Types of Operationalization in Social Sciences: Direct Observation, Survey/Self-Report Data.

Developing Measures

Drawbacks: May not capture the richness of experiences; structure may not match respondent experiences.
Indicators: Identifying indicators of your variable.
Crafting Items: Creating questions or items based on indicators.
Composite Measures: Using multiple items to create a comprehensive measure.

Example Scales

Likert Scale: Used for measuring attitudes or feelings.
Semantic Differential: Measures attitudes using pairs of adjectives.

Choosing a Scale

Measure data at the most sophisticated level possible.
Avoid using categorical/ordinal scales for ratio variables.
Write more items than needed to ensure reliability.
Use precise and clear language.
Keep items short.
Avoid jargon.
Use language respondents understand.
Provide specific frames (e.g., "When thinking of...").
Avoid absolutes like "always" or "never" in items.

Language

Language: Use language that respondents understand.
Specific Frames: Provide specific contexts for questions.
Avoid Absolutes: Avoid words like "always" or "never."

Respondent Considerations

Ensure respondents are willing and able to answer.
Be aware of social desirability bias.
Ensure respondents have the knowledge or experience to answer.
Consider if respondents can remember the information.

Sampling

Sample: A subset of elements or units selected from a population.
Population: The entire group you wish to generalize the results to.
Types of Population:
- All the people in the world.
- Population of Canada.
- Married couples.
- Teenagers and screen time.
- Organizations, messages, words, locations.
Why Not Use the Whole Population?: It becomes difficult to create an accurate census.
Subset of the Population: Generally, impossible to study everyone, so we study a small sample.
Goal: To get a sample that is representative of the population we wish to study.

Probability Sampling (Gold Standard)

Random Sampling: Each population member has an equal chance of being selected.
Simple Random Sampling:

Nonprobability Sampling

Why Use Nonprobability Sampling?: Probability sampling is theoretically best but practically impossible.
Convenience Sampling: Reliance on available participants, introducing biases like nonrepresentative sample and self-selection.
Purposive Sample: Also called judgmental sample, specific people chosen based on the research question.
Snowball Sample: Data from a few known members of a target population, asking them to provide other members.
Quota Sampling: Participants selected nonrandomly based on their known proportion in the population.
Are Nonprobability Samples OK?: Convenience sampling can contribute to valid inferences if representative and repeated (replication is key).

Pragmatic Approaches...

Specify the population → Consider bias → Measure demographics.

Week 4: Experimental Design

Experimental Design: Introducing some action/manipulation/treatment by the experimenter and observing the consequences.
Control: Essential for establishing causality.

Criteria for Causality

X is related to Y.
X temporally precedes Y.
X and Y are not related through some third variable.
Relationship between X and Y is not spurious.

True Experiments

Control Group: Helps control participant-related threats to internal validity (e.g., Hawthorne Effect, Placebo Effect, Maturation, Experimental Mortality).
Experimental Group: Also called an experimental condition or treatment condition.
Random Assignment: Ensures experimental and control groups are equivalent, eliminating selection bias.

Experimental Designs

Double-Blind Experiment
Manipulation Checks
Pretest-Posttest Control Group Design
Posttest Only Control Group Design
Solomon Four-Group Design
Quasi-Experimental Design
Time Series Design
Nonequivalent Control Group Design
Multiple Time Series Design
One-Shot Case Study
One-Group Pretest-Posttest
Static Group Comparison

Reliability and Validity

Reliability: Consistency of a measure (same result with repeated application).
Techniques:
- Test-Retest: Consistency over time.
- Internal Consistency:
  - Split-Half Reliability: Correlation between two subsets of items.
  - Item-Total Reliability: Correlation between individual items and the total score
Validity: Accuracy of a measure (reflects intended concept)
Types:
- Content Validity:
  - Face Validity: Measure appears to represent the concept.
  - Expert Panel Validity: Experts evaluate the measure.
- Criterion Validity:
  - Predictive Validity: Measure predicts future behavior.
  - Concurrent Validity: Measure correlates with similar measures.
- Construct Validity:
  - Convergent Validity: Measure correlates with related variables.
  - Divergent Validity: Measure does not correlate with different variables.

Threats to Validity

Internal Validity Threats: Placebo Effect, Hawthorne Effect, Maturation, Mortality, Researcher Threats.
External Validity Threats: Ecological Validity.

Week 5: Limitations of Quantitative Research

All Studies Are Wrong: How wrong? What are the limitations of the study? How much confidence should we have in the results?
Robust Findings: Findings that can be consistently replicated.
Quantitative Concerns: Ecological Validity
Threats to External Validity: Testing Interaction, Selection Interaction, History Interaction.

Week 7: Research Ethics and REBs

Technology: Makes large datasets easily searchable, scrapable, and analyzable.
Ethical Considerations: Is it ethical to use this data?
Informed Consent: Understanding Algorithms, Reading Terms of Service, Public Data, Participant Perspective
Risk Minimization Strategies: Obtain informed consent, Delete names and identifiable information.

2. Association of Internet Researchers Guidelines

Types of Research:
- Big Data from Scrapes/API
- Social Listening
- Social Analytics
- Building Dashboards
- Algorithm Design
- User Experience
- Semantic/Sentiment Analysis
- Digital Rhetoric
Venue/Platform Considerations
Legal Considerations
Ethical Considerations
Data Handling: (Data Management)
- Management, storage, and representation of data.
- Securing sensitive data (e.g., information on self-harm or criminal activity).
- Anonymization -Ethical considerations of anonymizing data.
Data Minimization: Collect and store only the necessary data necessary to answer research questions.
Legal Considerations: Compliance, adherence to Terms of Service, Avoiding legal risks, Maintaining privacy
Data Utility: Can the data answer relevant questions? Understanding the sample (data, users, etc.). Proprietary nature of social media and internet data limits generalizability for independent researchers.

3. Whose Ethics?

Ethics Legislation: Historical Context; Tuskegee Syphilis Studies, The Stanford Prison Study, Milgram’s Obedience Studies

Week 8: Qualitative Approaches

Theoretical Process: What are the theoretical processes?
Observations: What observations support these concepts?
Evolution of Concepts: Evolution of Concept
Number of Concepts: How many concepts are there?
Systematic Relationships: Are those concepts systematically related?
Conceptual Density; Development: Are the concepts well-developed?

Variation

Range of Conditions: Were the concepts observed under a range of different conditions?
Theoretical Accounting: Is the variation accounted for by the theoretical concepts?

Substantive

New Information: Does the study deliver new information?
Guidelines for Action: Does it produce guidelines for action?
Theory-Building: Are the results theory-building?
Contribution: What has society or the discipline gained by having this researcher do this project?

Qualitative Coding

Definition: Shorthand devices to label, separate, compile, and organize data.
Interpretation: Breaking data down into component parts and naming those parts.
Deductive Coding (from a developed codebook):
- Development: from theory or extant research.
- Process: often indicative of a more quantitative process.
Inductive Coding (emergent from the data):
- Source: Emergent coding from the data.
- Development: Theory develops from the codes.
- Process: often indicative of a more qualitative process.

Iterative Process

Analysis: Takes place after some data has been collected, influencing further data collection

Stages of Coding

Open Coding
Axial Coding
Selective Coding
Code Books
Explanations

Qualitative Approaches

Qualitative Ontology: Multiple Realities.
Purpose of Qualitative Research: Explain and Describe, Focus: Crafting plausible or credible explanations of social processes and human behavior

Common Methods

Grounded Theory
In-Depth Interviews
Focus Groups
Ethnography
Autoethnography
Content/Discourse Analysis

Qualitative Terms

Saturation: Point at which no new information is being discovered.
Member Validation: Participants validate the findings.
Sensitizing Concepts: Initial ideas that guide the research.
Iterative: Repeated cycles of data collection and analysis.
Rich/Thick Descriptions: Detailed understanding, providing detailed and nuanced descriptions of the research context and findings.

Week 9: Interviews

Definition: An interview develops a view of something between (inter) people (Brenner, 1985).
Purposes of the Qualitative Interview

Interview Structures

Structured Interview: (Fixed format)
Semi-Structured Interview: (Specific topics but flexible)
Unstructured Interview: (Conversational)

Interview Questions

Interview Schedule: Variety of questions
Chronological Order
Broad to Specific (Funnel)
Specific to Broad (Inverted Funnel)
Types of Questions (Introducing, Direct, Indirect, Follow Up).

Focus Groups

Definition
Purpose (collaboration; deeper understanding)
Focus Group Size (7-12 people)
Focus Group Makeup (homogenous; members not knowing each other)
Advantages
Captures real-life data
Flexible
High face validity: Non-researchers can understand the results

Week 10: Ethnography

Ethnography: Long-term immersion in a field site to write about cultures.
Ethnography Components (ethno: culture + graph: something written).
Researcher Roles (Participant? Observer? Positionality)
Characteristics:
- Time-Consuming
- Ethical questions emerge
Good Field Note Habits (Journaling, Allocating Time)
Field Types & Notes (full and scratch notes, allocates time).

Access (in Ethnography)

Preparation: How to access the site of the study and the community
Questions to Consider: What is the goal? Are they a member? What resources are needed?
Types of Access: Overt, Covert
Overt: direct permission from group leaders/members.
Covert: Don't tell participants about the research.

Week 11: Limitations of Qualitative Research

Qualitative is not Quantitative
Qualitative is Empirical
Qualitative is NOT Generalizable (thick descriptions)

Qualitative Criteria

Support for Conclusions: Does the process adequately support conclusions?
Qualitative Reporting
Theoretical Consistency
Major Categories, Events, Incidents, Actions, Relationships
Credibility Considerations
Triangulation

Qualitative Research Methodology Summary

Comparison across two or more forms of evidence.
Can be with multiple sources, multiple methods, and/or multiple researchers.
Negative Case Analysis (new data disconfirming previous explanations.)
Member Validation (checking perceptions of findings; participants)
Time Consuming
- Field Time
- Data Preparation (coding/etc).
Balance
Participant and Researcher Expertise
Writing
Trust
Avoiding Exploitation: Important to avoid community exploitation.
Audience-Dependent: The goal of qualitative research is ultimately audience-dependent.

Week 12: Transcending Quantitative and Qualitative Research

Methods as Tools (question-driven)
Analogy: "If all you have is a hammer, everything looks like a nail."
Common Goals: Deeper knowledge of human behavior and social processes

Quantitative vs Qualitative Research

Quantitative: Focus: Behavior, Approach: Deductive, Data: Numbers, Setting: Artificial
Qualitative: Focus: Meaning, Approach: Inductive, Data: Words, Setting: Natural

Multi-Strategy Research

Purpose: Use one type of research to corroborate findings from another type.
Example: Follow up on survey data with qualitative interviews, focus groups, create possible survey items.
Complementarity: Combining aspects of an investigation using different strategies (e.g., using survey methods to assess an experience and interviews to get more in-depth reactions).

Rich Understanding

Rich Understanding: Understanding symbolic and cultural engagement in depth.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Description

Test your knowledge on key concepts in research methodology. This quiz covers essential topics such as latent variables, self-report bias, and ethical concerns in research. Perfect for students studying psychology and social sciences.