Week 4: Experimental Design PDF

**Week 4: Experimental Design ** Chapter 2: research designs - Chapter 2 focuses on the strategic choices researchers make when selecting their specific research design - It explains that a research design is a framework guiding the process of data collection and analysis. - Which...

**Week 4: Experimental Design ** Chapter 2: research designs - Chapter 2 focuses on the strategic choices researchers make when selecting their specific research design - It explains that a research design is a framework guiding the process of data collection and analysis. - Which is crucial because the design chosen depends on the research goals and the types of questions being asked. - **For example**, researchers must decide whether their goal is to identify causal relationships, understand how phenomena evolve over time, or explore the meanings attached to social actions.  - The chapter makes a distinction between **nomothetic explanations** - **Nomothetic explanations:** Aim to establish generalizable principles **(common in quantitative research)** - **Idiographic explanations**: provide rich, detailed descriptions of specific cases **(common in qualitative research).** The choice between these approaches heavily influences the research design. - One key research design discussed in the chapter is **quasi-experiments** - Which share some characteristics with true experiments but lack random assignment of participants to experimental and control groups - **Quasi-experiments** are often used in real-world settings - Such as in evaluations of policy changes or organizational innovations - For example, researchers might study the effects of an earthquake on civic pride by comparing an effected city with an unaffected one - While these types of studies can offer external validity (i.e., they examine real world conditions) - They are also limited by challenges like pre-existing differences between the groups being compared. - **Cross-sectional designs:** involve collecting data at a single point in time - Cross-sectional designs is useful for identifying relationships between variables but cannot prove causality  - For example: a study might show that people with higher education tend to have higher incomes - But it cannot determine whether education causes higher income or if other factors are at play. - Cross-sectional studies are beneficial for understanding current social conditions but have limitations in establishing causal relationships. - **Longitudinal designs:** which follow participants over time and allow researchers to track changes in variables, making it easier to establish causal relationships. **There are 2 main types of longitudinal studies:** - **Panel studies:** which track the same individuals or groups over time - **Cohort studies:** which focus on a specific group with a shared experience (e.g., individuals born in the same year) **Lecture on this week:** - Talks about reliability and validity in the context of experimental research **Reliability: is about consistency whether the result of a study would be the same if repeated** - This concept is crucial in ensuring that research measurements are stable over time. - **Test-retest reliability:** which ensures that results are consistent over time - **Internal consistency:** which examines whether items on a scale are correlated - **Item-total reliability:** measures whether individual items on a scale correlate with the total score  - **Reliability coefficients (such as cronbach's alpha - so the weird a):** quantity the degree of consistency. A higher coefficient indicates higher reliability, with values close to 1.00 considered ideal **Validity:** refers to how accurately a study measures what it intends to measure - **Content validity**: ensures that a measure covers all relevant aspects of a concept - **Criterion validity:** ensures that the measure aligns with other established measures - **Predictive validity** (whether a measure can predict future behavior) - **Concurrent validity** (whether scores on a measure align with scores from similar measures - **Construct validity:** examines whether a measure fits within the theoretical framework it is supposed to represent - **Convergent validity:** (where the measure correlates with related variables) - **Discriminant validity:** (where it does not correlate with unrelated variables) The lecture then moves on to potential threats to experimental validity **Internal validity** concerns whether the results of an experiment can be attributed to the manipulation of the independent variable, without interference from other factors - Threats to internal validity include the placebo effect - **Placebo effect:** where participants change their behavior simply because they know they are part of an experiment  - **Hawthorne effect:** where participants alter their behavior because they are being observed - Other threats include **maturation (natural changes over time)** - **And mortality: where participants drop out of the study, potentially skewing the results** - The lecture also discusses observer bias: where researcher may unintentionally behave differently toward patients, affecting the results - Additionally, **researcher attribute effects:** highlight how characteristics of the researchers (such as gender, race, or demeanor) can influence the outcomes of the study **External validity:** - Deals with the generalizability of the study results beyond the simple Threats to external validity include: - Testing interaction: where the artificial setting of an experiment may affect how participants behave - Selection interaction: where the sample may differ from the broader population - History interaction: refers to how the time period of the study could influence its results To address these issues, researchers can use **creative procedures or field experiments, which take place in a more natural setting to increase ecological validity.** **Connection between lecture and chapters:** - The Chapter and Lecture are connected in how they emphasize the importance of careful research design to ensure the reliability and validity of findings.  - The chapter outlines various research designs, such as quasi-experiments, cross-sectional, and longitudinal studies, and discusses their strengths and weaknesses.  - These designs are influenced by issues of validity and reliability, as highlighted in the lecture.  - The lecture provides a deeper understanding of how reliability (consistency) and validity (accuracy) affect the interpretation of experimental results.  - Both the chapter and the lecture stress that a sound research design not only helps in answering research questions but also ensures that the conclusions drawn are valid and reliable.  - Together, they demonstrate that the quality of data collection and analysis is paramount to making informed and accurate conclusions in social research.