Quasi-Experimental Research Design PDF

Quasi- experiment al research design Group 2 Arooj Ihtisham Rida Muskan Bakh Noor Khan Faryal Shakir CONTENT Quasi-experimental research design Characteristics of Quasi-experimental design Process of Quasi-experimental...

Quasi- experiment al research design Group 2 Arooj Ihtisham Rida Muskan Bakh Noor Khan Faryal Shakir CONTENT Quasi-experimental research design Characteristics of Quasi-experimental design Process of Quasi-experimental design Types of Quasi-experimental design Threats to internal validity in quasi-experimental designs Advantages and Limitations  AROOJ IHTISHAM 007969  Quasi-experimental design The quasi means “resembling.” Thus quasi-experimental research is research that resembles experimental research but is not true experimental research. Although the independent variable is manipulated, participants are not randomly assigned to conditions or orders of conditions (Cook & Campbell, 1979). A quasi-experimental design aims to establish a cause-and-effect relationship between an independent and dependent variable. This design is commonly used in fields like psychology, education, health, and social sciences. However, the lack of randomization makes controlling for external variables challenging, so researchers must account for possible biases and threats to validity.  CHARACTERISTICS 1. Absence of 2. Comparative 3. Focus on 5. Use of Pre- 6. Natural Random Analysis: Quasi- Causality:Quasi- Existing or Settings: Assignment: experiments involve experiments aim to Naturally Conducted in Groups are pre- comparing groups, identify causal Formed real-life settings, existing rather than typically a treatment relationships. Groups: Groups making them randomly assigned, group (exposed to & are often based more reflective of such as different the intervention) and 4. Pretest and on pre-existing actual behaviors schools or a or comparison Post-test conditions or or outcomes. departments. group. Measurements characteristics.  Process of Quasi-Experimental Design 1. Define Research 2. Choose the 3. Select Groups 4. Pretest Question: Design Type and Variables Clearly define the Choose a quasi- Select pre-existing Conduct a pretest to study's aims, Does a experimental design groups and determine establish a baseline new teaching method based on feasibility variables to measure for comparison improve students and ethics. against post- score? intervention results. 5. Implement the 6. Post-test 7. Analyze and Intervention Interpret Results Implement the intervention to Measure the outcome in both Analyze and interpret results the treatment group groups post-intervention, using statistical analysis to determine the intervention's effect. Types of Quasi-Experimental Design 1. One Group design 2. The Non equivalent control group design 3. Time series design 4. Interrupted time series design 5. Regression Discontinuity Design (RDD) 6. Natural experiment. 7. Combination design. One Group Design When only one group is observed the study lacks a comparison group. There are two types of One group design. 1) One group post test only design 2) One group pre test post test design  One group post test only design The type of quasi experimental most Scenario: A fitness instructor introduces susceptible to threats to internal validity is the a new workout program and wants to one group post test only design (Campbell & evaluate its effectiveness. Stanley, 1966). In this, a researcher measures Intervention (X): The instructor implements a 6-week workout program for a group of a dependent variable for one group of participants. participants following a treatment Posttest (O): At the end of the program, the limitations instructor measures participants' fitness levels No Baseline Comparison: It’s unclear if using a standardized fitness test (e.g., how the participants improved or were already at a many push-ups they can complete in one similar fitness level before the program. Symbolic Representation minute). No Control Group: Difficulty in identifying intervention or external factors. X O  Pre test pro test design In the One-Group pretest post-test design is a research method used to measure the effects of an intervention by comparing results from before (pretest) and after (post-test) the intervention. Symbolic Representation Scenario: A teacher wants to test if a new teaching method improves students' math skills. O₁ X O₂ Pretest(O₁): Students take a math test before the new teaching method is implemented. Intervention(X): The teacher uses the new teaching method for one month. Posttest(O₂): Students take the same (or a similar) math test after one month. Measurement: The teacher compares the scores from the pretest Example Data Representation Example: Let’s say the math test scores for a group of students are as follows: Student Pretest (O₁) Posttest (O₂) Change (O₂ - O₁) 1 60 75 +15 2 70 80 +10 3 65 85 +20 Average 65 80 +15 Analysis of Measurement Mean Difference: 1. Average pretest score = 65 2. Average posttest score = 80 3. Improvement = 80 - 65 = +15 points FARYAL SHAKIR 006273  Non-equivalent Control Group A Design quasi-experimental design involving both an experimental and a control group, but without random assignment to these groups. This quasi-experimental design includes a treatment group and a non-equivalent control group that does not receive the treatment. Characteristics: Pretest-Posttest Comparison: Both groups are tested before and after the intervention. No Randomization: Groups may differ on various characteristics due to lack of random allocation. Symbolic Representation Where: O1, O3 = Pretests for both groups O1 X O2 O2, O4 = Posttests for both groups O3 X O4 X = Treatment/intervention continue... Uses: Often used in education, healthcare, and social sciences where randomization is not feasible. Example: Studying the effect of a new teaching method on students in two separate classrooms. Often used in education, healthcare, and social sciences where randomization is not feasible. Example: Studying the effect of a new teaching method on students in two separate classrooms. Strengths: Can be conducted in natural environments. Allows for comparisons even without experimental control. Limitations: Selection Bias: Differences between groups may affect outcomes. Difficult to control for external or confounding variables. Time Series Design Involves taking measurements of a variable at regular intervals before and after an intervention to observe trends. This design involves multiple observations before and after the treatment for a single group. Characteristics: Longitudinal Approach: Data is collected multiple times. Focus on Trends: Evaluates gradual or abrupt changes following an intervention. Symbolic Representation Where: O1, O2, O3 = Observations before the treatment O4, O5, O6 = Observations after the treatment O 1 O 2 O 3 X O 4 O5 O6 X = Treatment/intervention continue... Uses: Suitable for tracking progressive effects in one group over time. Example: Monitoring customer satisfaction monthly after implementing a service change. Strengths: Provides rich data to understand the effect of interventions over time. Helps identify seasonal or periodic trends. Limitations: Requires frequent, consistent data collection. Results may be influenced by external events or trends unrelated to the intervention. Interrupted Time Series Design A specific time series design where an intervention creates a clear “interruption” in the data, separating pre- and post-intervention periods. This design is a variation of the time series design where the intervention occurs as a significant event (an “interruption”) in the timeline of measurements. Characteristics: Interruption Point: Intervention is introduced at a specific point in time. Causal Analysis: Compares pre-intervention and post-intervention trends. Symbolic Representation O 1 O 2 O 3 X O 4 O5 O6 continue... Uses: Common in policy evaluations to assess the impact of laws or programs. Example: Examining crime rates before and after a curfew policy is implemented. Strengths: Provides stronger causal inferences compared to other non-random designs. Captures both immediate and long-term effects of interventions. Limitations: Confounding Variables: Results may be impacted by events coinciding with the intervention. Requires extensive baseline data for accurate trend comparisons. Key Differences Between Designs Non-equivalent Interrupted Control Group Time Series Time Series Design: Design: Design: Compares two groups Observes one group over Similar to time series but but lacks multiple time points. focuses on the effect of an randomization. Focuses on trends within intervention introduced at a Focuses on differences the same group specific point. between experimental and control groups. RIDA MUSKAN 006264 Regression discontinuity Design (RDD) A quasi-experimental design that assigns treatment based on a cutoff or threshold. Individuals just above and below the cutoff are compared to estimate causal effects. key features: Strict assignment (above the cutoff = treatment, below = no treatment). The design compares participants just above and just below the cutoff to isolate the treatment effect. Example1: A university offers a scholarship to students who score above 85% on an entrance exam. Researchers compare students who scored 84.9% to those who scored 85.1% to estimate the impact of the scholarship on their subsequent academic performance. Example 2: A company offers performance bonuses to employees whose monthly sales exceed $10,000. Researchers could compare the performance and job satisfaction of employees earning just above $10,000 to those earning just below to estimate the effect of the bonus. Regression discontinuity Design (RDD) Strengths: Provides strong causal evidence near the cutoff. High internal validity, as the comparison groups are assumed to be nearly identical except for the treatment. Limitations: Results may only apply to individuals near the cutoff, limiting external validity. Risk of manipulation of the cutoff (e.g., employees reporting inflated sales to qualify for the bonus). Natural experiment A natural experiment occurs when researchers take advantage of naturally occurring events or circumstances that mimic random assignment, without direct control by the researcher. Key Features: Researchers study real-world interventions or events where the treatment is assigned naturally (e.g., policy changes, natural disasters). Participants are exposed to the treatment based on circumstances beyond their control. Example 1: Smoking bans implemented in different regions provide a natural experiment for studying the effects on public health. Researchers can compare smoking-related diseases before and after the ban in areas with and without the legislation. Example 2: Minimum wage increase in a specific state allows researchers to compare employment rates in that state before and after the policy change, while comparing with a neighboring state where the policy was not implemented. Strengths: Real-world applicability, as the treatment reflects natural conditions. High external validity, since the findings can generalize to real-world situations. Limitations: Less control over confounding variables. It can be challenging to isolate the treatment effect due to external factors influencing the outcome. Combination design Combines multiple quasi-experimental techniques to strengthen the causal inference and address threats to validity (e.g., combining non-equivalent groups with pretest-posttest designs). Key Features: Integrates multiple designs (e.g., non-equivalent control group, interrupted time series) to control for biases and increase confidence in results. Allows for cross-validation of findings using different methods. Example 1: A government program to provide free tutoring to students in low-income schools. Researchers compare test scores of students who received the tutoring to a control group from another school district. They also measure changes before and after the program started using a pretest-posttest design. Example 2: A new public transportation initiative is launched in a city to reduce traffic congestion. Researchers compare traffic patterns in areas with the new service to areas without it, using both a pretest-posttest design (before and after the program) and a control group (a nearby city that didn't implement the program). Strengths: Increases robustness by controlling for more confounders. More reliable causal inference, especially when single designs are insufficient. Limitations: Can be complex to implement and analyze. Requires more data and resources for proper execution. Final cmparison RDD is most useful Natural experiments Combination designs are the when you have a are ideal for most comprehensive clear cutoff for studying real-world approach, using multiple assigning phenomena where quasi-experimental treatment, and you random assignment methods to strengthen want to estimate is not possible, causal inference and effects for those such as policy address a wider range of near the threshold. changes or natural potential biases. disasters. BAKHT NOOR KHAN 006267  Internal validity and External validity Internal validity External validity Internal validity refers to the degree to which a External validity refers to the extent to which the study can establish a causal relationship results of a study can be generalized between the independent variable and the beyond the specific conditions of the dependent variable, free from confounding research, including different populations, factors. It focuses on whether the observed settings, or times. It addresses how effects are genuinely due to the applicable the findings are to real-world intervention and not other variables. situations. Threats to internal validity Threats to internal validity are factors that can interfere with the ability to establish a clear cause-and-effect relationship in a study. Common threats include: 1. Selection Bias 2. History Effects 3. Maturation 4. Testing Effects 5. Instrumentation 6. Regression to the Mean 7. Attrition (Mortality) 8. Diffusion of Treatment 9. Compensatory Rivalry 10. Compensatory Equalization 11. Experimenter Bias  Selection Bias Maturation Differences between groups being Natural changes in participants over compared may exist before the time (e.g., aging, fatigue) may intervention, affecting the outcomes. account for observed effects. History Effects Testing Effects External events occurring during the Repeated testing or exposure to the study may influence the results, same test may influence unrelated to the intervention. participants' performance (e.g., practice or familiarity). Instrumentation Attrition (Mortality) Changes in measurement tools, Loss of participants over time can lead to techniques, or observers during biased results if those who drop out differ the study can affect the results. systematically from those who remain. Regression to the Mean Diffusion of Treatment Extreme scores on a pretest are likely When participants in different groups interact, to move closer to the average on the intervention's effects may "spread" to subsequent testing, independent the control group. of the intervention. Compensatory Rivalry Experimenter Bias Control group participants may alter Researchers' expectations or interactions their behavior to compete with may inadvertently influence the intervention group, affecting participants’ responses or outcomes. outcomes. Compensatory Equalization Researchers or facilitators may provide additional resources or attention to the control group to "balance" perceived inequalities. Advantages of Quasi experiment 1. Quasi-experiments design can be perfect to determine what is best for the population (external validity). 2. It gives the researchers the power over the variables by being able to control them. 3. Quasi experiment methods can be combined with other experimental methods too. 4. They are less expensive and require fewer resources compared with individual randomized controlled trials (RCTs) or cluster randomized trials. Therefore, quasi- experimental studies may also be more generalizable and have better external validity. 5. It provides transferability to a greater extent. 6. It is intuitive process that is well shaped by the researchers. 7. Involve real world problems and solutions and not any artificial ones. 8. Offers better control over the third variable known as the confounding variable which influences the cause and effect. Disadvantages of Quasi experiment 1. Quasi-experimental designs do not use random sampling in constructing experimental and control groups. Using non-uniform comparison groups can limit the generalization of the findings because non-controlled variables may have influenced the results. 2. It demands more time and resources. 3. No control over extraneous variables influencing the dependent variables. 4. It has less internal validity than true experiment. 5. It has scope for human errors. 6. It can allow the researchers personal bias to get involved. 7. Using old or backdated data can be incorrect and inadequate for the study. 8. Absence of control group and absence of control over research settings makes the result less reliable and weak for the establishment of a causal relationship between IV and DV. 9. Results are highly subjective due to the possibility of human error. Because experimental research requires specific levels of variable control, it is at a high risk of experiencing human error at some point during the research. Thank you!