Quasi-Experiments PDF

Experimental Designs: Quasi-Experiments Prepared by: Mabeth B. Francia, RGC, RPm, RPsy Quasi-experimental design is a research method that seeks to evaluate the causal relationships between variables, but without the full control over the independent variable(s) that is available in a tru...

Experimental Designs: Quasi-Experiments Prepared by: Mabeth B. Francia, RGC, RPm, RPsy Quasi-experimental design is a research method that seeks to evaluate the causal relationships between variables, but without the full control over the independent variable(s) that is available in a true experimental design. The researcher uses an existing group of participants that is not randomly assigned to the experimental and control groups. Instead, the groups are selected based on pre-existing characteristics or conditions such as age, gender or the presence of a certain condition. The purpose of quasi-experimental design is to investigate the causal relationship between two or more variables when it is not feasible or ethical to conduct a randomized controlled trial (RCT). Quasi-experimental designs attempt to emulate the randomized control trial by mimicking the control group and the intervention group as much as possible. The key purpose of quasi-experimental design is to evaluate the impact of an intervention, policy, or program on a targeted outcome while controlling for potential confounding factors that may affect the outcome. Quasi-experimental designs aim to answer questions such as: Did the intervention cause the change in the outcome? Would the outcome have changed without the intervention? And was the intervention effective in achieving its intended goals?  Types of Quasi-Experimental Design NON-EQUIVALENT CONTROL GROUP DESIGN selecting two groups of participants that are similar in every way except for the independent variable that the researcher is testing. one group receives the treatment or intervention being studied, while the other group does not. The two groups are then compared to see if there are any significant differences in the outcomes. INTERRUPTED TIME-SERIES DESIGN collecting data on the dependent variable over a period of time, both before and after an intervention or event. The researcher can then determine whether there was a significant change in the dependent variable following the intervention or event. PRETEST-POSTTEST DESIGN involves measuring the dependent variable before and after an intervention or event but without a control group. This design can be usefule for determining whether the intervention or event had an effect but it does not allow for control over other factors that may have influenced the outcomes. REGRESSION DISCONTINUITY DESIGN This design involves selecting participants based on a specific cutoff point on a continuous variable, such as a test score. Participants on either side of the cutoff point are then compared to determine whether the intervention or event had an effect. NATURAL EXPERIMENTS involves studying the effects of an intervention or event that occurs naturally, without the researcher’s intervention. This design is useful when true experiments are not feasible or ethical. A researcher might study the effects of a new law or policy that affects certain groups of people. DATA ANALYSIS METHODS DESCRIPTIVE STATISTICS involves summarizing the data collected during a study using measures such as mean, median, mode, range and standard deviation. helps researchers identify trends or patterns int eh data, and can also be useful for identifying outliers or anomalies. INFERENTIAL STATISTICS involves using statistical tests to determine whether the results of a study are statistically significant. Help researchers make generalizations about a population based on the sample data collected during the study. Common statistical tests used in quasi-experimental designs include t-test, ANOVA and regression analysis. PROPENSITY SCORE MATCHING used to reduce bias in quasi-experimental designs by matching participants in the intervention group with participants in the control group who have similar characteristics. This help reduce the impact of confounding variables that may affect the study’s results. DATA ANALYSIS METHODS DIFFERENCE-IN-DIFFERENCES ANALYSIS used to compare the difference in outcomes between two groups over time. used to determine whether a particular intervention has had an impact on the target population over time. INTERRUPTED TIME SERIES ANALYSIS used to examine the impact of an intervention or treatment over time by comparing data collected before and after the intervention or treatment. help researchers determine whether an intervention had a significant impact on the target population. REGRESSION DISCONTINUITY ANALYSIS used to compare the outcomes of participants who fall on either side of a predetermined cutoff point. helps researcher determine whether an intervention had a significant impact on a target population. WHEN TO USE QUASI-EXPERIMENTAL DESIGN When the research question involves investigating the effectiveness of an intervention, policy, or program: In situations where it is not feasible or ethical to randomly assign participants to intervention and control groups, quasi-experimental designs can be used to evaluate the impact of the intervention on the targeted outcome. When the sample size is small: In situations where the sample size is small, it may be difficult to randomly assign participants to intervention and control groups. Quasi-experimental designs can be used to investigate the impact of an intervention without requiring a large sample size. When the research question involves investigating a naturally occurring event: In some situations, researchers may be interested in investigating the impact of a naturally occurring event, such as a natural disaster or a major policy change. Quasi-experimental designs can be used to evaluate the impact of the event on the targeted outcome. WHEN TO USE QUASI-EXPERIMENTAL DESIGN When the research question involves investigating a long-term intervention: In situations where the intervention or program is long- term, it may be difficult to randomly assign participants to intervention and control groups for the entire duration of the intervention. Quasi-experimental designs can be used to evaluate the impact of the intervention over time. When the research question involves investigating the impact of a variable that cannot be manipulated: In some situations, it may not be possible or ethical to manipulate a variable of interest. Quasi- experimental designs can be used to investigate the relationship between the variable and the targeted outcome. QUASI-EXPERIMENTAL DESIGN EXAMPLES Evaluating the impact of a new teaching method: In this study, a group of students are taught using a new teaching method, while another group is taught using the traditional method. The test scores of both groups are compared before and after the intervention to determine whether the new teaching method had a significant impact on student performance. Assessing the effectiveness of a public health campaign: In this study, a public health campaign is launched to promote healthy eating habits among a targeted population. The behavior of the population is compared before and after the campaign to determine whether the intervention had a significant impact on the target behavior. Examining the impact of a new medication: In this study, a group of patients is given a new medication, while another group is given a placebo. The outcomes of both groups are compared to determine whether the new medication had a significant impact on the targeted health condition. QUASI-EXPERIMENTAL DESIGN EXAMPLES Evaluating the effectiveness of a job training program: In this study, a group of unemployed individuals is enrolled in a job training program, while another group is not enrolled in any program. The employment rates of both groups are compared before and after the intervention to determine whether the training program had a significant impact on the employment rates of the participants. Assessing the impact of a new policy: In this study, a new policy is implemented in a particular area, while another area does not have the new policy. The outcomes of both areas are compared before and after the intervention to determine whether the new policy had a significant impact on the targeted behavior or outcome.  Advantages and Limitations of Quasi-Experimental Design ADVANTAGES LIMITATIONS Greater external validity: Quasi-experimental designs Selection Bias: Quasi-experimental designs may suffer are more likely to have greater external validity than from selection bias because participants are not laboratory experiments because they are conducted in randomly assigned to groups. Participants may self- naturalistic settings. This means that the results are select into groups or be assigned based on pre-existing more likely to generalize to real-world situations. characteristics, which may introduce bias into the Ethical considerations: Quasi-experimental designs study. often involve naturally occurring events, such as natural History and Maturation: Quasi-experimental designs disasters or policy changes. This means that are susceptible to history and maturation effects, where researchers do not need to manipulate variables, the passage of time or other events may influence the which can raise ethical concerns. outcome of the study. More practical: Quasi-experimental designs are often Lack of Control: Quasi-experimental designs may lack more practical than experimental designs because they control over extraneous variables that could influence are less expensive and easier to conduct. They can the outcome of the study. This can limit the ability to also be used to evaluate programs or policies that draw causal inferences from the study. have already been implemented, which can save time and resources.  Advantages and Limitations of Quasi-Experimental Design ADVANTAGES LIMITATIONS No random assignment: Quasi-experimental designs Lack of Randomization: Quasi-experimental designs do do not require random assignment, which can be not involve randomization of participants into groups, difficult or impossible in some cases, such as when which means that the groups being studied may differ studying the effects of a natural disaster. This means in important ways that could affect the outcome of the that researchers can still make causal inferences, study. This can lead to problems with internal validity although they must use statistical techniques to control and limit the ability to make causal inferences. for potential confounding variables. Limited Generalizability: Quasi-experimental designs Greater generalizability: Quasi-experimental designs may have limited generalizability because the results are often more generalizable than experimental may only apply to the specific population and context designs because they include a wider range of being studied. participants and conditions. This can make the results more applicable to different populations and settings. Thank you for your kind attention! Reference: Hassan, M. (2024, January 4). Quasi-Experimental Research Design-Types, Methods. Researchmethod.net. https://researchmethod.net/quasi-experimental-design/