Research Design: Experimental Design Principles

Symbolism for Diagramming Experimental Designs

• X represents the exposure of a group to an experimental treatment.
• 0 represents the observation or measurement of the dependent variable; subscripts indicate temporal order.
• R denotes the random assignment of test units, where individuals are randomly assigned to experimental groups.

RCT vs. Controlled Trial (Quasi-Experiments)

• When research questions deal with "What happens if...?", randomized assignment to groups is considered.
• Subjects can be randomly assigned to groups in experimental designs.
• Subjects cannot be randomly assigned to groups in quasi-experimental designs.
• In both experimental and quasi-experimental designs, pretest data either can or cannot be collected which leads into after-only or pretest-posttest designs.

Quasi or Pre-Experimental Designs

• These designs do not adequately control for problems associated with loss of external or internal validity.
• They cannot be classified as true experiments.
• These designs are often used in exploratory research and are not considered causal.
• Examples includue: one-shot design, one-group pretest-posttest design, and static group design.

One-Shot Design

• It is also known as "after-only design".
• A single measure is recorded after treatment administration.
• Studies lack comparison or control of extraneous influences.
• Taste tests may be the only viable choice.
• It is diagrammed as X O1

One-Group Pretest-Posttest Design

• Subjects in the experimental group are measured before and after treatment.
• There is no control group.
• This offers comparison of the same individuals before and after treatment.
• It is diagrammed as O1 X O2

Static Group Design

• It is also known as "after-only design with control group".
• An experimental group is measured after exposure to the treatment.
• Control group is measured lacking exposure to the treatment.
• No pre-measure is taken.
• The major weakness is the lack of assurance that the groups were equal on variables of interest prior to the treatment.
• It is diagrammed as experimental and a control group: O1 X O1

Randomized (Pretest-Posttest) Control Group Design

• Also known as "Before-After with Control."
• This is a true experimental design.
• An experimental group is tested before and after treatment exposure
• Control group tested at same two times lacking exposure to treatment.
• It includes random assignment to groups.
• The effect of all extraneous variables is presumed to be the same on both groups.

Pretest-Posttest Control Group Design - Diagram

• Experimental Group: R O1 X O2
• Control Group: R O3 O4
• Effect equals: (O2-O1) -- (O4-O3)

Posttest-Only Control Group Design

• Also known as "After-Only with Control."
• This is a true experimental design.
• An experimental group is tested after treatment exposure.
• Control group tested at same time without exposure to the experimental treatment.
• Random assignment to groups is used.
• The effect of all extraneous variables is assumed to be the same on both groups.
• It is often used only in situations when one cannot pretest.

Posttest-Only Control Group Design - Example

• Assume there is an athlete's foot remedy.
• The goal is to demonstrate the product is better than the competition.
• It is not possible to pretest the effectiveness of the remedy.
• Diagrammed as: Experimental Group R X O1 and Control Group R X O2
• Effect of the experimental treatment equals: (O2 – O1)

Davis and Topf Study

• In 1993, Davis and Topf used a posttest-only control group to examine if CCU noise affects REM sleep.
• Researchers randomly assigned 70 women without hearing or sleeping problems.
• Participants were assigned in the study to sleep in the following conditions: a noisy enviroment or a quiet enviroment.
• It was found that CCU sounds can cause poorer REM sleep.

Randomized Controlled Trial

• Parallel: Patients remain in the same group (treatment or control) throughout the study.
• Crossover: Patients initially assigned to one group but are switched to the other group during the trial.

Randomization

• Allocation of treatments is carried out using a chance mechanism.
• Neither the patient nor the physician knows in advance which therapy will be assigned.
• Simplest Case: Each patient has the same chance of receiving any of the treatments under study.
• The opposite is convenience sampling ("grab"), which is easily accessed but not random.

Simple Randomization

• Imagine tossing a coin to determine group assignment.
• HEADS for Treatment A, TAILS for Treatment B.
• Approximately 1/2 are assigned to treatments A and B.
• It is usually done using a randomization schedule or a computerized random number generator.

Problem with Simple Randomization

• May result in imbalance in the number of subjects assigned to each group.
• The solution: Use blocking and/or stratified randomization.

Block Randomization Example

• If have two treatment groups A/B with equal allocation, and a block size of 4, then the random assignments would be chosen from the blocks
• Blocking ensures balance after every 4th assignment.
• The block size must be the multiplier of the sum of the treatment ratio.
• For example, for a treatment assignment is A: B in 1:1 ratio, the block size must be 2, 4, 6, or 8.
• For a treatment assignment is A:B in 2:1 ratio, the block size must be 3, 6, 9, or 12.

Block Randomization

• Within blocks is also used randomly assigned subjects.
• The balance is achieved within blocks, based on the randomization ratio

Stratified Randomized Sampling

• This ensures balance on an important baseline factor.
• It creates strata and sets up separate randomization schedules within each stratum.
• Example: to prevent an imbalance on age in an osteoporosis study, first create the strata “< 75 years" and " 75 years" and then randomize within each stratum separately.
• Blocking should be used within each stratum.

Stratification

• "Normal" randomization: Does not address how gender is spread through the groups in a study.
• Stratification solves this by creating even groups for comparison.

Blinding

• Involves masking the identity of the assigned interventions.
• The main goal is to avoid potential bias caused by conscious or subconscious factors.
• Open Label: Both investigators and patients know the treatment being given.
• Single blind: The patient is blinded.
• Double blind: both patient and assessing investigator are blinded.
• Triple blind: committee monitoring response variables (e.g. statistician) is blinded.

Blinding in Practice

• Patients who know they are getting an intervention may report certain side effects over others.
• Patients not on new or experimental treatment may be more likely to drop out of a study.
• It may be impossible (or unethical) to blind certain things with characteristic side effects.

Placebo Effect

• There is a beneficial effect from the treatment that cannot be attributed to the placebo itself.
• The effect must be due to the patient's belief in that treatment.
• Some researchers believe that antidepressants are not more effective than placebos and only outperform placebos due to systematic error.

Placebos and Antidepressants

• Placebo responses in antidepressant trials also activate similar regions of the brain as active medications.

Inclusion and Exclusion Criteria

• Establishing these criteria for study participants is a standard, required practice when designing high-quality research protocols.
• Defining inclusion and exclusion criteria increases the likelihood of producing reliable and reproducible results, minimizes the likelihood of harm to the subjects.
• Inclusion criteria are characteristics that the prospective subjects must have.
• Typical inclusion criteria include demographic, clinical, and geographic characteristics such as age, gender, race, ethnicity, marital status, educational experience, language, type of occupation, physical activity, medical conditions, and the presence of medical, or psychosocial.
• Exclusion criteria are defined as features of the potential study participants who meet the inclusion criteria but present with contradictory factors.
• Common exclusion criteria include characteristics of eligible individuals that make them highly likely to be lost to follow-up, miss scheduled appointments to collect data, provide inaccurate data, or have comorbidities that could bias the results of the study.
• Criteria that are too narrow may result in difficulty recruiting the right amount of patients for the study.
• Criteria that are too broad may include individuals less likely to comply.

Study and Eligibility Criteria Examples

• A lifestyle intervention study example for women between the ages of 45 and 75 diagnosed with menopause.
• An exclusion criterion for this particular study might be individuals who, upon meeting the inclusion criteia, also have abnormal blood tests.

Objective of Study

• To evaluate the effectiveness of perineal massage with pregnant women.
• Women were studied with and without vaginal births for prevention of perineal trauma at birth.
• 5 hospitals participated in this single-blind, randomized, controlled trial from Quebec, Canada.
• The results showed women with an intact perineum who complied with perineal messaging experienced a better outcome.
• Preineal massage for women with a first vaginal delivery is an effective way to help delivery.

Factors Influencing Southeastern U.S. Mothers' Participation in Baby-Friendly Practices Study

• The inclusion criteria for mothers included are a minimum of 18 years of age, and also African American (qualitative only).
• Exclusion criteria are if the mother is unable to breastfeed due to illness or delivery complications, taking contraindicated medications or substances, diagnosed with HIV, Department of Social Services involvement, and not being English-speaking.
• The infants had to be 38 week's gestation and older.
• Exclusions were admission to the neonatal intensive care unit, congenital abbormalities, and infant death.

Question 1

• All of the following are considered principle criteria that MUST be met in a true experiment EXCEPT:
• Correct answer is D. Being a double-blind study.

Questions cover static groups, inclusion/exclusion criteria, random assignment, pretests, and posttest-only designs. This quiz is designed for students studying experimental designs, to test the knowledge of key design principles.