PHAR 628 Lecture 7: Randomized Controlled Trial II PDF

PHAR 628 Research Methods & Biostatistics Lecture 7: Randomized Controlled Trial II Lixian Zhong, Ph.D. Feb 7th, 2025 1 Quick Review Characteristics of Experiments To be...

PHAR 628 Research Methods & Biostatistics Lecture 7: Randomized Controlled Trial II Lixian Zhong, Ph.D. Feb 7th, 2025 1 Quick Review Characteristics of Experiments To be considered a true experiment, a study must have three essential characteristics: 1. The independent variable must be manipulated by the experimenter 2. The subjects must be randomly assigned to groups 3. A control group must be incorporated within the design 2 What is the difference between random sampling and random assignment? Random Assignment Occurs Here Random Sampling Occurs Here 3 Manipulation of variables occurs here (i.e. “intervention”) 4 5 Learning Objectives ❖ Evaluate experiments - Internal validity - external validity ❖ Experimental study design - Blinding - Intention-to-treat 6 Internal Validity Given a statistical relationship between the independent and dependent variables, is there evidence that one causes the other? Internal validity helps us determine if the experimental intervention (e.g. treatment) really caused the observed change in the dependent variable (e.g. outcome) – Or if other (extraneous) factors may be responsible for that change 7  Internal Validity Confounding (extraneous) variables present threats to internal validity because they offer competing explanations for the observed relationship between the independent and dependent variables – They interfere with cause-and-effect inferences 8 External Validity Can the results be generalized to persons, settings, and times that are different from those employed in the experimental situation? External validity refers to the extent to which the results of a study can be generalized beyond the internal specifications of the study sample External validity is concerned with the usefulness of that information outside the experimental situation 10 Main Considerations for Evaluating External Validity - Selection of patients - Setting of the trial - Difference between trial protocol and routine practice - Outcome measures and follow up - Adverse effects of treatment 11  Study Designs to Enhance Internal Validity (Design for most accurate results…) ✓Manipulation of independent variable ESSENTIAL ✓Random Assignment ✓Control Groups NON-ESSENTIAL Blinding BUT HELPS Intention-to-treat 12  Study Designs to Enhance Internal Validity (Design for most accurate results…) ✓Manipulation of independent variable ESSENTIAL ✓Random Assignment ✓Control Groups NON-ESSENTIAL Blinding BUT HELPS Intention-to-treat 13 Observational Bias 14  Blinding The statue of Justitia or "Lady Justice" is quite well known in the context of law and order. Since the 15th century, Lady Justice has often been depicted wearing a blindfold. The blindfold represents objectivity, in that justice is or should be meted out objectively, without fear or favor, regardless of money, wealth, power, or identity; blind justice and impartiality. Source: http://en.wikipedia.org/wiki/Lady_Justice 15 Blinding The potential for bias (observational, reporting etc.) is an important concern in experimental studies The participants' knowledge of their treatment status or the investigator's expectations can, consciously or unconsciously, influence performance or the recording and reporting of outcomes Protection against this form of bias is best achieved by using a double-blind study, where neither the subjects nor the investigators are aware of the identity of the treatment groups until after data are collected 16 Blinding In its most complete form, a blind design can involve hiding the identity of group assignment ✓ From subjects ✓ From those who provide treatment and measure outcomes ✓ From those who analyze the data 17 Single-blind Trial A study in which one party, either the investigator or participant, is unaware of what medication the participant is taking; – Also called single-masked study – Usually the participants (subjects) do not know Subjects (blinding) Those who provide treatment and measure outcomes (no blinding) Those who will analyze the data (no blinding) Source: http://www.clinicaltrials.gov/ct2/info/glossary 18 Double-blind Trial A clinical trial design in which neither the participating individuals nor the investigators knows which participants are receiving the experimental drug and which are receiving a placebo (or another therapy). Double-blind trials are thought to produce objective results, since the expectations of the researcher and the participant about the experimental drug do not affect the outcome – Also called double-masked study. Source: http://www.clinicaltrials.gov/ct2/info/glossary 19 20  Triple-blind Trial None of the participants, investigators and data analysts know the identity of group assignment Subjects (blinding) Those who provide treatment and measure outcomes (blinding) Those who will reduce and analyze the data (blinding) 21 Open-label Trial Both the researchers and the participants know the identity of group assignment Subjects (no blinding) Those who provide treatment and measure outcomes (no blinding) Those who will reduce and analyze the data (no blinding) 22 Blinding As assessments become more subjective, the need for blinding increases. To whatever extent is possible within an experiment, blinding will substantially strengthen the validity of conclusions. The technique of blinding requires that treatments be coded in some way, so that when data collection is complete, the code can be broken and group assignments revealed 23 Issues related to blinding - Blinding may not always to feasible. E.g., surgery - Drug related sided effects have the potential to un-blind subjects and investigators. ◦ FDA recommends that “…administer a questionnaire at study completion to investigate the effectiveness of blinding the subjects and treating and evaluating physicians” ◦ CONSORT (Consolidated Standards of Reporting Trials) recommends reporting “how the success of blinding was evaluated.” ◦ Observed serious adverse events during blinded clinical trials: FDA says sponsors should compare the rate of observed serious adverse events in participants in the investigational arm to those in the control arm of the study. Because such assessments would involve unblinded data, FDA says they should only be done by a data monitoring committee or a “firewalled” and independent specially constituted safety committee. 24 Blinding occurs here 25  Study Designs to Enhance Validity (Design for most accurate results…) ✓Manipulation of independent variable ESSENTIAL ✓Random Assignment ✓Control Groups NON-ESSENTIAL Blinding BUT HELPS Intention-to-treat 26 Compliance with Treatment Compliance (adherence): the extent to which patient correctly follows medical advice. Most often it refers to medication compliance: how patient acts in accordance with the prescribed interval & dose of a dosing regimen In RCTs, some patients do not comply with their treatment. Question: Should we exclude or include these subjects in the analysis? 27 Withdrawal from Trial Some subjects choose to end participation before the end of the trial As a result, outcome information is usually not available – missing outcomes! Should we include or exclude these subjects from the analysis? 28 Intention-to-Treat (ITT) The ITT analysis includes all randomized patients in the groups to which they were randomly assigned, regardless of their adherence with the entry criteria, regardless of the treatment they actually received, and regardless of subsequent withdrawal from treatment or deviation from the protocol. - (Lloyd) Fisher et al., 1990 29 Intention-to-Treat (ITT) Intention-to-treat means that data are analyzed assuming each patient receives the treatment to which he was assigned We analyze the data according to the way we intended to treat the subjects, not the way they were actually treated Noncompliant subjects are also included in analysis according to the original random assignment 30 Intention-to-Treat (ITT) “Once randomized, always analyzed” 31 Non-ITT analyses As-Treated analysis: patients are included in the group corresponding to the treatment they actually receive. Patient compliance and “switchovers” are considered. Per-protocol analysis: patients who do not meet all of the eligibility criteria or do not adhere to the protocol are excluded. It restricts the comparison to the ideal patients who complied perfectly to the clinical trial instructions. 32 E.g., Coronary Drug Project Coronary Drug Project (1966-1975): randomized, multi-center, double-blind, placebo-controlled trial of clofibrate for treatment of coronary heart disease - 1103 men on clofibrate - 2789 men on placebo ITT analysis of 5-year mortality on clofibrate was 20.0%, 20.9% on placebo (p=0.55) 33 E.g., Coronary Drug Project There was speculation that good compliers would show the clofibrate benefit and poor compliers would have mortality similar to placebo subjects Good compliance defined as 80% of protocol prescribed treatment taken In clofibrate group, mortality rate at 5 years were: - Compliant: 15% - Non-compliant: 24.6% Compliant subjects had significantly lower mortality (p=0.0001)! As-treated analysis! 34 35 E.g., Coronary Drug Project Wait… In placebo group, mortality rates at 5 years were - Compliant: 15.1% - Non-compliant 28.2% Subjects compliant with placebo also had significantly lower mortality (p