Study Designs Overview PDF

Study designs, overview Prof Ghaith M. Al-Taani, PhD, BCPS Clinical Pharmacy and Pharmacy Practice (UK) Study design Refers to two concepts: Whether the study was well designed: Presence of three errors(random error, bias, and confounding) Which study design was used...

Study designs, overview Prof Ghaith M. Al-Taani, PhD, BCPS Clinical Pharmacy and Pharmacy Practice (UK) Study design Refers to two concepts: Whether the study was well designed: Presence of three errors(random error, bias, and confounding) Which study design was used in the studies in question? Randomized clinical trial Cohort study Case–control study Analyses of secular trends Case series Case reports Study Design Table 2.4 presents the study designs typically used for epidemiologic studies, or in fact for any clinical studies. They are organized in a hierarchical fashion. As one advances from the designs at the bottom of the table to those at the top, studies get progressively harder to perform, but are progressively more convincing. Associations shown by studies using designs at the top of the list are more likely to be causal associations than associations shown by studies using designs at the bottom of the list. The association between cigarette smoking and lung cancer has been reproduced in multiple well-designed studies, using analyses of secular trends, case–control studies, and cohort studies. However, it has not been shown using a randomized clinical trial, which is the “Cadillac” of study designs. CONCEPTS IN STUDY DESIGN Study purpose: Descriptive versus analytical Time orientation: Prospective versus retrospective design Prospective: Begin in the present and progress forward, collecting data from subjects whose outcomes lie in the future Retrospective: Begin and end in the present; however, this design involves a major backward look to collect information about events that occurred in the past Investigator orientation: a. Experimental trials b. Quasi-experimental trials c. Observational trials CASE REPORTS Case reports are simply reports of events observed in single patients. As used in PE, a case report describes a single patient who was exposed to a drug and experiences a particular, usually adverse, outcome. For example, one might see a published case report about a young woman who was taking oral contraceptives and who suffered a pulmonary embolism. CASE REPORTS Case reports are useful for raising hypotheses about drug effects, to be tested with more rigorous study designs. Certainly, one cannot usually determine whether the adverse outcome was due to the drug exposure or would have happened anyway. It is very rare that a case report can be used to make a statement about causation: Exception: When the outcome is so rare and so characteristic of the exposure that one knows that it was likely to be due to the exposure, even if the history of exposure were unclear. An example of this is clear cell vaginal adenocarcinoma occurring in young women exposed in utero to diethylstilbestrol CASE SERIES Case series are collections of patients, all of whom have a single exposure, whose clinical outcomes are then evaluated and described. Often they are from a single hospital or medical practice Case series can also be collections of patients with a single outcome, looking at their antecedent exposures. For example, one might observe 100 consecutive women under the age of 50 who suffer from a pulmonary embolism, and note that 30 of them had been taking oral contraceptives. CASE SERIES After drug marketing, case series are most useful for two related purposes. First, they can be useful for quantifying the incidence of an adverse reaction. Postmarketing surveillance study of prazosin was conducted to quantitate the incidence of first-dose syncope from prazosin. Second, they can be useful for being certain that any particular adverse effect of concern does not occur when observed in a population which is larger than that studied prior to drug marketing: Metiamide was an H-2 blocker, which was withdrawn after marketing outside the US because it caused agranulocytosis. Since cimetidine is chemically related to metiamide there was a concern that cimetidine might also cause agranulocytosis. CASE SERIES In the absence of a control group, one cannot be certain which features in the description of the patients are unique to the exposure, or outcome. Case series are also not very useful in determining causation, but provide clinical descriptions of a disease or of patients who receive an exposure CASE REPORTS/CASE SERIES Advantages and Disadvantages 1. Advantages: Hypotheses are formed, which may be the first step in describing an important clinical problem. Easy to perform and inexpensive 2. Disadvantages: Does not provide explanation other than conjecture and does not establish causality or association ANALYSES OF SECULAR TRENDS Analyses of secular trends, also called “ecological studies,” Examine trends in an exposure that is a presumed cause and trends in a disease that is a presumed effect and test whether the trends coincide These trends can be examined over time or across geographic boundaries. In other words, one could analyze data from a single region and examine how the trend changes over time, or one could analyze data from a single time period and compare how the data differ from region to region or country to country. Correlation (Ecological) Studies  They use the data from entire populations to compare disease frequencies between different groups during the same period of time  The figure shows the correlation b/w per capita daily consumption of meat & rates of colon cancer from a large number of countries.  There is a very striking positive relationship. - It’s not possible to link an exposure to occurrence of a disease in the same person - There might be other differences between countries in factors that are associated with the level of meat consumption that might themselves account for the observed differences in the prevalence of colon cancer ANALYSES OF SECULAR TRENDS Analyses of secular trends are useful for rapidly providing evidence for or against a hypothesis However, these studies lack data on individuals; they utilize only aggregated group data. As such, they are unable to control for confounding variables CASE–CONTROL STUDIES Case–control studies are studies that compare cases with a disease to controls without the disease, looking for differences in antecedent exposures. As an example, one could select cases of young women with venous thromboembolism and compare them to controls without venous thromboembolism, looking for differences in antecedent oral contraceptive use. Several such studies have been performed, generally demonstrating a strong association between the use of oral contraceptives and venous thromboembolism. Case control study - Structure The study identifies one group of subjects with the disease and another without it, then looks backward to find differences in predictor variables that may explain why the cases got the disease and the controls did not. The design of a case–control study is challenging because of the increased opportunities for bias, but there are many examples of well-designed case–control studies that have yielded important results Case–control studies cannot yield estimates of the incidence or prevalence of a disease CASE–CONTROL STUDIES Particularly useful when one wants to study multiple possible causes of a single disease, as one can use the same cases and controls to examine any number of exposures as potential risk factors. Also, useful when one is studying a relatively rare disease, as it guarantees a sufficient number of cases with the disease. Example: A study of diethylstilbestrol and clear cell vaginal adenocarcinoma required only 8 cases and 40 controls rather than the many thousands of exposed subjects that would have been required for a cohort study of this question. both cohort and cross-sectional studies of general population samples are expensive Each would require thousands of subjects to identify risk factors for a rare disease like stomach cancer. CASE–CONTROL STUDIES Case–control studies generally obtain their information on exposures retrospectively, i.e., by recalling events that happened in the past. Information on past exposure to potential risk factors is generally obtained From: Medical records or by administering questionnaires or interviews. As such, case–control studies are subject to limitations in the validity of retrospectively collected exposure information. In addition, the proper selection of controls can be a challenging task, and inappropriate control selection can lead to a selection bias, which may lead to incorrect conclusions. Nevertheless, when case–control studies are done well, subsequent well-done cohort studies or randomized clinical trials, if any, will generally confirm their results. As such, the case–control design is a very useful approach for PE studies CASE–CONTROL STUDIES Useful method (and perhaps the only practical way) to study exposures in rare diseases or diseases that take long periods to develop Critical assumptions to minimize bias a. Cases are selected to be representative of those who have the disease. b. Controls are representative of the general population that does not have the disease and are as identical as possible to the cases, minus the presence of the disease. c. Information is collected from cases and controls in the same way. CASE–CONTROL STUDIES Usefulness for Generating Hypotheses The retrospective approach of case–control studies, and their ability to examine a large number of predictor variables makes them useful for generating hypotheses about the causes of a new outbreak of disease CASE–CONTROL STUDIES 6. Advantages a. Inexpensive and can be conducted quickly b. Allows investigation of several possible exposures or associations 7. Disadvantages Confounding must be controlled. Only one outcome can be studied (the presence or absence of the disease that was the criterion for drawing the two samples), It is susceptible to bias Observational and recall bias: Looking back to recall exposures and their possible levels of exposure (particularly in retrospective assessment) Selection bias: Case selection and control matching are difficult. ➔ Can be controlled via matching or population-based sample of cases COHORT STUDIES Cohort studies are studies that identify subsets of a defined population and follow them over time, looking for differences in their outcome. Cohort studies generally are used to compare exposed patients to unexposed patients, although they can also be used to compare one exposure to another descriptive , typically to describe the occurrence of certain outcomes over time; analytic , to analyze associations between predictors and those outcomes Example: One could compare women of reproductive age who use OC to users of other contraceptive methods, looking for the differences in the frequency of venous thromboembolism. When such studies were performed, they in fact confirmed the relationship between oral contraceptives and thromboembolism, which had been noted using analyses of secular trends and case–control studies. COHORT STUDIES There are two primary purposes: descriptive , typically to describe the occurrence of certain outcomes over time; analytic , to analyze associations between predictors and those outcomes. Useful in studying the causes of disease and natural history/progression of disease Usually prospective Know if subjects have been exposed to something Follow them through time Follow-up can be years PROSPECTIVE COHORT STUDIES In a prospective cohort study, the investigator begins by assembling a sample of subjects She measures characteristics in each subject that might predict the subsequent outcomes, and follows these subjects with periodic measurements of the outcomes of interest. RETROSPECTIVE COHORT STUDIES It differs from that of a prospective one in that the assembly of the cohort, baseline measurements, and follow-up have all happened in the past. This type of study is only possible if adequate data about the risk factors and outcomes are available on a cohort of subjects that has been assembled for other purposes. COHORT STUDIES Cohort studies can be performed either prospectively, or retrospectively. The major difference between cohort and case–control studies is the basis upon which patients are recruited into the study: Patients are recruited into case–control studies based on the presence or absence of a disease, and their antecedent exposures are then studied. Patients are recruited into cohort studies based on the presence or absence of an exposure, and their subsequent disease course is then studied. Cohort studies advantages Cohort studies have the major advantage of being free of the big problem that plagues case–control studies: The difficult process of selecting an undiseased control group. Prospective cohort studies are free of the problem of the questionable validity of retrospectively collected data. For these reasons, an association demonstrated by a cohort study is more likely to be a causal association than one demonstrated by a case– control study. Cohort studies are particularly useful when one is studying multiple possible outcomes from a single exposure, especially a relatively uncommon exposure. Thus, they are particularly useful in postmarketing drug surveillance studies, which are looking at any possible effect of a newly marketed drug. Cohort studies advantages The prospective cohort design is a powerful strategy for assessing incidence (the number of new cases of a condition in a specified time interval), It appropriate for common and immediate diseases Retrospective cohort studies are much less costly and time consuming than prospective cohort studies Cohort studies disadvantages Cohort studies can require extremely large sample sizes to study relatively uncommon outcomes Inefficiency for studying rare outcomes. This might need following large number of patients for a long period In addition, prospective cohort studies can require a prolonged time period to study delayed drug effects. Causal inference is challenging and interpretation often affected by the influences of confounding variables Expense Cross-sectional study The structure of a cross-sectional study is similar to that of a cohort study except that all the measurements are made at about the same time, with no follow-up period. Cross-sectional studies can also be used for examining associations, e.g. association between childhood obesity and hours spent watching television Cross-sectional study Cohort studies, which have a longitudinal time dimension and can be used to estimate incidence (the proportion who get a disease or condition over time), Cross-sectional studies can generally provide information only about prevalence, the proportion who have a disease or condition at one point in time Prevalence is useful to health planners who want to know how many people have certain diseases so that they can allocate enough resources to care for them Strengths of cross-sectional studies There is no waiting for the outcome to occur Fast and inexpensive, There is no loss to follow-up Weaknesses of cross-sectional studies The difficulty of establishing causal relationships from observational data collected in a cross-sectional time frame Cross-sectional studies are also impractical for the study of rare diseases if the design involves collecting data on a sample of individuals from the general population Cross-sectional studies can be done on rare diseases if the sample is drawn from a population of diseased patients rather than from the general population RANDOMIZED CLINICAL TRIALS Experimental studies are studies in which the investigator controls the therapy that is to be received by each participant. Gold standard for testing the effect of an intervention “A beautiful technique, with wide applicability” Experimental or interventional, investigator makes intervention and evaluates cause and effect. Examine etiology, cause, efficacy, using comparative groups. For example: One could theoretically randomly allocate sexually active women to use either oral contraceptives or no contraceptive, examining whether they differ in their incidence of subsequent venous thromboembolism. RANDOMIZED CLINICAL TRIALS The major strength of this approach is random assignment, which is the only way to make it likely that the study groups are comparable in potential confounding variables that are either unknown or unmeasurable. For this reason, associations demonstrated in randomized clinical trials are more likely to be causal associations than those demonstrated using one of the other study designs reviewed above. RANDOMIZED CLINICAL TRIALS Outline of RCT Participants are randomly allocated to either one intervention or another Each group is followed up for specified period of time Analysed in terms of specific outcomes defined at start of study Two groups should be identical as possible Only difference is the intervention Any difference in outcomes can then be attributed to the intervention  Recruited population Intervention group Control group Exposed to Intervention Not exposed to intervention Follow-up Follow-up Outcomes Outcomes Questions that can be answered by a RCT Is this drug better than placebo or the current established treatment? Is this new surgical procedure better than current practice? Is a leaflet better than verbal advice in helping patients make informed choices about treatment options? Is this new service better than the usual service? RANDOMIZED CLINICAL TRIALS Design allows assessment of causality. a. Sufficient cause b. Necessary cause c. Risk factor Minimizes bias through randomization and/or stratification a. Randomization b. Block randomization c. Stratification d. Cluster randomization Treatment controls a. Placebo controlled b. Active controlled A RCT requires advance planning Want to test the effect of a new drug on cholesterol Recruitment of patients Sample size Chance of detecting a true difference between the groups, if one really exists Likelihood of detecting a true difference is the power of the study (1 – ß) ß is the probability of Type II error Most studies state between 80-90% power Power Analysis of RCT Question: How many participants do I need?  Answer: The minimum necessary to detect the level of effect you want/expect with sufficient power  Question: How do I find that out?  Answer: Conduct a power analysis Power analysis involves following variables: 1. Alpha = Probability of a type I error p =.05 1. Directionality of statistical test (one tailed/two tailed) 2. Power = 1 – Beta (probability of type II error;.20) =.80 3. ES (effect size) = size of the effect or strength of the association (e.g., size of group differences) 4. N = number of participants needed If you have any 3 of these 4 variables you can calculate the 4th missing one. Many online calculators…Power Calculators (e.g., G*Power) RANDOMIZED CLINICAL TRIALS Randomisation 50:50 probability that a patient will end up in one group or the other Generation of random numbers Needs to be truly random If not random, can produce bias in the study Every other patient who attends a clinic Obvious pattern emerges Want to end up with comparable groups RANDOMIZED CLINICAL TRIALS Concealment to allocation An allocation concealment procedure keeps clinicians and participants unaware of upcoming assignment No idea who is going to end up in a control or intervention groups Protects randomisation When those enrolling patients are unaware and cannot control the arm to which the patient is allocated, we refer to randomization as concealed. In unconcealed trials, those responsible for recruitment may systematically enroll sicker—or less sick—patients to either treatment or control groups. This behavior will compromise the purpose of randomization and the study will yield a biased result Violation of concealment Captopril Prevention Project (CAPPP) was intended to be a properly randomised trial. The small, but highly significant difference between the two treatment groups in pre-randomisation height, weight, systolic and diastolic BP show that the process of randomisation by sealed envelope was violated. Presumably at some centres, those responsible for entering patients sometimes unsealed the envelopes before the next patient was formally entered and then let knowledge of what the next treatment would be influence their decision as to whether that patient would be entered and assigned that foreknown treatment. Richard Peto, Lancet 1999; 354: 73 Randomisation is a beautiful thing Residents Intervention Control sites sites Total numbers 173 (51.8%) 161 (48.2%) Average age 82.5 82.7 (years) % Women 125 (72.2%) 119 (73.9%) % receiving 1-5 med 19 (11%) 21 (13.%) 6-10 meds 60 (34.7%) 66 (41%) 10+ meds 94 (54.35) 74 (46%) On psychoactive 113 (65.6%) 108 (67%) drugs Avoiding confounding Correct randomisation Avoids confounding Subjects have an equal chance of ending up in either group Characteristics should be randomly dispersed between the two groups Comparing ‘like’ with ‘like’ RANDOMIZED CLINICAL TRIALS Blinding methods a. Single-blind: Either subjects or investigators are unaware of subject assignment to active/control. b. Double-blind: Both subjects and investigators are unaware of subject assignment to active/control. c. Triple-blind: Both subjects and investigators are unaware of subject assignment to active/control; in addition, an analysis group is unaware. D. Open-label: Everyone is aware of subject assignment to active/control Blinding Five Groups That Should, if Possible, Be Blind to Treatment Assignment Patients: To avoid placebo effects Clinicians: To prevent differential administration of therapies that affect the outcome of interest (cointervention) Data collectors: To prevent bias in data collection Adjudicators of outcome: To prevent bias in decisions about whether or not a patient has had an outcome of interest Data analysts: To avoid bias in decisions regarding data analysis Examples of Considerations for Controlled Trials 1. Are the results of the study valid (methods)? Randomization, all subjects accounted for, blinding, inclusion and exclusion criteria appropriate, sample size sufficient, statistical tests appropriate, assessed: Surrogate markers or true outcomes 2. What were the results? How large was the treatment effect?, How precise was the effect (based on CIs significant) 3. Can I apply the results of this study to my patient population? Will they help me care for my patients? Can this study be applied to general practice, Can I apply this to my setting?, Do the expected benefits outweigh the expected and/or unanticipated risks RANDOMIZED CLINICAL TRIALS However, even randomized clinical trials are not without their problems: Ethical and logistical issues Expensive and artificial Usually conducted in the premarketing stages RCTs, however, remain the “gold standard” by which the other designs must be judged. Blinding Five Groups That Should, if Possible, Be Blind to Treatment Assignment Patients: To avoid placebo effects Clinicians: To prevent differential administration of therapies that affect the outcome of interest (cointervention) Data collectors: To prevent bias in data collection Adjudicators of outcome: To prevent bias in decisions about whether or not a patient has had an outcome of interest Data analysts: To avoid bias in decisions regarding data analysis Blinding Five Groups That Should, if Possible, Be Blind to Treatment Assignment Patients: To avoid placebo effects Clinicians: To prevent differential administration of therapies that affect the outcome of interest (cointervention) Data collectors: To prevent bias in data collection Adjudicators of outcome: To prevent bias in decisions about whether or not a patient has had an outcome of interest Data analysts: To avoid bias in decisions regarding data analysis Blinding Five Groups That Should, if Possible, Be Blind to Treatment Assignment Patients: To avoid placebo effects Clinicians: To prevent differential administration of therapies that affect the outcome of interest (cointervention) Data collectors: To prevent bias in data collection Adjudicators of outcome: To prevent bias in decisions about whether or not a patient has had an outcome of interest Data analysts: To avoid bias in decisions regarding data analysis Measurement of outcomes Objective measurements Concealment of allocation/blinding will reduce measurement bias Need to decide how samples are to be taken, who will analyse them, how they will be stored Outcome Measures The measures selected for an evaluation will depend on the objectives of the service or intervention and its anticipated outcomes Physiological measures (e.g. blood glucose monitoring) Measurement of health status (to reflect the impact of drug or disease on the individual's ability to perform various activities of daily living) Measuring satisfaction and acceptability Acceptability to health professionals Final thoughts Thus, a series of different study designs are available, each with respective advantages and disadvantages. Case reports, case series, analyses of secular trends, case– control studies, and cohort studies have been referred to collectively as: Observational study designs or nonexperimental study designs In nonexperimental study designs the investigator does not control the therapy, but simply observes and evaluates the results of ongoing medical care