Bias and Causal Associations in Observational Research PDF

;~ ~ EPIDEMIOLOGY SERIES 2002 03 IEpidemiology series I Bias and causal associations in observational reseWofh.....

;~ ~ EPIDEMIOLOGY SERIES 2002 03 IEpidemiology series I Bias and causal associations in observational reseWofh.. lee. ThIS M~1·c.--:"'J ~1 Protected'. -"'",:" ,~ay Be m 01) C'-:;;l!':'~rr... < j \.rUe 17 U S ~'-/ "2' ,l L.al'l.. Ccce). David A Grimes, Kenneth F Schulz Readers of medical literature need to consider two types of validity, Internal and extemal. Internal validity means that the study measured what it set out to; external validity Is the ability to generalise from the study to the reader's patients. With respect to internal validity, selection bias, information bias, and confounding are present to some degree in all observational research. Selection bias stems from an absence of comparability between groups being studied. Information bias results from incorrect determination of exposure, outcome, or both. The effect of information bias depends on its type. If Information Is gathered differently for one group than for another, bias results. By contrast, non..(llfferential mlsclasslficatlon tends to obscure real differences. Confounding Is a mixing or blurring of effects: a researcher attempts to relate an exposure to an outcome but actually measures the effect of a third factor (the confounding variable). Confounding can be controlled In several ways: restriction, matching, stratification, and more sophisticated mUltivariate techniques. If a reader cannot explain away study results on the basis of selection, Information, or confounding bias, then chance might be another explanation. Chance should be examined last, however, since these biases can account for highly significant, though bogus resuHs. Differentiation between spurious, indirect, and causal associations can be difficult. Criteria such as temporal sequence, strength and consistency of an aSSOCiation, and evidence of a dose-response effect lend support to a causal link. Clinicians face two important questions as they read criteria. The filtering process for admission to medical research: is the report believable, and, if so, is it randomised trials might, therefore, result in "a type of relevant to my practice? Uncritical acceptance of hothouse flower, which cannot bloom or be successfully published research has led to serious errors and removed beyond its special greenery."5 squandered resources.! Here, we will frame these twO questions in tenns of study validity, describe a simple Bias checklist for readers, and offer some criteria by which to Bias undermines the internal validity of research. Unlike judge reported associations. the conventional meaning of bias-ie, prejudice-bias in research denotes deviation from the uuth. All Internal and external validity observational studies (and, regrettably, many badly done Analogous to a laboratory test, a study should have randomised controlled trialst,11 sexual 700 6·0% partner Restriction (n=800) 100 6·0% The simples t approac h is restricti on (also called exclusio n or specification).28 For example, if cigarett e smoking is suspect ed to be a confoun ding factor, a study can enrol RR",6'0% :=1-0 6-0% only non-sm okers. Althoug h this tactic avoids confoun ding, it also hinders recruitm ent (and thus Example of confounding in a hypothetical cohort study of power) and preclud es extrapo lation to smokers. Intrauterine device use and salpingitis of Restrict ion might increase the internal validity of a study When the crude relative risk is controlled for the confounding effect number of sexual partners, the raised risk disappears_ at the cost of poorer external validity. 250 THE LANCET · Vo1359· January 19, 2002· www.thelancet_com 3  EPlDE;\U OLOGY SERIES cohort of 2000 women, use of an IUD was strongly By contrast , indirect aSSOcl3tlOns (which stem from related to develop ment of salpingitis (relative risk 3'0; confoun ding) are real but not causal. 95% CI 1'7-5'4). Howeve r, the number of sexual Judgme nt of cause-effect relation s can be tough. Few partners was related to women 's choice of contrac eption rules apply, though criteria first suggest ed by Hill ha\'e and to their risk of upper-g enital-tr act infection. Here, a received the most attentio n (panel 2).I,."~,H The only iron- disprop ortionat e number of women with more than one clad criterion is temporality: the cause must antedat e the sexual partner chose to use an IUD (700 vs 300 women effect. Howeve r, in many studies, especially with chronic with only one partner). The number of partners was also diseases, answeri ng this chicken-egg questio n can be related to the risk of infection (6% among those with dauntin g. Strong associations argue for causatio n. > 1 partner vs 1% among those with only one partner). In Wherea s weak associations in observational studies can each stratum by number of partners , the relative risk is easily be due to bias, large amount s of bias would be 1'0, indicati ng no association between the IUD and necessary to produce strong associations. (This large bias salpingitis. The ManteI- Haensze l weighte d relative risk, is evident in repons that link IUD use with salpingitis:) which controls for this confoun ding effect, is 1·0 Some suggest that relative risks more than 3 in cohort (95% CI 0·5-2·0 ). In this fictitious example, the studies, or odds ratios greater than 4 in case-co ntrol apparen t three-fold increase in risk associated with IUD studies, provide strong suppon for causation."'"I use was all due to confoun ding bias. Consist ent observa tion of an association in differen t populat ions and with differen t study designs also lends Multivariate techniques suppon to a real effect. For example , results of studies In multiva riate techniq ues, mathem atical modelli ng done around the world have consiste ntly shown that oral examines the potentia l effect of one variable while contrace ptives protect against ovarian cancer; a causal simulta neously controll ing for the effect of many other relation can, therefore, be argued. E\idenc e of a factors. A major advanta ge of these approac hes is that biological gradien t suppon s a causal association too. For they can control for more factors that can stratification. instance , protecti on against ovarian cancer is directly For example , an investig ator might use multiva riate related to duratio n of use of oral contracepti.ves.d The logistic regression to study the effect of oral risk of death from lung cancer is linearly related to years contrac eptives on ovarian cancer risk. In this way, they of cigarette smoking. In both of these examples, could simulta neously control for age, race, family history, increasi ng exposur e is associat ed with an increasi ng parity, &c. Anothe r example would be use of a biological effect. proport ional hazards regression analysis for time to Other criteria of HiH's are less useful, Specificity is a death; this method could control simulta neously for age, weak criterion. With a few exceptions, such as the rabies blood pressure , smoking history, serum lipids, and other virus, few exposures lead to only one outcom e. Should an risk factors. 39 Disadva ntages of multiva riate approac hes, association be highly specific, this provide s suppon for for some research ers, include greater difficulty in causality. Howeve r, since many exposu res-eg, cigarett e underst anding the results, and loss of hands-o n feel for smoke- lead to numero us outcom es, lack of specificity the data. 2S does not argue against causatio n. Biological plausibility is another weak criterion , limited by our Jack of knowledge. Chance 300 years ago, clinicians would have rejected the If a reader cannot explain results on the basis of selection, suggestion that citrus fruits could prevent scun,}, or that infonna tion, or confoun ding bias, then chance might be mosquit oes were linked with blad."water fever. Ancillary another explana tion. The reason for examin ation of bias biological evidence that is coheren t \,ith the associat ion before chance is that biases can easily cause highly might be helpful. For exampl e, the effect of cigarett e significant (though bogus) results. Regrettably, many readers use the p value as the arbiter of validity, without conside ring these other, more importa nt, factors. Panel 2: Criteria for judgment of causal The venerab le p value measure s chance. It advises the assoclations1 7,42,43 reader of the likelihood of a false-positive conclusion: a Temporal sequence difference was seen in the study, althoug h it does not Did exposure precede outcome? exist in the broader population (type I error). Many clinicians are surprise d to learn, however, that the p value Strength of associat ion of 0·05 as a thresho ld has no basis in medicin e. Rather, it How strong is the effect, measured as relative risk or odds stems from agricult ural and industri al experim ents early ratio? in the 20th century.~o..n Should a study not achieve ConsIstency of associat ion significance at this level, one needs to see if the study had Has effect been seen by others? adequa te power to find a clinically importa nt difference. Many "negativ e" studies simply have too few particip ants Biological gradient (dose-response relation) to do the job.13·'~ Better yet, investigators should present Does increased exposure result in more of the outcome? measure s of associat ion with confide nce intervals~1 in Speclflclty of associat ion preferen ce to hypothe sis tests. Does exposure lead only to outcome? Biological plausibil ity Judgm ent of assoc iations Does the association make sense? Bogus, indirect, or real? When statistical aSSoclatlonS emerge from clinical Coherence with existing knowredge research , the next step is to judge what type of associat ion Is the association consistent with available evidence? exists. Statistical associations do not necessarily imply Experimental evidence causal associations. J7 Althoug h several classifications are Has a randomised controlled trial been done? available,28 a simple approac h include s just three types: spuriou s, indirect , and causal. Spuriou s associations are Analogy the result of selectio n bias, informa tion bias, and chance. Is the association similar to others? THE lANCET · Vol 359 January 19, 2002· \\'WW.thelancet.com 251  _M_ EPIDEM IOLOGY SERIES 18 Burkman RT. Association between intrauterine device and pelvic, smoke on the bronchi al epitheli um of animals is coheren t inflammatory disease. Obstet GynecoZl981; 57: 269-76. with an increase d risk of cancer in human beings. Finally, 19 Kronmal RA, Whitney CW, Mumford SD. The intrauterine device experimental evidence is seldom available, and reasonin g and pelvic inflammatory disease: the Women's Health Study by analogy has sometim es caused harm. Since reanalyzed. J Clin Epidemio ll991; 44: 109-22. thalidom ide can cause birth defects, for instance , some 20 Feinstein AR, Horwitz RI. Oestrogen treatment and endometrial carcinoma. BMJ 1977; 2: 766-67. lawyers (successfully) argued by analogy that Bendec tin 21 Seltzer CC, Bosse R, Garvey AJ. Mail survey response by smoking (an antieme tic widely used for nausea and vomiting in status. AmJ Epidemio ll974; 100: 453-57. pregnancy) could also cause birth defects, despite 22 Schull WJ, Cobb S. The intrafamilial transmission of rheumato id evidence to the contrary.46 arthritis: 3, the lack of support for a genetic hypothesis. J Chronic Dis 1969; 22: 217-22. Conclu sion 23 Bartholomew LL, Grimes DA. The alleged association between induced abortion and risk of breast cancer; biology or bias? Obstet Studies need to have both internal and external validity: GynecoZ Surv 1998; 53: 708-14. the results should be both correct and capable of 24 Lindefors-Harris BM, Eklund G, Adami HO, Meirik O. Response extrapo lation to the populat ion. A simple checklist for bias in a case-control study: analysis utilizing comparative data bias (selection, informa tion, and confoun ding) then concerning legal abortions from twO independ ent Swedish studies. AmJ Epidemio ll991; 134: 1003-08. chance can help readers deciphe r research reports. When of 25 Harris BM, Eklund G, Mcirik 0, Rutqvist LE, Wiklund K. Risk a statistical association appears in research, guidelines for cancer of the breast after legal abortion during first trimester: a judgme nt of associations can help a reader decide Swedish register study. BMJ 1989; 299: 1430-32. whether the association is bogus, indirect, or real. 26 Melbye M, Wohlfahrt J, Olsen JH, et a!. Induced abortion and the risk of breast cancer. N EnglJ Med 1997; 336: 81-85. s We thank Willard Cates and David L Sackett for their helpful comment 27 Abramson JH. Making sense of data. New York: Oxford University our on an earlier version of this report. Much of this material stems from Press, 1988. 15 years of teaching the Berlex Foundation Faculty Development Course. 28 Hulley SB, Cumming s SR, Browner WS, Grady D, Hearst N, Newman RE, eds. Designing clinical research: an epidemiologic approach, 2nd edn. Baltimore: Lippincott Williams and Wilkins, 2001. References l e of medical 29 Ory HW. Association between oral contraceptives and myocardia I Grimes DA. Technology follies: the uncritical acceptanc infarction: a review. JAMA 1977; 237: 2619-22. innovatio n.JAMA 1993; 269: 3030-33. 30 Schwingl PI, Ory HW, Visness CM. Estimates of the risk of 2 Last JM, ed. A dictionary of epidemiology, 2nd edn. New York: cardiovascular death attributable to low-dose oral contraceptives in Oxford University Press, 1988. the United States. Am J Obstet Gynecoll9 99; 180: 241-49. 3 Ahlborn A, Norell S. Introduct ion to modern epidemiology, 2nd edn. 31 Jain AK. Cigarette smoking, use of oral contraceptives, and Chestnut Hill, Massachusetts: Epidemiology Resources, 1990. myocardial infarction. Am J Obstet GynecoZ1976; 126: 301-07. 4 Chalmers TC, Celano P, Sacks HS, Smith H Jr. Bias in treatment 32 Lee NC, Rubin GL, Ory HW, Burkman RT. Type of intrauterine assignment in controlled clinical trials. N Engl J Med 1983; 309: device and the risk of pelvic inflammatory disease. Obstet Gyneco[ 1358-61. 1983; 62: 1-6. 5 Feinstein AR. Clinical epidemiology: the architecture of clinical 33 Lee NC, Rubin GL, Borucki R. The intrauterine device and pelvic research. Philadelphia: WB Saunders Company , 1985. inflammatory disease revisited: new results from the Women's Health 6 Anon. The National Diet-Hea rt Study Final Report. Circulation 1968; Study. Obstet Gyneco11988; 72: 1-6. 37: 11-428. 34 Lee NC, Rubin GL, Grimes riA. Measures of sexual "behavior and 7 Moinpour CM, Lovato LC, Thompso n IMJr, et a\. Profile of men the risk of pelvic inflammatory disease. Obslet Gynecol1991; 77: randomized to the prostate cancer prevention trial: baseline health- 425-30. related quality of life, urinary and sexual functioning, and health 35 Schlesselman JJ. Cancer of the breast and reproductive tract in behaviors. J Clin Onco/2000; 18: 1942-53. relation to use of oral contraceptives. Contraception 1989; 40: 1-38. 8 Halbert JA, SiJagy CA, Finucane P, Withers RT, Hamdorf PA. 36 Lacey]V Jr, Brinton LA, Abbas FM, et a1. Oral contraceptives as risk Recruitment of older adults for a randomized, controlled trial of factors for cervical adenocarcinomas and squamous cell carcinomas. exercise advice in a general practice setting. J Am Gen"atr Soc 1999; Cancer Epidemio/ Biomarkers Prev 1999; 8: 1079-85. 47: 477-8l. 37 Kjellberg L, Hailmans G, Ahren AM, et at Smoking, diet, pregnancy 9 Moher D, Pham B, Jones A, et al. Does quality of reports of and oral contraceptive use as risk factors for cervical intra-epithelial randomised trials affect estimates of intervention efficacy reported in neoplasia in relation to human papillomavirus infection. Br J Cancer meta-analyses? Lancet 1998; 352: 609-13. 2000;82: 1332-38. 10 Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence 38 Mantel N, Haenszel W. Statistical aspects of the analysis of data from of bias. Dimensions of methodological quality associated with retrospective studies of disease. J Natl Cancer Imt 1959; 22: 719-:48. estimates of treatment effects in controlled trials. JAMA 1995; 273: 39 Lang TA, Secic M. How to report statistics in medicine. 408-12. Philadelphia: American College of Physicians, 1997. 11 Rothman KJ. Modern epidemiology. Boston: Little, Brown and 40 Rothman KJ. A show of confidence. N EnglJ Med 1978; 299: Company, 1986. 1362-63. 12 Grimes DA. The case for confidence intervals. Obste! Gyneco[1992; 41 Sterne JA, Smith GD. Sifting the evidence: what's wrong with 80: 865-66. significance tests? BMJ 2001; 322: 226-31. 13 Freiman JA, Chalmers TC, Smith H Jr, Kuebler RR. The 42 Hill AS. The environment and disease association or causation. importance of beta, the type II error and sample size in the design Proc R Soc Med 1965; 58: 295-300. and interpretation of the randomized control trial: survey of 71 43 Streiner DL, Norman GR, Munroe Blum H. PDQ epidemiology. ''negative'' trials. N EnglJ Med 1978; 299: 690-94. and Toronto: BC Decker, 1989. 14 Moher D, Dulberg CS, Wells GA. Statistical power, sample size, 44 Sackett DL, Haynes. RB, Guyatt GH, Tugwell P. Clinical their reporting in randomized controlled trials. JAMA 1994; 272: epidemiology: a basic science for clinical medicine, 2nd cdn. Boston: 122-24. Little, Brown and Company, 1991. 15 Sackett DL. Bias in analytic research. J Chronic Dis 1979; 32: 45 Grimes DA, Economy KE. Primary prevention of gynecologic 51-63. cancers. Am J Obstet Gynecoll9 95; 172: 227-35. 16 Wingo PA, Higgins JE, Rubin GL, Zahniser SC, eds. An 46 McKeigue PM, Laqlm SH, Linn S, Kutcher JS. Bendectin and birth epidemiologic approach to reproductiVe health. Geneva: WHO, 1994. defects: 1, a meta~analysis of the epidemiologic studies. Teratology 17 Hennekens CH, Buring JE. Epidemiology in medicine. Boston: Little, 1994; 50: 27-37. Brown and Company, 1987. THE LANCET ' Vol 359 January 19, 2002 www.thelancet.com 252