Screening and Diagnostic Tests PDF

SCREENING AND DIAGNOSTIC TESTS EPI II: LECTURE #4 CONCEPT MAP Epidemiological question Study design Causality Analysis RCTs Confounding Statistics...

SCREENING AND DIAGNOSTIC TESTS EPI II: LECTURE #4 CONCEPT MAP Epidemiological question Study design Causality Analysis RCTs Confounding Statistics Effect measure Cohort Selection bias modification Case-Control Information bias THINK-PAIR-SHARE  First, jot down some diseases and tests used for screening. (1 minute)  Next, with a partner discuss what these diseases and tests have in common. (4 minutes)  Finally, let’s share as a class! LECTURE OUTLINE  Definitions  Requirements for Effective Screening  Test Properties  Se, Sp, PPV, NPV  ROC curves & AUC  Population Perspective  Timeline for Screening  Biases in Screening Evaluations DEFINITIONS DEFINITIONS – PREVENTION Primary Secondary Tertiary Avoid biological Minimize adverse Reduce disability onset outcomes through and complications Example: Polio early detection of advanced vaccine and treatment disease Example: Example: HAART Mammography DEFINITIONS – SCREENING Screening is the process of identifying those who have (or are very likely to have) a particular condition of interest, so that decisions about treatment (or non-treatment) can be made. Especially when early detection of a condition is beneficial for either prevention of additional cases or treatment. DEFINITIONS – DIAGNOSTIC TEST Symptomatic or Screened people are tested Clinical symptoms Positive/suspicious screening test result Indicates that a person has or does not have the disease Often not distinguished from a screening test in the literature HOW IS SCREENING DONE?  It depends on disease and the test GOALS OF SCREENING  Immediate goal  Determine if person is likely or unlikely to have disease  Ultimate goal  Reduce morbidity & mortality from disease REQUIREMENTS FOR EFFECTIVE SCREENING SCREENING IN THE NEWS WHAT MAKES FOR AN EFFECTIVE SCREENING? Suitable disease Reasonable Accurate test cost Benefits Effective outweigh treatment harms REQUIREMENTS – SUITABLE DISEASE Suitable  Sufficient burden disease  Consequences if left undetected or untreated Reasonable  What about rare disease? Accurate test cost  Suitable “window of opportunity”  Detectable preclinical period of sufficient length to allow early detection Benefits Effective outweigh treatment harms REQUIREMENTS – ACCURATE TEST Suitable  Reliable (Same result each time) disease  Valid (“Correct”) Reasonable Accurate test cost  Sensitive  Specific Benefits Effective outweigh treatment harms REQUIREMENTS – EFFECTIVE TREATMENT Suitable disease  There must be treatment for disease that: Reasonable  Favorably alters disease progression Accurate test cost  Is effective for screen-detected disease  Is available to people getting screened Benefits Effective outweigh treatment harms REQUIREMENTS – BENEFITS OUTWEIGH HARMS  Benefits Suitable  Prevents late manifestation of disease disease  Successful treatment of disease  Achieve better quality of life Reasonable Accurate test cost  Harms  Adverse events from invasive diagnostic procedures  Adverse effects of treatment Benefits  Psychological effects of false positives Effective outweigh treatment harms REQUIREMENTS – REASONABLE COST Suitable disease  Cost of…  Screening Reasonable Accurate test cost  Diagnosis  Treatment Benefits Effective outweigh treatment harms ONE MORE CONSIDERATION…  Screening in low- and middle-income countries  Healthcare system  Resources for follow-up and/or treatment  Unethical to screen without follow-up  Must have adequate diagnostic testing and treatment  Limited resources and burden of screening TEST PROPERTIES VISUAL OVERVIEW SENSITIVITY & SPECIFICITY Sensitivity Specificity  Probability of detecting a true positive case of  Probability of detecting a true negative case of disease disease  Probability of positive test given individual has disease  Probability of negative test given individual does not have disease  Example: 70% sensitivity  When someone truly has the disease, the test can expected  Example: 70% specificity to be positive 70% of the time  When someone truly does not have the disease, the test can expected to be negative 70% of the time  Independent of the prevalence of disease  Independent of the prevalence of disease MORE ON SENSITIVITY & SPECIFICITY Sensitivity Specificity True positive rate True negative rate 1-Sensitivity=false negative rate 1-Specificity=false positive rate Type II Error Type I Error CALCULATING SENSITIVITY Truth + -  Proportion of true cases reported as cases  Sensitivity = a / (a + c) a b + a+b Test result (TP) (FP)  Among those who have the disease what proportion tested positive c d - c+d (FN) (TN)  Sensitivity = Pr(Test + | Truth +) a+c b+d CALCULATING SPECIFICITY Truth + -  Proportion of true non-cases reported as non-cases  Specificity = d / (b + d) a b + a+b Test result (TP) (FP)  Among those who do not have the disease what proportion tested negative c d - c+d (FN) (TN)  Sensitivity = Pr(Test - | Truth -) a+c b+d LET’S PRACTICE! Truth + -  Calculate + 160 30 190 Sensitivity Test result   Specificity - 40 170 210 200 200 400 LET’S PRACTICE! Truth + -  Calculate + 160 30 190 Sensitivity = 160 / (160 + 40) = 80% Test result   Specificity = 170 / (30 + 170) = 85% - 40 170 210 200 200 400 TRADE-OFFS SENSITIVITY & SPECIFICITY  Test results presented as binary  Underlying mechanism is typically continuous (i.e. antigen levels)  Cut-point is selected  Determines “normal” and “abnormal”  How do we pick a good cut-point?  Maximize sensitivity and specificity  Increasing sensitivity decreases specificity (and vice versa) DIFFERENT CUT-POINTS EXAMPLE: HIGH SENSITIVITY & LOW SPECIFICITY  Increase sensitivity at the expense of specificity  Result in a very large number of false positives but few (or no) false negatives  BAD when:  Diagnostic procedures are invasive/costly  Disease has no treatment or is severe  GOOD when:  Diagnostic tests are cheap  Disease treatment is highly effective  Little or no emotional cost to being false positive EXAMPLE: LOW SENSITIVITY & HIGH SPECIFICITY  Increase specificity at the expense of sensitivity,  Result in a larger number of false negatives but fewer false positives  BAD when:  You are trying to prevent transmission of a disease  When treatment is available and early detection will decrease mortality (missed opportunity)  GOOD when:  High emotional cost to being false positives  Diagnostic tests are expensive EXAMPLE: MINIMAL ERROR WHEN TO PRIORITIZE ON TEST PROPERTY OVER THE OTHER? Diagnostic test for a fatal disease with no Screening to prevent transmission of a treatment preventable disease  Optimize specificity at cost of sensitivity  Optimize sensitivity at cost of specificity  False-negatives do little harm (no treatment)  False-negatives carry high risk of exposing others  False-positives inflict great emotional harm  False-positives runs risk of potential harms  Example: Late-stage cancer  Example: Screening donated blood for HIV HOW DO WE SELECT THE CUT-POINT?  Receiver Operating Characteristics (ROC) Curves  Used to select best cut-point for screening (or diagnostic tests)  Plot true positive rate (sensitivity) versus false positive rate (1- specificity) for different cut-points ROC CURVES  General guidelines for interpreting the AUC: .90-1.0 = excellent .80-.90 = good .70-.80 = fair .60-.70 = poor .50-.60 = fail PROBLEM…  Not everyone understands sensitivity or specificity  Sensitivity answers: I have the disease, how likely is it that I test positive?  Specificity answers: I do not have the disease, how likely is it that I test negative?  Instead, people want to know…  I tested positive, how like is it that I have the disease?  Positive predictive value  I tested negative, how likely is it that I do not have the disease?  Negative predictive value POSITIVE & NEGATIVE PREDICTIVE VALUES Positive predictive value Negative predictive value  Proportion of people with positive screening test  Proportion of people with negative screening test result who truly have disease result who truly do not have disease  Proportion with disease given a positive test  Proportion without disease given a negative test  Usually fairly small  Usually fairly large  Increases as prevalence increases  Decreases as prevalence increases  False “alarm rate”= 1 – PPV  False “reassurance rate” = 1 – NPV CALCULATING POSITIVE PREDICTIVE VALUE Truth + -  Proportion of those reported as cases who were truly cases a b  PPV = a / (a + b) + a+b Test result (TP) (FP)  Among those who screened positive what c d proportion had the disease - c+d (FN) (TN)  PPV = Pr(Truth + | Test +) a+c b+d CALCULATING NEGATIVE PREDICTIVE VALUE Truth + -  Proportion of those reported as non-cases who are truly non-cases a b  NPV = d / (c + d) + a+b Test result (TP) (FP)  Among those who screened negative what c d proportion did not have the disease - c+d (FN) (TN)  NPV = Pr(Truth - | Test -) a+c b+d LET’S PRACTICE! Truth + -  Calculate + 160 30 190 PPV Test result   NPV - 40 170 210 200 200 400 LET’S PRACTICE! Truth + -  Calculate + 160 30 190 PPV = 160 / (160 + 30) = 84% Test result   NPV = 170 / (40 + 170) = 81% - 40 170 210 200 200 400 SUMMARY OF PPV & NPV HOW TO MEMORIZE THESE PROPERTIES?  Sensitivity Truth  Sensitivity has a “n” = False Negative. High sensitivity, low false negatives  Work down the first column TP / (TP + FN) + -   PPV a b + a+b  Work across the first row Test result (TP) (FP)  TP / (TP + FP)  Specificity c d - c+d  Specificity has a “p” = False Positive. High specificity, low false positives (FN) (TN)  Work down the second column  TN / (FP + TN) a+c b+d  NPV  Work across the second row  TN / (FN + TN) OVERVIEW OF THE TEST PROPERTIES POPULATION PERSPECTIVE RELATIONSHIP WITH PREVALENCE PPV False Positives  #FP = (N)(1-Specificity)(1-prevalence)  N = Sample interested in screening INCIDENCE RATES & REPEATED SCREENING  Incidence rate increases sharply following initial screen  If repeat screen done before end of longest lead time gained among cases detected in first screen, increase in incidence rates after the second screen will be smaller than the first  If screening is continued indefinitely, average incidence rate increases above baseline (i.e., rate that existed before screening) indefinitely INCIDENCE RATES & SCREENING INCREASING EFFECTIVENESS OF SCREENING To increase number of cases Increase sensitivity detected Screen at more frequent intervals To decrease number of false- Decrease sensitivity positives Screen less frequently Increase true positives & decrease Highly sensitive test 1st; highly specific test 2nd false positives Screen only high-risk population INCREASING EFFECTIVENESS (CONT.) Most determinants of screening effectiveness (sensitivity, specificity, effectiveness of early treatment) are linked to natural history of disease Rate at which disease progresses Signs & symptoms Amenability to treatment TIMELINE FOR SCREENING NATURAL HISTORY OF DISEASE Latency *Detectable Detectable Preclinical Preclinical Period Biological Detectable Symptoms Diagnosed Becomes Onset by testing begin w/ disease disabling Primary Secondary Tertiary Prevention Prevention Prevention 42 LEAD TIME LEAD TIME DEFINITION Interval between detection due to screening and time at which diagnosis would have been made without screening Different for different individuals; Distribution of lead times in a screened population There must be enough lead time in enough cases for a screening program to be effective BIASES IN SCREENING EVALUATION BIAS IN SCREENING Lead time bias Length time bias Overdiagnosis bias LEAD TIME BIAS  Diagnosis is made earlier in the screened group  Assume earlier diagnosis does not  Death occurs at same age in screened and unscreened  BIAS: Length of time from diagnosis to survival will look artifactually better in screened cases  Screening does not impact mortality  Screening causes people to live “longer” with disease LENGTH TIME BIAS  Probability of detecting disease is related to the growth rate of the tumor  Aggressive, rapidly growing tumors  Short detectable preclinical period  Typically present with symptoms  Indolent, slow growing tumors  Long detectable preclinical period  More likely to be detected when asymptomatic  BIAS: Higher proportion of indolent tumors is found in the screened group, causing an apparent improvement in survival OVERDIAGNOSIS People with undetectable preclinical disease screen positive Serendipity Follow up test confirms screening No way to distinguish “true positives” from “lucky hits” Screen positive but Disease never progresses Pseudo- Regresses disease Die of other causes People do not benefit from screening QUESTIONS TO CONSIDER FOR SCREENING  What can I gain by screening?  GOAL: Decrease in mortality  Is there an available treatment?  Is there a cost-effective treatment?  GOAL: Decrease in transmission of disease  Did I catch disease early enough to change behavior(s)?  GOAL: Increase in survival time & quality of life  How much is a year of life worth? TAKE HOME POINTS (STUDENTS)  Point #1:  Point #2:  Point #3: TAKE HOME POINTS (INSTRUCTOR)  Point #1: Several requirements are needed to make an effective screening.  Point #2: Test proprieties include sensitivity, specificity, positive predictive value, and negative predictive value with PPV and NPV being affected by the prevalence of disease in the population.  Point #3: Lead time bias, length time bias, and overdiagnosis are major biases that can affect screening tests. ITEMS FOR WEEK #3  Lecture #5  Modern Epi 3rd Ed. Pgs. 87-99 & 511-531  Articles posted to Canvas  Lecture #6  Modern Epi 3rd Ed. Pgs. 100-110  Articles posted to Canvas  Assignments  Check-in #1 (1/20)