Week 3 and 4 Diagnostic Studies Tests PDF

 Accuracy of Diagnostic tests True classification of disease status Present Absent Positive A B A+B (present)...

 Accuracy of Diagnostic tests True classification of disease status Present Absent Positive A B A+B (present) True positives False positives Diagnostic Test result C D C+D Negative False negatives True negatives (absent) A+C B+D For sensitivity & specificity, your denominator is based on the gold standard “truth” Sensitivity = A/A+C = TP/TP+FN Specificity = D/D+B = TN/TN+FP For positive and negative predictive value, your denominator is based on the new test PV+ = A/A+B = TP/TP+FP 2 Predictive Value PV is sometimes called the posterior probability, since it is determined after knowing the test results Since PV+ takes into account information on both the test and the population being tested, it is a good measure of clinical usefulness Impact of prevalence or prior probability of disease on PV: As prevalence increases, PV+ increases As prevalence decreases, PV- increases 3  Without With palpable palpable masses masses Surgical biopsy Surgical biopsy FNA Cancer No 14 8 113 15 result Cancer No Positivecancer cancer 22 128 Positive 1 91 8 181 Negative 92 15 99 121 189 Negative 196 11 317 Higher Prevalence = 4 Prevalence = 38% 13% prevalenc Sensitivity = 93% e, higher Sensitivity = Specificity = 92% PV +, 93% PV + = 88% slightly Specificity = lower PV - 92% PV - = 96% PV + = 4 64% A psychiatrist devised a short screening test for depression. An independent blind comparison was made with a gold standard for diagnosis of depression among 200 psychiatric outpatients. Among the 50 outpatients found to be depressed according to the gold standard, 35 patients were positive for the test. Among 150 patients found not to be depressed according to the gold standard, 30 patients were found to be positive for the test. Which of the following is/are correct: (A) The sensitivity was 80% (B) The specificity was 80% (C) The positive predictive value was 70% (D) The negative predictive value was 88.9% (E) The prevalence of depression was 25% A psychiatrist devised a short screening test for depression. An independent blind comparison was made with a gold standard 35 30 65 for diagnosis of depression among 200 psychiatric outpatients. Among the 50 15 120 135 outpatients found to be depressed according to the gold standard, 35 patients were positive for the test. Among 150 patients 50 150 200 found not to be depressed according to the gold standard, 30 patients were found to be positive for the test. Which of the (A) The following sensitivity is/are correct: was 80% False; sens = 35/50 = 70% True; spec= 120/150 = 80% (B) The specificity was 80% False; PPV= 35/65 = 53.8% (C) The positive predictive value was 70% True; 88.9% (D) The negative predictive value was NPV= 120/135 = 88.9% True;25% (E) The prevalence of depression was Prev=50/200 =25% A general practitioner decides to use the screening test for his patients. The prevalence of depression among his patients is estimated to be 5%. The following measures are identical whether the test is applied to general practice patients or to the psychiatric outpatients: (A) Prevalence of depression (B) Sensitivity (C) Specificity (D) Positive predictive value (E) Negative predictive value  A rapid serologic test for Lyme disease is tested in 1100 people. 100 are known to have Lyme disease and 95 of them test positive by this new assay. Of the remaining subjects who did not have Lyme disease, 200 also test positive. Which of the following statements about this assay are correct? A) It has 95% sensitivity and 20% specificity B) It has 80 % sensitivity and 95% specificity C) It has 95% sensitivity and 80% specificity D) It has 20% sensitivity and 95% specificity  Disease No Disease Total Positive 95 200 295 Negative 5 800 805 Total 100 1000 1100 Sens = 95/100 = 95% Specificity = 800/1000 = 80% Correct answer is C  A screening test for a particular disease has a sensitivity of 92% and a specificity of 94%. You plan to screen a population in which the prevalence of the disease is 4%. What is the false positive rate?  Because sensitivity and specificity are only percentages you can pick any sample size population you want, 1000 people will be used. Number of people with disease = 1000 people *.04 = 40 people Number of people w/o disease = 1000 people – 40 people = 960 people Sensitivity = 0.92 = True Positives / 40 people True Positives = 0.92 * 40 people = 36.8 people approx. 37 people Specificity = 0.94 = True Negatives / 960 people True Negatives = 0.94 * 960 people = 902.4 people False Positives = Number of people w/o disease – True Negatives False Positives = 960 people – 902.4 people = 57.6 people approx. 58 False positive rate = 57.6/960 = 0.06 or 6% You could have also answered the question by recalling that Number of people without disease = FP + TN; True negative rate is really specificity and False positive rate is really 1-specificity. In this example, 1- 0.94=0.06 Incidental Adrenal Mass: Benign or Malignant? Am Fam Physician. 2001 Jan 15;63(2):288-295 Distinguishing benign from a malignant nonhyperfunctioning tumor Lee et al: Results N=38 benign (gold std) AV: -35.5 to 50.8 HU (mean 2.2±16) Size: 1.2 to 4.1 cm ( mean 2.1±0.7) N=28 malignant (gold std) AV: 2.6 to 52.3 HU (mean 28.9±10.6) Size: 1.5 to 10 cm ( mean 3.9±1.9) 7. If a threshold of >5 cm is used to describe the adrenal mass as malignant, which of the following is correct: A. The sensitivity is 100% Please see image on last slide for B. The specificity is 100% explanation C. Both sensitivity and specificity are 100% 8. If the threshold is reduced and >4 cm is used to describe the adrenal mass as malignant, which of the following is correct: A. The sensitivity will decrease, but specificity will increase B. The sensitivity will increase, but specificity will decrease Please see image on last slide C. Both sensitivity and specificity will increase D. Both sensitivity and specificity will decrease Likelihood Ratios Clinical Scenario You are asked to see a 70 year old woman who is 5 days post abdominal surgery. She had sudden onset dyspnea yesterday afternoon and this has been worsening. She has some chest discomfort especially with deep breathing more so on right side than left Physical exam is remarkable for respiratory rate of 30, occasional wheezing on bilateral lung examination and mild abdominal tenderness at surgical site Her oxygen saturation is 95% on room air. You suspect a pulmonary embolism. You would like to order a CT angiogram. In the small community hospital you work at, the CT angiogram is out of order, but the VQ scanner is available. VQ Result The VQ scan comes back as indeterminate. You realize you need to learn more about VQ scans to decide what to do next. So you look for an article on the diagnostic properties of a VQ scan. Articles About Diagnosis: Questions to Ask Are the results valid? What are the results? Can I apply the results to patient care? Users' Guides to the Medical Literature: A Manual for Evidence- Based Clinical Practice, 3rd ed. Gordon Guyatt, Drummond Rennie, Maureen O. Meade, Deborah J. Cook Are results of the study valid? Was there an independent, blind comparison with a reference standard? Did the patient sample include an appropriate spectrum of the sort of patients to whom the diagnostic test will be applied in clinical practice? Did the results of the test being evaluated influence the decision to perform the reference standard? Were the tests methods described clearly enough to permit replication? What Are the Results Sensitivity Specificity Positive Predictive Value Negative Predictive Value Likelihood Ratio What were the results? What Are Results: Definitions The prior (or pretest) probability/prevalence of disease in the 2x2 table before any testing is the proportion of the total population with the disease: (a+c)/(a+b+c+d). Sensitivity (or true positive rate) is the proportion of diseased people with a positive test: a/(a+c) Specificity (or true negative rate) is the proportion of non diseased people with a negative test: d/(b+d). SnNout: if a highly sensitive test is negative can rule out dz SpPin: if a highly specific test is positive can rule in dz What Are Results: Definitions Positive predictive value is the proportion of people with a positive test who have the disease: a/(a+b). Negative predictive value is the proportion of people with a negative test who don’t have the disease: d/(c+d). PIOPED Assuming only High Probability scans are positive Angiogram (Pulm. Embolism) VQ Scan Yes No High Probability 102 14 Others 149 616 Total 251 630 What is the Sensitivity of a high probability VQ scan? A) 41% B) 70% C) 30% D) Cannot calculate Correct Answer: A) Sensitivity: 102/251 = 41% What is the Specificity of a high probability VQ scan? A) 60% B) 98% C) 70% D) 40% Correct Answer: B) Specificity: 616/630 = 98% What is the PPV of a high probability VQ scan? A) 87% B) 70% C) 50% D) 88% Correct Answer: A) PPV = 102/102+14 = 87% What is the NPV of a high probability VQ scan? A) 87% B) 70% C) 81% D) 88% Correct Answer: C) NPV = 616/149+616 = 81% PIOPED Sensitivity: 102/251 = 41% Specificity: 616/630 = 98% PPV = 102/102+14 = 87% NPV = 616/149+616 = 81% PIOPED Results JAMA 1994; 271(9): 703-707. Likelihood Ratio The likelihood of a positive test result in patients with LR + disease = The likelihood of a positive test result in patients without disease a/a+c LR+ = Sensitivity/1-specificity = b/b+d  Definitions: Recall the 2x2 Table Disease + No Disease Total - Truth Positive + a b a+b Test Negative - c a c+d Total a+c b+d a+b+c+d PIOPED Study: Table Showing the Different Possible VQ Scan Results LR = 102/251 / 14/630 = 18.3 VQ Scan PE No PE LR High probability 102 14 18.3 Intermediate 105 217 1.20 Low probab 39 273 0.36 Normal/near n 5 126 0.10 Total 251 630  LR Calculations for the PIOPED Study Test Properties of Ventilation – Perfusion (V/Q) Scanning Pulmonary Embolism Present Absent V/Q Scan Result No. Proportion No. Proportion Likelihood Ratio High Probability 102 102/251= 14 14/630=0.022 18.3 0.406 Intermediate 105 105/251=0.41 217 217/630=0.34 1.2 Probability 8 4 Low Probability 39 39/251=0.155 273 273/630=0.43 0.36 3 Norman/ near 5 5/251=0.020 126 126/630=0.20 0.10 normal 0 Total 251 630 JAMA 1994; 271(9): 703-707 What Does it Mean? LR > 10 or < 0.1 generate large and often conclusive changes from pre- to posttest probability LR of 5-10 and 0.1-0.2 generate moderate shifts LR of 2-5 and 0.5-0.2 generate small changes LR 1-2 and 0.5-1 do not alter probability in any important way Nomogram for Interpreting Diagnostic Tests Nomogram for Interpreting Diagnostic Tests Scenarios: 1) Patient has a 20% 2) The green line is for pre-test probability of the same scenario but PE. The VQ test is red now VQ is being read as as high probability (LR intermediate or 18) represented by the indeterminate (LR 1.2) blue line, this increases your post-test 3) The red line is same probability of PE to pre-test probability with slightly over 80%. You a VQ scan read as low may decide to treat the probability for PE (LR patient without further 0.36) testing (or not if risk of anti-coagulation is high) Pretest Probabilities, Likelihood Ratios (LRs) of Ventilation- Perfusion Scan Results, and Posttest Probabilities in Two Patients with Pulmonary Embolus Pretest Probability, % Scan Result (LR) Posttest Probability, % (Range)* (Range)* 78-Year-Old Woman With Sudden Onset of Dyspnea Following Abdominal Surgery 70 (60-80) High Probability (18.3) 97 (96-99) 70 (60-80) Intermediate Probability 74 (64-83) (1.2) 70 (60-80) Low Probability (0.36) 46 (35-59) 70 (60-80) Normal/near normal (0.1) 19 (13-29) 28-Year-Old Man with Dyspnea and Atypical Chest Pain 20 (10-30) High Probability (18.3) 82 (67-89) 20 (10-30) Intermediate Probability 23 (12-34) (1.2) 20 (10-30) Low Probability (0.36) 8 (4-6) 20 (10-30) Normal/near normal (0.1) 2 (1-4) JAMA 1994; 271(9): 703-707 ROC Curves Receiver Operator Characteristic Curves Negative Positive Test Test True Negative True Positive No Disease Disease False Positives increased Where should the cut point be to maximize SPECIFICITY? (i) A (ii) B (iii) C TEST RESULT High specificity Low sensitivity No Disease Disease Negative A B C Positive Where should the cut point be to maximize SENSITIVITY? (i) A (ii) B (iii) C TEST RESULT High sensitivity Low specificity No Disease Disease Negative A B C Positive Where should the cut point be to maximize Sensitivity AND Specificity? (i) A (ii) B (iii) C TEST RESULT Maximize sensitivity and specificity No Disease Disease FN FP Negative A B C Positive  A Summary of the Relationship Between Sensitivity and Specificity: ROC* Curve proportion (sensitivity) True positive Signal Useless test: Cut off values sensitivity= 1- specificity False positive proportion (1-specificity) at any cut Noise off value *Receiver Operating Characteristic Receiver Operating Characteristic (ROC) Curve Curve Area Under the Curve (AUC ) AUROC Category 0.9-1.0 Very good 0.8-0.9 Good 0.7-0.8 Fair 0.6-0.7 Poor 0.5-0.6 Fail Each point on a ROC curve corresponds to a cut-off value and is associated with a test Se and Sp. Locating the cut-off point thus requires a compromise between Se and Sp. In some cases, Se is more important than Sp, for example when a disease is highly infectious or associated with serious complications. On the other hand, in certain circumstances, Sp may be preferred over Se, say when the subsequent diagnostic testing is risky or costly. If there is no preference between Se and Sp, nonetheless, a reasonable approach would be to maximize both indices. Practice Practice Problems: Diagnostic and Screening Tests A new screening test has been developed and is being calibrated prior to marketing. A. Cut-off point A At what cut-off point will accuracy be maximized? B. Cut-off point B C. Cut-off point C D. Cut-off point D E. Cut-off point E Comments This is a screening test diagram problem Focus: identifying critical cut-off points Cut-off point A: 100% sensitivity Cut-off point A: 100% negative predictive value Cut-off point C: maximum accuracy Cut-off point E: 100% specificity Cut-off point E: 100% positive predictive value  True Classification of Disease Status Diagnostic Test Result Practice Problems: Diagnostic and Screening Tests A new screening test has been developed and is being A. Cut-off point A calibrated prior to marketing. At what cut-off point will positive predictive B. Cut-off point B value (PPV) be 100%? C. Cut-off point C D. Cut-off point D E. Cut-off point E Comments This is a screening test diagram problem Focus: identifying critical cut-off points Cut-off point A: 100% sensitivity Cut-off point A: 100% negative predictive value Cut-off point C: maximum accuracy Cut-off point E: 100% specificity Cut-off point E: 100% positive predictive value Practice Problems: Diagnostic and Screening Tests A new screening test has been developed and is being calibrated prior to marketing. A. Cut-off point A At what cut-off point will negative predictive B. Cut-off point B value (NPV) be 100%? C. Cut-off point C D. Cut-off point D E. Cut-off point E Comments This is a screening test diagram problem Focus: identifying critical cut-off points Cut-off point A: 100% sensitivity Cut-off point A: 100% negative predictive value Cut-off point C: maximum accuracy Cut-off point E: 100% specificity Cut-off point E: 100% positive predictive value Practice Problems: Diagnostic and Screening Tests A new screening test has been developed and is being calibrated prior to marketing. A. Cut-off point A At what cut-off point will sensitivity be 100%? B. Cut-off point B C. Cut-off point C D. Cut-off point D E. Cut-off point E Comments This is a screening test diagram problem Focus: identifying critical cut-off points Cut-off point A: 100% sensitivity Cut-off point A: 100% negative predictive value Cut-off point C: maximum accuracy Cut-off point E: 100% specificity Cut-off point E: 100% positive predictive value Practice Question Systolic blood pressure (SBP) has been shown to be a major risk factor for coronary artery disease (CAD). Over a wide range of SBPs, the higher the SBP, the higher the CAD incidence rate. Therefore, an expert panel wants to know what would happen if the cut-off point for elevated SBP was lowered from 140 mmHg to 135 mmHg? A. True positive results will decrease B. Positive predictive value will decrease C. False negative test results will increase D. Specificity will increase E. Negative predictive value will decrease  Practice Question A. Ture positive test results will decrease B. Positive predictive value will decrease C. False negatives test results will decrease D. Specificity will increase E. Negative predictive value will decrease In screening tests, moving the cut-off point to the left increases sensitivity but decreases specificity, which P (All (All N can lead to a decrease in positive predictive value P) N) (PPV). The more specific the test, the less likely an TP TN individual with a positive test will be free from disease FP FN and the greater the positive predictive value. PPV is the ratio of true positives (TP) to (TP + false positives Sensitivity​ Specificity (FP)). A lower cut-off point correctly identifies more patients with the disease, but also leads to more false NPV PPV positives among healthy people  From Diagnosis to Treatment Test and Treatment Thresholds in the Diagnostic Process Probability of Diagnosis Users' Guides to the Test Treatment Medical Literature: A 0% Threshold Threshold 100 Manual for Evidence- % Based Clinical Practice, 3rd ed. Gordon Guyatt, Drummond Rennie, Maureen O. Meade, Deborah J. Cook Probability below Probability between Probability above test threshold: test and treatment treatment threshold; no testing threshold: further testing completed; warranted testing required treatment commences Summary When faced with diagnostic uncertainty, use history, clinical examination, knowledge of risk factors, to determine pre-test probability of disease, and generate a differential diagnosis. Order tests accordingly Use the diagnostic test properties to determine posttest probability and decide whether to treat or continue with further testing Be comfortable with the definitions of sensitivity, specificity, and likelihood ratio