04. Probability-3.pdf

Probability Foundations of Clinical Medicine CMS100 Objectives 1. Understand the concept of probability as it applies to medical diagnosis 2. Understand how bias can distort subjective probability 3. Determine useful pretest probabilities 4. Avoid common errors in using research to determine pretest...

Probability Foundations of Clinical Medicine CMS100 Objectives 1. Understand the concept of probability as it applies to medical diagnosis 2. Understand how bias can distort subjective probability 3. Determine useful pretest probabilities 4. Avoid common errors in using research to determine pretest probabilities 5. Apply the concept of pretest probability to patient cases Probability How likely something is Usually expressed on a scale of 0-100% (or 0-1) It is not possible by rationally updating one’s views, to reach complete certainty (0 or 100%), although you can approach this It follows that you will need to act before you are certain We will discuss this in greater detail when we discuss thresholds Consider probability for better decision-making in the context of uncertainty Room for Improvement Morgan et al. 2021 Accuracy of Practitioner Estimates of Probability of Diagnosis Before and After Testing Respondents overestimated the probability of diagnosis before and after testing “This overestimation is consistent with cognitive biases, including base rate neglect, anchoring bias, and confirmation bias” Room for Improvement Room for Improvement Morgan et al. 2021 Implications 1. Overestimated probability used in deciding whether to initiate therapy can result in medication overuse and excessive procedures with their associated harms 2. These errors would corrupt shared decision-making with patients because practitioners need to understand the likelihood of various outcomes in order to communicate them. Training in shared decision-making has often focused on communication skills, not on actually understanding the probability of disease IHD DVT Stroke risk Attia et al (2004) Room for Improvement “A worrying observation was that a number of clinicians indicated pre-test probabilities of 100%. This presumably reflects a cautious attitude, assuming that all patients have disease until proven otherwise. This method of operating only works if the tests ordered have powerful negative likelihood ratios, and if these tests indeed give negative results. Overestimation of disease risk leaves clinicians unable to judge false positive test results, and may result in more intervention than necessary and indicate a lack of appreciation for how diagnostic tests influence the probability of disease … We speculate that the variability in estimates may be due partly to the lack of “numeracy” skills in previous medical curricula (when older clinicians did their undergraduate training).” - Attia et al (2004) Biases Not a personal shortcoming, a human trait Example Substitute judgments of representativeness for judgments of actual probability Representativeness: the degree to which something is representative of, or similar to, the stereotype Biases A certain American man has been described by a neighbor as follows: “Steve is very shy and withdrawn, invariably helpful but with little interest in people or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.” Is Steve more likely to be a librarian or a farmer? From Thinking, Fast and Slow Biases “People who are asked to assess probability are not stumped, because they do not try to judge probability as statisticians and philosophers use the word. A question about probability or likelihood activates a mental shotgun evoking answers to easier questions. One of the easy answers is an automatic assessment of representativeness.” - Daniel Kahneman, in Thinking, Fast and Slow p.150 Biases An uncommon presentation of a common disease is more likely than a common presentation of a rare disease If this weren’t so, diagnosis might just be a matter of patternmatching “When you hear hoofbeats, think of horses, not zebras” Biases “The essential keys to disciplined Bayesian reasoning can be simply summarized: Anchor your judgment of the probability of an outcome on a plausible base rate [i.e. pretest probability] Question the diagnosticity of your evidence” Thinking, Fast and Slow p.154 Pretest Probability Best estimate of a disease probability before you do a test After doing a test, you will have a post-test probability A starting place from which to update probability Multiple ways to determine this but want to start with a good reference class Best reference class: the set of patients that most closely matches this patient Reference Classes Basic: the prevalence of a disease in a population Prevalence: The proportion of a population affected by a condition Pros: (relatively) easy to search for can specify sub-populations to get a more accurate estimate Cons: may be an underestimate if it is something that people frequently seek medical attention for less helpful for acute conditions Reference Classes More specific: studies that give eventual diagnosis in patients presenting with complaint(s) similar to your patient’s Pros: take the presenting symptom into account to provide a more accurate initial judgment takes into account that people tend to seek medical attention for some conditions more than others Cons: this research is less common (harder to find) clinical scenario in research may be different from your own Reference Classes Do NOT use: Incidence in the population The frequency of a disease over a period of time Lifetime prevalence The chances of developing the disease over a lifetime: will tend to be an overestimate Tip: consider finding different reference classes with the useful methods above to represent maximum and minimum estimates Consider adding pretest probabilities to illness scripts (epidemiology) Example Migraine F>M Epidemiology … versus Migraine Prevalence: F~15-20%; M~5-10% Epidemiology … Example A 33-year-old woman presents with fatigue. You are considering iron-deficiency anemia as a differential diagnosis You find the following: World prevalence of iron deficiency anemia: 27% Canadian prevalence of iron deficiency anemia among women her age: ~4% Frequency of anemia (of any type) among people presenting to general practice with fatigue: 3% What do you think is an appropriate pretest probability? Example A 26-year-old woman with a past history of anxiety presents with chest pain since yesterday. She started an oral contraceptive 1 week ago and has been having relationship problems for the past two months. Three differentials you’re considering are as follows: Anxiety Chest wall pain Pulmonary embolism Take a moment to guess their rank in terms of pretest probabilities Example Chest wall pain Looking for prevalence is not particularly helpful; this condition doesn’t often last especially long What about in patients presenting with chest pain in general practice? 45% in this study 47% in this study 49% in this study Anxiety Looking for prevalence, we find mostly statistics about lifetime prevalence, which are not useful What about patients presenting with chest pain in general practice? 7% in this study (though they use “psychopathology”, so this may be an overestimate) 9.5% in this study (though here they use “psychogenic”; again possibly an overestimate) In primary care overall? Generalized anxiety disorder: 8%, panic disorder: 6.8% (Herr, 2015) Pulmonary embolism We know prevalence won’t be especially useful, since PEs don’t last long What about patients presenting with chest pain in general practice? ~0.1% in this study Example You might decide to start with the following pretest probabilities: Chest wall pain: 45-50% Anxiety: 5% Pulmonary embolism: 0.1% Summary 1. Explicitly considering probability can help with better decisionmaking in the context of uncertainty 2. Biases such as using representativeness in place of probability can distort judgments of subjective probability 3. There are multiple ways to determine pretest probabilities 4. Using inappropriate or unrepresentative statistics can distort estimates of pretest probabilities